Robotics Intern - World Models - Multimodal Sensing
Toyota Research Institute · Cambridge, MA
About the role
At Toyota Research Institute (TRI), we’re on a mission to improve the quality of human life. We’re developing new tools and capabilities to amplify the human experience. To lead this transformative shift in mobility, we’ve built a world-class team advancing the state of the art in AI, robotics, driving, and material sciences. This is a paid 12-week internship opportunity and is a hybrid, in-office role. Here’s a glimpse into the Internship experience from some of our TRI interns! The Mission We are working to create general-purpose robots capable of accomplishing a wide variety of dexterous tasks. To do this, our team is building general-purpose machine learning foundation models for dexterous robot manipulation. These models, which we call Large Behavior Models, use generative AI techniques to produce robot actions from sensor data and human requests. To accomplish this, we are creating a large curriculum of embodied robot demonstration data and combining that data with a rich corpus of internet-scale text, image, and video data. We are also using high-quality simulation to augment real-world robot data with procedurally-generated synthetic demonstrations. The Team The Large Behavior Models team charter is to push the frontiers of research in robotics and machine learning to develop the future capabilities required for general-purpose robots able to operate in unstructured environments such as homes! The Internship We are looking for a research intern in the area of data-efficient and general algorithms for learning robust policies leveraging multiple sensing modalities, including proprioception, images, force, audio, and dense tactile sensing. The intern who joins our team will be expected to create working code and/or hardware prototypes, interact frequently with team members, run experiments with both simulated and real (physical) robots, and participate in publishing the work to peer-reviewed venues. We’re looking for an intern who is comfortable working with both existing large static datasets as well as a growing and dynamic corpus of robot data. Qualifications Strong hands-on experience with machine learning and familiarity with large datasets and generative models. Research experience in world models, action-conditioned prediction, or learned control, with an interest in contact-rich manipulation. Strong software development skills in Python and experience with modern deep learning frameworks such as PyTorch. A “make it happen” attitude and comfort with rapid prototyping and testing in simulation and on physical robots. A passion for learning how robot actions shape physical interactions, including contact dynamics, deformation, slip, and force transmission. A creative approach to world action models that integrate tactile and force sensing with vision and proprioception to predict interaction outcomes and guide robot behavior. Experience with tactile sensors, force/torque sensing, contact simulation, or force-controlled manipulation is strongly preferred.
Description from Toyota Research Institute's careers site. The posting there is the authoritative version.