Human Aware Interactions and Learning Intern

The Mission

We are working to create general-purpose robots capable of accomplishing a wide variety of dexterous tasks with humans. To do this, our team is exploring how Large Behavior Models and other general-purpose machine learning models use generative AI techniques to produce robot actions from sensor data and human interactions. We use both simulation and human-interactive data to augment real-world robot data towards such embodied AI policies.

The Challenge

We envision a future where robots assist with household chores and cooking, aid older individuals in maintaining their independence, and enable people to spend more time on the activities they enjoy most. To achieve this, robots need to be able to operate reliably in messy, unstructured environments. Our mission is to answer the question “What will it take to create truly general-purpose robots that can accomplish a wide variety of tasks in settings like human homes with minimal human supervision?” We believe that the answer lies in using large-scale datasets of physical interaction from a variety of sources and building on the latest advances in machine learning to learn general-purpose robot behaviors from this data.

The Internship

We are seeking a research intern to join our Human Aware Interaction and Learning (HAIL) team. The research conducted on this internship aligns with our broader mission and will focus on one or more of the following areas at the intersection of robot learning and human-robot interaction (HRI):

• Data-efficient and general algorithms for learning robust multimodal policies that can also reason over actions in the presence of a human. This research would examine what additional features should be modeled when humans are present in the scene, or in collaborative scenarios, including how roles are shared and reallocated between the human and the robot.

• Developing hierarchical reasoning frameworks for learned robot policies that can reason over human and shared task states (intent, cues, roles) in order to enable fluent human-robot collaboration.

• Robustifying data-driven robot policies to the presence of humans, particularly in the low-data regimes typical of HRI, including approaches for data aggregation and simulative approaches towards adaptation of vision-language-action (VLA) models to human-human and/or human-robot interaction datasets.

• Uncertainty-aware teaming: estimating and calibrating when a learned policy will succeed given human context, and using such estimates to decide how the robot should act in a collaborative setting.

• Exploring additional data sources when training human-interactive robot policies, including human-human and synthetic/semi-synthetic data, towards novel datasets/methodologies and algorithmic frameworks that enable robots to efficiently learn from collaborative demonstrations, and better understand human behavior within these collaborations.

• Developing shared autonomy in conjunction with VLA models to create more sophisticated fusion between autonomous and manual control beyond simple switching mechanisms. This could be studied both in the context of efficiently collecting data and runtime decision-making approaches.

The intern who joins our team will be expected to create working code prototypes, interact frequently with team members, run experiments with both simulated and real (physical) robots, and participate in publishing the work to peer-reviewed venues. We’re looking for an intern who is comfortable working with both existing large static datasets and helping build and maintain a new corpus of robot data.