Stanford Seminar - Robot Skill Acquisition: Policy Representation and Data Generation

()

Robot Perception and Manipulation

The speaker introduces their work on robot perception and manipulation, aiming to push the boundaries of robot capabilities by enabling them to perform complex tasks.
They describe their previous workflow, which involves designing task-specific action primitives, collecting robot data, and training policies with a few learnable parameters.
This approach requires significant engineering effort and is not general enough to represent all possible robot actions, especially those requiring high-rate and reactive behaviors.
The speaker proposes a new workflow based on diffusion policy, which allows robots to directly learn complex manipulation skills from human demonstration data.
Diffusion policy addresses the challenge of modeling complex action distributions, such as action multimodality, by using an iterative denoising process.
This approach results in precise predictions and captures multimodalities in the robot action space.
Diffusion policy is a practical framework for learning robot behaviors as long as sufficient data is available.
Diffusion policy outperforms existing baselines on multiple robot control benchmarks.

Data Collection for Robot Learning

Collecting high-quality robot data requires careful planning and consideration of the specific task and environment.
Three important aspects of data for robot learning are scalability, reusability, and completeness.
Scalable data collection methods, such as self-supervised learning and internet data, often lack critical information for robot learning.
Scaling up data collection in simulation environments is challenging due to the high setup cost for new tasks.
A recent project, Scaling Up and Down, addresses this problem by using large models to break down tasks into smaller subtasks and reduce engineering effort.
The speaker introduces a framework for scaling up and distilling down robot experiences to learn a visual motor policy.
The framework uses a large language model (LLM) to generate training data for various tasks in a simulated environment.
The LLM helps break down tasks, narrow down the search space, and generate reward functions for subtasks.
The system can self-correct mistakes and record recovery behaviors, providing valuable data for training.
The distilled visual motor policy can be applied in the real world without relying on simulation states.
The speaker highlights the importance of suboptimal data in training to enable robots to recover from failures.
Challenges in scaling up real-world data for robots are discussed, including the need for an intuitive and standardized interface.
The speaker proposes the "Grasping in the Wild" project as an example of an interface for collecting robot-complete data in various environments.
Limitations of the "Grasping in the Wild" interface are identified, such as restricted visual coverage, fast camera motions, and latency discrepancies between data collection and robot deployment.
The speaker discusses the limitations of using internet data for robot manipulation tasks due to low action diversity.
They propose modifications to a GoPro camera to enable a large variety of manipulation tasks, including:
- Switching to a fish-eye lens for a wider field of view.
- Adding small mirrors for implicit stereo depth estimation.
- Adding sensors to the fingers for tracking gripper width, contact information, and implicit force measurement.
The modified GoPro camera is compatible with different robot platforms.
The speaker demonstrates the device on several hard manipulation tasks, including tossing, manual folding, and dishwashing.
The system achieves an 80% success rate for tossing, can perform manual folding after 200 demonstrations, and can handle the complex dishwashing task with a 70% success rate.

Multi-Arm Coordination and Generalization

The speaker emphasizes the importance of considering synchronization and coordination between multiple robot arms.
The system is able to generalize to new situations and can correct for errors.
The speaker introduces the Umi gripper, a low-cost, portable robotic gripper that can be easily deployed in various environments.
The speaker discusses the challenges of collecting diverse training data for robots and how Umi gripper addresses these challenges.
The speaker presents a generalization experiment where a robot trained with diverse data collected using Umi gripper is able to perform a rearrangement task in unseen environments and with unseen objects.
The speaker emphasizes the importance of diverse robot action data for generalization and shows that pre-training a visual encoder on internet data is insufficient for generalization.

Challenges and Future Directions

The speaker concludes by encouraging roboticists to leverage their unique skills and knowledge to create data for robot learning and shape the next generation of big data.
The speaker demonstrates how with enough data, you can generalize Dev to change in environments with the same Hardware.
Generalizing among different Hardware platforms is still hard, but the same policy can be deployed on different robot arms with the same hand.
Generalizing to different hands requires more involved engineering, such as training a Dynamics model or a separate inverse model for robots.
It is possible to get Yumi out in the wild to the general public to gather data, but it