240119 AA289 Annie Chen

240119 AA289 Annie Chen

Reinforcement Learning for Autonomous Robots

  • Recent advances in autonomous robots have led to robots that can perform tasks in controlled environments.
  • However, these robots often struggle to adapt to unexpected circumstances and novel scenarios during real-world deployment.
  • Reinforcement learning provides a framework for robots to adapt autonomously, but it is challenging to apply directly during deployment due to the need for feedback, retries, and the ability to learn from scratch.

Reset-Free Reinforcement Learning

  • Reset-free reinforcement learning addresses some of these challenges by allowing robots to practice both learning the task and undoing it without human intervention.
  • Single-life reinforcement learning is introduced as a paradigm where the agent is given prior experience and must adapt to a new scenario without human intervention or supervision within a single episode.

Robust Autonomous Modulation (REALM)

  • The proposed method, Robust Autonomous Modulation (REALM), leverages the expressive power of each behavior's value function to guide behavior selection during adaptation.
  • REALM fine-tunes the value functions of pre-trained behaviors to correct for overestimation in out-of-distribution states.
  • The selection mechanism in REALM quickly identifies appropriate behaviors in a given situation, eliminating the need for a separate high-level controller or adaptation module.
  • REALM is agnostic to how the policies and value functions of the prior behaviors are trained and can provide improvements in new situations with either a small or large number of pre-trained behaviors.
  • The adaptation process in REALM happens within a single episode at test time, allowing robots to adapt to a variety of situations without the need for extensive online training.

Rome: A Simple Algorithm for Autonomous Deployment-Time Adaptation

  • Rome is a simple algorithm for autonomous deployment-time adaptation.
  • Rome outperforms prior methods in simulated and real-world experiments.
  • Rome can adapt to novel situations within a single episode.
  • Rome can handle dynamic changing payloads and unseen objects.
  • Rome can leverage parts of each relevant behavior to complete tasks.
  • Rome provides a mechanism for single-life test-time adaptation to unseen situations.

Overwhelmed by Endless Content?