Stanford Seminar - Learning-enabled Adaptation to Evolving Conditions for Robotics

()

Learning-Enabled Adaptation for Lifelong Deployment of Robots

Robots can respond to unexpected changes in conditions during deployment through learning-enabled adaptation.
Distribution shifts, where the input data distribution changes from the training distribution, can lead to incorrect predictions and unsafe outcomes.

The data life cycle for lifelong deployment involves input data, model predictions, uncertainty estimation, out-of-distribution detection, data labeling, and model fine-tuning.
Out-of-distribution detection can be achieved using Bayesian methods such as the Scott algorithm, which estimates functional uncertainty and thresholds it to identify out-of-distribution inputs.
To adapt to distribution shifts, a subset of out-of-distribution inputs are selected for labeling, and the model is fine-tuned or retrained using these labels.
The performance-cost trade-off should be considered when selecting data for labeling, as more labeled data improves performance but incurs higher costs.

Out-of-distribution (OOD) detection algorithms like SCOTT can identify inputs that are different from the training distribution.
Subsampling a diverse set of OOD inputs for labeling can improve model performance more efficiently than randomly selecting inputs or selecting based solely on uncertainty.
The proposed diverse subsampling (DS-SCOTT) algorithm uses information gain as a measure to select diverse inputs for labeling.
DS-SCOTT outperforms random selection and uncertainty-based selection in terms of the trade-off between labeling cost and model performance improvement.
DS-SCOTT can be used in a data life cycle to continuously improve model performance by requesting labels for a diverse set of OOD inputs.
By selecting a diverse subset of 5% or 75% of the data, the model achieves performance comparable to labeling 100% of the data, while only labeling about 50% of it.

To address the problem of forgetting original data during continual learning, uncertainty estimates can be used to generate a regularization term that prevents forgetting.
Runtime monitors can be constructed to distinguish between different classes of distribution shifts.
Self-supervised methods can be used to find invariant features between in-distribution and out-distribution data, reducing the need for labeled data.

The method is particularly useful in domains prone to distribution shifts, such as autonomous driving, where factors like rain, lighting, and sensor degradation can affect perception modules.
The diverse subset selection can be implemented entirely at the edge, and the authors are considering open-sourcing the code.