🤖 Awesome Test-Time Robot Learning

A curated research map of how robot policies improve after pre-training and task-level post-training, using context, test-time computation, deployment experience, or online interaction.

🎯 Scope

We use test-time robot learning as a broad deployment-stage umbrella: a post-trained robot policy encounters a concrete environment and improves within an inference call, an episode, or a sequence of online interactions. The list is centered on manipulation, while including general robot-learning methods when their test-time mechanism transfers directly.

A paper belongs here when it changes deployment context; action sampling, guidance, or selection; candidate generation and verification; a fast state, memory, or representation; or policy-side parameters using deployment data. Routine pre-training, ordinary task fine-tuning with a fixed offline dataset, and planning without a learned robot policy are outside the main scope.

🗺️ Research Map

The six lines are organized by their primary deployment-time mechanism and supervision source. Hybrid methods are expected; each paper is assigned to the line that best captures its central contribution.

Compare the six lines

Line	Core mechanism	Deployment signal	Expected effect	Main limitation
RL Post-Training	RL optimization after task-level post-training	Rewards, success labels, interventions, online rollouts	Produces an RL-improved policy, adapter, residual controller, or latent interface	Interaction cost, resets, safety, and reward design
Test-Time Policy Steering	Guides sampling, denoising, flow, or action selection at execution	Value functions, VLM rewards, dynamics, constraints	Improves action generation or selection for the current setting	Candidate coverage, guidance quality, and inference latency
Test-Time Adaptation and Training (TTA & TTT)	Adapts restricted parameters, state, or representations from deployment data	Self-supervised losses, feedback, predictive objectives, trajectory history	Responds to distribution shift, temporal context, or model mismatch	Objective alignment, stability, and forgetting
In-Context Learning and Prompting	Conditions a policy on demonstrations, video, language, or sensorimotor context	Prompt examples supplied at deployment	Uses demonstrations and instructions without a gradient update	Requires a policy trained to interpret the prompt modality
Test-Time Scaling	Allocates more inference compute to reasoning, sampling, search, prediction, refinement, or verification	Internal confidence, value, predicted outcomes, self-consistency, or reasoning traces	Improves decisions within the current inference call without collecting new demonstrations	Gains depend on useful candidate diversity or reasoning; latency grows with compute
Policy Self-Improvement	Repeats deployment data collection and model updates	Rollouts, failures, corrections, rewards, or model-generated experience	Improves behavior across deployment iterations	Requires safe collection, reliable feedback, resets, and control of policy or model drift

📚 Contents

• 🧪 RL Post-Training
• 🧭 Test-Time Policy Steering
• 🧠 Test-Time Adaptation and Training (TTA & TTT)
• 🎬 In-Context Learning and Prompting
• 📈 Test-Time Scaling
• 🔄 Policy Self-Improvement
• 🤝 Contributing

Papers are sorted by their first public release date, from oldest to newest. Venue denotes the latest known publication venue; otherwise it is listed as arXiv.

1. 🧪 RL Post-Training

These methods use reinforcement learning after task-level post-training to improve a policy, adapter, residual controller, or latent interface from reward-bearing interaction.

Typical mechanism: Policy-wide or restricted parameter optimization, residual control, latent-space optimization, or online actor-critic updates.

Core advantage: Can improve state-action associations using rewards and interaction beyond behavior cloning.

Main limitation: Interaction cost, resets, safety, reward design, and stability of online optimization.

Papers (15)

2. 🧭 Test-Time Policy Steering

These methods keep the base policy frozen and intervene in its action-generation process by re-ranking candidates, guiding denoising or flow trajectories, optimizing noise, or applying external value and constraint signals.

Typical mechanism: The sampling process, candidate action, noise variable, denoising or flow trajectory, or execution-time controller around a frozen policy.

Core advantage: It is modular and often data-light, making it attractive when the desired behavior already exists in the policy distribution.

Main limitation: It usually cannot repair missing support; guidance quality, repeated sampling, and external models can also add substantial latency.

Papers (12)

3. 🧠 Test-Time Adaptation and Training (TTA & TTT)

These methods adapt a policy, representation, or temporal state from deployment data. They include gradient-based self-supervised adaptation, feedback-driven test-time optimization, fast-weight updates, and adaptive memories.

Typical mechanism: Model parameters or restricted adapters, fast weights, adaptive memory, temporal state, or perception and control representations.

Core advantage: Can respond to visual changes, temporal context, feedback, or model mismatch without a dedicated offline adaptation dataset for each deployment setting.

Main limitation: The deployment objective or feedback signal may not track task success, and continual updates can drift, forget, or destabilize control.

Papers (11)

4. 🎬 In-Context Learning and Prompting

These methods use demonstrations, videos, language, or sensorimotor trajectories in the deployment-time context to specify robot behavior. The direct in-context imitation subset predicts actions from task examples without task-specific parameter updates; the broader prompting subset studies compatible task interfaces such as human video and multimodal instructions.

Typical mechanism: The deployment-time context sequence; policy parameters are usually unchanged.

Core advantage: A user can specify a task, behavior, or constraint through examples without a task-specific gradient update or online reward optimization.

Main limitation: The policy must be trained to read the prompt modality, and reliable prompting does not guarantee the required low-level behavior is in the policy's learned repertoire.

Papers (20)

5. 📈 Test-Time Scaling

These methods allocate additional inference compute to reason, sample, search, predict outcomes, refine actions, or verify candidates before execution. Verification is one mechanism within this broader line rather than a separate category.

Typical mechanism: Reasoning depth, candidate count, search depth, world-model rollouts, refinement steps, verifier calls, or an adaptively allocated inference budget.

Core advantage: Can improve a fixed policy at deployment and expose a measurable compute-performance trade-off without collecting new demonstrations.

Main limitation: More computation helps only when reasoning, candidate diversity, predictive models, or selection scores contain useful signal; latency and hardware cost increase with the budget.

Papers (12)

6. 🔄 Policy Self-Improvement

These methods use deployment rollouts, failures, corrections, or model-generated experience to improve future behavior across successive data-collection and update rounds.

Typical mechanism: The base policy, residual policy, critic, world model, recovery behavior, or data-generation loop between deployment iterations.

Core advantage: Turns autonomous experience and intervention data into better future behavior, reducing dependence on repeated full demonstrations.

Main limitation: Requires reliable rewards or success labels, safe data collection, practical resets, and controls against policy drift or errors in model-generated data.

Papers (8)

🤝 Contributing

Paper additions and corrections are welcome. Please read CONTRIBUTING.md, edit data/papers.json, and run:

python3 scripts/validate_data.py
python3 scripts/build_readme.py --check

You can also use the Add a paper issue template.

🙏 Acknowledgements

The repository structure draws inspiration from community-maintained collections such as Awesome World Models for Robotics, Awesome VLA Post-Training, and Awesome VLA.

📄 License

This repository is released under the MIT License.