pi05_xpolicylab

XPolicyLab: A unified standard and infrastructure for robot policy development and deployment.

XPolicyLab is the shared layer between policy code and evaluation environments. Keep each model's dependencies, checkpoints, and training recipes under policy/<POLICY>/; XPolicyLab handles the parts that are boring but easy to get wrong — serving, observation/action contracts, and eval wiring.

Start here for repo-level concepts and integration steps. For install commands, checkpoint layout, and training details, jump to that policy's README — it is the source of truth for its model.

📚 Contents

• What XPolicyLab Enables
• Supported Benchmarks And Infrastructure
• Integrated Policies
• Submit Your Policy
• Framework Overview
• Model Integration Guide
• Quick Start
• Common Workflow
• Deployment Flow
• Standard Data Formats
• Data And Checkpoints
• Checks
• Contact

🚀 What XPolicyLab Enables

• Environment isolation: run the policy model in its own conda/uv environment while the simulator, benchmark, or robot client runs separately.
• Remote deployment: connect the policy server and environment client through websocket, either on one machine or across machines.
• A common adapter contract: use the same high-level lifecycle for installation, data conversion, training, serving, and evaluation.
• A large policy zoo: reuse adapters for VLA/WAM policies, imitation-learning baselines, and reference templates.
• Benchmark and infra integration: mount XPolicyLab into benchmark or simulator workspaces without coupling policy code to one environment.

🌐 Supported Benchmarks And Infrastructure

Benchmarks

• RoboDojo: simulator-backed evaluation and RoboDojo-format data exports.
• RoboTwin: benchmark and data source through policy-specific adapters and conversion scripts.

Infrastructure

• RLinf: infrastructure target for policy development and deployment workflows.
• StarVLA: infrastructure and policy stack; see policy/starVLA.

🧭 Integrated Policies

Top-level adapters live in policy/. Each policy README documents that model's paper/repo link, environment, data format, training entrypoint, and checkpoint layout.

📤 Submit Your Policy

Community policies are welcome — open a PR that adds policy/<POLICY>/. A PR is also required to enter the official RoboDojo and RoboTwin leaderboards, together with the checkpoint that reproduces your results. The full adapter standard, testing steps, and PR description template live in CONTRIBUTING.md.

PR rules:

1. Follow the standard adapter layout (Framework Overview) and support the full lifecycle — install, data conversion, training, and eval. If the training code cannot be open-sourced yet, you may land eval-only support first: notify the maintainers (Contact) and share a timeline for releasing training.
2. Write the policy README so that environment setup, data, training, and eval all work by following the script and argument conventions in Common Workflow.
3. Run the closed loop locally first — at minimum the debug-mode eval in Checks, ideally a simulator-backed eval.
4. For official leaderboard evaluation, attach a checkpoint download script in the PR description (Hugging Face or ModelScope preferred). We will evaluate your submission and publish the leaderboard entry as soon as possible.

🧩 Framework Overview

XPolicyLab separates model-side dependencies from environment-side dependencies.

Policy environment                         Evaluation / benchmark environment
------------------                         ----------------------------------
policy/<POLICY>/model.py     <---ws--->    env client / simulator / robot
policy server                              environment client
deploy.yml runtime config                  benchmark task and observation API

A typical adapter contains:

policy/<POLICY>/
├── README.md                    # policy-specific guide
├── INSTALLATION.md              # optional detailed setup notes
├── install.sh                   # environment setup
├── process_data.sh              # optional data conversion
├── train.sh                     # optional training
├── eval.sh                      # same-machine evaluation
├── setup_eval_policy_server.sh  # policy-side server
├── setup_eval_env_client.sh     # environment-side client
├── deploy.yml                   # runtime config
├── deploy.py                    # evaluation loop
└── model.py                     # model adapter

model.py implements the model-facing API. deploy.py bridges environment observations to model-server calls. Use policy/demo_policy as the minimal adapter reference.

model.py should define a Model class with this shape:

The policy server decodes camera colors before update_obs / update_obs_batch, so obs["vision"][<camera>]["color"] always arrives as an image array — model.py never decodes.

The default policy-server protocol is websocket (protocol: ws in deploy.yml); legacy_tcp exists only for adapters that have not migrated yet. The transport handles reconnects, retries, keepalive, and long model-loading cold starts for you — a normal adapter never touches it.

🛠️ Model Integration Guide

The fastest way to add a model is to copy the reference adapter, keep the XPolicyLab boundary small, and debug the adapter before touching a real simulator.

1. Learn the reference adapter: read policy/demo_policy, especially model.py, deploy.py, deploy.yml, eval.sh, setup_eval_policy_server.sh, and setup_eval_env_client.sh.
2. Understand the arguments: keep bench_name, task_name, ckpt_name, env_cfg_type, action_type, and seed consistent across data, training, and eval.
3. Create a skeleton: run bash scripts/create_policy.sh <POLICY_NAME> and immediately fill in policy/<POLICY_NAME>/README.md.
4. Implement model.py first: load model resources in __init__, store observations in update_obs, translate observations to model-native inputs, return XPolicyLab action dictionaries from get_action, and reset state in reset.
5. Keep deployment simple: put runtime defaults in deploy.yml; keep deploy.py aligned with demo_policy/deploy.py unless the environment loop truly differs.
6. Debug without a simulator: run EVAL_ENV_TYPE=debug to check imports, server startup, observation serialization, action keys, action dimensions, and batch logic.
7. Move to simulator or remote deployment: after debug mode passes, use EVAL_ENV_TYPE=sim or split policy server and environment client across machines.

Integrate <POLICY_NAME> into XPolicyLab.

Use policy/demo_policy as the reference.
1. Inspect the upstream model's inference API and dependencies.
2. Create or update policy/<POLICY_NAME>/README.md with install, checkpoint, train, and eval commands.
3. Implement install.sh and, if needed, process_data.sh and train.sh.
4. Implement model.py with Model.__init__, update_obs, get_action, reset, and batch methods.
5. Keep deploy.py aligned with policy/demo_policy/deploy.py.
6. Put runtime defaults in deploy.yml and use protocol: ws.
7. Run EVAL_ENV_TYPE=debug eval.sh and fix shape/action-key/server errors.
8. Summarize supported action_type, env_cfg_type, checkpoint layout, and remaining limitations.

⚡ Quick Start

Clone XPolicyLab as a normal Python project for adapter development, offline checks, training from prepared data, or your own environment client:

mkdir demo_env
cd demo_env
git clone https://github.com/XPolicyLab/XPolicyLab.git
cd XPolicyLab
pip install -e .

You do not need a simulator to start model-side development: the bundled downloader fetches prepared RoboDojo data — several simulator export versions plus HDF5 RoboDojo_real real-world data — for training and offline debugging. If you use XPolicyLab/ as a subpackage inside the RoboDojo repository, follow RoboDojo's own data download scripts instead.

Download a small Hugging Face demo bundle and keep the data next to XPolicyLab/:

# From demo_env/XPolicyLab
bash scripts/RoboDojo/download_robodojo_data.sh demo

This creates:

demo_env/
├── data/        # demo data, including a small 10-episode HuggingFace bundle
└── XPolicyLab/

The same script pulls the full exports — hdf5, lerobot_v3.0, lerobot_v2.1, and real (real-world HDF5) — each into its own ../data/ folder.

With this setup, you can test data conversion, model loading, training scripts, and debug-mode evaluation before connecting to a simulator-backed benchmark.

export EVAL_ENV_TYPE=debug
cd policy/demo_policy
bash install.sh
bash eval.sh RoboDojo stack_bowls demo arx_x5 joint 0 0 0 base base

The template for any adapter is the same — swap demo_policy and the argument values:

export EVAL_ENV_TYPE=debug
cd policy/<POLICY>
bash eval.sh <bench_name> <task_name> <ckpt_name> <env_cfg_type> <action_type> \
  <seed> <policy_gpu_id> <env_gpu_id> <policy_env_or_uv_path> <eval_env_conda_env>

For RoboDojo simulation, mount XPolicyLab/ beside the simulator-side env_cfg/, scripts/, src/eval_client/, and task/ directories.

🔄 Common Workflow

Most adapters expose the same top-level shape. Some policies add extra arguments, consume upstream-native datasets, or skip training support. Follow the policy README when it differs from this template.

cd policy/<POLICY>

# Install the policy runtime.
bash install.sh

# Optional: convert or prepare policy-specific data.
bash process_data.sh <bench_name> <ckpt_name> <env_cfg_type> <action_type> [extra_args...]

# Optional: train.
bash train.sh <bench_name> <ckpt_name> <env_cfg_type> <action_type> <seed> <gpu_id> [extra_args...]

# Evaluate on one machine.
bash eval.sh <bench_name> <task_name> <ckpt_name> <env_cfg_type> <action_type> <seed> \
  <policy_gpu_id> <env_gpu_id> <policy_env_or_uv_path> <eval_env_conda_env>

What the arguments mean

When you run eval.sh, you are mostly answering: which benchmark family, which task to run now, which checkpoint to load, which robot setup, joint or end-effector actions, and which seed. The same names travel through process_data.sh, train.sh, and eval.sh, so you do not have to rename things at every step.

How ckpt_name resolves. Usually you pass the short nickname used during training, such as cotrain, and XPolicyLab combines it with the other args into checkpoints/RoboDojo-cotrain-arx_x5-joint-0/. You can also pass the full folder name, or a path — relative paths resolve from the policy directory, absolute paths work too. Some adapters honor explicit keys in deploy.yml (checkpoint_path, model_path, ...). When in doubt, check the policy README.

A concrete eval example:

cd policy/AHA_WAM
bash eval.sh RoboDojo stack_bowls cotrain arx_x5 joint 0 0 0 aha_wam robodojo
# loads checkpoints/RoboDojo-cotrain-arx_x5-joint-0/ and evaluates on stack_bowls

🔌 Deployment Flow

During evaluation, the policy server and the environment client talk over websocket. That split is what lets you keep Isaac Sim / robot drivers on one machine and a heavy VLA on another.

For same-machine evaluation, eval.sh is enough — it starts the server, runs the client, and cleans up when you are done.

For split-machine deployment, start the policy server on the GPU machine and bind to 0.0.0.0 so other machines can reach it. The client connects to the policy machine's real IP, not 0.0.0.0.

cd policy/<POLICY>
bash setup_eval_policy_server.sh \
  <bench_name> <task_name> <ckpt_name> <env_cfg_type> <action_type> <seed> \
  <policy_gpu_id> <policy_env_or_uv_path> <policy_server_port> 0.0.0.0

Then start the environment client on the simulator or robot machine:

cd policy/<POLICY>
bash setup_eval_env_client.sh \
  <bench_name> <task_name> <ckpt_name> <env_cfg_type> <action_type> <seed> \
  <env_gpu_id> <eval_env_conda_env> <additional_info> \
  <policy_server_port> <policy_server_ip>

<additional_info> is a comma-separated key=value string forwarded to the environment client. eval.sh builds it automatically as ckpt_name=<ckpt_name>,action_type=<action_type>, which is the right default for most adapters.

EVAL_ENV_TYPE selects the environment-side backend:

• unset or sim: real simulator-backed evaluation, when the integration is installed.
• debug: offline wiring check — no Isaac, no robot, just shapes and IO.
• real: real-robot client path, where the hardware integration exists.

📐 Standard Data Formats

XPolicyLab standardizes the observation and trajectory dictionaries passed between adapters, converters, and environment clients. Individual policies may convert this standard format into their upstream-native format.

All pose values use [x, y, z, qw, qx, qy, qz]. Images are RGB end to end — stored image bits are encoded from RGB frames and no channel conversion happens anywhere in the pipeline (the only medium-adapter exceptions are listed with the converter helpers below). Note one naming quirk: runtime observations carry camera extrinsics as extrinsics_matrix, while trajectory files store extrinsic_matrix.

Observation Data Format
├── data_format_version                        string, optional
├── instruction / instructions                 string or list[str]
├── env_idx                                    int, optional for batched eval
├── additional_info/
│   └── frequency                              int, optional
├── vision/
│   ├── cam_head/
│   │   ├── color                              (H, W, 3) RGB, decoded by the server
│   │   ├── depth                              (H, W) or (H, W, 1), optional
│   │   ├── intrinsic_matrix                   (3, 3), optional
│   │   ├── extrinsics_matrix                  (4, 4), optional
│   │   └── shape                              (2,) or (3,), optional
│   ├── cam_left_wrist/                        optional
│   ├── cam_right_wrist/                       optional
│   ├── cam_wrist/                             optional for single-arm robots
│   └── cam_third_view/                        optional
└── state/
    ├── left_arm_joint_state                   (DOF,), optional
    ├── left_ee_joint_state                    (EEF_DOF,), optional
    ├── left_ee_pose                           (7,), optional
    ├── left_tcp_pose                          (7,), optional
    ├── left_delta_ee_pose                     (7,), optional
    ├── right_arm_joint_state                  (DOF,), optional
    ├── right_ee_joint_state                   (EEF_DOF,), optional
    ├── right_ee_pose                          (7,), optional
    ├── right_tcp_pose                         (7,), optional
    ├── right_delta_ee_pose                    (7,), optional
    ├── arm_joint_state                        (DOF,), optional for single-arm robots
    ├── ee_joint_state                         (EEF_DOF,), optional for single-arm robots
    ├── ee_pose                                (7,), optional for single-arm robots
    ├── tcp_pose                               (7,), optional for single-arm robots
    ├── delta_ee_pose                          (7,), optional for single-arm robots
    └── mobile/                                optional
        ├── base_pose                          (7,)
        └── base_twist                         (6,), [vx, vy, vz, wx, wy, wz]

Trajectory Data Format
├── data_format_version                        string, e.g. "v1.0"
├── instruction / instructions                 string, or JSON-serialized list[str]
├── subtasks                                   JSON-serialized annotations, optional
├── additional_info/
│   └── frequency                              int
├── vision/
│   ├── cam_head/
│   │   ├── colors                             (T, H, W, 3), uint8 RGB or encoded stream
│   │   ├── depths                             (T, H, W) or (T, H, W, 1), optional
│   │   ├── intrinsic_matrix                   (3, 3) or (T, 3, 3), optional
│   │   ├── extrinsic_matrix                   (4, 4) or (T, 4, 4), optional
│   │   └── shape                              (2,) or (3,), optional
│   ├── cam_left_wrist/                        optional
│   ├── cam_right_wrist/                       optional
│   ├── cam_wrist/                             optional for single-arm robots
│   └── cam_third_view/                        optional
├── action/                                    action targets, same key naming as state/ below
└── state/
    ├── left_arm_joint_states                  (T, DOF), optional
    ├── left_ee_joint_states                   (T, EEF_DOF), optional
    ├── left_ee_poses                          (T, 7), optional
    ├── left_tcp_poses                         (T, 7), optional
    ├── left_delta_ee_poses                    (T, 7), optional
    ├── right_arm_joint_states                 (T, DOF), optional
    ├── right_ee_joint_states                  (T, EEF_DOF), optional
    ├── right_ee_poses                         (T, 7), optional
    ├── right_tcp_poses                        (T, 7), optional
    ├── right_delta_ee_poses                   (T, 7), optional
    ├── arm_joint_states                       (T, DOF), optional for single-arm robots
    ├── ee_joint_states                        (T, EEF_DOF), optional for single-arm robots
    ├── ee_poses                               (T, 7), optional for single-arm robots
    ├── tcp_poses                              (T, 7), optional for single-arm robots
    ├── delta_ee_poses                         (T, 7), optional for single-arm robots
    └── mobile/                                optional
        ├── base_poses                         (T, 7)
        └── base_twists                        (T, 6), [vx, vy, vz, wx, wy, wz]

Useful converter helpers:

from XPolicyLab.utils.load_file import load_hdf5
from XPolicyLab.utils.process_data import decode_image_bit, get_robot_action_dim_info

decode_image_bit turns encoded image streams into arrays and returns already-decoded values untouched. get_robot_action_dim_info(env_cfg_type) returns robot-specific arm_dim and ee_dim lists, so adapters do not need to hard-code action dimensions.

Three image rules that cause silent, hard-to-debug bugs when broken:

• model.py never decodes. The policy server decodes every observation it forwards, so obs["vision"][<camera>]["color"] already is an image array. This covers update_obs / update_obs_batch and any custom RPC a policy exposes that carries an observation.
• Offline code decodes only via decode_image_bit. In conversion scripts and training dataloaders, never hand-roll cv2.imdecode / np.frombuffer / PIL decoding — RoboTwin and RoboDojo store image bits in legacy layouts that only this function reads correctly.
• Everything is RGB, end to end. No channel conversion belongs in conversion, training, or evaluation. The only exceptions are medium adapters: COLOR_RGB2BGR right before cv2.VideoWriter.write(...), and COLOR_BGR2RGB right after cv2.VideoCapture.read().

Robot action dimensions are registered in utils/robot/_robot_info.json: each top-level key is an env_cfg_type such as arx_x5, with arm_dim / ee_dim lists for per-arm joint and end-effector/gripper dimensions. Update it when adding a new robot configuration so conversion, training, and deployment code can infer action shapes consistently.

💾 Data And Checkpoints

Training and data prep usually name things predictably so eval can find them without guesswork:

<bench_name>-<ckpt_name>-<env_cfg_type>-<action_type>
<bench_name>-<ckpt_name>-<env_cfg_type>-<action_type>-<seed>

So if you trained with bench_name=RoboDojo, ckpt_name=cotrain, env_cfg_type=arx_x5, action_type=joint, seed=0, the run lands in checkpoints/RoboDojo-cotrain-arx_x5-joint-0/. How ckpt_name maps back to these folders at eval time is covered in Common Workflow.

Policies may also use upstream-native layouts or explicit paths in deploy.yml. Check the policy README before assuming a naming convention. For a small local dataset to play with, see Quick Start.

✅ Checks

Static checks, from the XPolicyLab repo root:

git diff --check
bash -n policy/<POLICY>/*.sh
python -m py_compile policy/<POLICY>/model.py policy/<POLICY>/deploy.py

Adapter wiring check (no simulator required) — the same debug-mode eval as Quick Start, run from policy/<POLICY>/:

export EVAL_ENV_TYPE=debug
bash eval.sh RoboDojo stack_bowls demo arx_x5 joint 0 0 0 \
  <policy_env_or_uv_path> <eval_env_conda_env>

The debug client sends plain image arrays by default. Re-run with DEBUG_OBS_ENCODED=1 to make it send encoded camera colors instead — a JPEG buffer, raw bytes, and a plain array across the three cameras — which exercises the server-side decode path that real environment clients rely on.

For a quick smoke test, policy/demo_policy accepts placeholder env names such as base. Argument details live in Common Workflow.

📬 Contact

Tianxing Chen: chentianxing2002@gmail.com