sim-only installs simulation / LIBERO / RoboCasa runtime dependencies plus the pinned RoboCasa source checkouts under ./src, real-only installs real-robot and remote-inference dependencies, and full installs both. Pass --skip-robocasa if you do not need the RoboCasa checkouts. RoboCasa simulator assets are downloaded by default whenever the installer installs the RoboCasa source checkouts (sim-only, full, or real-only --with-robocasa). The installer calls scripts/download_robocasa_assets.py and uses FLUXVLA_ROBOCASA_ASSET_ENDPOINT (default: HF_ENDPOINT, then https://hf-mirror.com). Use --skip-robocasa-assets to skip only the assets, or --skip-robocasa to skip both the source checkouts and the assets.

The installer selects a CUDA PyTorch profile automatically from the current CUDA toolkit / nvcc version first: CUDA >= 12.8 selects cu128, otherwise it selects cu124. If no toolkit is visible, it falls back to driver-reported CUDA and then GPU generation. Override it with --profile cu128 or --profile cu124.

After PyTorch is installed, the FlashAttention wheel is selected from the actual Python tag, PyTorch version, CUDA major version, C++ ABI, and CPU architecture. If your platform has no matching prebuilt wheel, set FLASH_ATTN_WHEEL_URL explicitly or pass --skip-flash-attn.

av is installed from the pip wheel first by default to avoid slow conda dependency resolution; if no wheel is available, the installer falls back to conda. Set FLUXVLA_AV_INSTALLER=conda if you specifically want the conda-forge package.

On Linux x86_64, the installer also installs ffmpeg=7 from conda-forge before TorchCodec. TorchCodec needs FFmpeg shared libraries such as libavutil.so.59; the imageio-ffmpeg executable and the PyAV wheel do not replace those libraries. To repair an existing Torch 2.8 environment, run:

conda install -y -c conda-forge "ffmpeg=7"
python -m pip install --force-reinstall "torchcodec==0.7.0"
python -c "from torchcodec.decoders import VideoDecoder; print('TorchCodec OK')"

Real-robot runners still require the system ROS installation itself. On ROS Noetic machines, source ROS before launching inference:

source /opt/ros/noetic/setup.bash

FlashAttention wheels are large, so GitHub release downloads can dominate a fresh install on slow networks. For repeated installs, put the exact wheel file in ./wheelhouse/, ./wheels/, or ~/.cache/fluxvla/wheels/; the installer will use it before any network request. You can also point at a local file or an internal mirror:

FLASH_ATTN_WHEEL_FILE=/path/to/flash_attn-2.8.3.post1+cu12torch2.8cxx11abiTRUE-cp310-cp310-linux_x86_64.whl \
bash scripts/install_env.sh sim-only --profile cu128

FLASH_ATTN_WHEEL_BASE_URLS="https://your-mirror.example.com/fluxvla/wheels" \
bash scripts/install_env.sh sim-only --profile cu128

The installer respects your existing pip configuration first. If that index is missing a package, or if no pip index is configured, it probes PyPI plus several common mirrors and retries by response time instead of pinning one mirror globally. For slow or unstable networks, customize the candidate list and timeouts:

PIP_INDEX_CANDIDATES="https://mirrors.aliyun.com/pypi/simple https://mirrors.cloud.tencent.com/pypi/simple https://pypi.tuna.tsinghua.edu.cn/simple https://pypi.org/simple" \
PIP_INSTALL_TIMEOUT=7200 \
PIP_NETWORK_TIMEOUT=900 \
GH_PROXY=https://ghfast.top \
bash scripts/install_env.sh full

git pull
conda install -y -c conda-forge "ffmpeg=7"
python -m pip install --upgrade -r requirements-base.txt
python -m pip install --upgrade --only-binary=:all: "av==14.2.0"
python -m pip install --upgrade "torchcodec==0.7.0"  # Torch 2.8; use 0.2.1 for Torch 2.6
python -m pip install "mujoco==3.2.6" gymnasium lxml bddl==1.0.1 hydra-core==1.2.0 robomimic==0.2.0
python -m pip install --force-reinstall --no-deps "libero @ git+https://github.com/yinchimaoliang/LIBERO.git@058fda1ddebe92918af091cb6816759ca6d003f0"
python -m pip install --force-reinstall --no-deps "robosuite @ git+https://github.com/yinchimaoliang/robosuite.git@e293cc32ff3c48957a4ebcad09952432b0dc9049"
python -m pip install --no-build-isolation -e .
python -c "import av, diffusers, peft, transformers; from diffusers import Cosmos2_5_PredictBasePipeline; print(av.__version__, diffusers.__version__, peft.__version__, transformers.__version__)"

conda create -n fluxvla python=3.10 -y
conda activate fluxvla

Important: Before running pip install -r requirements.txt, you must install PyTorch from the official CUDA index first. The default PyPI index cannot fetch CUDA-enabled builds.

# CUDA 12.8
pip install torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu128

For other CUDA versions, replace cu128 with the corresponding value (e.g., cu118, cu121). See: https://pytorch.org/get-started/locally/ and https://pytorch.org/get-started/previous-versions/.

The one-command installer downloads a prebuilt FlashAttention wheel from the official release assets. For manual installation, install the wheel matching your Python, PyTorch, and C++ ABI instead of building from source:

PYTAG=$(python - <<'PY'
import sys
print(f"cp{sys.version_info.major}{sys.version_info.minor}")
PY
)
ABI=$(python - <<'PY'
import torch
print(str(torch._C._GLIBCXX_USE_CXX11_ABI).upper())
PY
)

pip install --no-deps \
  "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.3.post1/flash_attn-2.8.3.post1+cu12torch2.8cxx11abi${ABI}-${PYTAG}-${PYTAG}-linux_x86_64.whl"

If you installed PyTorch 2.6, replace torch2.8 in the wheel URL with torch2.6.

FlashAttention wheels are tied to the installed Python, PyTorch, CUDA, and C++ ABI. flash-attn==2.5.5 is not forbidden, but it is only safe to keep when it was built for the exact PyTorch/CUDA stack you are still using. After any PyTorch upgrade, reinstall a matching FlashAttention wheel.

conda install -c conda-forge "ffmpeg=7" av=14.2.0

pip install -r requirements.txt
pip install --no-build-isolation -e .

Note: requirements.txt now composes requirements-base.txt, requirements-sim.txt, and requirements-real.txt. It does not install PyTorch; install CUDA PyTorch first or use scripts/install_env.sh. TorchCodec is also installed by the environment scripts because its version must match PyTorch. For a manual x86_64 install, use torchcodec==0.7.0 with Torch 2.8 or torchcodec==0.2.1 with Torch 2.6. TorchCodec also requires the conda/system FFmpeg shared libraries; PyAV and imageio-ffmpeg alone are not sufficient. Linux aarch64 uses the PyAV fallback.

For Jetson Orin setup, see docs/orin_flashing.md for initial flashing and JetPack setup, and docs/orin_docker_runtime.md for the validated FluxVLA Docker runtime workflow.

The validated Orin runtime is published as a Docker image:

docker pull fluxvla/fluxvla:fluxvla-orin-1.0.0
scripts/run_docker.sh

scripts/run_docker.sh uses this image by default and mounts the current repository to /workspace/FluxVLA. Runtime details are documented in docs/orin_docker_runtime.md.

GR00T N1.5 and N1.7 configs are organized under configs/gr00tn15/ and configs/gr00tn17/, respectively. RoboCasa GR00T N1.5 configs such as configs/gr00tn15/gr00tn15_eagle_3b_robocasa_30_eps_full_finetune.py require the pinned Isaac-GR00T and RoboCasa GR1 task checkouts. The one-click installer handles them for sim-only and full by default and places them under ./src:

bash scripts/install_env.sh sim-only

Use FLUXVLA_ROBOCASA_SRC_ROOT=/path/to/src to choose another checkout root, --skip-robocasa to skip these source installs, and --with-robocasa to force them in real-only mode. Runtime dependencies and the patched robosuite build are installed from requirements-sim.txt.

If you are not using the installer, the equivalent manual commands are:

pip install "mujoco==3.2.6" gymnasium lxml
pip install "robosuite @ git+https://github.com/yinchimaoliang/robosuite.git@e293cc32ff3c48957a4ebcad09952432b0dc9049"

git clone https://github.com/NVIDIA/Isaac-GR00T.git ./src/Isaac-GR00T
git -C ./src/Isaac-GR00T checkout 4af2b622892f7dcb5aae5a3fb70bcb02dc217b96
pip install --no-deps -e ./src/Isaac-GR00T

git clone https://github.com/robocasa/robocasa-gr1-tabletop-tasks.git \
  ./src/robocasa-gr1-tabletop-tasks
git -C ./src/robocasa-gr1-tabletop-tasks checkout 4840e671596f93ca03651524b9f72ffb1aadfeff
pip install --no-deps -e ./src/robocasa-gr1-tabletop-tasks

--no-deps is recommended for editable installs so the RoboCasa packages do not replace the pinned FluxVLA model stack dependencies. RoboCasa assets and datasets are covered in Data & Assets Preparation.

If you want to evaluate LIBERO on devices that do not support ray tracing (e.g., A100), please refer to EGL Device GPU Rendering Configuration.

scripts/install_env.sh sim-only and scripts/install_env.sh full now probe MuJoCo EGL automatically. If EGL devices are not visible, the installer tries to install the system packages below, creates the NVIDIA GLVND vendor file, and writes a conda activation hook for MUJOCO_GL=egl. Use FLUXVLA_EGL_SETUP=always to make this check strict, or --skip-egl-setup to skip it.

Install system dependencies

export MUJOCO_GL=egl
export PYOPENGL_PLATFORM=egl
sudo apt-get update
sudo apt-get install -y libegl1 libglvnd0 libopengl0 libegl-dev libgl1-mesa-dev libx11-dev libglew-dev libosmesa6-dev

Environment checks

Make sure /proc/1/environ contains the following environment variables:

• NVIDIA_DRIVER_CAPABILITIES=all
• NVARCH=x86_64
• NVIDIA_REQUIRE_CUDA=cuda>=12.4
• brand=tesla and driver>=470

Create an EGL configuration file

Create file /usr/share/glvnd/egl_vendor.d/10_nvidia.json with the following content:

{
    "file_format_version": "1.0.0",
    "ICD": {
        "library_path": "libEGL_nvidia.so.0"
    }
}

Then launch eval with __EGL_VENDOR_LIBRARY_FILENAMES=/usr/share/glvnd/egl_vendor.d/10_nvidia.json unless your environment already exports it.

To ensure code quality and consistency (especially for C++/CUDA code), install pre-commit hooks:

pip install pre-commit
pre-commit install

This will automatically check and format code before every commit.

Weights & Biases is used for experiment tracking and visualization. Configure it as follows:

1. Install wandb (included in requirements.txt):

pip install wandb

2. Log in to your wandb account:

wandb login

3. Set environment variables:

export WANDB_PROJECT=fluxvla        # project name (default: fluxvla)
export WANDB_ENTITY=your-team-name  # team name or username (default: None)
export WANDB_MODE=online            # online, offline, or disabled (default: online)

4. If you want to disable wandb logging during training, set:

export WANDB_MODE=disabled

Note: all wandb configuration is read from environment variables; no additional settings are needed in config files.

TensorBoard is supported as an optional logging backend for experiment metric visualization. Configure it as follows:

1. Add 'tensorboard' to active_trackers in your config file:

metric=dict(
    type='VLAMetric',
    active_trackers=('jsonl', 'wandb', 'tensorboard'),
    ...
)

Alternatively, enable it via command line without modifying the config file:

--cfg-options 'runner.metric.active_trackers=[jsonl,wandb,tensorboard]'

2. After training, launch TensorBoard to view metrics:

tensorboard --logdir work_dirs/tensorboard

Note: event files are saved to {work_dir}/tensorboard/{run_id}/ per run, enabling automatic comparison across experiments. If the TENSORBOARD_LOG_PATH environment variable is set, it will be used directly as the log directory.

Download the required datasets and place them under ./datasets. Download only the datasets you need according to your configuration.

For example, download the libero-10 dataset:

huggingface-cli download limxdynamics/FluxVLAData --repo-type dataset --include "libero_10_no_noops_lerobotv2.1/*" --local-dir ./datasets

Replace libero_10_no_noops_lerobotv2.1 with the corresponding folder name of the dataset you want to download.

For RoboCasa GR00T training with the released 30-demo subset, download the dataset under ./datasets:

huggingface-cli download limxdynamics/FluxVLAData \
  --repo-type dataset \
  --include "robocasa_gr1_24tasks_first30ep/*" \
  --local-dir ./datasets

For full-data RoboCasa GR1 training, replace the include pattern with robocasa_lerobot_V2.1/*.

Recompute normalization statistics whenever the training data, robot action semantics, action horizon, or terminal-padding policy changes. The automatic training path and the command-line tool call the same implementation, so they produce the same statistics when given the same profile and dataset settings.

For configs that opt in with auto_compute_statistics, the startup priority is:

1. use inline dataset_statistics when present;
2. otherwise use dataset_statistics_path when present;
3. otherwise compute transformed statistics once on rank 0 and save dataset_statistics.json plus dataset_statistics_metadata.json in the work directory.

The PI0.5 UR3, dual-Franka, and Tron2 training configs use this automatic path. The ALOHA configs use the official OpenPI PI0.5 Trossen statistics from gs://openpi-assets/checkpoints/pi05_base/assets/trossen/norm_stats.json for both training normalization and action denormalization. RoboCasa keeps checked-in dataset-specific statistics to avoid scanning the full dataset at every startup. To run a dataset-specific calculation manually from the repository root:

conda activate fluxvla

# UR3: six relative joints and an absolute gripper.
python tools/compute_transformed_dataset_stats.py /path/to/ur3 \
  --profile ur3 --action-horizon 50 \
  --variable-name _PI05_UR3_STATS --output /tmp/ur3_stats.py

# Dual Franka joint-position actions: relative joints and absolute grippers.
python tools/compute_transformed_dataset_stats.py /path/to/franka \
  --profile franka-qpos --action-horizon 50 \
  --variable-name _PI05_FRANKA_QPOS_STATS \
  --output /tmp/franka_qpos_stats.py

# Dual Franka Cartesian poses: fully absolute actions.
python tools/compute_transformed_dataset_stats.py /path/to/franka \
  --profile franka-eepose --action-horizon 50 \
  --variable-name _PI05_FRANKA_EEPOSE_STATS \
  --output /tmp/franka_eepose_stats.py

# Tron2: fully absolute arm, head, and gripper qpos targets.
python tools/compute_transformed_dataset_stats.py /path/to/tron2 \
  --profile tron2 --action-horizon 50 \
  --variable-name _TRON2_STATS --output /tmp/tron2_stats.py

# RoboCasa GR1: relative arm/waist joints and absolute Fourier-hand commands.
python tools/compute_transformed_dataset_stats.py /path/to/robocasa_lerobot_V2.1 \
  --profile robocasa-joint-delta --action-horizon 16 \
  --statistic-name robocasa_gr1_24tasks_joint_delta \
  --variable-name _PI05_ROBOCASA_STATS \
  --output /tmp/pi05_robocasa_joint_delta_stats.py

The tool applies configured robot coordinate/sign transforms first, converts only selected action dimensions to deltas second, and computes statistics last. Fully absolute policies can use --profile absolute or --no-delta; for example, a GR00T policy whose action column already contains absolute qpos can use auto_compute_statistics=dict(profile='absolute'). Override state_key or action_key when the dataset uses different columns.

The output contains mean, std, min, max, q01, and q99, so it can serve mean/std, min/max, or PI0.5 quantile normalization. Automatic computation inherits action_window_size, window_start_idx, supervise_terminal_padding, and statistic_name directly from the training config.

For OpenPI parity, do not regenerate the ALOHA statistics from the local training split. The checked-in _PI05_ALOHA_STATS is a direct copy of the official PI0.5 Trossen asset. If a different ALOHA calibration or data domain intentionally requires dataset-specific statistics, generate a replacement with:

python tools/compute_transformed_dataset_stats.py /path/to/aloha \
  --profile aloha --action-key observation.state \
  --gripper-input-range=-0.01,0.08 --action-horizon 50 \
  --variable-name _PI05_ALOHA_STATS --output /tmp/aloha_stats.py

When overriding the official asset, use the same replacement dictionary for both train_dataloader.dataset.dataset_statistics and inference.denormalize_action.norm_stats.

The legacy tools/compute_pi05_norm_stats.py command remains available as a compatibility wrapper around the generic tool.

Terminal padding is included by default. Add --exclude-terminal-padding if the config masks padded actions from the loss. The default action-window start is 0; set --window-start-index only when the config intentionally uses a different offset. For large datasets, use --temp-dir /path/with/free-space to place temporary memory-mapped files on a disk with sufficient capacity.

Run python tools/compute_transformed_dataset_stats.py --help for the available profiles and overrides such as custom state/action keys and delta masks.

The built-in ARM example config configs/arm/arm_clip_aloha_example.py expects a progress-labeled LeRobot v3.x dataset at ./datasets/ARM_manual_test_10Episodes_lerobotv3.0.

Download the released example dataset to the expected location with:

huggingface-cli download limxdynamics/FluxVLAData \
  --repo-type dataset \
  --include "ARM_manual_test_10Episodes_lerobotv3.0/*" \
  --local-dir ./datasets

ARM training reads the progress column directly from this dataset. For RA-BC / AW-BC on policy or DAgger datasets that do not already contain progress, first train or load an ARM checkpoint, then generate arm_progress.parquet with scripts/compute_arm_awbc_progress.py. See docs/arm.md and tools/arm_awbc/README.md.

Use the FluxVLA asset downloader below as the supported path for RoboCasa GR1 tabletop tasks. The table lists the upstream archives used by the script; manually downloading and extracting those archives is not sufficient for this stack because the script also fixes the directory layout and normalizes Objaverse XML metadata for the pinned RoboCasa GR1 checkout.

When using scripts/install_env.sh, this downloader runs by default together with the RoboCasa source checkouts unless --skip-robocasa or --skip-robocasa-assets is passed. For manual installation or refreshing the assets, run this command from the FluxVLA repository root. It downloads the required archives through the selected Hugging Face endpoint, extracts them into the RoboCasa asset directory, and normalizes the Objaverse XML metadata:

python scripts/download_robocasa_assets.py --endpoint https://hf-mirror.com

If the archives or extracted assets already exist locally, still run this script so the XML compatibility step is applied. For assets that have already been extracted into ./src/robocasa-gr1-tabletop-tasks/robocasa/models/assets, you can run only the validation and XML normalization step:

python scripts/download_robocasa_assets.py --normalize-only

Symlinks are not required; they are only a convenience when the assets already live on another local disk or shared storage.

FluxVLA SARM workflows accept standard LeRobot v2.1 or v3.x datasets. Besides the usual observation / action fields, the dataset must carry SARM subtask annotations in episodes metadata.

Published SARM example datasets on Hugging Face:

• LeRobot v3.x manual sparse+dense annotations for training / inference: limxdynamics/FluxVLAData/SARM_manual_test_10Episodes_lerobotv3.0
• LeRobot v3.x unlabeled dataset kept for manual or VLM labeling: limxdynamics/FluxVLAData/SARM_vlm_test_10Episodes_lerobotv3.0
• New LeRobot v2.1 manual conversion for training / inference and legacy-tool compatibility: limxdynamics/FluxVLAData/SARM_manual_test_10Episodes_lerobotv2.1
• New LeRobot v2.1 unlabeled conversion for manual or VLM labeling workflows: limxdynamics/FluxVLAData/SARM_vlm_test_10Episodes_lerobotv2.1

Download them under ./datasets with:

huggingface-cli download limxdynamics/FluxVLAData --repo-type dataset --include "SARM_manual_test_10Episodes_lerobotv3.0/*" --local-dir ./datasets
huggingface-cli download limxdynamics/FluxVLAData --repo-type dataset --include "SARM_vlm_test_10Episodes_lerobotv3.0/*" --local-dir ./datasets
huggingface-cli download limxdynamics/FluxVLAData --repo-type dataset --include "SARM_manual_test_10Episodes_lerobotv2.1/*" --local-dir ./datasets
huggingface-cli download limxdynamics/FluxVLAData --repo-type dataset --include "SARM_vlm_test_10Episodes_lerobotv2.1/*" --local-dir ./datasets

Use the manual_* datasets directly for training / inference. Use the vlm_* datasets as clean starting points for manual stage writing or VLM auto-annotation. Prefer the v2.1 pair when another tool expects meta/episodes.jsonl plus per-episode videos; prefer the v3.0 pair when you want to keep native LeRobot v3.x metadata layout.

Before using a LeRobot v3.x SARM dataset, sanity-check the video metadata:

• LeRobot v3.x allows either many episodes in one MP4 or one MP4 per episode.

• If many episodes share one MP4, each episode that points to that file must use correct from_timestamp / to_timestamp offsets.

• If videos are already split as file-000.mp4, file-001.mp4, ..., each episode should point to its own file_index, and from_timestamp will usually reset to 0.0.

• If the directory contains multiple MP4 files but all episodes still point to file-000.mp4, the dataset metadata is malformed and should be fixed before use.

• For ready-to-use SARM dataset structure, annotation columns, and progress inference usage, see docs/sarm.md.

• For writing manual stages or generating VLM-based annotations, see tools/sarm_annotate/README.md.

If you train with fluxvla on private datasets, you need to convert your raw data (e.g., HDF5 files collected by ALOHA robots) into the LeRobot Dataset v2.1 format. For a step-by-step conversion guide, see Data Conversion Guide.

For SARM specifically, FluxVLA supports both LeRobot v2.1 and v3.x datasets as long as the required SARM annotation columns are present. The SARM-specific metadata contract is documented in docs/sarm.md.

The converted dataset should follow this directory structure:

├── data
│   └── chunk-000
│   │   └── episode_000000.parquet
│   │   └── episode_000001.parquet
│   │   └── ... (more parquet files)
│   │   └── episode_00000N.parquet
│   └── chunk-001
│   └── ... (more chunks)
│   └── chunk-00N
├── meta
│   └── episodes.jsonl
│   └── episodes_stats.jsonl
│   └── info.json
│   └── tasks.jsonl
├── videos
│   └── chunk-000
│   │   └── camera name 0
│   │   │   └── episode_000000.mp4
│   │   │   └── episode_000001.mp4
│   │   │   └── ...(more mp4 files)
│   │   │   └── episode_00000N.mp4
│   │   └── camera name 1
│   │   └── ...(more cameras)
│   │   └── camera name N
│   └── chunk-001
│   └── ... (more chunks)
│   └── chunk-00N

Download the PI0 RoboCasa full-data checkpoint while preserving the path expected by its config:

hf download limxdynamics/FluxVLAEngine \
  --include "pi0_paligemma_robocasa_full_data_full_finetune_bs256/*" \
  --local-dir ./checkpoints

DiT4DiT requires the Cosmos-Predict2.5-2B Diffusers checkpoint. Download the exact revision and directory layout expected by configs/dit4dit/dit4dit_libero_all_full_finetune.py:

hf download nvidia/Cosmos-Predict2.5-2B \
  --revision diffusers/base/post-trained \
  --local-dir ./checkpoints/Cosmos-Predict2.5-2B

Download the FluxVLA DiT4DiT LIBERO checkpoint linked in the performance table:

hf download limxdynamics/FluxVLAEngine \
  --include "dit4dit_libero_all_full_finetune_bs256/*" \
  --local-dir ./checkpoints

The official DiT4DiT LIBERO checkpoint is optional for the default train-from-Cosmos recipe. Download it when reproducing the released model or when setting model.pretrained_name_or_path to _official_dit4dit_ckpt:

hf download mondo-robotics/dit4dit-model \
  --include "dit4dit_libero/*" \
  --local-dir ./checkpoints/dit4dit-model

Tip: You can speed up downloads with huggingface-cli download <model-name> --local-dir ./checkpoints/<model-name>.

For the built-in ARM and SARM configs, place the CLIP files under ./checkpoints/clip-vit-base-patch32:

huggingface-cli download openai/clip-vit-base-patch32 --local-dir ./checkpoints/clip-vit-base-patch32

If you use VLM-based SARM annotation, place the official SARM VLM under ./checkpoints/Qwen3-VL-30B-A3B-Instruct.

• Manage key parameters for data, models, training, evaluation, inference, and deployment through a single config file (easier to reproduce and deploy).

• Supports OpenVLA, LlavaVLA, Gr00t, Pi0, Pi0.5, FastWAM, and DiT4DiT.

• Supports Llama, Gemma, and Qwen-family LLM backbones.
• Supports DINOv2 and SigLIP vision backbones.
• Supports PaliGemma and Qwen-VL VLM backbones.

• Supports SARM training, annotation, and progress inference on LeRobot v2.1/v3.x datasets. See docs/sarm.md for details.
• Supports ARM reward modeling, progress reconstruction, and RA-BC / AW-BC sample reweighting. See docs/arm.md for details.

• Supports FSDP together with DDP, and supports LoRA training mode.
• Supports eval-after-train.
• Supports resuming training from checkpoints.

• Supports Parquet datasets and loading LeRobot-format data.
• Supports model weights in safetensors format.

• Supports multi-GPU evaluating libero on devices without ray tracing.
• Supports uploading LIBERO and RoboCasa evaluation summaries to Feishu Sheets; see Feishu Evaluation Reporting.
• Supports remote inference infrastructure with ZMQ-based server/client architecture, enabling GPU-offloaded inference for resource-constrained edge devices. See Remote Inference Serving.
• Supports RTC (Real-Time Chunking) to improve cross-chunk trajectory continuity.
• Supports accelerated inference for GR00T and PI0.5; see Inference Acceleration, including Triton fused kernels, CUDA Graph capture, and CUDA custom operators.
• Provides a minimal Oli humanoid whole-body (loco-manipulation) real-robot inference path (rospy sensor input + WebSocket control; base/hand commands are robot-SDK integration points). See docs/oli_whole_body.md.

/root/miniconda3/envs/fluxvla/bin/torchrun --standalone --nnodes 1 --nproc-per-node [NUM_GPUS] scripts/train.py --config [CONFIG_PATH] --work-dir [WORK_DIR] --cfg-options train_dataloader.per_device_batch_size=[PER_DEVICE_BATCH_SIZE]

Example:

export WANDB_MODE=disabled
/root/miniconda3/envs/fluxvla/bin/torchrun --standalone --nnodes 1 --nproc-per-node 2 scripts/train.py --config configs/pi05/pi05_paligemma_libero_10_full_finetune.py --work-dir ./checkpoints/pi05_paligemma_libero_10_full_finetune --cfg-options train_dataloader.per_device_batch_size=2

RoboCasa GR00T smoke training example:

WANDB_MODE=disabled TOKENIZERS_PARALLELISM=false \
torchrun --standalone --nnodes 1 --nproc-per-node 1 scripts/train.py \
  --config configs/gr00tn15/gr00tn15_eagle_3b_robocasa_30_eps_full_finetune.py \
  --work-dir work_dirs/smoke_groot_robocasa_train \
  --cfg-options \
    runner.type=FSDPTrainRunner \
    runner.sharding_strategy=no-shard \
    train_dataloader.per_device_batch_size=1 \
    runner.enable_gradient_checkpointing=False \
    runner.max_steps=2 \
    runner.save_iter_interval=1 \
    runner.max_keep_ckpts=2 \
    "runner.metric.active_trackers=('jsonl',)"

/root/miniconda3/envs/fluxvla/bin/torchrun --standalone --nnodes 1 --nproc-per-node [NUM_GPUS] scripts/eval.py --config [CONFIG_PATH] --ckpt-path [CKPT_PATH] --cfg-options [CFG_OPTIONS]

Example:

export WANDB_MODE=disabled
/root/miniconda3/envs/fluxvla/bin/torchrun --standalone --nnodes 1 --nproc-per-node 2 scripts/eval.py --config configs/pi05/pi05_paligemma_libero_10_full_finetune.py --ckpt-path checkpoints/pi05_paligemma_libero_10_full_finetune_bs64/checkpoints/step-028548-epoch-18-loss=0.0111.safetensors

RoboCasa GR00T evaluation example:

MUJOCO_GL=egl WANDB_MODE=disabled TOKENIZERS_PARALLELISM=false \
PYTHONHASHSEED=7 \
torchrun --standalone --nnodes 1 --nproc-per-node 1 scripts/eval.py \
  --config configs/gr00tn15/gr00tn15_eagle_3b_robocasa_30_eps_full_finetune.py \
  --ckpt-path work_dirs/gr00t_eagle_3b_robocasa_gr1_24x30_finetune_bs64/checkpoints/step-010000.safetensors \
  --cfg-options \
    eval.norm_stats_path=work_dirs/official_groot_gr1_dataset_statistics.json \
    eval.output_dir=work_dirs/gr00t_eagle_3b_robocasa_eval \
    eval.num_trials_per_task=50 \
    eval.seed=7

eval.seed controls the RoboCasa episode seeds and stochastic GR00T action sampling seeds during evaluation. PYTHONHASHSEED is independent and must be set before Python starts; using the same value is recommended when reproducing reported RoboCasa results.

export WANDB_MODE=disabled
bash scripts/train.sh [CONFIG] [WORK_DIR] --cfg-options train_dataloader.per_device_batch_size=[PER_DEVICE_BATCH_SIZE] train_dataloader.batch_size=[GLOBAL_BATCH_SIZE] runner.max_steps=[MAX_STEPS] runner.save_interval=[SAVE_INTERVAL] runner.max_keep_ckpts=[MAX_KEEP_CKPTS] --eval-after-train

To resume training from a checkpoint, use the --resume-from argument to specify the checkpoint file path. Training will continue from the saved global step, epoch, model state, and optimizer state.

Local training example:

export WANDB_MODE=disabled
/root/miniconda3/envs/fluxvla/bin/torchrun --standalone --nnodes 1 --nproc-per-node 2 scripts/train.py \
  --config configs/pi05/pi05_paligemma_libero_10_full_finetune.py \
  --work-dir ./work_dirs/pi05_paligemma_libero_10_full_finetune \
  --resume-from ./work_dirs/pi05_paligemma_libero_10_full_finetune/checkpoints/checkpoint_epoch_5.pt \
  --cfg-options train_dataloader.per_device_batch_size=2

Cluster training example:

export WANDB_MODE=disabled
bash scripts/train.sh [CONFIG] [WORK_DIR] \
  --resume-from [CHECKPOINT_PATH] \
  --cfg-options train_dataloader.per_device_batch_size=[PER_DEVICE_BATCH_SIZE] runner.max_steps=[MAX_STEPS]

export WANDB_MODE=disabled
bash scripts/eval.sh [CONFIG] [CKPT_PATH] --cfg-options [CFG_OPTIONS]

When running inference on a real robot, first install the environment on the robot side, and then run:

python scripts/inference_real_robot.py --config [CONFIG] -- ckpt-path [CKPT_PATH]

A: If you encounter Hugging Face connectivity issues (e.g., slow downloads, timeouts, or connection refused), set the following environment variable before running the command and use hf-mirror:

export HF_ENDPOINT="https://hf-mirror.com"

A: You can use the libmamba solver to speed up dependency resolution:

conda install -c conda-forge av=14.2.0 --solver=libmamba

A: This is expected. GR00T's performance on LIBERO is sensitive to random seeds, the hardware environment, and the number of training epochs. Small changes in these factors may cause noticeable fluctuations in evaluation results. It is recommended to run experiments with multiple random seeds and select the best checkpoint based on evaluation performance.

A: egl_probe needs CMake to build. Install it via conda (recommended) or apt:

conda install -c conda-forge cmake
# or
sudo apt install cmake

Note: Do not use pip install cmake. The pip package is a Python wrapper and may fail because pip isolates the build environment.

A: This is usually because your CMake version is too new for the egl_probe CMakeLists.txt. Set the following environment variable before installing:

CMAKE_POLICY_VERSION_MINIMUM=3.5 pip install -r requirements.txt

A: During installation, some dependencies may overwrite the pinned NumPy version. Reinstall the correct version directly:

pip install numpy==1.26.4