---

📖  Documentation  |  🤗  Model Hub

1. Model Introduction

FluxVLA Engine is an integrated engineering platform designed for embodied intelligence applications. It follows the core design principles of unified configuration, standardized interfaces, module decoupling, and deployability, forming a complete engineering loop from data collection to real-world deployment. With a focus on building a "standardized industrial-academic-research foundation," FluxVLA significantly lowers the engineering threshold for VLA (Visual Language Agent) research and development.

Key Features

🧩 Rich VLA Model Zoo: Built-in support for mainstream VLA families including OpenVLA, LLaVA-VLA, GR00T, Pi0, Pi0.5, and DreamZero, enabling fast comparison, fine-tuning, and deployment across different model architectures.

⚡ Extreme Inference Speed: CUDA-accelerated operator fusion. With the GR00T-N1.5, the system achieves a high-frequency of 42.8Hz on the RTX 5090.

🛠 All-in-One: A unified configuration file manages critical parameters for data, models, training, evaluation, inference, and deployment, ensuring easy reproducibility and deployment.

🖥 Flexible Configuration: Supports mainstream visual backbones, LLM (Large Language Models), VLM (Vision-Language Models), and VLA (Vision-Language Agents). You can also create and customize your own VLA.

🤖 Real-world Deployment Ready: In addition to mainstream simulators, FluxVLA comes with ready-to-use deployment scripts for Aloha, Tron2, and UR3, eliminating the need for custom integration.

💨 Smooth Motion: Optimized for real-world training/inference with RTC trajectory guidance, ensuring accurate and continuous motion even at 40Hz+ frequencies.

📊 Strong Benchmark Performance: Provides competitive LIBERO benchmark results across spatial, object, goal, and long-horizon tasks, making it easier to evaluate VLA models under standardized settings.

🛰 Remote Inference Ready: Includes a ZMQ-based server/client inference framework, enabling GPU-offloaded remote inference for resource-constrained robot-side devices.

🧪 Scalable Training Pipeline: Supports distributed training with DDP/FSDP, LoRA fine-tuning, eval-after-train, and checkpoint-based resume, covering both local debugging and cluster-scale training workflows.

2. Model Zoo

FluxVLA Engine supports multiple VLA model families, each optimized for different use cases:

Available Pretrained Models

Trained FluxVLA Checkpoints

3. Evaluation Results

FluxVLA demonstrates state-of-the-art performance across multiple LIBERO benchmark suites:

Inference Speed Comparison

Note: Optimized versions use CUDA-accelerated operator fusion and Triton kernels.

4. Architecture Overview

FluxVLA Engine supports flexible architecture combinations:

5. Installation & Deployment

Quick Installation

# Clone repository
git clone https://github.com/limxdynamics/FluxVLA.git
cd FluxVLA

# Create environment
conda create -n fluxvla python=3.10 -y
conda activate fluxvla

# Install PyTorch (CUDA 12.4)
pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 \
  --index-url https://download.pytorch.org/whl/cu124

# Install flash-attention
pip install psutil ninja packaging
MAX_JOBS=8 pip install flash-attn==2.5.5 --no-build-isolation

# Install FluxVLA
pip install -r requirements.txt
pip install --no-build-isolation -e .

Deployment Options

FluxVLA supports multiple deployment scenarios:

1. Local Training & Evaluation: Single-node or multi-node distributed training with DDP/FSDP
2. Real-Robot Deployment: Direct on-robot inference with optimized runtime
3. Remote Inference: GPU server + robot client architecture via ZMQ
4. Simulation: LIBERO, Isaac Sim integration

Hardware Requirements

6. Model Usage

Training

Train a VLA model on LIBERO-10 dataset:

# Single-node training
torchrun \
  --standalone \
  --nnodes 1 \
  --nproc-per-node 2 \
  scripts/train.py \
  --config configs/pi05/pi05_paligemma_libero_10_full_finetune.py \
  --work-dir ./checkpoints/pi05_paligemma_libero_10_full_finetune \
  --cfg-options train_dataloader.per_device_batch_size=2

# Or use the launcher script
bash scripts/train.sh \
  configs/gr00t/gr00t_eagle_3b_libero_10_full_finetune.py \
  work_dirs/gr00t_eagle_3b_libero_10_full_finetune

Evaluation

Evaluate a trained model on LIBERO benchmarks:

# Evaluate on LIBERO-10
torchrun \
  --standalone \
  --nnodes 1 \
  --nproc-per-node 2 \
  scripts/eval.py \
  --config configs/pi05/pi05_paligemma_libero_10_full_finetune.py \
  --ckpt-path checkpoints/pi05_paligemma_libero_10_full_finetune_bs64/checkpoints/latest-checkpoint.safetensors

# Or use the launcher script
bash scripts/eval.sh \
  configs/pi05/pi05_paligemma_libero_10_full_finetune.py \
  checkpoints/pi05_paligemma_libero_10_full_finetune_bs64/checkpoints/latest-checkpoint.safetensors

Inference

Simulation Inference

import torch
from fluxvla.models import build_model
from fluxvla.datasets import build_dataset
from fluxvla.transforms import build_transforms

# Load configuration
config = load_config('configs/pi05/pi05_paligemma_libero_10_full_finetune.py')

# Build model
model = build_model(config.model)
model.load_checkpoint('path/to/checkpoint.safetensors')
model.eval()
model.cuda()

# Build transforms
transforms = build_transforms(config.transforms)

# Inference loop
with torch.no_grad():
    # Prepare input
    image = transforms['image'](raw_image)
    instruction = "Pick up the red cube"
    
    # Get action prediction
    action = model.predict(
        image=image,
        instruction=instruction,
        proprioception=robot_state
    )
    
    # Execute action on robot
    robot.execute(action)

Real-Robot Inference

# Direct on-robot inference
python scripts/inference_real_robot.py \
  --config configs/pi05/pi05_paligemma_aloha_full_finetune.py \
  --ckpt-path checkpoints/pi05_paligemma_libero_10_full_finetune_bs64/checkpoints/latest-checkpoint.safetensors

Remote Inference

For deployment scenarios where the robot cannot host the full model:

# On GPU server
python scripts/remote_inference_server.py \
  --config configs/pi05/pi05_paligemma_aloha_remote_inference.py \
  --ckpt-path checkpoints/pi05_paligemma_libero_10_full_finetune_bs64/checkpoints/latest-checkpoint.safetensors \
  --host 0.0.0.0 \
  --port 5555

# On robot client
bash scripts/remote_inference_client.sh

7. Advanced Features

RTC (Real-Time Control) Trajectory Guidance

FluxVLA includes RTC trajectory guidance for smooth, continuous motion control:

RTC improves action continuity by conditioning on recent action history, reducing jitter and improving success rates in real-world deployment.

# Enable RTC in config
model = dict(
    type='Pi05VLA',
    rtc_enabled=True,
    rtc_prefix_length=5,
    rtc_temperature=0.1,
)

Inference Acceleration

FluxVLA provides multiple acceleration techniques:

1. CUDA Operator Fusion: Custom CUDA kernels for attention and MLP layers
2. Triton Kernels: Optimized Triton implementations for key operations
3. CUDA Graph: Graph capture for reduced kernel launch overhead
4. Mixed Precision: FP16/BF16 inference with minimal accuracy loss

# Enable acceleration in config
python scripts/inference.py \
  --config configs/pi05/pi05_paligemma_libero_10_full_finetune.py \
  --ckpt-path checkpoints/latest.safetensors \
  --use-cuda-graph \
  --use-triton-kernels

Distributed Training

FluxVLA supports both DDP and FSDP for multi-GPU training:

# Multi-node training with FSDP
torchrun \
  --nnodes 4 \
  --nproc-per-node 8 \
  --rdzv-backend c10d \
  --rdzv-endpoint $MASTER_ADDR:$MASTER_PORT \
  scripts/train.py \
  --config configs/pi05/pi05_paligemma_libero_10_full_finetune.py \
  --work-dir ./checkpoints/distributed_training

LoRA Fine-tuning

For efficient fine-tuning with limited compute:

# Enable LoRA in config
lora = dict(
    enabled=True,
    r=16,
    lora_alpha=32,
    target_modules=['q_proj', 'v_proj', 'k_proj', 'o_proj'],
    lora_dropout=0.05,
)

8. Supported Datasets

FluxVLA supports multiple robot learning datasets:

9. Citation

If you use FluxVLA in your research, please cite:

@software{fluxvla2026,
  title={FluxVLA Engine: An All-in-One VLA Engineering Platform for Embodied AI},
  author={LimX Dynamics},
  year={2026},
  url={https://github.com/limxdynamics/FluxVLA},
  note={GitHub repository}
}

10. License

The FluxVLA code is released under the Apache-2.0 License. See LICENSE for details.

Model checkpoints derived from third-party base models remain subject to their respective licenses. The Cosmos3-Edge checkpoints are derived from NVIDIA Cosmos3-Edge and are subject to the Open Model Development and Weight License 1.1 (OpenMDW 1.1).

11. Acknowledgements

FluxVLA builds on and benefits from the following open-source projects:

• LeRobot - Robot learning datasets and tools
• NVIDIA Isaac GR00T - Foundation model for humanoid robots
• OpenVLA - Open-source VLA baseline
• OpenPI - Flow-matching VLA framework
• LLaVA - Vision-language model architecture
• Qwen - Large language models
• DeepSpeed - Distributed training optimization
• Triton - GPU programming framework
• RTC - Real-time control trajectory guidance

We thank the authors and contributors of these projects for their excellent work.

12. Contact Us

For questions, feedback, collaboration, or technical support, please contact us:

• Mason: mason@limxdynamics.com
• Wayne Mao: waynemao@limxdynamics.com

You can also:

• Open an issue on GitHub
• Visit our documentation: English | 中文
• Check out our Hugging Face page: limxdynamics/FluxVLAEngine

---

Project Links:

• 🌐 GitHub: https://github.com/limxdynamics/FluxVLA
• 🤗 Hugging Face: https://huggingface.co/limxdynamics/FluxVLAEngine
• 📖 English Documentation: https://fluxvla.limxdynamics.com
• 📖 中文文档: https://fluxvla.limxdynamics.com/zh/