OpenHA: A Series of Open-Source Hierarchical Agentic Models in Minecraft

---

⭐️ Updates

• [2025.12] Released our paper: "Training One Model to Master Cross-Level Agentic Actions via Reinforcement Learning". We also released the related code in the CrossAgent folder.
• [2025.09] Released our paper: "OpenHA: A Series of Open-Source Hierarchical Agentic Models in Minecraft".
• [2025.09] Released the codebase.

---

🎈 Getting Started

🔧 Installation

Clone repo and install dependencies:

git clone --recurse-submodules https://github.com/CraftJarvis/OpenHA.git
conda create -n openha python=3.10
conda activate openha
pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu124  # check your CUDA version
cd OpenHA
conda install --channel=conda-forge openjdk=8 -y
pip install -e .

⚠️ Note: The script will install minestudio automatically. If you have not used MineStudio before, please check the tutorial.

For grounding-HA integration with ROCKET-1, you need a slightly modified SAM2:

cd external/SAM2
pip install -e .

---

🚀 Inference

OpenHA supports multiple ways to serve and load models.
We recommend vLLM for efficient multi-GPU / multi-process rollout. Example:

CUDA_VISIBLE_DEVICES=0,1,2,3 vllm serve CraftJarvis/minecraft-openha-qwen2vl-7b-2509  \
    --served-model-name minecraft-openha-qwen2vl-7b-2509  \
    --port 11000 \
    --limit-mm-per-prompt image=25  \
    --trust-remote-code --gpu-memory-utilization 0.90  \
    --pipeline-parallel-size 1  \
    --tensor-parallel-size 4  \
    --max-num-seqs 16 \
    --max-logprobs 20 \
    --max-model-len 32768

Once the model is loaded, run rollout:

python examples/rollout_openha.py --output_mode text_action  \
    --vlm_client_mode online \
    --system_message_tag text_action \
    --model_ips localhost --model_ports 11000 \
    --model_id minecraft-openha-qwen2vl-7b-2509 \
    --record_path "/DATA/limuyao/evaluate" \
    --max_steps_num 200 \
    --num_rollouts 8

OpenHA also supports HuggingFace Transformers (hf) or offline vllm loading.
Just change the --vlm_client_mode argument accordingly.

---

🎮 Interaction Details

You can control the output format of OpenHA via system_message_tag in rollout_openha.py.

Corresponding output_mode values:

MODE_SYSTEM_PROMPT_MAP = {
    "greedy": {"motion_coa", "grounding_coa"},
    "text_action": {"text_action"},
    "grounding": {"grounding_action"},
    "motion": {"motion_action"},
}

---

📦 Models on 🤗 Hugging Face

---

📊 Datasets on 🤗 Hugging Face

---

😊 Acknowledgement

We thank the following projects for their excellent work:

• minerl
• malmo
• MineStudio
• ROCKET-1
• SAM2

---

📝 Citation

If you find OpenHA useful, please give us a ⭐ on GitHub or cite us:

@article{wang2025openha,
      title={OpenHA: A Series of Open-Source Hierarchical Agentic Models in Minecraft}, 
      author={Zihao Wang and Muyao Li and Kaichen He and Xiangyu Wang and Zhancun Mu and Anji Liu and Yitao Liang},
      journal = {arXiv preprint arXiv:2509.13347},
      year={2025},
      url={https://arxiv.org/abs/2509.13347}, 
}