[NeurIPS 2025] AffordBot: 3D Fine-grained Embodied
Reasoning via Multimodal Large Language Models

Xinyi Wang, Xun Yang, Yanlong Xu, Yuchen Wu, Zhen Li, Na Zhao

AffordBot pioneers the Fine-grained 3D Embodied Reasoning task, introducing a unified framework that integrates multimodal large language models to jointly perform affordance grounding and motion estimation.

🔧 Installation

Clone the repository:

git clone https://github.com/hannahwxy/AffordBot.git
cd AffordBot

Set up the environment:

conda env create -f environment.yml
conda activate affordbot

If the installation of PyTorch3D fails, please comment out the pytorch3d line in the environment.yml and install it manually following official installation guide.

🗂️ Data Preparation

Download the Qwen2.5-VL-72B-Instruct, and place it in:

./ckpts/

Download the SceneFun3D dataset, and place it in:

./data/scenefun3d/dataset/

Download the SceneFun3D segmentation results generated with Mask3D, and place them in:

./data/scenefun3d/pred_mask/

Folder Structure:

AffordBot
├── ckpts
│   ├── Qwen2.5-VL-72B-Instruct
├── data
│   ├── scenefun3d
│   │   ├── annotation
│   │   ├── dataset
│   │   ├── ...
│   │   └── utils
├── eval
├── infer
├── outputs
├── ...
└── README.md

🚀 Usage

Descriptor Generation

python infer/export_descriptor.py

MLLM Deployment

python -m vllm.entrypoints.openai.api_server --model <path_to_mllm>  --served-model-name <model_name> --tensor-parallel-size 4

Inference

python infer/inference_scenefun3d.py --exp_name <experiment_name>

• Replace <experiment_name> with a custom name for your experiment.
• The results will be saved under ./outputs/experiment_name/

Evaluation

python eval/evaluate_scenefun3d.py --exp_name <experiment_name> --wmotion

• Replace <experiment_name> with the name used during inference.

🧩 Acknowledgments

We would like to thank the following repositories for their contributions:

• Fun3DU
• SceneFun3D
• SeeGround