T-FunS3D: Task-Driven Hierarchical Open-Vocabulary 3D Functionality Segmentation

[ICRA 2026] The repository contains the official implementation of T-FunS3D. T-FunS3D constructs an open-vocabulary scene graph using 3D point cloud and posed RGB-D images of an indoor environment. Once a free-form task query is assigned, T-FunS3D segments the functional interactive object parts in the 3D point cloud.

T-FunS3D: Task-Driven Hierarchical Open-Vocabulary 3D Functionality Segmentation
Jingkun Feng, Reza Sabzevari
ICRA 2026

[Paper] [Project Page]

Requirements

• Python: 3.10
• PyTorch: 2.1+ with CUDA 12.1+
• GPU: NVIDIA GPU with ≥16GB VRAM.

Installation

# Clone the repository
git clone --recursive https://github.com/EdwardjkFeng/T-FunS3D.git
cd T-FunS3D

# Automated setup (run from the repository root)
bash install.sh

# Run the commands in the script one by one in case automatic setup fails.
# Check the alternatives provided in the script.

# Download checkpoints and example data (also from the repository root)
bash download_data.sh

Data Preparation

We download the data split of SceneFun3D using the published scripts of Fun3DU.

1. Create dataset root folder $ROOT (datasets/scenefun3d/ is the default path in the scripts).
2. Download the file lists folder from the original dataset repo and put it in the $ROOT.
3. Create the lists of two splits by running the following scripts:

cd data_preparation
python make_video_list.py train
python make_video_list.py val

4. Download the data splits:

python sun3d/data_asset_download.py --split custom --video_id_csv $ROOT/benchmark_file_lists/val_set.csv --download_dir $ROOT/val --dataset_asset laser_scan_5mm crop_mask annotations descriptions hires_wide hires_wide_intrinsics hires_depth hires_poses

python sun3d/data_asset_download.py --split custom --video_id_csv $ROOT/benchmark_file_lists/train_set.csv --download_dir $ROOT/train --dataset_asset laser_scan_5mm crop_mask annotations descriptions hires_wide hires_wide_intrinsics hires_depth hires_poses

Run the following script to prepare the scenefun3d data for the pipeline. The step includes converting data structures and preprocessing the point clouds.

bash scenefun3d_batch_data_preprocess.sh <batch_id> # The script prepares the data batch-wise. One batch contains 10 scenes by default.

Run T-FunS3D on SceneFun3D

Before running this script, adjust the folowings:

1. ROOT: dataset root
2. OUTPUT_DIRECTORY, OUTPUT_FOLDER_DIRECTORY: output paths

bash run_t-funs3d.sh

BibTeX

If you find T-FunS3D useful for your research and applications, please cite us using this BibTex:

@inproceedings{feng2026tfuns3d,
  author    = {Feng, Jingkun and Sabzevari, Reza},
  title     = {T-FunS3D: Task-Driven Hierarchical Open-Vocabulary 3D Functionality Segmentation},
  booktitle = {2026 IEEE International Conference on Robotics and Automation (ICRA)},
  year      = {2026}
}

License

This project is licensed under the MIT License. See LICENSE for full terms. Code from Third-party (e.g., OpenMask3D, SceneFun3D, Fun3DU) retains its original license.

Acknowledgements

We build on prior advances in open-vocabulary 3D segementation, foundation models, and vision-language models. Our codebase is implemented based on OpenMask3D, SceneFun3D, Fun3DU, FG-CLIP, QWen3, and Molmo. We sincerely appreciate the authors for their wonderful work and for releasing their code, models, and data processing scripts.