README
GRAFT: Geometric Refinement and Fitting Transformer for Human Scene Reconstruction
Pradyumna YM · Yuxuan Xue · Yue Chen · Nikita Kister · István Sárándi · Gerard Pons-Moll
Project page · arXiv · Code
TL;DR — A learned HSI prior that predicts iterative geometry-grounded refinements to reconstruct humans in scenes from a single image. Matches optimization-based contact quality at ~50× lower runtime; as a plug-and-play prior it boosts existing methods by up to 44% contact F1 — no retraining needed.
This repository holds the released weights and the benchmark assets we produced.
What is here
| file | what it is |
|---|---|
graft.pt | the main model (16.2 M parameters) |
graft_no_visual.pt | the "without visual features" ablation — same architecture with drop_vis_feats=True, so the refinement sees scene geometry only |
eval/scene_image.png | the PROX empty-scene view the model consumes as its second input |
eval/rich_scene_imgs/ | the same for RICH, per (sequence, camera) — 33 files |
eval/contact_vertice_weights.pkl | per-vertex weights for the vertex-to-scene metric |
eval/prox_quantitative_depth.pkl | PROX scene depth, carried by the evaluation context |
These are our artifacts. The datasets themselves (PROX, RICH), the SMPL-X body
models, and the NLF detector are not redistributed here — the repository's
scripts/download_assets.py tells you where to obtain each.
Licence
These weights are released for non-commercial research use only (CC BY-NC 4.0). The code in the GitHub repository is MIT; this split follows NLF's.
The pipeline also loads MapAnything's weights (CC BY-NC 4.0) and NLF's (non-commercial research), and SMPL-X is under its own non-commercial research licence — so use of GRAFT as released is non-commercial research use.
Usage
git clone https://github.com/pradyumnaym/graft && cd graft
bash install.sh
python scripts/download_assets.py # pulls this repository
from graft import GRAFTModel
model = GRAFTModel.from_pretrained("graft").cuda().eval()
out = model.predict("photo.jpg", scene_image="empty_room.jpg")
out.vertices # (n_people, 10475, 3) in the scene's camera frame, metres
The two checkpoints load into the same class and differ only in configuration.
Note they use different SMPL-X hand-mean conventions, which is why
from_pretrained will not guess a config for a bare checkpoint path — getting that
flag wrong changes the hands silently rather than failing.
Results
| model | benchmark | PA-MPJPE | V2S L2 | V2S angle | Prec | Rec | F1 |
|---|---|---|---|---|---|---|---|
graft | PROX | 49.73 | 184.36 | 50.92 | 0.556 | 0.638 | 0.594 |
graft | RICH | 46.32 | 224.66 | 34.60 | 0.473 | 0.743 | 0.578 |
graft-no-visual | PROX | 52.56 | 188.47 | 51.92 | 0.534 | 0.510 | 0.522 |
graft-no-visual | RICH | 45.52 | 223.10 | 34.06 | 0.434 | 0.717 | 0.540 |
Inference is 174 ms per image on an H100 (one person, three refinement iterations). See the repository for the per-stage breakdown and for what "reproduced exactly" is scoped to.
Citation
@misc{ym2026graft,
title = {GRAFT: Geometric Refinement and Fitting Transformer for Human Scene Reconstruction},
author = {Pradyumna YM and Yuxuan Xue and Yue Chen and Nikita Kister and Istv{\'a}n S{\'a}r{\'a}ndi and Gerard Pons-Moll},
year = {2026},
eprint = {2604.19624},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2604.19624},
}
Contributors
11 commits
Similar repos
README
GRAFT: Geometric Refinement and Fitting Transformer for Human Scene Reconstruction
Pradyumna YM · Yuxuan Xue · Yue Chen · Nikita Kister · István Sárándi · Gerard Pons-Moll
Project page · arXiv · Code
TL;DR — A learned HSI prior that predicts iterative geometry-grounded refinements to reconstruct humans in scenes from a single image. Matches optimization-based contact quality at ~50× lower runtime; as a plug-and-play prior it boosts existing methods by up to 44% contact F1 — no retraining needed.
This repository holds the released weights and the benchmark assets we produced.
What is here
| file | what it is |
|---|---|
graft.pt | the main model (16.2 M parameters) |
graft_no_visual.pt | the "without visual features" ablation — same architecture with drop_vis_feats=True, so the refinement sees scene geometry only |
eval/scene_image.png | the PROX empty-scene view the model consumes as its second input |
eval/rich_scene_imgs/ | the same for RICH, per (sequence, camera) — 33 files |
eval/contact_vertice_weights.pkl | per-vertex weights for the vertex-to-scene metric |
eval/prox_quantitative_depth.pkl | PROX scene depth, carried by the evaluation context |
These are our artifacts. The datasets themselves (PROX, RICH), the SMPL-X body
models, and the NLF detector are not redistributed here — the repository's
scripts/download_assets.py tells you where to obtain each.
Licence
These weights are released for non-commercial research use only (CC BY-NC 4.0). The code in the GitHub repository is MIT; this split follows NLF's.
The pipeline also loads MapAnything's weights (CC BY-NC 4.0) and NLF's (non-commercial research), and SMPL-X is under its own non-commercial research licence — so use of GRAFT as released is non-commercial research use.
Usage
git clone https://github.com/pradyumnaym/graft && cd graft
bash install.sh
python scripts/download_assets.py # pulls this repository
from graft import GRAFTModel
model = GRAFTModel.from_pretrained("graft").cuda().eval()
out = model.predict("photo.jpg", scene_image="empty_room.jpg")
out.vertices # (n_people, 10475, 3) in the scene's camera frame, metres
The two checkpoints load into the same class and differ only in configuration.
Note they use different SMPL-X hand-mean conventions, which is why
from_pretrained will not guess a config for a bare checkpoint path — getting that
flag wrong changes the hands silently rather than failing.
Results
| model | benchmark | PA-MPJPE | V2S L2 | V2S angle | Prec | Rec | F1 |
|---|---|---|---|---|---|---|---|
graft | PROX | 49.73 | 184.36 | 50.92 | 0.556 | 0.638 | 0.594 |
graft | RICH | 46.32 | 224.66 | 34.60 | 0.473 | 0.743 | 0.578 |
graft-no-visual | PROX | 52.56 | 188.47 | 51.92 | 0.534 | 0.510 | 0.522 |
graft-no-visual | RICH | 45.52 | 223.10 | 34.06 | 0.434 | 0.717 | 0.540 |
Inference is 174 ms per image on an H100 (one person, three refinement iterations). See the repository for the per-stage breakdown and for what "reproduced exactly" is scoped to.
Citation
@misc{ym2026graft,
title = {GRAFT: Geometric Refinement and Fitting Transformer for Human Scene Reconstruction},
author = {Pradyumna YM and Yuxuan Xue and Yue Chen and Nikita Kister and Istv{\'a}n S{\'a}r{\'a}ndi and Gerard Pons-Moll},
year = {2026},
eprint = {2604.19624},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2604.19624},
}
Similar repos
Linked in READMEs
Contributors
11 commits