EPIC-Contact

Towards in-the-wild Egocentric 3D Hand-Object Pose Estimation  ·  ECCV 2026

Siddhant Bansal¹  ·  Zhifan Zhu¹  ·  Shashank Tripathi²  ·  Jiahe Zhao¹  ·  Michael J. Black²  ·  Dima Damen^1
1University of Bristol     ²Max Planck Institute for Intelligent Systems, Tübingen

Paper  |  Project Page  |  Dataset  |  Models

Per-clip propagation of the central-frame hand–object GT across an EPIC-Contact clip.

EPIC-Contact is an in-the-wild egocentric 3D hand–object dataset built on the EPIC-Grasps stable-grasp clips of EPIC-Kitchens. It provides posed MANO hand + object meshes and bijective hand–object contact correspondences across ≈2,272 clips and 9 object classes (pan, plate, bowl, bottle, glass, mug, cup, can, saucepan), split into 2,035 train / 237 test clips. Labels come in two tiers: a manually-verified true-GT central frame per clip (contact + posed meshes) and propagated per-frame pseudo-GT for every other frame — GT-quality relative geometry with approximate absolute camera placement, shipped with per-frame confidence scores for filtering.

It accompanies "Towards in-the-wild Egocentric 3D Hand-Object Pose Estimation" (Bansal et al., ECCV 2026).

• License: CC BY-NC 4.0 (non-commercial, attribution). Hand data is additionally under the MANO license.
• Total size: ~22 GB.

📖 Read DATASET.md first — it is the authoritative datasheet (formats, camera convention, contact schema, splits, license, citation). This file is just a map of the contents.

⬇️ Where the files live. This Hugging Face repo hosts everything except the raw frames (source_rgb/, 7.9 GB of EPIC-Kitchens images). To get the frames, download the full bundle zip (annotations + frames, ~22 GB) from OneDrive — https://uob-my.sharepoint.com/:u:/g/personal/hc23777_bristol_ac_uk/IQCk5iVfQin6Qr2iGQpEHWiwAZp6cUNJ5oHDkNnGNAI9Ly8?e=HEylx7 — unzip it, and place source_rgb/ at the dataset root so the demo/model paths resolve. The OneDrive zip also serves as a complete one-file mirror of the whole dataset.

Two label tiers

• True GT — central frames (central_frames/): one manually-verified central frame per clip, with contact correspondences + posed hand/object meshes. Trust these fully.
• Propagated pseudo-GT — per frame (per_frame_projected_npz/): all other frames, broadcast from the central frame. Relative geometry is GT-quality; absolute camera placement is approximate — filter with the per-frame confidence scores (quality_csvs/).

Contents

Which files do I need? (see DATASET.md §2)

• Just the dataset (any framework, no PyTorch): central_frames/ + per_frame_projected_npz/ + quality_csvs/ + source_rgb/ (+ contact_assets/ for dense contact, + object_assets/ for object templates/keypoints).
• Reproduce the HOPformer model: hopformer_repro_pkls/ + source_rgb/ — the model code reads the .pkls and RGB directly; no .npz needed.

source_rgb/ is not in this HF repo — both audiences get it from the OneDrive full-bundle zip (https://uob-my.sharepoint.com/:u:/g/personal/hc23777_bristol_ac_uk/IQCk5iVfQin6Qr2iGQpEHWiwAZp6cUNJ5oHDkNnGNAI9Ly8?e=HEylx7): unzip and place source_rgb/ at the dataset root.

Tools (quick start)

Two small, dependency-light scripts (numpy, pillow; the visualizer also needs trimesh, matplotlib). Run python <script>.py for full options.

# 1) Overlay the central-frame GT meshes on the frame:
python epic_contact_demo.py --clip_dir central_frames/<clip>/ --out overlay.png

# 2) Overlay a propagated per-frame mesh on its native 854x480 source frame:
python epic_contact_demo.py --clip_npz per_frame_projected_npz/<split>/<clip>.npz \
    --frame_num <N> --frame_image source_rgb/epicgrasps_storage/images/<vid>/<frame>.jpg --out f.png

# 3) Visualize the bijective hand<->object contact (2D overlay + 3D meshes):
python epic_contact_contact_viz.py --clip_dir central_frames/<clip>/ --out viz