Display Inverse Rendering Dataset

• 📄 Paper (ArXiv)
• 🌐 Project Page
• 💻 GitHub Repository

Introduction

This dataset is created for display inverse rendering, including multi-light stereo images captured by polarization cameras, and GT geometry (pixel-aligned point cloud and surface normals) scanned by high-precision 3D scanner.

Structure

• DIR-basic: The basic version of the dataset released with the paper. It includes stereo polarized RAW images, RGB images from a reference view, and ground-truth surface normals and point clouds. All images are captured under a multi-light configuration projected through 16×9 superpixels on the display.

  ├── A
  │  ├──GT_geometry (for reference(main) view)
  │  │  ├──'normal.npy',
  │  │  ├──'normal.png',
  │  │  ├──'point_cloud_gt.npy'
  │  ├──main
  │  │  ├──diffuseNspecular
  │  │  │  ├──'000 - 143.png',
  │  │  │  ├──'black.png',
  │  │  │  ├──'white.png',
  │  │  ├──RAW_polar
  │  │  │  ├──'000 - 143_[SHUTTER_TIME].png',
  │  │  │  ├──'black_[SHUTTER_TIME].png',
  │  │  │  ├──'white_[SHUTTER_TIME].png',
  │  ├──side
  │  │  ├──diffuseNspecular
  │  │  │  ├──'000 - 143.png',
  │  │  │  ├──'black.png',
  │  │  │  ├──'white.png',
  │  │  ├──RAW_polar
  │  │  │  ├──'000 - 143_[SHUTTER_TIME].png',
  │  │  │  ├──'black_[SHUTTER_TIME].png',
  │  │  │  ├──'white_[SHUTTER_TIME].png',
  │  ├──mask.png
  │  ├──point_cloud.npy (unprojected pixel w.r.t. depth & focal length)

• DIR-pms: This dataset follows the DiLiGeNT format and has the same composition as DIR-basic. It provides multi-light RGB images from the reference view along with related information and the ground-truth normal maps.

  ├── A [Suffix (default "PNG")]
  │  ├──'000 - 143.png',
  │  ├──'filenames.txt',
  │  ├──'light_directions.txt'
  │  ├──'light_intensities.txt',
  │  ├──'mask.png'
  │  ├──'Normal_gt.mat'

TODO

• Release training code.
• Release Display Inverse Rendering (DIR) dataset.
• Release EXPANDED version of DIR datset (HDR).
• Release EXPANDED version of DIR datset (multi-distance).