Dataset Card for StableText2Brick

This dataset contains over 47,000 toy brick structures of over 28,000 unique 3D objects accompanied by detailed captions. It was used to train BrickGPT, the first approach for generating physically stable toy brick models from text prompts, as described in Generating Physically Stable and Buildable Brick Structures from Text.

Dataset Details

Dataset Description

• Curated by: Carnegie Mellon University Generative Intelligence Lab
• Funded by: This work is partly supported by the Packard Foundation, Cisco Research Grant, and Amazon Faculty Award. This work is also in part supported by the Manufacturing Futures Institute, Carnegie Mellon University, through a grant from the Richard King Mellon Foundation. KD is supported by the Microsoft Research PhD Fellowship.
• License: MIT

Dataset Sources

• Repository: AvaLovelace1/BrickGPT
• Paper: Generating Physically Stable and Buildable Brick Structures from Text
• Project Page: https://avalovelace1.github.io/BrickGPT/

Dataset Structure

Each row of the dataset contains a different brick structure with the following fields:

• structure_id: A unique ID identifying the brick structure.
• object_id: The ID of the ShapeNet object that the brick structure was created from. Multiple brick structures may correspond to the same object and have the same object_id.
• category_id: The ID of the ShapeNet category to which the object belongs; e.g., 03001627 for "chair".
• captions: A list of caption strings describing the brick structure.
• bricks: The brick structure in text format. Each line of the text is in the format hxw (x,y,z), and represents a 1-unit-tall rectangular brick with dimensions hxw placed at coordinates (x,y,z). Each structure fits within a 20x20x20 world grid.
• stability_scores: A 20x20x20 array containing the stability scores of each voxel in the world grid, as computed in the paper. Each score is a real number in [0,1]. For voxels containing bricks, a nonzero score indicates a stable brick, while a zero score indicates an unstable brick. Empty voxels are assigned a score of 1.

Dataset Creation

See the paper for details on dataset creation.

Limitations

All structures in this dataset are composed of 1-unit-tall cuboid bricks and fit within a 20x20x20 grid. The dataset contains only 21 object categories: basket, bed, bench, birdhouse, bookshelf, bottle, bowl, bus, camera, car, chair, guitar, jar, mug, piano, pot, sofa, table, tower, train, vessel.

Citation

If you find this dataset useful for your research, please cite the following work.

@inproceedings{pun2025brickgpt,
    title     = {Generating Physically Stable and Buildable Brick Structures from Text},
    author    = {Pun, Ava and Deng, Kangle and Liu, Ruixuan and Ramanan, Deva and Liu, Changliu and Zhu, Jun-Yan},
    booktitle = {ICCV},
    year      = {2025}
}

Dataset Card Contact

Ava Pun (apun@andrew.cmu.edu)