A list of Text/Img-to-3D works. This repo mainly contains the 3D learning from 2D priors model (stable diffusion, CLIP...) works. Recently, training 3D generative models directly on 3D data has also shown promising results. Therefore, this repository has listed these methods separately.
Zero-Shot Text-Guided Object Generation with Dream Fields, Ajay Jain et al., CVPR 2022 | github
CLIP-Forge: Towards Zero-Shot Text-to-Shape Generation, Aditya Sanghi et al., CVPR 2022 | github
CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields, Can Wang et al., CVPR 2022 | github
Clip-Mesh: Generating textured meshes from text using pretrained image-text models, Mohammad Khalid, Nasir, et al., SIGGRAPH Asia 2022 | github
Text2Mesh: Text-Driven Neural Stylization for Meshes, Oscar Michel, et al., CVPR 2022 | github
DreamFusion: Text-to-3D using 2D Diffusion, Ben Poole, et al., ICLR 2022 | project page github
Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation, Haochen Wang, et al., CVPR 2023 | project page github
Magic3D: High-Resolution Text-to-3D Content Creation, Chen-Hsuan Lin, et al ., CVPR 2023 | project page github
Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures, Gal Metzer, et al., CVPR 2023 | github
TAPS3D: Text-Guided 3D Textured Shape Generation from Pseudo Supervision, Jiacheng Wei, et al., CVPR 2023 | github
Shap·E: Generating Conditional 3D Implicit Functions, Heewoo Jun, et al., | github
ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation, Zhengyi Wang, et al., | github
Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions, Ayaan Haque, et al., | github
Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation, Rui Chen, et al., ICCV 2023 | github
ATT3D: Amortized Text-to-3D Object Synthesis, Jonathan Lorraine., ICCV 2023 | project page
DreamEditor: Text-Driven 3D Scene Editing with Neural Fields, Jingyu Zhang, et al., Arxiv 2023
Vox-E Text-guided Voxel Editing of 3D Objects, Etai Sella, et al., ICCV 2023 | github
SKED: Sketch-guided Text-based 3D Editing, Aryan Mikaeili, et al., ICCV 2023 | project page
TextMesh: Generation of Realistic 3D Meshes From Text Prompts, Christina Tsalicoglou, et al., Arxiv 2023 | github
Re-imagine the Negative Prompt Algorithm: Transform 2D Diffusion into 3D, alleviate Janus problem and Beyond. Mohammadreza Armandpour, et al., Arxiv 2023 | github
IT3D: Improved Text-to-3D Generation with Explicit View Synthesis. Yiwen Chen, et al., Arxiv 2023 | github
Collaborative Score Distillation for Consistent Visual Synthesis Subin Kim, et al., Arxiv 2023 | project page
MVDREAM: MULTI-VIEW DIFFUSION FOR 3D GENERATION Yichun Shi, et al., Arxiv 2023 | project page
EfficientDreamer: High-Fidelity and Robust 3D Creation via Orthogonal-view Diffusion Prior Minda Zhao, et al., Arxiv 2023
TextMesh: Generation of Realistic 3D Meshes From Text Prompts Christina Tsalicoglou, et al., Arxiv 2023 | github
MATLABER: Material-Aware Text-to-3D via LAtent BRDF auto-EncodeR Xudong Xu, et al., Arxiv 2023 | project page
DREAMGAUSSIAN: GENERATIVE GAUSSIAN SPLATTING FOR EFFICIENT 3D CONTENT CREATION Jiaxiang Tang, et al., Arxiv 2023 | github
TEXT-TO-3D USING GAUSSIAN SPLATTING Zilong Chen, et al., Arxiv 2023 | github
Dreameditor: Text-driven 3d scene editing with neural fields Jingyu Zhuang, et al., SIGGRAPH Asia 2023
SWEETDREAMER: ALIGNING GEOMETRIC PRIORS IN 2D DIFFUSION FOR CONSISTENT TEXT-TO-3D Weiyu Li, et al., Arxiv 2023 | project page
Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Models Jianglong Ye, et al., Arxiv 2023 |project page
ED-NeRF: Efficient Text-Guided Editing of 3D Scene using Latent Space NeRF Jangho Park, et al., Arxiv 2023
T3Bench: Benchmarking Current Progress in Text-to-3D Generation Yuze He, et al., Arxiv 2023 | project page
IPDreamer: Appearance-Controllable 3D Object Generation with Image Prompts Bohan Zeng, et al., Arxiv 2023
Progressive3D: Progressively Local Editing for Text-to-3D Content Creation with Complex Semantic Prompts Xinhua Cheng, et al., Arxiv 2023 | project page
ENHANCING HIGH-RESOLUTION 3D GENERATION THROUGH PIXEL-WISE GRADIENT CLIPPING Zijie Pan, et al., Arxiv 2023 | github
TAMING MODE COLLAPSE IN SCORE DISTILLATION FOR TEXT-TO-3D GENERATION Openreview 2023
STEINDREAMER: VARIANCE REDUCTION FOR TEXTTO-3D SCORE DISTILLATION VIA STEIN IDENTITY Openreview 2023
TEXT-TO-3D WITH CLASSIFIER SCORE DISTILLATION Xin Yu, et al., Arxiv 2023., | project page
Noise-Free Score Distillation Oren Katzir, et al., Arxiv 2023 | github
TEXT-TO-3D GENERATION WITH BIDIRECTIONAL DIFFUSION USING BOTH 2D AND 3D PRIORS Openreview 2023
LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matching Yixun Liang, et al., Arxiv 2023 | github
GaussianDiffusion: 3D Gaussian Splatting for Denoising Diffusion Probabilistic Models with Structured Noise Xinhai Li, et al., Arxiv 2023
RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3D Lingteng Qiu., et al., Arxiv 2023 | project page
Learn to Optimize Denoising Scores for 3D Generation - A Unified and Improved Diffusion Prior on NeRF and 3D Gaussian Splatting Xiaofeng Yang., et al., Arxiv 2023 | project page
GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models Taoran Yi., et al., Arxiv 2023 | project page
Text2Immersion: Generative Immersive Scene with 3D Gaussians Hao Ouyang, et al., Arxiv 2023 | project page
StableDreamer: Taming Noisy Score Distillation Sampling for Text-to-3D Pengsheng Guo, et al., Arxiv 2023
DreamPropeller: Supercharge Text-to-3D Generation with Parallel Sampling Linqi Zhou, et al., Arxiv 2023 | project page
HyperFields:Towards Zero-Shot Generation of NeRFs from Text Sudarshan Babu, et al., ICML 2024 | project page
RealFusion 360◦ Reconstruction of Any Object from a Single Image, Luke Melas-Kyriazi, et al., ICCV 2023 | github
Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors, Guocheng Qian, et al., | github
One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization, Minghua Liu, et al., | github
Nerdi: Single-view nerf synthesis with language-guided diffusion as general image priors Congyue Deng, et al., CVPR 2023
NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360° Views Dejia Xu et al., CVPR 2023 | github
Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior Junshu Tang et al., ICCV 2023 | github
Zero-1-to-3: Zero-shot One Image to 3D Object Ruoshi Liu, et al., ICCV2023 | github
SyncDreamer: Generating Multiview-consistent Images from a Single-view Image Yuan Liu, et al., Arxiv 2023 | github
MVDream: Multi-view Diffusion for 3D Generation Yichun Shi, et al., Arxiv 2023 | github
Consistent123: One Image to Highly Consistent 3D Asset Using Case-Aware Diffusion Priors Yukang Lin, et al., Arxiv 2023 | github
HiFi-123: Towards High-fidelity One Image to 3D Content Generation Wangbo Yu, et al., Arxiv 2023 | github
ConsistNet: Enforcing 3D Consistency for Multi-view Images Diffusion Jiayu Yang, et al., Arxiv 2023 | Project Page
DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior Jingxiang Sun, et al., Arxiv 2023 | Project Page github
Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model Ruoxi Shi, et al., Arxiv 2023 | github
Wonder3D: Single Image to 3D using Cross-Domain Diffusion Xiaoxiao Long, et al., Arxiv 2023 | github
ImageDream: Image-Prompt Multi-view Diffusion for 3D Generation Peng Wang, et al., Arxiv 2023 | Project Page
One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion Minghua Liu, et al., Arxiv 2023 | github Project Page
Free3D: Consistent Novel View Synthesis without 3D Representation Chuanxia Zheng, et al., Arxiv 2023 | github
Repaint123: Fast and High-quality One Image to 3D Generation with Progressive Controllable 2D Repainting Junwu Zhang, et al., Arxiv 2023 | github
DMV3D:Denoising Multi-View Diffusion using 3D Large Reconstruction Model Yinghao Xu, et al., Arxiv 2023 | Project Page
PF-LRM: Pose-Free Large Reconstruction Model for Joint Pose and Shape Prediction Peng Wang, et al., Arxiv 2023 | Project Page
Instant3D : Instant Text-to-3D Generation Ming Li, et al., Arxiv 2023 | Project Page
LRM: Large Reconstruction Model for Single Image to 3D Yicong Hong., et al., Arxiv 2023 | project page
MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers Yawar Siddiqui, et al., Arxiv 2023 | project page
CAD: Photorealistic 3D Generation via Adversarial Distillation Ziyu Wan, et al., Arxiv 2023 | project page
A list of Text/Img-to-3D works. This repo mainly contains the 3D learning from 2D priors model (stable diffusion, CLIP...) works. Recently, training 3D generative models directly on 3D data has also shown promising results. Therefore, this repository has listed these methods separately.
Zero-Shot Text-Guided Object Generation with Dream Fields, Ajay Jain et al., CVPR 2022 | github
CLIP-Forge: Towards Zero-Shot Text-to-Shape Generation, Aditya Sanghi et al., CVPR 2022 | github
CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields, Can Wang et al., CVPR 2022 | github
Clip-Mesh: Generating textured meshes from text using pretrained image-text models, Mohammad Khalid, Nasir, et al., SIGGRAPH Asia 2022 | github
Text2Mesh: Text-Driven Neural Stylization for Meshes, Oscar Michel, et al., CVPR 2022 | github
DreamFusion: Text-to-3D using 2D Diffusion, Ben Poole, et al., ICLR 2022 | project page github
Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation, Haochen Wang, et al., CVPR 2023 | project page github
Magic3D: High-Resolution Text-to-3D Content Creation, Chen-Hsuan Lin, et al ., CVPR 2023 | project page github
Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures, Gal Metzer, et al., CVPR 2023 | github
TAPS3D: Text-Guided 3D Textured Shape Generation from Pseudo Supervision, Jiacheng Wei, et al., CVPR 2023 | github
Shap·E: Generating Conditional 3D Implicit Functions, Heewoo Jun, et al., | github
ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation, Zhengyi Wang, et al., | github
Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions, Ayaan Haque, et al., | github
Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation, Rui Chen, et al., ICCV 2023 | github
ATT3D: Amortized Text-to-3D Object Synthesis, Jonathan Lorraine., ICCV 2023 | project page
DreamEditor: Text-Driven 3D Scene Editing with Neural Fields, Jingyu Zhang, et al., Arxiv 2023
Vox-E Text-guided Voxel Editing of 3D Objects, Etai Sella, et al., ICCV 2023 | github
SKED: Sketch-guided Text-based 3D Editing, Aryan Mikaeili, et al., ICCV 2023 | project page
TextMesh: Generation of Realistic 3D Meshes From Text Prompts, Christina Tsalicoglou, et al., Arxiv 2023 | github
Re-imagine the Negative Prompt Algorithm: Transform 2D Diffusion into 3D, alleviate Janus problem and Beyond. Mohammadreza Armandpour, et al., Arxiv 2023 | github
IT3D: Improved Text-to-3D Generation with Explicit View Synthesis. Yiwen Chen, et al., Arxiv 2023 | github
Collaborative Score Distillation for Consistent Visual Synthesis Subin Kim, et al., Arxiv 2023 | project page
MVDREAM: MULTI-VIEW DIFFUSION FOR 3D GENERATION Yichun Shi, et al., Arxiv 2023 | project page
EfficientDreamer: High-Fidelity and Robust 3D Creation via Orthogonal-view Diffusion Prior Minda Zhao, et al., Arxiv 2023
TextMesh: Generation of Realistic 3D Meshes From Text Prompts Christina Tsalicoglou, et al., Arxiv 2023 | github
MATLABER: Material-Aware Text-to-3D via LAtent BRDF auto-EncodeR Xudong Xu, et al., Arxiv 2023 | project page
DREAMGAUSSIAN: GENERATIVE GAUSSIAN SPLATTING FOR EFFICIENT 3D CONTENT CREATION Jiaxiang Tang, et al., Arxiv 2023 | github
TEXT-TO-3D USING GAUSSIAN SPLATTING Zilong Chen, et al., Arxiv 2023 | github
Dreameditor: Text-driven 3d scene editing with neural fields Jingyu Zhuang, et al., SIGGRAPH Asia 2023
SWEETDREAMER: ALIGNING GEOMETRIC PRIORS IN 2D DIFFUSION FOR CONSISTENT TEXT-TO-3D Weiyu Li, et al., Arxiv 2023 | project page
Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Models Jianglong Ye, et al., Arxiv 2023 |project page
ED-NeRF: Efficient Text-Guided Editing of 3D Scene using Latent Space NeRF Jangho Park, et al., Arxiv 2023
T3Bench: Benchmarking Current Progress in Text-to-3D Generation Yuze He, et al., Arxiv 2023 | project page
IPDreamer: Appearance-Controllable 3D Object Generation with Image Prompts Bohan Zeng, et al., Arxiv 2023
Progressive3D: Progressively Local Editing for Text-to-3D Content Creation with Complex Semantic Prompts Xinhua Cheng, et al., Arxiv 2023 | project page
ENHANCING HIGH-RESOLUTION 3D GENERATION THROUGH PIXEL-WISE GRADIENT CLIPPING Zijie Pan, et al., Arxiv 2023 | github
TAMING MODE COLLAPSE IN SCORE DISTILLATION FOR TEXT-TO-3D GENERATION Openreview 2023
STEINDREAMER: VARIANCE REDUCTION FOR TEXTTO-3D SCORE DISTILLATION VIA STEIN IDENTITY Openreview 2023
TEXT-TO-3D WITH CLASSIFIER SCORE DISTILLATION Xin Yu, et al., Arxiv 2023., | project page
Noise-Free Score Distillation Oren Katzir, et al., Arxiv 2023 | github
TEXT-TO-3D GENERATION WITH BIDIRECTIONAL DIFFUSION USING BOTH 2D AND 3D PRIORS Openreview 2023
LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matching Yixun Liang, et al., Arxiv 2023 | github
GaussianDiffusion: 3D Gaussian Splatting for Denoising Diffusion Probabilistic Models with Structured Noise Xinhai Li, et al., Arxiv 2023
RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3D Lingteng Qiu., et al., Arxiv 2023 | project page
Learn to Optimize Denoising Scores for 3D Generation - A Unified and Improved Diffusion Prior on NeRF and 3D Gaussian Splatting Xiaofeng Yang., et al., Arxiv 2023 | project page
GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models Taoran Yi., et al., Arxiv 2023 | project page
Text2Immersion: Generative Immersive Scene with 3D Gaussians Hao Ouyang, et al., Arxiv 2023 | project page
StableDreamer: Taming Noisy Score Distillation Sampling for Text-to-3D Pengsheng Guo, et al., Arxiv 2023
DreamPropeller: Supercharge Text-to-3D Generation with Parallel Sampling Linqi Zhou, et al., Arxiv 2023 | project page
HyperFields:Towards Zero-Shot Generation of NeRFs from Text Sudarshan Babu, et al., ICML 2024 | project page
RealFusion 360◦ Reconstruction of Any Object from a Single Image, Luke Melas-Kyriazi, et al., ICCV 2023 | github
Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors, Guocheng Qian, et al., | github
One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization, Minghua Liu, et al., | github
Nerdi: Single-view nerf synthesis with language-guided diffusion as general image priors Congyue Deng, et al., CVPR 2023
NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360° Views Dejia Xu et al., CVPR 2023 | github
Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior Junshu Tang et al., ICCV 2023 | github
Zero-1-to-3: Zero-shot One Image to 3D Object Ruoshi Liu, et al., ICCV2023 | github
SyncDreamer: Generating Multiview-consistent Images from a Single-view Image Yuan Liu, et al., Arxiv 2023 | github
MVDream: Multi-view Diffusion for 3D Generation Yichun Shi, et al., Arxiv 2023 | github
Consistent123: One Image to Highly Consistent 3D Asset Using Case-Aware Diffusion Priors Yukang Lin, et al., Arxiv 2023 | github
HiFi-123: Towards High-fidelity One Image to 3D Content Generation Wangbo Yu, et al., Arxiv 2023 | github
ConsistNet: Enforcing 3D Consistency for Multi-view Images Diffusion Jiayu Yang, et al., Arxiv 2023 | Project Page
DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior Jingxiang Sun, et al., Arxiv 2023 | Project Page github
Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model Ruoxi Shi, et al., Arxiv 2023 | github
Wonder3D: Single Image to 3D using Cross-Domain Diffusion Xiaoxiao Long, et al., Arxiv 2023 | github
ImageDream: Image-Prompt Multi-view Diffusion for 3D Generation Peng Wang, et al., Arxiv 2023 | Project Page
One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion Minghua Liu, et al., Arxiv 2023 | github Project Page
Free3D: Consistent Novel View Synthesis without 3D Representation Chuanxia Zheng, et al., Arxiv 2023 | github
Repaint123: Fast and High-quality One Image to 3D Generation with Progressive Controllable 2D Repainting Junwu Zhang, et al., Arxiv 2023 | github
DMV3D:Denoising Multi-View Diffusion using 3D Large Reconstruction Model Yinghao Xu, et al., Arxiv 2023 | Project Page
PF-LRM: Pose-Free Large Reconstruction Model for Joint Pose and Shape Prediction Peng Wang, et al., Arxiv 2023 | Project Page
Instant3D : Instant Text-to-3D Generation Ming Li, et al., Arxiv 2023 | Project Page
LRM: Large Reconstruction Model for Single Image to 3D Yicong Hong., et al., Arxiv 2023 | project page
MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers Yawar Siddiqui, et al., Arxiv 2023 | project page
CAD: Photorealistic 3D Generation via Adversarial Distillation Ziyu Wan, et al., Arxiv 2023 | project page