colorful-liyu/awesome-3D-editing

6 commits

updated Oct 26, 2023

README

awesome-3D-editing

Open-source Toolboxes and Foundation Models

Methods	Task	Github
threestudio	T23D Generation
DreamGaussian	T23D Generation

Preliminary: milestones for generation and editing
3D general editing
Appearance-specific editing
Deformation
- Cage
- Control point
Avatar-related editing
Scene editing
4D editing

Paper list (Latest Update: 2023/10/26)

Preliminary: milestones for generation and editing

Classical representation

Title	arXiv	WebSite	Github	Pub.
3D Gaussian Splatting for Real-Time Radiance Field Rendering				SIGGRAPH 2023
Instant Neural Graphics Primitives				SIGGRAPH 2022

Classical text-3d generation

Title	WebSite	Github	Pub.
ATT3D: Amortized Text-to-3D Object Synthesis		-	ICCV 2023
FantASIA3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation			ICCV 2023
Zero-1-to-3: Zero-shot One Image to 3D Object			ICCV 2023
Magic3D: High-Resolution Text-to-3D Content Creation		-	CVPR 2023
Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model	-		ArXiv, 2023, 10
SyncDreamer: Generating Multiview-consistent Images from a Single-view Image			ArXiv, 2023, 09
MVDream: Multi-view Diffusion for 3D Generation			ArXiv, 2023, 08
One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization			ArXiv, 2023, 06
Shap·E: Generating Conditional 3D Implicit Functions	-		ArXiv, 2023, 05
Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures	-		ArXiv, 2022, 11
DreamFusion: Text-to-3D using 2D Diffusion			ArXiv, 2022, 09
DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation			ArXiv
Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Models		-	3DV 2024

Single-view reconstruction

Title	WebSite	Pub.
Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior		ICCV 2023
Zero-1-to-3: Zero-shot One Image to 3D Object		ICCV 2023
Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model	-	ArXiv, 2023, 10
SyncDreamer: Generating Multiview-consistent Images from a Single-view Image		ArXiv, 2023, 09
Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors		ArXiv, 2023, 06
DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation		ArXiv

3D general editing

3D local editing

Title	WebSite	Github	Pub.
FocalDreamer: Text-driven 3D Editing via Focal-fusion Assembly			ArXiv, 2023, 08
DreamEditor: Text-Driven 3D Scene Editing with Neural Fields		-	SIGGRAPH ASIA 2023
OBJect 3DIT: Language-guided 3D-aware Image Editing		-	NeurIPS 2023
Vox-E: Text-guided Voxel Editing of 3D Objects			ICCV 2023
Locally Stylized Neural Radiance Fields	-	-	ICCV 2023
Seal-3D: Interactive Pixel-Level Editing for Neural Radiance Fields			ICCV 2023
Local 3D Editing via 3D Distillation of CLIP Knowledge	-	-	CVPR 2023
RePaint-NeRF: NeRF Editting via Semantic Masks and Diffusion Models	-	-	CVPR 2023
SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field			CVPR 2023
SKED: Sketch-guided Text-based 3D Editing	-	-	CVPR 2023
Nerflets: Local Radiance Fields for Efficient Structure-Aware 3D Scene Representation from 2D Supervision		-	CVPR 2023
Progressive3D: Progressively Local Editing for Text-to-3D Content Creation with Complex Semantic Prompts			ArXiv
Language-driven Object Fusion into Neural Radiance Fields with Pose-Conditioned Dataset Updates	-	-	ArXiv
ED-NeRF: Efficient Text-Guided Editing of 3D Scene using Latent Space NeRF	-	-	ArXiv

3D (global) editing

Title	WebSite	Github	Pub.
NeRF-Art: Text-Driven Neural Radiance Fields Stylization			TVCG 2023
3D-aware Blending with Generative NeRFs			ICCV 2023
Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions			ICCV 2023
Vox-E: Text-guided Voxel Editing of 3D Objects			ICCV 2023
3D Semantic Subspace Traverser: Empowering 3D Generative Model with Shape Editing Capability	-		ICCV 2023
DreamBooth3D: Subject-Driven Text-to-3D Generation		-	ICCV 2023
Blending-NeRF: Text-Driven Localized Editing in Neural Radiance Fields		-	ICCV 2023
Learning Unified Decompositional and Compositional NeRF for Editable Novel View Synthesis		-	ICCV 2023
Edit-DiffNeRF: Editing 3D Neural Radiance Fields using 2D Diffusion Model		-	CVPR 2023
Frequency-Modulated Point Cloud Rendering with Easy Editing	-		CVPR 2023
IT3D: Improved Text-to-3D Generation with Explicit View Synthesis	-		ArXiv, 2023, 09
Blended-NeRF: Zero-Shot Object Generation and Blending in Existing Neural Radiance Fields	-	-	ArXiv, 2023, 06
Watch Your Steps: Local Image and Scene Editing by Text Instructions		-	ArXiv

Part-based editing

Title	Github	Pub.
PartNeRF: Generating Part-Aware Editable 3D Shapes without 3D Supervision		CVPR 2023
FocalDreamer: Text-driven 3D Editing via Focal-fusion Assembly		ArXiv, 2023, 08
VQ-NeRF: Neural Reflectance Decomposition and Editing with Vector Quantization	-	ArXiv

Inpainting

Title	WebSite	Github	Pub.
OBJect 3DIT: Language-guided 3D-aware Image Editing		-	NeurIPS 2023
Reference-guided Controllable Inpainting of Neural Radiance Fields		-	ICCV 2023
Removing Objects From Neural Radiance Fields			CVPR 2023
SPIn-NeRF: Multiview Segmentation and Perceptual Inpainting with Neural Radiance Fields			CVPR 2023
InpaintNeRF360: Text-Guided 3D Inpainting on Unbounded Neural Radiance Fields	-	-	ArXiv, 2023, 05
OR-NeRF: Object Removing from 3D Scenes Guided by Multiview Segmentation with Neural Radiance Fields	-		ArXiv, 2023, 05
NeRF-In: Free-Form NeRF Inpainting with RGB-D Priors	-	-	ArXiv, 2022, 06

Sketch-based editing

Title	WebSite	Github	Pub.
FantASIA3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation			ICCV 2023
Sketch2Mesh: Reconstructing and editing 3D shapes from sketches	-		ICCV 2021
SKED: Sketch-guided Text-based 3D Editing	-	-	CVPR 2023
Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures	-		ArXiv, 2022, 11

Reference-image based editing

Title	WebSite	Github	Pub.
Seal-3D: Interactive Pixel-Level Editing for Neural Radiance Fields			ICCV 2023
SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field			CVPR 2023
Language-driven Object Fusion into Neural Radiance Fields with Pose-Conditioned Dataset Updates	-	-	ArXiv

Appearance-specific editing

Stylization

Title	WebSite	Github	Pub.
NeRF-Art: Text-Driven Neural Radiance Fields Stylization			TVCG 2023
TeSTNeRF: Text-Driven 3D Style Transfer via Cross-Modal Learning	-	-	IJCAI 2023
X-Mesh:Towards Fast and Accurate Text-driven 3D Stylization via Dynamic Textual Guidance			ICCV 2023
Locally Stylized Neural Radiance Fields	-	-	ICCV 2023
Transforming Radiance Field with Lipschitz Network for Photorealistic 3D Scene Stylization	-	-	CVPR 2023
PaletteNeRF: Palette-based Appearance Editing of Neural Radiance Fields			CVPR 2023
StyleRF: Zero-shot 3D Style Transfer of Neural Radiance Fields			CVPR 2023
Ref-NPR: Reference-Based Non-Photorealistic Radiance Fields for Controllable Scene Stylization			CVPR 2023
MM-NeRF: Multimodal-Guided 3D Multi-Style Transfer of Neural Radiance Field	-	-	ArXiv
Text-driven Editing of 3D Scenes without Retraining			ArXiv

Re-texture

Title	WebSite	Github	Pub.
ICE-NeRF: Interactive Color Editing of NeRFs via Decomposition-Aware Weight Optimization		-	ICCV 2023
Texture Generation on 3D Meshes with Point-UV Diffusion			ICCV 2023
X-Mesh:Towards Fast and Accurate Text-driven 3D Stylization via Dynamic Textual Guidance			ICCV 2023
Text2Tex: Text-driven Texture Synthesis via Diffusion Models			ICCV 2023
NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editing			ECCV 2022
PaletteNeRF: Palette-based Appearance Editing of Neural Radiance Fields			CVPR 2023
StyleRF: Zero-shot 3D Style Transfer of Neural Radiance Fields			CVPR 2023
Ref-NPR: Reference-Based Non-Photorealistic Radiance Fields for Controllable Scene Stylization			CVPR 2023
TensoIR: Tensorial Inverse Rendering			CVPR 2023
Text-guided High-definition Consistency Texture Model	-	-	ArXiv, 2023, 05
Farm3D: Learning Articulated 3D Animals by Distilling 2D Diffusion		-	ArXiv, 2023, 04
ProteusNeRF: Fast Lightweight NeRF Editing using 3D-Aware Image Context		-	ArXiv
Text-driven Editing of 3D Scenes without Retraining			ArXiv
Dyn-E: Local Appearance Editing of Dynamic Neural Radiance Fields		-	ArXiv
TEGLO: High Fidelity Canonical Texture Mapping from Single-View Images		-	ArXiv
RecolorNeRF: Layer Decomposed Radiance Fields for Efficient Color Editing of 3D Scenes			ACM MM 2023
RecolorNeRF: Layer Decomposed Radiance Fields for Efficient Color Editing of 3D Scenes			ACM MM 2023

Re-light

Title	WebSite	Github	Pub.
ENVIDR: Implicit Differentiable Renderer with Neural Environment Lighting			ICCV 2023
ReNeRF: Relightable Neural Radiance Fields with Nearfield Lighting	-	-	ICCV 2023
IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis			ICCV 2023
Neural Fields meet Explicit Geometric Representations for Inverse Rendering of Urban Scenes	-	-	CVPR 2023
TensoIR: Tensorial Inverse Rendering			CVPR 2023

Deformation

Title	WebSite	Github	Pub.
Real-time Locally Injective Volumetric Deformation	-		TOG 2021
Variational Harmonic Maps for Space Deformation	-	-	TOG 2009
GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latents		-	SIGGRAPH 2023
TextDeformer: Geometry Manipulation using Text Guidance			SIGGRAPH 2023
Neural Shape Deformation Priors			NeurIPS 2022
ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators	-		ICCV 2021
Deforming Radiance Fields with Cages			ECCV 2022
NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editing			ECCV 2022
NeuralEditor: Editing Neural Radiance Fields via Manipulating Point Clouds			CVPR 2023
NeRF-Editing: Geometry Editing of Neural Radiance Fields	-		CVPR 2022
Neural Cages for Detail-Preserving 3D Deformations	-		CVPR 2020
SPIDR: SDF-based Neural Point Fields for Illumination and Deformation			ArXiv, 2022, 10
VolTeMorph: Realtime, Controllable and Generalisable Animation of Volumetric Representations	-	-	ArXiv, 2022, 08
MovingParts: Motion-based 3D Part Discovery in Dynamic Radiance Field			ArXiv

Cage

Title	arXiv	WebSite	Github	Pub.
Neural Cages for Detail-Preserving 3D Deformations		-		CVPR 2020

Control point

Title	WebSite	Github	Pub.
Real-time Locally Injective Volumetric Deformation	-		TOG 2021
Variational Harmonic Maps for Space Deformation	-	-	TOG 2009
ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators	-		ICCV 2021
EditableNeRF: Editing Topologically Varying Neural Radiance Fields by Key Points		-	CVPR 2023
NeRF-Editing: Geometry Editing of Neural Radiance Fields	-		CVPR 2022

Title	WebSite	Github	Pub.
AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatars		-	SIGGRAPH ASIA 2023
Single-Shot Implicit Morphable Faces with Consistent Texture Parameterization		-	SIGGRAPH 2023
CLIPFace: Text-guided Editing of Textured 3D Morphable Models			SIGGRAPH 2023
DreamFace: Progressive Generation of Animatable 3D Faces under Text Guidance		-	SIGGRAPH 2023
AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars			SIGGRAPH 2022
FDNeRF: Semantics-Driven Face Reconstruction, Prompt Editing and Relighting with Diffusion Models	-		NeurIPS 2023
DeformToon3D: Deformable 3D Toonification from Neural Radiance Fields			ICCV 2023
Towards High-Fidelity Text-Guided 3D Face Generation and Manipulation Using only Images	-	-	ICCV 2023
FaceCLIPNeRF: Text-driven 3D Face Manipulation using Deformable Neural Radiance Fields		-	ICCV 2023
CLIP-Actor: Text-Driven Recommendation and Stylization for Animating Human Meshes			ECCV 2022
Learning Locally Editable Virtual Humans			CVPR 2023
MEGANE: Morphable Eyeglass and Avatar Network		-	CVPR 2023
UV Volumes for Real-time Rendering of Editable Free-view Human Performance			CVPR 2023
HumanGen: Generating Human Radiance Fields with Explicit Priors	-	-	CVPR 2023
TECA: Text-Guided Generation and Editing of Compositional 3D Avatars		-	ArXiv, 2023, 09
MagicAvatar: Multimodal Avatar Generation and Animation			ArXiv, 2023, 08
HeadSculpt: Crafting 3D Head Avatars with Text		-	ArXiv, 2023, 06
Control4D: Dynamic Portrait Editing by Learning 4D GAN from 2D Diffusion-based Editor		-	ArXiv, 2023, 05
Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion		-	ArXiv, 2022, 12
LatentSwap3D: Semantic Edits on 3D Image GANs			ArXiv
Semantic 3D-aware Portrait Synthesis and Manipulation Based on Compositional Neural Radiance Field	-		AAAI 2023
TADA! Text to Animatable Digital Avatars			3DV 2024

Scene editing

Title	WebSite	Github	Pub.
IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis			ICCV 2023
Diffusion-based Generation, Optimization, and Planning in 3D Scenes		-	CVPR 2023
I2-SDF: Intrinsic Indoor Scene Reconstruction and Editing via Raytracing in Neural SDFs			CVPR 2023
Neural Fields meet Explicit Geometric Representations for Inverse Rendering of Urban Scenes	-	-	CVPR 2023
SIEDOB: Semantic Image Editing by Disentangling Object and Background	-		CVPR 2023
Text2Scene: Text-driven Indoor Scene Stylization with Part-aware Details	-	-	CVPR 2023
NeRFEditor: Differentiable Style Decomposition for Full 3D Scene Editing		-	ArXiv 2022
Set-the-Scene: Global-Local Training for Generating Controllable NeRF Scenes			ArXiv

4D editing

Title	WebSite	Github	Pub.
NeuPhysics: Editable Neural Geometry and Physics from Monocular Videos	-		NeurIPS 2022
Text-guided High-definition Consistency Texture Model	-	-	ArXiv, 2023, 05
Dyn-E: Local Appearance Editing of Dynamic Neural Radiance Fields		-	ArXiv
Make-It-4D: Synthesizing a Consistent Long-Term Dynamic Scene Video from a Single Image	-	-	ACM MM 2023

Similar repos

yyeboah/Awesome-Text-to-3D

A growing curation of Text-to-3D, Diffusion-to-3D works.

607

StellarCheng/Awesome-Text-to-3D

161

EricLee0224/awesome-nerf-editing

🧙🏻‍♂️A list of papers curated for you to dive into the Awesome Radiance Field-based 3D Editing.

506

2hiTee/awesome-3D-Generation

This is a collective repository for all 3D and 4D Object Generation papers

cwchenwang/awesome-3d-diffusion

A collection of papers on diffusion models for 3D generation.

1,257

chenyingshu/advances_3d_neural_stylization

Awesome 3D Stylization - Advances in 3D Neural Stylization

150

fnzhan/Generative-AI

[TPAMI 2023] Multimodal Image Synthesis and Editing: The Generative AI Era

753

mdyao/Awesome-3D-AIGC

A curated list of papers and open-source resources focused on 3D AIGC.

352