awesome-3d-photography

A paper list of 3D photography and cinemagraph.

This list is non-exhaustive. Feel free to pull requests or create issues to add papers.

Following this repo, I use some icons to (imprecisely) differentiate the 3D representations:

• :leaves: Layered Depth Image
• :gem: Mesh
• :airplane: Multiplane Images
• :taxi: Nerf
• :cloud: Point Cloud
• :space_invader: Voxel
• ☃️ 3D Gaussian Splatting

3D Photography

Novel view synthesis with a single input image based on 3D geometry.

• [arXiv 2024] GenWarp: Single Image to Novel Views with Semantic-Preserving Generative Warping [paper] [project page] ☃️
• [CVPR 2024] WonderJourney: Going from Anywhere to Everywhere [paper] [code] [project page] :cloud:
• [CVPR 2024] MultiDiff: Consistent Novel View Synthesis from a Single Image [paper] [project page]
• [arXiv 2023] Novel View Synthesis with View-Dependent Effects from a Single Image [paper] [project page]
• [SA 2023] SinMPI: Novel View Synthesis from a Single Image with Expanded Multiplane Images [paper] [code] :leaves:
• [TOG 2023] ReShader: View-Dependent Highlights for Single Image View-Synthesis [paper] [code] [project page]
• [NeurIPS 2023] Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervision [paper] [code] [project page]
• [ICCV 2023] Diffuse3D: Wide-Angle 3D Photography via Bilateral Diffusion [paper] [code] :leaves:
• [ICCV 2023] LoLep: Single-View View Synthesis with Locally-Learned Planes and Self-Attention Occlusion Inference [paper] :airplane:
• [arXiv 2023] Generative Novel View Synthesis with 3D-Aware Diffusion Models [paper] [project page]
• [CVPR 2023] Structural Multiplane Image: Bridging Neural View Synthesis and 3D Reconstruction [paper] :airplane:
• [TIP 2023] Single-view Neural Radiance Fields with Depth Teacher [paper] :taxi:
• [ECCV 2022] InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Images [paper] [project page]
• [SIGGRAPH 2022] Single-View View Synthesis in the Wild with Learned Adaptive Multiplane Images [paper] [code] [project page] :airplane:
• [CVPR 2022] Efficient Geometry-aware 3D Generative Adversarial Networks [paper] [code] [project page]
• [CVPRW 2022] Artistic Style Novel View Synthesis Based on A Single Image [paper] [code] [project page] :cloud:
• [CVPR 2022] 3D Photo Stylization: Learning to Generate Stylized Novel Views from a Single Image [paper] [code] [project page] :cloud:
• [ICCV 2021] Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Image [paper] [code] [project page] :gem:
• [ICCV 2021] MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesis [paper] [code] [project page] :airplane: :taxi:
• [ICCV 2021] PixelSynth: Generating a 3D-Consistent Experience from a Single Image [paper] [code] [project page] :cloud:
• [ICCV 2021] SLIDE: Single Image 3D Photography with Soft Layering and Depth-aware Inpainting [paper] [project page] :gem:
• [ICCV 2021] Video Autoencoder: self-supervised disentanglement of static 3D structure and motion [paper] [code] [project page] :space_invader:
• [ICCV 2021] Worldsheet: Wrapping the World in a 3D Sheet for View Synthesis from a Single Image [paper] [code] [project page] :gem:
• [CVPR 2021] Layout-Guided Novel View Synthesis from a Single Indoor Panorama [paper] [dataset]
• [WACV 2021] Adaptive Multiplane Image Generation from a Single Internet Picture [paper] :airplane:
• [CVPR 2020] Single-View View Synthesis with Multiplane Images [paper] [code] [project page] :airplane:
• [CVPR 2020] SynSin: End-to-end View Synthesis from a Single Image [paper] [code] [project page] :cloud:
• [CVPR 2020] 3D Photography using Context-aware Layered Depth Inpainting [paper] [code] [project page] :leaves:
• [Trans. Graph. 2020] One Shot 3D Photography [paper] [code] [project page] :leaves: :gem:
• [Trans. Graph. 2019] 3D Ken Burns Effect from a Single Image [paper] [code] :cloud:
• [ICCV 2019] Monocular Neural Image-based Rendering with Continuous View Control [paper] [code]
• [ECCV 2018] Layer-structured 3D Scene Inference via View Synthesis [paper] [code] [project page] :leaves:
• [SIGGRAPH Posters 2011] Layered Photo Pop-Up [poster] [abstract] [project page]

Binocular-input novel view synthesis. This list is not exhaustive.

• [CVPR 2022] 3D Moments from Near-Duplicate Photos [paper] [code] [project page] :leaves::cloud:
• [CVPR 2022] Stereo Magnification with Multi-Layer Images [paper] [code] [project page] :airplane::gem:
• [ICCV 2019] Extreme View Synthesis [paper] [code]
• [CVPR 2019] Pushing the Boundaries of View Extrapolation with Multiplane Images [paper] :airplane:
• [SIGGRAPH 2018] Stereo Magnification: Learning View Synthesis using Multiplane Images [paper] [code] [project page] :airplane:

3D photography of other kinds, such as relying on multi-view input, using modalities other than natural images, using generative methods without explicit 3D modeling.

• [SIGGRAPH 2023] ViP-NeRF: Visibility Prior for Sparse Input Neural Radiance Fields [paper] [code] [project page] :taxi:
• [CVPR 2023] Painting 3D Nature in 2D: View Synthesis of Natural Scenes from a Single Semantic Mask [paper] [project page] :gem:
• [ICLR 2023] Make-A-Video: Text-to-Video Generation without Text-Video Data [paper] [project page]
• [ECCV 2022] SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Image [paper] [code] [project page] :taxi:
• [CVPR 2022] Look Outside the Room: Synthesizing A Consistent Long-Term 3D Scene Video from A Single Image [paper] [code] [project page]
• [ICCV 2021] Geometry-Free View Synthesis: Transformers and no 3D Priors [paper] [code] [project page]
• [ICCV 2021] Learning to Stylize Novel Views [paper] [code] [project page] :cloud:
• [ICCV 2021] Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis [paper] [code] [project page] :taxi:
• [ICCV 2017] Personalized Cinemagraphs using Semantic Understanding and Collaborative Learning [paper]

3D Photography Videos

3D Photography with dynamics. The works that use a single image as input:

• [arXiv 2024] VividDream: Generating 3D Scene with Ambient Dynamics [paper] [project page] :cloud:☃️
• [ACM MM 2023] Make-It-4D: Synthesizing a Consistent Long-Term Dynamic Scene Video from a Single Image [paper] :leaves::cloud:
• [IJCAI 2023] Learning 3D Photography Videos via Self-supervised Diffusion on Single Images [paper]
• [CVPR 2023] 3D Cinemagraphy from a Single Image [paper] [code] [project page] :leaves::cloud:

The works with multi-view or video input:

• [SIGGRAPH 2024] Modeling Ambient Scene Dynamics for Free-view Synthesis [paper] [project page] ☃️
• [arXiv 2024] LoopGaussian: Creating 3D Cinemagraph with Multi-view Images via Eulerian Motion Field [paper] [code] [project page] :cloud:
• [CVPR 2023] 3D Video Loops from Asynchronous Input [paper] [code] [project page] :airplane:

Cinemagraph (Dynamics or Motion Generation)

Landscape Animation: running water, moving clouds, etc.

• [CVPR 2024] StyleCineGAN: Landscape Cinemagraph Generation using a Pre-trained StyleGAN [paper] [code] [project page]
• [TOG 2023] Text-Guided Synthesis of Eulerian Cinemagraphs [paper] [code] [project page]
• [SA 2022] Water Simulation and Rendering from a Still Photograph [paper] [project page]
• [ICCV 2023] DiffDreamer: Consistent Single-view Perpetual View Generation with Conditional Diffusion Models [paper] [project page]
• [ICLR 2023] Towards Smooth Video Composition [paper] [project page]
• [ICCV 2023] Simulating Fluids in Real-World Still Images [paper] [code] [project page]
• [CVPR 2022] Controllable Animation of Fluid Elements in Still Images [paper] [project page]
• [CVPR 2022] StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2 [paper] [code] [project page]
• [CVPR 2021] Animating Pictures with Eulerian Motion Fields [paper] [project page]
• [MultiMedia 2021] Learning Fine-Grained Motion Embedding for Landscape Animation [paper]
• [ECCV 2020] DeepLandscape: Adversarial Modeling of Landscape Videos [paper] [code] [project page]
• [ECCV 2020] DTVNet: Dynamic Time-lapse Video Generation via Single Still Image [paper] [code]
• [SIGGRAPH Asia 2019] Animating Landscape: Self-Supervised Learning of Decoupled Motion and Appearance for Single-Image Video Synthesis [paper] [code] [project page]
• [CVPR 2018] Learning to Generate Time-lapse Videos Using Multi-stage Dynamic Generative Adversarial Networks [paper] [code] [project page]

Other papers.

• [arXiv 2024] Explorative Inbetweening of Time and Space [paper] [project page]
• [arXiv 2024] Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modeling [paper] [project page]
• [arXiv 2023] AnimateAnything: Fine-Grained Open Domain Image Animation with Motion Guidance [paper] [code] [project page]
• [arXiv 2023] LivePhoto: Real Image Animation with Text-guided Motion Control [paper] [code] [project page]
• [arXiv 2023] DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors [paper] [code]
• [CVPR 2024] Generative Image Dynamics [paper] [project page]
• [arXiv 2023] DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectory [paper] [project page]
• [arXiv 2023] Motion-Conditioned Diffusion Model for Controllable Video Synthesis [paper] [project page]
• [CVPR 2023] Conditional Image-to-Video Generation with Latent Flow Diffusion Models [paper] [code]
• [CVPR 2023] Blowing in the Wind: CycleNet for Human Cinemagraphs from Still Images [paper]
• [ICCV 2021] iPOKE: Poking a Still Image for Controlled Stochastic Video Synthesis [paper] [code] [project page]
• [CVPR 2021] Stochastic Image-to-Video Synthesis Using cINNs [paper] [code] [project page]
• [CVPR 2021] Understanding Object Dynamics for Interactive Image-to-Video Synthesis [paper] [code] [project page]
• [SIGGRAPH 2021] Endless Loops: Detecting and Animating Periodic Patterns in Still Images [paper] [project page]
• [ECCV 2018] Flow-Grounded Spatial-Temporal Video Prediction from Still Images [paper] [code]
• [CVPR 2018] Controllable Video Generation with Sparse Trajectories [paper] [code] [project page]
• [CVPR 2018] MoCoGAN: Decomposing Motion and Content for Video Generation [paper] [code]