GANs

Courses

• Deeplearning.ai Generative Adversarial Networks Specialization
  • Build Basic Generative Adversarial Networks (GANs)
  • Build Better Generative Adversarial Networks (GANs)
  • Apply Generative Adversarial Networks (GANs)
• Deep Unsupervised Learning - Spring 2020 (UC Berkeley)
  • YouTube Lectures
  • Assignments
• Deep Generative Models - Fall 2019 (Stanford)
  • Course Notes/Blog
  • Probability Review

Conference Workshops

• NeurIPS 2020: Machine Learning for Creativity and Design
• ECCV 2020: Perception Through Structured Generative Models
• CVPR 2020: AI for Content Creation Workshop

Tutorials and Blogs

• Discovering Interpretable GAN controls 00:35:02
• StyleGAN v2: notes on training and latent space exploration (Fashion Dataset)
  • Improving Initialization for GAN inversion
  • Making Anime Faces With StyleGAN[1/2] - Tips and tricks for effective StyleGAN use
  • From GAN basic to StyleGAN2 [Explains - Perceptual Path Length]
• CVPR'20 iMLCV tutorial: Exploring and Exploiting Interpretable Semantics in GANs by Bolei Zhou 0:46:46
• Using Artificial Intelligence to Augment Human Intelligence
• Apple Machine Learning Blog

Training GANs with Limited Data

• NeurIPS 2020: StyleGAN2 Adaptive Discriminator Augmentation - Training Generative Adversarial Networks with Limited Data [Tensorflow Code] [PDF] [Unofficial Video]
  • Blog
• NeurIPS 2020: Differentiable Augmentation for Data-Efficient GAN Training

GAN Architectures (StyleGAN, ProgressiveGAN types)

• NeurIPS 2020: SariGAN - Semantic-aware-relative-importance GAN - Learning Semantic-aware Normalization for Generative Adversarial Networks
• CVPR 2020: A U-Net Based Discriminator for Generative Adversarial Networks [PyTorch Code] [Video]
• CVPR 2020: StyleGAN2 - Analyzing and Improving the Image Quality of StyleGAN
  • From GAN basic to StyleGAN2 [Explains - Perceptual Path Length]
  • Pytorch Code Repositories - rosinality/stylegan2-pytorch, lucidrains/stylegan2-pytorch
• CVPR 2019: StyleGAN - A Style-Based Generator Architecture for Generative Adversarial Networks
• ICLR 2018: Progressive GAN - Progressive Growing of GANs for Improved Quality, Stability, and Variation

GAN Editing

• CVPR 2020: Editing in Style: Uncovering the Local Semantics of GANs [Video]
• CVPR 2020: Disentangled Image Generation Through Structured Noise Injection [Video] [Tensorflow Code]
• CVPR 2020: The GAN that Warped: Semantic Attribute Editing with Unpaired Data

GAN Inversion: Inverting Real Faces to Latent Code (Image2StyleGAN types)

• ACCV 2020: MagGAN: High-Resolution Face Attribute Editing with Mask-Guided Generative Adversarial Network
• ECCV 2020: Pix2latent - Transforming and Projecting Images into Class-conditional Generative Networks
• arxiv 2020: Generative Hierarchical Features from Synthesizing Images
• SIGGRAPH 2020: MichiGAN: Multi-Input-Conditioned Hair Image Generation for Portrait Editing
• ECCV 2020: StyleGAN2 Distillation for Feed-forward Image Manipulation
  • Comments: Create synthetic datasets using style mixing and linear interpolation. Train paired pix2pixhd on these synthetic datasets for tasks such as male->female, female->male, etc.
• ECCV 2020: Rewriting a Deep Generative Model
• ECCV 2020: Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation
• ECCV 2020: DeepLandscape: Adversarial Modeling of Landscape Videos
• CVPR 2020: SEAN: Image Synthesis with Semantic Region-Adaptive Normalization
• CVPR 2020: Image Processing Using Multi-Code GAN Prior
• CVPR 2020: Image2StyleGAN++: How to Edit the Embedded Images?
• SIGGRAPH 2019: Semantic Photo Manipulation with a Generative Image Prior
• ICCV 2019: Image2StyleGAN - How to Embed Images Into the StyleGAN Latent Space?
  • Comments: Introduce extended latent codes to embed real images that are different from the dataset on which GAN has been trained. Each layer of Generator receives different latent codes from this extend latent space. Problem: Overfitting to given image; doesn't very well support the manipulation. Resulting code might be outside the original latent domain (due to unconstrained optimization)

Flow based networks (Invertible by design)

• SIGGRAPH ASIA 2020: StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows
  • What are Normalizing Flows? [Video]
• NeurIPS 2018: Glow: Generative Flow with Invertible 1x1 Convolutions

Adding encoder to GAN generator (Reconstructions are not good)

• ECCV 2020: In-Domain GAN Inversion for Real Image Editing
  • Comments Novel Encoder training for GAN Inversion - Introduce adversarial loss for training encoder; Encoder trained using real images. Resolves out-of-domain image inversion (from Image2StyleGAN) by doing an Encoder-constrained Optimization (i.e. minimize distance between encoder predicted latent code and optimized latent code). They compute loss on both reconstructed image and the predicted latent code. Hence, we can use these latent codes for image editing. Check application: Semantic Diffusion
• CVPR 2020: Adversarial Latent Autoencoders
• NeurIPS 2019: BigBiGAN - Large Scale Adversarial Representation Learning

Interpretability

Require supervision in form of off-the-shelf supervised classifiers

• ICLR 2020: On the "steerability" of generative adversarial networks
  • Comments: Explore correspondence of latent space trajectories in GANs to simple image transformations. Dataset biases limit the extent of transformations (e.g: can't convert red firetruck to blue firetruck by moving in the blueness direction in the latent space). Data augmentation and jointly training the walk trajectory and the generator weights imroves steerability, resulting in larger transformation effects.
• CVPR 2020: InterFaceGAN - Interpreting the Latent Space of GANs for Semantic Face Editing
  • Comments: Similar to HiGAN below, they use off-the-shelf image classifiers (like male/female, old/young, smile/no-smile, artifacts/no-artifacts) to find semantic boundaries in the latent space. Check for metrics to measure the disentanglement of faces.
• CVPRW 2020: HiGAN - Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis
  • Comments: Investigates the causality between latent space vectors and generated image attributes/semantics. For normal GANs, they use off-the-shelf image classifiers (like cloud/no-cloud, lighting/no-lighting) to find semantic boundaries in the latent space. For StyleGAN-like architectures, where stochasticity/randomness is introduced at multiple layers, they find that by perturbing input latent vectors at different layer depths, different semantics are controlled: Layout -> Objects -> Attributes -> Color Schemes
• ICCV 2019: GANalyze: Toward Visual Definitions of Cognitive Image Properties
  • Comments: Learn transformation in latent space (via a Transformer network) to improve memorability of generated images. Also check MemNet
• ICLR 2019: GAN Dissection: Visualizing and Understanding Generative Adversarial Networks, Video
  • Comments: It's a framework to interpret and label the internal units inside the Generator. Labels are associated by checking correlation of feature activations of individual units with the segmentation mask of the generated image

Unsupervised Attribute Discovery in GANs

• NeurIPS 2020: GANSpace: Discovering Interpretable GAN Controls [Video]
  • Comments Discover Interpretable latent space directions by performing PCA analysis on the latent activations
• ICML 2020: Unsupervised Discovery of Interpretable Directions in the GAN Latent Space
  • Comments: They learn a set of directions in the latet space that induce "orthogonal" image transformations that are easy to distinguish from each other. E.g. -background blur+, -background removal+, -hair+, etc.

Disentanglement of Variation Factors in Generative Models

• NeurIPS 2020: Self-learning Transformations for Improving Gaze and Head Redirection
• arxiv 2020: Closed-Form Factorization of Latent Semantics in GANs
• ECCV 2020: The Hessian Penalty - A Weak Prior for Unsupervised Disentanglement
• CVPR 2020: DiscoFaceGAN - Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning
• arxiv 2020: Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation
• arxiv 2020: Face Identity Disentanglement via Latent Space Mapping

Image to Image (Pix2Pix and CycleGAN types)

• NeurIPS 2020: Swapping Autoencoder for Deep Image Manipulation
• ECCV 2020: Contrastive Learning for Unpaired Image-to-Image Translation
• ECCV 2020: Learning to Factorize and Relight a City
• CVPR 2020: StarGAN v2 - Diverse Image Synthesis for Multiple Domains
• ICLR 2020: Mask Based Unsupervised Content Transfer
  • Comments: They tackle the problem of translating domain specific content (such as glasses/no-glasses, moustache/no-moustache) between two domains. Their proposed method disentangles the common and separate parts of these domains, and, through the generation of a mask, focuses the attention of the underlying network to the desired augmentation alone, without wastefully reconstructing the entire target.
• CVPR 2019: GauGAN/SPADE - Semantic Image Synthesis with Spatially-Adaptive Normalization
  • Comments: Parallel work to pix2pixhd (superior results) - CHECK
• ICCV 2019: Everybody dance now [Code]
• ICCV 2019: SinGAN - Learning a Generative Model from a Single Natural Image
• CVPR 2018: StarGAN - Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation
• CVPR 2018: Pix2PixHD - High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
• ICCV 2017: CycleGAN - Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• CVPR 2017: Pix2Pix - Image-to-Image Translation with Conditional Adversarial Networks
• CVPR 2017: SimGAN - Learning from Simulated and Unsupervised Images through Adversarial Training
  • Apple Blog
• SURVEY, Aug'20: Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications

Improving GANs

• NeurIPS 2020: Top-k Training of GANs: Improving GAN Performance by Throwing Away Bad Samples

Other Applications

• CVPR 2020: VOGUE: Try-On by StyleGAN Interpolation Optimization [Video]
• arxiv 2020: House-GAN: Relational Generative Adversarial Networks for Graph-constrained House Layout Generation
• IJCV 2020: Compositional GAN: Learning Image-Conditional Binary Composition
• CVPR 2020: PSGAN: Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfer
• CVPR 2020: Learning to Simulate Dynamic Environments with GameGAN
• CVPR 2020: Controllable Person Image Synthesis with Attribute-Decomposed GAN
• WACV 2020: TailorGAN: Making User-Defined Fashion Designs
• SIGGRAPH 2020: Unpaired Motion Style Transfer from Video to Animation
• ICCV 2019: InGAN: Capturing and Remapping the "DNA" of a Natural Image
• SIGGRAPH 2018: Non-Stationary Texture Synthesis by Adversarial Expansion

Quantitative Analysis

• CVPR 2019: Perceptual Path Metric: StyleGAN
• NeurIPS 2019: Improved Precision and Recall Metric for Assessing Generative Models
• Build Better Generative Adversarial Networks (GANs), Week 1
  • NeurIPS 2018: Assessing Generative Models via Precision and Recall
  • NeurIPS 2017: Frechet Inception Distance

====

• https://genforce.github.io/
• https://github.com/zhoubolei/awesome-generative-modeling/blob/master/README.md