Awesome Optimal Transport in Deep Learning

:wave: Hi! This repo is a collection of AWESOME things about :star2:Optimal Transport in Deep Learning:star2:, including useful materials, papers, code. Feel free to star and fork.

TODO:

• Update recent papers within the last 3 years
• Add pdf&code links

Contents

• Awesome Optimal Transport in Deep Learning
• Contents
• Useful Materials
  • Tutorials
  • Courses
  • Books
  • Libraries
  • AWESOME Repos
• Papers
  • Survey
  • Theory
  • Learning Problems
    • Domain Adaptation
    • Clustering/Self-Supervised Learning
    • Semi-Supervised Learning
    • Label Refinery
    • Class-Imbalanced/Long-Tailed Learning
    • Positive-Unlabeled Learning
    • Out-of-Distribution Detection/Open Set Recognition
    • Federated Learning
    • Others
  • Graph Learning
  • Cross-Modal Learning
  • Generative Modeling
    • GAN
    • Diffusion Model
    • Schrödinger Bridge
    • Normalizing Flow Model
    • Neural Optimal Transport
  • Reinforcement Learning
  • Imitation Learning
  • Computer Vision Tasks
    • 2D CV Tasks
    • 3D CV Tasks
  • NLP Tasks

Useful Materials

Tutorials

• [Marco Cuturi] A Primer on Optimal Transport [video-p1] [video-p2] [video-p3] [slides]
• [Justin Solomon] Shape Analysis (Lecture 19): Optimal transport [video] [slides]
• [Gabriel Peyré] Optimal transport for machine learning [video]
• [Nicolas Courty] Optimal transport for graphs [video]
• [Gabriel Peyré & Marco Cuturi] Optimal Transportation and Application - 25 october 2022 [video]
• [Rémi Flamary] Optimal Transport for Machine Learning tutorial [page]

Courses

• [Université Paris-Saclay] Lénaïc Chizat: Introduction to Optimal Transport Theory [page]
• [ENSAE] Marco Cuturi: Optimal Transport (Spring 2023) [page]
• [Data Science Summer School] Rémi Flamary: Optimal Transport and Machine Learning DS3 2018 [page]

Books

• [Gabriel Peyré & Marco Cuturi] Computational Optimal Transport [pdf] [page] [course notes]

Libraries

• POT: Python Optimal Transport (Python Optimal Transport library) [GitHub] [page]
• OTT-JAX: Optimal Transport Tools implemented with the JAX framework [GitHub] [page]
• GeomLoss: efficient GPU implementations for Geometric loss functions between point clouds, images and volumes [GitHub] [page]

AWESOME Repos

• kilianFatras/awesome-optimal-transport [GitHub]
• changwxx/Awesome-Optimal-Transport-in-Deep-Learning [GitHub]
• abdelwahed/OT_for_big_data [GitHub]

Papers

Survey

• [arXiv 2305] Earth Movers in The Big Data Era: A Review of Optimal Transport in Machine Learning [pdf] [page]
• [arXiv 2306] Recent Advances in Optimal Transport for Machine Learning [pdf]

Theory

• [NIPS 2013] Sinkhorn Distances: Lightspeed Computation of Optimal Transport [pdf]
• [SIAM Journal on Scientific Computing 2015] Iterative Bregman Projections for Regularized Transportation Problems [pdf]
• [Mathematics of Computation 2018] Scaling algorithms for unbalanced optimal transport problems [pdf]
• [ICML 2016] Gromov-wasserstein averaging of kernel and distance matrices [pdf]

Learning Problems

Domain Adaptation

• [ECML PKDD 2014] Domain adaptation with regularized optimal transport [pdf]
• [TPAMI 2016] Optimal Transport for Domain Adaptation [pdf]
• [NIPS 2017] Joint distribution optimal transportation for domain adaptation [pdf] [code]
• [ECCV 2018] DeepJDOT: Deep Joint Distribution Optimal Transport for Unsupervised Domain Adaptation [pdf]
• [AISTATS 2019] Optimal transport for multi-source domain adaptation under target shift [pdf]
• [CVPR 2020] Reliable weighted optimal transport for unsupervised domain adaptation [pdf]
• [ICML 2021] Unbalanced minibatch Optimal Transport; applications to Domain Adaptation [pdf] [code]
• [ICML 2022] Improving minibatch optimal transport via partial transportation [pdf]
• [CVPR 2023] COT: Unsupervised Domain Adaptation With Clustering and Optimal Transport [pdf]

Open-Set/Partial/Universal DA

• [IJCAI 2020] Joint Partial Optimal Transport for Open Set Domain Adaptation [pdf]
• [KDD 2021] Open Set Domain Adaptation using Optimal Transport [pdf]
• [NeurIPS 2022] Unified optimal transport framework for universal domain adaptation [pdf] [code] [page]
• [AAAI 2023] Prototypical Partial Optimal Transport for Universal Domain Adaptation [pdf] [code]
• [CVPR 2023] MOT: Masked Optimal Transport for Partial Domain Adaptation [pdf]

Clustering/Self-Supervised Learning

• [arXiv 2019] Differentiable Deep Clustering with Cluster Size Constraints[pdf]
• [ICLR 2020] Self-labelling via simultaneous clustering and representation learning[pdf]
• [NIPS 2020] Unsupervised Learning of Visual Features by Contrasting Cluster Assignments[pdf]
• [NeurIPS 2022] Wasserstein K-means for clustering probability distributions[pdf]
• [AAAI 2024] Unsupervised Cross-Domain Image Retrieval via Prototypical Optimal Transport[pdf] [code]
• [ICLR 2024] P^2OT: Progressive Partial Optimal Transport for Deep Imbalanced Clustering [pdf] [code]

Novel Class Discovery

• [ICCV 2021] A unified objective for novel class discovery[pdf] [code]
• [TMLR 2023] Novel class discovery for long-tailed recognition[pdf] [code]

Semi-Supervised Learning

• [IJCAI 2018] Semi-Supervised Optimal Transport for Heterogeneous Domain Adaptation[pdf]
• [ECCV 2020] Transporting labels via hierarchical optimal transport for semi-supervised learning[pdf]
• [ICML 2021] Sinkhorn label allocation: Semi-supervised classification via annealed self-training[pdf]
• [CVPR 2021] Self-Supervised Wasserstein Pseudo-Labeling for Semi-Supervised Image Classification[pdf]
• [arXiv 2310] OTMatch: Improving Semi-Supervised Learning with Optimal Transport [pdf]
• [ICCV 2023] Dual Pseudo-Labels Interactive Self-Training for Semi-Supervised Visible-Infrared Person Re-Identification [pdf] [code]

Label Refinery

• [CVPR 2021] Group-aware Label Transfer for Domain Adaptive Person Re-identification[pdf] [code]
• [ICASSP 2022] OT cleaner: Label correction as optimal transport[pdf]
• [CVPR 2023] OT-Filter: An Optimal Transport Filter for Learning With Noisy Labels[pdf]
• [NeurIPS 2023] CSOT: Curriculum and Structure-Aware Optimal Transport for Learning with Noisy Labels[pdf] [code]

Class-Imbalanced/Long-Tailed Learning

• [AAAI 2018] Label Distribution Learning by Optimal Transport[pdf]
• [CVPR workshop 2022] SAR: Self-adaptive refinement on pseudo labels for multiclass-imbalanced semi-supervised learning[pdf]
• [ICLR 2022] Optimal Transport for Long-Tailed Recognition with Learnable Cost Matrix[pdf]
• [NeurIPS 2022] SoLar: Sinkhorn Label Refinery for Imbalanced Partial-Label Learning[pdf] [code]
• [NeurIPS 2022] Learning to re-weight examples with optimal transport for imbalanced classification[pdf] [code]
• [TMLR 2023] Novel class discovery for long-tailed recognition[pdf] [code]
• [NeurIPS 2023] Relative Entropic Optimal Transport: a (Prior-aware) Matching Perspective to (Unbalanced) Classification[pdf]
• [NeurIPS 2023] Enhancing Minority Classes by Mixing: An Adaptative Optimal Transport Approach for Long-tailed Classification[pdf] [code]
• [IJCV 2023] An Optimal Transport View of Class-Imbalanced Visual Recognition[pdf]
• [ICLR 2024] P^2OT: Progressive Partial Optimal Transport for Deep Imbalanced Clustering [pdf] [code]
• [arXiv 2404] SP^2OT: Semantic-Regularized Progressive Partial Optimal Transport for Imbalanced Clustering [pdf] [code]

Positive-Unlabeled Learning

• [NeurIPS 2020] Partial optimal tranport with applications on positive-unlabeled learning[pdf]
• [NeurIPS 2020] Partial Gromov-Wasserstein with Applications on Positive-Unlabeled Learning[pdf] [code]
• [IJCAI 2022] Posistive-Unlabeled Learning via Optimal Transport and Margin Distribution[pdf]
• [ICLR 2023] Computing all Optimal Partial Transports[pdf]

Out-of-Distribution Detection/Open Set Recognition

• [ICCV 2023] FedPD: Federated Open Set Recognition with Parameter Disentanglement [pdf]
• [CVPR 2023] Uncertainty-Aware Optimal Transport for Semantically Coherent Out-of-Distribution Detection [pdf] [code]
• [NeurIPS 2023] Characterizing Out-of-Distribution Error via Optimal Transport [pdf] [code]

Federated Learning

• [NeurIPS 2020] Robust Federated Learning: The Case of Affine Distribution Shifts [pdf]
• [ICML 2022] Orchestra: Unsupervised Federated Learning via Globally Consistent Clustering [pdf]
• [ICCV 2023] FedPD: Federated Open Set Recognition with Parameter Disentanglement [pdf]
• [arXiv 2301] Decentralized Entropic Optimal Transport for Privacy-preserving Distributed Distribution Comparison [pdf]
• [ICLR 2024] Federated Wasserstein Distance[pdf]
• [CVPR 2024] Global and Local Prompts Cooperation via Optimal Transport for Federated Learning [pdf] [code]

Model Fusion

• [NeurIPS 2020] Model Fusion via Optimal Transport [pdf] [code]
• [ICLR 2024] Transformer Fusion with Optimal Transport[pdf] [code]

Others

• [WACV 2021] Zero-Shot Recognition via Optimal Transport[pdf]
• [NeurIPS 2023] Performance Scaling via Optimal Transport: Enabling Data Selection from Partially Revealed Sources[pdf][code]
• [NeurIPS 2022] Adaptive Distribution Calibration for Few-Shot Learning with Hierarchical Optimal Transport[pdf][code]
• [TMLR 2023] Transport with Support: Data-Conditional Diffusion Bridges[pdf]

Graph Learning

• [NeurIPS 2022] Template based Graph Neural Network with Optimal Transport Distances[pdf]
• [NeurIPS 2022] OTKGE: Multi-modal Knowledge Graph Embeddings via Optimal Transport [pdf]
• [NeurIPS 2022] Certifying Robust Graph Classification under Orthogonal Gromov-Wasserstein Threats[pdf]
• [NeurIPS 2023] Fused Gromov-Wasserstein Graph Mixup for Graph-level Classifications[pdf]

Cross-Modal Learning

• [IJCAI 2018] Semi-Supervised Optimal Transport for Heterogeneous Domain Adaptation[pdf]
• [ICML 2020] Graph optimal transport for cross-domain alignment[pdf] [code]
• [ECCV 2020] UNITER: UNiversal Image-Text Representation Learning[pdf] [code]
• [NeurIPS 2022] Keypoint-Guided Optimal Transport with Applications in Heterogeneous Domain Adaptation[pdf] [code]
• [NeurIPS 2022] OTKGE: Multi-modal Knowledge Graph Embeddings via Optimal Transport [pdf]
• [ICLR 2023 Spotlight] PLOT: Prompt Learning with Optimal Transport for Vision-Language Models[pdf] [code]
• [ICCV 2023] Multimodal Optimal Transport-based Co-Attention Transformer with Global Structure Consistency for Survival Prediction[pdf] [code]
• [NeurIPS 2023] Extremal Domain Translation with Neural Optimal Transport[pdf] [code]
• [ICLR 2024] Bridging Vision and Language Spaces with Assignment Prediction[pdf]
• [ICLR 2024] Revisiting Deep Audio-Text Retrieval Through the Lens of Transportation[pdf] [code]

Generative Modeling

GAN

• [Arxiv 2017] Wasserstein GAN. [Code] [pdf]
• [NeurIPS 2017] Improved Training of Wasserstein GANs. [Code] [pdf]
• [ICML 2017] A Two-Step Computation of the Exact GAN Wasserstein Distance. [Code] [pdf]
• [ECCV 2018] Wasserstein Divergence for GANs. [Code] [pdf]
• [CVPR 2018] Generative Modeling Using the Sliced Wasserstein Distance. [Code] [pdf]
• [CVPR 2018] HashGAN: Deep Learning to Hash With Pair Conditional Wasserstein GAN. [Code] [pdf]
• [NeurIPS 2018] Banach wasserstein gan. [Code] [pdf]
• [ICML 2019] Wasserstein of Wasserstein Loss for Learning Generative Models. [Code] [pdf]
• [CVPR 2019] Sliced Wasserstein Generative Models. [Code] [pdf]
• [CVPR 2019] Max-Sliced Wasserstein Distance and Its Use for GANs. [Code] [pdf]
• [ICCV 2019] Wasserstein GAN With Quadratic Transport Cost. [Code] [pdf]
• [NeurIPS 2019] Multi-marginal Wasserstein GAN. [Code] [pdf]
• [ICLR 2024] Analyzing and Improving OT-based Adversarial Networks[pdf]
• [ICLR 2024] SAN: Inducing Metrizability of GAN with Discriminative Normalized Linear Layer[pdf] [code]

Diffusion Model

• [NeurIPS 2021] Score-based generative neural networks for large-scale optimal transport. [Code] [pdf]
• [NeurIPS 2022] Score-based Generative Modeling Secretly Minimizes the Wasserstein Distance. [Code] [pdf]
• [ICLR 2022] Generative Modeling with Optimal Transport Maps. [Code] [pdf]
• [NeurIPS 2022] Amortized Projection Optimization for Sliced Wasserstein Generative Models. [Code] [pdf]
• [ICCV 2023] DPM-OT: A New Diffusion Probabilistic Model Based on Optimal Transport [Code] [pdf]
• [NeurIPS 2023] Optimal Transport-Guided Conditional Score-Based Diffusion Model [Code] [pdf]
• [NeurIPS 2023] Formulating Discrete Probability Flow Through Optimal Transport. [Code] [pdf]
• [ICLR 2023] Dual Diffusion Implicit Bridges for Image-to-Image Translation. [Code] [pdf]
• [ICLR 2023] Understanding DDPM Latent Codes Through Optimal Transport. [Code] [pdf]
• [ICLR 2024] Transport meets Variational Inference: Controlled Monte Carlo Diffusions[pdf]

Schrödinger Bridge

• [NeurIPS 2021] Diffusion Schrödinger Bridge with Applications to Score-Based Generative Modeling. [Code] [pdf]
• [ICLR 2023] Neural Lagrangian Schrödinger Bridge: Diffusion Modeling for Population Dynamics. [Code] [pdf]
• [Arxiv 2024] Reflected Schrödinger Bridge for Constrained Generative Modeling. [Code] [pdf]
• [Arxiv 2023] Generalized Schrödinger Bridge Matching. [Code] [pdf]
• [Arxiv 2023] Building the Bridge of Schrödinger: A Continuous Entropic Optimal Transport Benchmark. [Code] [pdf]
• [NeurIPS 2023] Diffusion Schrödinger Bridge Matching. [Code] [pdf]
• [TMLR 2023] Transport with Support: Data-Conditional Diffusion Bridges. [Code] [pdf]
• [AISTATS 2023] The Schrödinger Bridge between Gaussian Measures has a Closed Form. [Code] [pdf]

Normalizing Flow Model

• [NeurIPS 2018] Large Scale Optimal Transport and Mapping Estimation. [Code] [pdf]
• [AAAI 2021] Ot-flow: Fast and accurate continuous normalizing flows via optimal transport. [Code] [pdf]
• [Arxiv 2023] Training-free Linear Image Inversion via Flows. [Code] [pdf]
• [ICML 2023] On Kinetic Optimal Probability Paths for Generative Models. [Code] [pdf]
• [ICML'W 2023] Improving and generalizing flow-based generative models with minibatch optimal transport. [Code] [pdf]
• [ICLR 2023] Flow Matching for Generative Modeling. [Code] [pdf]
• [ICLR 2023] Building Normalizing Flows with Stochastic Interpolants. [Code] [pdf]
• [ICML 2023] Multisample Flow Matching: Straightening Flows with Minibatch Couplings. [Code] [pdf]
• [ICLR 2023] Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow. [Code] [pdf]
• [JMLR 2023] Diffusion Bridge Mixture Transports, Schrödinger Bridge Problems and Generative Modeling. [Code] [pdf]

Neural Optimal Transport

• [NeurIPS 2021] Do Neural Optimal Transport Solvers Work? A Continuous Wasserstein-2 Benchmark [Code] [pdf]
• [NeurIPS 2022] Kantorovich Strikes Back! Wasserstein GANs are not Optimal Transport? [Code] [pdf]
• [ICLR 2023] Neural Optimal Transport. [Code] [pdf]
• [ICLR 2023] Kernel Neural Optimal Transport. [Code] [pdf]
• [Arxiv 2023] Building the Bridge of Schrödinger: A Continuous Entropic Optimal Transport Benchmark. [Code] [pdf]
• [NeurIPS 2023] Generative Modeling through the Semi-dual Formulation of Unbalanced Optimal Transport. [Code] [pdf]
• [NeurIPS 2023] Extremal Domain Translation with Neural Optimal Transport. [Code] [pdf]
• [Arxiv 2023] Neural Gromov-Wasserstein Optimal Transport. [Code] [pdf]
• [Arxiv 2023] Neural Gromov-Wasserstein Optimal Transport. [Code] [pdf]
• [Arxiv 2023] ANALYZING AND IMPROVING OT-BASED ADVERSARIAL NETWORKS. [Code] [pdf]
• [NeurIPS 2023 Oral] Entropic Neural Optimal Transport via Diffusion Processes. [Code] [pdf]
• [ICLR 2024] Neural Optimal Transport with General Cost Functionals [pdf]
• [ICLR 2024] Energy-guided Entropic Neural Optimal Transport [pdf]
• [ICLR 2024] Light Schrödinger Bridge. [Code] [pdf]
• [ICLR 2024] Unbalancedness in Neural Monge Maps Improves Unpaired Domain Translation[pdf]

Reinforcement Learning

• [ICML 2022] Curriculum reinforcement learning via constrained optimal transport [Code] [pdf]
• [NeurIPS 2022] Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptation[pdf]
• [NeurIPS 2022] Trust Region Policy Optimization with Optimal Transport Discrepancies: Duality and Algorithm for Continuous Actions[pdf]
• [NeurIPS 2023] Accelerating Motion Planning via Optimal Transport[Code] [pdf]
• [NeurIPS 2023] Wasserstein Gradient Flows for Optimizing Gaussian Mixture Policies[pdf]

Imitation Learning

• [ICLR 2021] Primal Wasserstein Imitation Learning[pdf]
• [ICLR 2022] Cross-Domain Imitation Learning via Optimal Transport[pdf] [video]
• [ICLR 2023] Optimal Transport for Offline Imitation Learning [pdf] [code]
• [NeurIPS Workshop 2023] Understanding Reward Ambiguity Through Optimal Transport Theory in Inverse Reinforcement Learning[pdf]
• [arXiv 2402] Align Your Intents: Offline Imitation Learning via Optimal Transport[pdf]

Computer Vision Tasks

2D CV Tasks

Object Detection

• [CVPR 2021] OTA: Optimal Transport Assignment for Object Detection[pdf] [code]
• [CVPR 2023] Unbalanced Optimal Transport: A Unified Framework for Object Detection[pdf] [code]

Semantic Segmentation

• [ICCV 2021] Deep transport network for unsupervised video object segmentation[pdf]
• [ICLR 2023] Modeling Multimodal Aleatoric Uncertainty in Segmentation with Mixture of Stochastic Experts[pdf] [code]
• [ICCV 2023] Point2Mask: Point-supervised Panoptic Segmentation via Optimal Transport[pdf] [code]

Crowd Counting

• [NeruIPS 2020] Distribution Matching for Crowd Counting [pdf] [code]
• [IJCAI 2021] Bipartite Matching for Crowd Counting with Point Supervision [pdf]
• [AAAI 2021] Learning to Count via Unbalanced Optimal Transport [pdf]
• [CVPR 2021] A Generalized Loss Function for Crowd Counting and Localization [pdf]
• [CVPR 2023] Optimal Transport Minimization: Crowd Localization on Density Maps for Semi-Supervised Counting [pdf] [code]
• [ACM MM 2023] DAOT: Domain-Agnostically Aligned Optimal Transport for Domain-Adaptive Crowd Counting [pdf] [code]

Person ReID

• [CVPR 2021] Group-aware Label Transfer for Domain Adaptive Person Re-identification[pdf] [code]
• [ECCV 2022] Optimal Transport for Label-Efficient Visible-Infrared Person Re-Identification[pdf] [code]

Others

• [ACM MM 2022] Weakly-Supervised Temporal Action Alignment Driven by Unbalanced Spectral Fused Gromov-Wasserstein Distance[pdf]
• [NeurIPS 2022] Optimal Transport-based Identity Matching for Identity-invariant Facial Expression Recognition[pdf] [code]
• [NeurIPS 2022] Aligning individual brains with fused unbalanced Gromov Wasserstein[pdf]
• [CVPR 2023] Knowledge Distillation for 6D Pose Estimation by Aligning Distributions of Local Predictions[pdf] [code]
• [NeurIPS 2023] Deep Optimal Transport: A Practical Algorithm for Photo-realistic Image Restoration[pdf] [code]

3D CV Tasks

• [CVPR 2019] Learning with batch-wise optimal transport loss for 3d shape recognition[pdf]
• [CVPR 2020] Synchronizing Probability Measures on Rotations via Optimal Transport[pdf]
• [CVPR 2021] Self-point-flow: Self-supervised scene flow estimation from point clouds with optimal transport and random walk[pdf]
• [NeurIPS 2021] Accurate Point Cloud Registration with Robust Optimal Transport[pdf] [code]
• [ICCV workshop 2023] DELO: Deep Evidential LiDAR Odometry using Partial Optimal Transport[pdf]

NLP Tasks

• [ICLR 2019] Improving Sequence-to-Sequence Learning via Optimal Transport[pdf] [code]
• [IJCAI 2020] Evaluating Natural Language Generation via Unbalanced Optimal Transport[pdf]
• [AAAI 2020] Sequence Generation with Optimal-Transport-Enhanced Reinforcement Learning[pdf]
• [CVPR 2020] Hierarchical Optimal Transport for Document Representation[pdf] [code]
• [ACL 2021 best] Vocabulary learning via optimal transport for neural machine translation[pdf] [code]
• [ACL 2022] Transferring Knowledge via Neighborhood-Aware Optimal Transport for Low-Resource Hate Speech Detection[pdf]