A curated paper list for "Foundation Neural Operators: A Survey on Pretraining Methods, Data Ecosystems, and Efficient Adaptation".
37
11 commits
updated Feb 14, 2026
[Paper]
A curated paper list for "Foundation Neural Operators: A Survey on Pretraining Methods, the Data Ecosystem, and Efficient Adaptation".
This repository organizes recent advances in foundation-style neural operators into three parts(Some articles overlap across domains):
Last updated: 2026-01-21
| Year | Model (Link) | Team | Key idea / paradigm | Pretraining I/O |
|---|---|---|---|---|
| 2025 | GPhyT | UVA / RWTH | In-context learning (ICL) for dynamics: imitate LLM-style prompting, infer future evolution from short trajectory prompts | short-trajectory prompt → future physical evolution |
| 2025 | PDEformer-2 | Peking University | Formula-encoding paradigm: convert PDE into computation graph as input; align symbols with fields | PDE computation graph → solution query at arbitrary coordinates |
| 2025 | PI-MFM | Yale (Lu Lu) | Physics-informed multimodal paradigm: embed symbolic-driven PDE residual loss directly into self-supervised objectives | symbolic formula + IC/BC → operator solution field |
| 2025 | Walrus | Polymathic AI | Autoregressive evolution with harmonic stabilization (Patch Jittering) to suppress long-horizon error accumulation | spatiotemporal field sequence → next-step prediction |
| 2025 | Flow Marching | MIT | Flow Matching for operators: combine operator learning with probability paths to support uncertainty modeling | noisy state → clean physical field |
| 2025 | DISCO | ENS / Flatiron | Hypernetwork discovery: infer evolution-operator parameters dynamically from short trajectories | short state trajectory → operator network parameters |
| 2025 | BCAT | UCLA | Block-causal paradigm: treat fields as spatiotemporal causal “blocks” for sequence prediction | history block sequence → next physical block/frame |
| 2025 | VICON | UCLA | Vision ICL: patch-wise operations for efficient high-dimensional ICL | 2D function-pair sequence → next-step prediction |
| 2025 | MORPH | LANL | Shape-agnostic foundation operator: joint pretraining over 1D/2D/3D in one model | arbitrary mesh/point cloud → field response |
| 2025 | PDE-PFN | - | Prior-data fitting (PFN): use noisy PINN simulations as universal prior; Bayesian ICL | grid/point cloud → posterior distribution |
| 2025 | PDE-FM | IBM | State-space (Mamba) paradigm: linear scaling for 3D complex physical systems | spatial–spectral tokens → operator solution |
| 2025 | LOGLO-FNO | - | Schwartz-operator paradigm: add mode interactions to address FNO pretraining expressivity bottleneck | discrete field → enhanced mapping field |
| 2025 | OmniField | UCLA | Continuous neural field (CNF) paradigm: multimodal pretraining for sparse/noisy experimental data | sparse sensor streams → continuous-field reconstruction |
| 2025 | OmniLearn | - | Injection physics foundation model: general pretraining for high-energy particle trajectory evolution | particle configuration → physical property prediction |
| 2024 | POSEIDON | ETH Zurich | Multiscale operator Transformer: large-scale pretraining using PDE semigroup structure | initial condition + time step → evolution field |
| 2024 | Aurora | Microsoft | Atmospheric foundation model: 3D Swin Transformer trained on massive global weather data | heterogeneous weather variables → high-res forecast |
| 2024 | DPOT | Tsinghua / Alibaba | Autoregressive denoising with Fourier Attention for large heterogeneous PDE corpora | history sequence → next-step prediction |
| 2024 | Unisolver | Tsinghua University | Conditional-guided paradigm: embed PDE symbols, BCs, etc. as deep conditions in Transformer | PDE component symbols → solution field |
| 2024 | PROSE-FD | UCLA | Multimodal fluids: fuse symbolic expressions and numerical fields for zero-shot fluid simulation | symbolic description + numerical → future trajectory |
| 2024 | UPTS | - | Universal Physics Solver: adapt general LLM to time-dependent PDE solver | serialized PDE trajectory → solution prediction |
| 2024 | PITT | Harvard University&Google | Physics-Informed Token Transformer: tokenize symbols to learn analytic correction terms | system state + symbolic encoding → dynamics correction |
| 2023 | ICON | Osher team | Pioneering in-context operator learning: identify new operators via function-pair sequences | function-pair sequence → new operator prediction |
| 2023 | MPP | Polymathic AI | Pioneering multi-physics pretraining: demonstrate transfer gain from joint training across equations | heterogeneous physical fields → spatiotemporal evolution |
| 2023 | FactFormer | - | Axial decomposition paradigm: scalable Transformer for multi-D PDE pretraining (DPOT precursor) | multi-D grid data → field prediction |
| 2025 | OmniArch | Beihang University | Unified multi-physics architecture + cross-dim joint pretraining; PDEAligner for consistency | unified modeling format for multi-physics multi-input |
| 2025 | OmniFluids | Renmin University of China | Physics-first CFD foundation solver: physics pretraining → coarse-grid operator distillation → few-shot fine-tuning | unified modeling format for multi-physics multi-input |
| 2025 | PDE-Transformer | Technical University of Munich | Channel-independent interaction: embed physical variables independently as tokens + channel-level self-attention | unified modeling format for multi-physics multi-input |
| 2024 | Text2PDE | Carnegie Mellon University | Text-to-PDE full-field visualization via diffusion conditioned on PDE text | text → complete PDE field generation |
| 2023 | Scaling & Transfer Behavior | Lawrence Berkeley National Lab | Scaling + transfer behavior of pretraining→downstream | history trajectories → future trajectories |
| 2024 | CoDA-NO | NVIDIA | Codomain attention pretraining for multiphysics neural operators | history trajectories → future trajectories |
| 2024 | Data-Efficient Operator Learning (UP + ICL) | Simon Fraser University | Unsupervised pretraining + downstream tuning | history trajectories → future trajectories |
| 2026 | NCWNO | UIUC | MoE + pretraining + fine-tuning to mitigate forgetting | Parametric PDE inputs (fields + parameters) → solution field prediction |
| 2025 | MoE-POT | USTC | MoE-based pretraining framework (DPOT-like I/O) | same as DPOT |
| 2025 | LLMs as Multi-Modal DE Solvers | University of California | Multi-modal in-context operator learning (ICON + text prompting) | ICON-style + text prompts |
| 2025 | ENMA | NeurIPS 2025 | Token autoregressive diffusion, improving VICON | same as VICON |
| 2025 | Zero-shot PDE | - | Text-trained foundation models extrapolate spatiotemporal dynamics from PDE solutions, without fine-tuning or prompts. | Context → target |
| Year | Technique / Dataset (Link) | Team | Key data innovation |
|---|---|---|---|
| 2025 | The Well (15 TB) | Polymathic AI | Largest physics corpus: standardized dataset spanning 19 heterogeneous scenarios |
| 2025 | PaPQS | TAMU / PNNL | Active query synthesis (AL): use information gain (EIG) to generate most informative PDE samples |
| 2025 | UPTF-7 format | LANL | Unified physics tensor format: standardize metadata & tokenization for 1D–3D heterogeneous data |
| 2025 | Dimension augmentation | Polymathic AI | Cross-dimension alignment: embed 2D trajectories into 3D space for joint pretraining (3D data scarcity) |
| 2025 | Universal-form generation | Peking University | Symbolic auto-synthesis: random coefficient masking/zeroing over 8 PDE templates to create massive corpora |
| 2024 | All2All sampling (POSEIDON) | ETH Zurich | Semigroup augmentation: mine O(T^2) training pairs from one trajectory using time separability |
| 2024 | PDE Control Gym | University of California | PDE control benchmark focusing on boundary-condition control |
| 2022 | PDE Bench | NEC Labs Europe | First large PDE benchmark |
| 2023 | CFDBench | Tsinghua University | Fluid dynamics benchmark |
| 2022 | PDEArena | Microsoft | Large PDE corpus & benchmark suite |
| 2024 | POSEIDON (Downstream suite) | ETH Zurich | 15 downstream tasks across broad PDE types (elliptic/parabolic/hyperbolic/mixed, linear/nonlinear, etc.) |
| 2025 | Realistic Spatiotemporal Multiphysics Flows | - | Realistic multiphysics flow benchmark |
| 2026 | REALPDEBENCH | Westlake University | Benchmark for complex physical systems with real-world data |
| 2025 | Multiphysics Bench | HKU / Shanghai AI Lab | Multiphysics datasets & benchmarking |
| 2023 | BubbleML | UC Irvine | Bubble multiphysics dataset & benchmarks |
| 2025 | Open-CK | USTC | Large multi-physics field-coupling benchmark in combustion kinetics |
| 2024 | DrivAerNet++ | MIT | Largest aero-design corpus: 8,000 diverse car geometries with multimodal high-fidelity CFD (flow fields, meshes, coefficients) |
| 2024 | AhmedML | AWS | High-fidelity bluff body benchmark: 500 geometric variants using hybrid RANS-LES to capture complex flow separation & vortices |
| 2024 | WindsorML | AWS | GPU-native WMLES benchmark: 355 square-back variants with >280M cells for ultra-high-precision transient flow analysis |
| 2024 | DrivAerML | AWS | Industrial high-fidelity benchmark: 500 parametric car variants using scale-resolving hybrid RANS-LES simulations (DES) |
| Year | Technique (Link) | Team | Key optimization idea (efficient finetuning/inference) |
|---|---|---|---|
| 2025 | F-Adapter | - | Frequency-adaptive PEFT: allocate parameters dynamically by spectral complexity; fix high-frequency finetuning collapse |
| 2025 | Transolver++ | Tsinghua | Eidetic state parallelism: first linear scaling on single GPU for 1.2M grid points |
| 2025 | MoE-POT | USTC | Hierarchical routing: activate only 1/4 parameters, reduce inference cost by ~60% |
| 2025 | Mondrian | University of California Irvine | Sliding-window attention: shifted-window mechanism makes 3D operator learning linear-time |
| 2025 | PhysicsCorrect | University of Pennsylvania | Jacobian-cached correction: precompute PDE Jacobians to fix long-horizon drift at inference |
| 2025 | PITA | USTC | Alternating-direction optimization: jointly optimize predictor & equation features, reduce shortcut bias |
| 2024 | MATEY | Oak Ridge National Laboratory | Adaptive tokenization: adjust patch size by local variance (e.g., shocks), save ~8× compute |
| 2024 | SPUS | CAI | Extreme parameter-efficiency: lightweight residual U-Net for strong generalization pretraining |
| 2025 | DeepONet as a Multi-Operator Extrapolation Model | - | Distributed NO training (D2NO/MODNO) + physics-informed losses for zero-shot adaptation |
| 2025 | PI-MFM | Yale | Compute PDE residual losses via AD + finite differences from symbolic PDE inputs |
| 2025 | SC-FNO | Penn State University | Sensitivity-constrained training to improve robustness in few-shot settings |
| 2023 | Scaling & Transfer Behavior | Lawrence Berkeley National Lab | Study pretraining→downstream transfer |
| 2024 | CoDA-NO | NVIDIA | Codomain attention pretraining for multiphysics NOs |
| 2024 | Data-Efficient Operator Learning | - | Unsupervised pretraining + ICL for data efficiency |
| 2026 | NCWNO | UIUC | MoE + pretraining + finetuning to mitigate forgetting |
If you find this work useful, welcome to cite us.
@article{Ren_2026,
title={Foundation Neural Operators: A Survey on Pretraining Methods, the Data Ecosystem, and Efficient Adaptation},
url={http://dx.doi.org/10.36227/techrxiv.177100284.48520964/v1},
DOI={10.36227/techrxiv.177100284.48520964/v1},
publisher={Institute of Electrical and Electronics Engineers (IEEE)},
author={Ren, Xingyu and Wang, Pengkai and Liu, Pengwei and Yue, Xihang and Dong, Huanshuo and Huang, Zhenxin and Hao, Zhongkai and Hu, Ziqian and Huang, Zhen and Wang, Yian and Ni, Dong and Wang, Hong},
year={2026},
month=feb }
A curated paper list for "Foundation Neural Operators: A Survey on Pretraining Methods, Data Ecosystems, and Efficient Adaptation".
37
11 commits
updated Feb 14, 2026
[Paper]
A curated paper list for "Foundation Neural Operators: A Survey on Pretraining Methods, the Data Ecosystem, and Efficient Adaptation".
This repository organizes recent advances in foundation-style neural operators into three parts(Some articles overlap across domains):
Last updated: 2026-01-21
| Year | Model (Link) | Team | Key idea / paradigm | Pretraining I/O |
|---|---|---|---|---|
| 2025 | GPhyT | UVA / RWTH | In-context learning (ICL) for dynamics: imitate LLM-style prompting, infer future evolution from short trajectory prompts | short-trajectory prompt → future physical evolution |
| 2025 | PDEformer-2 | Peking University | Formula-encoding paradigm: convert PDE into computation graph as input; align symbols with fields | PDE computation graph → solution query at arbitrary coordinates |
| 2025 | PI-MFM | Yale (Lu Lu) | Physics-informed multimodal paradigm: embed symbolic-driven PDE residual loss directly into self-supervised objectives | symbolic formula + IC/BC → operator solution field |
| 2025 | Walrus | Polymathic AI | Autoregressive evolution with harmonic stabilization (Patch Jittering) to suppress long-horizon error accumulation | spatiotemporal field sequence → next-step prediction |
| 2025 | Flow Marching | MIT | Flow Matching for operators: combine operator learning with probability paths to support uncertainty modeling | noisy state → clean physical field |
| 2025 | DISCO | ENS / Flatiron | Hypernetwork discovery: infer evolution-operator parameters dynamically from short trajectories | short state trajectory → operator network parameters |
| 2025 | BCAT | UCLA | Block-causal paradigm: treat fields as spatiotemporal causal “blocks” for sequence prediction | history block sequence → next physical block/frame |
| 2025 | VICON | UCLA | Vision ICL: patch-wise operations for efficient high-dimensional ICL | 2D function-pair sequence → next-step prediction |
| 2025 | MORPH | LANL | Shape-agnostic foundation operator: joint pretraining over 1D/2D/3D in one model | arbitrary mesh/point cloud → field response |
| 2025 | PDE-PFN | - | Prior-data fitting (PFN): use noisy PINN simulations as universal prior; Bayesian ICL | grid/point cloud → posterior distribution |
| 2025 | PDE-FM | IBM | State-space (Mamba) paradigm: linear scaling for 3D complex physical systems | spatial–spectral tokens → operator solution |
| 2025 | LOGLO-FNO | - | Schwartz-operator paradigm: add mode interactions to address FNO pretraining expressivity bottleneck | discrete field → enhanced mapping field |
| 2025 | OmniField | UCLA | Continuous neural field (CNF) paradigm: multimodal pretraining for sparse/noisy experimental data | sparse sensor streams → continuous-field reconstruction |
| 2025 | OmniLearn | - | Injection physics foundation model: general pretraining for high-energy particle trajectory evolution | particle configuration → physical property prediction |
| 2024 | POSEIDON | ETH Zurich | Multiscale operator Transformer: large-scale pretraining using PDE semigroup structure | initial condition + time step → evolution field |
| 2024 | Aurora | Microsoft | Atmospheric foundation model: 3D Swin Transformer trained on massive global weather data | heterogeneous weather variables → high-res forecast |
| 2024 | DPOT | Tsinghua / Alibaba | Autoregressive denoising with Fourier Attention for large heterogeneous PDE corpora | history sequence → next-step prediction |
| 2024 | Unisolver | Tsinghua University | Conditional-guided paradigm: embed PDE symbols, BCs, etc. as deep conditions in Transformer | PDE component symbols → solution field |
| 2024 | PROSE-FD | UCLA | Multimodal fluids: fuse symbolic expressions and numerical fields for zero-shot fluid simulation | symbolic description + numerical → future trajectory |
| 2024 | UPTS | - | Universal Physics Solver: adapt general LLM to time-dependent PDE solver | serialized PDE trajectory → solution prediction |
| 2024 | PITT | Harvard University&Google | Physics-Informed Token Transformer: tokenize symbols to learn analytic correction terms | system state + symbolic encoding → dynamics correction |
| 2023 | ICON | Osher team | Pioneering in-context operator learning: identify new operators via function-pair sequences | function-pair sequence → new operator prediction |
| 2023 | MPP | Polymathic AI | Pioneering multi-physics pretraining: demonstrate transfer gain from joint training across equations | heterogeneous physical fields → spatiotemporal evolution |
| 2023 | FactFormer | - | Axial decomposition paradigm: scalable Transformer for multi-D PDE pretraining (DPOT precursor) | multi-D grid data → field prediction |
| 2025 | OmniArch | Beihang University | Unified multi-physics architecture + cross-dim joint pretraining; PDEAligner for consistency | unified modeling format for multi-physics multi-input |
| 2025 | OmniFluids | Renmin University of China | Physics-first CFD foundation solver: physics pretraining → coarse-grid operator distillation → few-shot fine-tuning | unified modeling format for multi-physics multi-input |
| 2025 | PDE-Transformer | Technical University of Munich | Channel-independent interaction: embed physical variables independently as tokens + channel-level self-attention | unified modeling format for multi-physics multi-input |
| 2024 | Text2PDE | Carnegie Mellon University | Text-to-PDE full-field visualization via diffusion conditioned on PDE text | text → complete PDE field generation |
| 2023 | Scaling & Transfer Behavior | Lawrence Berkeley National Lab | Scaling + transfer behavior of pretraining→downstream | history trajectories → future trajectories |
| 2024 | CoDA-NO | NVIDIA | Codomain attention pretraining for multiphysics neural operators | history trajectories → future trajectories |
| 2024 | Data-Efficient Operator Learning (UP + ICL) | Simon Fraser University | Unsupervised pretraining + downstream tuning | history trajectories → future trajectories |
| 2026 | NCWNO | UIUC | MoE + pretraining + fine-tuning to mitigate forgetting | Parametric PDE inputs (fields + parameters) → solution field prediction |
| 2025 | MoE-POT | USTC | MoE-based pretraining framework (DPOT-like I/O) | same as DPOT |
| 2025 | LLMs as Multi-Modal DE Solvers | University of California | Multi-modal in-context operator learning (ICON + text prompting) | ICON-style + text prompts |
| 2025 | ENMA | NeurIPS 2025 | Token autoregressive diffusion, improving VICON | same as VICON |
| 2025 | Zero-shot PDE | - | Text-trained foundation models extrapolate spatiotemporal dynamics from PDE solutions, without fine-tuning or prompts. | Context → target |
| Year | Technique / Dataset (Link) | Team | Key data innovation |
|---|---|---|---|
| 2025 | The Well (15 TB) | Polymathic AI | Largest physics corpus: standardized dataset spanning 19 heterogeneous scenarios |
| 2025 | PaPQS | TAMU / PNNL | Active query synthesis (AL): use information gain (EIG) to generate most informative PDE samples |
| 2025 | UPTF-7 format | LANL | Unified physics tensor format: standardize metadata & tokenization for 1D–3D heterogeneous data |
| 2025 | Dimension augmentation | Polymathic AI | Cross-dimension alignment: embed 2D trajectories into 3D space for joint pretraining (3D data scarcity) |
| 2025 | Universal-form generation | Peking University | Symbolic auto-synthesis: random coefficient masking/zeroing over 8 PDE templates to create massive corpora |
| 2024 | All2All sampling (POSEIDON) | ETH Zurich | Semigroup augmentation: mine O(T^2) training pairs from one trajectory using time separability |
| 2024 | PDE Control Gym | University of California | PDE control benchmark focusing on boundary-condition control |
| 2022 | PDE Bench | NEC Labs Europe | First large PDE benchmark |
| 2023 | CFDBench | Tsinghua University | Fluid dynamics benchmark |
| 2022 | PDEArena | Microsoft | Large PDE corpus & benchmark suite |
| 2024 | POSEIDON (Downstream suite) | ETH Zurich | 15 downstream tasks across broad PDE types (elliptic/parabolic/hyperbolic/mixed, linear/nonlinear, etc.) |
| 2025 | Realistic Spatiotemporal Multiphysics Flows | - | Realistic multiphysics flow benchmark |
| 2026 | REALPDEBENCH | Westlake University | Benchmark for complex physical systems with real-world data |
| 2025 | Multiphysics Bench | HKU / Shanghai AI Lab | Multiphysics datasets & benchmarking |
| 2023 | BubbleML | UC Irvine | Bubble multiphysics dataset & benchmarks |
| 2025 | Open-CK | USTC | Large multi-physics field-coupling benchmark in combustion kinetics |
| 2024 | DrivAerNet++ | MIT | Largest aero-design corpus: 8,000 diverse car geometries with multimodal high-fidelity CFD (flow fields, meshes, coefficients) |
| 2024 | AhmedML | AWS | High-fidelity bluff body benchmark: 500 geometric variants using hybrid RANS-LES to capture complex flow separation & vortices |
| 2024 | WindsorML | AWS | GPU-native WMLES benchmark: 355 square-back variants with >280M cells for ultra-high-precision transient flow analysis |
| 2024 | DrivAerML | AWS | Industrial high-fidelity benchmark: 500 parametric car variants using scale-resolving hybrid RANS-LES simulations (DES) |
| Year | Technique (Link) | Team | Key optimization idea (efficient finetuning/inference) |
|---|---|---|---|
| 2025 | F-Adapter | - | Frequency-adaptive PEFT: allocate parameters dynamically by spectral complexity; fix high-frequency finetuning collapse |
| 2025 | Transolver++ | Tsinghua | Eidetic state parallelism: first linear scaling on single GPU for 1.2M grid points |
| 2025 | MoE-POT | USTC | Hierarchical routing: activate only 1/4 parameters, reduce inference cost by ~60% |
| 2025 | Mondrian | University of California Irvine | Sliding-window attention: shifted-window mechanism makes 3D operator learning linear-time |
| 2025 | PhysicsCorrect | University of Pennsylvania | Jacobian-cached correction: precompute PDE Jacobians to fix long-horizon drift at inference |
| 2025 | PITA | USTC | Alternating-direction optimization: jointly optimize predictor & equation features, reduce shortcut bias |
| 2024 | MATEY | Oak Ridge National Laboratory | Adaptive tokenization: adjust patch size by local variance (e.g., shocks), save ~8× compute |
| 2024 | SPUS | CAI | Extreme parameter-efficiency: lightweight residual U-Net for strong generalization pretraining |
| 2025 | DeepONet as a Multi-Operator Extrapolation Model | - | Distributed NO training (D2NO/MODNO) + physics-informed losses for zero-shot adaptation |
| 2025 | PI-MFM | Yale | Compute PDE residual losses via AD + finite differences from symbolic PDE inputs |
| 2025 | SC-FNO | Penn State University | Sensitivity-constrained training to improve robustness in few-shot settings |
| 2023 | Scaling & Transfer Behavior | Lawrence Berkeley National Lab | Study pretraining→downstream transfer |
| 2024 | CoDA-NO | NVIDIA | Codomain attention pretraining for multiphysics NOs |
| 2024 | Data-Efficient Operator Learning | - | Unsupervised pretraining + ICL for data efficiency |
| 2026 | NCWNO | UIUC | MoE + pretraining + finetuning to mitigate forgetting |
If you find this work useful, welcome to cite us.
@article{Ren_2026,
title={Foundation Neural Operators: A Survey on Pretraining Methods, the Data Ecosystem, and Efficient Adaptation},
url={http://dx.doi.org/10.36227/techrxiv.177100284.48520964/v1},
DOI={10.36227/techrxiv.177100284.48520964/v1},
publisher={Institute of Electrical and Electronics Engineers (IEEE)},
author={Ren, Xingyu and Wang, Pengkai and Liu, Pengwei and Yue, Xihang and Dong, Huanshuo and Huang, Zhenxin and Hao, Zhongkai and Hu, Ziqian and Huang, Zhen and Wang, Yian and Ni, Dong and Wang, Hong},
year={2026},
month=feb }