Contents

• Foundation Models
• Theory
• Prior Design
• Interpretability
• Fast Inference
• Causal Inference
• Benchmarks and Evaluation
• Software and Libraries
• Educational

Foundation Models

• TabPFN: A transformer that solves small tabular classification problems in a second (2022), N. Hollmann et al. [pdf] - The original TabPFN for small tabular classification via in-context learning.
• TabDPT: Scaling Tabular Foundation Models (2024), J. Ma et al. [pdf] - Combines in-context learning with self-supervised learning and demonstrates scaling laws for tabular foundation models.
• Accurate predictions on small data with a tabular foundation model (TabPFN v2) (2025), N. Hollmann et al. [pdf] - Extends TabPFN to regression, multi-class classification, and larger datasets up to 10,000 samples, published in Nature.
• TabICL: A Tabular Foundation Model for In-Context Learning on Large Data (2025), J. Qu et al. [pdf] - Column-then-row attention architecture for efficient in-context learning, ~10x faster than TabPFN v2 on large datasets.
• TabFlex (2025), Y. Zeng et al. [pdf] - Employs linear attention to scale tabular in-context learning to millions of samples.
• ContextTab (2025), M. Spinaci et al. [pdf] - Incorporates semantic awareness and is trained on real-world data rather than purely synthetic priors.
• Real-TabPFN (2025), A. Garg et al. [pdf] - Improves TabPFN through continued pre-training on curated real-world datasets.
• LimiX (2025), X. Zhang et al. [pdf] - Models the joint distribution over variables and missingness patterns.
• Mitra (2025), X. Zhang et al. [pdf] - Combines curated synthetic priors with fine-tuning strategies.
• TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Models (2025), L. Grinsztajn et al. [pdf] - Scales to 50,000 samples and 2,000 features; state-of-the-art on the TabArena benchmark.
• TabICL v2: A better, faster, scalable, and open tabular foundation model (2026), J. Qu et al. [pdf] - Improves speed and performance over TabICL; adds regression support.

Theory

• Transformers Can Do Bayesian Inference (2022), S. Müller et al. [pdf] - Introduces Prior-Data Fitted Networks (PFNs), showing that transformers can approximate Bayesian posterior predictive distributions via in-context learning.
• Statistical Foundations of Prior-Data Fitted Networks (2023), T. Nagler. [pdf] - Provides a theoretical analysis of PFNs, including a frequentist interpretation and conditions under which bias and variance vanish.

Prior Design

In addition to the foundation model papers:

• Improving TabPFN's Synthetic Data Generation by Integrating Causal Structure (2026), D. Tugnoli et al. [pdf] - Proposes DAG-aware conditioning to align TabPFN's autoregressive generation with causal structure, reducing spurious correlations.
• TabForestPFN (2024), F. den Breejen et al. [pdf] - Uses forest-based synthetic data generation for complex decision boundaries.

Interpretability

• MotherNet: Fast Training and Inference via Hyper-Network Transformers (2023), A. Müller, C. Curino, and R. Ramakrishnan. [pdf] - Generates the weights of a compact child neural network in a single forward pass, enabling fast inference without in-context overhead.
• GAMformer: In-Context Learning for Generalized Additive Models (2024), A. Mueller et al. [pdf] - Estimates interpretable GAM shape functions in a single forward pass, bridging foundation models and interpretable ML.
• Interpretable Machine Learning for TabPFN (2024), D. Rundel et al. [pdf] - Adapts post-hoc IML methods such as feature effects, Shapley values, and data valuation to TabPFN, leveraging in-context learning to avoid costly retraining.
• TabPFN Through The Looking Glass: An Interpretability Study of TabPFN (2026), A. Gupta et al. [pdf] - A mechanistic interpretability study probing TabPFN's internal representations to understand what the model learns across layers.

Causal Inference

• Do-PFN: In-Context Learning for Causal Effect Estimation (2025), J. Robertson et al. [pdf] - Pre-trains PFNs on synthetic data from diverse causal structures including interventions to estimate causal effects without knowing the causal graph.
• CausalPFN: Amortized Causal Effect Estimation via In-Context Learning (2025), V. Balazadeh et al. [pdf] - Amortizes Bayesian causal inference, mapping observational data directly to treatment effect estimates in a single forward pass.
• CausalFM: Foundation Models for Causal Inference via Prior-Data Fitted Networks (2025), Y. Ma et al. [pdf] - A comprehensive framework for PFN-based causal inference covering back-door, front-door, and instrumental variable adjustment.

Fast Inference

Related to interpretability. Anything else?

Benchmarks and Evaluation

• TabArena: A Living Benchmark for Machine Learning on Tabular Data (2025), N. Erickson et al. [pdf] [website] - A collaborative, living benchmark platform for evaluating tabular foundation models.

Software and Libraries

Educational

• nanoTabPFN: A Lightweight and Educational Reimplementation of TabPFN (2025), A. Pfefferle et al. [pdf] - Under 500 lines of code; achieves competitive performance within one minute of pre-training on a single GPU.
• NanoTabICL (2026), Soda team at Inria. [code] - Minimal and educational reimplementation of the TabICLv2 architecture.

Contributing

Contributions of any kind welcome, just follow the guidelines!

Contributors

Thanks goes to these contributors!

Related Lists

• awesome-tabpfn - List of awesome applications of TabPFN, curated by the Prior Labs team.