LimiX-2M

LimiX-2M is a 2M-parameter tabular foundation model (TFM) introduced in the paper LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models. It utilizes a unified tokenize-and-route framework to improve conditioning and shallow-layer effective rank, outperforming larger baselines on widely used tabular benchmarks.

• Project Page: https://www.limix.ai/
• GitHub Repository: https://github.com/limix-ldm-ai/LimiX

Sample Usage

The following example demonstrates how to use the LimiXPredictor for a classification task as described in the repository's documentation:

from sklearn.datasets import load_breast_cancer
from sklearn.metrics import accuracy_score, roc_auc_score
from sklearn.model_selection import train_test_split
from huggingface_hub import hf_hub_download
import numpy as np
import torch
import os, sys

# Set environment variables for initialization
os.environ["RANK"] = "0"
os.environ["WORLD_SIZE"] = "1"
os.environ["MASTER_ADDR"] = "127.0.0.1"
os.environ["MASTER_PORT"] = "29500"

# Assuming the LimiX repository is cloned and in the path
from inference.predictor import LimiXPredictor

# Load data
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=42)

# Download model checkpoint
model_file = hf_hub_download(repo_id="stableai-org/LimiX-2M", filename="LimiX-2M.ckpt", local_dir="./cache")

# Initialize predictor and predict
clf = LimiXPredictor(device=torch.device('cuda' if torch.cuda.is_available() else 'cpu'), model_path=model_file, inference_config='config/cls_default_retrieval.json')
prediction = clf.predict(X_train, y_train, X_test)

print("roc_auc_score:", roc_auc_score(y_test, prediction[:, 1]))
print("accuracy_score:", accuracy_score(y_test, np.argmax(prediction, axis=1)))

Citation

@article{zhang2025limix,
  title={Limix: Unleashing structured-data modeling capability for generalist intelligence},
  author={Zhang, Xingxuan and Ren, Gang and Yu, Han and Yuan, Hao and Wang, Hui and Li, Jiansheng and Wu, Jiayun and Mo, Lang and Mao, Li and Hao, Mingchao and others},
  journal={arXiv preprint arXiv:2509.03505},
  year={2025}
}