This repository contains the artifacts for LimiX-2M, a 2M-parameter tabular foundation model designed to mitigate low-rank collapse and attention bottlenecks in structured data.

• Paper: LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models
• GitHub Repository: https://github.com/limix-ldm-ai/LimiX
• Project Page: https://www.limix.ai/

Model Description

LimiX-2M utilizes a unified tokenize-and-route framework. It expands scalar features into compact localized RBF features (RaBEL) and uses a reordered bidirectional block (S$\rightarrow$N$\rightarrow$F) to align computation with the readout. This architecture allows the model to outperform larger baselines while reducing training and inference costs.

Sample Usage

The following example demonstrates how to use the LimiXPredictor for a classification task. Note that using the predictor requires the source code from the GitHub repository.

from sklearn.datasets import load_breast_cancer
from sklearn.metrics import accuracy_score, roc_auc_score
from sklearn.model_selection import train_test_split
from huggingface_hub import hf_hub_download
import torch
import numpy as np
import os
from inference.predictor import LimiXPredictor

# Setup environment
os.environ["RANK"] = "0"
os.environ["WORLD_SIZE"] = "1"
os.environ["MASTER_ADDR"] = "127.0.0.1"
os.environ["MASTER_PORT"] = "29500"

# Load data
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=42)

# Download model
model_file = hf_hub_download(repo_id="stableai-org/LimiX-2M", filename="LimiX-2M.ckpt", local_dir="./cache")

# Initialize and predict
clf = LimiXPredictor(
    device=torch.device('cuda' if torch.cuda.is_available() else 'cpu'), 
    model_path=model_file, 
    inference_config='config/cls_default_retrieval.json'
)
prediction = clf.predict(X_train, y_train, X_test)

print("roc_auc_score:", roc_auc_score(y_test, prediction[:, 1]))
print("accuracy_score:", accuracy_score(y_test, np.argmax(prediction, axis=1)))

Citation

@article{limix2m2026,
  title={LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models},
  author={Zhang, Xingxuan and others},
  journal={arXiv preprint arXiv:2606.04485},
  year={2026}
}