TabDPT Classifier Pipeline

DIMER-ready inference pipeline for TabDPT v1.2 (TabDPT-Turbo), an open-weight tabular foundation model from Layer 6 AI for in-context supervised learning.

What this repository provides

• supervised tabular classification with TabDPTClassifier;
• train-fitted preprocessing for mixed numeric/categorical CSV tables;
• immutable upstream weight provenance and SHA-256 verification;
• a DIMER manifest plus executable local/on-prem runtime adapter;
• deterministic validation splitting and support-row capping from DIMER runtime controls;
• a DIMER Notebook Specification 2.2 standalone tutorial pair (E2E + ARTIFACT-INFERENCE, generated by tools/build_notebook.py; see tutorials/README.md) with a verified no-refit artifact reload, plus CI tests that do not download the 254 MB model in routine CI.

Pinned model identity

The weight bytes are not committed to this repository. Runtime acquisition is pinned to the immutable Hugging Face revision and verified before model construction. The local snapshot layout is weights/tabdpt-1.2/ (the MODEL_KEY): the committed dimer-base-manifest.json there lists the checkpoint's path, byte size and SHA-256, stage_missing_files fetches only what is absent at the pinned revision, verify_snapshot re-hashes every entry, and TabDPTClassificationPipeline.from_pretrained(weights_dir=...) pins the verified file as the base checkpoint (see weights/README.md).

DIMER runtime contract

DIMER's current tabular transport is reused rather than inventing a new channel:

• datasetPreprocessing → DIMER_PREPROCESSING_ARGS_JSON
• modelFinetuning → DIMER_HYPERPARAMETERS_JSON

For TabDPT, modelFinetuning is a compatibility transport namespace only. fine_tune must remain false; the other fields control ICL context/inference (n_ensembles, context_size, batch_size, temperature, seed). Every manifest key is consumed by tabdpt_classifier_pipeline.dimer_runtime, and CI checks the manifest/runtime key sets for exact equality.

dimer_entrypoint.py reads DIMER_DATASET_DIR, accepts train.csv plus optional val.csv directly or inside one ZIP, applies the requested validation split/support cap, executes TabDPT, and writes result.json, training_context.parquet, and artifact.json under the configured output paths.

Quick start

python -m venv .venv
. .venv/bin/activate
pip install -e '.[model]'

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from tabdpt_classifier_pipeline import TabDPTClassificationPipeline

frame = load_breast_cancer(as_frame=True).frame
train, test = train_test_split(frame, test_size=0.2, random_state=42, stratify=frame['target'])
pipe = TabDPTClassificationPipeline()
pipe.fit(train, target_column='target')
print(pipe.evaluate(test))

For DIMER, mount/cache the verified base weight through DIMER_BASE_MODEL_PATH so execution does not depend on live internet access.

Tutorials

tutorials/tabdpt_classifier_colab.ipynb (E2E) and its companion tutorials/tabdpt_classifier_artifact_inference_colab.ipynb (ARTIFACT-INFERENCE) are declared under DIMER Notebook Specification 2.2 and are standalone (§4) and GUIDED: generated by tools/build_notebook.py, each carries the package's three modules verbatim, a stage runner, the model identity, the snapshot manifest digests and a hash-locked requirements file (tutorials/requirements-colab.lock.txt) that a pinned uv installs into an isolated environment (nothing is installed into the kernel, so Run all needs no restart), so the exported notebooks run without this repository (parity enforced by tests/test_notebook_parity.py and tools/validate_release_assets.py). The E2E default path loads scikit-learn's bundled breast-cancer table (no download), validates it with validate_inputs into an input manifest, splits it with class coverage preserved, compares majority_class_baseline, conditions the model, writes an evaluation_report (sample-sanity: accuracy / log loss / ROC-AUC on one seeded holdout), scores new rows with class probabilities under an explicit argmax rule, exports JSON/CSV plus the DIMER serving artifact, and reloads it across a fresh boundary without refitting preprocessing. The companion's default path uses the pinned sample artifact and rows carried from examples/sample-artifact/ (trusted SHA-256 checked; support rows of the public breast-cancer table and fitted preprocessing statistics, no weights); your own artifact comes by upload or explicit path. It validates the artifact before reconstruction, validates new unlabelled rows into an input manifest, predicts, and writes a not-measurable evaluation report. BYOD is optional and gated off by default. See tutorials/README.md for the registry and docs/release-verification.md for the release gate.

Release status

Candidate. Static/unit checks do not constitute clean-runtime notebook evidence. No clean-runtime run of the current notebook blobs is recorded: the previous repository-installing pair (spec 1.0) has a local clean-kernel run (2026-09-11) and an earlier standalone blob has a Kaggle T4 run (2026-09-14, outcome only), and neither transfers to the current blobs. A hosted one-pass Colab T4 run of each current blob is pending. Complete docs/release-verification.md against the exact release revision before calling either notebook release-grade.

Model card

MODEL_CARD.md documents this pipeline against the DIMER Model Card Specification, at the version its front matter declares: intended uses and users, out-of-scope uses, factors, metrics and decision thresholds, and ethical considerations.

Evaluation caveat

TabDPT was pretrained on real-world tabular datasets. Public tutorial/benchmark datasets may overlap directly or indirectly with upstream pretraining. Tutorial metrics are plumbing/sanity checks, not clean evidence of out-of-distribution quality.

Upstream references

• Hosseinzadeh et al., TabDPT-Turbo: Efficient In-Context Learning for Tabular Prediction, arXiv:2608.01400 (2026). This is the paper for the v1.2 / TabDPT-Turbo release packaged here.
• Ma et al., TabDPT: Scaling Tabular Foundation Models, arXiv:2410.18164 / NeurIPS 2025. This is the base TabDPT work.

This repository is an integration project and is not affiliated with or endorsed by Layer 6 AI or The Toronto-Dominion Bank.

AI Assistance Disclosure

This repository’s code and accompanying documentation were developed with generative AI assistance for code development and technical writing under maintainer direction. The maintainer remains responsible for reviewing the implementation, validating results, and making release decisions. AI assistance does not constitute independent verification, provider endorsement, or release approval.