---

annotations_creators: [] license: [] pretty_name: tabular_benchmark tags: [] task_categories:

• tabular-classification
• tabular-regression configs:
• config_name: clf_cat_albert data_files: clf_cat/albert.csv
• config_name: clf_cat_compas-two-years data_files: clf_cat/compas-two-years.csv
• config_name: clf_cat_covertype data_files: clf_cat/covertype.csv
• config_name: clf_cat_default-of-credit-card-clients data_files: clf_cat/default-of-credit-card-clients.csv
• config_name: clf_cat_electricity data_files: clf_cat/electricity.csv
• config_name: clf_cat_eye_movements data_files: clf_cat/eye_movements.csv
• config_name: clf_cat_road-safety data_files: clf_cat/road-safety.csv
• config_name: clf_num_Bioresponse data_files: clf_num/Bioresponse.csv
• config_name: clf_num_Diabetes130US data_files: clf_num/Diabetes130US.csv
• config_name: clf_num_Higgs data_files: clf_num/Higgs.csv
• config_name: clf_num_MagicTelescope data_files: clf_num/MagicTelescope.csv
• config_name: clf_num_MiniBooNE data_files: clf_num/MiniBooNE.csv
• config_name: clf_num_bank-marketing data_files: clf_num/bank-marketing.csv
• config_name: clf_num_california data_files: clf_num/california.csv
• config_name: clf_num_covertype data_files: clf_num/covertype.csv
• config_name: clf_num_credit data_files: clf_num/credit.csv
• config_name: clf_num_default-of-credit-card-clients data_files: clf_num/default-of-credit-card-clients.csv
• config_name: clf_num_electricity data_files: clf_num/electricity.csv
• config_name: clf_num_eye_movements data_files: clf_num/eye_movements.csv
• config_name: clf_num_heloc data_files: clf_num/heloc.csv
• config_name: clf_num_house_16H data_files: clf_num/house_16H.csv
• config_name: clf_num_jannis data_files: clf_num/jannis.csv
• config_name: clf_num_pol data_files: clf_num/pol.csv
• config_name: reg_cat_Airlines_DepDelay_1M data_files: reg_cat/Airlines_DepDelay_1M.csv
• config_name: reg_cat_Allstate_Claims_Severity data_files: reg_cat/Allstate_Claims_Severity.csv
• config_name: reg_cat_Bike_Sharing_Demand data_files: reg_cat/Bike_Sharing_Demand.csv
• config_name: reg_cat_Brazilian_houses data_files: reg_cat/Brazilian_houses.csv
• config_name: reg_cat_Mercedes_Benz_Greener_Manufacturing data_files: reg_cat/Mercedes_Benz_Greener_Manufacturing.csv
• config_name: reg_cat_SGEMM_GPU_kernel_performance data_files: reg_cat/SGEMM_GPU_kernel_performance.csv
• config_name: reg_cat_abalone data_files: reg_cat/abalone.csv
• config_name: reg_cat_analcatdata_supreme data_files: reg_cat/analcatdata_supreme.csv
• config_name: reg_cat_delays_zurich_transport data_files: reg_cat/delays_zurich_transport.csv
• config_name: reg_cat_diamonds data_files: reg_cat/diamonds.csv
• config_name: reg_cat_house_sales data_files: reg_cat/house_sales.csv
• config_name: reg_cat_medical_charges data_files: reg_cat/medical_charges.csv
• config_name: reg_cat_nyc-taxi-green-dec-2016 data_files: reg_cat/nyc-taxi-green-dec-2016.csv
• config_name: reg_cat_particulate-matter-ukair-2017 data_files: reg_cat/particulate-matter-ukair-2017.csv
• config_name: reg_cat_seattlecrime6 data_files: reg_cat/seattlecrime6.csv
• config_name: reg_cat_topo_2_1 data_files: reg_cat/topo_2_1.csv
• config_name: reg_cat_visualizing_soil data_files: reg_cat/visualizing_soil.csv
• config_name: reg_num_Ailerons data_files: reg_num/Ailerons.csv
• config_name: reg_num_Bike_Sharing_Demand data_files: reg_num/Bike_Sharing_Demand.csv
• config_name: reg_num_Brazilian_houses data_files: reg_num/Brazilian_houses.csv
• config_name: reg_num_MiamiHousing2016 data_files: reg_num/MiamiHousing2016.csv
• config_name: reg_num_abalone data_files: reg_num/abalone.csv
• config_name: reg_num_cpu_act data_files: reg_num/cpu_act.csv
• config_name: reg_num_delays_zurich_transport data_files: reg_num/delays_zurich_transport.csv
• config_name: reg_num_diamonds data_files: reg_num/diamonds.csv
• config_name: reg_num_elevators data_files: reg_num/elevators.csv
• config_name: reg_num_house_16H data_files: reg_num/house_16H.csv
• config_name: reg_num_house_sales data_files: reg_num/house_sales.csv
• config_name: reg_num_houses data_files: reg_num/houses.csv
• config_name: reg_num_medical_charges data_files: reg_num/medical_charges.csv
• config_name: reg_num_nyc-taxi-green-dec-2016 data_files: reg_num/nyc-taxi-green-dec-2016.csv
• config_name: reg_num_pol data_files: reg_num/pol.csv
• config_name: reg_num_sulfur data_files: reg_num/sulfur.csv
• config_name: reg_num_superconduct data_files: reg_num/superconduct.csv
• config_name: reg_num_wine_quality data_files: reg_num/wine_quality.csv
• config_name: reg_num_yprop_4_1 data_files: reg_num/yprop_4_1.csv

---

Tabular Benchmark

Dataset Description

This dataset is a curation of various datasets from openML and is curated to benchmark performance of various machine learning algorithms.

• Repository: https://github.com/LeoGrin/tabular-benchmark/community
• Paper: https://hal.archives-ouvertes.fr/hal-03723551v2/document

Dataset Summary

Benchmark made of curation of various tabular data learning tasks, including:

• Regression from Numerical and Categorical Features
• Regression from Numerical Features
• Classification from Numerical and Categorical Features
• Classification from Numerical Features

Supported Tasks and Leaderboards

• tabular-regression
• tabular-classification

Dataset Structure

Data Splits

This dataset consists of four splits (folders) based on tasks and datasets included in tasks.

• reg_num: Task identifier for regression on numerical features.
• reg_cat: Task identifier for regression on numerical and categorical features.
• clf_num: Task identifier for classification on numerical features.
• clf_cat: Task identifier for classification on categorical features.

Depending on the dataset you want to load, you can load the dataset by passing task_name/dataset_name to data_files argument of load_dataset like below:

from datasets import load_dataset
dataset = load_dataset("inria-soda/tabular-benchmark", data_files="reg_cat/house_sales.csv")

Dataset Creation

Curation Rationale

This dataset is curated to benchmark performance of tree based models against neural networks. The process of picking the datasets for curation is mentioned in the paper as below:

• Heterogeneous columns. Columns should correspond to features of different nature. This excludes images or signal datasets where each column corresponds to the same signal on different sensors.
• Not high dimensional. We only keep datasets with a d/n ratio below 1/10.
• Undocumented datasets We remove datasets where too little information is available. We did keep datasets with hidden column names if it was clear that the features were heterogeneous.
• I.I.D. data. We remove stream-like datasets or time series.
• Real-world data. We remove artificial datasets but keep some simulated datasets. The difference is subtle, but we try to keep simulated datasets if learning these datasets are of practical importance (like the Higgs dataset), and not just a toy example to test specific model capabilities.
• Not too small. We remove datasets with too few features (< 4) and too few samples (< 3 000). For benchmarks on numerical features only, we remove categorical features before checking if enough features and samples are remaining.
• Not too easy. We remove datasets which are too easy. Specifically, we remove a dataset if a simple model (max of a single tree and a regression, logistic or OLS) reaches a score whose relative difference with the score of both a default Resnet (from Gorishniy et al. [2021]) and a default HistGradientBoosting model (from scikit learn) is below 5%. Other benchmarks use different metrics to remove too easy datasets, like removing datasets perfectly separated by a single decision classifier [Bischl et al., 2021], but this ignores varying Bayes rate across datasets. As tree ensembles are superior to simple trees and logistic regresison [Fernández-Delgado et al., 2014], a close score for the simple and powerful models suggests that we are already close to the best achievable score.
• Not deterministic. We remove datasets where the target is a deterministic function of the data. This mostly means removing datasets on games like poker and chess. Indeed, we believe that these datasets are very different from most real-world tabular datasets, and should be studied separately

Source Data

Numerical Classification

Categorical Classification

Numerical Regression

Categorical Regression

Dataset Curators

Léo Grinsztajn, Edouard Oyallon, Gaël Varoquaux.

Licensing Information

[More Information Needed]

Citation Information

Léo Grinsztajn, Edouard Oyallon, Gaël Varoquaux. Why do tree-based models still outperform deep learning on typical tabular data?. NeurIPS 2022 Datasets and Benchmarks Track, Nov 2022, New Orleans, United States. ffhal-03723551v2f