TadA-Bench

A Million-Variant Benchmark for Future-Round Discovery Toward Agentic Protein Engineering

Jin Gao¹, Juntu Zhao¹, Zirui Zeng¹, Jiaqi Shen¹, Junhao Shi¹, Dukun Zhao¹, Yuming Lu^1,†, Dequan Wang^1,2,†

¹Shanghai Jiao Tong University   ·   ²Shanghai Innovation Institute
^†Corresponding authors

Dataset Summary

TadA-Bench is a fixed-data wet-lab replay benchmark built from 31 rounds of TadA directed evolution. The benchmark asks whether biological language models can use earlier experimental evidence to rank variants that appear only in later rounds, matching the candidate prioritization problem faced by iterative protein-engineering workflows.

The release provides aligned protein, DNA, and RNA views for the same TadA engineering campaign. Each example is sequence-defined and includes a Seq2Graph-derived activity label, enabling reproducible future-round evaluation without rerunning wet-lab experiments.

Splits

Official comparisons should use the fixed chronological splits without reshuffling. In the paper protocol, models train on rounds 1-27, validate on round 28, and test on rounds 29-31.

Fields

Loading

Use the fixed data revision for reproducible ICML 2026 comparisons. This SHA identifies the released benchmark data files used for the official experiments.

from datasets import load_dataset

REVISION = "07168448caaafab4efb26eca04ec3e503edf1c04"

ds = load_dataset(
    "JinGao/TadA-Bench",
    split="all.AA.test",
    revision=REVISION,
)
print(ds[0])

To inspect every split:

from datasets import load_dataset

REVISION = "07168448caaafab4efb26eca04ec3e503edf1c04"

ds = load_dataset(
    "JinGao/TadA-Bench",
    revision=REVISION,
)
print(ds)

For lightweight inspection, use streaming:

from datasets import load_dataset
from itertools import islice

REVISION = "07168448caaafab4efb26eca04ec3e503edf1c04"

ds = load_dataset(
    "JinGao/TadA-Bench",
    split="all.AA.test",
    revision=REVISION,
    streaming=True,
)
print(list(islice(ds, 3)))

Evaluation Code

The official code repository provides fixed-split loaders, baseline configs, metric utilities, and leaderboard validation:

https://github.com/shiyegao/TadA-Bench

Leaderboard submissions should report Spearman, Recall@10%, and nDCG@10% on the selected fixed split. See the GitHub repository for submission metadata schema, format-only validation, and full-split validation commands.

Citation

If you use TadA-Bench, please cite the accompanying ICML 2026 paper: arXiv:2606.02624.

@inproceedings{gao2026tadabench,
  title = {TadA-Bench: A Million-Variant Benchmark for Future-Round Discovery Toward Agentic Protein Engineering},
  author = {Gao, Jin and Zhao, Juntu and Zeng, Zirui and Shen, Jiaqi and Shi, Junhao and Zhao, Dukun and Lu, Yuming and Wang, Dequan},
  booktitle = {Proceedings of the 43rd International Conference on Machine Learning},
  year = {2026}
}

License

This dataset is released under the Apache License 2.0.

Contact

Jin Gao: Homepage | gaojin@sjtu.edu.cn