ViroBench: Viral Understanding & Safety Benchmark for Nucleotide Foundation Models

ViroBench is a curated viral genomics dataset and benchmark suite designed to stress-test nucleotide foundation models (NFMs) under realistic distribution shifts and to support responsible viral sequence modeling.

• Paper: ViroBench: Benchmarking Nucleotide Foundation Models on Viral Genomics Tasks
• Repository: https://github.com/QIANJINYDX/ViroBench
• Project Page / Leaderboard: https://sii-agi4s.github.io/ViroBench/

Dataset Summary

ViroBench is a dual-axis diagnostic benchmark for viral sequence modeling, evaluating both:

• Understanding Axis: Taxonomy classification & host prediction (12 classification tasks).
• Biosecurity Axis: Genome modeling & CDS completion (6 generation tasks).

The full dataset contains 58,314 viral samples curated from NCBI (GenBank + RefSeq), spanning DNA, RNA, and unknown nucleic-acid types.

ViroBench-CLS-Lite

A curated lightweight subset of 8,000 sequences designed for rapid prototyping, hyperparameter screening, and efficient evaluation of models on taxonomy and host classification tasks.

Benchmark Tasks

1. Understanding Axis (Classification)

• Taxonomy Classification: Prediction at Kingdom, Phylum, Class, Order, and Family levels.
• Host Prediction: Predicting the virus host across different nucleic-acid types.
• Split Strategies: Uses Genus-disjoint splits (to reduce phylogenetic leakage) and Temporal splits (to capture distribution drift).

2. Biosecurity Axis (Generation)

• Genome Modeling: Evaluation using bits-per-base (BPB) across length-bucketed tasks (short, medium, long).
• CDS Completion: Evaluating coding sequence (CDS) continuation under functional constraints such as coding validity and frame consistency.

Sample Usage

To run the benchmark using the official scripts, follow these examples from the repository:

Classification Benchmark

# Example: genus-disjoint taxonomy (all NA types)
python script/run_all.py \
  --model_name evo2_1b_base \
  --dataset_name ALL-taxon-genus

Generation Diagnostics (Genome Modeling)

python script/run_all_ppl.py \
  --model_name evo2_1b_base \
  --dataset_name genome-long

CDS Completion

python script/run_all_gen.py \
  --model_name evo2_1b_base \
  --dataset_name cds-medium

Citation

@article{ye2026virobench,
  title={ViroBench: Benchmarking Nucleotide Foundation Models on Viral Genomics Tasks},
  author={Ye, Dongxin and Hu, Fang and Hu, Han and Hu, Shu and Tan, Yang and Ouyang, Wanli and Li, Stan Z and Cui, Jie and Dong, Nanqing},
  journal={arXiv preprint arXiv:2605.25388},
  year={2026}
}