ToxiMol: A Benchmark for Structure-Level Molecular Detoxification

🔥 News

• 🏆 ToxiMol has been accepted as an Oral presentation at the 32nd SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2026), AI for Sciences Track.

Overview

ToxiMol is the first comprehensive benchmark for molecular toxicity repair tailored to general-purpose Multimodal Large Language Models (MLLMs). This is the dataset repository for the paper "Breaking Bad Molecules: Are MLLMs Ready for Structure-Level Molecular Detoxification?".

Key Features

🧬 Comprehensive Dataset

• 660 representative toxic molecules spanning diverse toxicity mechanisms and varying granularities
• 11 primary toxicity repair tasks based on Therapeutics Data Commons (TDC) platform
• Multi-granular coverage: Tox21 (12 sub-tasks), ToxCast (10 sub-tasks), and 9 additional datasets
• Multimodal inputs: SMILES strings + 2D molecular structure images rendered using RDKit

🎯 Challenging Task Definition

The molecular toxicity repair task requires models to:

1. Identify potential toxicity endpoints from molecular structures
2. Interpret semantic constraints from natural language descriptions
3. Generate structurally similar substitute molecules that eliminate toxic fragments
4. Satisfy drug-likeness and synthetic feasibility requirements

📊 Systematic Evaluation

• ToxiEval framework: Automated evaluation integrating toxicity prediction, synthetic accessibility, drug-likeness, and structural similarity
• Comprehensive analysis: Evaluation of ~30 mainstream MLLMs with ablation studies
• Multi-dimensional metrics: Success rate analysis across different evaluation criteria and failure modes

Dataset Structure

Task Overview

Data Structure

Each entry contains:

{
  "task": "string",      // Toxicity task identifier
  "id": "int",           // Molecule ID
  "smiles": "string",    // SMILES representation
  "image": "binary" // 2D molecular structure image binary
}

Available Subdatasets

subdatasets = [
    "ames", "carcinogens_lagunin", "clintox", "dili", 
    "herg", "herg_central", "herg_karim", "ld50_zhu", 
    "skin_reaction", "tox21", "toxcast"
]

# Load all datasets
datasets = {}
for name in subdatasets:
    datasets[name] = load_dataset("DeepYoke/ToxiMol-benchmark", data_dir=name)

Experimental Results

Our systematic evaluation of ~30 mainstream MLLMs reveals:

• Current limitations: Overall success rates remain relatively low across models
• Emerging capabilities: Models demonstrate initial potential in toxicity understanding, semantic constraint adherence, and structure-aware molecule editing
• Key challenges: Structural validity, multi-dimensional constraint satisfaction, and failure mode attribution

Citation

If you use this dataset in your research, please cite:

@misc{lin2026breakingbadmoleculesmllms,
      title={Breaking Bad Molecules: Are MLLMs Ready for Structure-Level Molecular Detoxification?}, 
      author={Fei Lin and Ziyang Gong and Cong Wang and Tengchao Zhang and Yonglin Tian and Yining Jiang and Ji Dai and Chao Guo and Xiaotong Yu and Xue Yang and Gen Luo and Fei-Yue Wang},
      year={2026},
      eprint={2506.10912},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2506.10912}, 
}

License

This project is licensed under the MIT License - see the LICENSE file for details.