Protein language models have emerged as key paradigms for peptide lead discovery by learning evolutionary information. However, their ability to encode the physicochemical factors that govern peptide function and properties remains largely unexplored. Here, we present a physicochemical perspective benchmark to evaluate protein language models for peptide drug discovery. Specifically, to simulate the key processes of peptide discovery, we selected multiple databases covering peptide-protein affinity, therapeutic function, and toxicity prediction. Subsequently, eight protein language models, whose parameter sizes span from the million-level to billion-level, were evaluated under full fine-tuning, low-rank adaptation (LoRA), and unsupervised paradigms, respectively. Furthermore, based on the physicochemical space, we analyzed and compared the performance of full and LoRA fine-tuning strategies, revealing how these mechanisms guide models toward convergent sample spaces with coincident physicochemical distributions.
We used the following public datasets for benchmarking:
| Dataset | Tasks |
|---|---|
| 🔗 PPIKB | Peptide–protein affinity |
| ☠️ ToxTeller | Peptide toxicity prediction |
| ☠️ ToxinPred 3.0 | Peptide toxicity prediction |
| 💊 PrMFTP | Multi-functional therapeutic peptide prediction |
ESM2: Lin Z, Akin H, Rao R, et al. Evolutionary-scale prediction of atomic-level protein structure with a language model[J]. Science, 2023, 379(6637): 1123-1130. paper, code.
ESM3: Hayes T, Rao R, Akin H, et al. Simulating 500 million years of evolution with a language model[J]. Science, 2025, 387(6736): 850-858. paper, code.
xTrimoPGLM: Chen B, Cheng X, Li P, et al. Xtrimopglm: unified 100-billion-parameter pretrained transformer for deciphering the language of proteins[J]. Nature Methods, 2025: 1-12. paper, code.
ProtT5: Elnaggar A, Heinzinger M, Dallago C, et al. Prottrans: Toward understanding the language of life through self-supervised learning[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 44(10): 7112-7127. paper, code.
AminoBert: Chowdhury R, Bouatta N, Biswas S, et al. Single-sequence protein structure prediction using a language model and deep learning[J]. Nature Biotechnology, 2022, 40(11): 1617-1623. paper , code.
ProtBert: Elnaggar A, Heinzinger M, Dallago C, et al. Prottrans: Toward understanding the language of life through self-supervised learning[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 44(10): 7112-7127. paper, code.
OmegaPLM: Wu R, Ding F, Wang R, et al. High-resolution de novo structure prediction from primary sequence[J]. BioRxiv, 2022: 2022.07. 21.500999. paper, code.
CARP: Yang K K, Fusi N, Lu A X. Convolutions are competitive with transformers for protein sequence pretraining[J]. Cell Systems, 2024, 15(3): 286-294. e2. paper, code.
Make sure that your dataset is stored in the following structure so that scripts can read it correctly.
BenchPLM/
├── dataset/ # This catalog is designed to contain the datasets needed for model. Put it in the same directory where the script you want to run is located.
│ ├── PPIKB/
│ ├── PrMFTP/
│ ├── Toxinpred3.0/
│ └── ToxTeller/
│
├── model/ # This directory contains all the methods' full fine-tuning and LoRA code files.
│ ├── AminoBERT/
│ ├── CARP/
│ ├── ESM2/
│ │ ├── README.md # Specific operation instructions for this model
│ │ ├── full_fine-tuning # Full fine-tuning script
│ │ └── lora # Full fine-tuning script
│ ├── ... # Other models folder
│ └── xTrimoPGLM/
│
├── data_utils.py # For peptide–protein affinity data preprocessing, put it in the same directory where the script you want to run is located.
├── evaluation.py # For multi-tag evaluation, put it in the same directory where the script you want to run is located.
└── README.md
For specific details, you can refer to ./model, where each method includes a file that includes a README file and its implementation on the dataset. For example, the operation step of ESM3 can find in ./model/ESM3/README.md.
Protein language models have emerged as key paradigms for peptide lead discovery by learning evolutionary information. However, their ability to encode the physicochemical factors that govern peptide function and properties remains largely unexplored. Here, we present a physicochemical perspective benchmark to evaluate protein language models for peptide drug discovery. Specifically, to simulate the key processes of peptide discovery, we selected multiple databases covering peptide-protein affinity, therapeutic function, and toxicity prediction. Subsequently, eight protein language models, whose parameter sizes span from the million-level to billion-level, were evaluated under full fine-tuning, low-rank adaptation (LoRA), and unsupervised paradigms, respectively. Furthermore, based on the physicochemical space, we analyzed and compared the performance of full and LoRA fine-tuning strategies, revealing how these mechanisms guide models toward convergent sample spaces with coincident physicochemical distributions.
We used the following public datasets for benchmarking:
| Dataset | Tasks |
|---|---|
| 🔗 PPIKB | Peptide–protein affinity |
| ☠️ ToxTeller | Peptide toxicity prediction |
| ☠️ ToxinPred 3.0 | Peptide toxicity prediction |
| 💊 PrMFTP | Multi-functional therapeutic peptide prediction |
ESM2: Lin Z, Akin H, Rao R, et al. Evolutionary-scale prediction of atomic-level protein structure with a language model[J]. Science, 2023, 379(6637): 1123-1130. paper, code.
ESM3: Hayes T, Rao R, Akin H, et al. Simulating 500 million years of evolution with a language model[J]. Science, 2025, 387(6736): 850-858. paper, code.
xTrimoPGLM: Chen B, Cheng X, Li P, et al. Xtrimopglm: unified 100-billion-parameter pretrained transformer for deciphering the language of proteins[J]. Nature Methods, 2025: 1-12. paper, code.
ProtT5: Elnaggar A, Heinzinger M, Dallago C, et al. Prottrans: Toward understanding the language of life through self-supervised learning[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 44(10): 7112-7127. paper, code.
AminoBert: Chowdhury R, Bouatta N, Biswas S, et al. Single-sequence protein structure prediction using a language model and deep learning[J]. Nature Biotechnology, 2022, 40(11): 1617-1623. paper , code.
ProtBert: Elnaggar A, Heinzinger M, Dallago C, et al. Prottrans: Toward understanding the language of life through self-supervised learning[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 44(10): 7112-7127. paper, code.
OmegaPLM: Wu R, Ding F, Wang R, et al. High-resolution de novo structure prediction from primary sequence[J]. BioRxiv, 2022: 2022.07. 21.500999. paper, code.
CARP: Yang K K, Fusi N, Lu A X. Convolutions are competitive with transformers for protein sequence pretraining[J]. Cell Systems, 2024, 15(3): 286-294. e2. paper, code.
Make sure that your dataset is stored in the following structure so that scripts can read it correctly.
BenchPLM/
├── dataset/ # This catalog is designed to contain the datasets needed for model. Put it in the same directory where the script you want to run is located.
│ ├── PPIKB/
│ ├── PrMFTP/
│ ├── Toxinpred3.0/
│ └── ToxTeller/
│
├── model/ # This directory contains all the methods' full fine-tuning and LoRA code files.
│ ├── AminoBERT/
│ ├── CARP/
│ ├── ESM2/
│ │ ├── README.md # Specific operation instructions for this model
│ │ ├── full_fine-tuning # Full fine-tuning script
│ │ └── lora # Full fine-tuning script
│ ├── ... # Other models folder
│ └── xTrimoPGLM/
│
├── data_utils.py # For peptide–protein affinity data preprocessing, put it in the same directory where the script you want to run is located.
├── evaluation.py # For multi-tag evaluation, put it in the same directory where the script you want to run is located.
└── README.md
For specific details, you can refer to ./model, where each method includes a file that includes a README file and its implementation on the dataset. For example, the operation step of ESM3 can find in ./model/ESM3/README.md.