Zygosity-Aware DNA Language Modeling

Zygosity-aware DNA language models leverage both haplotypes to improve ancestry and gene expression prediction. This repo provides HyenaDNA, CNNs, and Nucleotide Transformer-based code, embeddings generation, classification head training scripts, and example data to reproduce the study's key results.

Table of Contents

• Project Structure
• Folder Descriptions
• Getting Started
• Usage
• Contributing

Project Structure

Zygosity_DNALM/
├── src/            # Source code and model implementations
├── results/        # Output results and predictions
└── README.md       # This file

Folder Descriptions

src/

This directory contains all source code for the project:

• Model Implementations: HyenaDNA, CNNs, and Nucleotide Transformer architectures
• Embedding Generation: Scripts to generate DNA sequence embeddings
• Classification Heads: Code for training classification layers on top of embeddings
• Data Processing: Utilities for loading and preprocessing genomic data
• Training Scripts: End-to-end training pipelines
• Evaluation Metrics: Functions to compute model performance metrics
• Figure Generation: Code for generating each figure in the paper

results/

This directory stores output files generated by the models:

• Evaluation Metrics: Accuracy, F1-scores, AUC-ROC curves in .csv and .npy

Getting Started

Follow these steps to set up your environment, download the data, and run the training scripts for our DNA-LM zygosity-aware analysis.

1. Clone the Repository bash

git clone https://github.com/huRashidy/Zygosity_DNALM.git`
cd Zygosity_DNALM

3. Download the Data from Zenodo The full data archive (including all embeddings, .csv/.npy files, etc.) is available via Zenodo, with the link https://zenodo.org/records/17533617 .

a. Manual download: Go to the Zenodo page and download the .rar (or .zip) file to your computer.

b. Command-line download: You can use wget if a direct download link is available, or zenodo-get if preferred.

Example using wget (replace with the actual Zenodo download link and filename):

bash

wget https://zenodo.org/records/17533617/files/chr6.rar`

c. Extract the archive Move the downloaded archive into your project directory and unpack:

bash

mkdir data
# If .rar (install unrar if needed)
unrar x chr6.rar data/
unrar x results.rar data/

All extracted data files must be placed in the data subfolder at the root of the repo. Make sure you replace the './' in all "./data/results" paths is replaced with the root directory.

2. Install Required Python Dependencies Ensure you are using Python 3.8+ and run:

bash

pip install -r requirements.txt

3. If you want to train the classification head

cd train_head
python chr6_5fold_full_embeddings.py --chr 6 --model 'XGB'
##In case you want the results per window
python chr6_5fold_per_window --chr 6 --model 'XGB'

Citation

Here is the citation if you like our work.

@article {El Rashidy2025.11.19.689326,
	author = {El Rashidy, Hussin and Saadat, Ali and Fellay, Jacques},
	title = {Zygosity-Aware DNA Language Modeling Improves Ancestry and Gene Expression Prediction},
	elocation-id = {2025.11.19.689326},
	year = {2025},
	doi = {10.1101/2025.11.19.689326},
	publisher = {Cold Spring Harbor Laboratory},
	abstract = {DNA language models (DNA-LMs) are transforming how genomic sequence information is represented and interpreted. Yet most current approaches treat DNA as a single sequence, overlooking the diploid structure and zygosity information that distinguish the two parental copies of the genome. Here, we systematically evaluate explicit diploid, zygosity-aware representations in DNA-LMs for two downstream tasks: ancestry classification and gene expression prediction. For ancestry, we use HyenaDNA embeddings of the extended MHC region and show that concatenating maternal and paternal haplotype embeddings consistently improves predictive performance across five superpopulations compared to single-haplotype inputs. For gene expression, we compare convolutional neural networks (CNNs) trained from scratch with Nucleotide Transformer models using reference-only, single-copy, and two-copy (zygosity-aware) sequence encodings. CNNs showed increased performance by incorporating genetic variation and zygosity via simple additive genotype encoding, whereas na{\"\i}vely injecting variation into pretrained Nucleotide Transformer models yields mixed effects, highlighting a mismatch between current pretraining objectives and variation-sensitive prediction. Together, our results demonstrate that zygosity-aware representations can capture biologically meaningful information beyond reference-only views and underscore the need for diploid- and population-aware pretraining strategies in future DNA-LMs for variant interpretation and precision medicine.Competing Interest StatementThe authors have declared no competing interest.},
	URL = {https://www.biorxiv.org/content/early/2025/11/20/2025.11.19.689326},
	eprint = {https://www.biorxiv.org/content/early/2025/11/20/2025.11.19.689326.full.pdf},
	journal = {bioRxiv}
}

License

License information to be added.

Contact

For questions or issues, please open an issue on GitHub or contact the repository maintainer.