ychuest/Awesome-LLMs-meet-genomes

Explore a comprehensive collection of basic theories, applications, papers, and best practices about Large Language Models (LLMs) in genomes.

157

207 commits

updated Oct 2, 2026

See the code

README

Awesome-LLMs-meet-genomes

Awesome-LLMs-meet-genomes is a collection of state-of-the-art, novel, exciting LLMs methods on genomes. It contains papers, codes, datasets, evaluations, and analyses. Any additional information about LLMs for bioinformatics is welcome, and we are glad to add you to the contributor list here. Any problems, please contact yangchengyjs@163.com. If you find this repository useful to your research or work, it is really appreciated to star this repository. ✨

This project is sponsored by E-API.

E-API provides unified multi-model API access, supporting integrations and collaborations for open-source projects, developer tools, and AI applications.

New users who register through the referral link below may receive RMB 5 in product credits: https://eapi.site/register?aff=3M9ZLY

This sponsorship and referral relationship is not affiliated with, endorsed by, or officially connected to GitHub.

Learn more about E-API / Collaboration


Made with Python GitHub stars GitHub forks visitors

Table of Content


🔔 News

  • 🧬✔️ [2024/09] Benchmarks for classification of genomic sequences link.
  • 💥 [2024/08] Some real-world experience in training LLMs link.
  • 💥 [2024/08] Three ways of Fine-tuning link.
  • 💥 [2024/08] Visualisation of the Transformer Principle link.
  • 🌟 [2024/08] The Cultivation Method of Large Language Models: A Path to Success link.
  • 📖 [2024/08] Large Language Models: From Theory to Practice link.

Important Survey Papers

YearTitleVenuePaperCode
2026.02Generative Artificial Intelligence for Biology: Toward Unifying Models, Algorithms, and ModalitiesarXivLink-
2025.05Back to recurrent processing at the crossroad of transformers and state-space modelsNature Machine IntelligenceLink-
2025.03Transformers and genome language modelsNature Machine IntelligenceLink-
2025.03Language modelling techniques for analysing the impact of human genetic variationarXivLink-
2025.03Biological Sequence with Language Model Prompting: A SurveyarXivLink-
2025.01Large Language Models for BioinformaticsarXivLink-
2024.09Genomic Language Models: Opportunities and ChallengesarXivLink-
2024.07Scientific Large Language Models: A Survey on Biological & Chemical DomainsarXivLinklink
2024.01Large language models in bioinformatics: applications and perspectivesarXivLink-
2023.11To Transformers and Beyond: Large Language Models for the GenomearXivLink-
2023.01Applications of transformer-based language models in bioinformatics: a surveyBioinformatics AdvancesLink-

Important Benchmarks

YearTitleVenuePaperCode
2026.06GENEB: Why Genomic Models Are Hard to CompareICML'26LinkHuggingface
2025.11DNALONGBENCH: a benchmark suite for long-range DNA prediction tasksnature communicationsLinkLink

Genomic Large Language Models (Gene-LLMs)

Generic Base Models

YearTitleVenuePaperCode
2026.05PlasmidLM: A Promptable DNA Language Model via Verifiable-Reward Post-TrainingbioXivlink-
2026.04From nucleotides to semantics: genomic representation learning via joint-embedding predictive architecturebioXivlink-
2026.02JEPA-DNA: Grounding Genomic Foundation Models through Joint-Embedding Predictive ArchitecturesarXivlinklink
2026.02Extending Sequence Length is Not All You Need: Effective Integration of Multimodal Signals for Gene Expression PredictionICLR'26 Orallinklink
2026.01DNACHUNKER: Learnable Tokenization for DNA Language ModelsICML'26linklink
2026.01GENERator: A Long-Context Generative Genomic Foundation ModelarXivlinklink
2026.01Functional In-Context Learning in Genomic Language Models with Nucleotide-Level Supervision and Genome Compression (GENERator-v2 Technical Report)BioRxivlinklink
2025.11Context-Aware Regularization with Markovian Integration for Attention-Based Nucleotide AnalysisNeurIPS'25linklink
2025.11MergeDNA: Context-aware Genome Modeling with Dynamic Tokenization through Token MergingAAAI'26 Orallink-
2025.10Annotating the genome at single-nucleotide resolution with DNA foundation modelsNature Methodslinklink
2025.07JanusDNA: A Powerful Bi-directional Hybrid DNA Foundation ModelNeurIPS'25linklink
2025.07BMFM-DNA: A SNP-aware DNA foundation model to capture variant effectsarXivlinklink
2025.07TrinityDNA: A Bio-Inspired Foundational Model for Efficient Long-Sequence DNA ModelingAAAI'26 Submissionlink-
2025.06Generalized biological foundation model with unified nucleic acid and protein languageNature Machine Intelligencelinklink
2025.06eccDNAMamba: A Pre-Trained Model for Ultra-Long eccDNA Sequence AnalysisICML'25linklink
2025.06SPACE: Your Genomic Profile Predictor is a Powerful DNA Foundation ModelICLR'25linklink
2025.04Bridging Sequence-Structure Alignment in RNA Foundation ModelsAAAI'25linklink
2025.03HydraRNA: a hybrid architecture based full-length RNA language modelbioXivlinklink
2025.03Pre-training Genomic Language Model with Variants for Better Modeling Functional GenomicsbioXivlinklink
2025.03Enhancing DNA Foundation Models to Address Masking InefficienciesarXivlinklink
2025.02HybriDNA: A Hybrid Transformer-Mamba2 Long-Range DNA Language ModelarXivlink-
2025.02Omni-DNA: A Unified Genomic Foundation Model for Cross-Modal and Multi-Task LearningarXivlinklink
2025.01MutBERT: Probabilistic Genome Representation Improves Genomics Foundation ModelsBioXivlinklink
2025.01GENA-LM: a family of open-source foundational DNA language models for long sequencesNucleic Acids Researchlinklink
2024.12EpiGePT: a pretrained transformer-based language model for context-specific human epigenomicsGenome Biologylinklink
2024.11VIRALpre: Genomic Foundation Model Embedding Fused with K-mer Feature for Virus IdentificationbioRxivlink-
2024.11BEACON: Benchmark for Comprehensive RNA Tasks and Language ModelsNeurIPS'24linklink
2024.11DNA Language Models for RNA AnalysesICLR'25 Conference Submissionlink-
2024.10The OMG dataset: An Open MetaGenomic corpus for mixed-modality genomic language modelingBioRxivlinklink
2024.10Character-level Tokenizations as Powerful Inductive Biases for RNA Foundational ModelsNeurIPS'24linklink
2024.10Revisiting K-mer Profile for Effective and Scalable Genome Representation LearningNeurIPS'24linklink
2024.10A long-context language model for deciphering and generating bacteriophage genomesNature Communicationslinklink
2024.10Revisiting Convolution Architecture in the Realm of DNA Foundation ModelsICLR'25 Conference Submissionlink-
2024.10Hyperbolic Genome EmbeddingsICLR'25 Conference Submissionlink-
2024.10dnaGrinder: a lightweight and high-capacity genomic foundation modelICLR'25 Conference Submissionlink-
2024.10DNABERT-S: Pioneering Species Differentiation with Species-Aware DNA EmbeddingsICLR'25 Conference Submissionlink-
2024.10Long-range gene expression prediction with token alignment of large language modelarXivlink-
2024.09A Comparison of Tokenization Impact in Attention Based and State Space Genomic Language ModelsbioRxivlink-
2024.09Designing realistic regulatory DNA with autoregressive language modelsGenome Researchlink-
2024.08Understanding the Natural Language of DNA using Encoder-Decoder Foundation Models with Byte-level PrecisionBioinformatics Advanceslinklink
2024.08Unlocking Efficiency: Adaptive Masking for Gene Transformer ModelsECAI'24linklink
2024.07Genomics-FM: Universal Foundation Model for Versatile and Data-Efficient Functional Genomic AnalysisbioRxivlinklink
2024.07 ✨✨✨VQDNA: Unleashing the Power of Vector Quantization for Multi-Species Genomic Sequence ModelingICML'24linklink
2024.07OmniGenome: Aligning RNA Sequences with Secondary Structures in Genomic Foundation ModelsarXivlinklink
2024.07Scorpio : Enhancing Embeddings to Improve Downstream Analysis of DNA sequencesbioRxivlinklink
2024.07DNA language model GROVER learns sequence context in the human genome (可用于蛋白质-DNA结合预测任务)Nature Machine Intelligencelinklink tutorials
2024.05Are Genomic Language Models All You Need? Exploring Genomic Language Models on Protein Downstream TasksbioRxivlinklink
2024.05GeneAgent: Self-verification Language Agent for Gene Set Knowledge Discovery using Domain DatabasesarXivlink-
2024.05DeepGene: An Efficient Foundation Model for Genomics based on Pan-genome Graph TransformerbioRxivlinklink
2024.05Self-Distillation Improves DNA Sequence Inference DatabasesarXivlinklink
2024.04Effect of tokenization on transformers for biological sequencesBioinformaticslinklink
2024.04DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species GenomeICLR'24linklink
2024.02Exploring Genomic Large Language Models: Bridging the Gap between Natural Language and Gene SequencesbioRxivlinklink data
2024.02Sequence modeling and design from molecular to genome scale with EvobioRxivlinklink
2024.01ProkBERT family: genomic language models for microbiome applicationsFrontiers in MicrobiologyLinklink
2023.09The Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human GenomicsbioRxivlinklink
2023.08DNAGPT: A Generalized Pre-trained Tool for Versatile DNA Sequence Analysis TasksbioRxivlinklink
2023.07EpiGePT: a Pretrained Transformer model for epigenomicsbioRxivlinklink
2023.07GeneMask: Fast Pretraining of Gene Sequences to Enable Few-Shot LearningECAI'23linklink
2023.06Transfer learning enables predictions in network biologynaturelinklink
2023.06GENA-LM: A Family of Open-Source Foundational DNA Language Models for Long SequencesbioRxivlinklink
2023.06HyenaDNA: long-range genomic sequence modeling at single nucleotide resolutionNIPS'23linklink
2023.01The Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human GenomicsbioRxivlinklink
2023.01Species-aware DNA language modelingbioRxivlinklink
2022.08MoDNA: motif-oriented pre-training for DNA language modelBCB'22linklink
2021.02DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genomeBioinformaticslinklink

DNA Sequence Design

TimeTitleVenuePaperCode
2026.06Interpretability Driven Evolutionary Approach for the Design of Biological SequencesICML'26linklink
2026.01Iterative Distillation for Reward-Guided Fine-Tuning of Diffusion Models in Biomolecular DesignICML'26link-
2025.05Discrete Diffusion Modeling by Estimating the Ratios of the Data DistributionICML'24 Best paperlinklink
2025.05Multi-Objective-Guided Discrete Flow Matchingfor Controllable Biological Sequence DesignICML'25linklink
2025.05Ctrl-DNA: Controllable Cell-Type-Specific Regulatory DNA Design via Constrained RLNeurIPS'25linklink
2025.03Regulatory dna sequence design with reinforcement learningICLR'25linklink
2025.02DNAdesign: feature-aware in silico design of synthetic DNA through mutationBioinformatics'25linklink
2024.10Absorb & Escape: Overcoming Single Model Limitations in Generating Genomic SequencesNeurIPS'24linklink
2024.09Designing realistic regulatory DNA with autoregressive language modelsGenome Researchlinklink
2024.05Dirichlet Flow Matching with Applications to DNA Sequence DesignICML'25linklink

DNA Multimodal

TimeTitleVenuePaperCode
2025.10BioReason: Incentivizing Multimodal Biological Reasoning within a DNA-LLM ModelNeurIPS'25linklink

RNA Foundation Models

TimeTitleVenuePaperCode
2024.10Orthrus: Towards Evolutionary and Functional RNA Foundation ModelsNeurIPS'24linklink

Sequence-to-Function

TimeTitleVenuePaperCode
2026.01Advancing regulatory variant effect prediction with AlphaGenomeNaturelinklink
2025.12A foundational model for joint sequence-function multi-species modeling at scale for long-range genomic predictionBioRxivlinklink Hugging Face
2025.12Enigma: An Efficient Model for Deciphering Regulatory GenomicsBioRxivlinklink
2025.11Nona: A unifying multimodal masking framework for functional genomicsBioRxivlink-
2025.06Assessing large-scale genomic language models in predicting personal gene expression: promises and limitationsBioRxivlinklink
2025.04HyDrop v2: Scalable atlas construction for training sequence-to-function modelsBioRxivlinklink

Sequence-to-Expression

TimeTitleVenuePaperCode
2026.02Extending Sequence Length is Not All You Need: Effective Integration of Multimodal Signals for Gene Expression PredictionICLR'26 Orallinklink
2025.02Learning to discover regulatory elements for gene expression predictionICLR'25 Orallinklink data

Downstream Tasks

Gene Pathogenicity Prediction

TimeTitleVenuePaperCode
2024.06PathoLM: Identifying pathogenicity from the DNA sequence through the Genome Foundation ModelarXivlink-
2024.06Gene Pathogenicity Prediction using Genomic Foundation ModelsAAAI'24 Spring Symposium on Clinical Foundation Modelslink-

Retrieval-Augmented Generation

TimeTitleVenuePaperCode
2024.06GeneRAG: Enhancing Large Language Models with Gene-Related Task by Retrieval-Augmented GenerationbioRxivlinklink

Function Prediction

TimeTitleVenuePaperCode
2024.07FGBERT: Function-Driven Pre-trained Gene Language Model for MetagenomicsarXivlink-
2023.07PLPMpro: Enhancing promoter sequence prediction with prompt-learning based pre-trained language modelCIBMlink-
2021.10Effective gene expression prediction from sequence by integrating long-range interactionsNature Methodslinklink

Perturbation

TimeTitleVenuePaperCode
2024.08Scouter: a transcriptional response predictor for unseen genetic perturbtions with LLM embeddingspypilinklink
2024.07Enhancing generative perturbation models with LLM-informed gene embeddingsICLR'24 Workshoplink-
2024.03A genome-scale deep learning model to predict gene expression changes of genetic perturbations from multiplex biological networksarXivlinklink

Variants and Evolution Prediction

TimeTitleVenuePaperCode
2025.10VariantFormer: A hierarchical transformer integrating DNA sequences with genetic variations and regulatory landscapes for personalized gene expression predictionBioRxivlinklink
2025.04BioToken and BioFM – Biologically-Informed Tokenization Enables Accurate and Efficient Genomic Foundation ModelsBioRxivlinklink
2025.02A SNP Foundation Model: Application in Whole-Genome Haplotype Phasing and Genotype ImputationBioRxivlink-
2025.01A DNA language model based on multispecies alignment predicts the effects of genome-wide variantsNature Biotechnologylinklink
2024.11Leveraging genomic deep learning models for non-coding variant effect predictionArXivlink-
2024.04Species-aware DNA language models capture regulatory elements and their evolutionGenome Biologylinklink
2023.10GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamicsThe International Journal of High Performance Computing Applicationslinklink
2023.08DNA language models are powerful predictors of genome-wide variant effectsPNASlinklink
2022.05SNP2Vec: Scalable Self-Supervised Pre-Training for Genome-Wide Association StudyACL'22linklink

Fine-tuning for Genomes and proteins

TimeTitleVenuePaperCode
2024.09Fine-tuning sequence-to-expression models on personal genome and transcriptome databioRxivlinklink
2024.08Enhancing recognition and interpretation of functional phenotypic sequences through fine-tuning pre-trained genomic modelsJournal of Translational Medicinelinklink
2024.08Fine-tuning protein language models boosts predictions across diverse tasksNature Communicationslinklink
2024.02Efficient and Scalable Fine-Tune of Language Models for Genome UnderstandingarXivlinklink
2023.11Parameter-Efficient Fine-Tune on Open Pre-trained Transformers for Genomic SequenceNeurIPS'23 Workshop GenBiolink-
2024.01ViraLM: Empowering Virus Discovery through the Genome Foundation ModelbioRxivlinklink

Interaction Prediction

TimeTitleVenuePaperCode
2024.08Large-Scale Multi-omic Biosequence Transformers for Modeling Peptide-Nucleotide InteractionsarXivlinklink
2024.04Genomic language model predicts protein co-regulation and functionnature communicationslinklink
2024.01Gene-associated Disease Discovery Powered by Large Language ModelsarXivlink-

Identification of Transcription Factor Binding Sites

TimeTitleVenuePaperCode
2025.03deepTFBS: Improving within- and cross-species prediction of transcription factor binding using deep multi-task and transfer learningBioRxivlinklink
2024.10DNA breathing integration with deep learning foundational model advances genome-wide binding prediction of human transcription factorsNucleic Acids Researchlinklink
2024.08BertSNR: an interpretable deep learning framework for single-nucleotide resolution identification of transcription factor binding sites based on DNA language modelBioinformaticslinklink
2024.05BERT-TFBS: a novel BERT-based model for predicting transcription factor binding sites by transfer learningBriefings in Bioinformaticslinklink
2024.01Multiomics-integrated deep language model enables in silico genome-wide detection of transcription factor binding site in unexplored biosamplesBioinformaticslink-

Origins of Replication Rite Prediction

TimeTitleVenuePaperCode
2024.01PLANNER: a multi-scale deep language model for the origins of replication site predictionIEEE Journal of Biomedical and Health Informaticslink-

DNA-binding Protein Prediction

TimeTitleVenuePaperCode
2024.09Improving prediction performance of general protein language model by domain-adaptive pretraining on DNA-binding proteinNature Communicationslinklink
2024.07Prediction of Protein-DNA Binding Sites Based on Protein Language Model and Deep LearningInternational Conference on Intelligent Computinglink-
2024.03 ✨✨✨EquiPNAS: improved protein–nucleic acid binding site prediction using protein-language-model-informed equivariant deep graph neural networksNucleic Acids Researchlinklink
2024.01Predictive Recognition of DNA-binding Proteins Based on Pre-trained Language Model BERTJournal of Bioinformatics and Computational Biologylink-
2024.01Protein–DNA binding sites prediction based on pre-trained protein language model and contrastive learningBriefings in Bioinformaticslinklink
2022.09Improving language model of human genome for DNA–protein binding prediction based on task-specific pre-trainingInterdisciplinary Sciences: Computational Life Scienceslinklink

RNA Prediction

TimeTitleVenuePaperCode
2024.07 ✨✨✨Single-sequence protein-RNA complex structure prediction by geometric attention-enabled pairing of biological language modelsbioRxivlinklink
2024.05RNAErnie: Multi-purpose RNA language modelling with motif-aware pretraining and type-guided fine-tuningNature Machine Intelligencelinklink
2024.02RiNALMo: General-Purpose RNA Language Models Can Generalize Well on Structure Prediction TasksarXivlinklink
2023.10Multiple sequence alignment-based RNA language model and its application to structural inferenceNucleic Acids Researchlinklink
2023.07Uni-RNA: Universal Pre-trained Models Revolutionize RNA ResearchbioRxivlink-
2023.06Prediction of Multiple Types of RNA Modifications via Biological Language ModelTCBBlinklink
2023.02Self-supervised learning on millions of pre-mRNA sequences improves sequence-based RNA splicing predictionbioRxivlinklink

Sequence Modeling

TimeTitleVenuePaperCode
2026.06Adaptive DNA Sequence Modeling via Synergistic Plasticity UnitsICML'26link-
2026.02GeneZip: Region-Aware Compression for Long Context DNA ModelingICML'26linklink
2025.05HAD: Hybrid Architecture Distillation Outperforms Teacher in Genomic Sequence ModelingarXivlink-
2025.04Leveraging State Space Models in Long Range GenomicsICLR'25 workshoplink-
2025.03Lyra: An Efficient and Expressive Subquadratic Architecture for Modeling Biological SequencesarXivlink-
2025.02Life-Code: Central Dogma Modeling with Multi-Omics Sequence UnificationarXivlink-
2024.11Unveiling Protein-DNA Interdependency: Harnessing Unified Multimodal Sequence Modeling, Understanding and Generation--link
2024.09Toward Understanding BERT-Like Pre-Training for DNA Foundation ModelsarXivlink-
2024.08LitGene: a transformer-based model that uses contrastive learning to integrate textual information into gene representationsbioRxivlinklink
2024.08BiRNA-BERT allows efficient RNA language modeling with adaptive tokenizationbioRxivlinklink
2024.07 ✨✨✨VQDNA: Unleashing the Power of Vector Quantization for Multi-Species Genomic Sequence ModelingICML'24linklink
2024.06 💥💥💥Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence ModelingICML’24linklink
2024.06Contrastive pre-training for sequence based genomics modelsbioRxivlinklink
2024.05Dirichlet Flow Matching with Applications to DNA Sequence DesignICML’24linklink
2024.05 🏋️🏋️Self-Distillation Improves DNA Sequence InferencearXivlinklink
2024.05 🏋️🏋️Accurate and efficient protein embedding using multi-teacher distillation learningarXivlinklink
2024.04Effect of tokenization on transformers for biological sequencesBioinformaticslinklink
2024.04A Sparse and Wide Neural Network Model for DNA SequencesSRNNlinklink
2024.03Self-supervised learning for DNA sequences with circular dilated convolutional networksNeural Networkslinklink
2024.01ProtHyena: A fast and efficient foundation protein language model at single amino acid ResolutionbioRxivlinklink
2023.06HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide ResolutionNeurIPS’23linklink

Reinforcement learning

TimeTitleVenuePaperCode
2026.06Emergent Biological Realism in RL-Trained DNA Language ModelsICML’26link-

Evolution

TimeTitleVenuePaperCode
2026.06Predicting evolutionary rate as a pretraining task improves genome language model representationsICML’26linklink

Basics of Sequence Modeling

TimeTitleVenuePaperCode
2025.06Comba: Improving Bilinear RNNs with Closed-loop ControlarXivlinklink
2025.05Efficient Long-range Language Modeling with Self-supervised Causal RetrievalICML'25link-
2025.04Learnable Multi-Scale Wavelet Transformer: A Novel Alternative to Self-AttentionarXivlink-
2025.03Linear-MoE: Linear Sequence Modeling Meets Mixture-of-ExpertsICLR'25linklink
2025.02MoM: Linear Sequence Modeling with Mixture-of-MemoriesarXivlinklink
2025.02Linear Attention for Efficient Bidirectional Sequence ModelingarXivlinklink
2024.10LongMamba: Enhancing Mamba's Long-Context Capabilities via Training-Free Receptive Field EnlargementICLR'25 Conference Submissionlink-
2024.09Reparameterized Multi-Resolution Convolutions for Long Sequence ModellingarXivlink-
2024.08SE(3)-Hyena Operator for Scalable Equivariant LearningarXivlink-
2024.04LongVQ: Long Sequence Modeling with Vector Quantization on Structured MemoryIJCAI'24link-
2024.02Transformer-VQ: Linear-Time Transformers via Vector QuantizationICLR’24link-
2024.02MoE-Mamba: Efficient Selective State Space Models with Mixture of ExpertsarXivlink-
2024.01Scavenging Hyena: Distilling Transformers into Long Convolution ModelsarXivlink-

Tokenization

TimeTitleVenuePaperCode
2026.06BioToken and BioFM – Biologically-Informed Tokenization Enables Accurate and Efficient Genomic Foundation ModelsICML'26linklink
2026.06LDARNet: DNA Adaptive Representation Network with Learnable Tokenization for Genomic ModelingICML'26linklink
2026.04EvoLen: Evolution-Guided Tokenization for DNA Language ModelarXivlink-
2026.01PatchDNA: A Flexible and Biologically-Informed Alternative to Tokenization for DNAICLR'26link-
2025.11MergeDNA: Context-aware Genome Modeling with Dynamic Tokenization through Token MergingAAAI'26 Orallink-
2025.05DNAZEN: Enhanced Gene Sequence Representations via Mixed Granularities of Coding UnitsRxivlinklink
2025.04Genomic Tokenizer: Toward a biology-driven tokenization in transformer models for DNA sequencesBioRxivlinklink
2025.01A partition cover approach to tokenizationarXivlinklink
2024.11Enhancing Large Language Models through Adaptive TokenizersNeurIPS'24link-
2024.11Theoretical Analysis of Byte-Pair EncodingarXivlink-
2024.10Generation with Dynamic VocabularyEMNLP'24linklink
2024.10Adaptive BPE Tokenization for Enhanced Vocabulary Adaptation in Finetuning Pretrained Language ModelsEMNLP'24 Findingslinklink
2024.10Model Decides How to Tokenize: Adaptive DNA Sequence Tokenization with MxDNANeurIPS'24linklink
2024.09BPE Gets Picky: Efficient Vocabulary Refinement During Tokenizer TrainingNeurIPS'24linklink
2024.09A Comparison of Tokenization Impact in Attention Based and State Space Genomic Language ModelsbioRxivlink-
2024.04Scaffold-BPE: Enhancing Byte Pair Encoding for Large Language Models with Simple and Effective Scaffold Token RemovalarXivlinklink
2024.04Effect of tokenization on transformers for biological sequencesBioinformaticslinklink
2024.02Tokenization Is More Than CompressionarXivlink-
2023.10Toward Understanding BERT-Like Pre-Training for DNA Foundation ModelsarXivlink-

Position Code

TimeTitleVenuePaperCode
2025.10Decoupling the "What" and "Where" With Polar Coordinate Positional EmbeddingsarXivlinklink
2025.05PaTH Attention: Position Encoding via Accumulating Householder TransformationsarXivlinklink
2025.04Effective Length Extrapolation via Dimension-Wise Positional Embeddings ManipulationarXivlinklink
2025.04Rethinking RoPE: A Mathematical Blueprint for N-dimensional Positional EncodingarXivlink-
2024.03YaRN: Efficient Context Window Extension of Large Language ModelsICLR'24linklink
2024.01Roformer: Enhanced transformer with rotary position embeddingNeurocomputinglinklink

Adversarial Attack

TimeTitleVenuePaperCode
2026.06Poisoning the Genome: Targeted Backdoor Attacks on DNA Foundation ModelsarXivlinklink
2025.05Fast and Low-Cost Genomic Foundation Models via Outlier RemovalICML'25linklink
2025.05GeneBreaker: Jailbreak Attacks against DNA Language Models with Pathogenicity GuidancearXivlinklink

Quantization

TimeTitleVenuePaperCode
2024.06Low-Rank Quantization-Aware Training for LLMsarXivlinklink

Fine-tuning

TimeTitleVenuePaperCode
2024.07LoRA+: Efficient Low Rank Adaptation of Large ModelsICML'24linklink
2021.10LoRA: Low-Rank Adaptation of Large Language ModelsarXivlinklink
2024.07DoRA: Weight-Decomposed Low-Rank AdaptationICML'24linklink
2024.07Accurate LoRA-Finetuning Quantization of LLMs via Information RetentionICML'24linklink
2024.05🏋️🏋️Self-Distillation Bridges Distribution Gap in Language Model Fine-TuningACL'24linklink

Reducing Knowledge Hallucination

TimeTitleVenuePaperCode
2024.06Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language ModelsICML'24linklink

Data processing

1、从 FASTA 文件中加载并查询基因组序列.
2、DNABERT2 Fine-Tuning for DHS Specificity Prediction.
3、Scaling-Laws-of-Genomic.
4、Deafness-mutation-sites.
5、DNABERT-2_CNN_BiLSTM.
6、1_Train_HG.
7、dbtk-dnabert.
8、DNABERT2_Tokenizer.
9、处理序列的R脚本.

  1. Awesome-LLM-Learning
  2. Scientific-LLM-Survey (Biological & Chemical Domains)
  3. LLM-FineTuning-Large-Language-Models
  4. Awesome-llms-fine-tuning (Explore a comprehensive collection of resources, tutorials, papers, tools, and best practices for fine-tuning Large Language Models (LLMs))
  5. Awesome-LLM4RS-Papers
  6. LLM4Rec-Awesome-Papers (A list of awesome papers and resources of recommender system on large language model (LLM))
  7. Awesome-Code-LLM (A curated list of language modeling researches for code and related datasets)

Contributors

ychuest

(back to top)

bioinformatics
downstream-tasks
genome
graph
llms
prediction
predictive-modeling

ychuest/Awesome-LLMs-meet-genomes

Explore a comprehensive collection of basic theories, applications, papers, and best practices about Large Language Models (LLMs) in genomes.

157

207 commits

updated Oct 2, 2026

See the code

README

Awesome-LLMs-meet-genomes

Awesome-LLMs-meet-genomes is a collection of state-of-the-art, novel, exciting LLMs methods on genomes. It contains papers, codes, datasets, evaluations, and analyses. Any additional information about LLMs for bioinformatics is welcome, and we are glad to add you to the contributor list here. Any problems, please contact yangchengyjs@163.com. If you find this repository useful to your research or work, it is really appreciated to star this repository. ✨

This project is sponsored by E-API.

E-API provides unified multi-model API access, supporting integrations and collaborations for open-source projects, developer tools, and AI applications.

New users who register through the referral link below may receive RMB 5 in product credits: https://eapi.site/register?aff=3M9ZLY

This sponsorship and referral relationship is not affiliated with, endorsed by, or officially connected to GitHub.

Learn more about E-API / Collaboration


Made with Python GitHub stars GitHub forks visitors

Table of Content


🔔 News

  • 🧬✔️ [2024/09] Benchmarks for classification of genomic sequences link.
  • 💥 [2024/08] Some real-world experience in training LLMs link.
  • 💥 [2024/08] Three ways of Fine-tuning link.
  • 💥 [2024/08] Visualisation of the Transformer Principle link.
  • 🌟 [2024/08] The Cultivation Method of Large Language Models: A Path to Success link.
  • 📖 [2024/08] Large Language Models: From Theory to Practice link.

Important Survey Papers

YearTitleVenuePaperCode
2026.02Generative Artificial Intelligence for Biology: Toward Unifying Models, Algorithms, and ModalitiesarXivLink-
2025.05Back to recurrent processing at the crossroad of transformers and state-space modelsNature Machine IntelligenceLink-
2025.03Transformers and genome language modelsNature Machine IntelligenceLink-
2025.03Language modelling techniques for analysing the impact of human genetic variationarXivLink-
2025.03Biological Sequence with Language Model Prompting: A SurveyarXivLink-
2025.01Large Language Models for BioinformaticsarXivLink-
2024.09Genomic Language Models: Opportunities and ChallengesarXivLink-
2024.07Scientific Large Language Models: A Survey on Biological & Chemical DomainsarXivLinklink
2024.01Large language models in bioinformatics: applications and perspectivesarXivLink-
2023.11To Transformers and Beyond: Large Language Models for the GenomearXivLink-
2023.01Applications of transformer-based language models in bioinformatics: a surveyBioinformatics AdvancesLink-

Important Benchmarks

YearTitleVenuePaperCode
2026.06GENEB: Why Genomic Models Are Hard to CompareICML'26LinkHuggingface
2025.11DNALONGBENCH: a benchmark suite for long-range DNA prediction tasksnature communicationsLinkLink

Genomic Large Language Models (Gene-LLMs)

Generic Base Models

YearTitleVenuePaperCode
2026.05PlasmidLM: A Promptable DNA Language Model via Verifiable-Reward Post-TrainingbioXivlink-
2026.04From nucleotides to semantics: genomic representation learning via joint-embedding predictive architecturebioXivlink-
2026.02JEPA-DNA: Grounding Genomic Foundation Models through Joint-Embedding Predictive ArchitecturesarXivlinklink
2026.02Extending Sequence Length is Not All You Need: Effective Integration of Multimodal Signals for Gene Expression PredictionICLR'26 Orallinklink
2026.01DNACHUNKER: Learnable Tokenization for DNA Language ModelsICML'26linklink
2026.01GENERator: A Long-Context Generative Genomic Foundation ModelarXivlinklink
2026.01Functional In-Context Learning in Genomic Language Models with Nucleotide-Level Supervision and Genome Compression (GENERator-v2 Technical Report)BioRxivlinklink
2025.11Context-Aware Regularization with Markovian Integration for Attention-Based Nucleotide AnalysisNeurIPS'25linklink
2025.11MergeDNA: Context-aware Genome Modeling with Dynamic Tokenization through Token MergingAAAI'26 Orallink-
2025.10Annotating the genome at single-nucleotide resolution with DNA foundation modelsNature Methodslinklink
2025.07JanusDNA: A Powerful Bi-directional Hybrid DNA Foundation ModelNeurIPS'25linklink
2025.07BMFM-DNA: A SNP-aware DNA foundation model to capture variant effectsarXivlinklink
2025.07TrinityDNA: A Bio-Inspired Foundational Model for Efficient Long-Sequence DNA ModelingAAAI'26 Submissionlink-
2025.06Generalized biological foundation model with unified nucleic acid and protein languageNature Machine Intelligencelinklink
2025.06eccDNAMamba: A Pre-Trained Model for Ultra-Long eccDNA Sequence AnalysisICML'25linklink
2025.06SPACE: Your Genomic Profile Predictor is a Powerful DNA Foundation ModelICLR'25linklink
2025.04Bridging Sequence-Structure Alignment in RNA Foundation ModelsAAAI'25linklink
2025.03HydraRNA: a hybrid architecture based full-length RNA language modelbioXivlinklink
2025.03Pre-training Genomic Language Model with Variants for Better Modeling Functional GenomicsbioXivlinklink
2025.03Enhancing DNA Foundation Models to Address Masking InefficienciesarXivlinklink
2025.02HybriDNA: A Hybrid Transformer-Mamba2 Long-Range DNA Language ModelarXivlink-
2025.02Omni-DNA: A Unified Genomic Foundation Model for Cross-Modal and Multi-Task LearningarXivlinklink
2025.01MutBERT: Probabilistic Genome Representation Improves Genomics Foundation ModelsBioXivlinklink
2025.01GENA-LM: a family of open-source foundational DNA language models for long sequencesNucleic Acids Researchlinklink
2024.12EpiGePT: a pretrained transformer-based language model for context-specific human epigenomicsGenome Biologylinklink
2024.11VIRALpre: Genomic Foundation Model Embedding Fused with K-mer Feature for Virus IdentificationbioRxivlink-
2024.11BEACON: Benchmark for Comprehensive RNA Tasks and Language ModelsNeurIPS'24linklink
2024.11DNA Language Models for RNA AnalysesICLR'25 Conference Submissionlink-
2024.10The OMG dataset: An Open MetaGenomic corpus for mixed-modality genomic language modelingBioRxivlinklink
2024.10Character-level Tokenizations as Powerful Inductive Biases for RNA Foundational ModelsNeurIPS'24linklink
2024.10Revisiting K-mer Profile for Effective and Scalable Genome Representation LearningNeurIPS'24linklink
2024.10A long-context language model for deciphering and generating bacteriophage genomesNature Communicationslinklink
2024.10Revisiting Convolution Architecture in the Realm of DNA Foundation ModelsICLR'25 Conference Submissionlink-
2024.10Hyperbolic Genome EmbeddingsICLR'25 Conference Submissionlink-
2024.10dnaGrinder: a lightweight and high-capacity genomic foundation modelICLR'25 Conference Submissionlink-
2024.10DNABERT-S: Pioneering Species Differentiation with Species-Aware DNA EmbeddingsICLR'25 Conference Submissionlink-
2024.10Long-range gene expression prediction with token alignment of large language modelarXivlink-
2024.09A Comparison of Tokenization Impact in Attention Based and State Space Genomic Language ModelsbioRxivlink-
2024.09Designing realistic regulatory DNA with autoregressive language modelsGenome Researchlink-
2024.08Understanding the Natural Language of DNA using Encoder-Decoder Foundation Models with Byte-level PrecisionBioinformatics Advanceslinklink
2024.08Unlocking Efficiency: Adaptive Masking for Gene Transformer ModelsECAI'24linklink
2024.07Genomics-FM: Universal Foundation Model for Versatile and Data-Efficient Functional Genomic AnalysisbioRxivlinklink
2024.07 ✨✨✨VQDNA: Unleashing the Power of Vector Quantization for Multi-Species Genomic Sequence ModelingICML'24linklink
2024.07OmniGenome: Aligning RNA Sequences with Secondary Structures in Genomic Foundation ModelsarXivlinklink
2024.07Scorpio : Enhancing Embeddings to Improve Downstream Analysis of DNA sequencesbioRxivlinklink
2024.07DNA language model GROVER learns sequence context in the human genome (可用于蛋白质-DNA结合预测任务)Nature Machine Intelligencelinklink tutorials
2024.05Are Genomic Language Models All You Need? Exploring Genomic Language Models on Protein Downstream TasksbioRxivlinklink
2024.05GeneAgent: Self-verification Language Agent for Gene Set Knowledge Discovery using Domain DatabasesarXivlink-
2024.05DeepGene: An Efficient Foundation Model for Genomics based on Pan-genome Graph TransformerbioRxivlinklink
2024.05Self-Distillation Improves DNA Sequence Inference DatabasesarXivlinklink
2024.04Effect of tokenization on transformers for biological sequencesBioinformaticslinklink
2024.04DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species GenomeICLR'24linklink
2024.02Exploring Genomic Large Language Models: Bridging the Gap between Natural Language and Gene SequencesbioRxivlinklink data
2024.02Sequence modeling and design from molecular to genome scale with EvobioRxivlinklink
2024.01ProkBERT family: genomic language models for microbiome applicationsFrontiers in MicrobiologyLinklink
2023.09The Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human GenomicsbioRxivlinklink
2023.08DNAGPT: A Generalized Pre-trained Tool for Versatile DNA Sequence Analysis TasksbioRxivlinklink
2023.07EpiGePT: a Pretrained Transformer model for epigenomicsbioRxivlinklink
2023.07GeneMask: Fast Pretraining of Gene Sequences to Enable Few-Shot LearningECAI'23linklink
2023.06Transfer learning enables predictions in network biologynaturelinklink
2023.06GENA-LM: A Family of Open-Source Foundational DNA Language Models for Long SequencesbioRxivlinklink
2023.06HyenaDNA: long-range genomic sequence modeling at single nucleotide resolutionNIPS'23linklink
2023.01The Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human GenomicsbioRxivlinklink
2023.01Species-aware DNA language modelingbioRxivlinklink
2022.08MoDNA: motif-oriented pre-training for DNA language modelBCB'22linklink
2021.02DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genomeBioinformaticslinklink

DNA Sequence Design

TimeTitleVenuePaperCode
2026.06Interpretability Driven Evolutionary Approach for the Design of Biological SequencesICML'26linklink
2026.01Iterative Distillation for Reward-Guided Fine-Tuning of Diffusion Models in Biomolecular DesignICML'26link-
2025.05Discrete Diffusion Modeling by Estimating the Ratios of the Data DistributionICML'24 Best paperlinklink
2025.05Multi-Objective-Guided Discrete Flow Matchingfor Controllable Biological Sequence DesignICML'25linklink
2025.05Ctrl-DNA: Controllable Cell-Type-Specific Regulatory DNA Design via Constrained RLNeurIPS'25linklink
2025.03Regulatory dna sequence design with reinforcement learningICLR'25linklink
2025.02DNAdesign: feature-aware in silico design of synthetic DNA through mutationBioinformatics'25linklink
2024.10Absorb & Escape: Overcoming Single Model Limitations in Generating Genomic SequencesNeurIPS'24linklink
2024.09Designing realistic regulatory DNA with autoregressive language modelsGenome Researchlinklink
2024.05Dirichlet Flow Matching with Applications to DNA Sequence DesignICML'25linklink

DNA Multimodal

TimeTitleVenuePaperCode
2025.10BioReason: Incentivizing Multimodal Biological Reasoning within a DNA-LLM ModelNeurIPS'25linklink

RNA Foundation Models

TimeTitleVenuePaperCode
2024.10Orthrus: Towards Evolutionary and Functional RNA Foundation ModelsNeurIPS'24linklink

Sequence-to-Function

TimeTitleVenuePaperCode
2026.01Advancing regulatory variant effect prediction with AlphaGenomeNaturelinklink
2025.12A foundational model for joint sequence-function multi-species modeling at scale for long-range genomic predictionBioRxivlinklink Hugging Face
2025.12Enigma: An Efficient Model for Deciphering Regulatory GenomicsBioRxivlinklink
2025.11Nona: A unifying multimodal masking framework for functional genomicsBioRxivlink-
2025.06Assessing large-scale genomic language models in predicting personal gene expression: promises and limitationsBioRxivlinklink
2025.04HyDrop v2: Scalable atlas construction for training sequence-to-function modelsBioRxivlinklink

Sequence-to-Expression

TimeTitleVenuePaperCode
2026.02Extending Sequence Length is Not All You Need: Effective Integration of Multimodal Signals for Gene Expression PredictionICLR'26 Orallinklink
2025.02Learning to discover regulatory elements for gene expression predictionICLR'25 Orallinklink data

Downstream Tasks

Gene Pathogenicity Prediction

TimeTitleVenuePaperCode
2024.06PathoLM: Identifying pathogenicity from the DNA sequence through the Genome Foundation ModelarXivlink-
2024.06Gene Pathogenicity Prediction using Genomic Foundation ModelsAAAI'24 Spring Symposium on Clinical Foundation Modelslink-

Retrieval-Augmented Generation

TimeTitleVenuePaperCode
2024.06GeneRAG: Enhancing Large Language Models with Gene-Related Task by Retrieval-Augmented GenerationbioRxivlinklink

Function Prediction

TimeTitleVenuePaperCode
2024.07FGBERT: Function-Driven Pre-trained Gene Language Model for MetagenomicsarXivlink-
2023.07PLPMpro: Enhancing promoter sequence prediction with prompt-learning based pre-trained language modelCIBMlink-
2021.10Effective gene expression prediction from sequence by integrating long-range interactionsNature Methodslinklink

Perturbation

TimeTitleVenuePaperCode
2024.08Scouter: a transcriptional response predictor for unseen genetic perturbtions with LLM embeddingspypilinklink
2024.07Enhancing generative perturbation models with LLM-informed gene embeddingsICLR'24 Workshoplink-
2024.03A genome-scale deep learning model to predict gene expression changes of genetic perturbations from multiplex biological networksarXivlinklink

Variants and Evolution Prediction

TimeTitleVenuePaperCode
2025.10VariantFormer: A hierarchical transformer integrating DNA sequences with genetic variations and regulatory landscapes for personalized gene expression predictionBioRxivlinklink
2025.04BioToken and BioFM – Biologically-Informed Tokenization Enables Accurate and Efficient Genomic Foundation ModelsBioRxivlinklink
2025.02A SNP Foundation Model: Application in Whole-Genome Haplotype Phasing and Genotype ImputationBioRxivlink-
2025.01A DNA language model based on multispecies alignment predicts the effects of genome-wide variantsNature Biotechnologylinklink
2024.11Leveraging genomic deep learning models for non-coding variant effect predictionArXivlink-
2024.04Species-aware DNA language models capture regulatory elements and their evolutionGenome Biologylinklink
2023.10GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamicsThe International Journal of High Performance Computing Applicationslinklink
2023.08DNA language models are powerful predictors of genome-wide variant effectsPNASlinklink
2022.05SNP2Vec: Scalable Self-Supervised Pre-Training for Genome-Wide Association StudyACL'22linklink

Fine-tuning for Genomes and proteins

TimeTitleVenuePaperCode
2024.09Fine-tuning sequence-to-expression models on personal genome and transcriptome databioRxivlinklink
2024.08Enhancing recognition and interpretation of functional phenotypic sequences through fine-tuning pre-trained genomic modelsJournal of Translational Medicinelinklink
2024.08Fine-tuning protein language models boosts predictions across diverse tasksNature Communicationslinklink
2024.02Efficient and Scalable Fine-Tune of Language Models for Genome UnderstandingarXivlinklink
2023.11Parameter-Efficient Fine-Tune on Open Pre-trained Transformers for Genomic SequenceNeurIPS'23 Workshop GenBiolink-
2024.01ViraLM: Empowering Virus Discovery through the Genome Foundation ModelbioRxivlinklink

Interaction Prediction

TimeTitleVenuePaperCode
2024.08Large-Scale Multi-omic Biosequence Transformers for Modeling Peptide-Nucleotide InteractionsarXivlinklink
2024.04Genomic language model predicts protein co-regulation and functionnature communicationslinklink
2024.01Gene-associated Disease Discovery Powered by Large Language ModelsarXivlink-

Identification of Transcription Factor Binding Sites

TimeTitleVenuePaperCode
2025.03deepTFBS: Improving within- and cross-species prediction of transcription factor binding using deep multi-task and transfer learningBioRxivlinklink
2024.10DNA breathing integration with deep learning foundational model advances genome-wide binding prediction of human transcription factorsNucleic Acids Researchlinklink
2024.08BertSNR: an interpretable deep learning framework for single-nucleotide resolution identification of transcription factor binding sites based on DNA language modelBioinformaticslinklink
2024.05BERT-TFBS: a novel BERT-based model for predicting transcription factor binding sites by transfer learningBriefings in Bioinformaticslinklink
2024.01Multiomics-integrated deep language model enables in silico genome-wide detection of transcription factor binding site in unexplored biosamplesBioinformaticslink-

Origins of Replication Rite Prediction

TimeTitleVenuePaperCode
2024.01PLANNER: a multi-scale deep language model for the origins of replication site predictionIEEE Journal of Biomedical and Health Informaticslink-

DNA-binding Protein Prediction

TimeTitleVenuePaperCode
2024.09Improving prediction performance of general protein language model by domain-adaptive pretraining on DNA-binding proteinNature Communicationslinklink
2024.07Prediction of Protein-DNA Binding Sites Based on Protein Language Model and Deep LearningInternational Conference on Intelligent Computinglink-
2024.03 ✨✨✨EquiPNAS: improved protein–nucleic acid binding site prediction using protein-language-model-informed equivariant deep graph neural networksNucleic Acids Researchlinklink
2024.01Predictive Recognition of DNA-binding Proteins Based on Pre-trained Language Model BERTJournal of Bioinformatics and Computational Biologylink-
2024.01Protein–DNA binding sites prediction based on pre-trained protein language model and contrastive learningBriefings in Bioinformaticslinklink
2022.09Improving language model of human genome for DNA–protein binding prediction based on task-specific pre-trainingInterdisciplinary Sciences: Computational Life Scienceslinklink

RNA Prediction

TimeTitleVenuePaperCode
2024.07 ✨✨✨Single-sequence protein-RNA complex structure prediction by geometric attention-enabled pairing of biological language modelsbioRxivlinklink
2024.05RNAErnie: Multi-purpose RNA language modelling with motif-aware pretraining and type-guided fine-tuningNature Machine Intelligencelinklink
2024.02RiNALMo: General-Purpose RNA Language Models Can Generalize Well on Structure Prediction TasksarXivlinklink
2023.10Multiple sequence alignment-based RNA language model and its application to structural inferenceNucleic Acids Researchlinklink
2023.07Uni-RNA: Universal Pre-trained Models Revolutionize RNA ResearchbioRxivlink-
2023.06Prediction of Multiple Types of RNA Modifications via Biological Language ModelTCBBlinklink
2023.02Self-supervised learning on millions of pre-mRNA sequences improves sequence-based RNA splicing predictionbioRxivlinklink

Sequence Modeling

TimeTitleVenuePaperCode
2026.06Adaptive DNA Sequence Modeling via Synergistic Plasticity UnitsICML'26link-
2026.02GeneZip: Region-Aware Compression for Long Context DNA ModelingICML'26linklink
2025.05HAD: Hybrid Architecture Distillation Outperforms Teacher in Genomic Sequence ModelingarXivlink-
2025.04Leveraging State Space Models in Long Range GenomicsICLR'25 workshoplink-
2025.03Lyra: An Efficient and Expressive Subquadratic Architecture for Modeling Biological SequencesarXivlink-
2025.02Life-Code: Central Dogma Modeling with Multi-Omics Sequence UnificationarXivlink-
2024.11Unveiling Protein-DNA Interdependency: Harnessing Unified Multimodal Sequence Modeling, Understanding and Generation--link
2024.09Toward Understanding BERT-Like Pre-Training for DNA Foundation ModelsarXivlink-
2024.08LitGene: a transformer-based model that uses contrastive learning to integrate textual information into gene representationsbioRxivlinklink
2024.08BiRNA-BERT allows efficient RNA language modeling with adaptive tokenizationbioRxivlinklink
2024.07 ✨✨✨VQDNA: Unleashing the Power of Vector Quantization for Multi-Species Genomic Sequence ModelingICML'24linklink
2024.06 💥💥💥Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence ModelingICML’24linklink
2024.06Contrastive pre-training for sequence based genomics modelsbioRxivlinklink
2024.05Dirichlet Flow Matching with Applications to DNA Sequence DesignICML’24linklink
2024.05 🏋️🏋️Self-Distillation Improves DNA Sequence InferencearXivlinklink
2024.05 🏋️🏋️Accurate and efficient protein embedding using multi-teacher distillation learningarXivlinklink
2024.04Effect of tokenization on transformers for biological sequencesBioinformaticslinklink
2024.04A Sparse and Wide Neural Network Model for DNA SequencesSRNNlinklink
2024.03Self-supervised learning for DNA sequences with circular dilated convolutional networksNeural Networkslinklink
2024.01ProtHyena: A fast and efficient foundation protein language model at single amino acid ResolutionbioRxivlinklink
2023.06HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide ResolutionNeurIPS’23linklink

Reinforcement learning

TimeTitleVenuePaperCode
2026.06Emergent Biological Realism in RL-Trained DNA Language ModelsICML’26link-

Evolution

TimeTitleVenuePaperCode
2026.06Predicting evolutionary rate as a pretraining task improves genome language model representationsICML’26linklink

Basics of Sequence Modeling

TimeTitleVenuePaperCode
2025.06Comba: Improving Bilinear RNNs with Closed-loop ControlarXivlinklink
2025.05Efficient Long-range Language Modeling with Self-supervised Causal RetrievalICML'25link-
2025.04Learnable Multi-Scale Wavelet Transformer: A Novel Alternative to Self-AttentionarXivlink-
2025.03Linear-MoE: Linear Sequence Modeling Meets Mixture-of-ExpertsICLR'25linklink
2025.02MoM: Linear Sequence Modeling with Mixture-of-MemoriesarXivlinklink
2025.02Linear Attention for Efficient Bidirectional Sequence ModelingarXivlinklink
2024.10LongMamba: Enhancing Mamba's Long-Context Capabilities via Training-Free Receptive Field EnlargementICLR'25 Conference Submissionlink-
2024.09Reparameterized Multi-Resolution Convolutions for Long Sequence ModellingarXivlink-
2024.08SE(3)-Hyena Operator for Scalable Equivariant LearningarXivlink-
2024.04LongVQ: Long Sequence Modeling with Vector Quantization on Structured MemoryIJCAI'24link-
2024.02Transformer-VQ: Linear-Time Transformers via Vector QuantizationICLR’24link-
2024.02MoE-Mamba: Efficient Selective State Space Models with Mixture of ExpertsarXivlink-
2024.01Scavenging Hyena: Distilling Transformers into Long Convolution ModelsarXivlink-

Tokenization

TimeTitleVenuePaperCode
2026.06BioToken and BioFM – Biologically-Informed Tokenization Enables Accurate and Efficient Genomic Foundation ModelsICML'26linklink
2026.06LDARNet: DNA Adaptive Representation Network with Learnable Tokenization for Genomic ModelingICML'26linklink
2026.04EvoLen: Evolution-Guided Tokenization for DNA Language ModelarXivlink-
2026.01PatchDNA: A Flexible and Biologically-Informed Alternative to Tokenization for DNAICLR'26link-
2025.11MergeDNA: Context-aware Genome Modeling with Dynamic Tokenization through Token MergingAAAI'26 Orallink-
2025.05DNAZEN: Enhanced Gene Sequence Representations via Mixed Granularities of Coding UnitsRxivlinklink
2025.04Genomic Tokenizer: Toward a biology-driven tokenization in transformer models for DNA sequencesBioRxivlinklink
2025.01A partition cover approach to tokenizationarXivlinklink
2024.11Enhancing Large Language Models through Adaptive TokenizersNeurIPS'24link-
2024.11Theoretical Analysis of Byte-Pair EncodingarXivlink-
2024.10Generation with Dynamic VocabularyEMNLP'24linklink
2024.10Adaptive BPE Tokenization for Enhanced Vocabulary Adaptation in Finetuning Pretrained Language ModelsEMNLP'24 Findingslinklink
2024.10Model Decides How to Tokenize: Adaptive DNA Sequence Tokenization with MxDNANeurIPS'24linklink
2024.09BPE Gets Picky: Efficient Vocabulary Refinement During Tokenizer TrainingNeurIPS'24linklink
2024.09A Comparison of Tokenization Impact in Attention Based and State Space Genomic Language ModelsbioRxivlink-
2024.04Scaffold-BPE: Enhancing Byte Pair Encoding for Large Language Models with Simple and Effective Scaffold Token RemovalarXivlinklink
2024.04Effect of tokenization on transformers for biological sequencesBioinformaticslinklink
2024.02Tokenization Is More Than CompressionarXivlink-
2023.10Toward Understanding BERT-Like Pre-Training for DNA Foundation ModelsarXivlink-

Position Code

TimeTitleVenuePaperCode
2025.10Decoupling the "What" and "Where" With Polar Coordinate Positional EmbeddingsarXivlinklink
2025.05PaTH Attention: Position Encoding via Accumulating Householder TransformationsarXivlinklink
2025.04Effective Length Extrapolation via Dimension-Wise Positional Embeddings ManipulationarXivlinklink
2025.04Rethinking RoPE: A Mathematical Blueprint for N-dimensional Positional EncodingarXivlink-
2024.03YaRN: Efficient Context Window Extension of Large Language ModelsICLR'24linklink
2024.01Roformer: Enhanced transformer with rotary position embeddingNeurocomputinglinklink

Adversarial Attack

TimeTitleVenuePaperCode
2026.06Poisoning the Genome: Targeted Backdoor Attacks on DNA Foundation ModelsarXivlinklink
2025.05Fast and Low-Cost Genomic Foundation Models via Outlier RemovalICML'25linklink
2025.05GeneBreaker: Jailbreak Attacks against DNA Language Models with Pathogenicity GuidancearXivlinklink

Quantization

TimeTitleVenuePaperCode
2024.06Low-Rank Quantization-Aware Training for LLMsarXivlinklink

Fine-tuning

TimeTitleVenuePaperCode
2024.07LoRA+: Efficient Low Rank Adaptation of Large ModelsICML'24linklink
2021.10LoRA: Low-Rank Adaptation of Large Language ModelsarXivlinklink
2024.07DoRA: Weight-Decomposed Low-Rank AdaptationICML'24linklink
2024.07Accurate LoRA-Finetuning Quantization of LLMs via Information RetentionICML'24linklink
2024.05🏋️🏋️Self-Distillation Bridges Distribution Gap in Language Model Fine-TuningACL'24linklink

Reducing Knowledge Hallucination

TimeTitleVenuePaperCode
2024.06Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language ModelsICML'24linklink

Data processing

1、从 FASTA 文件中加载并查询基因组序列.
2、DNABERT2 Fine-Tuning for DHS Specificity Prediction.
3、Scaling-Laws-of-Genomic.
4、Deafness-mutation-sites.
5、DNABERT-2_CNN_BiLSTM.
6、1_Train_HG.
7、dbtk-dnabert.
8、DNABERT2_Tokenizer.
9、处理序列的R脚本.

  1. Awesome-LLM-Learning
  2. Scientific-LLM-Survey (Biological & Chemical Domains)
  3. LLM-FineTuning-Large-Language-Models
  4. Awesome-llms-fine-tuning (Explore a comprehensive collection of resources, tutorials, papers, tools, and best practices for fine-tuning Large Language Models (LLMs))
  5. Awesome-LLM4RS-Papers
  6. LLM4Rec-Awesome-Papers (A list of awesome papers and resources of recommender system on large language model (LLM))
  7. Awesome-Code-LLM (A curated list of language modeling researches for code and related datasets)

Contributors

ychuest

(back to top)

bioinformatics
downstream-tasks
genome
graph
llms
prediction
predictive-modeling

Significant stargazers

Nick Doiron

465 followers · starred Jan 2026

Siyuan Li

83 followers · starred Jul 2025