Repository: micanonsens/argamba-dual-step44000
The Gamba models are a family of DNA language models from the ArGamba paper that jointly model DNA sequence and evolutionary rate information.
This repository includes the model weights and code for the Jamba autoregressive variant model (ArGamba) trained to predict both the human genome sequence and the Zoonomia 241-mammalian alignment-derived PhyloP scores using both Next Token Prediction (NTP) and Current Evolutionary-rate Prediction (CEP) tasks. For more details, see the GitHub repo.
All Gamba family models have checkpoints available at 44,000 steps:
| Checkpoint name | Architecture | Training task |
|---|---|---|
| ArGamba-dual | ArGamba (Jamba autoregressive) | NTP + CEP |
| ArGamba-seq_only | ArGamba (Jamba autoregressive) | NTP |
| ArGamba-cons_only | ArGamba (Jamba autoregressive) | CEP |
| BiGamba-dual | BiGamba (Mamba bidirectional) | MLM + MEM |
| BiGamba-seq_only | BiGamba (Mamba bidirectional) | MLM |
| BiGamba-cons_only | BiGamba (Mamba bidirectional) | MEM |
from transformers import AutoModel
model = AutoModel.from_pretrained(
"micanonsens/argamba-dual-step44000",
trust_remote_code=True
)
trust_remote_code=True is required.Repository: micanonsens/argamba-dual-step44000
The Gamba models are a family of DNA language models from the ArGamba paper that jointly model DNA sequence and evolutionary rate information.
This repository includes the model weights and code for the Jamba autoregressive variant model (ArGamba) trained to predict both the human genome sequence and the Zoonomia 241-mammalian alignment-derived PhyloP scores using both Next Token Prediction (NTP) and Current Evolutionary-rate Prediction (CEP) tasks. For more details, see the GitHub repo.
All Gamba family models have checkpoints available at 44,000 steps:
| Checkpoint name | Architecture | Training task |
|---|---|---|
| ArGamba-dual | ArGamba (Jamba autoregressive) | NTP + CEP |
| ArGamba-seq_only | ArGamba (Jamba autoregressive) | NTP |
| ArGamba-cons_only | ArGamba (Jamba autoregressive) | CEP |
| BiGamba-dual | BiGamba (Mamba bidirectional) | MLM + MEM |
| BiGamba-seq_only | BiGamba (Mamba bidirectional) | MLM |
| BiGamba-cons_only | BiGamba (Mamba bidirectional) | MEM |
from transformers import AutoModel
model = AutoModel.from_pretrained(
"micanonsens/argamba-dual-step44000",
trust_remote_code=True
)
trust_remote_code=True is required.