ESM-2 Protein Pipeline

DIMER-oriented pipeline for ESM-2 150M (facebook/esm2_t30_150M_UR50D), pinned to an immutable Hugging Face revision. The repository exposes mean-pooled protein sequence embeddings, a labelled-dataset contract with explicit ceilings, bounded gradient fine-tuning of a sequence-classification head, held-out classification metrics with trivial baselines, and a safetensors adapter artifact that is digest-verified before it is loaded.

Upstream alignment

• Model: facebook/esm2_t30_150M_UR50D
• Revision: a695f6045e2e32885fa60af20c13cb35398ce30c
• Upstream weight license: MIT
• Upstream task: protein language modelling (masked LM); this repository uses the encoder for sequence embeddings and sequence classification
• Repository adaptation: E2E (bounded fine-tuning of the classification head plus the last n encoder layers, with a portable safetensors adapter)

Quick start

from esm2_protein_pipeline import ESM2Pipeline, generate_sample_dataset, split_dataset

pipe = ESM2Pipeline.from_pretrained()            # verifies the snapshot under weights/ first
emb = pipe.embed(["MKTAYIAKQRQISFVKSHFSRQLEERLGLIEVQ"])
print(emb["dimension"], len(emb["embeddings"][0]))   # 640 640

splits = split_dataset(generate_sample_dataset())
pipe.adapt(splits["train"], splits["validation"])    # bounded AdamW fine-tuning
print(pipe.evaluate(splits["test"])["accuracy"])
print(pipe.classify(["MKTAYIAKQRQISFVKSHFSRQLEERLGLIEVQ"])["predictions"][0]["label"])

embed() and classify() take 1..64 sequences (MAX_SEQUENCES_PER_CALL) of at most 1,022 residues each (MAX_RESIDUES, the 1,024-token training window minus <cls>/<eos>), as uppercase one-letter strings over ACDEFGHIKLMNPQRSTVWY plus the ambiguity codes BUZOX; anything else — lowercase, gaps, whitespace — is rejected with an error naming the offending characters rather than silently mapped to <unk>. classify() requires a prior adapt() or from_artifact(). Datasets are {id, sequence, label} records: at least 12 rows and 3 per class, at most 5,000 rows and 20 classes, unique ids and unique sequences.

Weights layout

weights/esm2-t30-150m-ur50d/   config.json  model.safetensors  vocab.txt  tokenizer_config.json
                               special_tokens_map.json  README.md  dimer-base-manifest.json

from_pretrained() calls stage_missing_files() then verify_snapshot() (byte size + SHA-256 of every manifest entry; staging fetches only absent entries, only at the pinned revision, and only with allow_download=True), then loads with local_files_only=True, trust_remote_code=False and use_safetensors=True. The upstream pytorch_model.bin and tf_model.h5 are deliberately absent from the manifest and are never loaded. .safetensors files are git-ignored; the Git repository does not vendor the checkpoint. See docs/WEIGHTS.md.

Adapter artifacts

save_artifact(dir) writes adapter.safetensors (the trained tensors only — head, final layer norm and the unfrozen encoder layers; about 41 MB with the default two layers) plus a manifest.json recording the artifact format, the exact base model id and revision, the class order, the tensor names, the file size and SHA-256, and the full adaptation configuration. ESM2Pipeline.from_artifact(dir) re-verifies the base snapshot, then checks the artifact manifest, the base identity and every digest before deserialising, and refuses any tensor the base architecture does not have.

Tests

pip install -e . --no-deps
pytest -q -o addopts= tests

Tests are offline: injected backends and temporary manifests, never the weights.

Tutorial

tutorials/esm2_protein_colab.ipynb is declared E2E and is standalone (DIMER Notebook Specification 2.2 §4): it is generated by tools/build_notebook.py from tools/notebook_template.py and embeds the 3 package modules (pipeline.py, samples.py, metrics.py) verbatim in dependency order, the pinned model identity, the snapshot manifest and the exact runtime pins, so the exported .ipynb keeps working without this repository being reachable — no clone, no repository install, no repository import on its primary path. Do not edit it by hand; change the package or the template and regenerate (python tools/build_notebook.py; --check is enforced by the validator and CI). Its first cell builds an isolated Python 3.12.12 environment from the hash-locked tutorials/requirements-colab.lock.txt (manylinux wheels, so Linux x86_64 runtimes only: Colab, Kaggle or Linux Jupyter) and routes the remaining cells to it, so a one-pass Run all needs no restart. Its default path generates and validates a deterministic 96-sequence dataset whose classes are defined by a residue-order rule, splits it 56/20/20 stratified by class, stages and digest-verifies the pinned snapshot, extracts mean-pooled embeddings, measures four baselines (majority class, hydrophobic fraction, 20-residue composition nearest centroid, and the longest-hydrophobic-run rule that defines the classes), runs a 4-epoch bounded fine-tuning, evaluates accuracy/macro-F1/AUROC on the held-out test split, classifies six freshly generated sequences, exports the safetensors adapter and verifies reload parity. The fine-tuned model ties the one-line order rule on the synthetic test split, so the run shows that the adaptation contract works, not what pretraining contributes. BYOD (CSV/JSON/JSONL, by upload or BYOD_PATH) is optional and gated off by default. See tutorials/README.md.

Release status

Candidate. The notebook was revised after its Notebook Review Framework v1 review (isolated runtime, honest baselines, BYOD split contract, guided layer); no hosted run of the revised notebook is recorded yet. The previous blob's 2026-09-18 Kaggle Tesla T4 run passed only after a manual restart, which is not a one-pass Run all and not promotion evidence. The execution records and the promotion gate are in docs/release-verification.md.

Licensing

• Upstream weights: MIT (facebook/esm2_t30_150M_UR50D), redistributed unmodified from the pinned revision.
• This repository's code and documentation: Apache-2.0 (LICENSE).
• The upstream licence governs your use of the weights, including commercial use and redistribution; this repository grants no rights beyond it.

AI Assistance Disclosure

This repository’s code and accompanying documentation were developed with generative AI assistance for code development and technical writing under maintainer direction. The maintainer remains responsible for reviewing the implementation, validating results, and making release decisions. AI assistance does not constitute independent verification, provider endorsement, or release approval.