AI Gene Review

AI-assisted tool for reviewing and curating gene annotations with community feedback integration. This project provides a structured workflow for validating existing Gene Ontology (GO) annotations using AI-driven analysis combined with literature research, bioinformatics evidence, and crowdsourced expert feedback.

Overview

The AI Gene Review tool helps researchers and curators:

• Review existing GO annotations using strict, defined criteria
• Synthesize high-quality annotations from multiple evidence sources
• Fetch and organize gene data from UniProt and GOA databases
• Validate annotation files against LinkML schemas
• Manage references and supporting literature
• Collect community feedback through integrated voting and evaluation systems

Quick Start

Installation

1. Install uv for dependency management
2. Clone the repository and install dependencies:

   git clone https://github.com/cmungall/ai-gene-review.git
   cd ai-gene-review
   uv sync --group dev
   

Basic Usage

Fetch gene data:

uv run ai-gene-review fetch-gene human TP53

Validate a gene review file:

uv run ai-gene-review validate genes/human/TP53/TP53-ai-review.yaml

Fetch publications for a gene:

uv run ai-gene-review fetch-gene-pmids human TP53

Generate statistics report:

just stats                # Generate HTML report
just stats-open           # Generate and open in browser

Workflow Overview

1. Fetch Gene Data: Download UniProt records and GO annotations
2. Literature Research: Gather supporting publications and evidence
3. Create Review: Structure annotations using the YAML schema
4. Validate: Check against LinkML schema and best practices
5. Generate HTML: Render interactive web pages with voting capabilities
6. Collect Feedback: Community voting and expert evaluation forms
7. Iterate: Refine annotations based on validation results and community input

Key Features

• 🧬 Multi-organism support: Human, mouse, worm, and other model organisms
• 📚 Literature integration: Automatic PubMed citation fetching and caching
• ✅ Schema validation: LinkML-based validation for consistency
• 🛡️ Anti-hallucination validation: ID/label tuple checksums prevent AI fabrication of terms
• 🔄 Batch processing: Handle multiple genes efficiently
• 📊 Structured reviews: YAML-based gene annotation reviews
• 🔍 Evidence tracking: Detailed provenance and supporting text
• 🗳️ Community voting: Thumbs up/down feedback on AI decisions
• 📝 Expert evaluation: Detailed feedback forms for comprehensive gene review assessment
• 🎨 Interactive web interface: Rich HTML rendering with modern UI

Resources

Web Applications & Documentation

• Main Project Site: https://ai4curation.io/ai-gene-review (coming soon)
• Interactive Web App: Browse Gene Reviews - Search and explore gene annotation reviews
• Statistics Dashboard: Summary Statistics - Analytics and review metrics
• Evaluation Form: https://go.lbl.gov/gene-eval - Detailed expert feedback form
• Project Slides: Overview Presentation

Presentations

• ISMB 2026 (Function COSI): Caufield JH, Joachimiak MP, Mungall CJ. Agentic evaluation of AI function prediction pipelines. Washington, DC, 14 July 2026. Slides: doi:10.5281/zenodo.21810552

Documentation Pages

• Voting System Guide: Learn how to provide feedback on AI curation decisions
• Evaluation Form Guide: Comprehensive guide for detailed gene review evaluation
• GitHub Issues: Report bugs and feature requests

Gene Review Structure

Each gene review follows a structured YAML format containing:

• Gene metadata: UniProt ID, gene symbol, taxon information
• Description: Comprehensive summary of gene function
• References: Literature and bioinformatics sources
• Existing annotations: Review of current GO annotations with actions (ACCEPT, MODIFY, REMOVE, etc.)
• Core functions: Curated essential gene functions

Example structure:

id: Q9BRQ4
gene_symbol: CFAP300
taxon:
  id: NCBITaxon:9606
  label: Homo sapiens
description: >-
  CFAP300 is a cilium- and flagellum-specific protein...
existing_annotations:
  - term:
      id: GO:0005515
      label: protein binding
    action: MODIFY
    reason: "While evidence is strong, 'protein binding' is uninformative..."

Example Data

The repository includes example gene reviews for:

• Human: BRCA1, CFAP300, RBFOX3, TP53
• Mouse: Various examples
• Worm: lrx-1

Browse the genes/ directory to see complete examples.

Community Feedback System

The AI Gene Review project includes a comprehensive feedback system to improve AI curation through community input:

🗳️ Quick Voting System

Every gene review page includes thumbs up/down voting on:

• Individual annotation decisions (ACCEPT, REMOVE, MODIFY actions)
• Gene descriptions and summaries
• Core function definitions
• Suggested questions for experts
• Suggested experiments
• Documentation sections (Deep Research, Notes, Bioinformatics Results)
• Proposed new GO terms
• Pathway visualizations

Features:

• Anonymous voting with session-based tracking
• Instant visual feedback with vote persistence
• Rate limiting to prevent abuse
• Vote data collection via Google Apps Script

📝 Detailed Evaluation Form

For comprehensive feedback, use the evaluation form at https://go.lbl.gov/gene-eval:

• Structured assessment of annotation quality
• Pre-filled gene information when accessed from gene pages
• Multiple choice and free-text questions
• Expert-level feedback for improving AI curation

📹 Video Tutorial: Watch a step-by-step guide on performing evaluations (YouTube)

🎯 Feedback Integration

The feedback system enables:

• Data-driven improvements to AI curation algorithms
• Identification of problematic patterns in automated annotations
• Community validation of AI decisions
• Prioritization of genes needing expert attention

How to Provide Feedback

1. Quick feedback: Use 👍/👎 buttons on any gene review page
2. Detailed feedback: Click "📝 Provide Detailed Feedback" button or visit the evaluation form directly
3. Technical feedback: Submit issues and suggestions via GitHub Issues

Case Studies

PedH (Pseudomonas putida KT2440) - Lanthanide-Dependent Alcohol Dehydrogenase

The review of pedH revealed several important curation insights:

Key Discoveries

1. Lanthanide vs Calcium Dependency: PedH was incorrectly annotated with "calcium ion binding" (GO:0005509) when it actually requires lanthanide ions (La³⁺, Ce³⁺, Pr³⁺, Nd³⁺, Sm³⁺) for activity. This highlights the importance of reviewing automated annotations based on sequence similarity.

2. Cellular Localization Precision: Bioinformatics analysis confirmed PedH is a soluble periplasmic enzyme, not membrane-associated:

   • Signal peptide (aa 1-25) directs export, then is cleaved
   • No transmembrane regions in mature protein
   • Functions throughout periplasmic space, not just at membrane boundaries
   • Led to choosing GO:0042597 (periplasmic space) over GO:0030288 (outer membrane-bounded periplasmic space)

3. Dual Functional Roles: PedH serves both as:

   • Metabolic enzyme: Oxidizes alcohols in 2-phenylethanol degradation pathway
   • Regulatory sensor: Part of lanthanide-sensing system controlling gene expression via PedS2/PedR2 two-component system

4. Missing GO Terms Identified: The review revealed gaps in GO:

   • No term for "lanthanide ion binding" (distinct from transition metal binding)
   • No term for "lanthanide-dependent alcohol dehydrogenase activity"

Lessons for Curation

• Verify metal cofactors carefully - Don't assume calcium when other metals are possible
• Consider protein mobility - Soluble vs membrane-associated matters for localization terms
• Look for regulatory functions - Enzymes may have sensory/regulatory roles beyond catalysis
• Use bioinformatics to validate - Signal peptide and TM predictions can clarify localization

Anti-Hallucination Validation

The AI Gene Review system implements a robust anti-hallucination validation mechanism using ID/label tuple checksums to prevent AI systems from fabricating or misusing ontological terms.

How It Works

Every ontology term in the system requires both an id (semantic identifier) and label (human-readable name):

term:
  id: GO:0005515      # Ontology identifier
  label: protein binding  # Canonical label

Validation Process

The TermValidator performs multi-layer validation:

1. Format Validation: Ensures IDs follow proper CURIE patterns (PREFIX:NUMBER)
2. Existence Validation: Verifies terms exist in authoritative ontologies via OAK/OLS APIs
3. Label Matching: Cross-references provided labels against canonical ontology labels
4. Branch Validation: Ensures GO terms are in correct ontological branches (MF/BP/CC)
5. Obsolescence Checking: Flags outdated terms

Why This Prevents Hallucination

✅ Dual Verification: Both ID and label must be correct and consistent ✅ External Truth Source: Validates against authoritative ontologies (GO, HP, MONDO, etc.) ✅ Real-time Checking: Uses live API calls to catch fabricated terms ✅ Semantic Consistency: Ensures terms make sense in their context

Examples

# ❌ This would be caught as invalid
term:
  id: GO:0005515
  label: "DNA binding"  # Wrong label for GO:0005515

# ✅ This passes validation
term:
  id: GO:0005515  
  label: "protein binding"  # Correct canonical label

# ❌ This would be flagged as fabricated
term:
  id: GO:9999999
  label: "made up function"  # Non-existent term

Supported Ontologies

The validator supports 10+ major ontologies:

• GO: Gene Ontology (molecular functions, biological processes, cellular components)
• HP: Human Phenotype Ontology
• MONDO: Mondo Disease Ontology
• CL: Cell Ontology
• UBERON: Uberon Anatomy Ontology
• CHEBI: Chemical Entities of Biological Interest
• PR: Protein Ontology
• SO: Sequence Ontology
• PATO: Phenotype And Trait Ontology
• NCBITaxon: NCBI Taxonomy

This validation system represents a novel approach to preventing ontological hallucination in AI curation workflows and could serve as a model for other AI applications working with structured biological knowledge.

Annotation Rule Review

In addition to reviewing individual gene annotations, the AI Gene Review system supports systematic review of automated annotation rules used by UniProt (ARBA and UniRule systems). These rules generate millions of GO annotations across protein databases, making their quality critical for downstream research.

What are ARBA Rules?

ARBA (Association-Rule-Based Annotator) rules use combinations of:

• InterPro domains: Protein family signatures
• PANTHER families: Evolutionary classifications
• CATH FunFams: Functional families based on structure
• Taxonomic restrictions: Organism-specific constraints

When a protein matches all conditions in a rule, it receives the predicted GO annotation.

Why Review Rules?

Automated rules can produce systematic errors affecting thousands of proteins:

• Taxonomic over-annotation: Mammalian functions applied to fungi/plants
• GO term breadth: Using vague parent terms instead of specific functions
• Domain promiscuity: Structural domains with multiple functional contexts
• Catabolic/biosynthetic confusion: Grouping enzymes with opposite activities

Rule Review Workflow

# 1. Fetch rule data
just rules-fetch ARBA00089174

# 2. Run deep research (multiple providers)
just rules-deep-research-perplexity ARBA00089174
just rules-deep-research-falcon ARBA00089174

# 3. Create/update review YAML
# Reviews are stored in rules/arba/RULE_ID/RULE_ID-review.yaml

# 4. Validate the review
just rules-validate ARBA00089174

# 5. Validate all reviews
just rules-validate --all

Rule Review Structure

Each rule review evaluates:

Example Finding: False Positive Detection

When reviewing ARBA00089174 (adaptive thermogenesis), the system identified that S. pombe gene cps1 (a fungal carboxypeptidase) received this annotation despite adaptive thermogenesis being a mammalian-specific physiological process. This revealed the rule's Eukaryota scope was too broad and should be restricted to Mammalia.

Rule Review Files

rules/
  arba/
    ARBA00089174/
      ARBA00089174.enriched.json     # Raw rule data from UniProt
      ARBA00089174-review.yaml        # Comprehensive review
      ARBA00089174-deep-research-perplexity.md   # Literature research
      ARBA00089174-deep-research-falcon.md       # Additional research

Actions for Rules

• ACCEPT: Rule produces accurate annotations, no changes needed
• MODIFY: Rule has correct biological basis but needs refinement (taxonomic scope, GO term specificity, condition consolidation)
• REMOVE: Rule produces more false positives than valid annotations

Repository Structure

• genes/ - Gene review data organized by organism
  • human/, mouse/, worm/ - Species-specific gene directories
  • Each gene folder contains: YAML review, UniProt data, GO annotations, notes
• rules/ - Annotation rule reviews
  • arba/ - ARBA rule reviews organized by rule ID
  • Each rule folder contains: enriched JSON, review YAML, deep research files
• docs/ - MkDocs-managed documentation
• src/ai_gene_review/ - Core Python package
  • cli.py - Command-line interface
  • schema/ - LinkML schema definitions
  • etl/ - Data extraction and loading modules
• tests/ - Python tests and example data
• publications/ - Cached PubMed articles

Developer Tools

This project uses just command runner for development tasks.

Available commands:

just --list           # Show all available commands
just test             # Run tests, type checking, and linting
just format           # Run code formatting checks
just install          # Install project dependencies

CLI Commands:

uv run ai-gene-review --help                    # Show CLI help
uv run ai-gene-review fetch-gene human BRCA1    # Fetch gene data
uv run ai-gene-review validate <yaml-file>      # Validate review file
uv run ai-gene-review batch-fetch <input-file>  # Process multiple genes

HTML Rendering:

just render human BRCA1        # Render single gene to HTML
just render-all                # Render all gene reviews to HTML
just stage-pages               # Assemble existing generated output in _site/
just build-pages               # Render and assemble the complete publication tree
python -m ai_gene_review.render --all genes/    # Alternative rendering command

stage-pages assembles the GitHub Pages publication artifact. It preserves current public URL paths, writes an ignored _site/ directory, and reports the uncompressed publication size. Cleanup is restricted to the repository's _site/ directory, and the root is verified with Git before cleanup. The CLI always uses <repo-root>/_site. With PAGES_ARTIFACT_DEPLOY_ENABLED=true, the live site uses the validated artifact built from main. In this mode no regeneration PR is created: committed generated HTML is not refreshed and is not the live site. The repository's Pages source must separately be set to GitHub Actions. Artifact failures preserve the previous deployment and fail publication status.

The published homepage shows Site built in UTC and links to the originating Actions run. _site/build-info.json records that timestamp, the checked-out commit and the run URL. These describe the artifact: redeploying an older artifact preserves its original build date. Project HTML separately shows Source last changed, using the last commit touching its Markdown source, with links to the commit and that exact source revision. Stamps omit names, emails and commit messages. Source history follows the published branch's first parents: merged edits use the merge commit that introduced them on that branch, while discarded side-branch edits do not affect the stamp. History and the gene symbol index are each collected once per rendering batch. Local edits are marked; incomplete shallow history is labelled instead of assigning a misleading date. Standalone Markdown exports without Git history can still be rendered, without a source stamp. The maintained index.html must contain exactly one <!-- site-build-info --> marker; staging fails if it is missing or duplicated, and audits the injected link along with the other published links.

The legacy regeneration PR path is retained only when artifact publication is disabled; artifact-build failures do not block that path. It still enforces Git's 100 MiB blob limit and cannot handle the current browser payload, so disabling the flag is not a working rollback. Use the verified-artifact recovery workflow below instead. Disabling the flag does not change the repository's Pages source.

For disposable builds use just deploy-browser pages, just update-browser-data pages, just deploy-predictions-browser pages, or just deploy-propagation-browser pages. just build-pages selects this target for all three browsers. The default git target keeps the blob guard for output intended to be committed; BROWSER_TARGET=pages also selects artifact mode. Staging compresses the main annotation payload, but prediction and propagation payloads are still uncompressed. Total artifact limits do not guarantee browser memory or load-time performance.

Staging also follows local links from published HTML and CSS, plus literal JavaScript fetch()/import() URLs, copying reports, notes, images, and downloads at their existing paths. Downloaded scientific data and images keep their original bytes. Known source-relative HTML links and missing site prefixes are repaired against existing files; linked source directories get browsable indexes. In staged gene, project, and module HTML, exact static CSS/JavaScript blocks from the maintained templates are replaced with relative links to content-addressed shared files in _pages-assets/. This preserves page URLs, script order, and CSS contents while avoiding thousands of identical copies. Source HTML remains standalone and unchanged. Unknown/dynamic blocks, URL-relative CSS, and documents with a <base> element remain inline. The manifest reports net savings as shared_asset_bytes_saved. Dynamic JavaScript URLs still require browser checks. Deleted reviews' orphan HTML remains excluded. The manifest's linked_source_files_not_staged and corresponding byte count specifically report these excluded orphan review pages. Independently linked notes, reports, and images remain publishable even without a review YAML; removing a review alone is not a request to unpublish its supporting research.

Staged gene HTML is compacted while retaining preformatted text and inline spacing. The main review stays ordinary HTML. Supporting research, reference, documentation, and raw-YAML panels are stored losslessly in _pages-content/ and restored on page load. Each panel also offers a compressed HTML download if JavaScript is unavailable or loading fails. The annotation browser retains its data.js URL and ready event, with its complete columnar data loaded from data.json.gz. These features use the browser's native gzip decompressor; there is no truncation of research text or annotation rows. Net savings are reported as content_compaction_bytes_saved. Loaders accept both raw gzip and responses already decoded by the HTTP layer. Restoration requires JavaScript and DecompressionStream; without them, the primary gene review and section downloads remain available, but the annotation browser cannot initialize. Supporting panels become searchable in-page after loading; crawlers that do not execute JavaScript see only the primary HTML and links to the source material. Staging audits the emitted asset URLs again after compaction.

Run the real-browser transport regression (with Chromium installed) using: uv run pytest -m integration tests/test_pages_loaders_browser.py. It covers servers both with and without Content-Encoding: gzip.

Preview the artifact through HTTP so compressed supporting files can load. For example, mount it at its real URL prefix:

preview_dir=$(mktemp -d)
ln -s "$PWD/_site" "$preview_dir/ai-gene-review"
python3 -m http.server 8000 --directory "$preview_dir"
# Open http://localhost:8000/ai-gene-review/

Research-provider metadata can reference files that were never archived. These are explicitly labelled not archived, retaining their descriptions instead of presenting broken download/image links. The full list remains visible in unavailable_source_artifact_paths in the manifest. This is distinct from an orphaned review HTML omitted from the artifact, which is reported separately.

The publication boundary is reachable, non-hidden files inside this repository at the existing site paths. Navigation and dependencies are followed transitively, without extension filters, depth limits, or per-file truncation that would break existing links. Linking a directory publishes its reachable children too; local runtime caches are excluded. Oversized artifacts are reported and blocked, not silently pruned. If a referenced rendered document is absent but its Markdown source exists, the link serves that source rather than a nonexistent HTML page. The separate MkDocs build on gh-pages is not part of this publication artifact; linked docs such as the subtraction report use their repository Markdown source. Every rebuilt artifact must meet the budget before publication.

Publication assumes the contents of main are accepted. Biological validation (including module GO labels and obsoletion checks) runs independently in PR CI and the weekly validation workflow, not in build-pages or daily publication. Upstream ontology drift with no repository changes is therefore detected weekly, not daily. This is an accepted tradeoff: publishing updates does not wait for biological revalidation; an independent validation run can be dispatched when needed. Missing supporting links, suspected site-prefix errors, and links to orphaned review HTML remain warnings with full diagnostic lists in pages-diagnostics; they do not freeze the whole site. Rendering failures, missing required build outputs, invalid archives/checksums, and deployment failures remain blocking. This change does not add incremental rendering or fallback to older pages.

The deployment job is disabled unless PAGES_ARTIFACT_DEPLOY_ENABLED=true is set in repository Actions variables. It requires a successful upload, at most site content and a separately measured GNU tar below 10,000,000,000 bytes (including headers and padding), and a verified archive checksum. The CLI and CI use the same manifest deployable decision and size_budget_bytes. broken_local_links counts distinct missing paths; broken_local_link_paths lists them for diagnosis. off_base_path_links counts same-host URLs outside /ai-gene-review/ that match a safe repository file (including an existing Markdown source for an HTML target); off_base_path_urls lists those suspected prefix errors. Links to the umbrella homepage (including /index.html) are allowed. Unmatched URLs outside the site prefix may belong to other projects and are not checked. Diagnostic lists are retained in full for machine processing rather than truncated; the workflow summary shows only counts. This static audit does not guarantee dynamically constructed JavaScript URLs or fragment anchors work.

The live site uses GitHub Actions as its Pages source and PAGES_ARTIFACT_DEPLOY_ENABLED=true. Build and deploy site (generate-pages.yaml) builds and publishes from main; these settings are required for deployment.

Deploy existing Pages artifact (deploy-existing-pages.yaml) is a manual shortcut when rendering, validation, and uploads succeeded but deployment failed or was skipped, or when intentionally redeploying a known-good build. It is not part of normal daily publishing and does not restore lost data. Select a completed build run ID and dispatch it on main:

gh workflow run deploy-existing-pages.yaml --ref main -f source_run_id=RUN_ID

It verifies the original build and artifact, then publishes that exact snapshot, not current main. Choosing an older run publishes older content. For new content, failed builds, an expired artifact, or an older manifest marked nondeployable under the former link policy, run Build and deploy site instead. The Pages archive is retained for 3 days (diagnostics for 7). Only validated default-branch builds are accepted; checks include successful rendering and uploads, artifact provenance, actual archive size, and its recorded SHA-256. The former self-imposed 1 GB site / 1 GiB tar gates are removed. Both paths retain a strict less-than-10-GB ceiling matching the action's absolute archive cutoff. GitHub officially supports only 1 GB sites: larger deployments are an explicit temporary operational choice, not a promise of support or scalability. When deployment is enabled, a skipped or failed deployment fails the workflow.

To retry a retained build blocked solely by the old 1 GB policy, supply -f legacy_size_sha256=SHA256 to the manual workflow, using the SHA-256 of its original downloaded artifact.tar. This explicit override rechecks provenance and actual archive-size gates and verifies the supplied checksum. It does not permit failed integrity checks or rebuild the site. The manifest reports estimated archive_bytes and archive_size_budget_bytes. After upload, diagnostics record the actual archive_actual_bytes and archive_sha256; deployment requires that check to succeed. Recovery verifies that checksum, not exact equality with the estimate. Older validated builds without a checksum declaration remain recoverable using the trusted run/artifact provenance and the actual archive size.

Once enabled, the artifact built from the checked-out source on main is authoritative for the live site. Generated-file commits, App tokens, reviews and PR merges are skipped entirely in this mode. Git-tracked output is not updated.

The Build and deploy site workflow runs daily at 08:23 UTC and can also be started with GitHub Actions' Run workflow button. Each run rebuilds the full site; merged content appears after the next successful deployment. Generated-files PRs are only attempted in the disabled-artifact legacy mode, subject to the Git size limit described above. Agent cron profiles do not control this publication schedule. Manual runs wait for an active build to finish instead of cancelling it. Gene review validation remains in PR CI and the weekly full validation workflow.

Module authors: see the symbol-label rule for family representative labels, validation warnings, and accession-based reasoning.

Contributing

See CONTRIBUTING.md for detailed contribution guidelines including:

• Code of conduct and best practices
• Understanding LinkML schemas
• Pull request workflow
• Development setup

Credits

This project uses the template monarch-project-copier