Gaius-Augustus/Paludamentum

Paludamentum is the coat of the emperor - evidence processing pipeline

Python

11

173 commits

updated Oct 11, 2026

See the code

README

Paludamentum

Paludamentum logo
$\color{lightgray}{\tiny\textsf{Logo generated by Gemini}}$

Authors: Lars Gabriel & Katharina J. Hoff

The paludamentum was the cloak worn by Roman emperors and commanders. This repository is the cloak around the emperors of the Gaius-Augustus gene finder family: a Nextflow pipeline that prepares extrinsic evidence (proteins, RNA-Seq, Iso-Seq), derives high-confidence genes from it, and integrates them with the ab initio predictions of the deep learning gene finder Tiberius or Vipsania.

Table of contents

What the pipeline does

Paludamentum overview: the inputs genome, proteins, short reads and Iso-Seq reads; the six processing steps ab initio prediction, protein evidence, transcript evidence, high-confidence genes, integration, and post-processing and quality control; the output files

The numbers in the figure are the steps below.

  1. Ab initio prediction. The genome is split into chunks, the gene finder runs on each chunk on a GPU, and the chunk predictions are merged. If you already have a Tiberius or Vipsania prediction, pass it with --result and this step is skipped, see Existing predictions.
  2. Protein evidence. Proteins (your FASTA files and/or OrthoDB v12 partitions) are aligned with miniprot, scored with miniprot-boundary-scorer, and converted to hints with miniprothint. For very large protein databases the ab initio proteins are used to select the most relevant source species first (DIAMOND).
  3. Transcript evidence. Short reads are aligned with HISAT2, Iso-Seq reads with minimap2. Libraries with a low alignment rate are dropped. Transcripts are assembled with StringTie, and intron hints are extracted. If you already have a StringTie assembly, pass it with --stringtie instead of the reads, see Existing StringTie assemblies.
  4. High-confidence genes. Assembled transcripts get ORFs from TD2 (or TransDecoder 5.7.1 with transdecoder: td1). ORFs that are supported by protein homology (DIAMOND) and by the scored protein alignments become the high-confidence (HC) gene set; a comparison of the ORF finders is in docs/orf_finder_comparison.md.
  5. Integration. The HC genes are merged with the ab initio predictions into the final annotation.
  6. Post-processing and quality control. Transcripts with an internal stop codon or a broken CDS are removed, transcripts without UTRs get UTRs from the StringTie assemblies, and the annotation is written as GFF3 (checked with GenomeTools), GTF, protein and CDS sequences. Optional: completeness with compleasm and BUSCO (--busco_lineage), OMArk, a comparison with a reference annotation (--reference_annotation), rRNA, tRNA, Rfam and lncRNA genes (--ncrna) and GO terms with FANTASIA-Lite (--fantasia, GPU). report.html sums it up. See docs/postprocessing.md.

Steps 4 and 5 run in one of three ways. The clade of the gene finder model and the inputs decide which (conf/hc_genes.yaml):

HC genes (steps 4 and 5)ForProteinsDescribed in
TransDecoder flow: TD2 ORFs supported by the proteinsall other cladesrequired with readsstep 4 above
Drusilla flow with proteins: Drusilla ORFs, LightGBM filter and hint rescue of the ab initio genesvertebrates, mammalsgivenDrusilla flow
Drusilla flow without proteins: Drusilla ORFs merged with all ab initio genesinsects (recommended); any clade with a Drusilla modelnot neededInsects

Without any evidence input the pipeline runs steps 1 and 6 only. That is useful to parallelize a gene finder over several GPUs.

Step 6 in detail: the color of a step says which data it needs, the label on it which option switches it on.

Post-processing and quality control: the annotation runs through the sanity filter, gets UTRs from the StringTie assemblies in modes with transcripts, and is written as GFF3, GTF, proteins and CDS; gene set statistics always run, evidence support in evidence modes, completeness, OMArk, gffcompare, ncRNA and GO terms with their options; report.html sums up all steps

Requirements

The pipeline runs with Singularity (or Apptainer): the gene finders and all tools run in container images that the pipeline pulls itself. You do not install HISAT2, miniprot, StringTie, Tiberius, Vipsania or any other tool.

You need:

  • Singularity or Apptainer, on the machine that launches the pipeline and on the nodes that run its tasks
  • Nextflow 25.04 or newer
  • Java 17 or newer (required by Nextflow)
  • Python 3.9 or newer
  • an NVIDIA GPU on the nodes that run the gene finder step, up to and including the Blackwell generation (for example A100 or RTX PRO 6000)

Installation

git clone --recursive https://github.com/Gaius-Augustus/Paludamentum
cd Paludamentum
pip install -e .

This installs the launcher paludamentum. --recursive checks out the submodules tiberius/, vipsania/ and drusilla/; in an existing clone run git submodule update --init after git pull.

The first run pulls the images into ~/.cache/paludamentum/singularity, later runs reuse them. Each gene finder image is about 9.4 GB, the image of the evidence tools 0.6 GB. The images of BUSCO, FEELnc, OMArk and FANTASIA-Lite are pulled only when their steps are switched on. On a cluster this directory must be on a file system that the compute nodes can read; set NXF_SINGULARITY_CACHEDIR to use another directory.

Quick start

# Tiberius, ab initio only, parallelized over GPUs by Nextflow
paludamentum --nf_config slurm_generic --genome genome.fa --model_cfg eudicotyledons

# Tiberius with evidence, inputs from a params file
paludamentum --params_yaml params.yaml --nf_config slurm_generic

# Vipsania with evidence, inputs on the command line
paludamentum --genefinder vipsania --nf_config local --genome genome.fa --model Fungi \
    --proteins proteins.faa --rnaseq_paired "/abs/path/rnaseq/*_{1,2}.fastq.gz"

# Tiberius with proteins and an existing StringTie assembly instead of reads
paludamentum --nf_config local --genome genome.fa --model_cfg eudicotyledons \
    --proteins proteins.faa --stringtie stringtie.gtf

# Evidence for an existing Tiberius prediction (GFF3 or GTF); the gene finder does not run
paludamentum --nf_config local --genome genome.fa --result tiberius.gff3 \
    --proteins proteins.faa --rnaseq_paired "/abs/path/rnaseq/*_{1,2}.fastq.gz"

--nf_config takes a path or the name of a config in conf/ (local, slurm_generic, ...); local sizes the tasks to the machine it runs on, slurm_generic needs your GPU partition (see Containers and HPC). Evidence can be given on the command line (--proteins, --odb12Partitions, --rnaseq_paired, --rnaseq_single, --isoseq, --stringtie, --rnaseq_sra_paired, --rnaseq_sra_single, --isoseq_sra) or in the params file; command line values override the file. Useful options: --dry_run writes the params file and validates inputs and executables without starting Nextflow, --resume continues a previous run, --work_dir sets the Nextflow work directory, --cleanup deletes the task directories of the run from the work directory once it has finished successfully (see docs/hpc.md). Arguments after -- go to Nextflow. paludamentum --help lists everything.

Params file

Start from conf/parameters.yaml. Minimal example:

genome: /abs/path/genome.fa
proteins: /abs/path/proteins.faa
rnaseq_paired: "/abs/path/rnaseq/*_{1,2}.fastq.gz"
outdir: results
tiberius:
  run: true
  model_cfg: eudicotyledons

The launcher expands ~ and environment variables and resolves relative paths against the launch directory, not against the params file; it writes the merged parameters of a run with absolute paths to <outdir>/params.yaml. Paths must not contain spaces. All parameters are documented in docs/parameters.md.

Inputs and modes

ParameterContent
genomegenome FASTA, optionally gzipped (required)
tiberius.result, vipsania.result (--result)existing prediction of the gene finder, GFF3 or GTF, used instead of running it, see Existing predictions
proteinsone protein FASTA or a list; lists are concatenated
odb12PartitionsOrthoDB v12 partitions to download: Metazoa, Vertebrata, Viridiplantae, Arthropoda, Fungi, Alveolata, Stramenopiles, Amoebozoa, Euglenozoa, Eukaryota
rnaseq_paired, rnaseq_singleshort read FASTQ files (glob or list)
rnaseq_bamaligned short reads, for example from VARUS
stringtieStringTie assembly (GTF) made outside the pipeline, one file, a list or a glob; instead of or in addition to reads, see Existing StringTie assemblies
rnaseq_varus, isoseq_varus, mixed_varuspyVARUS output directories, which bring their StringTie assembly and intron hints, see docs/parameters.md
isoseqIso-Seq FASTQ files
rnaseq_sra_paired, rnaseq_sra_single, isoseq_sraSRA run accessions, downloaded by the pipeline; an accession whose download fails twice is dropped with an Error is ignored note, and -resume tries it again
keep_downloadscopy the reads downloaded from SRA to sra_downloads/ in outdir; default false, they stay in the work directory only
min_alignment_ratelibraries whose alignment rate (percent mapped) is below this value are dropped; default 80

The mode is inferred from the inputs and can be forced with mode (--mode). A forced mode without its inputs is an error, and so is transcript evidence without proteins outside the Drusilla flow:

ModeInputs
abinitio (historic name: tiberius)genome only
proteinsproteins
rnaseqshort reads or a StringTie assembly, and proteins
isoseqIso-Seq, and proteins
mixedshort reads and Iso-Seq, and proteins

The TransDecoder HC genes need protein evidence. The Drusilla flow does not: for a clade with a Drusilla model in conf/hc_genes.yaml (vertebrates, mammals, insects) the proteins of the modes rnaseq, isoseq and mixed are optional.

Existing StringTie assemblies

If the transcripts are already assembled, pass the StringTie GTF with --stringtie (stringtie in the params file). The pipeline then neither maps reads nor runs StringTie for them; the assembly goes straight to the HC gene step. Proteins are still required, except in the Drusilla flow.

paludamentum --nf_config local --genome genome.fa --model_cfg eudicotyledons \
    --proteins proteins.faa --stringtie stringtie.gtf
stringtie: /abs/path/stringtie.gtf      # or a list, or a glob such as "/abs/path/*.gtf"

The assembly may come from short reads, Iso-Seq or both (stringtie --mix), and it must be an assembly of this genome FASTA (same sequence names). It combines with --result: an existing prediction and an existing assembly need no GPU, no mapping and no assembly step. Notes:

  • Several assemblies can be given, also together with reads; they are merged with the assemblies that the pipeline makes from the reads.
  • With a vertebrate, mammal or insect gene finder model (a clade with hc: drusilla in conf/hc_genes.yaml) the Drusilla flow runs, which filters transcripts by the coverage and TPM that StringTie writes. It takes exactly one assembly of all reads and no other RNA-Seq or Iso-Seq input, and the GTF must come from stringtie on the reads, not from stringtie --merge (which drops coverage and TPM). Set drusilla.run: false to merge several assemblies in the TransDecoder flow instead.
  • A run with only an assembly is reported as mode rnaseq, whatever reads the assembly was made from. hintsfile.gff then has protein hints only (none without proteins); intron hints from reads do not influence the annotation.

Outputs

All files are written to outdir (default <genefinder>_results). <tool> is tiberius or vipsania.

FileContent
<tool>_evidence.gff3final annotation: ab initio predictions merged with HC genes, sanity filtered, with UTRs in modes with transcripts
<tool>_evidence.gtfthe final annotation as GTF
<tool>_evidence_proteins.faprotein sequences of all transcripts of the final annotation
<tool>_evidence_cds.facoding sequences of the final annotation
<tool>_ab_initio.gff3mode abinitio only: the final annotation (sanity filtered, with .gtf, _proteins.fa and _cds.fa as above)
<tool>_evidence_with_ncRNA.gff3final annotation plus rRNA, tRNA, Rfam and lncRNA genes (ncrna.run)
<tool>_evidence_go.gff3, <tool>_evidence_with_ncRNA_go.gff3the same with GO terms (fantasia.run)
report.htmlmethods, gene set statistics, completeness, evidence support, sanity filter, UTRs, ncRNA, versions and references of the run
qc/sanity filter and UTR reports, gene set statistics and plots, hint support, completeness, OMArk, gffcompare, FANTASIA, software versions, see docs/postprocessing.md
ncrna/rRNA, tRNA, Rfam and lncRNA annotations (ncrna.run)
intermediate/<tool>_ab_initio.gff3ab initio predictions before the sanity filter, in every mode; the file to give to --result of a later run
intermediate/<tool>_merged.gff3ab initio predictions merged with the HC genes, before the sanity filter
intermediate/<stem>_sanity_filtered.gff3the annotation after the sanity filter, before the UTRs
intermediate/hc.gff3high-confidence genes derived from the evidence (TransDecoder)
intermediate/drusilla_orfs.gtfDrusilla ORFs, the HC genes of the Drusilla flow
intermediate/<tool>_lgb_filtered.gtfab initio predictions kept by the LightGBM filter (Drusilla flow)
intermediate/<tool>_lgb_scores.tsvLightGBM class probabilities of all ab initio transcripts (Drusilla flow)
intermediate/hint_rescue.gtfab initio genes predicted again with protein hints (Drusilla flow; missing if the rescue was skipped)
hintsfile.gffprotein, RNA-Seq and Iso-Seq hints
sra_downloads/reads downloaded from SRA, with keep_downloads: true only
params.yamlthe merged parameters of this run, written by the launcher
citations.mdreferences of the gene finder and the tools that this run used, see Citation
methods.mda methods section for this run: the steps that ran, their main parameters and inline citations of citations.md, see Citation

Nextflow's timeline, trace and report files are written as well.

Gene finders

--genefinder tiberius|vipsania selects the gene finder. Without it, the launcher takes genefinder from the params file, else the block with run: true, else Vipsania if --model is given, else Tiberius. Only one gene finder runs per pipeline run.

Gene finderModelParameters
Tiberius--model_cfg, a name in tiberius/model_cfg of the submodule (for example eudicotyledons) or a pathdocs/parameters.md
Vipsania--model, a clade name (for example Fungi) or a model iddocs/vipsania.md

Model weights are downloaded once on the submitting host, so the GPU nodes need no internet; model_dir takes weights that you downloaded yourself. Vipsania finetuning on the target genome is off by default (--finetune).

Existing predictions

If the genome is already annotated with Tiberius or Vipsania, give that file to the pipeline instead of predicting again. The file may be the GFF3 or the GTF that tiberius.py or vipsania annotate wrote, or the intermediate/<tool>_ab_initio.gff3 of an earlier Paludamentum run (the predictions before the sanity filter; <tool>_ab_initio.gff3 in the output directory of mode abinitio is the filtered final annotation). It must belong to the genome FASTA of this run (same sequence names).

# Tiberius prediction
paludamentum --nf_config local --genome genome.fa --result tiberius.gff3 \
    --proteins proteins.faa --rnaseq_paired "/abs/path/rnaseq/*_{1,2}.fastq.gz"

# Vipsania prediction
paludamentum --genefinder vipsania --nf_config local --genome genome.fa --result vipsania.gff \
    --proteins proteins.faa --isoseq isoseq.fastq.gz

In a params file the option is result in the block of the gene finder:

tiberius:
  run: true
  result: /abs/path/tiberius.gff3

All other steps run as usual, and the outputs are the same as in a run that predicts itself. No model option is needed, and the gene finder does not run. Notes:

  • --genefinder names the gene finder that made the file (default: Tiberius). It sets the names of the output files (tiberius_*, vipsania_*).
  • For a prediction of a vertebrate or insect model, name the model as well (--model_cfg vertebrates for Tiberius, --model Vertebrata or --model Insecta for Vipsania). The model is not run; it tells the pipeline to use the Drusilla flow. Without it, runs with transcripts use the TransDecoder HC genes. The hint rescue of the Drusilla flow runs Tiberius on a GPU; all other steps of a run with --result need no GPU.
  • The pipeline renames the genes and transcripts of the file (gene_000001, gene_000001.t1, ...) and reads its exon, CDS and UTR features.

Drusilla flow

For vertebrate, mammal and insect gene finder models (clades with hc: drusilla in conf/hc_genes.yaml), runs with transcripts take another route through steps 4 and 5 of the overview: Drusilla ORFs replace the TransDecoder HC genes. With protein evidence, vertebrates and mammals also get a LightGBM filter of the ab initio predictions, and Tiberius predicts partial genes again with protein hints. The gene-level F1 is 80.70 on Takifugu rubripes with the Tiberius vertebrates model (Paludamentum 0.5.0, gffcompare 0.12.10) and 79.71 on Bos taurus with mammalia_softmasking_v2 (Paludamentum 0.4.0). The flow is switched on automatically (drusilla.run: auto). Conditions, steps, parameters and known issues are in docs/drusilla_flow.md.

The figure shows steps 3 to 5 of the overview for vertebrates and mammals with proteins; its numbers count the steps of this flow.

Drusilla flow: StringTie assembly, transcript filter, Drusilla ORFs and codon fix give the high-confidence genes; LightGBM filter and hint rescue treat the ab initio genes; both are merged into the final annotation

Insects

For insects, Paludamentum recommends RNA-Seq or Iso-Seq reads, Vipsania with the model Insecta fine-tuned on the genome, and no protein evidence:

paludamentum --nf_config local --genome genome.fa --model Insecta --finetune \
    --rnaseq_bam rnaseq.bam

The run then goes through the steps of the overview as follows; the numbers in the figure are those of the overview.

Insect flow: the reads are aligned and assembled with StringTie into one transcript set (step 3), which is filtered, and Drusilla predicts its ORFs with the insect model (step 4); Vipsania, fine-tuned on the genome, predicts the ab initio genes (step 1); all ab initio genes are merged with the ORFs (step 5) and post-processed into vipsania_evidence.gff3 (step 6); step 2, protein evidence, does not run; other choices get a warning

  1. Ab initio prediction. Vipsania fine-tunes its Insecta model on the genome, then annotates the genome with it. This is one GPU task on the whole genome (about 4 h on an A100), not one task per genome chunk.
  2. Protein evidence does not run. With it go the stop and start codon fix of the ORFs, the LightGBM filter and the hint rescue, which lowered the accuracy on insects.
  3. Transcript evidence. Short reads are aligned with HISAT2, Iso-Seq reads with minimap2 (a BAM, a pyVARUS directory or a StringTie assembly is used as given), and all reads are assembled into one StringTie transcript set.
  4. High-confidence genes. Transcripts are kept if they are at least 300 nt long, with a coverage of at least 3 and a TPM of at least 1. Drusilla, with its insect model, predicts the ORFs of the kept transcripts, and isoforms whose CDS is part of another one are collapsed. These ORFs are the HC genes (intermediate/drusilla_orfs.gtf).
  5. Integration. All ab initio genes, without any filter, are merged with the HC genes; transcripts with identical CDS are collapsed.
  6. Post-processing and quality control as for every run; the final annotation is vipsania_evidence.gff3.

On 11 insect genomes this flow reached a mean gene F1 of 75.2, as BRAKER3 with proteins (75.4), and more than Drusilla with Tiberius (66.0), Vipsania alone (60.4) or Tiberius alone (57.6) (evaluation of Lars Gabriel, gffcompare 0.12.10, see docs/drusilla_flow.md). On Drosophila melanogaster and Bombyx mori Paludamentum gives the same as his standalone merge, 84.88 and 74.12 against 84.49 and 73.85.

An insect run that differs from this flow still runs, but the Nextflow log warns and names each difference: Tiberius instead of Vipsania (also the Tiberius insecta and lepidoptera models, which run the Drusilla flow too), Vipsania without fine-tuning, no RNA-Seq or Iso-Seq, or protein evidence (then the ORFs get the codon fix, but there is still no LightGBM filter). The recommendation is recommended of the clade Insecta in conf/hc_genes.yaml.

Containers and HPC

The images are pinned in conf/base.config and listed in docs/containers.md.

--nf_config takes a path or the name of a config in conf/: local runs everything on one machine, slurm_generic is a starting point for SLURM. For your own cluster, copy conf/user_hpc_template.config and set your queues, GPU options and scratch paths, see docs/hpc.md. Downloads (SRA reads, OrthoDB partitions, model weights, BUSCO lineages, Rfam, the NCBI taxonomy for OMArk) run on the submitting host, which needs internet access.

Running without containers, and other setups for experts, are described in docs/advanced_setup.md.

Documentation

PageContent
docs/parameters.mdall parameters: inputs, gene finder blocks, Drusilla flow, mode
docs/vipsania.mdVipsania as gene finder: models, finetuning, offline nodes
docs/drusilla_flow.mdDrusilla flow for vertebrate and insect models, with and without proteins: conditions, steps, known issues
docs/postprocessing.mdpost-processing and quality control: sanity filter, UTRs, GFF3 contract, completeness, ncRNA, GO terms, report
docs/containers.mdcontainer images and how they are pinned
docs/hpc.mdNextflow config for your cluster, process labels
docs/advanced_setup.mdfor experts: running without containers, location of the checkout
docs/orf_finder_comparison.mdTransDecoder, TD2 and Drusilla as ORF finder of the HC gene step
docs/hint_rescue_comparison.mdhint rescue compared with the original scripts
docs/development.mdsubmodules and status, repository layout, notes for maintainers, testing, roadmap

License

Artistic License 1.0, see LICENSE, with three exceptions, each marked in the header of the file:

  • Scripts in bin/ copied from Tiberius stay under its MIT License, see LICENSE-Tiberius.
  • Scripts in bin/ copied or ported from BRAKER4 stay under its MIT License, see LICENSE-BRAKER4.
  • bin/fantasia_generate_embeddings.py is derived from FANTASIA-Lite and is under the GNU Affero General Public License v3.0, see LICENSE-FANTASIA-Lite. It is only used when --fantasia is on, where it is bound into the FANTASIA-Lite container.

Citation

Every run writes citations.md to outdir. It lists the references of the gene finder and of the tools that this run used, selected from its inputs and parameters. Please cite them in a publication that uses the results.

methods.md next to it describes, in the style of a Methods section and in the order of execution, what the run did (mode, gene finder and model, evidence processing, high-confidence genes, post-processing, quality control, ncRNA and GO terms as switched on), with inline citations of the references in citations.md. report.html shows it as well. Check and adapt the text before you use it in a publication.

References

Pipeline and gene finders

  • Paludamentum (the pipeline). Gabriel L, Hoff KJ. Paludamentum: evidence integration pipeline for the Gaius-Augustus gene finders. 2026. https://github.com/Gaius-Augustus/Paludamentum
  • Nextflow (workflow engine). Di Tommaso P, Chatzou M, Floden EW, Barja PP, Palumbo E, Notredame C. Nextflow enables reproducible computational workflows. Nature Biotechnology. 2017;35(4):316-319. doi:10.1038/nbt.3820
  • Tiberius (gene prediction). Gabriel L, Becker F, Hoff KJ, Stanke M. Tiberius: end-to-end deep learning with an HMM for gene prediction. Bioinformatics. 2024;40(12):btae685. doi:10.1093/bioinformatics/btae685
  • Tiberius (models of the non-mammalian clades). Gabriel L, Brůna T, Kaur A, Krishnan A, Ortmann F, Salamov A, Talbot S, Becker F, Krieg R, Wheat CW, Grigoriev IV, Stanke M, Hoff KJ. Accurate ab initio gene prediction in eukaryotes with Tiberius in multiple clades. bioRxiv. 2026. doi:10.64898/2026.04.24.720536
  • Vipsania (gene prediction). Krieg R, Becker F, Saenko S, Diehl J, Stanke M. Vipsania: Unsupervised Deep Gene Finding. bioRxiv. 2026. doi:10.64898/2026.08.26.747235

Protein evidence

  • OrthoDB v12 (protein database). Tegenfeldt F, Kuznetsov D, Manni M, Berkeley M, Zdobnov EM, Kriventseva EV. OrthoDB and BUSCO update: annotation of orthologs with wider sampling of genomes. Nucleic Acids Research. 2025;53(D1):D516-D522. doi:10.1093/nar/gkae987
  • miniprot (protein to genome alignment). Li H. Protein-to-genome alignment with miniprot. Bioinformatics. 2023;39(1):btad014. doi:10.1093/bioinformatics/btad014
  • miniprot-boundary-scorer and miniprothint (scoring of the protein alignments, protein hints). Brůna T, Li H, Guhlin J, Honsel D, Herbold S, Stanke M, Nenasheva N, Ebel M, Gabriel L, Hoff KJ. Galba: genome annotation with miniprot and AUGUSTUS. BMC Bioinformatics. 2023;24(1):327. doi:10.1186/s12859-023-05449-z
  • DIAMOND (protein homology of the ORFs, selection of source species in large protein databases). Buchfink B, Reuter K, Drost HG. Sensitive protein alignments at tree-of-life scale using DIAMOND. Nature Methods. 2021;18(4):366-368. doi:10.1038/s41592-021-01101-x

Transcript evidence

  • HISAT2 (alignment of short reads). Kim D, Paggi JM, Park C, Bennett C, Salzberg SL. Graph-based genome alignment and genotyping with HISAT2 and HISAT-genotype. Nature Biotechnology. 2019;37(8):907-915. doi:10.1038/s41587-019-0201-4
  • minimap2 (alignment of Iso-Seq reads). Li H. Minimap2: pairwise alignment for nucleotide sequences. Bioinformatics. 2018;34(18):3094-3100. doi:10.1093/bioinformatics/bty191
  • SAMtools (processing of alignments). Danecek P, Bonfield JK, Liddle J, Marshall J, Ohan V, Pollard MO, Whitwham A, Keane T, McCarthy SA, Davies RM, Li H. Twelve years of SAMtools and BCFtools. GigaScience. 2021;10(2):giab008. doi:10.1093/gigascience/giab008
  • AUGUSTUS (bam2hints) (intron hints from read alignments). Stanke M, Diekhans M, Baertsch R, Haussler D. Using native and syntenically mapped cDNA alignments to improve de novo gene finding. Bioinformatics. 2008;24(5):637-644. doi:10.1093/bioinformatics/btn013
  • VARUS (sampled and aligned reads of the pyVARUS input). Stanke M, Bruhn W, Becker F, Hoff KJ. VARUS: sampling complementary RNA reads from the sequence read archive. BMC Bioinformatics. 2019;20(1):558. doi:10.1186/s12859-019-3182-x
  • StringTie (transcript assembly). Kovaka S, Zimin AV, Pertea GM, Razaghi R, Salzberg SL, Pertea M. Transcriptome assembly from long-read RNA-seq alignments with StringTie2. Genome Biology. 2019;20(1):278. doi:10.1186/s13059-019-1910-1
  • GffRead and GffCompare (format conversion, protein sequences, comparison with a reference annotation). Pertea G, Pertea M. GFF Utilities: GffRead and GffCompare. F1000Research. 2020;9:304. doi:10.12688/f1000research.23297.2

High-confidence genes

  • TransDecoder (ORFs of the assembled transcripts). Haas BJ, Papanicolaou A, Yassour M, et al. De novo transcript sequence reconstruction from RNA-seq using the Trinity platform for reference generation and analysis. Nature Protocols. 2013;8(8):1494-1512. doi:10.1038/nprot.2013.084
  • TD2 (ORFs of the assembled transcripts). Mao A, Ji HJ, Haas BJ, Salzberg SL, Sommer MJ. TD2: finding protein coding regions in transcripts. bioRxiv. 2025. doi:10.1101/2025.04.13.648579
  • PSAURON (ORF scores of TD2). Sommer MJ, Zimin AV, Salzberg SL. PSAURON: a tool for assessing protein annotation across a broad range of species. NAR Genomics and Bioinformatics. 2025;7(1):lqae189. doi:10.1093/nargab/lqae189
  • Drusilla (ORFs of the assembled transcripts as high-confidence genes, filter of the gene predictions). Gabriel L, Hoff KJ. Annotating Eukaryotic Genomes by Combining Deep Learning with extrinsic evidence. Poster, German Conference on Bioinformatics (GCB). 2026. doi:10.13140/RG.2.2.24444.91521
  • LightGBM (filter of the gene predictions). Ke G, Meng Q, Finley T, Wang T, Chen W, Ma W, Ye Q, Liu TY. LightGBM: A Highly Efficient Gradient Boosting Decision Tree. Advances in Neural Information Processing Systems 30 (NIPS 2017). 2017. https://proceedings.neurips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html

Post-processing and quality control

  • GenomeTools (validation of the published GFF3 files). Gremme G, Steinbiss S, Kurtz S. GenomeTools: a comprehensive software library for efficient processing of structured genome annotations. IEEE/ACM Transactions on Computational Biology and Bioinformatics. 2013;10(3):645-656. doi:10.1109/TCBB.2013.68
  • BUSCO (completeness of genome and proteome). Manni M, Berkeley MR, Seppey M, Simão FA, Zdobnov EM. BUSCO Update: novel and streamlined workflows along with broader and deeper phylogenetic coverage for scoring of eukaryotic, prokaryotic, and viral genomes. Molecular Biology and Evolution. 2021;38(10):4647-4654. doi:10.1093/molbev/msab199
  • compleasm (completeness of genome and proteome). Huang N, Li H. compleasm: a faster and more accurate reimplementation of BUSCO. Bioinformatics. 2023;39(10):btad595. doi:10.1093/bioinformatics/btad595
  • OMArk (consistency and completeness of the proteome). Nevers Y, Warwick Vesztrocy A, Rossier V, Train CM, Altenhoff A, Dessimoz C, Glover NM. Quality assessment of gene repertoire annotations with OMArk. Nature Biotechnology. 2025;43(1):124-133. doi:10.1038/s41587-024-02147-w
  • OMAmer (protein families of the proteome for OMArk). Rossier V, Warwick Vesztrocy A, Robinson-Rechavi M, Dessimoz C. OMAmer: tree-driven and alignment-free protein assignment to subfamilies outperforms closest sequence approaches. Bioinformatics. 2021;37(18):2866-2873. doi:10.1093/bioinformatics/btab219
  • pybarrnap (rRNA genes). Shimoyama Y. pybarrnap: Python implementation of barrnap. 2024. https://github.com/moshi4/pybarrnap
  • barrnap (method and rRNA models that pybarrnap re-implements). Seemann T. barrnap 0.9: BAsic Rapid Ribosomal RNA Predictor. 2018. https://github.com/tseemann/barrnap
  • tRNAscan-SE (tRNA genes). Chan PP, Lin BY, Mak AJ, Lowe TM. tRNAscan-SE 2.0: improved detection and functional classification of transfer RNA genes. Nucleic Acids Research. 2021;49(16):9077-9096. doi:10.1093/nar/gkab688
  • Infernal (ncRNA genes of the Rfam families). Nawrocki EP, Eddy SR. Infernal 1.1: 100-fold faster RNA homology searches. Bioinformatics. 2013;29(22):2933-2935. doi:10.1093/bioinformatics/btt509
  • Rfam (RNA families database (release 15.1)). Ontiveros-Palacios N, Cooke E, Nawrocki EP, Triebel S, Marz M, Rivas E, Griffiths-Jones S, Petrov AI, Bateman A, Sweeney B. Rfam 15: RNA families database in 2025. Nucleic Acids Research. 2025;53(D1):D258-D267. doi:10.1093/nar/gkae1023
  • FEELnc (lncRNA genes). Wucher V, Legeai F, Hédan B, et al. FEELnc: a tool for long non-coding RNA annotation and its application to the dog transcriptome. Nucleic Acids Research. 2017;45(8):e57. doi:10.1093/nar/gkw1306
  • FANTASIA (GO terms from protein language model embeddings). Martínez-Redondo GI, Perez-Canales FM, Carbonetto B, Fernández JM, Barrios-Núñez I, Vázquez-Valls M, Cases I, Rojas AM, Fernández R. FANTASIA leverages language models to decode the functional dark proteome across the animal tree of life. Communications Biology. 2025;8(1):1227. doi:10.1038/s42003-025-08651-2
  • FANTASIA suite (FANTASIA-Lite) (GO term annotation software). Pérez-Canales FM, Domínguez-Rodríguez À, Carbonetto B, Fernández R, Cases I, Rojas AM. FANTASIA suite: a reproducible and configurable framework for embedding-based functional annotation of proteins. NAR Genomics and Bioinformatics. 2026;8(3):lqag106. doi:10.1093/nargab/lqag106
  • ProtT5 (protein language model of FANTASIA). Elnaggar A, Heinzinger M, Dallago C, Rehawi G, Wang Y, Jones L, Gibbs T, Feher T, Angerer C, Steinegger M, Bhowmik D, Rost B. ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2022;44(10):7112-7127. doi:10.1109/TPAMI.2021.3095381

Funding

Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation), project number 552910312: "AI-GUSTUS: Eine cloud-native Pipeline für genaue Genom-Annotation" (AI-GUSTUS: a cloud-native pipeline for accurate genome annotation).

Gaius-Augustus/Paludamentum

Paludamentum is the coat of the emperor - evidence processing pipeline

Python

11

173 commits

updated Oct 11, 2026

See the code

README

Paludamentum

Paludamentum logo
$\color{lightgray}{\tiny\textsf{Logo generated by Gemini}}$

Authors: Lars Gabriel & Katharina J. Hoff

The paludamentum was the cloak worn by Roman emperors and commanders. This repository is the cloak around the emperors of the Gaius-Augustus gene finder family: a Nextflow pipeline that prepares extrinsic evidence (proteins, RNA-Seq, Iso-Seq), derives high-confidence genes from it, and integrates them with the ab initio predictions of the deep learning gene finder Tiberius or Vipsania.

Table of contents

What the pipeline does

Paludamentum overview: the inputs genome, proteins, short reads and Iso-Seq reads; the six processing steps ab initio prediction, protein evidence, transcript evidence, high-confidence genes, integration, and post-processing and quality control; the output files

The numbers in the figure are the steps below.

  1. Ab initio prediction. The genome is split into chunks, the gene finder runs on each chunk on a GPU, and the chunk predictions are merged. If you already have a Tiberius or Vipsania prediction, pass it with --result and this step is skipped, see Existing predictions.
  2. Protein evidence. Proteins (your FASTA files and/or OrthoDB v12 partitions) are aligned with miniprot, scored with miniprot-boundary-scorer, and converted to hints with miniprothint. For very large protein databases the ab initio proteins are used to select the most relevant source species first (DIAMOND).
  3. Transcript evidence. Short reads are aligned with HISAT2, Iso-Seq reads with minimap2. Libraries with a low alignment rate are dropped. Transcripts are assembled with StringTie, and intron hints are extracted. If you already have a StringTie assembly, pass it with --stringtie instead of the reads, see Existing StringTie assemblies.
  4. High-confidence genes. Assembled transcripts get ORFs from TD2 (or TransDecoder 5.7.1 with transdecoder: td1). ORFs that are supported by protein homology (DIAMOND) and by the scored protein alignments become the high-confidence (HC) gene set; a comparison of the ORF finders is in docs/orf_finder_comparison.md.
  5. Integration. The HC genes are merged with the ab initio predictions into the final annotation.
  6. Post-processing and quality control. Transcripts with an internal stop codon or a broken CDS are removed, transcripts without UTRs get UTRs from the StringTie assemblies, and the annotation is written as GFF3 (checked with GenomeTools), GTF, protein and CDS sequences. Optional: completeness with compleasm and BUSCO (--busco_lineage), OMArk, a comparison with a reference annotation (--reference_annotation), rRNA, tRNA, Rfam and lncRNA genes (--ncrna) and GO terms with FANTASIA-Lite (--fantasia, GPU). report.html sums it up. See docs/postprocessing.md.

Steps 4 and 5 run in one of three ways. The clade of the gene finder model and the inputs decide which (conf/hc_genes.yaml):

HC genes (steps 4 and 5)ForProteinsDescribed in
TransDecoder flow: TD2 ORFs supported by the proteinsall other cladesrequired with readsstep 4 above
Drusilla flow with proteins: Drusilla ORFs, LightGBM filter and hint rescue of the ab initio genesvertebrates, mammalsgivenDrusilla flow
Drusilla flow without proteins: Drusilla ORFs merged with all ab initio genesinsects (recommended); any clade with a Drusilla modelnot neededInsects

Without any evidence input the pipeline runs steps 1 and 6 only. That is useful to parallelize a gene finder over several GPUs.

Step 6 in detail: the color of a step says which data it needs, the label on it which option switches it on.

Post-processing and quality control: the annotation runs through the sanity filter, gets UTRs from the StringTie assemblies in modes with transcripts, and is written as GFF3, GTF, proteins and CDS; gene set statistics always run, evidence support in evidence modes, completeness, OMArk, gffcompare, ncRNA and GO terms with their options; report.html sums up all steps

Requirements

The pipeline runs with Singularity (or Apptainer): the gene finders and all tools run in container images that the pipeline pulls itself. You do not install HISAT2, miniprot, StringTie, Tiberius, Vipsania or any other tool.

You need:

  • Singularity or Apptainer, on the machine that launches the pipeline and on the nodes that run its tasks
  • Nextflow 25.04 or newer
  • Java 17 or newer (required by Nextflow)
  • Python 3.9 or newer
  • an NVIDIA GPU on the nodes that run the gene finder step, up to and including the Blackwell generation (for example A100 or RTX PRO 6000)

Installation

git clone --recursive https://github.com/Gaius-Augustus/Paludamentum
cd Paludamentum
pip install -e .

This installs the launcher paludamentum. --recursive checks out the submodules tiberius/, vipsania/ and drusilla/; in an existing clone run git submodule update --init after git pull.

The first run pulls the images into ~/.cache/paludamentum/singularity, later runs reuse them. Each gene finder image is about 9.4 GB, the image of the evidence tools 0.6 GB. The images of BUSCO, FEELnc, OMArk and FANTASIA-Lite are pulled only when their steps are switched on. On a cluster this directory must be on a file system that the compute nodes can read; set NXF_SINGULARITY_CACHEDIR to use another directory.

Quick start

# Tiberius, ab initio only, parallelized over GPUs by Nextflow
paludamentum --nf_config slurm_generic --genome genome.fa --model_cfg eudicotyledons

# Tiberius with evidence, inputs from a params file
paludamentum --params_yaml params.yaml --nf_config slurm_generic

# Vipsania with evidence, inputs on the command line
paludamentum --genefinder vipsania --nf_config local --genome genome.fa --model Fungi \
    --proteins proteins.faa --rnaseq_paired "/abs/path/rnaseq/*_{1,2}.fastq.gz"

# Tiberius with proteins and an existing StringTie assembly instead of reads
paludamentum --nf_config local --genome genome.fa --model_cfg eudicotyledons \
    --proteins proteins.faa --stringtie stringtie.gtf

# Evidence for an existing Tiberius prediction (GFF3 or GTF); the gene finder does not run
paludamentum --nf_config local --genome genome.fa --result tiberius.gff3 \
    --proteins proteins.faa --rnaseq_paired "/abs/path/rnaseq/*_{1,2}.fastq.gz"

--nf_config takes a path or the name of a config in conf/ (local, slurm_generic, ...); local sizes the tasks to the machine it runs on, slurm_generic needs your GPU partition (see Containers and HPC). Evidence can be given on the command line (--proteins, --odb12Partitions, --rnaseq_paired, --rnaseq_single, --isoseq, --stringtie, --rnaseq_sra_paired, --rnaseq_sra_single, --isoseq_sra) or in the params file; command line values override the file. Useful options: --dry_run writes the params file and validates inputs and executables without starting Nextflow, --resume continues a previous run, --work_dir sets the Nextflow work directory, --cleanup deletes the task directories of the run from the work directory once it has finished successfully (see docs/hpc.md). Arguments after -- go to Nextflow. paludamentum --help lists everything.

Params file

Start from conf/parameters.yaml. Minimal example:

genome: /abs/path/genome.fa
proteins: /abs/path/proteins.faa
rnaseq_paired: "/abs/path/rnaseq/*_{1,2}.fastq.gz"
outdir: results
tiberius:
  run: true
  model_cfg: eudicotyledons

The launcher expands ~ and environment variables and resolves relative paths against the launch directory, not against the params file; it writes the merged parameters of a run with absolute paths to <outdir>/params.yaml. Paths must not contain spaces. All parameters are documented in docs/parameters.md.

Inputs and modes

ParameterContent
genomegenome FASTA, optionally gzipped (required)
tiberius.result, vipsania.result (--result)existing prediction of the gene finder, GFF3 or GTF, used instead of running it, see Existing predictions
proteinsone protein FASTA or a list; lists are concatenated
odb12PartitionsOrthoDB v12 partitions to download: Metazoa, Vertebrata, Viridiplantae, Arthropoda, Fungi, Alveolata, Stramenopiles, Amoebozoa, Euglenozoa, Eukaryota
rnaseq_paired, rnaseq_singleshort read FASTQ files (glob or list)
rnaseq_bamaligned short reads, for example from VARUS
stringtieStringTie assembly (GTF) made outside the pipeline, one file, a list or a glob; instead of or in addition to reads, see Existing StringTie assemblies
rnaseq_varus, isoseq_varus, mixed_varuspyVARUS output directories, which bring their StringTie assembly and intron hints, see docs/parameters.md
isoseqIso-Seq FASTQ files
rnaseq_sra_paired, rnaseq_sra_single, isoseq_sraSRA run accessions, downloaded by the pipeline; an accession whose download fails twice is dropped with an Error is ignored note, and -resume tries it again
keep_downloadscopy the reads downloaded from SRA to sra_downloads/ in outdir; default false, they stay in the work directory only
min_alignment_ratelibraries whose alignment rate (percent mapped) is below this value are dropped; default 80

The mode is inferred from the inputs and can be forced with mode (--mode). A forced mode without its inputs is an error, and so is transcript evidence without proteins outside the Drusilla flow:

ModeInputs
abinitio (historic name: tiberius)genome only
proteinsproteins
rnaseqshort reads or a StringTie assembly, and proteins
isoseqIso-Seq, and proteins
mixedshort reads and Iso-Seq, and proteins

The TransDecoder HC genes need protein evidence. The Drusilla flow does not: for a clade with a Drusilla model in conf/hc_genes.yaml (vertebrates, mammals, insects) the proteins of the modes rnaseq, isoseq and mixed are optional.

Existing StringTie assemblies

If the transcripts are already assembled, pass the StringTie GTF with --stringtie (stringtie in the params file). The pipeline then neither maps reads nor runs StringTie for them; the assembly goes straight to the HC gene step. Proteins are still required, except in the Drusilla flow.

paludamentum --nf_config local --genome genome.fa --model_cfg eudicotyledons \
    --proteins proteins.faa --stringtie stringtie.gtf
stringtie: /abs/path/stringtie.gtf      # or a list, or a glob such as "/abs/path/*.gtf"

The assembly may come from short reads, Iso-Seq or both (stringtie --mix), and it must be an assembly of this genome FASTA (same sequence names). It combines with --result: an existing prediction and an existing assembly need no GPU, no mapping and no assembly step. Notes:

  • Several assemblies can be given, also together with reads; they are merged with the assemblies that the pipeline makes from the reads.
  • With a vertebrate, mammal or insect gene finder model (a clade with hc: drusilla in conf/hc_genes.yaml) the Drusilla flow runs, which filters transcripts by the coverage and TPM that StringTie writes. It takes exactly one assembly of all reads and no other RNA-Seq or Iso-Seq input, and the GTF must come from stringtie on the reads, not from stringtie --merge (which drops coverage and TPM). Set drusilla.run: false to merge several assemblies in the TransDecoder flow instead.
  • A run with only an assembly is reported as mode rnaseq, whatever reads the assembly was made from. hintsfile.gff then has protein hints only (none without proteins); intron hints from reads do not influence the annotation.

Outputs

All files are written to outdir (default <genefinder>_results). <tool> is tiberius or vipsania.

FileContent
<tool>_evidence.gff3final annotation: ab initio predictions merged with HC genes, sanity filtered, with UTRs in modes with transcripts
<tool>_evidence.gtfthe final annotation as GTF
<tool>_evidence_proteins.faprotein sequences of all transcripts of the final annotation
<tool>_evidence_cds.facoding sequences of the final annotation
<tool>_ab_initio.gff3mode abinitio only: the final annotation (sanity filtered, with .gtf, _proteins.fa and _cds.fa as above)
<tool>_evidence_with_ncRNA.gff3final annotation plus rRNA, tRNA, Rfam and lncRNA genes (ncrna.run)
<tool>_evidence_go.gff3, <tool>_evidence_with_ncRNA_go.gff3the same with GO terms (fantasia.run)
report.htmlmethods, gene set statistics, completeness, evidence support, sanity filter, UTRs, ncRNA, versions and references of the run
qc/sanity filter and UTR reports, gene set statistics and plots, hint support, completeness, OMArk, gffcompare, FANTASIA, software versions, see docs/postprocessing.md
ncrna/rRNA, tRNA, Rfam and lncRNA annotations (ncrna.run)
intermediate/<tool>_ab_initio.gff3ab initio predictions before the sanity filter, in every mode; the file to give to --result of a later run
intermediate/<tool>_merged.gff3ab initio predictions merged with the HC genes, before the sanity filter
intermediate/<stem>_sanity_filtered.gff3the annotation after the sanity filter, before the UTRs
intermediate/hc.gff3high-confidence genes derived from the evidence (TransDecoder)
intermediate/drusilla_orfs.gtfDrusilla ORFs, the HC genes of the Drusilla flow
intermediate/<tool>_lgb_filtered.gtfab initio predictions kept by the LightGBM filter (Drusilla flow)
intermediate/<tool>_lgb_scores.tsvLightGBM class probabilities of all ab initio transcripts (Drusilla flow)
intermediate/hint_rescue.gtfab initio genes predicted again with protein hints (Drusilla flow; missing if the rescue was skipped)
hintsfile.gffprotein, RNA-Seq and Iso-Seq hints
sra_downloads/reads downloaded from SRA, with keep_downloads: true only
params.yamlthe merged parameters of this run, written by the launcher
citations.mdreferences of the gene finder and the tools that this run used, see Citation
methods.mda methods section for this run: the steps that ran, their main parameters and inline citations of citations.md, see Citation

Nextflow's timeline, trace and report files are written as well.

Gene finders

--genefinder tiberius|vipsania selects the gene finder. Without it, the launcher takes genefinder from the params file, else the block with run: true, else Vipsania if --model is given, else Tiberius. Only one gene finder runs per pipeline run.

Gene finderModelParameters
Tiberius--model_cfg, a name in tiberius/model_cfg of the submodule (for example eudicotyledons) or a pathdocs/parameters.md
Vipsania--model, a clade name (for example Fungi) or a model iddocs/vipsania.md

Model weights are downloaded once on the submitting host, so the GPU nodes need no internet; model_dir takes weights that you downloaded yourself. Vipsania finetuning on the target genome is off by default (--finetune).

Existing predictions

If the genome is already annotated with Tiberius or Vipsania, give that file to the pipeline instead of predicting again. The file may be the GFF3 or the GTF that tiberius.py or vipsania annotate wrote, or the intermediate/<tool>_ab_initio.gff3 of an earlier Paludamentum run (the predictions before the sanity filter; <tool>_ab_initio.gff3 in the output directory of mode abinitio is the filtered final annotation). It must belong to the genome FASTA of this run (same sequence names).

# Tiberius prediction
paludamentum --nf_config local --genome genome.fa --result tiberius.gff3 \
    --proteins proteins.faa --rnaseq_paired "/abs/path/rnaseq/*_{1,2}.fastq.gz"

# Vipsania prediction
paludamentum --genefinder vipsania --nf_config local --genome genome.fa --result vipsania.gff \
    --proteins proteins.faa --isoseq isoseq.fastq.gz

In a params file the option is result in the block of the gene finder:

tiberius:
  run: true
  result: /abs/path/tiberius.gff3

All other steps run as usual, and the outputs are the same as in a run that predicts itself. No model option is needed, and the gene finder does not run. Notes:

  • --genefinder names the gene finder that made the file (default: Tiberius). It sets the names of the output files (tiberius_*, vipsania_*).
  • For a prediction of a vertebrate or insect model, name the model as well (--model_cfg vertebrates for Tiberius, --model Vertebrata or --model Insecta for Vipsania). The model is not run; it tells the pipeline to use the Drusilla flow. Without it, runs with transcripts use the TransDecoder HC genes. The hint rescue of the Drusilla flow runs Tiberius on a GPU; all other steps of a run with --result need no GPU.
  • The pipeline renames the genes and transcripts of the file (gene_000001, gene_000001.t1, ...) and reads its exon, CDS and UTR features.

Drusilla flow

For vertebrate, mammal and insect gene finder models (clades with hc: drusilla in conf/hc_genes.yaml), runs with transcripts take another route through steps 4 and 5 of the overview: Drusilla ORFs replace the TransDecoder HC genes. With protein evidence, vertebrates and mammals also get a LightGBM filter of the ab initio predictions, and Tiberius predicts partial genes again with protein hints. The gene-level F1 is 80.70 on Takifugu rubripes with the Tiberius vertebrates model (Paludamentum 0.5.0, gffcompare 0.12.10) and 79.71 on Bos taurus with mammalia_softmasking_v2 (Paludamentum 0.4.0). The flow is switched on automatically (drusilla.run: auto). Conditions, steps, parameters and known issues are in docs/drusilla_flow.md.

The figure shows steps 3 to 5 of the overview for vertebrates and mammals with proteins; its numbers count the steps of this flow.

Drusilla flow: StringTie assembly, transcript filter, Drusilla ORFs and codon fix give the high-confidence genes; LightGBM filter and hint rescue treat the ab initio genes; both are merged into the final annotation

Insects

For insects, Paludamentum recommends RNA-Seq or Iso-Seq reads, Vipsania with the model Insecta fine-tuned on the genome, and no protein evidence:

paludamentum --nf_config local --genome genome.fa --model Insecta --finetune \
    --rnaseq_bam rnaseq.bam

The run then goes through the steps of the overview as follows; the numbers in the figure are those of the overview.

Insect flow: the reads are aligned and assembled with StringTie into one transcript set (step 3), which is filtered, and Drusilla predicts its ORFs with the insect model (step 4); Vipsania, fine-tuned on the genome, predicts the ab initio genes (step 1); all ab initio genes are merged with the ORFs (step 5) and post-processed into vipsania_evidence.gff3 (step 6); step 2, protein evidence, does not run; other choices get a warning

  1. Ab initio prediction. Vipsania fine-tunes its Insecta model on the genome, then annotates the genome with it. This is one GPU task on the whole genome (about 4 h on an A100), not one task per genome chunk.
  2. Protein evidence does not run. With it go the stop and start codon fix of the ORFs, the LightGBM filter and the hint rescue, which lowered the accuracy on insects.
  3. Transcript evidence. Short reads are aligned with HISAT2, Iso-Seq reads with minimap2 (a BAM, a pyVARUS directory or a StringTie assembly is used as given), and all reads are assembled into one StringTie transcript set.
  4. High-confidence genes. Transcripts are kept if they are at least 300 nt long, with a coverage of at least 3 and a TPM of at least 1. Drusilla, with its insect model, predicts the ORFs of the kept transcripts, and isoforms whose CDS is part of another one are collapsed. These ORFs are the HC genes (intermediate/drusilla_orfs.gtf).
  5. Integration. All ab initio genes, without any filter, are merged with the HC genes; transcripts with identical CDS are collapsed.
  6. Post-processing and quality control as for every run; the final annotation is vipsania_evidence.gff3.

On 11 insect genomes this flow reached a mean gene F1 of 75.2, as BRAKER3 with proteins (75.4), and more than Drusilla with Tiberius (66.0), Vipsania alone (60.4) or Tiberius alone (57.6) (evaluation of Lars Gabriel, gffcompare 0.12.10, see docs/drusilla_flow.md). On Drosophila melanogaster and Bombyx mori Paludamentum gives the same as his standalone merge, 84.88 and 74.12 against 84.49 and 73.85.

An insect run that differs from this flow still runs, but the Nextflow log warns and names each difference: Tiberius instead of Vipsania (also the Tiberius insecta and lepidoptera models, which run the Drusilla flow too), Vipsania without fine-tuning, no RNA-Seq or Iso-Seq, or protein evidence (then the ORFs get the codon fix, but there is still no LightGBM filter). The recommendation is recommended of the clade Insecta in conf/hc_genes.yaml.

Containers and HPC

The images are pinned in conf/base.config and listed in docs/containers.md.

--nf_config takes a path or the name of a config in conf/: local runs everything on one machine, slurm_generic is a starting point for SLURM. For your own cluster, copy conf/user_hpc_template.config and set your queues, GPU options and scratch paths, see docs/hpc.md. Downloads (SRA reads, OrthoDB partitions, model weights, BUSCO lineages, Rfam, the NCBI taxonomy for OMArk) run on the submitting host, which needs internet access.

Running without containers, and other setups for experts, are described in docs/advanced_setup.md.

Documentation

PageContent
docs/parameters.mdall parameters: inputs, gene finder blocks, Drusilla flow, mode
docs/vipsania.mdVipsania as gene finder: models, finetuning, offline nodes
docs/drusilla_flow.mdDrusilla flow for vertebrate and insect models, with and without proteins: conditions, steps, known issues
docs/postprocessing.mdpost-processing and quality control: sanity filter, UTRs, GFF3 contract, completeness, ncRNA, GO terms, report
docs/containers.mdcontainer images and how they are pinned
docs/hpc.mdNextflow config for your cluster, process labels
docs/advanced_setup.mdfor experts: running without containers, location of the checkout
docs/orf_finder_comparison.mdTransDecoder, TD2 and Drusilla as ORF finder of the HC gene step
docs/hint_rescue_comparison.mdhint rescue compared with the original scripts
docs/development.mdsubmodules and status, repository layout, notes for maintainers, testing, roadmap

License

Artistic License 1.0, see LICENSE, with three exceptions, each marked in the header of the file:

  • Scripts in bin/ copied from Tiberius stay under its MIT License, see LICENSE-Tiberius.
  • Scripts in bin/ copied or ported from BRAKER4 stay under its MIT License, see LICENSE-BRAKER4.
  • bin/fantasia_generate_embeddings.py is derived from FANTASIA-Lite and is under the GNU Affero General Public License v3.0, see LICENSE-FANTASIA-Lite. It is only used when --fantasia is on, where it is bound into the FANTASIA-Lite container.

Citation

Every run writes citations.md to outdir. It lists the references of the gene finder and of the tools that this run used, selected from its inputs and parameters. Please cite them in a publication that uses the results.

methods.md next to it describes, in the style of a Methods section and in the order of execution, what the run did (mode, gene finder and model, evidence processing, high-confidence genes, post-processing, quality control, ncRNA and GO terms as switched on), with inline citations of the references in citations.md. report.html shows it as well. Check and adapt the text before you use it in a publication.

References

Pipeline and gene finders

  • Paludamentum (the pipeline). Gabriel L, Hoff KJ. Paludamentum: evidence integration pipeline for the Gaius-Augustus gene finders. 2026. https://github.com/Gaius-Augustus/Paludamentum
  • Nextflow (workflow engine). Di Tommaso P, Chatzou M, Floden EW, Barja PP, Palumbo E, Notredame C. Nextflow enables reproducible computational workflows. Nature Biotechnology. 2017;35(4):316-319. doi:10.1038/nbt.3820
  • Tiberius (gene prediction). Gabriel L, Becker F, Hoff KJ, Stanke M. Tiberius: end-to-end deep learning with an HMM for gene prediction. Bioinformatics. 2024;40(12):btae685. doi:10.1093/bioinformatics/btae685
  • Tiberius (models of the non-mammalian clades). Gabriel L, Brůna T, Kaur A, Krishnan A, Ortmann F, Salamov A, Talbot S, Becker F, Krieg R, Wheat CW, Grigoriev IV, Stanke M, Hoff KJ. Accurate ab initio gene prediction in eukaryotes with Tiberius in multiple clades. bioRxiv. 2026. doi:10.64898/2026.04.24.720536
  • Vipsania (gene prediction). Krieg R, Becker F, Saenko S, Diehl J, Stanke M. Vipsania: Unsupervised Deep Gene Finding. bioRxiv. 2026. doi:10.64898/2026.08.26.747235

Protein evidence

  • OrthoDB v12 (protein database). Tegenfeldt F, Kuznetsov D, Manni M, Berkeley M, Zdobnov EM, Kriventseva EV. OrthoDB and BUSCO update: annotation of orthologs with wider sampling of genomes. Nucleic Acids Research. 2025;53(D1):D516-D522. doi:10.1093/nar/gkae987
  • miniprot (protein to genome alignment). Li H. Protein-to-genome alignment with miniprot. Bioinformatics. 2023;39(1):btad014. doi:10.1093/bioinformatics/btad014
  • miniprot-boundary-scorer and miniprothint (scoring of the protein alignments, protein hints). Brůna T, Li H, Guhlin J, Honsel D, Herbold S, Stanke M, Nenasheva N, Ebel M, Gabriel L, Hoff KJ. Galba: genome annotation with miniprot and AUGUSTUS. BMC Bioinformatics. 2023;24(1):327. doi:10.1186/s12859-023-05449-z
  • DIAMOND (protein homology of the ORFs, selection of source species in large protein databases). Buchfink B, Reuter K, Drost HG. Sensitive protein alignments at tree-of-life scale using DIAMOND. Nature Methods. 2021;18(4):366-368. doi:10.1038/s41592-021-01101-x

Transcript evidence

  • HISAT2 (alignment of short reads). Kim D, Paggi JM, Park C, Bennett C, Salzberg SL. Graph-based genome alignment and genotyping with HISAT2 and HISAT-genotype. Nature Biotechnology. 2019;37(8):907-915. doi:10.1038/s41587-019-0201-4
  • minimap2 (alignment of Iso-Seq reads). Li H. Minimap2: pairwise alignment for nucleotide sequences. Bioinformatics. 2018;34(18):3094-3100. doi:10.1093/bioinformatics/bty191
  • SAMtools (processing of alignments). Danecek P, Bonfield JK, Liddle J, Marshall J, Ohan V, Pollard MO, Whitwham A, Keane T, McCarthy SA, Davies RM, Li H. Twelve years of SAMtools and BCFtools. GigaScience. 2021;10(2):giab008. doi:10.1093/gigascience/giab008
  • AUGUSTUS (bam2hints) (intron hints from read alignments). Stanke M, Diekhans M, Baertsch R, Haussler D. Using native and syntenically mapped cDNA alignments to improve de novo gene finding. Bioinformatics. 2008;24(5):637-644. doi:10.1093/bioinformatics/btn013
  • VARUS (sampled and aligned reads of the pyVARUS input). Stanke M, Bruhn W, Becker F, Hoff KJ. VARUS: sampling complementary RNA reads from the sequence read archive. BMC Bioinformatics. 2019;20(1):558. doi:10.1186/s12859-019-3182-x
  • StringTie (transcript assembly). Kovaka S, Zimin AV, Pertea GM, Razaghi R, Salzberg SL, Pertea M. Transcriptome assembly from long-read RNA-seq alignments with StringTie2. Genome Biology. 2019;20(1):278. doi:10.1186/s13059-019-1910-1
  • GffRead and GffCompare (format conversion, protein sequences, comparison with a reference annotation). Pertea G, Pertea M. GFF Utilities: GffRead and GffCompare. F1000Research. 2020;9:304. doi:10.12688/f1000research.23297.2

High-confidence genes

  • TransDecoder (ORFs of the assembled transcripts). Haas BJ, Papanicolaou A, Yassour M, et al. De novo transcript sequence reconstruction from RNA-seq using the Trinity platform for reference generation and analysis. Nature Protocols. 2013;8(8):1494-1512. doi:10.1038/nprot.2013.084
  • TD2 (ORFs of the assembled transcripts). Mao A, Ji HJ, Haas BJ, Salzberg SL, Sommer MJ. TD2: finding protein coding regions in transcripts. bioRxiv. 2025. doi:10.1101/2025.04.13.648579
  • PSAURON (ORF scores of TD2). Sommer MJ, Zimin AV, Salzberg SL. PSAURON: a tool for assessing protein annotation across a broad range of species. NAR Genomics and Bioinformatics. 2025;7(1):lqae189. doi:10.1093/nargab/lqae189
  • Drusilla (ORFs of the assembled transcripts as high-confidence genes, filter of the gene predictions). Gabriel L, Hoff KJ. Annotating Eukaryotic Genomes by Combining Deep Learning with extrinsic evidence. Poster, German Conference on Bioinformatics (GCB). 2026. doi:10.13140/RG.2.2.24444.91521
  • LightGBM (filter of the gene predictions). Ke G, Meng Q, Finley T, Wang T, Chen W, Ma W, Ye Q, Liu TY. LightGBM: A Highly Efficient Gradient Boosting Decision Tree. Advances in Neural Information Processing Systems 30 (NIPS 2017). 2017. https://proceedings.neurips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html

Post-processing and quality control

  • GenomeTools (validation of the published GFF3 files). Gremme G, Steinbiss S, Kurtz S. GenomeTools: a comprehensive software library for efficient processing of structured genome annotations. IEEE/ACM Transactions on Computational Biology and Bioinformatics. 2013;10(3):645-656. doi:10.1109/TCBB.2013.68
  • BUSCO (completeness of genome and proteome). Manni M, Berkeley MR, Seppey M, Simão FA, Zdobnov EM. BUSCO Update: novel and streamlined workflows along with broader and deeper phylogenetic coverage for scoring of eukaryotic, prokaryotic, and viral genomes. Molecular Biology and Evolution. 2021;38(10):4647-4654. doi:10.1093/molbev/msab199
  • compleasm (completeness of genome and proteome). Huang N, Li H. compleasm: a faster and more accurate reimplementation of BUSCO. Bioinformatics. 2023;39(10):btad595. doi:10.1093/bioinformatics/btad595
  • OMArk (consistency and completeness of the proteome). Nevers Y, Warwick Vesztrocy A, Rossier V, Train CM, Altenhoff A, Dessimoz C, Glover NM. Quality assessment of gene repertoire annotations with OMArk. Nature Biotechnology. 2025;43(1):124-133. doi:10.1038/s41587-024-02147-w
  • OMAmer (protein families of the proteome for OMArk). Rossier V, Warwick Vesztrocy A, Robinson-Rechavi M, Dessimoz C. OMAmer: tree-driven and alignment-free protein assignment to subfamilies outperforms closest sequence approaches. Bioinformatics. 2021;37(18):2866-2873. doi:10.1093/bioinformatics/btab219
  • pybarrnap (rRNA genes). Shimoyama Y. pybarrnap: Python implementation of barrnap. 2024. https://github.com/moshi4/pybarrnap
  • barrnap (method and rRNA models that pybarrnap re-implements). Seemann T. barrnap 0.9: BAsic Rapid Ribosomal RNA Predictor. 2018. https://github.com/tseemann/barrnap
  • tRNAscan-SE (tRNA genes). Chan PP, Lin BY, Mak AJ, Lowe TM. tRNAscan-SE 2.0: improved detection and functional classification of transfer RNA genes. Nucleic Acids Research. 2021;49(16):9077-9096. doi:10.1093/nar/gkab688
  • Infernal (ncRNA genes of the Rfam families). Nawrocki EP, Eddy SR. Infernal 1.1: 100-fold faster RNA homology searches. Bioinformatics. 2013;29(22):2933-2935. doi:10.1093/bioinformatics/btt509
  • Rfam (RNA families database (release 15.1)). Ontiveros-Palacios N, Cooke E, Nawrocki EP, Triebel S, Marz M, Rivas E, Griffiths-Jones S, Petrov AI, Bateman A, Sweeney B. Rfam 15: RNA families database in 2025. Nucleic Acids Research. 2025;53(D1):D258-D267. doi:10.1093/nar/gkae1023
  • FEELnc (lncRNA genes). Wucher V, Legeai F, Hédan B, et al. FEELnc: a tool for long non-coding RNA annotation and its application to the dog transcriptome. Nucleic Acids Research. 2017;45(8):e57. doi:10.1093/nar/gkw1306
  • FANTASIA (GO terms from protein language model embeddings). Martínez-Redondo GI, Perez-Canales FM, Carbonetto B, Fernández JM, Barrios-Núñez I, Vázquez-Valls M, Cases I, Rojas AM, Fernández R. FANTASIA leverages language models to decode the functional dark proteome across the animal tree of life. Communications Biology. 2025;8(1):1227. doi:10.1038/s42003-025-08651-2
  • FANTASIA suite (FANTASIA-Lite) (GO term annotation software). Pérez-Canales FM, Domínguez-Rodríguez À, Carbonetto B, Fernández R, Cases I, Rojas AM. FANTASIA suite: a reproducible and configurable framework for embedding-based functional annotation of proteins. NAR Genomics and Bioinformatics. 2026;8(3):lqag106. doi:10.1093/nargab/lqag106
  • ProtT5 (protein language model of FANTASIA). Elnaggar A, Heinzinger M, Dallago C, Rehawi G, Wang Y, Jones L, Gibbs T, Feher T, Angerer C, Steinegger M, Bhowmik D, Rost B. ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2022;44(10):7112-7127. doi:10.1109/TPAMI.2021.3095381

Funding

Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation), project number 552910312: "AI-GUSTUS: Eine cloud-native Pipeline für genaue Genom-Annotation" (AI-GUSTUS: a cloud-native pipeline for accurate genome annotation).