sdodlapati3/BioPipelines

Comprehensive bioinformatics pipelines for NGS analysis

0

stars

348

commits

Python

primary language

Dec 5, 2025

updated

README

BioPipelines

AI-Powered Bioinformatics Workflow Generation
Natural language → Production Nextflow pipelines → Results

Python 3.10+ Nextflow License: MIT

🚀 Quick Start (30 seconds)

# 1. Activate environment
conda activate ~/envs/biopipelines

# 2. Set API keys (choose any provider)
export GEMINI_API_KEY="your-key"      # Google - FREE 1,500/day
export CEREBRAS_API_KEY="your-key"    # Fastest - FREE 14,400/day
export GROQ_API_KEY="your-key"        # Fast - FREE 14,400/day

# 3. Launch web interface
./scripts/start_gradio.sh

# 4. Open browser and type:
#    "RNA-seq differential expression for mouse, paired-end reads"

📖 Complete Architecture Guide - Technical deep-dive


🌐 LLM Provider Cascade

BioPipelines uses intelligent provider routing with automatic failover:

PriorityProviderFree TierSpeedBest For
1Google Gemini1,500 req/day~500msHigh-quality generation
2Cerebras14,400 req/day, 1M tokens~170msFastest inference
3Groq14,400 req/day~170msFast inference
4OpenRouter50 req/day (:free models)~2.8sModel variety
5Lightning.ai1,000 credits~400msDeepSeek models
6GitHub ModelsRequires approval~1sGPT-4 access
15OllamaUnlimited (local)VariablePrivacy/offline
16vLLMUnlimited (local)GPU-dependentCustom models
99OpenAIPay-per-use~800msBest quality

Automatic Failover: If Gemini is unavailable, system cascades to Cerebras → Groq → etc.

See FREE_LLM_PROVIDERS.md for detailed API key setup.


✨ Features

🤖 AI-Powered Workflow Composer

Generate production-ready Nextflow pipelines from natural language:

from workflow_composer import Composer

composer = Composer()  # Uses provider cascade automatically
workflow = composer.generate(
    "RNA-seq differential expression for mouse, treatment vs control"
)
workflow.save("my_rnaseq_workflow/")

🧬 UnifiedIntentParser (87.4% Accuracy)

Advanced natural language understanding:

  • Multi-model ensemble with arbiter voting
  • Automatic model selection based on query complexity
  • 40% LLM augmentation for ambiguous queries

🧬 10 Production-Ready Pipelines

All pipelines fully validated and containerized:

PipelineDescriptionStatus
DNA-seqVariant calling with GATK, FreeBayes✅ Validated
RNA-seqDifferential expression with DESeq2✅ Validated
scRNA-seqSingle-cell analysis with Scanpy✅ Validated
ChIP-seqPeak calling with MACS2✅ Validated
ATAC-seqChromatin accessibility✅ Validated
MethylationWGBS/RRBS bisulfite analysis⚠️ Core Complete
Hi-C3D genome organization⚠️ Core Complete
Long-readNanopore/PacBio SV detection✅ Validated
MetagenomicsTaxonomic profiling (Kraken2)✅ Validated
Structural VariantsMulti-tool SV calling✅ Validated

📦 Installation

# Clone the repository
git clone https://github.com/sdodlapati3/BioPipelines.git
cd BioPipelines

# Create conda environment
conda env create -f environment.yml
conda activate biopipelines

# Install Python package
pip install -e .

# Set up API keys (copy and edit)
cp docker/.env.example .env
# Edit .env with your API keys

🎯 Usage

./scripts/start_gradio.sh
# Open http://localhost:7860

CLI

# Generate workflow from natural language
biocomposer generate "ChIP-seq peak calling for human H3K4me3" -o chipseq_workflow/

# Interactive chat mode
biocomposer chat

# Search available tools
biocomposer tools --search "alignment"

# Check LLM providers
biocomposer providers --check

Python API

from workflow_composer import Composer
from workflow_composer.providers import check_providers

# Check available providers
status = check_providers()
print(status)
# {'gemini': True, 'cerebras': True, 'groq': True, ...}

# Generate workflow
composer = Composer()
workflow = composer.generate(
    "WGS germline variant calling for human samples"
)
workflow.save("variants_workflow/")

📁 Project Structure

BioPipelines/
├── src/workflow_composer/     # AI Workflow Composer (main package)
│   ├── providers/             # LLM providers (Gemini, Cerebras, Groq, etc.)
│   ├── core/                  # Intent parsing, tool selection, generation
│   ├── agents/                # ChatAgent, multi-agent orchestration
│   ├── cli.py                 # biocomposer CLI
│   └── composer.py            # Main Composer class
├── nextflow-pipelines/        # Production Nextflow pipelines
│   └── modules/               # Reusable Nextflow modules
├── containers/                # Singularity container definitions
│   ├── base/                  # Base bioinformatics container
│   ├── rna-seq/               # RNA-seq tools container
│   └── ...                    # Pipeline-specific containers
├── config/                    # Configuration files
│   ├── composer.yaml          # Workflow Composer config
│   └── tool_mappings.yaml     # Tool catalog
├── scripts/                   # Utility scripts
│   ├── start_gradio.sh        # Launch web interface
│   ├── start_server.sh        # Start API server
│   └── llm/                   # vLLM server scripts
├── data/                      # Data directory (gitignored)
├── docs/                      # Documentation
│   ├── ARCHITECTURE.md        # System architecture
│   ├── FREE_LLM_PROVIDERS.md  # Free LLM API guide
│   └── tutorials/             # Step-by-step guides
├── examples/                  # Example workflows
│   └── generated/             # AI-generated examples
├── tests/                     # Test suite
└── logs/                      # Runtime logs

🔧 Requirements

  • Python >= 3.10
  • Conda/Mamba (for environment management)
  • Nextflow >= 23.0 (for pipeline execution)
  • Singularity >= 3.8 (for containerized tools)
  • SLURM (optional, for HPC execution)

📚 Documentation


🤝 Contributing

Contributions welcome! Please open an issue first to discuss changes.

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing-feature)
  3. Commit your changes (git commit -m 'Add amazing feature')
  4. Push to branch (git push origin feature/amazing-feature)
  5. Open a Pull Request

📄 License

MIT License - see LICENSE for details.


📧 Contact

For questions, please open an issue on GitHub.

Repository: github.com/sdodlapati3/BioPipelines

Contributors

renoschubert

348 commits

sdodlapati3/BioPipelines

Comprehensive bioinformatics pipelines for NGS analysis

0

stars

348

commits

Python

primary language

Dec 5, 2025

updated

README

BioPipelines

AI-Powered Bioinformatics Workflow Generation
Natural language → Production Nextflow pipelines → Results

Python 3.10+ Nextflow License: MIT

🚀 Quick Start (30 seconds)

# 1. Activate environment
conda activate ~/envs/biopipelines

# 2. Set API keys (choose any provider)
export GEMINI_API_KEY="your-key"      # Google - FREE 1,500/day
export CEREBRAS_API_KEY="your-key"    # Fastest - FREE 14,400/day
export GROQ_API_KEY="your-key"        # Fast - FREE 14,400/day

# 3. Launch web interface
./scripts/start_gradio.sh

# 4. Open browser and type:
#    "RNA-seq differential expression for mouse, paired-end reads"

📖 Complete Architecture Guide - Technical deep-dive


🌐 LLM Provider Cascade

BioPipelines uses intelligent provider routing with automatic failover:

PriorityProviderFree TierSpeedBest For
1Google Gemini1,500 req/day~500msHigh-quality generation
2Cerebras14,400 req/day, 1M tokens~170msFastest inference
3Groq14,400 req/day~170msFast inference
4OpenRouter50 req/day (:free models)~2.8sModel variety
5Lightning.ai1,000 credits~400msDeepSeek models
6GitHub ModelsRequires approval~1sGPT-4 access
15OllamaUnlimited (local)VariablePrivacy/offline
16vLLMUnlimited (local)GPU-dependentCustom models
99OpenAIPay-per-use~800msBest quality

Automatic Failover: If Gemini is unavailable, system cascades to Cerebras → Groq → etc.

See FREE_LLM_PROVIDERS.md for detailed API key setup.


✨ Features

🤖 AI-Powered Workflow Composer

Generate production-ready Nextflow pipelines from natural language:

from workflow_composer import Composer

composer = Composer()  # Uses provider cascade automatically
workflow = composer.generate(
    "RNA-seq differential expression for mouse, treatment vs control"
)
workflow.save("my_rnaseq_workflow/")

🧬 UnifiedIntentParser (87.4% Accuracy)

Advanced natural language understanding:

  • Multi-model ensemble with arbiter voting
  • Automatic model selection based on query complexity
  • 40% LLM augmentation for ambiguous queries

🧬 10 Production-Ready Pipelines

All pipelines fully validated and containerized:

PipelineDescriptionStatus
DNA-seqVariant calling with GATK, FreeBayes✅ Validated
RNA-seqDifferential expression with DESeq2✅ Validated
scRNA-seqSingle-cell analysis with Scanpy✅ Validated
ChIP-seqPeak calling with MACS2✅ Validated
ATAC-seqChromatin accessibility✅ Validated
MethylationWGBS/RRBS bisulfite analysis⚠️ Core Complete
Hi-C3D genome organization⚠️ Core Complete
Long-readNanopore/PacBio SV detection✅ Validated
MetagenomicsTaxonomic profiling (Kraken2)✅ Validated
Structural VariantsMulti-tool SV calling✅ Validated

📦 Installation

# Clone the repository
git clone https://github.com/sdodlapati3/BioPipelines.git
cd BioPipelines

# Create conda environment
conda env create -f environment.yml
conda activate biopipelines

# Install Python package
pip install -e .

# Set up API keys (copy and edit)
cp docker/.env.example .env
# Edit .env with your API keys

🎯 Usage

./scripts/start_gradio.sh
# Open http://localhost:7860

CLI

# Generate workflow from natural language
biocomposer generate "ChIP-seq peak calling for human H3K4me3" -o chipseq_workflow/

# Interactive chat mode
biocomposer chat

# Search available tools
biocomposer tools --search "alignment"

# Check LLM providers
biocomposer providers --check

Python API

from workflow_composer import Composer
from workflow_composer.providers import check_providers

# Check available providers
status = check_providers()
print(status)
# {'gemini': True, 'cerebras': True, 'groq': True, ...}

# Generate workflow
composer = Composer()
workflow = composer.generate(
    "WGS germline variant calling for human samples"
)
workflow.save("variants_workflow/")

📁 Project Structure

BioPipelines/
├── src/workflow_composer/     # AI Workflow Composer (main package)
│   ├── providers/             # LLM providers (Gemini, Cerebras, Groq, etc.)
│   ├── core/                  # Intent parsing, tool selection, generation
│   ├── agents/                # ChatAgent, multi-agent orchestration
│   ├── cli.py                 # biocomposer CLI
│   └── composer.py            # Main Composer class
├── nextflow-pipelines/        # Production Nextflow pipelines
│   └── modules/               # Reusable Nextflow modules
├── containers/                # Singularity container definitions
│   ├── base/                  # Base bioinformatics container
│   ├── rna-seq/               # RNA-seq tools container
│   └── ...                    # Pipeline-specific containers
├── config/                    # Configuration files
│   ├── composer.yaml          # Workflow Composer config
│   └── tool_mappings.yaml     # Tool catalog
├── scripts/                   # Utility scripts
│   ├── start_gradio.sh        # Launch web interface
│   ├── start_server.sh        # Start API server
│   └── llm/                   # vLLM server scripts
├── data/                      # Data directory (gitignored)
├── docs/                      # Documentation
│   ├── ARCHITECTURE.md        # System architecture
│   ├── FREE_LLM_PROVIDERS.md  # Free LLM API guide
│   └── tutorials/             # Step-by-step guides
├── examples/                  # Example workflows
│   └── generated/             # AI-generated examples
├── tests/                     # Test suite
└── logs/                      # Runtime logs

🔧 Requirements

  • Python >= 3.10
  • Conda/Mamba (for environment management)
  • Nextflow >= 23.0 (for pipeline execution)
  • Singularity >= 3.8 (for containerized tools)
  • SLURM (optional, for HPC execution)

📚 Documentation


🤝 Contributing

Contributions welcome! Please open an issue first to discuss changes.

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing-feature)
  3. Commit your changes (git commit -m 'Add amazing feature')
  4. Push to branch (git push origin feature/amazing-feature)
  5. Open a Pull Request

📄 License

MIT License - see LICENSE for details.


📧 Contact

For questions, please open an issue on GitHub.

Repository: github.com/sdodlapati3/BioPipelines

Contributors

renoschubert

348 commits

Languages

Python

91.5%

Nextflow

4.3%

Shell

4.0%