Generative AI using Amazon SageMaker

A comprehensive repository showcasing Generative AI workflows on Amazon SageMaker AI. This collection provides end-to-end implementations spanning the complete ML lifecycle, from foundational concepts to enterprise-scale deployments, covering model training, fine-tuning, inference optimization, MLOps automation, distributed training, RAG systems, intelligent agents, and real-world industry applications.

🚀 Quick Start

New to Generative AI on SageMaker? Start here:

• Getting Started Guide - Essential setup, foundational concepts, and first steps

⚙️ Model Customization Recipes

Config-driven fine-tuning recipes for 20+ foundation models — Pick a model, choose a strategy (QLoRA, Spectrum, or Full Fine-tuning), and launch a SageMaker Training Job. The recipe generator handles everything from dataset formatting to adapter merging and evaluation.

• 20+ Pre-configured Models: Llama, Qwen, DeepSeek, Gemma, Phi, GPT-OSS, and more
• 3 Training Strategies: QLoRA (memory-efficient), Spectrum (balanced), Full Fine-tuning (maximum performance)
• End-to-End Automation: Recipe generator, instance sizing guide, training time estimates, and deployment pipeline

→ Get Started | → Available Models | → Instance Guide

---

🔬 Model Training & Customization — Deep Dives

Hands-on notebooks for specific models, frameworks, and training strategies — While the Recipes above offer a config-driven approach, the Distributed Training folder provides deep-dive implementations where you control every aspect: distributed strategies (DDP, FSDP, DeepSpeed ZeRO-3), reinforcement learning (DPO, GRPO), specialized frameworks (NVIDIA NeMo, veRL, Unsloth), and more.

• Model Fine-Tuning — SFT with DDP, FSDP, and DeepSpeed ZeRO-3 for Qwen, LLaMA, Mistral, Gemma, and GPT-OSS
• Reinforcement Learning — DPO and GRPO using TRL, Unsloth, veRL, and NVIDIA NeMo RL
• NVIDIA NeMo — NeMo RL (GRPO with Ray and vLLM) and NeMo AutoModel (SFT with FSDP2 and DTensor)
• Spectrum Fine-Tuning — SNR-based selective layer freezing for efficient training
• Diffusers — DreamBooth LoRA fine-tuning for FLUX.1-dev image generation
• Efficient Fine-tuning — Unsloth-powered instruction fine-tuning with 2x-5x speed improvements

→ Explore All Deep Dives

---

🤖 Models

This repository supports a comprehensive range of foundation models with various training methodologies. The table below shows model compatibility with different fine-tuning techniques, training frameworks, and deployment options.

Model Support Matrix

Training Infrastructure

• SageMaker HyperPod - High-performance computing clusters for large-scale training
• SageMaker Training Jobs - Standard managed training infrastructure

📚 Repository Structure

🎯 End-to-End GenAI Lifecycle

Complete production workflows covering the entire ML lifecycle with enterprise-grade practices

• Model Customization - Advanced fine-tuning techniques including instruction tuning, parameter-efficient methods (LoRA, QLoRA), and domain adaptation strategies
• Inference - Production deployment patterns, real-time and batch inference, auto-scaling configurations, and multi-model endpoints
• MLOps - Automated CI/CD pipelines using SageMaker Pipelines with integrated preprocessing, training, evaluation, model registration, and batch transform operations

⚡ Distributed Training — Deep Dives

Hands-on notebooks for specific models, frameworks, and training strategies — See the Distributed Training — Deep Dives section above for full details.

🔍 Retrieval-Augmented Generation (RAG)

Knowledge-enhanced AI systems with advanced embedding and retrieval techniques

• VoyageAI Embedding RAG - Production-ready RAG implementation featuring VoyageAI's state-of-the-art embeddings, Claude 3 integration, vector database optimization, and semantic search capabilities for enterprise knowledge bases

🤖 AI Agents

Intelligent multi-agent frameworks and orchestration systems

• DeepSeek CrewAI Agent - Multi-agent research and writing system using DeepSeek R1 Distilled LLaMA 70B with CrewAI orchestration for collaborative task execution
• LangGraph Model Context Protocol - Advanced agentic workflows with MCP integration for loan underwriting, featuring multi-step orchestration and role-based agent specialization
• ML Models as Agent Tools - Integration patterns for using SageMaker-deployed ML models as agent tools via MCP, including both direct implementation and Amazon Bedrock AgentCore approaches
• SageMaker Strands Integration - Enterprise-grade agent solutions with managed hosting and authentication

🎯 Use Cases

Real-world applications and industry-specific solutions

• RAG & Chatbots - Conversational AI with knowledge retrieval using FLAN-T5-XL and Falcon-7B models, featuring document processing and context-aware responses
• Text Summarization - Document and content summarization using AI21, Falcon-7B, and FLAN-T5-XL models with LangChain integration
• Text Summarization to Image - Multi-modal content generation pipeline combining text summarization with image generation capabilities
• Text-to-SQL - Natural language database querying using Code Llama with LangChain SQL query generation, complete with demo database and web interface

🚀 Inference Optimization

Performance and efficiency improvements for production deployments

• Post-Training Quantization - Model compression techniques using GPTQ and AWQ quantization methods, reducing memory footprint by 50-75% while maintaining accuracy, with automated SageMaker Training Job implementation

📊 LLM Performance Evaluation

Comprehensive benchmarking and performance analysis frameworks

• DeepSeek R1 Distilled - Performance evaluation and benchmarking tools for the DeepSeek R1 Distilled model series, including accuracy metrics, latency analysis, and cost optimization studies

📦 Archive

Legacy examples and deprecated implementations for reference and migration guidance

🛠️ Key Features

• Complete ML Lifecycle: From data preprocessing and model training to production deployment and monitoring
• Multiple Training Strategies: Single-node, multi-node distributed, and reinforcement learning approaches with automatic scaling
• Production-Ready MLOps: Automated CI/CD pipelines with SageMaker Pipelines, model registry, and deployment automation
• Advanced AI Patterns: RAG systems, multi-agent orchestration, and multi-modal applications
• Performance Optimization: Model quantization, distributed training, inference acceleration, and cost optimization
• Industry Use Cases: Financial services, healthcare, retail, and manufacturing applications
• Enterprise Security: IAM integration, VPC support, encryption at rest and in transit

🏗️ Technologies & Frameworks

Core Platform

• Amazon SageMaker - Managed ML platform with training, inference, and MLOps capabilities
• Amazon Bedrock - Managed foundation model service with enterprise security
• AWS Lambda & API Gateway - Serverless inference and API management

ML Frameworks

• Hugging Face Transformers - State-of-the-art model implementations and fine-tuning utilities
• PyTorch & TensorFlow - Deep learning frameworks with distributed training support
• FSDP & DeepSpeed - Memory-efficient distributed training frameworks
• Ray - Distributed computing framework for ML workloads

Agent & Orchestration

• LangGraph & LangChain - Agent frameworks and workflow orchestration
• CrewAI - Multi-agent system coordination and task delegation
• Model Context Protocol (MCP) - Standardized tool integration for AI agents

Optimization & Efficiency

• Unsloth - 2x-5x faster fine-tuning with reduced memory usage
• TRL (Transformer Reinforcement Learning) - RLHF and preference optimization
• NVIDIA NeMo - NeMo RL and NeMo AutoModel for distributed training with Ray, vLLM, and DTensor
• llm-compressor - Post-training quantization with GPTQ and AWQ
• vLLM - High-throughput inference serving

📋 Prerequisites

AWS Requirements

• AWS Account with SageMaker access and appropriate service limits
• IAM roles with SageMaker, S3, and related service permissions
• VPC configuration for secure deployments (optional but recommended)

Development Environment

• Python 3.8+ with virtual environment management
• Jupyter Lab/Notebook for interactive development
• AWS CLI configured with appropriate credentials
• Git for version control and collaboration

Knowledge Prerequisites

• Intermediate understanding of machine learning concepts
• Familiarity with Python programming and data science libraries
• Basic knowledge of AWS services and cloud computing
• Understanding of transformer architectures and LLMs (recommended)

🚀 Getting Started

1. Environment Setup

# Clone the repository
git clone <repository-url>
cd generative-ai-sagemaker

# Create and activate virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install core dependencies
pip install -r requirements.txt

2. AWS Configuration

# Configure AWS credentials
aws configure

# Verify SageMaker access
aws sagemaker list-training-jobs --max-items 1

3. Choose Your Learning Path

Beginners (New to GenAI/SageMaker)

1. Start with Getting Started Guide
2. Explore basic Inference examples
3. Try simple Use Cases like text summarization

Intermediate (Some ML/Cloud experience)

1. Dive into Model Customization
2. Explore Distributed Training techniques
3. Implement RAG systems for knowledge-enhanced applications

Advanced (Production-ready implementations)

1. Master MLOps pipelines
2. Build Multi-agent systems
3. Optimize with Quantization techniques

4. Quick Validation

Run a simple inference example to validate your setup:

# Example: Deploy a pre-trained model for text generation
from sagemaker.huggingface import HuggingFaceModel

model = HuggingFaceModel(
    transformers_version="4.28",
    pytorch_version="2.0",
    py_version="py310",
    role=role,
    model_data="s3://path-to-model"
)

predictor = model.deploy(
    initial_instance_count=1,
    instance_type="ml.g5.xlarge"
)

🎯 Example Workflows

Text Generation Pipeline

Data Preparation → Model Fine-tuning → Evaluation → Deployment → Monitoring
     ↓                    ↓              ↓           ↓           ↓
  S3 Storage      SageMaker Training   Model Registry  Endpoint   CloudWatch

RAG Implementation

Document Ingestion → Embedding Generation → Vector Storage → Query Processing → Response Generation
        ↓                     ↓                 ↓              ↓                    ↓
   Text Processing      SageMaker Endpoint   Vector DB    Retrieval Logic    LLM Inference

Multi-Agent System

Task Definition → Agent Orchestration → Tool Execution → Result Aggregation → Final Output
      ↓                  ↓                   ↓               ↓                  ↓
  LangGraph         CrewAI Framework    MCP Servers    Agent Coordination   Structured Response

🤝 Contributing

We welcome contributions from the community! Please see our Contributing Guidelines for details on:

• Reporting Issues: Bug reports, feature requests, and documentation improvements
• Code Contributions: Pull requests, code reviews, and testing procedures
• Standards: Code formatting, documentation requirements, and best practices
• Community Guidelines: Code of conduct and collaboration expectations

Contribution Areas

• New use case implementations
• Performance optimizations
• Documentation improvements
• Testing and validation
• Integration with new AWS services

🔒 Security

Security is our top priority. For security issue notifications and responsible disclosure, please see CONTRIBUTING.

Security Best Practices

• Use IAM roles with least privilege access
• Enable encryption at rest and in transit
• Implement VPC endpoints for secure communication
• Regular security audits and compliance checks

📄 License

This library is licensed under the MIT-0 License. See the LICENSE file for details.

🆘 Support & Resources

Community Support

• GitHub Issues: Bug reports and feature requests
• GitHub Discussions: Community Q&A and knowledge sharing
• Documentation: Comprehensive guides and API references

AWS Resources

• SageMaker Documentation: Official AWS Documentation
• AWS Support: Professional support plans available
• AWS Training: Certification and learning paths

Additional Resources

• Model Hub: Pre-trained models and configurations
• Best Practices: Performance optimization and cost management
• Case Studies: Real-world implementation examples

---

🌟 What's New

• Latest Updates: DeepSeek R1 integration, enhanced MCP support, improved quantization techniques
• Coming Soon: Multi-modal agents, advanced RAG patterns, cost optimization tools
• Community Highlights: Featured implementations and success stories

---

Ready to build the future of AI? Start exploring the examples and building your next Generative AI application on Amazon SageMaker! 🚀

This repository is actively maintained and regularly updated with the latest AWS services, model architectures, and best practices. Star ⭐ the repository to stay updated with new releases and features.