Generative AI on Amazon EKS

Deploy and run Large Language Model (LLM) inference workloads on Amazon EKS with GPU acceleration, observability, and model storage — all provisioned with Terraform.

📺 Workshop Walkthrough

In this video, you'll see how to deploy the GenAI on EKS workshop in your own AWS account using Terraform and run it end-to-end.

What's Included

• Terraform infrastructure — EKS Auto Mode cluster, VPC, S3 model storage, Amazon Managed Prometheus, Grafana dashboards, and IAM roles
• Kubernetes manifests — Ready-to-deploy configurations for inference workloads

Target Audience

This workshop is intended for Machine Learning Scientists/Engineers, Data Scientists/Engineers, Prompt Engineers, Developers, and Technical Founders.

While not mandatory, participants will benefit from:

• Basic knowledge of ML frameworks (PyTorch, Hugging Face Transformers)
• Fundamental understanding of Kubernetes concepts
• Familiarity with Python programming

New to Amazon EKS? We recommend completing the EKS Workshop first.

Prerequisites

• AWS CLI (>= 2.x)
• Terraform (>= 1.3)
• kubectl
• AWS account with GPU instance quota

Getting Started

See terraform/README.md for full deployment instructions.

Repository Structure

.
├── terraform/          # Infrastructure as Code (EKS, VPC, S3, AMP, Grafana)
│   ├── grafana-dashboards/   # Pre-built Grafana dashboard JSON files
│   ├── main.tf               # Provider and locals configuration
│   ├── eks.tf                # EKS cluster and S3 CSI driver
│   ├── vpc.tf                # VPC and networking
│   ├── helm.tf               # Observability stack (Prometheus, Grafana)
│   ├── amp.tf                # Amazon Managed Prometheus
│   ├── variables.tf          # Configurable variables
│   └── ...
└── manifests/          # Kubernetes manifests for inference workloads

Cleanup

cd terraform
terraform destroy --auto-approve

Security

See CONTRIBUTING for more information.

License

This library is licensed under the MIT-0 License. See the LICENSE file.