Cloud GPU performance benchmark recipes

This repository contains recipes that provide instructions to reproduce specific workload performance measurements, which are part of a confidential benchmarking program. These recipes focus on helping you reliably achieve performance metrics, such as throughput, that demonstrate the combined hardware and software stack on GPUs.

Note: The recipes in this repository are not designed as general-purpose code samples or tutorials for using Compute Engine-based products.

Intended audience

This content is for you if you are a customer or partner who needs to:

• Validate hardware performance with your suppliers.
• Inform purchasing decisions using the benchmarking data.
• Reproduce optimal performance scenarios before you customize workflows for your own requirements.

How to use these recipes

To reproduce a benchmark, follow these steps:

1. Identify your requirements: determine the model, GPU type, workload, framework, and orchestrator that you are interested in.
2. Select a recipe: based on your requirements use the Benchmark support matrix to find a recipe that meets your needs.
3. Follow the recipe: each recipe will provide you with procedures to complete the following tasks:
   • prepare your environment.
   • run the benchmark.
   • analyze the benchmarks results. This includes not just the results but detailed logs for further analysis. You can automate your infrastructure setup using Cluster Toolkit. For more information, see Automated GPU environment deployment with Cluster Toolkit.

Benchmarks support matrix

Training benchmarks A3 Mega

Training benchmarks A3 Ultra

Training benchmarks A4

Training benchmarks A4X

Training benchmarks A4X MAX

Inference benchmarks A3 Mega

Inference benchmarks A3 Ultra

Inference benchmarks A4

Inference benchmarks A4X

Inference benchmarks G4

Checkpointing benchmarks

Goodput benchmarks

Repository organization

• ./training: this directory contains recipes with instructions to reproduce training benchmarks with GPUs.
• ./inference: this directory contains recipes with instructions to reproduce inference benchmarks with GPUs.
• ./src: this directory contains the shared dependencies required to run benchmarks, such as Docker images and Helm charts.
• ./docs: this directory contains supporting documentation for explanations of benchmark methodologies or configurations.

Repository scope

This repository provides the steps that you can use to reproduce a specific benchmark. The actual performance measurements and the complete, confidential benchmark report are not included.

Methodology

Performance benchmarks measure the performance of various workloads on the platform. These benchmarks are primarily used to validate performance with hardware suppliers and to provide you with data for purchasing decisions.

Maintenance policy

Benchmark data is considered a point-in-time measurement and completed benchmarks are not repeated. We maintain and update the recipes in this repository on a best-effort basis.

Resources

For general guidance on how to get started using Compute products, refer to the official documentation and tutorials:

• Compute Engine overview
• Compute Engine samples
• Cloud GPU documentation
• AI Hypercomputer documentation
• Automated GPU environment deployment with Cluster Toolkit

Report issues

If you have questions or encounter problems with this repository, report them through GitHub Issues or reach out to your Google Cloud account team for assistance.

Contributor notes

Note: This is not an officially supported Google product. This project is not eligible for the Google Open Source Software Vulnerability Rewards Program.