tugot17/pmpp

Complete solutions to the Programming Massively Parallel Processors Edition 4

Jupyter Notebook

905

176 commits

updated Sep 7, 2026

See the code

README

Programming Massively Parallel Processors - Complete Solutions

Book Cover

CUDA Python License

Complete solutions to Kirk & Hwu's Programming Massively Parallel Processors (4th Edition)

Theoretical explanations + Working implementations + Performance analysis


Overview

This repository contains comprehensive solutions to all exercises in Programming Massively Parallel Processors by David Kirk and Wen-mei Hwu (4th Edition). Each chapter includes:

  • Detailed exercise solutions with step-by-step explanations
  • Working code implementations in both CUDA C and Python
  • Performance benchmarks comparing different approaches
  • Visual diagrams for complex algorithms

Chapter Organization

Each chapter follows this structure:

├── code/
│   ├── *.cu          # CUDA implementations
│   ├── *.py          # Python alternatives  
│   ├── Makefile      # Build configuration
│   └── ...
└── README.md         # Theory + Exercises + Solutions

Available Chapters

ChapterTopicFocus Areas
Chapter 2Heterogeneous Data Parallel ComputingVector operations, thread mapping, CUDA basics
Chapter 3Multidimensional Grids and DataGrid organization, thread hierarchy
Chapter 4Compute Architecture and SchedulingGPU architecture, warps, occupancy
Chapter 5Memory Architecture and Data LocalityMemory types, tiling, bandwidth optimization
Chapter 6Performance ConsiderationsMemory coalescing, latency hiding
Chapter 7ConvolutionConstant memory, caching, halo cells
Chapter 8Stencil2D/3D stencil computations, register tiling
Chapter 9Parallel HistogramAtomic operations, privatization, aggregation
Chapter 10ReductionTree reduction, divergence minimization
Chapter 11Prefix Sum (Scan)Work-efficient algorithms, Kogge-Stone, Brent-Kung
Chapter 12MergeCo-rank function, circular buffers
Chapter 13SortingRadix sort, merge sort optimization
Chapter 14Sparse Matrix ComputationSpMV, CSR/ELL/COO formats
Chapter 15Graph TraversalBFS algorithms, frontier-based approaches
Chapter 16Deep LearningCNN implementation, GEMM formulation
Chapter 17Iterative MRI ReconstructionMedical imaging algorithms
Chapter 18Electrostatic Potential MapScatter vs gather, cutoff binning
Chapter 19Parallel Programming and Computational ThinkingAlgorithm selection, problem decomposition
Chapter 20Heterogeneous Computing ClusterCUDA streams, MPI integration
Chapter 21CUDA Dynamic ParallelismRecursive algorithms, quadtrees

Quick Start

Prerequisites

  • NVIDIA GPU with CUDA support
  • CUDA Toolkit installed
  • Python 3.11+ (optional, for Python examples)

Setup

# Clone the repository
git clone <repository-url>
cd pmpp

# For Python examples (optional)
conda create -n pmpp python=3.11
conda activate pmpp
pip install -r requirements.txt

Running Examples

CUDA/C Examples:

cd chapter-XX/code
make
./program_name

Python Examples:

cd chapter-XX/code
python script_name.py

Contributing

Found an error? Please open an issue using this template:

Describe the bug

Describe where the problem is and what precisely is wrong.

Proposed solution

Here paste your proposed solution. Please include the reasoning behind why you believe your solution is correct.

Contribution Guidelines

  • Maintain the existing explanation style with clear reasoning
  • Include working code for any new implementations
  • Add performance data where relevant
  • Follow the existing code formatting standards

License

This project is licensed under the MIT License - see the LICENSE file for details.

tugot17/pmpp

Complete solutions to the Programming Massively Parallel Processors Edition 4

Jupyter Notebook

905

176 commits

updated Sep 7, 2026

See the code

README

Programming Massively Parallel Processors - Complete Solutions

Book Cover

CUDA Python License

Complete solutions to Kirk & Hwu's Programming Massively Parallel Processors (4th Edition)

Theoretical explanations + Working implementations + Performance analysis


Overview

This repository contains comprehensive solutions to all exercises in Programming Massively Parallel Processors by David Kirk and Wen-mei Hwu (4th Edition). Each chapter includes:

  • Detailed exercise solutions with step-by-step explanations
  • Working code implementations in both CUDA C and Python
  • Performance benchmarks comparing different approaches
  • Visual diagrams for complex algorithms

Chapter Organization

Each chapter follows this structure:

├── code/
│   ├── *.cu          # CUDA implementations
│   ├── *.py          # Python alternatives  
│   ├── Makefile      # Build configuration
│   └── ...
└── README.md         # Theory + Exercises + Solutions

Available Chapters

ChapterTopicFocus Areas
Chapter 2Heterogeneous Data Parallel ComputingVector operations, thread mapping, CUDA basics
Chapter 3Multidimensional Grids and DataGrid organization, thread hierarchy
Chapter 4Compute Architecture and SchedulingGPU architecture, warps, occupancy
Chapter 5Memory Architecture and Data LocalityMemory types, tiling, bandwidth optimization
Chapter 6Performance ConsiderationsMemory coalescing, latency hiding
Chapter 7ConvolutionConstant memory, caching, halo cells
Chapter 8Stencil2D/3D stencil computations, register tiling
Chapter 9Parallel HistogramAtomic operations, privatization, aggregation
Chapter 10ReductionTree reduction, divergence minimization
Chapter 11Prefix Sum (Scan)Work-efficient algorithms, Kogge-Stone, Brent-Kung
Chapter 12MergeCo-rank function, circular buffers
Chapter 13SortingRadix sort, merge sort optimization
Chapter 14Sparse Matrix ComputationSpMV, CSR/ELL/COO formats
Chapter 15Graph TraversalBFS algorithms, frontier-based approaches
Chapter 16Deep LearningCNN implementation, GEMM formulation
Chapter 17Iterative MRI ReconstructionMedical imaging algorithms
Chapter 18Electrostatic Potential MapScatter vs gather, cutoff binning
Chapter 19Parallel Programming and Computational ThinkingAlgorithm selection, problem decomposition
Chapter 20Heterogeneous Computing ClusterCUDA streams, MPI integration
Chapter 21CUDA Dynamic ParallelismRecursive algorithms, quadtrees

Quick Start

Prerequisites

  • NVIDIA GPU with CUDA support
  • CUDA Toolkit installed
  • Python 3.11+ (optional, for Python examples)

Setup

# Clone the repository
git clone <repository-url>
cd pmpp

# For Python examples (optional)
conda create -n pmpp python=3.11
conda activate pmpp
pip install -r requirements.txt

Running Examples

CUDA/C Examples:

cd chapter-XX/code
make
./program_name

Python Examples:

cd chapter-XX/code
python script_name.py

Contributing

Found an error? Please open an issue using this template:

Describe the bug

Describe where the problem is and what precisely is wrong.

Proposed solution

Here paste your proposed solution. Please include the reasoning behind why you believe your solution is correct.

Contribution Guidelines

  • Maintain the existing explanation style with clear reasoning
  • Include working code for any new implementations
  • Add performance data where relevant
  • Follow the existing code formatting standards

License

This project is licensed under the MIT License - see the LICENSE file for details.

Significant stargazers

Fangjun Kuang

763 followers · starred Aug 2025

Mish Ushakov

761 followers · starred Jul 2025

_Kerman

682 followers · starred Jul 2025

bssrdf

47 followers · starred Jul 2025