Shadowgraph
TrendingBuzzAbout
Sign in

coderonion/awesome-cuda-and-hpc

🚀🚀🚀 This repository lists some awesome public CUDA, cuda-python, cuBLAS, cuDNN, CUTLASS, TensorRT, TensorRT-LLM, Triton, TVM, MLIR, PTX and High Performance Computing (HPC) projects.

518

45 commits

updated Aug 2, 2025

See the code

README

Awesome-CUDA-and-HPC

Awesome

🚀🚀🚀 This repository lists some awesome public CUDA, cuda-python, cuBLAS, CUTLASS, cuDNN, NCCL, TensorRT, TensorRT-LLM, Triton, TileLang, TVM, MLIR, LLVM, PTX and High Performance Computing (HPC) projects.

Contents

Official Version

  • CUDA : CUDA is a parallel computing platform and programming model developed by NVIDIA for general computing on graphical processing units (GPUs).

  • NVIDIA/cuda-python : CUDA Python: Performance meets Productivity. nvidia.github.io/cuda-python/

  • cuBLAS : Basic Linear Algebra on NVIDIA GPUs. NVIDIA cuBLAS is a GPU-accelerated library for accelerating AI and HPC applications. It includes several API extensions for providing drop-in industry standard BLAS APIs and GEMM APIs with support for fusions that are highly optimized for NVIDIA GPUs. The cuBLAS library also contains extensions for batched operations, execution across multiple GPUs, and mixed- and low-precision execution with additional tuning for the best performance.

  • cuDNN : The NVIDIA CUDA Deep Neural Network library (cuDNN) is a GPU-accelerated library of primitives for deep neural networks. cuDNN provides highly tuned implementations for standard routines such as forward and backward convolution, attention, matmul, pooling, and normalization.

  • CUTLASS : CUDA Templates for Linear Algebra Subroutines. CUTLASS is a collection of CUDA C++ template abstractions for implementing high-performance matrix-matrix multiplication (GEMM) and related computations at all levels and scales within CUDA. It incorporates strategies for hierarchical decomposition and data movement similar to those used to implement cuBLAS and cuDNN.

  • TensorRT : NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT. developer.nvidia.com/tensorrt

  • TensorRT-LLM : TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines. nvidia.github.io/TensorRT-LLM

  • Triton : Development repository for the Triton language and compiler. This is the development repository of Triton, a language and compiler for writing highly efficient custom Deep-Learning primitives. The aim of Triton is to provide an open-source environment to write fast code at higher productivity than CUDA, but also with higher flexibility than other existing DSLs. triton-lang.org/

  • TileLang : Domain-specific language designed to streamline the development of high-performance GPU/CPU/Accelerators kernels. Tile Language (tile-lang) is a concise domain-specific language designed to streamline the development of high-performance GPU/CPU kernels (e.g., GEMM, Dequant GEMM, FlashAttention, LinearAttention). By employing a Pythonic syntax with an underlying compiler infrastructure on top of TVM, tile-lang allows developers to focus on productivity without sacrificing the low-level optimizations necessary for state-of-the-art performance. tilelang.com/

  • TVM : Open deep learning compiler stack for cpu, gpu and specialized accelerators. tvm.apache.org/

  • MLIR : Multi-Level Intermediate Representation Compiler Framework. The MLIR project is a novel approach to building reusable and extensible compiler infrastructure. MLIR aims to address software fragmentation, improve compilation for heterogeneous hardware, significantly reduce the cost of building domain specific compilers, and aid in connecting existing compilers together.

  • LLVM : The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.

  • PTX : PTX, a low-level parallel thread execution virtual machine and instruction set architecture (ISA).

Awesome List

Learning Resources

Frameworks

  • CUDA Frameworks

    • GPU Interface

      GPU接口
      • CPP Version

        • CCCL : CUDA C++ Core Libraries. The concept for the CUDA C++ Core Libraries (CCCL) grew organically out of the Thrust, CUB, and libcudacxx projects that were developed independently over the years with a similar goal: to provide high-quality, high-performance, and easy-to-use C++ abstractions for CUDA developers.

        • HIP : HIP: C++ Heterogeneous-Compute Interface for Portability. HIP is a C++ Runtime API and Kernel Language that allows developers to create portable applications for AMD and NVIDIA GPUs from single source code. rocmdocs.amd.com/projects/HIP/

      • Python Version
      • Rust Version
      • Julia Version

    • Performance Benchmark

      • NVIDIA/nvbench : CUDA Kernel Benchmarking Library.

      • FlagPerf : FlagPerf is an open-source software platform for benchmarking AI chips. FlagPerf是智源研究院联合AI硬件厂商共建的一体化AI硬件评测引擎，旨在建立以产业实践为导向的指标体系，评测AI硬件在软件栈组合（模型+框架+编译器）下的实际能力。

      • te42kyfo/gpu-benches : collection of benchmarks to measure basic GPU capabilities.

    • Scientific Computing Framework

      科学计算框架

      • cuBLAS : Basic Linear Algebra on NVIDIA GPUs. NVIDIA cuBLAS is a GPU-accelerated library for accelerating AI and HPC applications. It includes several API extensions for providing drop-in industry standard BLAS APIs and GEMM APIs with support for fusions that are highly optimized for NVIDIA GPUs. The cuBLAS library also contains extensions for batched operations, execution across multiple GPUs, and mixed- and low-precision execution with additional tuning for the best performance.

      • CUTLASS : CUDA Templates for Linear Algebra Subroutines.

      • MatX : MatX - GPU-Accelerated Numerical Computing in Modern C++. An efficient C++17 GPU numerical computing library with Python-like syntax. nvidia.github.io/MatX

      • DeepGEMM : DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling.

      • MUTLASS : MUSA Templates for Linear Algebra Subroutines.

      • CuPy : CuPy : NumPy & SciPy for GPU. cupy.dev

      • GenericLinearAlgebra.jl : Generic numerical linear algebra in Julia.

      • custos-math : This crate provides CUDA, OpenCL, CPU (and Stack) based matrix operations using custos.

    • Attention and Transformer Framework

      • FlashAttention : Fast and memory-efficient exact attention. "FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness". (arXiv 2022).

      • SageAttention : Quantized Attention that achieves speedups of 2.1-3.1x and 2.7-5.1x compared to FlashAttention2 and xformers, respectively, without lossing end-to-end metrics across various models. "SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration". (arXiv 2024). "SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantization". (arXiv 2024).

      • fla-org/flash-linear-attention : 🚀 Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton.

      • 66RING/tiny-flash-attention : flash attention tutorial written in python, triton, cuda, cutlass.

      • weishengying/tiny-flash-attention : 使用 cutlass 实现 flash-attention 精简版，具有教学意义。

      • jepeake/tiny-flash-attention : flash attention in ~20 lines.

    • Machine Learning Framework

      • cuDNN : The NVIDIA CUDA® Deep Neural Network library (cuDNN) is a GPU-accelerated library of primitives for deep neural networks. cuDNN provides highly tuned implementations for standard routines such as forward and backward convolution, attention, matmul, pooling, and normalization.

      • PyTorch : Tensors and Dynamic neural networks in Python with strong GPU acceleration. pytorch.org

      • MooreThreads/torch_musa : torch_musa is an open source repository based on PyTorch, which can make full use of the super computing power of MooreThreads graphics cards.

      • PaddlePaddle : PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）. www.paddlepaddle.org/

      • flashlight/flashlight : A C++ standalone library for machine learning. fl.readthedocs.io/en/latest/

      • yhwang-hub/dl_model_infer : his is a c++ version of the AI reasoning library. Currently, it only supports the reasoning of the tensorrt model. The follow-up plan supports the c++ reasoning of frameworks such as Openvino, NCNN, and MNN. There are two versions for pre- and post-processing, c++ version and cuda version. It is recommended to use the cuda version., This repository provides accelerated deployment cases of deep learning CV popular models, and cuda c supports dynamic-batch image process, infer, decode, NMS.

      • NVlabs/tiny-cuda-nn : Lightning fast C++/CUDA neural network framework.

      • zjhellofss/KuiperLLama : 《动手自制大模型推理框架》。KuiperLLama 动手自制大模型推理框架，支持LLama2/3和Qwen2.5。校招、秋招、春招、实习好项目，带你从零动手实现支持LLama2/3和Qwen2.5的大模型推理框架。

      • zjhellofss/KuiperInfer : 校招、秋招、春招、实习好项目！带你从零实现一个高性能的深度学习推理库，支持大模型 llama2 、Unet、Yolov5、Resnet等模型的推理。Implement a high-performance deep learning inference library step by step。

      • zjhellofss/kuiperdatawhale : 从零自制深度学习推理框架。

      • MarioSieg/magnetron : (WIP) A small but powerful, homemade PyTorch from scratch. Minimalistic homemade PyTorch alternative, written in C99 and Python.

      • lucasdelimanogueira/PyNorch : Recreating PyTorch from scratch (C/C++, CUDA, NCCL and Python, with multi-GPU support and automatic differentiation!)

    • AI Inference Framework

      AI推理框架
      • LLM Inference and Serving Engine

        • TensorRT : NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT. developer.nvidia.com/tensorrt

        • TensorRT-LLM : TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines. nvidia.github.io/TensorRT-LLM

        • NVIDIA/TensorRT-Model-Optimizer : TensorRT Model Optimizer is a unified library of state-of-the-art model optimization techniques such as quantization, pruning, distillation, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM or TensorRT to optimize inference speed on NVIDIA GPUs. nvidia.github.io/TensorRT-Model-Optimizer

        • Ollama : Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 2, and other large language models. ollama.com

        • vLLM : A high-throughput and memory-efficient inference and serving engine for LLMs. docs.vllm.ai

        • Nano-vLLM : A lightweight vLLM implementation built from scratch.

        • SGLang : SGLang is a fast serving framework for large language models and vision language models. docs.sglang.ai/

        • sgl-project/sgl-learning-materials : Materials for learning SGLang.

        • llama.cpp : LLM inference in C/C++.

        • FlashInfer : FlashInfer: Kernel Library for LLM Serving . flashinfer.ai

        • Chitu（赤兔） : High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability. Chitu (赤兔) 是一个专注于效率、灵活性和可用性的高性能大语言模型推理框架。

        • ztxz16/fastllm : fastllm是c++实现自有算子替代Pytorch的高性能全功能大模型推理库，可以推理Qwen, Llama, Phi等稠密模型，以及DeepSeek, Qwen-moe等moe模型。fastllm是后端无依赖的高性能大模型推理库。同时支持张量并行推理稠密模型和混合模式推理MOE模型，任意10G以上显卡即可推理满血DeepSeek。双路9004/9005服务器+单显卡部署DeepSeek满血满精度原版模型，单并发20tps；INT4量化模型单并发30tps，多并发可达60+。

        • MLC LLM : Universal LLM Deployment Engine with ML Compilation. llm.mlc.ai/

        • KTransformers : A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations. kvcache-ai.github.io/ktransformers/

        • Aphrodite : Large-scale LLM inference engine. aphrodite.pygmalion.chat

        • GPUStack : GPUStack is an open-source GPU cluster manager for running AI models. Manage GPU clusters for running AI models. gpustack.ai

        • Lamini : The Official Python Client for Lamini's API. lamini.ai/

        • triton-inference-server/tensorrtllm_backend : The Triton TensorRT-LLM Backend.

        • datawhalechina/self-llm : 《开源大模型食用指南》基于Linux环境快速部署开源大模型，更适合中国宝宝的部署教程。

        • ninehills/llm-inference-benchmark : LLM Inference benchmark.

        • csbench/csbench : "CS-Bench: A Comprehensive Benchmark for Large Language Models towards Computer Science Mastery". (arXiv 2024).

        • MooreThreads/vllm_musa : A high-throughput and memory-efficient inference and serving engine for LLMs. docs.vllm.ai

        • dusty-nv/jetson-inference : Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.

        • m0dulo/InferSpore : 🌱 A fully independent Large Language Model (LLM) inference engine, built leveraging cuBLAS and cub. 🧩

      • High Performance Kernel Library

        • Liger-Kernel : Efficient Triton Kernels for LLM Training. arxiv.org/pdf/2410.10989

        • FlashInfer : FlashInfer: Kernel Library for LLM Serving . flashinfer.ai

        • DeepGEMM : DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling.

        • FlashMLA : FlashMLA: Efficient MLA Decoding Kernel for Hopper GPUs.

        • DeepEP : DeepEP: an efficient expert-parallel communication library.

        • FlagGems : FlagGems is a high-performance general operator library implemented in OpenAI Triton. It aims to provide a suite of kernel functions to accelerate LLM training and inference.

      • C Implementation

        • llm.c : LLM training in simple, pure C/CUDA. There is no need for 245MB of PyTorch or 107MB of cPython. For example, training GPT-2 (CPU, fp32) is ~1,000 lines of clean code in a single file. It compiles and runs instantly, and exactly matches the PyTorch reference implementation.

        • llama2.c : Inference Llama 2 in one file of pure C. Train the Llama 2 LLM architecture in PyTorch then inference it with one simple 700-line C file (run.c).

      • CPP Implementation

        • llama.cpp : LLM inference in C/C++.

        • gemma.cpp : gemma.cpp is a lightweight, standalone C++ inference engine for the Gemma foundation models from Google.

        • whisper.cpp : High-performance inference of OpenAI's Whisper automatic speech recognition (ASR) model.

        • ztxz16/fastllm : fastllm是c++实现自有算子替代Pytorch的高性能全功能大模型推理库，可以推理Qwen, Llama, Phi等稠密模型，以及DeepSeek, Qwen-moe等moe模型。fastllm是后端无依赖的高性能大模型推理库。同时支持张量并行推理稠密模型和混合模式推理MOE模型，任意10G以上显卡即可推理满血DeepSeek。双路9004/9005服务器+单显卡部署DeepSeek满血满精度原版模型，单并发20tps；INT4量化模型单并发30tps，多并发可达60+。

        • ChatGLM.cpp : C++ implementation of ChatGLM-6B and ChatGLM2-6B.

        • MegEngine/InferLLM : InferLLM is a lightweight LLM model inference framework that mainly references and borrows from the llama.cpp project.

        • DeployAI/nndeploy : nndeploy是一款模型端到端部署框架。以多端推理以及基于有向无环图模型部署为内核，致力为用户提供跨平台、简单易用、高性能的模型部署体验。nndeploy-zh.readthedocs.io/zh/latest/

        • zjhellofss/KuiperInfer (自制深度学习推理框架) : 带你从零实现一个高性能的深度学习推理库，支持llama 、Unet、Yolov5、Resnet等模型的推理。Implement a high-performance deep learning inference library step by step.

        • skeskinen/llama-lite : Embeddings focused small version of Llama NLP model.

        • Const-me/Whisper : High-performance GPGPU inference of OpenAI's Whisper automatic speech recognition (ASR) model.

        • wangzhaode/ChatGLM-MNN : Pure C++, Easy Deploy ChatGLM-6B.

        • davidar/eigenGPT : Minimal C++ implementation of GPT2.

        • Tlntin/Qwen-TensorRT-LLM : 使用TRT-LLM完成对Qwen-7B-Chat实现推理加速。

        • FeiGeChuanShu/trt2023 : NVIDIA TensorRT Hackathon 2023复赛选题：通义千问Qwen-7B用TensorRT-LLM模型搭建及优化。

        • TRT2022/trtllm-llama : ☢️ TensorRT 2023复赛——基于TensorRT-LLM的Llama模型推断加速优化。

      • Mojo Implementation
      • Rust Implementation

      • Zig Implementation

        • llama2.zig : Inference Llama 2 in one file of pure Zig.

        • renerocksai/gpt4all.zig : ZIG build for a terminal-based chat client for an assistant-style large language model with ~800k GPT-3.5-Turbo Generations based on LLaMa.

        • EugenHotaj/zig_inference : Neural Network Inference Engine in Zig.

      • Go Implementation

        • Ollama : Get up and running with Llama 2, Mistral, Gemma, and other large language models. ollama.com

        • go-skynet/LocalAI : 🤖 Self-hosted, community-driven, local OpenAI-compatible API. Drop-in replacement for OpenAI running LLMs on consumer-grade hardware. Free Open Source OpenAI alternative. No GPU required. LocalAI is an API to run ggml compatible models: llama, gpt4all, rwkv, whisper, vicuna, koala, gpt4all-j, cerebras, falcon, dolly, starcoder, and many other. localai.io

    • Distributed and Multi-GPU Framework

      分布式以及多GPU框架

      • NVIDIA/nccl : Optimized primitives for collective multi-GPU communication.

      • NVIDIA/multi-gpu-programming-models : Examples demonstrating available options to program multiple GPUs in a single node or a cluster.

      • wilicc/gpu-burn : Multi-GPU CUDA stress test.

      • SCUDA : SCUDA: GPU-over-IP. SCUDA is a GPU over IP bridge allowing GPUs on remote machines to be attached to CPU-only machines.

    • Robotics Framework

      机器人框架
      • Cupoch : Robotics with GPU computing.

    • ZKP and Web3 Framework

      零知识证明和Web3框架

  • Triton Frameworks

    • Triton Machine Learning Framework

      • BobMcDear/attorch : A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.

    • Triton High Performance Kernel Library

      • Liger-Kernel : Efficient Triton Kernels for LLM Training. arxiv.org/pdf/2410.10989

      • FlagGems : FlagGems is an operator library for large language models implemented in the Triton Language. FlagGems is a high-performance general operator library implemented in OpenAI Triton. It builds on a collection of backend neutral kernels that aims to accelerate LLM training and inference across diverse hardware platforms.

      • harleyszhang/lite_llama : A light llama-like llm inference framework based on the triton kernel.

      • linxihui/dkernel : This repo contains customized CUDA kernels written in OpenAI Triton. As of now, it contains the sparse attention kernel used in phi-3-small models. The sparse attention is also supported in vLLM for efficient inference.

    • Triton Inference Framework

  • MLIR Frameworks

    • MLIR GPU Programming

      • 'gpu' Dialect : This dialect provides middle-level abstractions for launching GPU kernels following a programming model similar to that of CUDA or OpenCL.

      • 'amdgpu' Dialect : The AMDGPU dialect provides wrappers around AMD-specific functionality and LLVM intrinsics.

    • MLIR FFI Bindings

      • pyMLIR : Python interface for MLIR - the Multi-Level Intermediate Representation. pyMLIR is a full Python interface to parse, process, and output MLIR files according to the syntax described in the MLIR documentation. pyMLIR supports the basic dialects and can be extended with other dialects.

    • MLIR Machine Learning Framework

      • Torch-MLIR : The Torch-MLIR project aims to provide first class support from the PyTorch ecosystem to the MLIR ecosystem.

      • ONNX-MLIR : Representation and Reference Lowering of ONNX Models in MLIR Compiler Infrastructure.

      • TPU-MLIR : Machine learning compiler based on MLIR for Sophgo TPU. TPU-MLIR is an open-source machine-learning compiler based on MLIR for TPU. This project provides a complete toolchain, which can convert pre-trained neural networks from different frameworks into binary files bmodel that can be efficiently operated on TPUs.

      • IREE : IREE: Intermediate Representation Execution Environment. A retargetable MLIR-based machine learning compiler and runtime toolkit. iree.dev/

      • ByteIR : The ByteIR Project is a ByteDance model compilation solution. ByteIR includes compiler, runtime, and frontends, and provides an end-to-end model compilation solution. byteir.ai

      • Xilinx/mlir-aie : An MLIR-based toolchain for AMD AI Engine-enabled devices. This repository contains an MLIR-based toolchain for AI Engine-enabled devices, such as AMD Ryzen™ AI and Versal™.

  • HPC Frameworks

    • BLAS : BLAS (Basic Linear Algebra Subprograms). The BLAS (Basic Linear Algebra Subprograms) are routines that provide standard building blocks for performing basic vector and matrix operations. The Level 1 BLAS perform scalar, vector and vector-vector operations, the Level 2 BLAS perform matrix-vector operations, and the Level 3 BLAS perform matrix-matrix operations.

    • LAPACK : LAPACK development repository. LAPACK — Linear Algebra PACKage. LAPACK is written in Fortran 90 and provides routines for solving systems of simultaneous linear equations, least-squares solutions of linear systems of equations, eigenvalue problems, and singular value problems. The associated matrix factorizations (LU, Cholesky, QR, SVD, Schur, generalized Schur) are also provided, as are related computations such as reordering of the Schur factorizations and estimating condition numbers. Dense and banded matrices are handled, but not general sparse matrices. In all areas, similar functionality is provided for real and complex matrices, in both single and double precision.

    • OpenBLAS : OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version. www.openblas.net

    • BLIS : BLAS-like Library Instantiation Software Framework.

    • NumPy : The fundamental package for scientific computing with Python. numpy.org

    • SciPy : SciPy library main repository. SciPy (pronounced "Sigh Pie") is an open-source software for mathematics, science, and engineering. It includes modules for statistics, optimization, integration, linear algebra, Fourier transforms, signal and image processing, ODE solvers, and more. scipy.org

    • Gonum : Gonum is a set of numeric libraries for the Go programming language. It contains libraries for matrices, statistics, optimization, and more. www.gonum.org/

    • YichengDWu/matmul.mojo : High Performance Matrix Multiplication in Pure Mojo 🔥. Matmul.🔥 is a high performance muilti-threaded implimentation of the BLIS algorithm in pure Mojo 🔥.

Applications

  • CUDA Applications

    • Image Preprocess

    • Object Detection

    • Signal Processing

      • Alisah-Ozcan/GPU-FFT : Welcome to the GPU-FFT-Optimization repository! We present cutting-edge algorithms and implementations for optimizing the Fast Fourier Transform (FFT) on Graphics Processing Units (GPUs).

    • Mesh Processing

      • owensgroup/RXMesh : GPU-accelerated triangle mesh processing. RXMesh is a surface triangle mesh data structure and programming model for processing static meshes on the GPU. RXMesh aims at provides a high-performance, generic, and compact data structure that can handle meshes regardless of their quality (e.g., non-manifold).

    • Graph Analytics

Blogs

Videos

Interview

awesome
blas
cublas
cuda
cudnn
cutlass
deepseek
gemm
gpu
hpc
llama
llm
mlir
ptx
pytorch
tensorrt
tensorrt-llm
triton
tvm
vlm

Significant stargazers

Abi Aryan

58 followers · starred Oct 2025

Similar repos

gpu-mode/resource-stream

GPU programming related news and material links

2,351

goabiaryan/awesome-gpu-engineering

GPU Engineering for AI Systems

598

NVIDIA/cutlass

CUDA Templates and Python DSLs for High-Performance Linear Algebra

10,519

wafer-ai/gpu-perf-engineering-resources

A curated resource list for learning AI performance engineering, from GPU fundamentals to…

3,940

qpwo/TensorRT-LLM

mirror for CI comparison

0

NVIDIA/TensorRT-LLM

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs)…

14,756

yumin066/TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs)…

0

BBuf/how-to-optim-algorithm-in-cuda

how to optimize some algorithm in cuda.

3,296

coderonion/awesome-cuda-and-hpc

🚀🚀🚀 This repository lists some awesome public CUDA, cuda-python, cuBLAS, cuDNN, CUTLASS, TensorRT, TensorRT-LLM, Triton, TVM, MLIR, PTX and High Performance Computing (HPC) projects.

518

45 commits

updated Aug 2, 2025

See the code

README

Awesome-CUDA-and-HPC

Awesome

🚀🚀🚀 This repository lists some awesome public CUDA, cuda-python, cuBLAS, CUTLASS, cuDNN, NCCL, TensorRT, TensorRT-LLM, Triton, TileLang, TVM, MLIR, LLVM, PTX and High Performance Computing (HPC) projects.

Contents

Official Version

  • CUDA : CUDA is a parallel computing platform and programming model developed by NVIDIA for general computing on graphical processing units (GPUs).

  • NVIDIA/cuda-python : CUDA Python: Performance meets Productivity. nvidia.github.io/cuda-python/

  • cuBLAS : Basic Linear Algebra on NVIDIA GPUs. NVIDIA cuBLAS is a GPU-accelerated library for accelerating AI and HPC applications. It includes several API extensions for providing drop-in industry standard BLAS APIs and GEMM APIs with support for fusions that are highly optimized for NVIDIA GPUs. The cuBLAS library also contains extensions for batched operations, execution across multiple GPUs, and mixed- and low-precision execution with additional tuning for the best performance.

  • cuDNN : The NVIDIA CUDA Deep Neural Network library (cuDNN) is a GPU-accelerated library of primitives for deep neural networks. cuDNN provides highly tuned implementations for standard routines such as forward and backward convolution, attention, matmul, pooling, and normalization.

  • CUTLASS : CUDA Templates for Linear Algebra Subroutines. CUTLASS is a collection of CUDA C++ template abstractions for implementing high-performance matrix-matrix multiplication (GEMM) and related computations at all levels and scales within CUDA. It incorporates strategies for hierarchical decomposition and data movement similar to those used to implement cuBLAS and cuDNN.

  • TensorRT : NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT. developer.nvidia.com/tensorrt

  • TensorRT-LLM : TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines. nvidia.github.io/TensorRT-LLM

  • Triton : Development repository for the Triton language and compiler. This is the development repository of Triton, a language and compiler for writing highly efficient custom Deep-Learning primitives. The aim of Triton is to provide an open-source environment to write fast code at higher productivity than CUDA, but also with higher flexibility than other existing DSLs. triton-lang.org/

  • TileLang : Domain-specific language designed to streamline the development of high-performance GPU/CPU/Accelerators kernels. Tile Language (tile-lang) is a concise domain-specific language designed to streamline the development of high-performance GPU/CPU kernels (e.g., GEMM, Dequant GEMM, FlashAttention, LinearAttention). By employing a Pythonic syntax with an underlying compiler infrastructure on top of TVM, tile-lang allows developers to focus on productivity without sacrificing the low-level optimizations necessary for state-of-the-art performance. tilelang.com/

  • TVM : Open deep learning compiler stack for cpu, gpu and specialized accelerators. tvm.apache.org/

  • MLIR : Multi-Level Intermediate Representation Compiler Framework. The MLIR project is a novel approach to building reusable and extensible compiler infrastructure. MLIR aims to address software fragmentation, improve compilation for heterogeneous hardware, significantly reduce the cost of building domain specific compilers, and aid in connecting existing compilers together.

  • LLVM : The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.

  • PTX : PTX, a low-level parallel thread execution virtual machine and instruction set architecture (ISA).

Awesome List

Learning Resources

Frameworks

  • CUDA Frameworks

    • GPU Interface

      GPU接口
      • CPP Version

        • CCCL : CUDA C++ Core Libraries. The concept for the CUDA C++ Core Libraries (CCCL) grew organically out of the Thrust, CUB, and libcudacxx projects that were developed independently over the years with a similar goal: to provide high-quality, high-performance, and easy-to-use C++ abstractions for CUDA developers.

        • HIP : HIP: C++ Heterogeneous-Compute Interface for Portability. HIP is a C++ Runtime API and Kernel Language that allows developers to create portable applications for AMD and NVIDIA GPUs from single source code. rocmdocs.amd.com/projects/HIP/

      • Python Version
      • Rust Version
      • Julia Version

    • Performance Benchmark

      • NVIDIA/nvbench : CUDA Kernel Benchmarking Library.

      • FlagPerf : FlagPerf is an open-source software platform for benchmarking AI chips. FlagPerf是智源研究院联合AI硬件厂商共建的一体化AI硬件评测引擎，旨在建立以产业实践为导向的指标体系，评测AI硬件在软件栈组合（模型+框架+编译器）下的实际能力。

      • te42kyfo/gpu-benches : collection of benchmarks to measure basic GPU capabilities.

    • Scientific Computing Framework

      科学计算框架

      • cuBLAS : Basic Linear Algebra on NVIDIA GPUs. NVIDIA cuBLAS is a GPU-accelerated library for accelerating AI and HPC applications. It includes several API extensions for providing drop-in industry standard BLAS APIs and GEMM APIs with support for fusions that are highly optimized for NVIDIA GPUs. The cuBLAS library also contains extensions for batched operations, execution across multiple GPUs, and mixed- and low-precision execution with additional tuning for the best performance.

      • CUTLASS : CUDA Templates for Linear Algebra Subroutines.

      • MatX : MatX - GPU-Accelerated Numerical Computing in Modern C++. An efficient C++17 GPU numerical computing library with Python-like syntax. nvidia.github.io/MatX

      • DeepGEMM : DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling.

      • MUTLASS : MUSA Templates for Linear Algebra Subroutines.

      • CuPy : CuPy : NumPy & SciPy for GPU. cupy.dev

      • GenericLinearAlgebra.jl : Generic numerical linear algebra in Julia.

      • custos-math : This crate provides CUDA, OpenCL, CPU (and Stack) based matrix operations using custos.

    • Attention and Transformer Framework

      • FlashAttention : Fast and memory-efficient exact attention. "FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness". (arXiv 2022).

      • SageAttention : Quantized Attention that achieves speedups of 2.1-3.1x and 2.7-5.1x compared to FlashAttention2 and xformers, respectively, without lossing end-to-end metrics across various models. "SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration". (arXiv 2024). "SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantization". (arXiv 2024).

      • fla-org/flash-linear-attention : 🚀 Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton.

      • 66RING/tiny-flash-attention : flash attention tutorial written in python, triton, cuda, cutlass.

      • weishengying/tiny-flash-attention : 使用 cutlass 实现 flash-attention 精简版，具有教学意义。

      • jepeake/tiny-flash-attention : flash attention in ~20 lines.

    • Machine Learning Framework

      • cuDNN : The NVIDIA CUDA® Deep Neural Network library (cuDNN) is a GPU-accelerated library of primitives for deep neural networks. cuDNN provides highly tuned implementations for standard routines such as forward and backward convolution, attention, matmul, pooling, and normalization.

      • PyTorch : Tensors and Dynamic neural networks in Python with strong GPU acceleration. pytorch.org

      • MooreThreads/torch_musa : torch_musa is an open source repository based on PyTorch, which can make full use of the super computing power of MooreThreads graphics cards.

      • PaddlePaddle : PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）. www.paddlepaddle.org/

      • flashlight/flashlight : A C++ standalone library for machine learning. fl.readthedocs.io/en/latest/

      • yhwang-hub/dl_model_infer : his is a c++ version of the AI reasoning library. Currently, it only supports the reasoning of the tensorrt model. The follow-up plan supports the c++ reasoning of frameworks such as Openvino, NCNN, and MNN. There are two versions for pre- and post-processing, c++ version and cuda version. It is recommended to use the cuda version., This repository provides accelerated deployment cases of deep learning CV popular models, and cuda c supports dynamic-batch image process, infer, decode, NMS.

      • NVlabs/tiny-cuda-nn : Lightning fast C++/CUDA neural network framework.

      • zjhellofss/KuiperLLama : 《动手自制大模型推理框架》。KuiperLLama 动手自制大模型推理框架，支持LLama2/3和Qwen2.5。校招、秋招、春招、实习好项目，带你从零动手实现支持LLama2/3和Qwen2.5的大模型推理框架。

      • zjhellofss/KuiperInfer : 校招、秋招、春招、实习好项目！带你从零实现一个高性能的深度学习推理库，支持大模型 llama2 、Unet、Yolov5、Resnet等模型的推理。Implement a high-performance deep learning inference library step by step。

      • zjhellofss/kuiperdatawhale : 从零自制深度学习推理框架。

      • MarioSieg/magnetron : (WIP) A small but powerful, homemade PyTorch from scratch. Minimalistic homemade PyTorch alternative, written in C99 and Python.

      • lucasdelimanogueira/PyNorch : Recreating PyTorch from scratch (C/C++, CUDA, NCCL and Python, with multi-GPU support and automatic differentiation!)

    • AI Inference Framework

      AI推理框架
      • LLM Inference and Serving Engine

        • TensorRT : NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT. developer.nvidia.com/tensorrt

        • TensorRT-LLM : TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines. nvidia.github.io/TensorRT-LLM

        • NVIDIA/TensorRT-Model-Optimizer : TensorRT Model Optimizer is a unified library of state-of-the-art model optimization techniques such as quantization, pruning, distillation, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM or TensorRT to optimize inference speed on NVIDIA GPUs. nvidia.github.io/TensorRT-Model-Optimizer

        • Ollama : Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 2, and other large language models. ollama.com

        • vLLM : A high-throughput and memory-efficient inference and serving engine for LLMs. docs.vllm.ai

        • Nano-vLLM : A lightweight vLLM implementation built from scratch.

        • SGLang : SGLang is a fast serving framework for large language models and vision language models. docs.sglang.ai/

        • sgl-project/sgl-learning-materials : Materials for learning SGLang.

        • llama.cpp : LLM inference in C/C++.

        • FlashInfer : FlashInfer: Kernel Library for LLM Serving . flashinfer.ai

        • Chitu（赤兔） : High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability. Chitu (赤兔) 是一个专注于效率、灵活性和可用性的高性能大语言模型推理框架。

        • ztxz16/fastllm : fastllm是c++实现自有算子替代Pytorch的高性能全功能大模型推理库，可以推理Qwen, Llama, Phi等稠密模型，以及DeepSeek, Qwen-moe等moe模型。fastllm是后端无依赖的高性能大模型推理库。同时支持张量并行推理稠密模型和混合模式推理MOE模型，任意10G以上显卡即可推理满血DeepSeek。双路9004/9005服务器+单显卡部署DeepSeek满血满精度原版模型，单并发20tps；INT4量化模型单并发30tps，多并发可达60+。

        • MLC LLM : Universal LLM Deployment Engine with ML Compilation. llm.mlc.ai/

        • KTransformers : A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations. kvcache-ai.github.io/ktransformers/

        • Aphrodite : Large-scale LLM inference engine. aphrodite.pygmalion.chat

        • GPUStack : GPUStack is an open-source GPU cluster manager for running AI models. Manage GPU clusters for running AI models. gpustack.ai

        • Lamini : The Official Python Client for Lamini's API. lamini.ai/

        • triton-inference-server/tensorrtllm_backend : The Triton TensorRT-LLM Backend.

        • datawhalechina/self-llm : 《开源大模型食用指南》基于Linux环境快速部署开源大模型，更适合中国宝宝的部署教程。

        • ninehills/llm-inference-benchmark : LLM Inference benchmark.

        • csbench/csbench : "CS-Bench: A Comprehensive Benchmark for Large Language Models towards Computer Science Mastery". (arXiv 2024).

        • MooreThreads/vllm_musa : A high-throughput and memory-efficient inference and serving engine for LLMs. docs.vllm.ai

        • dusty-nv/jetson-inference : Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.

        • m0dulo/InferSpore : 🌱 A fully independent Large Language Model (LLM) inference engine, built leveraging cuBLAS and cub. 🧩

      • High Performance Kernel Library

        • Liger-Kernel : Efficient Triton Kernels for LLM Training. arxiv.org/pdf/2410.10989

        • FlashInfer : FlashInfer: Kernel Library for LLM Serving . flashinfer.ai

        • DeepGEMM : DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling.

        • FlashMLA : FlashMLA: Efficient MLA Decoding Kernel for Hopper GPUs.

        • DeepEP : DeepEP: an efficient expert-parallel communication library.

        • FlagGems : FlagGems is a high-performance general operator library implemented in OpenAI Triton. It aims to provide a suite of kernel functions to accelerate LLM training and inference.

      • C Implementation

        • llm.c : LLM training in simple, pure C/CUDA. There is no need for 245MB of PyTorch or 107MB of cPython. For example, training GPT-2 (CPU, fp32) is ~1,000 lines of clean code in a single file. It compiles and runs instantly, and exactly matches the PyTorch reference implementation.

        • llama2.c : Inference Llama 2 in one file of pure C. Train the Llama 2 LLM architecture in PyTorch then inference it with one simple 700-line C file (run.c).

      • CPP Implementation

        • llama.cpp : LLM inference in C/C++.

        • gemma.cpp : gemma.cpp is a lightweight, standalone C++ inference engine for the Gemma foundation models from Google.

        • whisper.cpp : High-performance inference of OpenAI's Whisper automatic speech recognition (ASR) model.

        • ztxz16/fastllm : fastllm是c++实现自有算子替代Pytorch的高性能全功能大模型推理库，可以推理Qwen, Llama, Phi等稠密模型，以及DeepSeek, Qwen-moe等moe模型。fastllm是后端无依赖的高性能大模型推理库。同时支持张量并行推理稠密模型和混合模式推理MOE模型，任意10G以上显卡即可推理满血DeepSeek。双路9004/9005服务器+单显卡部署DeepSeek满血满精度原版模型，单并发20tps；INT4量化模型单并发30tps，多并发可达60+。

        • ChatGLM.cpp : C++ implementation of ChatGLM-6B and ChatGLM2-6B.

        • MegEngine/InferLLM : InferLLM is a lightweight LLM model inference framework that mainly references and borrows from the llama.cpp project.

        • DeployAI/nndeploy : nndeploy是一款模型端到端部署框架。以多端推理以及基于有向无环图模型部署为内核，致力为用户提供跨平台、简单易用、高性能的模型部署体验。nndeploy-zh.readthedocs.io/zh/latest/

        • zjhellofss/KuiperInfer (自制深度学习推理框架) : 带你从零实现一个高性能的深度学习推理库，支持llama 、Unet、Yolov5、Resnet等模型的推理。Implement a high-performance deep learning inference library step by step.

        • skeskinen/llama-lite : Embeddings focused small version of Llama NLP model.

        • Const-me/Whisper : High-performance GPGPU inference of OpenAI's Whisper automatic speech recognition (ASR) model.

        • wangzhaode/ChatGLM-MNN : Pure C++, Easy Deploy ChatGLM-6B.

        • davidar/eigenGPT : Minimal C++ implementation of GPT2.

        • Tlntin/Qwen-TensorRT-LLM : 使用TRT-LLM完成对Qwen-7B-Chat实现推理加速。

        • FeiGeChuanShu/trt2023 : NVIDIA TensorRT Hackathon 2023复赛选题：通义千问Qwen-7B用TensorRT-LLM模型搭建及优化。

        • TRT2022/trtllm-llama : ☢️ TensorRT 2023复赛——基于TensorRT-LLM的Llama模型推断加速优化。

      • Mojo Implementation
      • Rust Implementation

      • Zig Implementation

        • llama2.zig : Inference Llama 2 in one file of pure Zig.

        • renerocksai/gpt4all.zig : ZIG build for a terminal-based chat client for an assistant-style large language model with ~800k GPT-3.5-Turbo Generations based on LLaMa.

        • EugenHotaj/zig_inference : Neural Network Inference Engine in Zig.

      • Go Implementation

        • Ollama : Get up and running with Llama 2, Mistral, Gemma, and other large language models. ollama.com

        • go-skynet/LocalAI : 🤖 Self-hosted, community-driven, local OpenAI-compatible API. Drop-in replacement for OpenAI running LLMs on consumer-grade hardware. Free Open Source OpenAI alternative. No GPU required. LocalAI is an API to run ggml compatible models: llama, gpt4all, rwkv, whisper, vicuna, koala, gpt4all-j, cerebras, falcon, dolly, starcoder, and many other. localai.io

    • Distributed and Multi-GPU Framework

      分布式以及多GPU框架

      • NVIDIA/nccl : Optimized primitives for collective multi-GPU communication.

      • NVIDIA/multi-gpu-programming-models : Examples demonstrating available options to program multiple GPUs in a single node or a cluster.

      • wilicc/gpu-burn : Multi-GPU CUDA stress test.

      • SCUDA : SCUDA: GPU-over-IP. SCUDA is a GPU over IP bridge allowing GPUs on remote machines to be attached to CPU-only machines.

    • Robotics Framework

      机器人框架
      • Cupoch : Robotics with GPU computing.

    • ZKP and Web3 Framework

      零知识证明和Web3框架

  • Triton Frameworks

    • Triton Machine Learning Framework

      • BobMcDear/attorch : A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.

    • Triton High Performance Kernel Library

      • Liger-Kernel : Efficient Triton Kernels for LLM Training. arxiv.org/pdf/2410.10989

      • FlagGems : FlagGems is an operator library for large language models implemented in the Triton Language. FlagGems is a high-performance general operator library implemented in OpenAI Triton. It builds on a collection of backend neutral kernels that aims to accelerate LLM training and inference across diverse hardware platforms.

      • harleyszhang/lite_llama : A light llama-like llm inference framework based on the triton kernel.

      • linxihui/dkernel : This repo contains customized CUDA kernels written in OpenAI Triton. As of now, it contains the sparse attention kernel used in phi-3-small models. The sparse attention is also supported in vLLM for efficient inference.

    • Triton Inference Framework

  • MLIR Frameworks

    • MLIR GPU Programming

      • 'gpu' Dialect : This dialect provides middle-level abstractions for launching GPU kernels following a programming model similar to that of CUDA or OpenCL.

      • 'amdgpu' Dialect : The AMDGPU dialect provides wrappers around AMD-specific functionality and LLVM intrinsics.

    • MLIR FFI Bindings

      • pyMLIR : Python interface for MLIR - the Multi-Level Intermediate Representation. pyMLIR is a full Python interface to parse, process, and output MLIR files according to the syntax described in the MLIR documentation. pyMLIR supports the basic dialects and can be extended with other dialects.

    • MLIR Machine Learning Framework

      • Torch-MLIR : The Torch-MLIR project aims to provide first class support from the PyTorch ecosystem to the MLIR ecosystem.

      • ONNX-MLIR : Representation and Reference Lowering of ONNX Models in MLIR Compiler Infrastructure.

      • TPU-MLIR : Machine learning compiler based on MLIR for Sophgo TPU. TPU-MLIR is an open-source machine-learning compiler based on MLIR for TPU. This project provides a complete toolchain, which can convert pre-trained neural networks from different frameworks into binary files bmodel that can be efficiently operated on TPUs.

      • IREE : IREE: Intermediate Representation Execution Environment. A retargetable MLIR-based machine learning compiler and runtime toolkit. iree.dev/

      • ByteIR : The ByteIR Project is a ByteDance model compilation solution. ByteIR includes compiler, runtime, and frontends, and provides an end-to-end model compilation solution. byteir.ai

      • Xilinx/mlir-aie : An MLIR-based toolchain for AMD AI Engine-enabled devices. This repository contains an MLIR-based toolchain for AI Engine-enabled devices, such as AMD Ryzen™ AI and Versal™.

  • HPC Frameworks

    • BLAS : BLAS (Basic Linear Algebra Subprograms). The BLAS (Basic Linear Algebra Subprograms) are routines that provide standard building blocks for performing basic vector and matrix operations. The Level 1 BLAS perform scalar, vector and vector-vector operations, the Level 2 BLAS perform matrix-vector operations, and the Level 3 BLAS perform matrix-matrix operations.

    • LAPACK : LAPACK development repository. LAPACK — Linear Algebra PACKage. LAPACK is written in Fortran 90 and provides routines for solving systems of simultaneous linear equations, least-squares solutions of linear systems of equations, eigenvalue problems, and singular value problems. The associated matrix factorizations (LU, Cholesky, QR, SVD, Schur, generalized Schur) are also provided, as are related computations such as reordering of the Schur factorizations and estimating condition numbers. Dense and banded matrices are handled, but not general sparse matrices. In all areas, similar functionality is provided for real and complex matrices, in both single and double precision.

    • OpenBLAS : OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version. www.openblas.net

    • BLIS : BLAS-like Library Instantiation Software Framework.

    • NumPy : The fundamental package for scientific computing with Python. numpy.org

    • SciPy : SciPy library main repository. SciPy (pronounced "Sigh Pie") is an open-source software for mathematics, science, and engineering. It includes modules for statistics, optimization, integration, linear algebra, Fourier transforms, signal and image processing, ODE solvers, and more. scipy.org

    • Gonum : Gonum is a set of numeric libraries for the Go programming language. It contains libraries for matrices, statistics, optimization, and more. www.gonum.org/

    • YichengDWu/matmul.mojo : High Performance Matrix Multiplication in Pure Mojo 🔥. Matmul.🔥 is a high performance muilti-threaded implimentation of the BLIS algorithm in pure Mojo 🔥.

Applications

  • CUDA Applications

    • Image Preprocess

    • Object Detection

    • Signal Processing

      • Alisah-Ozcan/GPU-FFT : Welcome to the GPU-FFT-Optimization repository! We present cutting-edge algorithms and implementations for optimizing the Fast Fourier Transform (FFT) on Graphics Processing Units (GPUs).

    • Mesh Processing

      • owensgroup/RXMesh : GPU-accelerated triangle mesh processing. RXMesh is a surface triangle mesh data structure and programming model for processing static meshes on the GPU. RXMesh aims at provides a high-performance, generic, and compact data structure that can handle meshes regardless of their quality (e.g., non-manifold).

    • Graph Analytics

Blogs

Videos

Interview

awesome
blas
cublas
cuda
cudnn
cutlass
deepseek
gemm
gpu
hpc
llama
llm
mlir
ptx
pytorch
tensorrt
tensorrt-llm
triton
tvm
vlm

Similar repos

gpu-mode/resource-stream

GPU programming related news and material links

2,351

goabiaryan/awesome-gpu-engineering

GPU Engineering for AI Systems

598

NVIDIA/cutlass

CUDA Templates and Python DSLs for High-Performance Linear Algebra

10,519

wafer-ai/gpu-perf-engineering-resources

A curated resource list for learning AI performance engineering, from GPU fundamentals to…

3,940

qpwo/TensorRT-LLM

mirror for CI comparison

0

NVIDIA/TensorRT-LLM

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs)…

14,756

yumin066/TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs)…

0

BBuf/how-to-optim-algorithm-in-cuda

how to optimize some algorithm in cuda.

3,296

Significant stargazers

Abi Aryan

58 followers · starred Oct 2025