Shadowgraph
TrendingAbout
Sign in

PrunaAI/awesome-ai-efficiency

A curated list of materials on AI efficiency

237

109 commits

updated Aug 6, 2026

See the code

README

🌟 Awesome AI Efficiency 🌟

Awesome MIT License

A curated list of resources dedicated to enhancing efficiency in AI systems. This repository covers a wide range of topics essential for optimizing AI models and processes, aiming to make AI faster, cheaper, smaller, and greener!

Topics Summary 🎨

TopicDescriptionTopics
QuantizationReducing precision of AI models without lossQuantization
PruningRemoving unnecessary model parameters for efficiencyPruning
CachingStoring computation results for faster reuseCaching
DistillationTransferring knowledge from a large model to a smaller oneDistillation
FactorizationBreaking down complex models into simpler, efficient componentsFactorization
CompilationOptimizing model code for specific hardware and environmentsCompilation
Parameter-Efficient Fine-tuningLearning a subset of parametersPEFT
Speculative DecodingDecoding with batchesSpecDec
HardwareLeveraging specialized hardware for faster model executionHardware
TrainingTechniques for making model training faster and more efficientTraining
InferenceOptimizing the speed and resource usage during model inferenceInference
SustainabilityStrategies to reduce the environmental impact of AI systemsSustainability
ScalabilityApproaches for scaling AI models and infrastructure efficientlyScalability

If you find this list helpful, give it a ⭐ on GitHub, share it, and contribute by submitting a pull request or issue!

Table of Contents

Facts 📊

  • 1% of Dutch electricity: Microsoft hyperscale data center in Middenmeer consumes over 1% of all Dutch electricity (Source, 2026)
  • 3-40Wh: Amount of energy consumed for one small to long ChatGPT query (Source, 2025)
  • 1L: Estimated amount of water required for 20-100 ChatGPT queries (Source, 2025)
  • 2 nuclear plants: Number of nuclear plants to constantly work ot generate enough energy if 80M people generate 5 pages per day (Source, 2025)
  • 1 smartphone charge: Amount of energy required to AI generate a couple of images or run a few thousands inference with an LLM (Source, 2024)
  • >10s: Time requried to generate 1 HD image with Flux on H100 or to generate 100 tokens with Llama 3 on T4 (Source and Source, 2024)
  • 7-10 smartphone charges: Amount of energy required to AI generate one video with Wan 2.1 (Source)
  • 61,848.0x: Difference between the highest and lowest energy use in energy leaderboard for AI models (Source, 2025).
  • 1,300MWh: GPT-3, for example, is estimated to use just under 1,300 megawatt hours (MWh) of electricity; about as much power as consumed annually by 130 US homes (Source, 2024)
  • 800M users/week: Amount of users using ChatGPT per week in 2025 (Source)
  • 1B messages/day: Amount of ChatGPT queries per day in 2025 (Source)
  • +160%: Expected increase of data center power consumption by 2030 (Source)
  • x3.8: Hardware acceleration (GPU/TPU) reduces energy consumption by a factor of 3.8 compared with the CPU, for the same task, but also reduces response time by up to 39% (Source)
  • x18:The carbon footprint of a task can vary by a factor of 18 depending on the model, framework and backend used (Source)

Tools 🛠️

  • :heart: Pruna :heart:: A package to make AI models faster, smaller, faster, greener by combining compression methods (incl. quantization, pruning, caching, compilation, distillation...) on various hardware.
  • TensorRT: High-performance deep learning inference library for NVIDIA GPUs.
  • ONNX: Open Neural Network Exchange format for interoperability among deep learning frameworks.
  • Code Carbon: Library to track energy and carbon efficiency of various hardware.
  • LLM Perf: A framework for benchmarking the performance of transformers models with different hardwares, backends and optimizations.
  • ML.ENERGY Leaderboard: An initiative to benchmark energy efficiency of AI models.
  • AI Energy Score: An initiative to establish comparable energy efficiency ratings for AI models, helping the industry make informed decisions about sustainability in AI development.
  • Model Optimization Toolkit: TensorFlow toolkit for optimizing machine learning models for deployment and execution.
  • Green Coding: LLM service that you can use to prompt most open source models and see the resource usage.
  • EcoLogits: EcoLogits is a python library that tracks the energy consumption and environmental footprint of using generative AI models through APIs.
  • Perplexity Kernels: GPU kernels by Perplexity.
  • Fast Tokenizer: Fast tokenizer is an efficient and optimized tokenizer engine for llm inference serving.
  • WeightWatcher: WeightWatcher (WW) is an open-source, diagnostic tool for analyzing Deep Neural Networks (DNN), without needing access to training or even test data..
  • Cockpit: A Practical Debugging Tool for Training Deep Neural Networks.
  • Electrictiy Map: A live map showing the origin of the electricity in world regions and their CO2 intensity.
  • MLCA: A tool for machine learning life cycle assessment.
  • TritonParse: A visualization and analysis tool for Triton IR files, designed to help developers analyze, debug, and understand Triton kernel compilation processes.
  • Routing on Random Forests: A framework for training and serving LLM based on random forest-based routers, thus allowing to optimize for costs.
  • LLMCache: An LLM serving engine extension to reduce time-to-first-token and increase throughput, especially under long-context scenarios.
  • ExLlamaV3: An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs.
  • FlashDeBERTa: Flash implementation of DeBERTa disentangled attention mechanism.
  • QuACK: An assortiment of Kernels for GPUs.
  • Pi-Quant: An assortiment of Kernels for CPUs.
  • pplx-kernels: An assortiment of Kernels for GPUs.
  • LMCache: an LLM serving engine extension to reduce TTFT and increase throughput, especially under long-context scenarios, by optimizing the KV caches.
  • FastWan: a family of video generation models trained via “sparse distillation”.
  • FastVideo: A unified inference and post-training framework for accelerated video generation.
  • GEAK Agent: This is an LLM-based multi-agent framework, which can generate functional and efficient gpu kernels automatically.
  • Fused Kernel Library: Implementation of a package that allows user to define GPU kernel fusion, for non CUDA programmers.
  • FlashPack: a library with efficient Disk-to-GPU Tensor loading.
  • Humming: a library with efficient kernels.
  • NVIDIA Model Optimizer: Model optimization for efficient deployment.
  • GigaToken: Language model tokenization at GB/s.
  • FlashLib: Fast and memory-efficient classical machine learning operators.
  • Mage: A lightweight 4B multimodal model family for efficient visual understanding and generation.
  • ltx2-simple-optims: Simple optimizations for LTX2.3, including a custom QK-Norm-RoPE kernel.

News Articles 📰

Reports 📈

Research Articles 📄

PaperYearVenueTags
D2Cache: Second-Order Delta Caching for Higher Video Diffusion Acceleration2026CVPRCaching Inference
Budget-Constrained Step-Level Diffusion Caching2026ICMLCaching
ReQAT: Achieving Full-Precision Reasoning Accuracy with 4-bit Floating-Point Quantization-Aware Training2026ICMLQuantization Training
Distribution Matching Distillation Meets Reinforcement Learning2026ECCVDistillation Training
DiffusionBlocks: Block-wise Neural Network Training via Diffusion Interpretation2026ICLRTraining
Parallel Decoding Distillation for Fast Image and Video Generation2026NoneDistillation Inference
Sol-Attn: Accelerating Video Generation Inference via On-the-Fly Attention Sparsification2026NoneInference
SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation2026NoneInference Caching
OrbitQuant: Data-Agnostic Quantization for Image and Video Diffusion Transformers2026NoneQuantization
Multi-Resolution Flow Matching: Training-Free Diffusion Acceleration via Staged Sampling2026NoneInference
Post-Training Pruning for Diffusion Transformers2026NonePruning
Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generation2026NoneInference Compilation
Training-free sparse attention based on cumulative energy filtering2026NoneInference
Adaptive Inference-Time Scaling via Early-Step Latent Verification for Image Editing2026NoneInference
HiLo-Token: Input-Adaptive High-Low Frequency Token Compression for Efficient Image Editing2026NonePruning Inference
TetherCache: Stabilizing Autoregressive Long-Form Video Generation with Gated Recall and Trusted Alignment2026NoneCaching Inference
High-Fidelity Two-Step Image Generation via Teacher-Aligned End-to-End Distillation2026NoneDistillation
RhymeFlow: Training-Free Acceleration for Video Generation with Asynchronous Denoising Flow Scheduling2026NoneInference
ReCache: Learning Budget-Aware Caching Schedules for Diffusion Models via REINFORCE2026NoneCaching
Qwen-Image-Flash: Beyond Objective Design2026NoneDistillation Inference
KVarN: Variance-Normalized KV-Cache Quantization Mitigates Error Accumulation in Reasoning Tasks2026NoneQuantization Caching
Dual-Rate Diffusion: Accelerating diffusion models with an interleaved heavy-light network2026NoneInference
ElasticDiT: Efficient Diffusion Transformers via Elastic Architecture and Sparse Attention for High-Resolution Image Generation on Mobile Devices2026NoneInference Pruning
Continuous-Time Distribution Matching for Few-Step Diffusion Distillation2026NoneDistillation
Statistically-Lossless Quantization of Large Language Models2026NoneQuantization
YOSE: You Only Select Essential Tokens for Efficient DiT-based Video Object Removal2026CVPRPruning Inference
Token Reduction via Local and Global Contexts Optimization for Efficient Video Large Language Models2026CVPRPruning
MASQuant: Modality-Aware Smoothing Quantization for Multimodal Large Language Models2026CVPRQuantization
SegQuant: A Semantics-Aware and Generalizable Quantization Framework for Diffusion Models2026CVPRQuantization
ResCa: Residual Caching for Diffusion Transformers Acceleration2026CVPRCaching
DisCa: Accelerating Video Diffusion Transformers with Distillation-Compatible Learnable Feature Caching2026CVPRCaching Distillation
SenCache: Accelerating Diffusion Model Inference via Sensitivity-Aware Caching2026CVPRCaching
Accelerating Diffusion-based Video Editing via Heterogeneous Caching: Beyond Full Computing at Sampled Denoising Timestep2026CVPRCaching
d2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching2026CVPRCaching Inference
Training-free, Perceptually Consistent Low-Resolution Previews with High-Resolution Image for Efficient Workflows of Diffusion Models2026CVPRInference
LinVideo: A Post-Training Framework towards O(n) Attention in Efficient Video Generation2026CVPRInference
An Efficient Token Compression Framework for Visual Object Tracking2026CVPRPruning Inference
VMonarch: Efficient Video Diffusion Transformers with Structured Attention2026CVPRInference
Otil: Accelerating Diffusion Model Inference via Communication-Efficient Multi-GPU Parallelism2026CVPRInference Scalability
NanoSD: Edge Efficient Foundation Model for Real Time Image Restoration2026CVPRDistillation Inference
Flash-DMD: Towards High-Fidelity Few-Step Image Generation with Efficient Distillation and Joint Reinforcement Learning2026CVPRDistillation
Attention-aware Inference Optimizations for Large Vision-Language Models with Memory-efficient Decoding2026CVPRCaching Inference
Attention Surgery: An Efficient Recipe to Linearize Your Video Diffusion Transformer2026CVPRInference Distillation
Trainable Log-linear Sparse Attention for Efficient Diffusion Transformers2026CVPRInference Training
Rethinking Asymmetric Quantization: Hidden Symmetry in Vision Model Weights2026CVPRQuantization
LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuning2026CVPR
S2FT: Parameter-Efficient Fine-Tuning in Sparse Spectrum Domain2026CVPRPEFT
vAttention: Verified Sparse Attention via Sampling2026ICLRInference
PLoP: Precise LoRA Placement for Efficient Finetuning of Large Models2026ICLRPEFT
How to train data-efficient LLMs2026ICLRTraining
ERTACache: Error Rectification and Timesteps Adjustment for Efficient Diffusion2026ICLRCaching
Fewer Battles, More Gain: An Information-Efficient Framework for Arena-based LLM Evaluation2026ICLRScalability
PASER: Post-Training Data Selection for Efficient Pruned Large Language Model Recovery2026ICLRPruning
Efficient Orthogonal Fine-Tuning with Principal Subspace Adaptation2026ICLRPEFT
MicroMix: Efficient Mixed-Precision Quantization with Microscaling Formats for Large Language Models2026ICLRQuantization
Fast-dLLM v2: Efficient Block-Diffusion LLM2026ICLRInference Caching
DSA: Efficient Inference For Video Generation Models via Distributed Sparse Attention2026ICLRInference
Q&C: When Quantization Meets Cache in Efficient Generation2026ICLRQuantization Caching
Plan and Budget: Effective and Efficient Test-Time Scaling on Reasoning Large Language Models2026ICLRInference
Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling2026ICLRSpecDec
GradPruner: Gradient-guided Layer Pruning Enabling Efficient Fine-Tuning and Inference for LLMs2026ICLRPruning Training
ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skipping2026ICLRInference Caching
Channel-Aware Mixed-Precision Quantization for Efficient Long-Context Inference2026ICLRQuantization Caching
Test-Time Iterative Error Correction for Efficient Diffusion Models2026ICLRInference
FideDiff: Efficient Diffusion Model for High-Fidelity Image Motion Deblurring2026ICLRInference
PreciseCache: Precise Feature Caching for Efficient and High-fidelity Video Generation2026ICLRCaching
Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs2026ICLRInference
SPRINT: Sparse-Dense Residual Fusion for Efficient Diffusion Transformers2026ICLRTraining Inference
ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inference2026ICLRQuantization
FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion2026ICLRCaching Inference
SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer2026ICLRInference Caching
Diffusion Models as Dataset Distillation Priors2026ICLRDistillation
Scale-wise Distillation of Diffusion Models2026ICLRDistillation
Streaming Autoregressive Video Generation via Diagonal Distillation2026ICLRDistillation
pi-Flow: Policy-Based Few-Step Generation via Imitation Distillation2026ICLRDistillation
PISA: Piecewise Sparse Attention Is Wiser for Efficient Diffusion Transformers2026ICMLInference
Beyond Fixed Formulas: Data-Driven Linear Predictor for Efficient Diffusion Models2026NoneCaching
RAPID: Reusing Attention Sparsity with Inter-step Adaptation for Efficient Video Diffusion2026CVPRCaching Inference
AnyFlow: Any-Step Video Diffusion Model with On-Policy Flow Map Distillation2026NoneDistillation
Towards Resource-Efficient LLMs: End-to-End Energy Accounting of Distillation Pipelines2026ICMLSustainability Distillation
Spectral Progressive Diffusion for Efficient Image and Video Generation2026NoneInference
SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer2026NoneInference Quantization
Sparser, Faster, Lighter Transformer Language Models2026NonePruning
Stream-T1: Test-Time Scaling for Streaming Video Generation2026NoneInference
Optimizing Few-Step Generation with Adaptive Matching Distillation2026ICMLDistillation
Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding2026NoneSpecDec Inference
Bonsai Image 4B: Low-Bit Diffusion on Apple Silicon and Consumer GPUs2026NoneQuantization Inference
Where Do the Joules Go? Diagnosing Inference Energy Consumption2026NoneSustainability Inference
Generative Modeling via Drifting2026NoneInference
Just on Time: Token-Level Early Stopping for Diffusion Language Models2026NoneInference
From Sketch to Fresco: Efficient Diffusion Transformer with Progressive Resolution2026NoneInference
Forecast the Principal, Stabilize the Residual: Subspace-Aware Feature Caching for Efficient Diffusion Transformers2026NoneCaching
ArcFlow: Unleashing 2-Step Text-to-Image Generation via High-Precision Non-Linear Flow Distillation2026NoneDistillation
FlowCast: Trajectory Forecasting for Scalable Zero-Cost Speculative Flow Matching2026NoneInference
Learning to Reason in 13 Parameters2026NonePEFT
Quant VideoGen: Auto-Regressive Long Video Generation via 2-Bit KV-Cache Quantization2026NoneQuantization
PixelRush: Ultra-Fast, Training-Free High-Resolution Image Generation via One-step Diffusion2026NoneInference
Omni-Video 2: Scaling MLLM-Conditioned Diffusion for Unified Video Generation and Editing2026None
Iterative Refinement Improves Compositional Image Generation2026NoneInference
LightCache: Memory-Efficient, Training-Free Acceleration for Video Generation2025NoneCaching
CoIn: Coverage and Informativeness-Guided Token Reduction for Efficient Large Multimodal Models2025NonePruning
Efficient Zero-Shot Inpainting with Decoupled Diffusion Guidance2025NoneInference
ComfyGPT: A Self-Optimizing Multi-Agent System for Comprehensive ComfyUI Workflow Generation2025None
Progressive Prompt Detailing for Improved Alignment in Text-to-Image Generative Models2025NoneInference
SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer2025NoneCaching
SSDD: Single-Step Diffusion Decoder for Efficient Image Tokenization2025NoneDistillation
Less is More: Recursive Reasoning with Tiny Networks2025NoneInference
Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models2025NoneInference
Optimal Stepsize for Diffusion Sampling2025NoneInference
VORTA: Efficient Video Diffusion via Routing Sparse Attention2025NeurIPSInference
VSA: Faster Video Diffusion with Trainable Sparse Attention2025NeurIPSInference Training
Efficient Reasoning Models: A Survey2025NoneInference
pi-Flow: Policy-Based Few-Step Generation via Imitation Distillation2025ICLRDistillation
REAP the Experts: Why Pruning Prevails for One-Shot MoE compression2025NonePruning
LLaDA2.0: Scaling Up Diffusion Language Models to 100B2025None
TiDAR: Think in Diffusion, Talk in Autoregression2025NoneSpecDec
HilbertA: Hilbert Attention for Image Generation with Diffusion Models2025NoneInference
Intelligence per Watt: Measuring Intelligence Efficiency of Local AI2025NoneSustainability Inference
LongVie 2: Multimodal Controllable Ultra-Long Video World Model2025None
HiStream: Efficient High-Resolution Video Generation via Redundancy-Eliminated Streaming2025NoneCaching
MAGI-1: Autoregressive Video Generation at Scale2025None
WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inference2025NoneInference
Blockwise Flow Matching: Improving Flow Matching Models For Efficient High-Quality Generation2025NoneInference
H-Net++: Hierarchical Dynamic Chunking for Tokenizer-Free Language Modelling in Morphologically-Rich Languages2025None
Circuit Sparsity2025NonePruning
SLiM: One-shot Quantization and Sparsity with Low-rank Approximation for LLM Weight Compression2024NoneQuantization Pruning
TokenSqueeze: Performance-Preserving Compression for Reasoning LLMs2025NeurIPSInference
Frequency-Aware Token Reduction for Efficient Vision Transformer2025NeurIPSPruning
Language Models (Mostly) Know When to Stop Reading2025NeurIPSInference
Why 1 + 1 < 1 in Visual Token Pruning2025NeurIPSPruning
Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models2025NeurIPSInference
The Overthinker’s DIET: Cutting Token Calories with Difficulty-Aware Training2025NeurIPSTraining
R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large Model Token Routing2025NeurIPSSpecDec
Training Language Models to Reason Efficiently2025NeurIPSTraining
AccuQuant: Simulating Multiple Denoising Steps for Quantizing Diffusion Models2025NeurIPSQuantization
FP4 All the Way: Fully Quantized Training of LLMs2025NeurIPSQuantization
Quartet: Native FP4 Training Can Be Optimal for Large Language Models2025NeurIPSQuantization
DartQuant: Efficient Rotational Distribution Calibration for LLM Quantization2025NeurIPSQuantization
DFloat11: Lossless Compression of LLMs and Diffusion Models for Efficient GPU Inference2024NeurIPSQuantization
A Token is Worth over 1,000 Tokens2025NeurIPSDistillation
HAODiff: Human-Aware One-Step Diffusion via Dual-Prompt Guidance2025NeurIPS
One-Step Diffusion-Based Image Compression with Semantic Distillation2025NeurIPSDistillation
Mean Flows for One-step Generative Modeling2025NeurIPS
Uni-Instruct: One-step Diffusion Model through Unified Diffusion Divergence Instruction2025NeurIPS
Simple Distillation for One-Step Diffusion Models2025NeurIPSDistillation
Why Knowledge Distillation Works in Generative Models2025NeurIPSDistillation
Knowledge Distillation Detection for Open-Weights Models2025NeurIPSDistillation
FFN Fusion: Rethinking Sequential Computation in Large Language Models2024NeurIPSPruning
Sign-In to the Lottery: Reparameterized Sparse Training2025NeurIPSPruning
Differentiable Sparsity via D-Gating2025NeurIPSPruning
The Graphon Limit Hypothesis2025NeurIPSPruning
Týr-the-Pruner: Structural Pruning LLMs via Global Sparsity Distribution Optimization2025NeurIPSPruning
DuoGPT: Training-free Dual Sparsity through Activation-aware Pruning in LLMs2025NeurIPSPruning
Spark Transformer: Reactivating Sparsity in FFN and Attention2025NeurIPSPruning
Multi-Token Prediction Needs Registers2025NeurIPSInference
Q3R: Quadratic Reweighted Rank Regularizer for Effective Low-Rank Training2025NeurIPSFactorization
Accurate and Efficient Low-Rank Model Merging in Core Space2025NeurIPSPEFT
FALQON: Accelerating LoRA Fine-tuning with Low-Bit Floating-Point Arithmetic2025NeurIPSPEFT
DEFT: Decompositional Efficient Fine-Tuning for Text-to-Image Models2025NeurIPSPEFT
KLASS: KL-Guided Fast Inference in Masked Diffusion Models2025NeurIPSInference
ScaleDiff: Higher-Resolution Image Synthesis via Efficient and Model-Agnostic Diffusion2025NeurIPSInference
AI Should Sense Better, Not Just Scale Bigger: Adaptive Sensing as a Paradigm Shift2025NeurIPSScalability
A Sustainable AI Economy Needs Data Deals That Work for Generators2024NeurIPSSustainability
We Should Chart an Atlas of All the World's Models2024NeurIPS
Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Models2025NeurIPSInference
Position: Require Frontier AI Labs To Release Small "Analog" Models2025NeurIPSSustainability
Accelerating Diffusion LLMs via Adaptive Parallel Decoding2025NeurIPSSpecDec
LongLive: Real-time Interactive Long Video Generation2025NoneCaching
MagCache: Fast Video Generation with Magnitude-Aware Cache2025NeurIPSCaching
Toward Efficient Inference for Mixture of Experts2025NeurIPS
Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization2025NeurIPSQuantization
DC-VideoGen: Efficient Video Generation with Deep Compression Video Autoencoder2025None
Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation2025NeurIPS
More than Carbon: Cradle-to-Grave environmental impacts of GenAI training on the Nvidia A100 GPU2025NoneSustainability
Does Efficiency Lead to Green Machine Learning Model Training? Analyzing Historical Trends in Impacts from Hardware, Algorithmic and Carbon Optimizations2025NoneSustainability Training
Measuring the environmental impact of delivering AI at Google Scale2025NoneSustainability
GANQ: GPU-Adaptive Non-Uniform Quantization for Large Language Models2025ICMLQuantization
Foresight: Adaptive Layer Reuse for Accelerated and High-Quality Text-to-Video Generation2025NoneCaching
HiCache: Training-free Acceleration of Diffusion Models via Hermite Polynomial-based Feature Caching2025ICMLCaching
ERTACache: Error Rectification and Timesteps Adjustment for Efficient Diffusion2025NoneCaching
DiCache: Let Diffusion Model Determine its Own Cache2025NoneCaching
Set Block Decoding is a Language Model Inference Accelerator2025None
Position: Small Language Models are the Future of Agentic AI2025None
Fast-dLLM v2: Efficient Block-Diffusion Large Language Model2025None
SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillation2025None
LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models2025None
Let LLM Tell What to Prune and How Much to Prune2025ICMLPruning
SlimLLM: Accurate Structured Pruning for Large Language Models2025ICMLPruning
Speculative Prefill: Turbocharging TTFT with Lightweight and Training-Free Token Importance Estimation2025ICMLSpecDec
OrthoRank: Token Selection via Sink Token Orthogonality for Efficient LLM inference2025ICML
SkipGPT: Each Token is One of a Kind2025ICMLPruning
AdaSplash: Adaptive Sparse Flash Attention2025ICML
Can Compressed LLMs Truly Act? An Empirical Evaluation of Agentic Capabilities in LLM Compression2025ICML
An Efficient Matrix Multiplication Algorithm for Accelerating Inference in Binary and Ternary Neural Networks2025ICML
Accelerating Large Language Model Reasoning via Speculative Search2025ICMLSpecDec
ReFrame: Layer Caching for Accelerated Inference in Real-Time Rendering2025ICMLCaching
HarmoniCa: Harmonizing Training and Inference for Better Feature Caching in Diffusion Transformer Acceleration2025ICMLCaching
Attention-Level Speculation2025ICMLSpecDec
EvoPress: Accurate Dynamic Model Compression via Evolutionary Search2025ICML
Hardware and Software Platform Inference2025ICMLInference
MoH: Multi-Head Attention as Mixture-of-Head Attention2025ICML
QoS-Efficient Serving of Multiple Mixture-of-Expert LLMs Using Partial Runtime Reconfiguration2025ICML
KV Cache Compression via Sparse Coding over Universal Dictionaries2025ICMLCaching
KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference2025ICMLQuantization
BlockDialect: Block-wise Fine-grained Mixed Format Quantization for Energy-Efficient LLM Inference2025ICMLQuantization
any4: Learned 4-bit Numeric Representation for LLMs2025ICMLQuantization
AdaDecode: Accelerating LLM Decoding with Adaptive Layer Parallelism2025ICMLSpecDec
BOA: Attention-aware Post-training Quantization without Backpropagation2025ICMLQuantization
Radio: Rate–Distortion Optimization for Large Language Model Compression2025ICMLQuantization
Auditing Prompt Caching in Language Model APIs2025ICML
Mind the Gap: A Practical Attack on GGUF Quantization2025ICMLQuantization
Olica: Efficient Structured Pruning of Large Language Models without Retraining2025ICMLPruning
TESS 2: A Large-Scale Generalist Diffusion Language Model2025None
Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference2025None
Your LLM Knows the Future: Uncovering Its Multi-Token Prediction Potential2025None
The Fused Kernel Library: A C++ API to Develop Highly-Efficient GPU Libraries2025None
QuarterMap: Efficient Post-Training Token Pruning for Visual State Space Models2025NonePruning
Fast Video Generation with Sliding Tile Attention2025ICML
Quartet: Native FP4 Training Can Be Optimal for Large Language Models2025NoneQuantization
How Hungry is AI? Benchmarking Energy, Water, and Carbon Footprint of LLM Inference2025NoneInference
MagCache: Fast Video Generation with Magnitude-Aware Cache2025NoneCaching
Compressing Language Models for Specialized Domains2025None
Dynamic Chunking for End-to-End Hierarchical Sequence Modeling2025None
SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-Bit Training2025None
XAttention: Block Sparse Attention with Antidiagonal Scoring2025ICML
Jenga: Effective Memory Management for Serving LLM with Heterogeneity2025None
Learning Few-Step Diffusion Models by Trajectory Distribution Matching2025ICCV
Radial Attention: O(nlogn) Sparse Attention with Energy Decay for Long Video Generation2025None
Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding2025NoneCaching
Chipmunk: Training-Free Acceleration of Diffusion Transformers with Dynamic Column-Sparse Deltas2025NoneCaching
Mirage: A Multi-Level Superoptimizer for Tensor Programs2025None
The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimization2025NoneSustainability
AB-Cache: Training-Free Acceleration of Diffusion Models via Adams-Bashforth Cached Feature Reuse2025NoneCaching
Hardware-Efficient Attention for Fast Decoding2025NoneHardware
Model-Preserving Adaptive Rounding2025NoneQuantization
Frugal AI: Introduction, Concepts, Development and Open Questions2025NoneSustainability
Making AI Less “Thirsty”: Uncovering and Addressing the Secret Water Footprint of AI Models2025NoneSustainability
Efficient Time Series Processing for Transformers and State-Space Models through Token Merging2025None
A Survey on Inference Engines for Large Language Models: Perspectives on Optimization and Efficiency2025NoneInference
SpargeAttn: Accurate Sparse Attention Accelerating Any Model Inference2025NoneInference
SpargeAttention2: Trainable Sparse Attention via Hybrid Top-k+Top-p Masking and Distillation Fine-Tuning2025NoneInference Training
s1: Simple test-time scaling2025NoneInference
BitNet b1.58 2B4T Technical Report2025NoneQuantization
NdLinear Is All You Need for Representation Learning2025NoneFactorization
LoRI: Reducing Cross-Task Interference in Multi-Task LowRank Adaptation2025ICLRPEFT
FISH-Tuning: Enhancing PEFT Methods with Fisher Information2025NonePEFT
Green Prompting2025None
Compression Scaling Laws:Unifying Sparsity and Quantization2025NonePruningQuantization
FasterCache: Training-Free Video Diffusion Model Acceleration with High Quality2025ICLRCaching
LANTERN: Accelerating Visual Autoregressive Models with Relaxed Speculative Decoding2025ICLRSpecDec
Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Models2025NoneQuantization
Real-Time Video Generation with Pyramid Attention Broadcast2025ICLRCaching
Not All Prompts Are Made Equal: Prompt-based Pruning of Text-to-Image Diffusion Models2025ICLRPruning
Probe Pruning: Accelerating LLMs through Dynamic Pruning via Model-Probing2025ICLRPruning
Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention2025None
FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality Samples with Less Compute2025None
Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling2025NoneInference
SpinQuant: LLM Quantization with Learned Rotations2025ICLRQuantization
Making AI Less “Thirsty”: Uncovering and Addressing the Secret Water Footprint of AI Models2025None
Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps2025NoneInference
QuEST: Stable Training of LLMs with 1-Bit Weights and Activations2025NoneQuantization
Distillation Scaling Laws2025NoneDistillation
From Efficiency Gains to Rebound Effects: The Problem of Jevons' Paradox in AI's Polarized Environmental Debate2025None
Coca4ai: checking energy behaviors on AI data centers2024NoneSustainability Scalability
Scaling up Masked Diffusion Models on Text2024None
LTX-Video: Realtime Video Latent Diffusion2024CVPR
Constant Acceleration Flow2024NoneInference
LoRA vs Full Fine-tuning: An Illusion of Equivalence2024NeurIPSPEFT
How Green Can AI Be? A Study of Trends in Machine Learning Environmental Impacts2024NoneSustainability
QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs2024NeurIPSQuantization
The Iterative Optimal Brain Surgeon: Faster Sparse Recovery by Leveraging Second-Order Information2024NeurIPS
Palu: Compressing KV-Cache with Low-Rank Projection2024NoneQuantization
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration2024MLSysQuantization
LOFIT: Localized Fine-tuning on LLM Representations2024NeurIPSPEFT
Outlier Weighed Layerwise Sparsity: A Missing Secret Sauce for Pruning LLMs to High Sparsity2024ICMLPruning
FasterCache: Training-Free Video Diffusion Model Acceleration with High Quality2024NoneCaching
QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks2024ICMLQuantization
Better & Faster Large Language Models via Multi-token Prediction2024None
QTIP: Quantization with Trellises and Incoherence Processing2024NeurIPSQuantization
VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models2024EMNLPQuantization
QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs2024NeurIPSQuantization
QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving2024NoneQuantization
Extreme Compression of Large Language Models via Additive Quantization2024ICMLQuantization
Fast Matrix Multiplications for Lookup Table-Quantized LLMs2024NoneQuantization
GPTVQ: The Blessing of Dimensionality for LLM Quantization2024NoneQuantization
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey2024NonePEFT
SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration2024NoneSpecDec
SpecExec: Massively Parallel Speculative Decoding for Interactive LLM Inference on Consumer Devices2024NeurIPSSpecDec
ShortGPT: Layers in Large Language Models are More Redundant Than You Expecthttps://arxiv.org/pdf/2403.038532024NonePruning
Canvas: End-to-End Kernel Architecture Search in Neural Networks2024NoneCompilation
Scaling Laws for Precision2024NoneQuantization
DeepCache: Accelerating Diffusion Models for Free2024CVPRCaching
Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding2024ACLDistillation
Power Hungry Processing: Watts Driving the Cost of AI Deployment?2024FaccT
Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compression2024ICMLPruningQuantization
Pushing the Limits of Large Language Model Quantization via the Linearity Theorem2024NoneQuantization
Position: Tensor Networks are a Valuable Asset for Green AI2024NoneFactorization
Hype, Sustainability, and the Price of the Bigger-is-Better Paradigm in AI2024NoneSustainability
Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes2024ICLRPruning
Exploring the Carbon Footprint of Hugging Face's ML Models: A Repository Mining Study2023ESEMSustainability
Efficient Memory Management for Large Language Model Serving with PagedAttention2023SOSPCaching
Broken Neural Scaling Laws2023ICLR
Model Compression in Practice: Lessons Learned from Practitioners Creating On-device Machine Learning Experiences2023None
Post Training Mixed Precision Quantization of Neural Networks using First-Order Information2023ICCVQuantization
Ring Attention with Blockwise Transformers for Near-Infinite Context2023None
A Practical Mixed Precision Algorithm for Post-Training Quantization2023NoneQuantization
SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models2023ICMLQuantization
PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs2023NonePEFTPruning
Trends in AI inference energy consumption: Beyond the performance-vs-parameter laws of deep learning2023Sustainable Computing: Informatics and SystemsSustainability
An experimental comparison of software-based power meters: focus on CPU and GPU2023CCGridHardware
Fast Inference from Transformers via Speculative Decoding2023ICMLCaching
Efficient Streaming Language Models with Attention Sinks2023ICLR
Q-Diffusion: Quantizing Diffusion Models2023ICCVQuantization
GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers2023NoneQuantization
QReg: On Regularization Effects of Quantization2022NoneQuantization
Mixed-Precision Neural Network Quantization via Learned Layer-wise Importance2022ECCVQuantization
Knowledge Distillation: A Good Teacher is Patient and Consistent2022CVPRDistillation
LoRA: Low-Rank Adaptation of Large Language Models2022ICLRPEFT
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale2022NeurIPSQuantization
Optimal Clipping and Magnitude-aware Differentiation for Improved Quantization-aware Training2022ICMLQuantization
Sustainable AI: Environmental Implications, Challenges and Opportunities2022NoneSustainability
Learnable Lookup Table for Neural Network Quantization2022CVPRQuantization
Training Compute-Optimal Large Language Models2022None
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness2022None
Towards a Unified View of Parameter-Efficient Transfer Learning2022ICLRPEFT
Parameter-Efficient Transfer Learning with Diff Pruning2021ACLPEFTPruning
What is the State of Neural Network Pruning?2020MLSysPruning
Scaling Laws for Autoregressive Generative Modeling2020None
Model Compression via Distillation and Quantization2018ICLRQuantization
Optimal Brain Damage1989NeurIPsPruning

Blogs 📰

Books 📚

Lectures 🎓

  • AI Efficiency Courses: Slides, Exercises (2025) - Lecture by Bertrand Charpentier
  • Data Compression, Theory and Applications: YouTube, Slides (2024) - Stanford
  • MIT Han's Lab (2024) - Lecture by Song Han
  • GPU Mode (2020) - Tutorials by GPU mode community

People 🧑‍💻

NameAffiliationResearch InterestsSocial Media
James MartinBetter TechAI SustainabilityLinkedIn
Saleh AshkboosETH ZurichQuantizationLinkedIn
Dan AlistarhIST AustriaAI CompressionLinkedIn
Elias FrantarOpenAIQuantizationLinkedIn
Tim DettmersCMUQuantizationLinkedIn
Song HanMITAI EfficiencyLinkedIn
Scott ChamberlinTBDAI EfficiencyLinkedIn
Benoit PetitBoavistaData Center EfficiencyLinkedIn
Samuel RincéGen AI ImpactAI Efficiency, SustainabilityLinkedIn
Théo Alves Da CostaEkimetricsAI Efficiency, SustainabilityLinkedIn
Sasha LuccioniHugging FaceAI SustainabilityLinkedIn
Anne-Laure LigozatENSIEEAI SustainabilityLinkedIn
Boris GamazaychikovSales ForceAI SustainabilityLinkedIn
Julie RavillonSales ForceAI SustainabilityLinkedIn
Will AlpineEnabled EMissions CamapaignsAI SustainabilityLinkedIn
Holly AlpineEnabled EMissions CamapaignsAI SustainabilityLinkedIn
Drew WilkinsonClimate Leadership CollectiveAI SustainabilityLinkedIn
Maren CostaWorkforClimateAI SustainabilityLinkedIn
Lou WelgrynData4GoodAI Ethics, SustainabilityLinkedIn
Caroline Jean-PierreGen AI ImpactAI SustainabilityLinkedIn
Claire SaignolGen AI ImpactAI Sustianability LinkedIn
Juliette FropierFrench MinistryAI SustainabilityLinkedIn
Helene Costa de BeauregardFrench MinistryAI SustainabilityLinkedIn
Rémy MarroneIndependentAI SustainabilityLinkedIn
Mark ButcherPositive CloudCloud sustainabilityLinkedIn
Robert KeusGreen PTAI SustainabilityLinkedIn
Cas BurggraafGreenPTAI SustainabilityLinkedIn
Wilco BurggraafGreenPTAI SustainabilityLinkedIn
Anna Lerner NesbittClimate CollectiveAI SustainabilityLinkedIn
Scott ChamberlinNeural WattAI SustainabilityLinkedIn
Jeremy TamaniniDual CItizen LLCAI SustainabilityLinkedIn
Emma StrubellCMUAI Sustainability
Lin Feng ZhangPersonal siteAI EfficiencyWebsite

Organizations 🌍

OrganizationDescriptionWebsite
Data4GoodA platform that connects data scientists with social impact projects to address global challenges using data.data4good.org
Gen AI ImpactA platform dedidaceted to understand generative AI environmental footprint.genai-impact.org
Make.orgA global platform that empowers citizens to propose and take action on social and environmental issues through collective projects.make.org
CodeCarbonA tool that helps track the carbon emissions of machine learning models and optimizes them for sustainability.codecarbon.io
Sustainable AI CoalitionAn organization dedicated to advancing sustainability in AI technologies and promoting best practices for green AI.sustainableaicoalition.org
FruitPunch AIA community that solves AI solutions for impact organizations that contribute to the SDG's.fruitpunch.ai

Contributing 🤝

Contributions are welcome! Please follow our contribution guidelines to add new resources or suggest improvements that promote AI efficiency. Youc can contact @sharpenb if you have any questions.

License 📄

This project is licensed under the MIT License. Feel free to share and use the resources as needed.

Contributors

sharpenb

108 commits

davidberenstein1957

1 commits

Similar repos

PrunaAI/pruna

Pruna is a model optimization framework built for developers, enabling you to deliver faster, more…

1,285

same org

AIoT-MLSys-Lab/Efficient-LLMs-Survey

[TMLR 2024] Efficient Large Language Models: A Survey

1,257

HuangOwen/Awesome-LLM-Compression

Awesome LLM compression research papers and tools.

1,876

horseee/Awesome-Efficient-LLM

A curated list for Efficient Large Language Models

2,043

pprp/Awesome-LLM-Quantization

Awesome list for LLM quantization

448

sihyeong/Awesome-LLM-Inference-Engine

245

tiingweii-shii/Awesome-Resource-Efficient-LLM-Papers

a curated list of high-quality papers on resource-efficient LLMs 🌱

165

NVIDIA/Model-Optimizer

A unified library of SOTA model optimization techniques like quantization, distillation, pruning,…

3,847

PrunaAI/awesome-ai-efficiency

A curated list of materials on AI efficiency

237

109 commits

updated Aug 6, 2026

See the code

README

🌟 Awesome AI Efficiency 🌟

Awesome MIT License

A curated list of resources dedicated to enhancing efficiency in AI systems. This repository covers a wide range of topics essential for optimizing AI models and processes, aiming to make AI faster, cheaper, smaller, and greener!

Topics Summary 🎨

TopicDescriptionTopics
QuantizationReducing precision of AI models without lossQuantization
PruningRemoving unnecessary model parameters for efficiencyPruning
CachingStoring computation results for faster reuseCaching
DistillationTransferring knowledge from a large model to a smaller oneDistillation
FactorizationBreaking down complex models into simpler, efficient componentsFactorization
CompilationOptimizing model code for specific hardware and environmentsCompilation
Parameter-Efficient Fine-tuningLearning a subset of parametersPEFT
Speculative DecodingDecoding with batchesSpecDec
HardwareLeveraging specialized hardware for faster model executionHardware
TrainingTechniques for making model training faster and more efficientTraining
InferenceOptimizing the speed and resource usage during model inferenceInference
SustainabilityStrategies to reduce the environmental impact of AI systemsSustainability
ScalabilityApproaches for scaling AI models and infrastructure efficientlyScalability

If you find this list helpful, give it a ⭐ on GitHub, share it, and contribute by submitting a pull request or issue!

Table of Contents

Facts 📊

  • 1% of Dutch electricity: Microsoft hyperscale data center in Middenmeer consumes over 1% of all Dutch electricity (Source, 2026)
  • 3-40Wh: Amount of energy consumed for one small to long ChatGPT query (Source, 2025)
  • 1L: Estimated amount of water required for 20-100 ChatGPT queries (Source, 2025)
  • 2 nuclear plants: Number of nuclear plants to constantly work ot generate enough energy if 80M people generate 5 pages per day (Source, 2025)
  • 1 smartphone charge: Amount of energy required to AI generate a couple of images or run a few thousands inference with an LLM (Source, 2024)
  • >10s: Time requried to generate 1 HD image with Flux on H100 or to generate 100 tokens with Llama 3 on T4 (Source and Source, 2024)
  • 7-10 smartphone charges: Amount of energy required to AI generate one video with Wan 2.1 (Source)
  • 61,848.0x: Difference between the highest and lowest energy use in energy leaderboard for AI models (Source, 2025).
  • 1,300MWh: GPT-3, for example, is estimated to use just under 1,300 megawatt hours (MWh) of electricity; about as much power as consumed annually by 130 US homes (Source, 2024)
  • 800M users/week: Amount of users using ChatGPT per week in 2025 (Source)
  • 1B messages/day: Amount of ChatGPT queries per day in 2025 (Source)
  • +160%: Expected increase of data center power consumption by 2030 (Source)
  • x3.8: Hardware acceleration (GPU/TPU) reduces energy consumption by a factor of 3.8 compared with the CPU, for the same task, but also reduces response time by up to 39% (Source)
  • x18:The carbon footprint of a task can vary by a factor of 18 depending on the model, framework and backend used (Source)

Tools 🛠️

  • :heart: Pruna :heart:: A package to make AI models faster, smaller, faster, greener by combining compression methods (incl. quantization, pruning, caching, compilation, distillation...) on various hardware.
  • TensorRT: High-performance deep learning inference library for NVIDIA GPUs.
  • ONNX: Open Neural Network Exchange format for interoperability among deep learning frameworks.
  • Code Carbon: Library to track energy and carbon efficiency of various hardware.
  • LLM Perf: A framework for benchmarking the performance of transformers models with different hardwares, backends and optimizations.
  • ML.ENERGY Leaderboard: An initiative to benchmark energy efficiency of AI models.
  • AI Energy Score: An initiative to establish comparable energy efficiency ratings for AI models, helping the industry make informed decisions about sustainability in AI development.
  • Model Optimization Toolkit: TensorFlow toolkit for optimizing machine learning models for deployment and execution.
  • Green Coding: LLM service that you can use to prompt most open source models and see the resource usage.
  • EcoLogits: EcoLogits is a python library that tracks the energy consumption and environmental footprint of using generative AI models through APIs.
  • Perplexity Kernels: GPU kernels by Perplexity.
  • Fast Tokenizer: Fast tokenizer is an efficient and optimized tokenizer engine for llm inference serving.
  • WeightWatcher: WeightWatcher (WW) is an open-source, diagnostic tool for analyzing Deep Neural Networks (DNN), without needing access to training or even test data..
  • Cockpit: A Practical Debugging Tool for Training Deep Neural Networks.
  • Electrictiy Map: A live map showing the origin of the electricity in world regions and their CO2 intensity.
  • MLCA: A tool for machine learning life cycle assessment.
  • TritonParse: A visualization and analysis tool for Triton IR files, designed to help developers analyze, debug, and understand Triton kernel compilation processes.
  • Routing on Random Forests: A framework for training and serving LLM based on random forest-based routers, thus allowing to optimize for costs.
  • LLMCache: An LLM serving engine extension to reduce time-to-first-token and increase throughput, especially under long-context scenarios.
  • ExLlamaV3: An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs.
  • FlashDeBERTa: Flash implementation of DeBERTa disentangled attention mechanism.
  • QuACK: An assortiment of Kernels for GPUs.
  • Pi-Quant: An assortiment of Kernels for CPUs.
  • pplx-kernels: An assortiment of Kernels for GPUs.
  • LMCache: an LLM serving engine extension to reduce TTFT and increase throughput, especially under long-context scenarios, by optimizing the KV caches.
  • FastWan: a family of video generation models trained via “sparse distillation”.
  • FastVideo: A unified inference and post-training framework for accelerated video generation.
  • GEAK Agent: This is an LLM-based multi-agent framework, which can generate functional and efficient gpu kernels automatically.
  • Fused Kernel Library: Implementation of a package that allows user to define GPU kernel fusion, for non CUDA programmers.
  • FlashPack: a library with efficient Disk-to-GPU Tensor loading.
  • Humming: a library with efficient kernels.
  • NVIDIA Model Optimizer: Model optimization for efficient deployment.
  • GigaToken: Language model tokenization at GB/s.
  • FlashLib: Fast and memory-efficient classical machine learning operators.
  • Mage: A lightweight 4B multimodal model family for efficient visual understanding and generation.
  • ltx2-simple-optims: Simple optimizations for LTX2.3, including a custom QK-Norm-RoPE kernel.

News Articles 📰

Reports 📈

Research Articles 📄

PaperYearVenueTags
D2Cache: Second-Order Delta Caching for Higher Video Diffusion Acceleration2026CVPRCaching Inference
Budget-Constrained Step-Level Diffusion Caching2026ICMLCaching
ReQAT: Achieving Full-Precision Reasoning Accuracy with 4-bit Floating-Point Quantization-Aware Training2026ICMLQuantization Training
Distribution Matching Distillation Meets Reinforcement Learning2026ECCVDistillation Training
DiffusionBlocks: Block-wise Neural Network Training via Diffusion Interpretation2026ICLRTraining
Parallel Decoding Distillation for Fast Image and Video Generation2026NoneDistillation Inference
Sol-Attn: Accelerating Video Generation Inference via On-the-Fly Attention Sparsification2026NoneInference
SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation2026NoneInference Caching
OrbitQuant: Data-Agnostic Quantization for Image and Video Diffusion Transformers2026NoneQuantization
Multi-Resolution Flow Matching: Training-Free Diffusion Acceleration via Staged Sampling2026NoneInference
Post-Training Pruning for Diffusion Transformers2026NonePruning
Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generation2026NoneInference Compilation
Training-free sparse attention based on cumulative energy filtering2026NoneInference
Adaptive Inference-Time Scaling via Early-Step Latent Verification for Image Editing2026NoneInference
HiLo-Token: Input-Adaptive High-Low Frequency Token Compression for Efficient Image Editing2026NonePruning Inference
TetherCache: Stabilizing Autoregressive Long-Form Video Generation with Gated Recall and Trusted Alignment2026NoneCaching Inference
High-Fidelity Two-Step Image Generation via Teacher-Aligned End-to-End Distillation2026NoneDistillation
RhymeFlow: Training-Free Acceleration for Video Generation with Asynchronous Denoising Flow Scheduling2026NoneInference
ReCache: Learning Budget-Aware Caching Schedules for Diffusion Models via REINFORCE2026NoneCaching
Qwen-Image-Flash: Beyond Objective Design2026NoneDistillation Inference
KVarN: Variance-Normalized KV-Cache Quantization Mitigates Error Accumulation in Reasoning Tasks2026NoneQuantization Caching
Dual-Rate Diffusion: Accelerating diffusion models with an interleaved heavy-light network2026NoneInference
ElasticDiT: Efficient Diffusion Transformers via Elastic Architecture and Sparse Attention for High-Resolution Image Generation on Mobile Devices2026NoneInference Pruning
Continuous-Time Distribution Matching for Few-Step Diffusion Distillation2026NoneDistillation
Statistically-Lossless Quantization of Large Language Models2026NoneQuantization
YOSE: You Only Select Essential Tokens for Efficient DiT-based Video Object Removal2026CVPRPruning Inference
Token Reduction via Local and Global Contexts Optimization for Efficient Video Large Language Models2026CVPRPruning
MASQuant: Modality-Aware Smoothing Quantization for Multimodal Large Language Models2026CVPRQuantization
SegQuant: A Semantics-Aware and Generalizable Quantization Framework for Diffusion Models2026CVPRQuantization
ResCa: Residual Caching for Diffusion Transformers Acceleration2026CVPRCaching
DisCa: Accelerating Video Diffusion Transformers with Distillation-Compatible Learnable Feature Caching2026CVPRCaching Distillation
SenCache: Accelerating Diffusion Model Inference via Sensitivity-Aware Caching2026CVPRCaching
Accelerating Diffusion-based Video Editing via Heterogeneous Caching: Beyond Full Computing at Sampled Denoising Timestep2026CVPRCaching
d2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching2026CVPRCaching Inference
Training-free, Perceptually Consistent Low-Resolution Previews with High-Resolution Image for Efficient Workflows of Diffusion Models2026CVPRInference
LinVideo: A Post-Training Framework towards O(n) Attention in Efficient Video Generation2026CVPRInference
An Efficient Token Compression Framework for Visual Object Tracking2026CVPRPruning Inference
VMonarch: Efficient Video Diffusion Transformers with Structured Attention2026CVPRInference
Otil: Accelerating Diffusion Model Inference via Communication-Efficient Multi-GPU Parallelism2026CVPRInference Scalability
NanoSD: Edge Efficient Foundation Model for Real Time Image Restoration2026CVPRDistillation Inference
Flash-DMD: Towards High-Fidelity Few-Step Image Generation with Efficient Distillation and Joint Reinforcement Learning2026CVPRDistillation
Attention-aware Inference Optimizations for Large Vision-Language Models with Memory-efficient Decoding2026CVPRCaching Inference
Attention Surgery: An Efficient Recipe to Linearize Your Video Diffusion Transformer2026CVPRInference Distillation
Trainable Log-linear Sparse Attention for Efficient Diffusion Transformers2026CVPRInference Training
Rethinking Asymmetric Quantization: Hidden Symmetry in Vision Model Weights2026CVPRQuantization
LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuning2026CVPR
S2FT: Parameter-Efficient Fine-Tuning in Sparse Spectrum Domain2026CVPRPEFT
vAttention: Verified Sparse Attention via Sampling2026ICLRInference
PLoP: Precise LoRA Placement for Efficient Finetuning of Large Models2026ICLRPEFT
How to train data-efficient LLMs2026ICLRTraining
ERTACache: Error Rectification and Timesteps Adjustment for Efficient Diffusion2026ICLRCaching
Fewer Battles, More Gain: An Information-Efficient Framework for Arena-based LLM Evaluation2026ICLRScalability
PASER: Post-Training Data Selection for Efficient Pruned Large Language Model Recovery2026ICLRPruning
Efficient Orthogonal Fine-Tuning with Principal Subspace Adaptation2026ICLRPEFT
MicroMix: Efficient Mixed-Precision Quantization with Microscaling Formats for Large Language Models2026ICLRQuantization
Fast-dLLM v2: Efficient Block-Diffusion LLM2026ICLRInference Caching
DSA: Efficient Inference For Video Generation Models via Distributed Sparse Attention2026ICLRInference
Q&C: When Quantization Meets Cache in Efficient Generation2026ICLRQuantization Caching
Plan and Budget: Effective and Efficient Test-Time Scaling on Reasoning Large Language Models2026ICLRInference
Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling2026ICLRSpecDec
GradPruner: Gradient-guided Layer Pruning Enabling Efficient Fine-Tuning and Inference for LLMs2026ICLRPruning Training
ES-dLLM: Efficient Inference for Diffusion Large Language Models by Early-Skipping2026ICLRInference Caching
Channel-Aware Mixed-Precision Quantization for Efficient Long-Context Inference2026ICLRQuantization Caching
Test-Time Iterative Error Correction for Efficient Diffusion Models2026ICLRInference
FideDiff: Efficient Diffusion Model for High-Fidelity Image Motion Deblurring2026ICLRInference
PreciseCache: Precise Feature Caching for Efficient and High-fidelity Video Generation2026ICLRCaching
Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs2026ICLRInference
SPRINT: Sparse-Dense Residual Fusion for Efficient Diffusion Transformers2026ICLRTraining Inference
ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inference2026ICLRQuantization
FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion2026ICLRCaching Inference
SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer2026ICLRInference Caching
Diffusion Models as Dataset Distillation Priors2026ICLRDistillation
Scale-wise Distillation of Diffusion Models2026ICLRDistillation
Streaming Autoregressive Video Generation via Diagonal Distillation2026ICLRDistillation
pi-Flow: Policy-Based Few-Step Generation via Imitation Distillation2026ICLRDistillation
PISA: Piecewise Sparse Attention Is Wiser for Efficient Diffusion Transformers2026ICMLInference
Beyond Fixed Formulas: Data-Driven Linear Predictor for Efficient Diffusion Models2026NoneCaching
RAPID: Reusing Attention Sparsity with Inter-step Adaptation for Efficient Video Diffusion2026CVPRCaching Inference
AnyFlow: Any-Step Video Diffusion Model with On-Policy Flow Map Distillation2026NoneDistillation
Towards Resource-Efficient LLMs: End-to-End Energy Accounting of Distillation Pipelines2026ICMLSustainability Distillation
Spectral Progressive Diffusion for Efficient Image and Video Generation2026NoneInference
SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer2026NoneInference Quantization
Sparser, Faster, Lighter Transformer Language Models2026NonePruning
Stream-T1: Test-Time Scaling for Streaming Video Generation2026NoneInference
Optimizing Few-Step Generation with Adaptive Matching Distillation2026ICMLDistillation
Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding2026NoneSpecDec Inference
Bonsai Image 4B: Low-Bit Diffusion on Apple Silicon and Consumer GPUs2026NoneQuantization Inference
Where Do the Joules Go? Diagnosing Inference Energy Consumption2026NoneSustainability Inference
Generative Modeling via Drifting2026NoneInference
Just on Time: Token-Level Early Stopping for Diffusion Language Models2026NoneInference
From Sketch to Fresco: Efficient Diffusion Transformer with Progressive Resolution2026NoneInference
Forecast the Principal, Stabilize the Residual: Subspace-Aware Feature Caching for Efficient Diffusion Transformers2026NoneCaching
ArcFlow: Unleashing 2-Step Text-to-Image Generation via High-Precision Non-Linear Flow Distillation2026NoneDistillation
FlowCast: Trajectory Forecasting for Scalable Zero-Cost Speculative Flow Matching2026NoneInference
Learning to Reason in 13 Parameters2026NonePEFT
Quant VideoGen: Auto-Regressive Long Video Generation via 2-Bit KV-Cache Quantization2026NoneQuantization
PixelRush: Ultra-Fast, Training-Free High-Resolution Image Generation via One-step Diffusion2026NoneInference
Omni-Video 2: Scaling MLLM-Conditioned Diffusion for Unified Video Generation and Editing2026None
Iterative Refinement Improves Compositional Image Generation2026NoneInference
LightCache: Memory-Efficient, Training-Free Acceleration for Video Generation2025NoneCaching
CoIn: Coverage and Informativeness-Guided Token Reduction for Efficient Large Multimodal Models2025NonePruning
Efficient Zero-Shot Inpainting with Decoupled Diffusion Guidance2025NoneInference
ComfyGPT: A Self-Optimizing Multi-Agent System for Comprehensive ComfyUI Workflow Generation2025None
Progressive Prompt Detailing for Improved Alignment in Text-to-Image Generative Models2025NoneInference
SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer2025NoneCaching
SSDD: Single-Step Diffusion Decoder for Efficient Image Tokenization2025NoneDistillation
Less is More: Recursive Reasoning with Tiny Networks2025NoneInference
Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models2025NoneInference
Optimal Stepsize for Diffusion Sampling2025NoneInference
VORTA: Efficient Video Diffusion via Routing Sparse Attention2025NeurIPSInference
VSA: Faster Video Diffusion with Trainable Sparse Attention2025NeurIPSInference Training
Efficient Reasoning Models: A Survey2025NoneInference
pi-Flow: Policy-Based Few-Step Generation via Imitation Distillation2025ICLRDistillation
REAP the Experts: Why Pruning Prevails for One-Shot MoE compression2025NonePruning
LLaDA2.0: Scaling Up Diffusion Language Models to 100B2025None
TiDAR: Think in Diffusion, Talk in Autoregression2025NoneSpecDec
HilbertA: Hilbert Attention for Image Generation with Diffusion Models2025NoneInference
Intelligence per Watt: Measuring Intelligence Efficiency of Local AI2025NoneSustainability Inference
LongVie 2: Multimodal Controllable Ultra-Long Video World Model2025None
HiStream: Efficient High-Resolution Video Generation via Redundancy-Eliminated Streaming2025NoneCaching
MAGI-1: Autoregressive Video Generation at Scale2025None
WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inference2025NoneInference
Blockwise Flow Matching: Improving Flow Matching Models For Efficient High-Quality Generation2025NoneInference
H-Net++: Hierarchical Dynamic Chunking for Tokenizer-Free Language Modelling in Morphologically-Rich Languages2025None
Circuit Sparsity2025NonePruning
SLiM: One-shot Quantization and Sparsity with Low-rank Approximation for LLM Weight Compression2024NoneQuantization Pruning
TokenSqueeze: Performance-Preserving Compression for Reasoning LLMs2025NeurIPSInference
Frequency-Aware Token Reduction for Efficient Vision Transformer2025NeurIPSPruning
Language Models (Mostly) Know When to Stop Reading2025NeurIPSInference
Why 1 + 1 < 1 in Visual Token Pruning2025NeurIPSPruning
Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models2025NeurIPSInference
The Overthinker’s DIET: Cutting Token Calories with Difficulty-Aware Training2025NeurIPSTraining
R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large Model Token Routing2025NeurIPSSpecDec
Training Language Models to Reason Efficiently2025NeurIPSTraining
AccuQuant: Simulating Multiple Denoising Steps for Quantizing Diffusion Models2025NeurIPSQuantization
FP4 All the Way: Fully Quantized Training of LLMs2025NeurIPSQuantization
Quartet: Native FP4 Training Can Be Optimal for Large Language Models2025NeurIPSQuantization
DartQuant: Efficient Rotational Distribution Calibration for LLM Quantization2025NeurIPSQuantization
DFloat11: Lossless Compression of LLMs and Diffusion Models for Efficient GPU Inference2024NeurIPSQuantization
A Token is Worth over 1,000 Tokens2025NeurIPSDistillation
HAODiff: Human-Aware One-Step Diffusion via Dual-Prompt Guidance2025NeurIPS
One-Step Diffusion-Based Image Compression with Semantic Distillation2025NeurIPSDistillation
Mean Flows for One-step Generative Modeling2025NeurIPS
Uni-Instruct: One-step Diffusion Model through Unified Diffusion Divergence Instruction2025NeurIPS
Simple Distillation for One-Step Diffusion Models2025NeurIPSDistillation
Why Knowledge Distillation Works in Generative Models2025NeurIPSDistillation
Knowledge Distillation Detection for Open-Weights Models2025NeurIPSDistillation
FFN Fusion: Rethinking Sequential Computation in Large Language Models2024NeurIPSPruning
Sign-In to the Lottery: Reparameterized Sparse Training2025NeurIPSPruning
Differentiable Sparsity via D-Gating2025NeurIPSPruning
The Graphon Limit Hypothesis2025NeurIPSPruning
Týr-the-Pruner: Structural Pruning LLMs via Global Sparsity Distribution Optimization2025NeurIPSPruning
DuoGPT: Training-free Dual Sparsity through Activation-aware Pruning in LLMs2025NeurIPSPruning
Spark Transformer: Reactivating Sparsity in FFN and Attention2025NeurIPSPruning
Multi-Token Prediction Needs Registers2025NeurIPSInference
Q3R: Quadratic Reweighted Rank Regularizer for Effective Low-Rank Training2025NeurIPSFactorization
Accurate and Efficient Low-Rank Model Merging in Core Space2025NeurIPSPEFT
FALQON: Accelerating LoRA Fine-tuning with Low-Bit Floating-Point Arithmetic2025NeurIPSPEFT
DEFT: Decompositional Efficient Fine-Tuning for Text-to-Image Models2025NeurIPSPEFT
KLASS: KL-Guided Fast Inference in Masked Diffusion Models2025NeurIPSInference
ScaleDiff: Higher-Resolution Image Synthesis via Efficient and Model-Agnostic Diffusion2025NeurIPSInference
AI Should Sense Better, Not Just Scale Bigger: Adaptive Sensing as a Paradigm Shift2025NeurIPSScalability
A Sustainable AI Economy Needs Data Deals That Work for Generators2024NeurIPSSustainability
We Should Chart an Atlas of All the World's Models2024NeurIPS
Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Models2025NeurIPSInference
Position: Require Frontier AI Labs To Release Small "Analog" Models2025NeurIPSSustainability
Accelerating Diffusion LLMs via Adaptive Parallel Decoding2025NeurIPSSpecDec
LongLive: Real-time Interactive Long Video Generation2025NoneCaching
MagCache: Fast Video Generation with Magnitude-Aware Cache2025NeurIPSCaching
Toward Efficient Inference for Mixture of Experts2025NeurIPS
Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization2025NeurIPSQuantization
DC-VideoGen: Efficient Video Generation with Deep Compression Video Autoencoder2025None
Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation2025NeurIPS
More than Carbon: Cradle-to-Grave environmental impacts of GenAI training on the Nvidia A100 GPU2025NoneSustainability
Does Efficiency Lead to Green Machine Learning Model Training? Analyzing Historical Trends in Impacts from Hardware, Algorithmic and Carbon Optimizations2025NoneSustainability Training
Measuring the environmental impact of delivering AI at Google Scale2025NoneSustainability
GANQ: GPU-Adaptive Non-Uniform Quantization for Large Language Models2025ICMLQuantization
Foresight: Adaptive Layer Reuse for Accelerated and High-Quality Text-to-Video Generation2025NoneCaching
HiCache: Training-free Acceleration of Diffusion Models via Hermite Polynomial-based Feature Caching2025ICMLCaching
ERTACache: Error Rectification and Timesteps Adjustment for Efficient Diffusion2025NoneCaching
DiCache: Let Diffusion Model Determine its Own Cache2025NoneCaching
Set Block Decoding is a Language Model Inference Accelerator2025None
Position: Small Language Models are the Future of Agentic AI2025None
Fast-dLLM v2: Efficient Block-Diffusion Large Language Model2025None
SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillation2025None
LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models2025None
Let LLM Tell What to Prune and How Much to Prune2025ICMLPruning
SlimLLM: Accurate Structured Pruning for Large Language Models2025ICMLPruning
Speculative Prefill: Turbocharging TTFT with Lightweight and Training-Free Token Importance Estimation2025ICMLSpecDec
OrthoRank: Token Selection via Sink Token Orthogonality for Efficient LLM inference2025ICML
SkipGPT: Each Token is One of a Kind2025ICMLPruning
AdaSplash: Adaptive Sparse Flash Attention2025ICML
Can Compressed LLMs Truly Act? An Empirical Evaluation of Agentic Capabilities in LLM Compression2025ICML
An Efficient Matrix Multiplication Algorithm for Accelerating Inference in Binary and Ternary Neural Networks2025ICML
Accelerating Large Language Model Reasoning via Speculative Search2025ICMLSpecDec
ReFrame: Layer Caching for Accelerated Inference in Real-Time Rendering2025ICMLCaching
HarmoniCa: Harmonizing Training and Inference for Better Feature Caching in Diffusion Transformer Acceleration2025ICMLCaching
Attention-Level Speculation2025ICMLSpecDec
EvoPress: Accurate Dynamic Model Compression via Evolutionary Search2025ICML
Hardware and Software Platform Inference2025ICMLInference
MoH: Multi-Head Attention as Mixture-of-Head Attention2025ICML
QoS-Efficient Serving of Multiple Mixture-of-Expert LLMs Using Partial Runtime Reconfiguration2025ICML
KV Cache Compression via Sparse Coding over Universal Dictionaries2025ICMLCaching
KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference2025ICMLQuantization
BlockDialect: Block-wise Fine-grained Mixed Format Quantization for Energy-Efficient LLM Inference2025ICMLQuantization
any4: Learned 4-bit Numeric Representation for LLMs2025ICMLQuantization
AdaDecode: Accelerating LLM Decoding with Adaptive Layer Parallelism2025ICMLSpecDec
BOA: Attention-aware Post-training Quantization without Backpropagation2025ICMLQuantization
Radio: Rate–Distortion Optimization for Large Language Model Compression2025ICMLQuantization
Auditing Prompt Caching in Language Model APIs2025ICML
Mind the Gap: A Practical Attack on GGUF Quantization2025ICMLQuantization
Olica: Efficient Structured Pruning of Large Language Models without Retraining2025ICMLPruning
TESS 2: A Large-Scale Generalist Diffusion Language Model2025None
Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference2025None
Your LLM Knows the Future: Uncovering Its Multi-Token Prediction Potential2025None
The Fused Kernel Library: A C++ API to Develop Highly-Efficient GPU Libraries2025None
QuarterMap: Efficient Post-Training Token Pruning for Visual State Space Models2025NonePruning
Fast Video Generation with Sliding Tile Attention2025ICML
Quartet: Native FP4 Training Can Be Optimal for Large Language Models2025NoneQuantization
How Hungry is AI? Benchmarking Energy, Water, and Carbon Footprint of LLM Inference2025NoneInference
MagCache: Fast Video Generation with Magnitude-Aware Cache2025NoneCaching
Compressing Language Models for Specialized Domains2025None
Dynamic Chunking for End-to-End Hierarchical Sequence Modeling2025None
SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-Bit Training2025None
XAttention: Block Sparse Attention with Antidiagonal Scoring2025ICML
Jenga: Effective Memory Management for Serving LLM with Heterogeneity2025None
Learning Few-Step Diffusion Models by Trajectory Distribution Matching2025ICCV
Radial Attention: O(nlogn) Sparse Attention with Energy Decay for Long Video Generation2025None
Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding2025NoneCaching
Chipmunk: Training-Free Acceleration of Diffusion Transformers with Dynamic Column-Sparse Deltas2025NoneCaching
Mirage: A Multi-Level Superoptimizer for Tensor Programs2025None
The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimization2025NoneSustainability
AB-Cache: Training-Free Acceleration of Diffusion Models via Adams-Bashforth Cached Feature Reuse2025NoneCaching
Hardware-Efficient Attention for Fast Decoding2025NoneHardware
Model-Preserving Adaptive Rounding2025NoneQuantization
Frugal AI: Introduction, Concepts, Development and Open Questions2025NoneSustainability
Making AI Less “Thirsty”: Uncovering and Addressing the Secret Water Footprint of AI Models2025NoneSustainability
Efficient Time Series Processing for Transformers and State-Space Models through Token Merging2025None
A Survey on Inference Engines for Large Language Models: Perspectives on Optimization and Efficiency2025NoneInference
SpargeAttn: Accurate Sparse Attention Accelerating Any Model Inference2025NoneInference
SpargeAttention2: Trainable Sparse Attention via Hybrid Top-k+Top-p Masking and Distillation Fine-Tuning2025NoneInference Training
s1: Simple test-time scaling2025NoneInference
BitNet b1.58 2B4T Technical Report2025NoneQuantization
NdLinear Is All You Need for Representation Learning2025NoneFactorization
LoRI: Reducing Cross-Task Interference in Multi-Task LowRank Adaptation2025ICLRPEFT
FISH-Tuning: Enhancing PEFT Methods with Fisher Information2025NonePEFT
Green Prompting2025None
Compression Scaling Laws:Unifying Sparsity and Quantization2025NonePruningQuantization
FasterCache: Training-Free Video Diffusion Model Acceleration with High Quality2025ICLRCaching
LANTERN: Accelerating Visual Autoregressive Models with Relaxed Speculative Decoding2025ICLRSpecDec
Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Models2025NoneQuantization
Real-Time Video Generation with Pyramid Attention Broadcast2025ICLRCaching
Not All Prompts Are Made Equal: Prompt-based Pruning of Text-to-Image Diffusion Models2025ICLRPruning
Probe Pruning: Accelerating LLMs through Dynamic Pruning via Model-Probing2025ICLRPruning
Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention2025None
FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality Samples with Less Compute2025None
Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling2025NoneInference
SpinQuant: LLM Quantization with Learned Rotations2025ICLRQuantization
Making AI Less “Thirsty”: Uncovering and Addressing the Secret Water Footprint of AI Models2025None
Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps2025NoneInference
QuEST: Stable Training of LLMs with 1-Bit Weights and Activations2025NoneQuantization
Distillation Scaling Laws2025NoneDistillation
From Efficiency Gains to Rebound Effects: The Problem of Jevons' Paradox in AI's Polarized Environmental Debate2025None
Coca4ai: checking energy behaviors on AI data centers2024NoneSustainability Scalability
Scaling up Masked Diffusion Models on Text2024None
LTX-Video: Realtime Video Latent Diffusion2024CVPR
Constant Acceleration Flow2024NoneInference
LoRA vs Full Fine-tuning: An Illusion of Equivalence2024NeurIPSPEFT
How Green Can AI Be? A Study of Trends in Machine Learning Environmental Impacts2024NoneSustainability
QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs2024NeurIPSQuantization
The Iterative Optimal Brain Surgeon: Faster Sparse Recovery by Leveraging Second-Order Information2024NeurIPS
Palu: Compressing KV-Cache with Low-Rank Projection2024NoneQuantization
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration2024MLSysQuantization
LOFIT: Localized Fine-tuning on LLM Representations2024NeurIPSPEFT
Outlier Weighed Layerwise Sparsity: A Missing Secret Sauce for Pruning LLMs to High Sparsity2024ICMLPruning
FasterCache: Training-Free Video Diffusion Model Acceleration with High Quality2024NoneCaching
QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks2024ICMLQuantization
Better & Faster Large Language Models via Multi-token Prediction2024None
QTIP: Quantization with Trellises and Incoherence Processing2024NeurIPSQuantization
VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models2024EMNLPQuantization
QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs2024NeurIPSQuantization
QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving2024NoneQuantization
Extreme Compression of Large Language Models via Additive Quantization2024ICMLQuantization
Fast Matrix Multiplications for Lookup Table-Quantized LLMs2024NoneQuantization
GPTVQ: The Blessing of Dimensionality for LLM Quantization2024NoneQuantization
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey2024NonePEFT
SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration2024NoneSpecDec
SpecExec: Massively Parallel Speculative Decoding for Interactive LLM Inference on Consumer Devices2024NeurIPSSpecDec
ShortGPT: Layers in Large Language Models are More Redundant Than You Expecthttps://arxiv.org/pdf/2403.038532024NonePruning
Canvas: End-to-End Kernel Architecture Search in Neural Networks2024NoneCompilation
Scaling Laws for Precision2024NoneQuantization
DeepCache: Accelerating Diffusion Models for Free2024CVPRCaching
Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding2024ACLDistillation
Power Hungry Processing: Watts Driving the Cost of AI Deployment?2024FaccT
Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compression2024ICMLPruningQuantization
Pushing the Limits of Large Language Model Quantization via the Linearity Theorem2024NoneQuantization
Position: Tensor Networks are a Valuable Asset for Green AI2024NoneFactorization
Hype, Sustainability, and the Price of the Bigger-is-Better Paradigm in AI2024NoneSustainability
Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes2024ICLRPruning
Exploring the Carbon Footprint of Hugging Face's ML Models: A Repository Mining Study2023ESEMSustainability
Efficient Memory Management for Large Language Model Serving with PagedAttention2023SOSPCaching
Broken Neural Scaling Laws2023ICLR
Model Compression in Practice: Lessons Learned from Practitioners Creating On-device Machine Learning Experiences2023None
Post Training Mixed Precision Quantization of Neural Networks using First-Order Information2023ICCVQuantization
Ring Attention with Blockwise Transformers for Near-Infinite Context2023None
A Practical Mixed Precision Algorithm for Post-Training Quantization2023NoneQuantization
SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models2023ICMLQuantization
PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs2023NonePEFTPruning
Trends in AI inference energy consumption: Beyond the performance-vs-parameter laws of deep learning2023Sustainable Computing: Informatics and SystemsSustainability
An experimental comparison of software-based power meters: focus on CPU and GPU2023CCGridHardware
Fast Inference from Transformers via Speculative Decoding2023ICMLCaching
Efficient Streaming Language Models with Attention Sinks2023ICLR
Q-Diffusion: Quantizing Diffusion Models2023ICCVQuantization
GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers2023NoneQuantization
QReg: On Regularization Effects of Quantization2022NoneQuantization
Mixed-Precision Neural Network Quantization via Learned Layer-wise Importance2022ECCVQuantization
Knowledge Distillation: A Good Teacher is Patient and Consistent2022CVPRDistillation
LoRA: Low-Rank Adaptation of Large Language Models2022ICLRPEFT
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale2022NeurIPSQuantization
Optimal Clipping and Magnitude-aware Differentiation for Improved Quantization-aware Training2022ICMLQuantization
Sustainable AI: Environmental Implications, Challenges and Opportunities2022NoneSustainability
Learnable Lookup Table for Neural Network Quantization2022CVPRQuantization
Training Compute-Optimal Large Language Models2022None
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness2022None
Towards a Unified View of Parameter-Efficient Transfer Learning2022ICLRPEFT
Parameter-Efficient Transfer Learning with Diff Pruning2021ACLPEFTPruning
What is the State of Neural Network Pruning?2020MLSysPruning
Scaling Laws for Autoregressive Generative Modeling2020None
Model Compression via Distillation and Quantization2018ICLRQuantization
Optimal Brain Damage1989NeurIPsPruning

Blogs 📰

Books 📚

Lectures 🎓

  • AI Efficiency Courses: Slides, Exercises (2025) - Lecture by Bertrand Charpentier
  • Data Compression, Theory and Applications: YouTube, Slides (2024) - Stanford
  • MIT Han's Lab (2024) - Lecture by Song Han
  • GPU Mode (2020) - Tutorials by GPU mode community

People 🧑‍💻

NameAffiliationResearch InterestsSocial Media
James MartinBetter TechAI SustainabilityLinkedIn
Saleh AshkboosETH ZurichQuantizationLinkedIn
Dan AlistarhIST AustriaAI CompressionLinkedIn
Elias FrantarOpenAIQuantizationLinkedIn
Tim DettmersCMUQuantizationLinkedIn
Song HanMITAI EfficiencyLinkedIn
Scott ChamberlinTBDAI EfficiencyLinkedIn
Benoit PetitBoavistaData Center EfficiencyLinkedIn
Samuel RincéGen AI ImpactAI Efficiency, SustainabilityLinkedIn
Théo Alves Da CostaEkimetricsAI Efficiency, SustainabilityLinkedIn
Sasha LuccioniHugging FaceAI SustainabilityLinkedIn
Anne-Laure LigozatENSIEEAI SustainabilityLinkedIn
Boris GamazaychikovSales ForceAI SustainabilityLinkedIn
Julie RavillonSales ForceAI SustainabilityLinkedIn
Will AlpineEnabled EMissions CamapaignsAI SustainabilityLinkedIn
Holly AlpineEnabled EMissions CamapaignsAI SustainabilityLinkedIn
Drew WilkinsonClimate Leadership CollectiveAI SustainabilityLinkedIn
Maren CostaWorkforClimateAI SustainabilityLinkedIn
Lou WelgrynData4GoodAI Ethics, SustainabilityLinkedIn
Caroline Jean-PierreGen AI ImpactAI SustainabilityLinkedIn
Claire SaignolGen AI ImpactAI Sustianability LinkedIn
Juliette FropierFrench MinistryAI SustainabilityLinkedIn
Helene Costa de BeauregardFrench MinistryAI SustainabilityLinkedIn
Rémy MarroneIndependentAI SustainabilityLinkedIn
Mark ButcherPositive CloudCloud sustainabilityLinkedIn
Robert KeusGreen PTAI SustainabilityLinkedIn
Cas BurggraafGreenPTAI SustainabilityLinkedIn
Wilco BurggraafGreenPTAI SustainabilityLinkedIn
Anna Lerner NesbittClimate CollectiveAI SustainabilityLinkedIn
Scott ChamberlinNeural WattAI SustainabilityLinkedIn
Jeremy TamaniniDual CItizen LLCAI SustainabilityLinkedIn
Emma StrubellCMUAI Sustainability
Lin Feng ZhangPersonal siteAI EfficiencyWebsite

Organizations 🌍

OrganizationDescriptionWebsite
Data4GoodA platform that connects data scientists with social impact projects to address global challenges using data.data4good.org
Gen AI ImpactA platform dedidaceted to understand generative AI environmental footprint.genai-impact.org
Make.orgA global platform that empowers citizens to propose and take action on social and environmental issues through collective projects.make.org
CodeCarbonA tool that helps track the carbon emissions of machine learning models and optimizes them for sustainability.codecarbon.io
Sustainable AI CoalitionAn organization dedicated to advancing sustainability in AI technologies and promoting best practices for green AI.sustainableaicoalition.org
FruitPunch AIA community that solves AI solutions for impact organizations that contribute to the SDG's.fruitpunch.ai

Contributing 🤝

Contributions are welcome! Please follow our contribution guidelines to add new resources or suggest improvements that promote AI efficiency. Youc can contact @sharpenb if you have any questions.

License 📄

This project is licensed under the MIT License. Feel free to share and use the resources as needed.

Similar repos

PrunaAI/pruna

Pruna is a model optimization framework built for developers, enabling you to deliver faster, more…

1,285

same org

AIoT-MLSys-Lab/Efficient-LLMs-Survey

[TMLR 2024] Efficient Large Language Models: A Survey

1,257

HuangOwen/Awesome-LLM-Compression

Awesome LLM compression research papers and tools.

1,876

horseee/Awesome-Efficient-LLM

A curated list for Efficient Large Language Models

2,043

pprp/Awesome-LLM-Quantization

Awesome list for LLM quantization

448

sihyeong/Awesome-LLM-Inference-Engine

245

tiingweii-shii/Awesome-Resource-Efficient-LLM-Papers

a curated list of high-quality papers on resource-efficient LLMs 🌱

165

NVIDIA/Model-Optimizer

A unified library of SOTA model optimization techniques like quantization, distillation, pruning,…

3,847

Contributors

sharpenb

108 commits

davidberenstein1957

1 commits