ESP-DL [中文]

ESP-DL is a lightweight and efficient neural network inference framework designed specifically for ESP series chips. With ESP-DL, you can easily and quickly develop AI applications using Espressif's System on Chips (SoCs).

Overview

ESP-DL offers APIs to load, debug, and run AI models. The framework is easy to use and can be seamlessly integrated with other Espressif SDKs. ESP-PPQ serves as the quantization tool for ESP-DL, capable of quantizing models from ONNX, Pytorch, and TensorFlow, and exporting them into the ESP-DL standard model format.

• ESP-DL Standard Model Format: This format is similar to ONNX but uses FlatBuffers instead of Protobuf, making it more lightweight and supporting zero-copy deserialization, with a file extension of .espdl.
• Flexible Mixed Quantization: ESP-DL supports 8-bit (w8a8), 16-bit (w16a16), and mixed-precision (w8a16) quantization. These schemes can be freely combined within a model to preserve accuracy while maximizing inference efficiency.
• Efficient Operator Implementation: ESP-DL efficiently implements common AI operators such as Conv, Gemm, Add, and Mul. The list of supported operators. For performance benchmark data of some operators on ESP32-S3 and ESP32-P4, please refer to benchmark_report.md.
• Static Memory Planner: The memory planner automatically allocates different layers to the optimal memory location based on the user-specified internal RAM size, ensuring efficient overall running speed while minimizing memory usage.
• Dual Core Scheduling: Automatic dual-core scheduling allows computationally intensive operators to fully utilize the dual-core computing power. Currently, Conv2D and DepthwiseConv2D support dual-core scheduling.
• 8bit LUT Activation: All activation functions except for ReLU and PReLU are implemented using an 8-bit LUT (Look Up Table) method in ESP-DL to accelerate inference. You can use any activation function, and their computational complexity remains the same.

News

• [2026/9/1] Starting from ESP-DL v3.3.11, w8a16 mixed quantization is fully supported.
• [2026/7/30] Added PP-OCRv6 OCR model and example, recognizing Chinese, English and 46 Latin-script languages on-device on ESP32-P4.
• [2026/6/5] Introduce two skills, esp32s3-pie-simd and esp32p4-pie-simd, to help the code agent rewrite the code into a PIE-accelerated version.
• [2026/5/13] Introduced espdl-quantize skill, which automatically iterates to search for optimal esp-ppq quantization strategies, reducing manual tuning costs and improving quantized model accuracy. See how_to_use_espdl_quantize_skill.rst.
• [2026/4/23] Support model quantization via AutoQuant tool, which automatically explores optimal quantization strategies and parameters, significantly reducing manual tuning effort. See how_to_use_AutoQuant.rst.
• [2026/4/20] Conv and Gemm operators on ESP32-P4 have been upgraded to per-channel quantization, significantly improving model accuracy. Note: This requires ESP-PPQ ≥ 1.2.10 and ESP-DL ≥ 3.3.1. The new ESP-DL maintains backward compatibility, and existing per-tensor models are not affected.
• [2026/3/12] YOLO26 model has been added, thanks to BoumedineBillal for the contribution. Start training and deploying your own YOLO26 model.
• [2026/3/11] The ESP-DL Agent Tool has been evolved into espdl-operator skill, making it easier for AI Agents to automatically implement operators. See how_to_implement_operators_with_ai_agent.rst.
• [2026/2/27] Support quantizing models with TrainedQuantizedThresholdPass. See quantize_model_with_TQT.rst.
• [2026/1/14] Support visualizing .espdl models in netron to facilitate model debugging.
• [2025/11/4] Release ESP-DL Agent Tool v1.0. This tool is designed for code agent system to assist with implementing and optimizing operators for the ESP-DL framework.
• [2025/10/20] We released ESP-DL v3.2.0 with improved model quantization and deployment capabilities.
• [2025/07/22] Rename the ESP-PPQ package from ppq to esp-ppq to avoid conflicts with ppq, and upgrade all ESP-PPQ dependencies to the latest versions. Note: In the quantization script, pay attention to modifying the package name.
• [2025/04/30] We released a new esp-detection project and the ESPDet-Pico model, which can easily train and deploy object detection models. espdet_pico_224_224_cat espdl model and example is a cat detection model trained by esp-detection. Feel free to try it and share your feedback!
• [2025/02/18] We supported yolo11n espdl model and example.
• [2025/01/09] We updated the schema of espdl model and released ESP-DL v3.1.0. Note: previous models can be load by new schema, but new model is not compatible with previous version.
• [2024/12/20] We released ESP-DL v3.0.0.

Getting Started

Software Requirements

• ESP-IDF

ESP-DL runs based on ESP-IDF. For detailed instructions on how to get ESP-IDF, please see ESP-IDF Programming Guide.

Please use ESP-IDF release/v5.3 or above.

• ESP-PPQ

ESP-PPQ is a quantization tool based on ppq. If you want to quantize your own model, you can install esp-ppq using the following method. For more installation methods, please refer to the Getting Started.

   pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
   pip install esp-ppq

[!NOTE]

• The example code installs the Linux PyTorch CPU version. Please install the appropriate PyTorch version based on your actual needs.

Model Quantization

First, please refer to the ESP-DL Operator Support State to ensure that the operators in your model are already supported.

ESP-PPQ can directly read ONNX models for quantization. Pytorch and TensorFlow need to be converted to ONNX models first, so make sure your model can be converted to ONNX models. For more details about quantization, please refer to

how to quantize model
how to quantize MobileNetV2
how to quantize YOLO11n

Model Deployment

ESP-DL provides a series of API to quickly load and run models. A typical example is as follows:

how to load test profile model
how to run model

ESP-DL Model Zoo

The directory models contains a collection of pre-trained and quantized models optimized for ESP32 series chips. Each model is provided as an ESP-IDF component for easy integration. For more details, please refer to ESP-DL Model Zoo.

Support Operators

If you encounter unsupported operators, please point them out in the issues, and we will support them as soon as possible. Contributions to this ESP-DL are also welcomed, please refer to Creating a New Module (Operator) for more details.

ESP-DL Operator Support State