D-Robotics/rdk_model_zoo

Python

387

286 commits

updated Sep 24, 2026

See the code

README

RDK Model Zoo Logo

RDK Model Zoo

Out-of-the-Box AI Model Deployment Pipelines and Full-Link Conversion Tutorials Based on D-Robotics BPU

English | 简体中文

Stars Forks PRs Welcome License Community

Introduction

Mission: Dedicated to providing D-Robotics developers with extreme performance, out-of-the-box, and full-scenario AI deployment validation experiences.

This repository is the official collection of BPU model examples and tools (Model Zoo) provided by D-Robotics. It is oriented towards AI model deployment and application development on BPU (Brain Processing Unit), helping developers to quickly get started with BPU and fast-track model inference workflows.

The repository includes BPU-ready models across multiple AI domains and provides complete reference implementations from Original Model (PyTorch/ONNX) -> Fixed-point Quantization -> Inference Execution -> Result Parsing -> Example Validation, helping users understand and utilize BPU capabilities at minimal cost.

Core Value

  • 🚀 Quick BPU Adoption: Provides out-of-the-box inference pipelines to help users complete BPU inference validation and performance evaluation in the shortest time.
  • 🧩 Complete End-to-End Examples: Covers the entire process from algorithm export and fixed-point quantization to efficient on-board execution (.bin / .hbm). Includes model loading, preprocessing, BPU inference execution, post-processing, and result visualization.
  • 📐 Standardized Design & Documentation: Provides unified directory structures and sample code specifications, supporting Python (hbm_runtime) and C/C++ interfaces for easy understanding, secondary development, and reduced integration/maintenance costs.
  • 🌐 Full Scenario Coverage: Covers classification, detection, segmentation, pose estimation, OCR, and multi-modal models.

Hardware & System Support

This repository uses hardware-specific branches to keep maintained samples, legacy demos, and board-specific documents clearly separated. The current rdk_x5 branch is the primary delivery branch for RDK X5. The previous main branch has been renamed to rdk_x5_legacy and is kept only as the historical archive branch.

Target HardwareBranchDescription
RDK X5rdk_x5Primary delivery branch for RDK X5. Recommended system version: RDK OS >= 3.5.0, based on Ubuntu 22.04 aarch64 and TROS-Humble.
RDK X5 legacy demosrdk_x5_legacyHistorical archive branch for the previous RDK X5 demos. Use it only when you need to reference legacy demo content.
RDK X3rdk_x3Branch for RDK X3 devices.
RDK S seriesrdk_sBranch for RDK S series boards. Historical archived demos for RDK S series boards are kept in RDK Model Zoo S.

Browse the online model catalog →

The online catalog presents released models as searchable cards with model assets, measured performance and accuracy results, and their documented test conditions.


Directory Structure

Click to expand project directory architecture
rdk_model_zoo/
|-- samples/
|   |-- robotics/
|   |   `-- himloco/              # Unitree Go2 locomotion policy
|   |-- vision/
|   |   |-- clip/                 # Image-text multimodal matching
|   |   |-- convnext/             # Image classification
|   |   |-- edgenext/             # Image classification
|   |   |-- efficientformer/      # Image classification
|   |   |-- efficientformerv2/    # Image classification
|   |   |-- efficient_sam/        # Promptable image segmentation
|   |   |-- mobile_sam/           # Promptable image segmentation
|   |   |-- efficientnet/         # Image classification
|   |   |-- efficientvit/         # Image classification
|   |   |-- fasternet/            # Image classification
|   |   |-- fastvit/              # Image classification
|   |   |-- fcos/                 # Object detection
|   |   |-- googlenet/            # Image classification
|   |   |-- hgnetv2/              # Image classification
|   |   |-- lprnet/               # License plate recognition
|   |   |-- mobilenetv1/          # Image classification
|   |   |-- mobilenetv2/          # Image classification
|   |   |-- mobilenetv3/          # Image classification
|   |   |-- mobilenetv4/          # Image classification
|   |   |-- mobileone/            # Image classification
|   |   |-- modnet/               # Image matting
|   |   |-- paddleocr/            # OCR text detection and recognition
|   |   |-- pp_liteseg/           # Semantic segmentation
|   |   |-- repghost/             # Image classification
|   |   |-- repvgg/               # Image classification
|   |   |-- repvit/               # Image classification
|   |   |-- resnet/               # Image classification
|   |   |-- resnext/              # Image classification
|   |   |-- unet/                 # Semantic segmentation
|   |   |-- ultralytics_yolo/     # Detection, segmentation, pose, classification
|   |   |-- ultralytics_yolo26/   # Detection, segmentation, pose, classification
|   |   |-- vargconvnet/          # Image classification
|   |   |-- yolo26_depth/         # Monocular depth estimation
|   |   |-- yolov5/               # Object detection
|   |   `-- yoloworld/           # Open-vocabulary object detection
|-- docs/                  # Project guidelines and reference documentation
|   `-- tros/              # TROS integration guides and examples
|-- datasets/              # Sample datasets and download scripts
|-- skills/                # RDK Model Zoo skills (placeholder)
|-- utils/                 # Shared C++ / Python utilities

Quick Start

  1. Check system version: Ensure the target board is running RDK OS >= 3.5.0.
  2. Connect hardware: Ensure your RDK board is powered and network-connected. SSH or VSCode Remote SSH is recommended.
  3. Read the model README first: Always open the target directory README.md before running commands.
  4. Run the Ultralytics YOLO11x detection sample:
cd samples/vision/ultralytics_yolo/model
wget -nc https://archive.d-robotics.cc/downloads/rdk_model_zoo/rdk_x5/ultralytics_YOLO/yolo11x_detect_bayese_640x640_nv12.bin

cd ../runtime/python
python3 main.py \
  --task detect \
  --model-path ../../model/yolo11x_detect_bayese_640x640_nv12.bin \
  --test-img ../../../../../datasets/coco/assets/bus.jpg \
  --img-save-path ../../test_data/inference_yolo11x.jpg

Inference Result:

YOLO11x Inference Result

Model List

CategoryModel NameModel PathSupported PlatformDetails
Image ClassificationConvNeXtsamples/vision/convnextRDK X5Details
Image ClassificationEdgeNeXtsamples/vision/edgenextRDK X5Details
Image ClassificationEfficientFormersamples/vision/efficientformerRDK X5Details
Image ClassificationEfficientFormerV2samples/vision/efficientformerv2RDK X5Details
Image ClassificationEfficientNetsamples/vision/efficientnetRDK X5Details
Image ClassificationEfficientViTsamples/vision/efficientvitRDK X5Details
Image ClassificationFasterNetsamples/vision/fasternetRDK X5Details
Image ClassificationFastViTsamples/vision/fastvitRDK X5Details
Image ClassificationGoogLeNetsamples/vision/googlenetRDK X5Details
Image ClassificationHGNetV2samples/vision/hgnetv2RDK X5Details
Image ClassificationMobileNetV1samples/vision/mobilenetv1RDK X5Details
Image ClassificationMobileNetV2samples/vision/mobilenetv2RDK X5Details
Image ClassificationMobileNetV3samples/vision/mobilenetv3RDK X5Details
Image ClassificationMobileNetV4samples/vision/mobilenetv4RDK X5Details
Image ClassificationMobileOnesamples/vision/mobileoneRDK X5Details
Image ClassificationRepGhostsamples/vision/repghostRDK X5Details
Image ClassificationRepVGGsamples/vision/repvggRDK X5Details
Image ClassificationRepViTsamples/vision/repvitRDK X5Details
Image ClassificationResNetsamples/vision/resnetRDK X5Details
Image ClassificationResNeXtsamples/vision/resnextRDK X5Details
Image ClassificationVargConvNetsamples/vision/vargconvnetRDK X5Details
Promptable Image SegmentationEfficientSAM-Tinysamples/vision/efficient_samRDK X5Details
Promptable Image SegmentationMobileSAMsamples/vision/mobile_samRDK X5Details
Semantic SegmentationPP-LiteSegsamples/vision/pp_litesegRDK X5Details
Semantic SegmentationUNet ResNet Familysamples/vision/unetRDK X5Details
Object DetectionFCOSsamples/vision/fcosRDK X5Details
Object DetectionYOLOv5samples/vision/yolov5RDK X5Details
Object Detection / Instance Segmentation / Pose Estimation / Image ClassificationUltralytics YOLO (YOLOv5u / YOLOv8 / YOLOv9 / YOLOv10 / YOLO11 / YOLO12 / YOLO13)samples/vision/ultralytics_yoloRDK X5Details
Object Detection / Instance Segmentation / Pose Estimation / Image ClassificationUltralytics YOLO26samples/vision/ultralytics_yolo26RDK X5Details
Monocular Depth EstimationYOLO26 Depthsamples/vision/yolo26_depthRDK X5Details
Image MattingMODNetsamples/vision/modnetRDK X5Details
OCR Text Detection and RecognitionPaddleOCRsamples/vision/paddleocrRDK X5Details
License Plate RecognitionLPRNetsamples/vision/lprnetRDK X5Details
Image-Text Multimodal MatchingCLIPsamples/vision/clipRDK X5Details
Open-Vocabulary Object DetectionYOLOWorldsamples/vision/yoloworldRDK X5Details
Legged LocomotionHIMLoco (Unitree Go2)samples/robotics/himlocoRDK X5Details

Documentation & Resources


FAQ

1. Model accuracy doesn't meet expectations?
  • Ensure OpenExplorer Docker and board-side libdnn.so versions are up-to-date.
  • Check if model export followed the structure adjustments/operator replacements required in the model's README.
  • Verify cosine similarity of each output node is >= 0.999 (minimum 0.99) during quantization validation.
2. Inference speed doesn't meet expectations?
  • Python API performance is lower than C/C++. For maximum performance, use C/C++.
  • Benchmark data (pure forward) excludes pre/post-processing. Models with NV12 input usually achieve peak BPU throughput.
  • Ensure CPU/BPU frequency is locked to maximum.
  • Check for other resource-heavy processes.
3. How to fix quantization precision loss?
  • Refer to the PTQ accuracy debugging section in the platform documentation.
  • If INT8 loss is severe due to model characteristics, consider Mixed Precision or QAT (Quantization-Aware Training).
4. Error "Can't reshape 1354752 in (1,3,640,640)"?

Update the resolution in preprocess.py to match your ONNX model's input size. Delete old calibration data and re-run the calibration script.

5. mAP accuracy is lower than official results (e.g., Ultralytics)?
  • Deployment uses fixed shape and INT8 quantization, unlike dynamic shape/float official tests.
  • Slight implementation differences in evaluation scripts (e.g., pycocotools).
  • NCHW-RGB to NV12 conversion adds minimal pixel-level loss.
6. Does the model use CPU during inference?

Yes. Non-quantizable or BPU-unsupported operators fallback to CPU. Even for pure BPU models, input/output quantization/dequantization nodes are executed by the CPU.


Community & Contribution

Star History

Star History Chart

We warmly welcome contributions! Please raise an issue on GitHub Issues or discuss on the Developer Community.

License

This project is licensed under the Apache License 2.0 agreement.

Contributors

maxma615

144 commits

Bartonzany

54 commits

WuChao-2024

49 commits

frank05080

11 commits

D-Robotics/rdk_model_zoo

Python

387

286 commits

updated Sep 24, 2026

See the code

README

RDK Model Zoo Logo

RDK Model Zoo

Out-of-the-Box AI Model Deployment Pipelines and Full-Link Conversion Tutorials Based on D-Robotics BPU

English | 简体中文

Stars Forks PRs Welcome License Community

Introduction

Mission: Dedicated to providing D-Robotics developers with extreme performance, out-of-the-box, and full-scenario AI deployment validation experiences.

This repository is the official collection of BPU model examples and tools (Model Zoo) provided by D-Robotics. It is oriented towards AI model deployment and application development on BPU (Brain Processing Unit), helping developers to quickly get started with BPU and fast-track model inference workflows.

The repository includes BPU-ready models across multiple AI domains and provides complete reference implementations from Original Model (PyTorch/ONNX) -> Fixed-point Quantization -> Inference Execution -> Result Parsing -> Example Validation, helping users understand and utilize BPU capabilities at minimal cost.

Core Value

  • 🚀 Quick BPU Adoption: Provides out-of-the-box inference pipelines to help users complete BPU inference validation and performance evaluation in the shortest time.
  • 🧩 Complete End-to-End Examples: Covers the entire process from algorithm export and fixed-point quantization to efficient on-board execution (.bin / .hbm). Includes model loading, preprocessing, BPU inference execution, post-processing, and result visualization.
  • 📐 Standardized Design & Documentation: Provides unified directory structures and sample code specifications, supporting Python (hbm_runtime) and C/C++ interfaces for easy understanding, secondary development, and reduced integration/maintenance costs.
  • 🌐 Full Scenario Coverage: Covers classification, detection, segmentation, pose estimation, OCR, and multi-modal models.

Hardware & System Support

This repository uses hardware-specific branches to keep maintained samples, legacy demos, and board-specific documents clearly separated. The current rdk_x5 branch is the primary delivery branch for RDK X5. The previous main branch has been renamed to rdk_x5_legacy and is kept only as the historical archive branch.

Target HardwareBranchDescription
RDK X5rdk_x5Primary delivery branch for RDK X5. Recommended system version: RDK OS >= 3.5.0, based on Ubuntu 22.04 aarch64 and TROS-Humble.
RDK X5 legacy demosrdk_x5_legacyHistorical archive branch for the previous RDK X5 demos. Use it only when you need to reference legacy demo content.
RDK X3rdk_x3Branch for RDK X3 devices.
RDK S seriesrdk_sBranch for RDK S series boards. Historical archived demos for RDK S series boards are kept in RDK Model Zoo S.

Browse the online model catalog →

The online catalog presents released models as searchable cards with model assets, measured performance and accuracy results, and their documented test conditions.


Directory Structure

Click to expand project directory architecture
rdk_model_zoo/
|-- samples/
|   |-- robotics/
|   |   `-- himloco/              # Unitree Go2 locomotion policy
|   |-- vision/
|   |   |-- clip/                 # Image-text multimodal matching
|   |   |-- convnext/             # Image classification
|   |   |-- edgenext/             # Image classification
|   |   |-- efficientformer/      # Image classification
|   |   |-- efficientformerv2/    # Image classification
|   |   |-- efficient_sam/        # Promptable image segmentation
|   |   |-- mobile_sam/           # Promptable image segmentation
|   |   |-- efficientnet/         # Image classification
|   |   |-- efficientvit/         # Image classification
|   |   |-- fasternet/            # Image classification
|   |   |-- fastvit/              # Image classification
|   |   |-- fcos/                 # Object detection
|   |   |-- googlenet/            # Image classification
|   |   |-- hgnetv2/              # Image classification
|   |   |-- lprnet/               # License plate recognition
|   |   |-- mobilenetv1/          # Image classification
|   |   |-- mobilenetv2/          # Image classification
|   |   |-- mobilenetv3/          # Image classification
|   |   |-- mobilenetv4/          # Image classification
|   |   |-- mobileone/            # Image classification
|   |   |-- modnet/               # Image matting
|   |   |-- paddleocr/            # OCR text detection and recognition
|   |   |-- pp_liteseg/           # Semantic segmentation
|   |   |-- repghost/             # Image classification
|   |   |-- repvgg/               # Image classification
|   |   |-- repvit/               # Image classification
|   |   |-- resnet/               # Image classification
|   |   |-- resnext/              # Image classification
|   |   |-- unet/                 # Semantic segmentation
|   |   |-- ultralytics_yolo/     # Detection, segmentation, pose, classification
|   |   |-- ultralytics_yolo26/   # Detection, segmentation, pose, classification
|   |   |-- vargconvnet/          # Image classification
|   |   |-- yolo26_depth/         # Monocular depth estimation
|   |   |-- yolov5/               # Object detection
|   |   `-- yoloworld/           # Open-vocabulary object detection
|-- docs/                  # Project guidelines and reference documentation
|   `-- tros/              # TROS integration guides and examples
|-- datasets/              # Sample datasets and download scripts
|-- skills/                # RDK Model Zoo skills (placeholder)
|-- utils/                 # Shared C++ / Python utilities

Quick Start

  1. Check system version: Ensure the target board is running RDK OS >= 3.5.0.
  2. Connect hardware: Ensure your RDK board is powered and network-connected. SSH or VSCode Remote SSH is recommended.
  3. Read the model README first: Always open the target directory README.md before running commands.
  4. Run the Ultralytics YOLO11x detection sample:
cd samples/vision/ultralytics_yolo/model
wget -nc https://archive.d-robotics.cc/downloads/rdk_model_zoo/rdk_x5/ultralytics_YOLO/yolo11x_detect_bayese_640x640_nv12.bin

cd ../runtime/python
python3 main.py \
  --task detect \
  --model-path ../../model/yolo11x_detect_bayese_640x640_nv12.bin \
  --test-img ../../../../../datasets/coco/assets/bus.jpg \
  --img-save-path ../../test_data/inference_yolo11x.jpg

Inference Result:

YOLO11x Inference Result

Model List

CategoryModel NameModel PathSupported PlatformDetails
Image ClassificationConvNeXtsamples/vision/convnextRDK X5Details
Image ClassificationEdgeNeXtsamples/vision/edgenextRDK X5Details
Image ClassificationEfficientFormersamples/vision/efficientformerRDK X5Details
Image ClassificationEfficientFormerV2samples/vision/efficientformerv2RDK X5Details
Image ClassificationEfficientNetsamples/vision/efficientnetRDK X5Details
Image ClassificationEfficientViTsamples/vision/efficientvitRDK X5Details
Image ClassificationFasterNetsamples/vision/fasternetRDK X5Details
Image ClassificationFastViTsamples/vision/fastvitRDK X5Details
Image ClassificationGoogLeNetsamples/vision/googlenetRDK X5Details
Image ClassificationHGNetV2samples/vision/hgnetv2RDK X5Details
Image ClassificationMobileNetV1samples/vision/mobilenetv1RDK X5Details
Image ClassificationMobileNetV2samples/vision/mobilenetv2RDK X5Details
Image ClassificationMobileNetV3samples/vision/mobilenetv3RDK X5Details
Image ClassificationMobileNetV4samples/vision/mobilenetv4RDK X5Details
Image ClassificationMobileOnesamples/vision/mobileoneRDK X5Details
Image ClassificationRepGhostsamples/vision/repghostRDK X5Details
Image ClassificationRepVGGsamples/vision/repvggRDK X5Details
Image ClassificationRepViTsamples/vision/repvitRDK X5Details
Image ClassificationResNetsamples/vision/resnetRDK X5Details
Image ClassificationResNeXtsamples/vision/resnextRDK X5Details
Image ClassificationVargConvNetsamples/vision/vargconvnetRDK X5Details
Promptable Image SegmentationEfficientSAM-Tinysamples/vision/efficient_samRDK X5Details
Promptable Image SegmentationMobileSAMsamples/vision/mobile_samRDK X5Details
Semantic SegmentationPP-LiteSegsamples/vision/pp_litesegRDK X5Details
Semantic SegmentationUNet ResNet Familysamples/vision/unetRDK X5Details
Object DetectionFCOSsamples/vision/fcosRDK X5Details
Object DetectionYOLOv5samples/vision/yolov5RDK X5Details
Object Detection / Instance Segmentation / Pose Estimation / Image ClassificationUltralytics YOLO (YOLOv5u / YOLOv8 / YOLOv9 / YOLOv10 / YOLO11 / YOLO12 / YOLO13)samples/vision/ultralytics_yoloRDK X5Details
Object Detection / Instance Segmentation / Pose Estimation / Image ClassificationUltralytics YOLO26samples/vision/ultralytics_yolo26RDK X5Details
Monocular Depth EstimationYOLO26 Depthsamples/vision/yolo26_depthRDK X5Details
Image MattingMODNetsamples/vision/modnetRDK X5Details
OCR Text Detection and RecognitionPaddleOCRsamples/vision/paddleocrRDK X5Details
License Plate RecognitionLPRNetsamples/vision/lprnetRDK X5Details
Image-Text Multimodal MatchingCLIPsamples/vision/clipRDK X5Details
Open-Vocabulary Object DetectionYOLOWorldsamples/vision/yoloworldRDK X5Details
Legged LocomotionHIMLoco (Unitree Go2)samples/robotics/himlocoRDK X5Details

Documentation & Resources


FAQ

1. Model accuracy doesn't meet expectations?
  • Ensure OpenExplorer Docker and board-side libdnn.so versions are up-to-date.
  • Check if model export followed the structure adjustments/operator replacements required in the model's README.
  • Verify cosine similarity of each output node is >= 0.999 (minimum 0.99) during quantization validation.
2. Inference speed doesn't meet expectations?
  • Python API performance is lower than C/C++. For maximum performance, use C/C++.
  • Benchmark data (pure forward) excludes pre/post-processing. Models with NV12 input usually achieve peak BPU throughput.
  • Ensure CPU/BPU frequency is locked to maximum.
  • Check for other resource-heavy processes.
3. How to fix quantization precision loss?
  • Refer to the PTQ accuracy debugging section in the platform documentation.
  • If INT8 loss is severe due to model characteristics, consider Mixed Precision or QAT (Quantization-Aware Training).
4. Error "Can't reshape 1354752 in (1,3,640,640)"?

Update the resolution in preprocess.py to match your ONNX model's input size. Delete old calibration data and re-run the calibration script.

5. mAP accuracy is lower than official results (e.g., Ultralytics)?
  • Deployment uses fixed shape and INT8 quantization, unlike dynamic shape/float official tests.
  • Slight implementation differences in evaluation scripts (e.g., pycocotools).
  • NCHW-RGB to NV12 conversion adds minimal pixel-level loss.
6. Does the model use CPU during inference?

Yes. Non-quantizable or BPU-unsupported operators fallback to CPU. Even for pure BPU models, input/output quantization/dequantization nodes are executed by the CPU.


Community & Contribution

Star History

Star History Chart

We warmly welcome contributions! Please raise an issue on GitHub Issues or discuss on the Developer Community.

License

This project is licensed under the Apache License 2.0 agreement.

Contributors

maxma615

144 commits

Bartonzany

54 commits

WuChao-2024

49 commits

frank05080

11 commits

Languages

Python

99.3%