linghu8812/tensorrt_inference

C++

716

94 commits

updated Aug 23, 2023

See the code

README

TensorRT Models Deploy from ONNX

Install Depends

see INSTALL.md

Build from Docker

docker build -t tensorrt_inference:0.1.0_rc .

How to Use

Build tensorrt_inference project:

git clone https://github.com/linghu8812/tensorrt_inference.git
cd tensorrt_inference/project
mkdir build && cd build
cmake ..
make -j

after these steps, the executable program tensorrt_inference and all support model shared libraries have been built.

Run tensorrt_inference

./tensorrt_inference

it will show:

Please design model arch, config file and folder name!

Inference with models, take yolov5 model as example, put yolov5s.onnx model into weights folder:

cd ../../bin/
./tensorrt_inference yolov5 ../configs/yolov5/config.yaml ../samples/detection_segmentation

Use one model library example can reference here example

Supported Models

download transfered ONNX models, and put them into folder weights: https://pan.baidu.com/s/1Ff_SA9Q66DUnZjSipPa74Q, code: opkp

models	framework	instruction
lenet	PyTorch	An example from model training to TensorRT model deploy
alexnet	MXNet Gluon	MXNet Gluon example
arcface	MXNet Symbol	MXNet Symbol and face recognition example
CenterFace	ONNX	rewrite ONNX model and face detection example
efficientnet	Keras	Keras to ONNX example
face_alignment	MXNet Symbol	MXNet Symbol and face key points detection example
fast-reid	PyTorch	PyTorch and pedestrian reid example
FCN	GluonCV	MXNet GluonCV semantic segmentation example
gender-age	MXNet Symbol	MXNet Symbol and face gender and age recognize example
ghostnet	PyTorch	PyTorch example
MiniFASNet	PyTorch	PyTorch face anti spoofing example
mmpose	PyTorch	PyTorch person key points detect example
nanodet	PyTorch	PyTorchlightweight anchor-free object detection example
RetinaFace	MXNet Symbol	MXNet Symbol and face detection example
ScaledYOLOv4	PyTorch	YOLOv4 large with PyTorch implementation
scrfd	PyTorch	PyTorch scrfd face detection example
seresnext	PyTorch	PyTorch example
Swin-Transformer	timm	timm image classification example
yolor	PyTorch	PyTorch and object detection example
Yolov4	darknet	darknet and object detection example
yolov5	PyTorch	PyTorch and object detection example
YOLOv6	PyTorch	PyTorch and object detection example
yolov7	PyTorch	PyTorch and object detection example
yolov8	ultralytics	ultralytics object detection example

Discussion Groups

QQ group: 588966055

Contributors

87 commits

3 commits

2 commits

1 commits

Similar repos

wang-xinyu/tensorrtx

Implementation of popular deep learning networks with TensorRT network definition API

7,835

jkjung-avt/tensorrt_demos

TensorRT MODNet, YOLOv4, YOLOv3, SSD, MTCNN, and GoogLeNet

1,792

marcoslucianops/DeepStream-Yolo

NVIDIA DeepStream SDK 8.0 / 7.1 / 7.0 / 6.4 / 6.3 / 6.2 / 6.1.1 / 6.1 / 6.0.1 / 6.0 / 5.1…

2,071

Tianxiaomo/pytorch-YOLOv4

PyTorch ,ONNX and TensorRT implementation of YOLOv4

4,520

Tencent/TNN

TNN: developed by Tencent Youtu Lab and Guangying Lab, a uniform deep learning inference framework…

4,654

Tencent-Hunyuan/TensorRT-libs

Model

HunyuanDiT TensorRT Acceleration

triton-inference-server/server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

11,003

NVIDIA-AI-IOT/yolov4_deepstream

yolo model qat and deploy with deepstream&tensorrt

607

Languages

C++

50.9%

Python

36.4%

CMake

11.7%