tiny-tensorrt

a simple, efficient, easy-to-use nvidia TensorRT wrapper for cnn with c++ and python api,support caffe, uff and onnx format models. you will be able use tiny-tensorrt deploy your model with few lines of code!

// create engine
trt.CreateEngine(prototxt,caffemodel,engingefile,outputBlob,calibratorData,maxBatchSize,runMode);
// transfer you input data to tensorrt engine
trt.DataTransfer(input,0,True);
// inference!!!
trt.Forward();
//  retrieve network output
trt.DataTransfer(output, outputIndex, False) // you can get outputIndex in CreateEngine phase

Features

  • [x] Support TensorRT 7 now --- 2019-12-25 :christmas_tree::christmas_tree::christmas_tree:
  • [x] Custom plugin tutorial and well_commented sample! ---2019-12-11 :fire::fire::fire:
  • [x] Custom onnx model output node ---2019.10.18
  • [x] Upgrade with TensorRT 6.0.1.5 --- 2019.9.29
  • [x] Support onnx,caffe and tensorflow model
  • [ ] Support more model and layer --working on
  • [x] PReLU and up-sample plugin
  • [x] Engine serialization and deserialization
  • [x] INT8 support for caffe model
  • [x] Python api support
  • [x] Set device

System Requirements

cuda 10.0+

TensorRT 6 or 7

for python api, python 2.x/3.x and numpy in needed

this project is fully test in ubuntu 16.04. I tested it with 1060ti, 1050ti, 1080ti, 1660ti, 2080, 2080ti and p4.

GitHub