MMDet to TensorRT

This project aims to convert the mmdetection model to TensorRT model end2end. Focus on object detection for now. Mask support is experiment.


  • fp16
  • int8(experiment)
  • batched input
  • dynamic input shape
  • combination of different modules
  • deepstream support

Any advices, bug reports and stars are welcome.


This project is released under the Apache 2.0 license.


  • install mmdetection:

    # mim is so cool!
    pip install openmim
    mim install mmdet==2.14.0

  • install torch2trt_dynamic:

    git clone torch2trt_dynamic
    cd torch2trt_dynamic
    python develop
  • install amirstan_plugin:

    • Install tensorrt: TensorRT

    • clone repo and build plugin

      git clone --depth=1
      cd amirstan_plugin
      git submodule update --init --progress --depth=1
      mkdir build
      cd build
      make -j10
    • DON’T FORGET setting the envoirment variable(in ~/.bashrc):

      export AMIRSTAN_LIBRARY_PATH=${amirstan_plugin_root}/build/lib



git clone
cd mmdetection-to-tensorrt
python develop


Build docker image

# cuda11.1 TensorRT7.2.2 pytorch1.8 cuda11.1
sudo docker build -t mmdet2trt_docker:v1.0 docker/

You can also specify CUDA, Pytorch and Torchvision versions with docker build args by:

# cuda11.1 tensorrt7.2.2 pytorch1.6 cuda10.2
sudo docker build -t mmdet2trt_docker:v1.0 --build-arg TORCH_VERSION=1.6.0 --build-arg TORCHVISION_VERSION=0.7.0 --build-arg CUDA=10.2 --docker/

Run (will show the help for the CLI entrypoint)

sudo docker run --gpus all -it --rm -v ${your_data_path}:${bind_path} mmdet2trt_docker:v1.0

Or if you want to open a terminal inside de container:

sudo docker run --gpus all -it --rm -v ${your_data_path}:${bind_path} --entrypoint bash mmdet2trt_docker:v1.0

Example conversion:

sudo docker run --gpus all -it --rm -v ${your_data_path}:${bind_path} mmdet2trt_docker:v1.0 ${bind_path}/ ${bind_path}/checkpoint.pth ${bind_path}/output.trt


how to create a TensorRT model from mmdet model (converting might take few minutes)(Might have some warning when converting.) detail can be found in



Run mmdet2trt -h for help on optional arguments.


<div class="highlight highlight-source-python position-relative overflow-auto" data-snippet-clipboard-copy-content="opt_shape_param=[
[1,3,320,320], # min shape
[1,3,800,1344], # optimize shape
[1,3,1344,1344], # max shape

        [1,3,320,320],      # min shape
        [1,3,800,1344],     # optimize shape
        [1,3,1344,1344],    # max shape
max_workspace_size=1<<30    # some module and tactic need large workspace.
trt_model = mmdet2trt(cfg_path, weight_path, opt_shape_param=opt_shape_param, fp16_mode=True, max_workspace_size=max_workspace_size)

# save converted model, save_model_path)

# save engine if you want to use it in c++ api
with open(save_engine_path, mode='wb') as f: