TransTrack

Multiple-Object Tracking with Transformer.

Introduction

TransTrack: Multiple-Object Tracking with Transformer

Models

Training data Training time Validation MOTA download
crowdhuman, mot_half 36h + 1h 65.4 model
crowdhuman 36h 53.8 model
mot_half 8h 61.6 model

Models are also available in Baidu Drive by code m4iv.

Notes

  • Evaluating crowdhuman-training model and mot-training model use different command lines, see Steps.
  • We observe about 1 MOTA noise.
  • If the resulting MOTA of your self-trained model is not desired, playing around with the --track_thresh sometimes gives a better performance.
  • The training time is on 8 NVIDIA V100 GPUs with batchsize 16.
  • We use the models pre-trained on imagenet.

Installation

The codebases are built on top of Deformable DETR and CenterTrack.

Requirements

  • Linux or macOS with Python ≥ 3.6
  • PyTorch ≥ 1.5 and torchvision that matches the PyTorch installation.
    You can install them together at pytorch.org to make sure of this
  • OpenCV is optional and needed by demo and visualization

Steps

  1. Install and build libs
git clone https://github.com/PeizeSun/TransTrack.git
cd TransTrack
cd models/ops
python setup.py build install
cd ../..
pip install -r requirements.txt
  1. Prepare dataset
mkdir -p crowdhuman/annotations
cp -r /path_to_crowdhuman_dataset/annotations/CrowdHuman_val.json crowdhuman/annotations/CrowdHuman_val.json
cp -r /path_to_crowdhuman_dataset/annotations/CrowdHuman_train.json crowdhuman/annotations/CrowdHuman_train.json
cp -r /path_to_crowdhuman_dataset/CrowdHuman_train crowdhuman/CrowdHuman_train
cp -r /path_to_crowdhuman_dataset/CrowdHuman_val crowdhuman/CrowdHuman_val
mkdir mot
cp -r /path_to_mot_dataset/train mot/train
cp -r /path_to_mot_dataset/test mot/test
python track_tools/convert_mot_to_coco.py

CrowdHuman dataset is available in CrowdHuman. We provide annotations of json format.

MOT dataset is available in MOT.

  1. Pre-train on crowdhuman
sh track_exps/crowdhuman_train.sh
python track_tools/crowdhuman_model_to_mot.py

The pre-trained model is available crowdhuman_final.pth.

  1. Train TransTrack
sh track_exps/crowdhuman_mot_trainhalf.sh
  1. Evaluate TransTrack
sh track_exps/mot_val.sh
sh track_exps/mot_eval.sh

Notes

  • Evaluate pre-trained CrowdHuman model on MOT
sh track_exps/det_val.sh
sh track_exps/mot_eval.sh

GitHub