3D_adapt_auto_driving

This paper has been accpeted by Conference on Computer Vision and Pattern Recognition (CVPR) 2020.

Train in Germany, Test in The USA: Making 3D Object Detectors Generalize

by Yan Wang*, Xiangyu Chen*, Yurong You, Li Erran, Bharath Hariharan, Mark Campbell, Kilian Q. Weinberger, Wei-Lun Chao*

Dependencies

Usage

Prepare Datasets (Jupyter notebook)

We develop our method on these datasets:

  1. Configure dataset_path in config_path.py.

    Raw datasets will be organized as the following structure:

     dataset_path/
         | kitti/               # KITTI object detection 3D dataset
             | training/
             | testing/
         | argo/                # Argoverse dataset v1.1
             | train1/
             | train2/
             | train3/
             | train4/
             | val/
             | test/
         | nusc/                # nuScenes dataset v1.0
             | maps/
             | samples/
             | sweeps/
             | v1.0-trainval/
         | lyft/                # Lyft Level 5 dataset v1.02
             | v1.02-train/
         | waymo/               # Waymo dataset v1.0
             | training/
             | validation/
     
  2. Download all datasets.

    For KITTI, Argoverse and Waymo, we provide scripts for automatic download.

    cd scripts/
    python download.py [--datasets kitti+argo+waymo]
    

    nuScenes and Lyft need to downloaded manually.

  3. Convert all datasets to KITTI format.

    cd scripts/
    python -m pip install -r convert_requirements.txt
    python convert.py [--datasets argo+nusc+lyft+waymo]
    
  4. Split validation set

    We provide the train/val split used in our experiments under split folder.

    cd split/
    python replace_split.py
    
  5. Generate car subset

    We filter scenes and only keep those with cars.

    cd scripts/
    python gen_car_split.py
    

Statistical Normalization (Jupyter notebook)

  1. Compute car size statistics of each dataset.
    The computed statistics are stored as label_stats_{train/val/test}.json under KITTI format dataset root.

    cd stat_norm/
    python stat.py
    
  2. Generate rescaled datasets according to car size statistics.
    The rescaled datasets are stored under $dataset_path/rescaled_datasets by default.

    cd stat_norm/
    python norm.py [--path $PATH]
    

Training (To be updated)

We use PointRCNN to validate our method.

  1. Setup PointRCNN

    cd pointrcnn/
    ./build_and_install.sh
    
  2. Build datasets in PointRCNN format.

    cd pointrcnn/tools/
    python generate_multi_data.py
    python generate_gt_database.py --root ...
    
  3. Download the models pretrained on source domains from google drive using gdrive.

    cd pointrcnn/tools/
    gdrive download -r 14MXjNImFoS2P7YprLNpSmFBsvxf5J2Kw
    
  4. Adapt to a new domain by re-training with rescaled data.

    cd pointrcnn/tools/
    
    python train_rcnn.py --cfg_file ...
    

Inference

cd pointrcnn/tools/
python eval_rcnn.py --ckpt /path/to/checkpoint.pth --dataset $dataset --output_dir $output_dir 

Evaluation

We provide evaluation code with

  • old (based on bbox height) and new (based on distance) difficulty metrics
  • output transformation functions to locate domain gap
python evaluate/
python evaluate.py --result_path $predictions --dataset_path $dataset_root --metric [old/new]

Citation

@inproceedings{wang2020train,
  title={Train in germany, test in the usa: Making 3d object detectors generalize},
  author={Yan Wang and Xiangyu Chen and Yurong You and Li Erran and Bharath Hariharan and Mark Campbell and Kilian Q. Weinberger and Wei-Lun Chao},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={11713-11723},
  year={2020}
}

GitHub

https://github.com/cxy1997/3D_adapt_auto_driving