Towards High-Quality Instance Segmentation with Fine-Grained Features

This repo is the official implementation of RefineMask: Towards High-Quality Instance Segmentation with Fine-Grained Features.

Main Results

Results on COCO

Method Backbone Schedule AP AP* Checkpoint
Mask R-CNN R50-FPN 1x 34.7 36.8
RefineMask R50-FPN 1x 37.3 40.6 download
Mask R-CNN R50-FPN 2x 35.4 37.7
RefineMask R50-FPN 2x 37.8 41.2 download
Mask R-CNN R101-FPN 1x 36.1 38.4
RefineMask R101-FPN 1x 38.6 41.8 download
Mask R-CNN R101-FPN 2x 36.6 39.3
RefineMask R101-FPN 2x 39.0 42.4 download

Note: No data augmentations except standard horizontal flipping were used.

Results on LVIS

Method Backbone Schedule AP APr APc APf Checkpoint
Mask R-CNN R50-FPN 1x 22.1 10.1 21.7 30.0
RefineMask R50-FPN 1x 25.7 13.8 24.9 31.8 download
Mask R-CNN R101-FPN 1x 23.7 12.3 23.2 29.1
RefineMask R101-FPN 1x 27.1 15.6 26.2 33.1 download

Results on Cityscapes

Method Backbone Schedule AP APS APM APL Checkpoint
Mask R-CNN R50-FPN 1x 33.8 12.0 31.5 51.8
RefineMask R50-FPN 1x 37.6 14.0 35.4 57.9 download

Efficiency of RefineMask

Method AP AP* FPS
Mask R-CNN 34.7 36.8 15.7
PointRend 35.6 38.7 11.4
HTC 37.4 40.7 4.4
RefineMask 37.3 40.9 11.4

Usage

Requirements

  • Python 3.6+
  • Pytorch 1.5.0
  • mmcv-full 1.0.5

Datasets

data
  ├── coco
  |   ├── annotations
  │   │   │   ├── instances_train2017.json
  │   │   │   ├── instances_val2017.json
  │   │   │   ├── lvis_v0.5_val_cocofied.json
  │   ├── train2017
  │   │   ├── 000000004134.png
  │   │   ├── 000000031817.png
  │   │   ├── ......
  │   ├── val2017
  │   ├── test2017
  ├── lvis
  |   ├── annotations
  │   │   │   ├── lvis_v1_train.json
  │   │   │   ├── lvis_v1_val.json
  │   ├── train2017
  │   │   ├── 000000004134.png
  │   │   ├── 000000031817.png
  │   │   ├── ......
  │   ├── val2017
  │   ├── test2017
  ├── cityscapes
  |   ├── annotations
  │   │   │   ├── instancesonly_filtered_gtFine_train.json
  │   │   │   ├── instancesonly_filtered_gtFine_val.json
  │   ├── leftImg8bit
  │   |   ├── train
  │   │   ├── val
  │   │   ├── test

Note: We used the lvis-v1.0 dataset which consists of 1203 categories.

Training

./scripts/dist_train.sh ./configs/refinemask/coco/r50-refinemask-1x.py 8 work_dirs/r50-refinemask-1x

Note: The codes only support batch size 1 per GPU, and we trained all models with a total batch size 16x1. If you train models with a total batch size 8x1, the performance may drop. We will support batch size 2 or more per GPU later. You can use ./scripts/slurm_train.sh for training with multi-nodes. Multiple images per GPU during training has been supported now.

Inference

./scripts/dist_test.sh ./configs/refinemask/coco/r50-refinemask-1x.py 8 work_dirs/r50-refinemask-1x

Citation

@InProceedings{Zhang_2021_CVPR,
    author    = {Zhang, Gang and Lu, Xin and Tan, Jingru and Li, Jianmin and Zhang, Zhaoxiang and Li, Quanquan and Hu, Xiaolin},
    title     = {RefineMask: Towards High-Quality Instance Segmentation With Fine-Grained Features},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2021},
    pages     = {6861-6869}
}

GitHub

https://github.com/zhanggang001/RefineMask