Unseen Object Amodal Instance Segmentation (UOAIS)

Seunghyeok Back, Joosoon Lee, Taewon Kim, Sangjun Noh, Raeyoung Kang, Seongho Bak, Kyoobin Lee

This repository contains source codes for the paper “Unseen Object Amodal Instance Segmentation via Hierarchical Occlusion Modeling.”

[Paper] [Project Website] [Video]

Updates & TODO Lists

(2021.09.26) UOAIS-Net has been released
Add train and evaluation code
Release synthetic dataset (UOAIS-Sim) and amodal annotation (OSD-Amodal)
Add ROS inference node

Getting Started

Environment Setup

Tested on Titan RTX with python 3.7, pytorch 1.8.0, torchvision 0.9.0, CUDA 10.2.

Download

git clone https://github.com/gist-ailab/uoais.git
cd uoais
mkdir output

Download the checkpoint at GDrive and move the downloaded folders to the output folder

Set up a python environment

conda create -n uoais python=3.7
conda activate uoais
pip install torch torchvision 
pip install shapely torchfile opencv-python pyfastnoisesimd rapidfuzz

Install detectron2
Build and install custom AdelaiDet

python setup.py build develop

Run with Sample Data

UOAIS-Net (RGB-D)

python tools/run_sample_data.py

License

This repository is released under the MIT license.

Citation

If you use our work in a research project, please cite our work:

@misc{back2021unseen,
      title={Unseen Object Amodal Instance Segmentation via Hierarchical Occlusion Modeling}, 
      author={Seunghyeok Back and Joosoon Lee and Taewon Kim and Sangjun Noh and Raeyoung Kang and Seongho Bak and Kyoobin Lee},
      year={2021},
      eprint={2109.11103},
      archivePrefix={arXiv},
      primaryClass={cs.RO}
}