saic_depth_completion

Official implementation of "Decoder Modulation for Indoor Depth Completion".

Installation

This implementation is based on Python 3+ and Pytorch 1.4+. We provide two ways of setting up an environment. If you are using Anaconda, the following code performs necessary installation:

conda env create -f environment.yaml
conda activate depth-completion
python setup.py install

The same procedure can be done with pip:

pip3 install -r requirements.txt
python setup.py install

Training

We provide a code for training on Matterport3D. Download Matterpord3D dataset and reorder your root folder as follows:

ROOT/
  ├── data/
  └── splits/
        ├── train.txt
        ├── val.txt
        └── test.txt

and data directory is should be configured in this order. Be sure that ROOT path in matterport.py is valid.
Now you can start training with the following command:

# for LRN decoder with efficientnet-b4 backbone
python train_matterport.py --default_cfg='LRN' --config_file='../configs/LRN_efficientnet-b4_lena.yaml' --postfix='example_lrn' 
# for DM-LRN decoder with efficientnet-b4 backbone
python train_matterport.py --default_cfg='DM-LRN' --config_file='../configs/DM-LRN_efficientnet-b4_pepper.yaml' --postfix='example_dm_lrn'

Evaluation

We provide scripts for evaluation on Matterport3D. If you need to perform test on NYUv2, see directly into a code since it may be changed in the future. Following instructions performs evaluation on Matterport3D test set:

# for LRN decoder with efficientnet-b4 backbone
python test_net.py --default_cfg='LRN' --config_file='../configs/LRN_efficientnet-b4_lena.yaml' --weights=<path to lrn_b4.pth>
# for DM-LRN decoder with efficientnet-b4 backbone
python test_net.py --default_cfg='DM-LRN' --config_file='../configs/DM-LRN_efficientnet-b4_pepper.yaml' --weights=<path to dm-lrn_b4.pth>
# if you need to visualize the results just add --save_dir argument
python test_net.py --default_cfg='DM-LRN' --config_file='../configs/DM-LRN_efficientnet-b4_pepper.yaml' --weights=<path to dm-lrn_b4.pth> --save_dir=<path to existing folder>

Model ZOO

This repository includes all models mentioned in original paper.

Backbone Decoder
type		 Encoder
input		 Training loss Link Config
efficientnet-b0 LRN RGBD LogDepthL1loss [lrn_b0.pth][lrn_b0] LRN_efficientnet-b0_suzy.yaml
efficientnet-b1 LRN RGBD LogDepthL1loss [lrn_b1.pth][lrn_b1] LRN_efficientnet-b1_anabel.yaml
efficientnet-b2 LRN RGBD LogDepthL1loss [lrn_b2.pth][lrn_b2] LRN_efficientnet-b2_irina.yaml
efficientnet-b3 LRN RGBD LogDepthL1loss [lrn_b3.pth][lrn_b3] LRN_efficientnet-b3_sara.yaml
efficientnet-b4 LRN RGBD LogDepthL1loss [lrn_b4.pth][lrn_b4] LRN_efficientnet-b4_lena.yaml
efficientnet-b4 LRN RGBD BerHu [lrn_b4_berhu.pth][lrn_b4_berhu] LRN_efficientnet-b4_helga.yaml
efficientnet-b4 LRN RGBD+M LogDepthL1loss [lrn_b4_mask.pth][lrn_b4_mask] LRN_efficientnet-b4_simona.yaml
efficientnet-b0 DM-LRN RGBD LogDepthL1Loss [dm-lrn_b0.pth][dm-lrn_b0] DM_LRN_efficientnet-b0_camila.yaml
efficientnet-b1 DM-LRN RGBD LogDepthL1Loss [dm-lrn_b1.pth][dm-lrn_b1] DM_LRN_efficientnet-b1_pamela.yaml
efficientnet-b2 DM-LRN RGBD LogDepthL1Loss [dm-lrn_b2.pth][dm-lrn_b2] DM_LRN_efficientnet-b2_rosaline.yaml
efficientnet-b3 DM-LRN RGBD LogDepthL1Loss [dm-lrn_b3.pth][dm-lrn_b3] DM_LRN_efficientnet-b3_jenifer.yaml
efficientnet-b4 DM-LRN RGBD LogDepthL1Loss [dm-lrn_b4.pth][dm-lrn_b4] DM_LRN_efficientnet-b4_pepper.yaml
efficientnet-b4 DM-LRN RGBD BerHu [dm-lrn_b4_berhu.pth][dm-lrn_b4_berhu] DM_LRN_efficientnet-b4_amelia.yaml

GitHub