Volume Rendering of Neural Implicit Surfaces
Project Page | Paper | Data
This repository contains an implementation for the NeurIPS 2021 paper:
Volume Rendering of Neural Implicit Surfaces
Lior Yariv1, Jiatao Gu2, Yoni Kasten1, Yaron Lipman1,2
1Weizmann Institute of Science, 2Facebook AI Research
The paper introduce VolSDF: a volume rendering framework for implicit neural surfaces, allowing to learn high fidelity geometry from a sparse set of input images.
Setup
Installation Requirmenets
The code is compatible with python 3.8 and pytorch 1.7. In addition, the following packages are required:
numpy, pyhocon, plotly, scikit-image, trimesh, imageio, opencv, torchvision.
You can create an anaconda environment called volsdf with the required dependencies by running:
conda env create -f environment.yml
conda activate volsdf
Data
We apply our multiview surface reconstruction model to real 2D images from two datasets: DTU and BlendedMVS.
The selected scans data evaluated in the paper can be downloaded using:
bash data/download_data.sh
For more information on the data convention and how to run VolSDF on a new data please have a look at data convention.
Usage
Multiview 3D reconstruction
For training VolSDF run:
cd ./code
python training/exp_runner.py --conf ./confs/dtu.conf --scan_id SCAN_ID
where SCAN_ID is the id of the scene to reconstruct.
To run on the BlendedMVS dataset, which have more complex background, use --conf ./confs/bmvs.conf.
Evaluation
To produce the meshed surface and renderings, run:
cd ./code
python evaluation/eval.py --conf ./confs/dtu.conf --scan_id SCAN_ID --checkpoint CHECKPOINT [--eval_rendering]
where CHECKPOINT is the epoch you wish to evaluate or ‘latest’ if you wish to take the most recent epoch.
Turning on --eval_rendering will further produce and evaluate PSNR of train image reconstructions.
Citation
If you find our work useful in your research, please consider citing:
@inproceedings{yariv2021volume,
title={Volume rendering of neural implicit surfaces},
author={Yariv, Lior and Gu, Jiatao and Kasten, Yoni and Lipman, Yaron},
booktitle={Thirty-Fifth Conference on Neural Information Processing Systems},
year={2021}
}
