uORF
Unsupervised Discovery of Object Radiance Fields
by Hong-Xing Yu, Leonidas J. Guibas and Jiajun Wu from Stanford University.
arXiv link: https://arxiv.org/abs/2107.07905
Project website: https://kovenyu.com/uorf
Environment
We recommend using Conda:
conda env create -f environment.yml
conda activate uorf-3090
or install the packages listed therein. Please make sure you have NVIDIA drivers supporting CUDA 11.0, or modify the version specifictions in environment.yml.
Data and model
Please download datasets and models here.
Evaluation
We assume you have a GPU.
If you have already downloaded and unzipped the datasets and models into the root directory,
simply run
bash scripts/eval_nvs_seg_chair.sh
from the root directory. Replace the script filename with eval_nvs_seg_clevr.sh, eval_nvs_seg_diverse.sh,
and eval_scene_manip.sh for different evaluations. Results will be saved into ./results/.
During evaluation, the results on-the-fly will also be sent to visdom in a nicer form, which can be accessed from
localhost:8077.
Training
We assume you have a GPU with no less than 24GB memory (evaluation does not require this as rendering can be done ray-wise but some losses are defined on the image space),
e.g., 3090. Then run
bash scripts/train_clevr_567.sh
or other training scripts. If you unzip datasets on some other place, add the location as the first parameter:
bash scripts/train_clevr_567.sh PATH_TO_DATASET
Training takes ~6 days on a 3090 for CLEVR-567 and Room-Chair, and ~9 days for Room-Diverse.
It can take even longer for less powerful GPUs (e.g., ~10 days on a titan RTX for CLEVR-567 and Room-Chair).
During training, visualization will be sent to localhost:8077.
Bibtex
@article{yu2021unsupervised
author = {Yu, Hong-Xing and Guibas, Leonidas J. and Wu, Jiajun},
title = {Unsupervised Discovery of Object Radiance Fields},
journal = {arXiv preprint arXiv:2107.07905},
year = {2021},
}
Acknowledgement
Our code framework is adapted from Jun-Yan Zhu's CycleGAN.
Some code related to adversarial loss is adapted from a pytorch implementation of StyleGAN2.
Some snippets are adapted from pytorch slot attention and NeRF.
If you find any problem please don't hesitate to email me at [email protected] or open an issue.
