MarioNette | Webpage | Paper | Video
MarioNette: Self-Supervised Sprite Learning
Dmitriy Smirnov, Michaël Gharbi, Matthew Fisher, Vitor Guizilini, Alexei A. Efros, Justin Solomon
NeurIPS 2021
Set-up
To install the neecssary dependencies, run:
conda env create -f environment.yml
conda activate MarioNette
Also, be sure to execute export PYTHONPATH=:$PYTHONPATH prior to running any of the scripts.
Training
To train a MarioNette model, run:
python scripts/train.py --checkpoint_dir out_dir --data data_dir
Your dataset should be stored in data_dir, with each input frame named #.png. If the images are not 128×128 pixels, specify the resolution using the --canvas_size flag. Optionally, pass a --layer_size flag to specify the anchor grid resolution, --num_layers to specify the number of layers, or --num_classes to specify the size of the spirte dictionary.
To monitor the training, launch a TensorBoard instance with --logdir out_dir.
BibTeX
@article{smirnov2021marionette,
title={{MarioNette}: Self-Supervised Sprite Learning},
author={Smirnov, Dmitriy and Gharbi, Michael and Fisher, Matthew and Guizilini, Vitor and Efros, Alexei A. and Solomon, Justin},
year={2021},
journal={Conference on Neural Information Processing Systems}
}
