MarioNette | Webpage | Paper | Video


MarioNette: Self-Supervised Sprite Learning
Dmitriy Smirnov, Michaƫl Gharbi, Matthew Fisher, Vitor Guizilini, Alexei A. Efros, Justin Solomon
NeurIPS 2021


To install the neecssary dependencies, run:

conda env create -f environment.yml
conda activate MarioNette

Also, be sure to execute export PYTHONPATH=:$PYTHONPATH prior to running any of the scripts.


To train a MarioNette model, run:

python scripts/ --checkpoint_dir out_dir --data data_dir

Your dataset should be stored in data_dir, with each input frame named #.png. If the images are not 128×128 pixels, specify the resolution using the --canvas_size flag. Optionally, pass a --layer_size flag to specify the anchor grid resolution, --num_layers to specify the number of layers, or --num_classes to specify the size of the spirte dictionary.

To monitor the training, launch a TensorBoard instance with --logdir out_dir.


  title={{MarioNette}: Self-Supervised Sprite Learning},
  author={Smirnov, Dmitriy and Gharbi, Michael and Fisher, Matthew and Guizilini, Vitor and Efros, Alexei A. and Solomon, Justin},
  journal={Conference on Neural Information Processing Systems}


View Github