/ Machine Learning

Pytorch implementation of GANILLA

Pytorch implementation of GANILLA

GANILLA

We provide PyTorch implementation for:

GANILLA: Generative Adversarial Networks for Image to Illustration Translation.

Paper
Arxiv

Dataset Stats:

dataset_stats

Sample Images:

ill_dataset

GANILLA:

GANILLA results on the illustration dataset:

ganilla_res

Comparison with other methods:

sota_comp

Style transfer using Miyazaki's anime images:

miyazaki_res

Ablation Experiments:

ablation_experiments

Prerequisites

  • Linux or macOS
  • Python 2 or 3
  • CPU or NVIDIA GPU + CUDA CuDNN

Getting Started

Installation

  • Clone this repo:
git clone https://github.com/giddyyupp/ganilla.git
cd ganilla
pip install -r requirements.txt
  • For Conda users, we include a script ./scripts/conda_deps.sh to install PyTorch and other libraries.

GANILLA train/test

  • Download a GANILLA/CycleGAN dataset (e.g. maps):
bash ./datasets/download_cyclegan_dataset.sh maps
  • Train a model:
#!./scripts/train_ganilla.sh
python train.py --dataroot ./datasets/maps --name maps_cyclegan --model cycle_gan --netG resnet_fpn
  • To view training results and loss plots, run python -m visdom.server and click the URL http://localhost:8097. To see more intermediate results, check out ./checkpoints/maps_cyclegan/web/index.html
  • Test the model:
#!./scripts/test_cyclegan.sh
python test.py --dataroot ./datasets/maps --name maps_cyclegan --model cycle_gan --netG resnet_fpn

The test results will be saved to a html file here: ./results/maps_cyclegan/latest_test/index.html.

You can find more scripts at scripts directory.

Apply a pre-trained model (GANILLA)

  • You can download pretrained models using following link

Put a pretrained model under ./checkpoints/{name}_pretrained/100_net_G.pth.

  • To test the model, you also need to download the monet2photo dataset and use trainB images as source:
bash ./datasets/download_cyclegan_dataset.sh monet2photo
  • Then generate the results using
python test.py --dataroot datasets/monet2photo/testB --name {name}_pretrained --model test

The option --model test is used for generating results of GANILLA only for one side. python test.py --model cycle_gan will require loading and generating results in both directions, which is sometimes unnecessary. The results will be saved at ./results/. Use --results_dir {directory_path_to_save_result} to specify the results directory.

  • If you would like to apply a pre-trained model to a collection of input images (rather than image pairs), please use --dataset_mode single and --model test options. Here is a script to apply a model to Facade label maps (stored in the directory facades/testB).
#!./scripts/test_single.sh
python test.py --dataroot ./datasets/monet2photo/testB/ --name {your_trained_model_name} --model test

You might want to specify --netG to match the generator architecture of the trained model.

Training/Test Tips

Best practice for training and testing your models.

Frequently Asked Questions

Before you post a new question, please first look at the above Q & A and existing GitHub issues.

Citation

If you use this code for your research, please cite our papers.

@article{hicsonmez2020ganilla,
  title={GANILLA: Generative adversarial networks for image to illustration translation},
  author={Hicsonmez, Samet and Samet, Nermin and Akbas, Emre and Duygulu, Pinar},
  journal={Image and Vision Computing},
  pages={103886},
  year={2020},
  publisher={Elsevier}
}

@inproceedings{Hicsonmez:2017:DDN:3078971.3078982,
 author = {Hicsonmez, Samet and Samet, Nermin and Sener, Fadime and Duygulu, Pinar},
 title = {DRAW: Deep Networks for Recognizing Styles of Artists Who Illustrate Children's Books},
 booktitle = {Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval},
 year = {2017}
}

Acknowledgments

Our code is heavily inspired by CycleGAN.

The numerical calculations reported in this work were fully performed at TUBITAK ULAKBIM, High Performance and Grid Computing Center (TRUBA resources).

GitHub

Comments