Add styles from famous paintings to any photo in a fraction of a second!
It takes 100ms on a 2015 Titan X to style the MIT Stata Center (1024×680) like Udnie, by Francis Picabia.
Our implementation is based off of a combination of Gatys’ A Neural Algorithm of Artistic Style, Johnson’s Perceptual Losses for Real-Time Style Transfer and Super-Resolution, and Ulyanov’s Instance Normalization.
THe repository i based on https://github.com/lengstrom/fast-style-transfer.git.
We added styles from various paintings to a photo of Chicago. Click on thumbnails to see full applied style images.
Our implementation uses TensorFlow to train a fast style transfer network. We use roughly the same transformation network as described in Johnson, except that batch normalization is replaced with Ulyanov’s instance normalization, and the scaling/offset of the output
tanh layer is slightly different. We use a loss function close to the one described in Gatys, using VGG19 instead of VGG16 and typically using “shallower” layers than in Johnson’s implementation (e.g. we use
relu1_1 rather than
relu1_2). Empirically, this results in larger scale style features in transformations.
Virtual Environment Setup (Anaconda) – Windows/Linux
|Operating System||Windows 10 Home|
|GPU||Nvidia GTX 2080 TI|
Step 1：Install Anaconda
Step 2：Build a virtual environment
Run the following commands in sequence in Anaconda Prompt:
conda create -n tf-gpu tensorflow-gpu=2.1.0 conda activate tf-gpu
Run the following command in the notebook or just conda install the package:
!pip install moviepy==1.0.2
Follow the commands below to use fast-style-transfer
Training Style Transfer Networks
style.py to train a new style transfer network. Run
python style.py to view all the possible parameters. Training takes 4-6 hours on a Maxwell Titan X. More detailed documentation here. Before you run this, you should run
setup.sh. Example usage:
python main.py --style path/to/style/img.jpg \ --checkpoint-dir checkpoint/path \ --test path/to/test/img.jpg \ --test-dir path/to/test/dir \ --content-weight 1.5e1 \ --checkpoint-iterations 1000 \ --batch-size 20
Evaluating Style Transfer Networks
evaluate.py to evaluate a style transfer network. Run
python evaluate.py to view all the possible parameters. Evaluation takes 100 ms per frame (when batch size is 1) on a Maxwell Titan X. More detailed documentation here. Takes several seconds per frame on a CPU. Models for evaluation are located here. Example usage:
python eval.py --checkpoint path/to/style/model.ckpt \ --in-path dir/of/test/imgs/ \ --out-path dir/for/results/
You will need the following to run the above:
- TensorFlow 0.11.0
- Python 2.7.9, Pillow 3.4.2, scipy 0.18.1, numpy 1.11.2
- If you want to train (and don’t want to wait for 4 months):
- A decent GPU
- All the required NVIDIA software to run TF on a GPU (cuda, etc)
- ffmpeg 3.1.3 if you want to stylize video