C2F-FWN

data/code repository of "C2F-FWN: Coarse-to-Fine Flow Warping Network for Spatial-Temporal Consistent Motion Transfer"
(https://arxiv.org/abs/2012.08976)

Example Results

Prerequisites

  • Ubuntu
  • Python 3
  • NVIDIA GPU (>12GB memory) + CUDA10 cuDNN7
  • PyTorch 1.0.0

Other Dependencies

DConv (modified from original [DConv])

cd models/dconv
bash make.sh

FlowNet_v2 (directly ported from the original [flownet2] following the steps described in [vid2vid])

cd models/flownet2-pytorch
bash install.sh

Getting Started

It's a preview version of our source code. We will clean it up in the near future.

Notes

  1. Main functions for training and testing can be found in "train_stage1.py", "train_stage2.py", "train_stage2.py", "test_all_stages.py";
  2. Data preprocessings of all the stages can be found in "data" folder;
  3. Model definitions of all the stages can be found in "models" folder;
  4. Training and testing options can be found in "options" folder;
  5. Training and testing scripts can be found in "scripts" folder;
  6. Tool functions can be found in "util" folder.

Data Preparation

Download all the data packages from [google drive] or [baidu pan (extraction code:gle4], and uncompress them.
You should create a directory named 'SoloDance' in the root (i.e., 'C2F-FWN') of this project, and then put 'train' and 'test' folders to 'SoloDance' you just created.
The structure should look like this:
-C2F-FWN
---SoloDance
------train
------test

Training

1.Train the layout GAN of stage 1:

bash scripts/stage1/train_1.sh

2.Train our C2F-FWN of stage 2:

bash scripts/stage2/train_2_tps_only.sh
bash scripts/stage2/train_2.sh

3.Train the composition GAN of stage 3:

bash scripts/stage3/train_3.sh

Testing all the stages together (separate testing scripts for different stages will be updated in the near future)

bash scripts/full/test_full.sh

Acknowledgement

A large part of the code is borrowed from NVIDIA/vid2vid. Thanks for their wonderful works.

Citation

If you find this project useful for your research, please cite our paper using the following BibTeX entry.

@article{wei2020c2f,
  title={C2F-FWN: Coarse-to-Fine Flow Warping Network for Spatial-Temporal Consistent Motion Transfer},
  author={Wei, Dongxu and Xu, Xiaowei and Shen, Haibin and Huang, Kejie},
  journal={arXiv preprint arXiv:2012.08976},
  year={2020}
}

GitHub

https://github.com/wswdx/C2F-FWN