/ Machine Learning

Guided Image-to-Image Translation with Bi-Directional Feature Transformation

Guided Image-to-Image Translation with Bi-Directional Feature Transformation

Guided-pix2pix

Official Pytorch implementation for Guided Image-to-Image Translation with Bi-Directional Feature Transformation. Please contact Badour AlBahar ([email protected]) if you have any questions.

Prerequisites

This codebase was developed and tested with:

  • Python2.7
  • Pytorch 0.4.1.post2
  • CUDA 8.0

Installation

Datasets

Train

1. Pose transfer:

python train.py --dataroot /root/DeepFashion/ --name exp_name --netG bFT_resnet --dataset_mode pose --input_nc 3 --guide_nc 18 --output_nc 3 --lr 0.0002 --niter 100 --niter_decay 0 --batch_size 8 --use_GAN --netD basic --beta1 0.9 --checkpoints_dir ./pose_checkpoints

2. Texture transfer:

python train.py --dataroot /root/training_handbags_pretrain/ --name exp_name --netG bFT_unet --dataset_mode texture --input_nc 1 --guide_nc 4 --output_nc 3 --niter 100 --niter_decay 0 --batch_size 256 --lr 0.0002 --use_GAN --netD basic --n_layers 7 --beta1 .9 --checkpoints_dir ./texture_checkpoints

3. Depth Upsampling:

python train.py --dataroot /root/NYU_RGBD_matfiles/ --name exp_name --netG bFT_resnet --dataset_mode depth --input_nc 1     --guide_nc 3 --output_nc 1 --lr 0.0002 --niter 500 --niter_decay 0 --batch_size 2 --checkpoints_dir ./depth_checkpoints --depthTask_scale [4, 8, or 16]

Test

You can specify which epoch to test by specifying --epoch or use the default which is the latest epoch. Results will be saved in --results_dir.

1. Pose transfer:

python test.py --dataroot /root/DeepFashion/ --name exp_name --netG bFT_resnet --dataset_mode pose --input_nc 3 --guide_nc 18 --output_nc 3 --checkpoints_dir ./pose_checkpoints --task pose --results_dir ./pose_results

2. Texture transfer:

python test.py --dataroot /root/training_handbags_pretrain/ --name exp_name --netG bFT_unet --n_layers 7 --dataset_mode texture --input_nc 1 --guide_nc 4 --output_nc 3 --checkpoints_dir ./texture_checkpoints --task texture --results_dir ./texture_results

3. Depth Upsampling:

python test.py --dataroot /root/NYU_RGBD_matfiles/ --name exp_name --netG bFT_resnet --dataset_mode depth --input_nc 1 --guide_nc 3 --output_nc 1 --checkpoints_dir ./depth_checkpoints --task depth  --depthTask_scale [4, 8, or 16] --results_dir ./depth_results

Pretrained checkpoints

  • Download the pretrained checkpoints here.

  • Test: For example, to test the depth upsampling task with scale 16:

python test.py --dataroot /root/NYU_RGBD_matfiles/ --name depth_16 --netG bFT_resnet --dataset_mode depth --input_nc 1 --guide_nc 3 --output_nc 1 --checkpoints_dir ./checkpoints/pretrained/ --task depth  --depthTask_scale 16 --results_dir ./depth_results

Evaluate

You can specify which epoch to evaluate by specifying --epoch or use the default which is the latest epoch. Results will be saved in --results_dir.

1. Pose transfer:

python evaluate.py --dataroot /root/DeepFashion/ --name pose --netG bFT_resnet --dataset_mode pose --input_nc 3 --guide_nc 18 --output_nc 3 --checkpoints_dir ./checkpoints/pretrained/ --task pose --results_dir ./pose_results

This will save the results in --results_dir and compute both SSIM and IS metrics.

2. Texture transfer:
Please download the pretrained model of textureGAN in ./resources from bags, shoes, and clothes. For example, to test the pretrained texture transfer model for the bags dataset:

python evaluate.py --dataroot /root/training_handbags_pretrain/ --name texture_bags --netG bFT_unet --n_layers 7 --dataset_mode texture --input_nc 1 --guide_nc 4 --output_nc 3 --checkpoints_dir ./checkpoints/pretrained/ --task texture --results_dir ./texture_results

This will save the output of bFT and textureGAN in --results_dir for 10 random input texture patches per test image. The results can then be used to compute FID and LPIPS.

3. Depth Upsampling:

python evaluate.py --dataroot /root/NYU_RGBD_matfiles/ --name depth_16 --netG bFT_resnet --dataset_mode depth --input_nc 1 --guide_nc 3 --output_nc 1 --checkpoints_dir ./checkpoints/pretrained/ --task depth  --depthTask_scale 16 --results_dir ./depth_results

This will save the results in --results_dir and compute their RMSE metric.

GitHub