ABME (ICCV2021)

Junheum Park,
Chul Lee,
and Chang-Su Kim

Official PyTorch Code for "Asymmetric Bilateral Motion Estimation for Video Frame Interpolation" [paper]

Requirements

PyTorch 1.7
CUDA 11.0
CuDNN 8.0.5
python 3.8

Installation

Create conda environment:

    $ conda create -n ABME python=3.8 anaconda
    $ conda activate ABME
    $ pip install opencv-python
    $ conda install pytorch==1.7 torchvision cudatoolkit=11.0 -c pytorch

Download repository:

    $ git clone https://github.com/JunHeum/ABME.git

Download pre-trained model parameters:

    $ unzip ABME_Weights.zip

Check your nvcc version:

    $ nvcc --version

To install correlation layer, you should match your nvcc version with cudatoolkit version of your conda environment. [nvcc_setting]

Install correlation layer:

    $ cd correlation_package
    $ python setup.py install

Quick Usage

Generate an intermediate frame on your pair of frames:

    $ python run.py --first images/im1.png --second images/im3.png --output images/im2.png

Test

Download the datasets.
Copy the path of the test dataset. (e.g., /hdd/vimeo_interp_test)
Parse this path into the --dataset_root argument.
(optional) You can ignore the --is_save. But, it yields a slightly different performance than evaluation on saved images.

    $ python test.py --name ABME --is_save --Dataset ucf101 --dataset_root /where/is/your/ucf101_dataset/path
    $ python test.py --name ABME --is_save --Dataset vimeo --dataset_root /where/is/your/vimeo_dataset/path
    $ python test.py --name ABME --is_save --Dataset SNU-FILM-all --dataset_root /where/is/your/FILM_dataset/path
    $ python test.py --name ABME --is_save --Dataset Xiph_HD --dataset_root /where/is/your/Xiph_dataset/path
    $ python test.py --name ABME --is_save --Dataset X4K1000FPS --dataset_root /where/is/your/X4K1000FPS_dataset/path

Experimental Results

We provide interpolated frames on test datasets for fast comparison or users with limited GPU memory. Especially, the test on X4K1000FPS requires at least 20GB of GPU memory.

Train

We plan to share train codes soon!

Citation

Please cite the following paper if you feel this repository useful.

    @inproceedings{park2021ABME,
        author    = {Park, Junheum and Lee, Chul and Kim, Chang-Su}, 
        title     = {Asymmetric Bilateral Motion Estimation for Video Frame Interpolation}, 
        booktitle = {International Conference on Computer Vision},
        year      = {2021}
    }