/ Machine Learning

A Robust Facial Landmark Detection Framework for Motion-Blurred Videos

A Robust Facial Landmark Detection Framework for Motion-Blurred Videos

FAB

We present a framework named FAB that takes advantage of structure consistency in the temporal dimension for facial landmark detection in motion-blurred videos. A structure predictor is proposed to predict the missing face structural information temporally, which serves as a geometry prior. This allows our framework to work as a virtuous circle. It is also a flexible video-based framework that can incorporate any static image-based methods to provide a performance boost on video datasets. Extensive experiments on Blurred-300VW, the proposed Real-world Motion Blur (RWMB) datasets and 300VW demonstrate the superior performance to the state-of-the-art methods.

Moreover, we proposed a new benchmark named Real-World Motion Blur (RWMB). It contains videos with obvious motion blur picked from YouTube, which include dancing, boxing, jumping, etc. A detailed description of the system can be found in our paper.

Citation

If you use this code or RWMB dataset for your research, please cite our paper.

@inproceedings{keqiang2019fab,
 author = {Sun, Keqiang and Wu, Wayne and Liu, Tinghao and Yang, Shuo and Wang, Quan and Zhou, Qiang and and Ye, Zuochang and Qian, Chen},
 title = {FAB: A Robust Facial Landmark Detection Framework for Motion-Blurred Videos},
 booktitle = {ICCV},
 month = October,
 year = {2019}
}

Prerequisites

Getting Started

Blurred-300VW Dataset Download

Blurred-300VW is a video facial landmark dataset with artifical motion blur, based on Original 300VW.

  1. Blurred-300VW [Google Drive] [Baidu Drive]
  2. Unzip the package and put them on './data/Blurred-300VW'

Wider Facial Landmark in the Wild (WFLW) Dataset Download

Real-World Motion Blur(RWMB) is a newly proposed facial landmark benchmark with read-world motion blur.

  1. RWMB Testing images [Google Drive] [Baidu Drive]
  2. Unzip the package and put them on './data/RWMB'

Training FAB on Blurred-300VW

bash ./scripts/train.sh

Testing FAB on Blurred-300VW

bash ./scripts/test.sh

To Do List

Supported dataset

Supported models

  • [ ] [Pretrained Model of Structure Predictor Block]
  • [ ] [Pretrained Model of Video Deblur Block]
  • [ ] [Pretrained Model of Resnet Block]
  • [ ] [Pretrained Model of Final model]

GitHub