Vision Transformer Based Video Hashing Retrieval for Tracing the Source of Fake Videos?️

✨ Introduction


Video Tracing and Tampering Localization (VTL). A novel framework to detect fake video (clipping, cropping, blur, etc.) by tracing the source video of fake video. 1) Training hash centers as HCs. 2) Finding index of source video from HCs. 3) Masking the different between fake video and source video as a result of comparison (auxiliary information).

Trace Samples and Acc of HashBits

Although the source videos are very similar, we can accurately find the source videos of the fake video clips.

DFTL Dataset Samples

Same person with different scenes. You can download full 16 minutes videos of source video and fake video by follows link.

Different fake videos from same source.

Source Video

Fake Videos of Different Face Swap Methods

DAVIS2016-TL Dataset Samples

The first gif of boat is source video, and remaining five videos generated by different inpainting methods.

? Train or Test

Datasets Download

BaiduNetdisk code:VTLs

  • actors: Source videos and fake videos of full 16 minutes. You can use these videos to make richer datasets.
  • DFTL: Dataset of DFTL, the DFTL build from actors.
  • DAVIS2016-TL: Extension of DAVIS2016

Extract to the same directory as the code (vtl). Example:

├─other files
│  ├─vrf: dataset of DFTL
│  ├─inpainting: dataset of DAVIS2016-TL
│  └─vtl: our code
│      ├─CSQ: Central Similarity Quantization for Efficient Image and Video Retrieval
│      ├─dmac: Compared method of Localization
│      └─codes


Pretrained models and hash centers

pip install -r requirements.txt

ViTHash 64-1024bits 64-1024bits
Generator link link


  • local_rank: gpu id
  • path: dataset path
  • type: choice dataloader
    • 0: DFTL dataloader, dir name is vrf
    • 1: DAVIS2016-TL dataloader, dir name is inpainting

Train ViTHash

python --local_rank=0 --path=../vrf --type=0 --bits=128

Train Generator

python --local_rank=0 --path=../vrf --type=0


Test IOU

The test script will test Generator of VTL and DMAC together on DFTL and DAVIS2016-TL.
You can modify it for yourself.


Test ViTHash

  1. type: choice dataloader
    • 0: DFTL dataloader, dir name is vrf
    • 1: DAVIS2016-TL dataloader, dir name is inpainting
  2. path: dataset path
  3. hashbits: 128 256 512 or 1024, will load different pre-trained model and hash JSON file.

python 1 ../inpainting 512

Test CSQ

  1. cd ./CSQ
  2. run test script

python --dataset=Inpainting --pretrained_3d=./Inpainting_64bits.pth

?️ Tracing

Trace Samples

?️ Localization

Localization Samples




View Github