ManTra-Net: Manipulation Tracing Network For Detection And Localization of Image Forgeries With Anomalous Features.
ManTraNet is an end-to-end image forgery detection and localization solution, which means it takes a testing image as input, and predicts pixel-level forgery likelihood map as output. Comparing to existing methods, the proposed ManTraNet has the following advantages:
- Simplicity: ManTraNet needs no extra pre- and/or post-processing
- Fast: ManTraNet puts all computations in a single network, and accepts an image of arbitrary size.
- Robustness: ManTraNet does not rely on working assumptions other than the local manipulation assumption, i.e. some region in a testing image is modified differently from the rest.
Technically speaking, ManTraNet is composed of two sub-networks as shown below:
- Image Manipulation Trace Feature Extractor: the feature extraction network for the image manipulation classification task, which is sensitive to different manipulation types, and encodes the image manipulation in a patch into a fixed dimension feature vector.
- Local Anomaly Detection Network: the anomaly detection network to compare a local feature against the dominant feature averaged from a local region, whose activation depends on how far a local feature deviates from the reference feature instead of the absolute value of a local feature.
ManTraNet is pretrained with all synthetic data. To prevent overfitting, we
- Pretrain the Image Manipulation Classification (385 classes) task to obtain the Image Manipulation Trace Feature Extractor
- Train ManTraNet with four types of synthetic data, i.e. copy-move, splicing, removal, and enhancement
To extend the provided ManTraNet, one may introduce the new manipulation either to the IMC pretrain task, or to the end-to-end ManTraNet task, or both. It is also worth noting that the IMC task can be a self-supervised task.
ManTraNet is written in Keras with the TensorFlow backend.
- Keras: 2.2.0
- TensorFlow: 1.8.0
Other versions might also work, but are not tested.
One may simply download the repo and play with the provided ipython notebook.
Alternatively, one may play with the inference code using this google colab link.