We are happy that our paper titled “Harmonizing Image Forgery Detection & Localization: Fusion of Complementary Approaches” was accepted at the Journal of Imaging. This work is based on the master thesis of Louis De Neve “Towards Reliable (Deep)Fake Detection: A Fusion of Image Forgery Detection Methods”.
Image manipulation is easier than ever, often facilitated using accessible AI-based tools. This poses significant risks when used to disseminate disinformation, false evidence, or fraud, which highlights the need for image forgery detection and localization methods to combat this issue. While some recent detection methods demonstrate good performance, there is still a significant gap to be closed to consistently and accurately detect image manipulations in the wild.
This paper aims to enhance forgery detection and localization by combining existing detection methods that complement each other. First, we analyze these methods’ complementarity, with an objective measurement of complementariness, and calculation of a target performance value using a theoretical oracle fusion. Then, we propose a novel fusion method that combines the existing methods’ outputs**.
The proposed fusion method is trained using a Generative Adversarial Network architecture. It combines the outputs of the following forgery detection methods: ADQ1, BLK, CAGI, DCT, Comprint, Noiseprint, Comprint+Noiseprint, and CAT-Net.
Our experiments demonstrate improved detection and localization performance on a variety of datasets. Although our fusion method is hindered by a lack of generalization, this is a common problem in supervised learning, and hence a motivation for future work.
Examples of forged images and the corresponding heatmaps by the individual forgery localization algorithms and the proposed fusion method. The fusion method often has the most accurate detection.
The fusion method is integrated in the COM-PRESS dashboard, where anyone can upload images for manipulation analysis.
Example of fusion. By combining several individual forgery detection methods in the fusion method, we filter false positive detections and keep the most confident detection. Full example page: https://com-press.ilabt.imec.be/result/EnrabScJq.
In conclusion, this work deepens our understanding of forgery detection methods’ complementariness and how to harmonize them. As such, we contribute to better protection against image manipulations and the battle against disinformation.
Paper: Harmonizing Image Forgery Detection & Localization: Fusion of Complementary Approaches
The source code of FusionIDLab is available on GitHub.com.
The COM-PRESS consortium consists of the research groups IDLab-MEDIA and mict (both from UGent-imec), and Apache (De Werktitel cv) as media partner. Moreover, the consortium is working with CheckFirst. The project received subsidies from the Flemish government’s Department of Culture, Youth & Media (Departement Cultuur Jeugd & Media).
