logo

SCIENTIA SINICA Informationis, Volume 51 , Issue 9 : 1451(2021) https://doi.org/10.1360/SSI-2020-0064

Deep visual identity forgery and detection

More info
  • ReceivedMar 19, 2020
  • AcceptedAug 3, 2020
  • PublishedSep 10, 2021

Abstract


Funded by

2030 新一代人工智能重大专项(2018AAA0103202)

国家自然科学基金(61806152)

陕西省重点研发项目(2020ZDLGY08-08)


References

[1] Long K, Ma Y, Zhu Q C. How will deepfake technology influence national security: emerging challenges and policy implications. China Inform Secur, 2019, 10: 21--34. Google Scholar

[2] Cao J F. Deepfake technology: the legal challenge and response. China Inform Secur, 2019, 10: 8. Google Scholar

[3] Yang R, Huang J W, Luo W Q. 多媒体取证. Sci Sin-Inf, 2013, 43: 1654-1672 CrossRef Google Scholar

[4] LeCun Y, Bengio Y, Hinton G. Deep learning. Nature, 2015, 521: 436-444 CrossRef ADS Google Scholar

[5] Finlayson S G, Bowers J D, Ito J. Adversarial attacks on medical machine learning. Science, 2019, 363: 1287-1289 CrossRef ADS Google Scholar

[6] Leslie D. Raging robots, hapless humans: the AI dystopia. Nature, 2019, 574: 32-33 CrossRef ADS Google Scholar

[7] Korshunova I, Shi W, Dambre J, et al. Fast face-swap using convolutional neural networks. In: Proceedings of IEEE International Conference on Computer Vision, Venice, 2017. 3677--3685. Google Scholar

[8] Nirkin Y, Masi I, Tuan A T, et al. On face segmentation, face swapping, and face perception. In: Proceedings of IEEE International Conference on Automatic Face & Gesture Recognition, Xi'an, 2018. 98--105. Google Scholar

[9] Natsume R, Yatagawa T, Morishima S. RSGAN: face swapping and editing using face and hair representation in latent spaces. 2018,. arXiv Google Scholar

[10] Natsume R, Yatagawa T, Morishima S. Fsnet: an identity-aware generative model for image-based face swapping. In: Proceedings of Asian Conference on Computer Vision, Perth, 2018. 117--132. Google Scholar

[11] Bao J M, Chen D, Wen F, et al. Towards open-set identity preserving face synthesis. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, 2018. 6713--6722. Google Scholar

[12] Nirkin Y, Keller Y, Hassner T. Fsgan: subject agnostic face swapping and reenactment. In: Proceedings of IEEE International Conference on Computer Vision, Seoul, 2019. 7184--7193. Google Scholar

[13] Li L Z, Bao J M, Yang H, et al. FaceShifter: towards high fidelity and occlusion aware face wwapping. 2019,. arXiv Google Scholar

[14] Isola P, Zhu J Y, Zhou T, et al. Image-to-image translation with conditional adversarial networks. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Hawaii, 2017. 1125--1134. Google Scholar

[15] Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of IEEE International Conference on Computer Vision, Venice, 2017. 2223--2232. Google Scholar

[16] Shen W, Liu R J. Learning residual images for face attribute manipulation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Hawaii, 2017. 4030--4038. Google Scholar

[17] Lample G, Zeghidour N, Usunier N, et al. Fader networks: manipulating images by sliding attributes. In: Proceedings of Advances in Neural Information Processing Systems, Long Beach, 2017. 5967--5976. Google Scholar

[18] Choi Y, Choi M, Kim M, et al. Stargan: unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, 2018. 8789--8797. Google Scholar

[19] Choi Y, Uh Y, Yoo J, et al. StarGAN v2: diverse image synthesis for multiple domains. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Seattle, 2020. Google Scholar

[20] He Z, Zuo W, Kan M. AttGAN: Facial Attribute Editing by Only Changing What You Want. IEEE Trans Image Process, 2019, 28: 5464-5478 CrossRef ADS Google Scholar

[21] Liu M, Ding Y K, Xia M, et al. STGAN: a unified selective transfer network for arbitrary image attribute editing. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, 2019. 3673--3682. Google Scholar

[22] Guo J T, Qian Z Z, Zhou Z W, et al. MulGAN: facial attribute editing by exemplar. 2019,. arXiv Google Scholar

[23] Chen Y C, Lin H J, Shu M, et al. Facelet-bank for fast portrait manipulation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, 2018. 3541--3549. Google Scholar

[24] Chen Y C, Shen X H, Lin Z, et al. Semantic component decomposition for face attribute manipulation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, 2019. 9859--9867. Google Scholar

[25] Qian S J, Lin K Y, Wu W, et al. Make a face: towards arbitrary high fidelity face manipulation. In: Proceedings of IEEE International Conference on Computer Vision, Seoul, 2019. 10033--10042. Google Scholar

[26] Lee C H, Liu Z W, Wu L Y, et al. MaskGAN: towards diverse and interactive facial image manipulation. 2019,. arXiv Google Scholar

[27] Usman B, Dufour N, Saenko K, et al. PuppetGAN: cross-domain image manipulation by demonstration. In: Proceedings of IEEE International Conference on Computer Vision, Seoul, 2019. 9450--9458. Google Scholar

[28] Thies J, Zollh?fer M, Nie?ner M. Real-time expression transfer for facial reenactment. ACM Trans Graph, 2015, 34: 1-14 CrossRef Google Scholar

[29] Thies J, Zollhofer M, Stamminger M, et al. Face2face: real-time face capture and reenactment of rgb videos. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 2016. 2387--2395. Google Scholar

[30] Thies J, Zollh?fer M, Theobalt C. ACM Trans Graph, 2018, 37: 1-13 CrossRef Google Scholar

[31] Kim H, Garrido P, Tewari A. Deep video portraits. ACM Trans Graph, 2018, 37: 1-14 CrossRef Google Scholar

[32] Song L X, Lu Z H, He R, et al. Geometry guided adversarial facial expression synthesis. In: Proceedings of ACM International Conference on Multimedia, Seoul, 2018. 627--635. Google Scholar

[33] Wu W, Zhang Y X, Li C, et al. ReenactGAN: learning to reenact faces via boundary transfer. In: Proceedings of European Conference on Computer Vision, Munich, 2018. 603--619. Google Scholar

[34] Otberdout N, Daoudi M, Kacem A, et al. Dynamic facial expression generation on hilbert hypersphere with conditional wasserstein generative adversarial nets. 2019,. arXiv Google Scholar

[35] Huang Y, Khan S M. Generating photorealistic facial expressions in dyadic interactions. In: Proceedings of British Machine Vision Conference, Newcastle, 2018. Google Scholar

[36] Ding H, Sricharan K, Chellappa R. ExprGAN: facial expression editing with controllable expression intensity. In: Proceedings of AAAI Conference on Artificial Intelligence, New Orleans, 2018. Google Scholar

[37] Pumarola A, Agudo A, Martinez A M, et al. GANimation: anatomically-aware facial animation from a single image. In: Proceedings of European Conference on Computer Vision, Munich, 2018. 818--833. Google Scholar

[38] Tripathy S, Kannala J, Rahtu E. ICface: interpretable and controllable face reenactment using gans. In: Proceedings of IEEE Winter Conference on Applications of Computer Vision, Colorado, 2020. 3385--3394. Google Scholar

[39] Zakharov E, Shysheya A, Burkov E, et al. Few-shot adversarial learning of realistic neural talking head models. In: Proceedings of IEEE International Conference on Computer Vision, Seoul, 2019. 9459--9468. Google Scholar

[40] Ha S, Kersner M, Kim B, et al. MarioNETte: few-shot face reenactment preserving identity of unseen targets. 2019,. arXiv Google Scholar

[41] Suwajanakorn S, Seitz S M, Kemelmacher-Shlizerman I. Synthesizing Obama. ACM Trans Graph, 2017, 36: 1-13 CrossRef Google Scholar

[42] Oh T H, Dekel T, Kim C, et al. Speech2Face: learning the face behind a voice. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, 2019. 7539--7548. Google Scholar

[43] Wen Y, Raj B, Singh R. Face reconstruction from voice using generative adversarial networks. In: Proceedings of Advances in Neural Information Processing Systems, Vancouver, 2019. 5266--5275. Google Scholar

[44] Vougioukas K, Petridis S, Pantic M. Realistic Speech-Driven Facial Animation with GANs. Int J Comput Vis, 2020, 128: 1398-1413 CrossRef Google Scholar

[45] Jamaludin A, Chung J S, Zisserman A. You Said That?: Synthesising Talking Faces from Audio. Int J Comput Vis, 2019, 127: 1767-1779 CrossRef Google Scholar

[46] Song L, Wu W, Qian C, et al. Everybody's talkin': let me talk as you want. 2020,. arXiv Google Scholar

[47] Fried O, Tewari A, Zollh?fer M. Text-based editing of talking-head video. ACM Trans Graph, 2019, 38: 1-14 CrossRef Google Scholar

[48] Chen X, Qing L B, He X H, et al. FTGAN: a fully-trained generative adversarial networks for text to face generation. 2019,. arXiv Google Scholar

[49] Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets. In: Proceedings of Advances in Neural Information Processing Systems, Montreal, 2014. 2672--2680. Google Scholar

[50] Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks. 2015,. arXiv Google Scholar

[51] Miyato T, Kataoka T, Koyama M, et al. Spectral normalization for generative adversarial networks. 2018,. arXiv Google Scholar

[52] Zhang H, Goodfellow I, Metaxas D, et al. Self-attention generative adversarial networks. 2018,. arXiv Google Scholar

[53] Karras T, Aila T, Laine S, et al. Progressive growing of gans for improved quality, stability, and variation. 2017,. arXiv Google Scholar

[54] Karras T, Laine S, Aila T. A style-based generator architecture for generative adversarial networks. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, 2019. 4401--4410. Google Scholar

[55] Karras T, Laine S, Aittala M, et al. Analyzing and improving the image quality of stylegan. 2019,. arXiv Google Scholar

[56] Brock A, Donahue J, Simonyan K. Large scale gan training for high fidelity natural image synthesis. 2018,. arXiv Google Scholar

[57] Gui J, Sun Z J, Wen Y G, et al. A review on generative adversarial networks: algorithms, theory, and applications. 2020,. arXiv Google Scholar

[58] Cheng X Y, Xie L, Zhu J X, et al. Review of generative adversarial network. Comput Sci, 2019, 46: 74--81. Google Scholar

[59] Liang J J, Wei J J, Jiang Z F. Generating against network GAN overview. J Front Comput Sci Technol, 2019, 14: 1--17. Google Scholar

[60] Gao Y F, Hu Y J, Yu Z Q, et al. Evaluation and comparison of five popular fake face detection entworks. J Appl Sci, 2019, 37: 590--608. Google Scholar

[61] Nguyen T T, Nguyen C M, Nguyen D T, et al. Deep learning for deepfakes creation and detection. 2019,. arXiv Google Scholar

[62] Tolosana R, Vera-Rodriguez R, Fierrez J, et al. Deepfakes and beyond: a survey of face manipulation and fake detection. 2020,. arXiv Google Scholar

[63] Verdoliva L. Media forensics and deepfakes: an overview. 2020,. arXiv Google Scholar

[64] Zhang Y, Zheng L L, Thing V L L. Automated face swapping and its detection. In: Proceedings of IEEE International Conference on Signal and Image Processing, Singapore, 2017. 15--19. Google Scholar

[65] Li H D, Chen H, Li B, et al. Can forensic detectors identify gan generated images? In: Proceedings of Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, Honolulu, 2018. 722--727. Google Scholar

[66] Korshunov P, Marcel S. Deepfakes: a new threat to face recognition? assessment and detection. 2018,. arXiv Google Scholar

[67] Li H D, Li B, Tan S Q, et al. Detection of deep network generated images using disparities in color components. 2018,. arXiv Google Scholar

[68] He P S, Li H L, Wang H X. Detection of fake images via the ensemble of deep representations from multi color spaces. In: Proceedings of IEEE International Conference on Image Processing, Taipei, 2019. 2299--2303. Google Scholar

[69] Li Y Z, Lyu S W. Exposing deepfake videos by detecting face warping artifacts. 2018,. arXiv Google Scholar

[70] Matern F, Riess C, Stamminger M. Exploiting visual artifacts to expose deepfakes and face manipulations. In: Proceedings of IEEE Winter Applications of Computer Vision Workshops, Hawaii, 2019. 83--92. Google Scholar

[71] Durall R, Keuper M, Pfreundt F J, et al. Unmasking deepfakes with simple features. 2019,. arXiv Google Scholar

[72] Yu N, Davis L S, Fritz M. Attributing fake images to GANs: learning and analyzing GAN fingerprints. In: Proceedings of IEEE International Conference on Computer Vision, Seoul, 2019. 7556--7566. Google Scholar

[73] Nguyen H H, Fang F M, Yamagishi J, et al. Multi-task learning for detecting and segmenting manipulated facial images and videos. 2019,. arXiv Google Scholar

[74] Huang Y H, Juefei-Xu F, Wang R, et al. FakeLocator: robust localization of GAN-based face manipulations via semantic segmentation networks with bells and whistles. 2020,. arXiv Google Scholar

[75] Wang R, Ma L, Juefei-Xu F, et al. Fakespotter: a simple baseline for spotting ai-synthesized fake faces. 2019,. arXiv Google Scholar

[76] Nguyen H H, Yamagishi J, Echizen I. Capsule-forensics: using capsule networks to detect forged images and videos. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, Brighton, 2019. 2307--2311. Google Scholar

[77] Liu Z Z, Qi X J, Jia J Y, et al. Global texture enhancement for fake face detection in the Wild. 2020,. arXiv Google Scholar

[78] McCloskey S, Albright M. Detecting GAN-generated imagery using saturation cues, In: Proceedings of IEEE International Conference on Image Processing, Taipei, 2019. 4584--4588. Google Scholar

[79] Nataraj L, Mohammed T M, Manjunath B S, et al. Detecting GAN generated fake images using co-occurrence matrices. 2019,. arXiv Google Scholar

[80] Kumar P, Vatsa M, Singh R. Detecting Face2Face facial reenactment in videos. In: Proceedings of IEEE Winter Conference on Applications of Computer Vision, Colorado, 2020. 2589--2597. Google Scholar

[81] Zhang X, Karaman S, Chang S F. Detecting and simulating artifacts in gan fake images. 2019,. arXiv Google Scholar

[82] Stehouwer J, Dang H, Liu F, et al. On the detection of digital face manipulation. 2019,. arXiv Google Scholar

[83] Mo H X, Chen B L, Luo W Q. Fake faces identification via convolutional neural network. In: Proceedings of ACM Workshop on Information Hiding and Multimedia Security, Innsbruck, 2018. 43--47. Google Scholar

[84] Xuan X S, Peng B, Wang W, et al. On the generalization of GAN image forensics. In: Proceedings of Chinese Conference on Biometric Recognition, Zhuzhou, 2019. 134--141. Google Scholar

[85] Zhou P, Han X T, Morariu V I, et al. Two-stream neural networks for tampered face detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, 2017. 1831--1839. Google Scholar

[86] Dang L M, Hassan S I, Im S. Face image manipulation detection based on a convolutional neural network. Expert Syst Appl, 2019, 129: 156-168 CrossRef Google Scholar

[87] Marra F, Gragnaniello D, Verdoliva L, et al. Do GANs leave artificial fingerprints? In: Proceedings of IEEE Conference on Multimedia Information Processing and Retrieval, Shenzhen, 2019. 506--511. Google Scholar

[88] Tarasiou M, Zafeiriou S. Extracting deep local features to detect manipulated images of human faces. 2019,. arXiv Google Scholar

[89] Li J, Shen T, Zhang W, et al. Zooming into face forensics: a pixel-level analysis. 2019,. arXiv Google Scholar

[90] Jeon H, Bang Y, Woo S S. FDFtNet: facing off fake images using fake detection fine-tuning network. 2020,. arXiv Google Scholar

[91] Li Y Z, Chang M C, Lyu S W. In ICTU oculi: exposing AI created fake videos by detecting eye blinking. In: Proceedings of IEEE International Workshop on Information Forensics and Security, Hong Kong, 2018. Google Scholar

[92] Yang X, Li Y Z, Lyu S W. Exposing deep fakes using inconsistent head poses. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, Brighton, 2019. 8261--8265. Google Scholar

[93] Ciftci U A, Demir I. FakeCatcher: detection of synthetic portrait videos using biological signals. 2019,. arXiv Google Scholar

[94] Fernandes S, Raj S, Ortiz E, et al. Predicting heart rate variations of deepfake videos using neural ODE. In: Proceedings of IEEE International Conference on Computer Vision Workshops, Seoul, 2019. Google Scholar

[95] Güera D, Delp E J. Deepfake video detection using recurrent neural networks. In: Proceedings of IEEE International Conference on Advanced Video and Signal Based Surveillance, Auckland, 2018. Google Scholar

[96] Amerini I, Galteri L, Caldelli R, et al. Deepfake video detection through optical flow based CNN. In: Proceedings of IEEE International Conference on Computer Vision Workshops, Seoul, 2019. Google Scholar

[97] Afchar D, Nozick V, Yamagishi J, et al. Mesonet: a compact facial video forgery detection network. In: Proceedings of IEEE International Workshop on Information Forensics and Security, Hong Kong, 2018. Google Scholar

[98] Agarwal S, Farid H, Gu Y, et al. Protecting world leaders against deep fakes. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, 2019. 38--45. Google Scholar

[99] Baltrusaitis T, Zadeh A, Lim Y C, et al. Openface 2.0: facial behavior analysis toolkit. In: Proceedings of IEEE International Conference on Automatic Face & Gesture Recognition, Xi'an, 2018. 59--66. Google Scholar

[100] Sabir E, Cheng J X, Jaiswal A, et al. Recurrent convolutional strategies for face manipulation detection in videos. 2019,. arXiv Google Scholar

[101] Cozzolino D, Thies J, Rössler A, et al. Forensictransfer: weakly-supervised domain adaptation for forgery detection. 2018,. arXiv Google Scholar

[102] Du M N, Pentyala S, Li Y N, et al. Towards generalizable forgery detection with locality-aware autoencoder. 2019,. arXiv Google Scholar

[103] Marra F, Saltori C, Boato G, et al. Incremental learning for the detection and classification of GAN-generated images. 2019,. arXiv Google Scholar

[104] Wang S Y, Wang O, Zhang R, et al. CNN-generated images are surprisingly easy to spot... for now. 2019,. arXiv Google Scholar

[105] Li L Z, Bao J M, Zhang T, et al. Face x-ray for more general face forgery detection. 2019,. arXiv Google Scholar

[106] 严 , 余 , 张 . 可信计算的研究与发展. Sci Sin-Inf, 2010, 40: 139-166 CrossRef Google Scholar

[107] Zhang S S, Zuo X, Liu J W. The problem of the adversarial examples in deep learning. Chinese J Comput, 2019, 8: 15. Google Scholar

[108] Dong Y P, Su H, Wu B Y, et al. Efficient decision-based black-box adversarial attacks on face recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2019. 7714--7722. Google Scholar

[109] Zhang H C, Wang J Y. Towards adversarially robust object detection. In: Proceedings of IEEE International Conference on Computer Vision, 2019. 421--430. Google Scholar

[110] Huang R, Fang F M, Nguyen H H, et al. Security of facial forensics models against adversarial attacks. 2019,. arXiv Google Scholar

[111] Neves J C, Tolosana R, Vera-Rodriguez R, et al. Real or fake? Spoofing state-of-the-art face synthesis detection systems. 2019,. arXiv Google Scholar

[112] Ruiz N, Bargal S, Sclaroff S. Disrupting deepfakes: adversarial attacks against conditional image translation networks and facial manipulation systems. 2020,. arXiv Google Scholar

[113] Huang Y H, Juefei-Xu F, Wang R, et al. FakePolisher: making deepfakes more detection-evasive by shallow reconstruction. 2020,. arXiv Google Scholar

[114] Liu Z W, Luo P, Wang X G, et al. Deep learning face attributes in the wild. In: Proceedings of IEEE International Conference on Computer Vision, Santiago, 2015. 3730--3738. Google Scholar

[115] Khodabakhsh A, Ramachandra R, Raja K, et al. Fake face detection methods: Can they be generalized? In: Proceedings of International Conference of the Biometrics Special Interest Group, Darmstadt, 2018. Google Scholar

[116] Rossler A, Cozzolino D, Verdoliva L, et al. Faceforensics+: learning to detect manipulated facial images. In: Proceedings of IEEE International Conference on Computer Vision, Seoul, 2019. Google Scholar

[117] Li Y Z, Yang X, Sun P, et al. Celeb-df: a new dataset for deepfake forensics. 2019,. arXiv Google Scholar

[118] Dolhansky B, Howes R, Pflaum B, et al. The deepfake detection challenge (DFDC) preview dataset. 2019,. arXiv Google Scholar

[119] Jiang L M, Wu W, Li R, et al. DeeperForensics-1.0: a large-scale dataset for real-world face forgery detection. 2020,. arXiv Google Scholar

[120] Thies J, Zollh?fer M, Nie?ner M. Deferred neural rendering. ACM Trans Graph, 2019, 38: 1-12 CrossRef Google Scholar

  • Figure 1

    (Color online) Example types of deep face forgery

  • Figure 2

    (Color online) Examples of target-generic face forgery

  • Figure 3

    Classification of forgery detection methods. (a) Research on face forgery detection; (b) classification on face forgery detection

  • Figure 4

    (Color online) Experimental results on DeepfakeTIMIT dataset. (a) Two-streamNN [85], (b) EVA [70],protect łinebreak (c) Multi-task [73], (d) Capsule [76], (e) Xception [116], (f) FWA [69], (g) Mesonet [97], (h) HeadPose [92]

  • Table 1   Summary of face forgery detection methods based on spatial clues
    Category Motivation Method Detection clues Face forgery types
    Based on color Existing face forgery methods [67] Translating RGB images into HSV, YCbCr color spaces Target-generic fake face generation
    artifacts usually ignore constrains on color spaces [68] Translating RGB images into HSV, YCbCr color spaces, together with high-pass filtering Target-generic fake face generation
    Based on visual Existing forgery results contain [69] Using CNN to detect face swap contours Face swap
    disparities visual disparities in details [70] Detecting visual disparities in details, such as eye colors, varying lighting, and teeth details Face swap, reenactment, fake face generation
    Based on image processing Using traditional image processing techniques [71] Conducting discrete Fourier transform, and apply classification models on frequency domain Face swap, fake face generation
    Based on image classification Considering forgery detection as image [72] Assuming fake images generated from the same GAN model as one class Target-generic fake face generation
    classification task [73] Using AdaBoost and XGBoost to deal with unbalanced data between real and fake images Face swap
    Based on multi-task Performing forgery detection [73] Using multi-task learning to detect and segment forgery faces Face swap, reenactment
    learning and location together [74] Introducing semantic segmentation into forgery detection task Fake facegeneration
    Based on specific Replying on specific deep [75] Monitoring neuron behavior for face forgery detection Fake facegeneration, reenactment,
    deep networks networks for forgery detection [76] Using capsule networks for forgery feature extraction Face swap, reenactment
    [77] Introducing Gram-Net architecture for face forgery detection Fake face generation
  • Table 2   Summary of face forgery detection methods based on temporal clues
    Category Motivation Method Detection clues Forgery types
    Eye blinking Detecting the frequency of eye blinking [91] Using CNN and RNN to detect eye blinking Face swap
    Head movement Detecting head poses [92] Locating disparities between head pose and facial pose Face swap
    Biological Extracting biological [93] Using PPG signals with SVM Face swap
    signals signals for detection [94] Detecting heart rates for forgery classification Face swap
    Frame-to-frame Detecting disparities [95] Using RNN to extract spatial features Face swap
    disparities between frames [96] Using optical flow for forgery detection Face swap
  • Table 3   Summary of face forgery detection datasets
    Dataset Date Types Real data (number) Fake data (number)
    CelebA [114] 2015 Real face images Image (202K)
    CelebA-HQ [53] 2018 Real face images Image (30K)
    FFHQ [54] 2018 Real face images Image (70K)
    100K-Faces-StyleGAN [54] 2018 Target-generic fake faces Image (100K)
    100K-Faces-StyleGAN2 [55] 2019 Target-generic fake faces Image (100K)
    UADFV [92] 2018 Target-specific fake faces Video (49) Video (49)
    FFW [115] 2018 Target-specific fake faces Video (150)
    DeepfakeTIMIT [66] 2018 Target-specific fake faces Video (320) Video (640)
    FaceForensics+ [116] 2019 Target-specific fake faces Video (1000) Video (4000)
    Celeb-DF [117] 2019 Target-specific fake faces Video (408) Video (795)
    Celeb-DF(v2) [117] 2019 Target-specific fake faces Video (590) Video (5639)
    DFDC [118] 2019 Target-specific fake faces Video (1131) Video (4113)
    DeeperForensics-1.0 [119] 2020 Target-specific fake faces Video (10000) Video (10000)
  • Table 4   Experimental results on FaceForensics+ dataset (%)
    Category Method Result (FaceSwap) Result (Face2Face) Result (DFD)
    Two-streamNN [85] AUC (70.1) Acc (NC: 99.9, HQ: 96, LQ: 86.8) AUC (52.8)
    EVA [70] AUC (78) AUC (86.6) AUC (77.2)
    Multi-task [73] AUC (76.3), EER (HQ: 7.1) AUC (54.1), Acc (92.8),
    EER (HQ: 15.1)EER (8.2)
    Capsule [76] AUC (96.6) Acc (NC: 99.3, HQ: 96.5, LQ: 81) AUC (64)
    Spatial Xception [116] AUC (99.7) AUC (53.9)
    clue FWA [69] AUC (80.1) AUC (74.3)
    Multi-Resnet [80] Acc (NC: 99.9, HQ: 99.1, LQ: 91.2)
    Attention [82] AUC (99.4), EER (3.4)
    DFT [71] Acc (91)
    Mosenet [97] AUC (84.7) Acc (NC: 96.8, HQ: 93.4, LQ: 83.2) AUC (76)
    Temporal Headpose [92] AUC (47.3) AUC (56.1)
    clue RCNN [100] AUC (LQ: 96.3) Acc (LQ: 94.3)
    Opticalflow [96] Acc (81.6)
  • Table 5   Experimental results on UADFV and Celeb-DF datasets (%)
    Category Method Result (UADFV) Result (Celeb-DF)
    Two-streamNN [85] AUC (85.1) AUC (55.7)
    EVA [70] AUC (70.2) AUC (48.8)
    Spatial clue Multi-task [73] AUC (65.8) AUC (36.5)
    Capsule [76] AUC (61.3)
    Xception [116] AUC (80.4) AUC (38.7)
    FWA [69] AUC (97.4) AUC (53.8)
    Temporal clue Mesonet [97] AUC (84.3) AUC (53.6)
    Headpose [92] AUC (89) AUC (54.8)
    Generalizability Face X-ray [105] AUC (80.6)
qqqq

Contact and support