. .

Project Sponsors

Bundesamt für Sicherheit in der Informationstechnik (BSI)
Norwegian Research Council (RCN)
European Commission

Project Partners

Hochschule Darmstadt (CRISP)
NTNU (NBL)
secunet Security Networks AG (SEC)
Salzburg University (Wavelab)
National Institute of Standards and Technology (NIST)
Universita di Bologna (UBO)
Universiteit Twente (UTW)
Fraunhofer (IGD)

Contact

Christoph Busch
Email: christoph.busch@h-da.de

Juan Tapia
Mathias Ibsen
Sushma Venkatesh
Haoyu Zhang
Christian Rathgeb
Raghu Ramachandra
Kiran Raja
Siri Lorenz
Robert Nichols
Daniel Fischer

Morphing Attack Detection (MAD)

Morphing Attacks and Impact

Biometrics refers to the automated recognition of individuals based on their biological and behavioral characteristics. Due to the strong link between subjects and their biometric samples, the wide acceptance, and their user convenience, biometric systems become increasingly popular and important for border control.
Even though the security of biometric systems is increasing, specifically face recognition systems are vulnerable with regard to the unique link between the biometric facial sample and its subject. By enrolling an artificial sample, generated by merging samples of two or multiple subjects in image domain, the contributing subjects might be verified successfully against the manipulated reference. This morphing attack can be done, for instance, in the passport application process, where in most European countries the applicant brings his own printed photograph. This way, the unique link between individuals and their biometric reference data is annulled. The impact of this vulnerability has been illustrated in this video
To prevent the aforementioned face morphing attacks, an automatic detection of morphs is required at border control as well as early in the application process. This must be considered as the most challenging research task to date and long-term research will be need to fully address the challenge. However on this page reports recent progress in this research area.

Application Scenarios

Focusing on the workflow of a generic biometric system, two Morphing Attack Detection (MAD) tasks can be distinguished: (1) detection during enrolment, e.g. the passport application process, where the MAD mechanism processes a one suspected facial image, referred to as single image morphing attack detection (S-MAD) and depicted in the figure on the top; and (2) detection at the time of identity validation, e.g. passing through an Automated Border Control (ABC) gates at borders, where a trusted live capture from the gate serves as additional source of information for the MAD mechanism, referred to as differential morphing attack detection (D-MAD) and depicted in figure at the bottom.

picture 1

History of Morphing Attacks

The problem of morphing attacks has been addressed in the biometric research community only recently, despite that the issue was discussed back in 2004 in the presentation
Matthew Lewis, Philip Statham: "CESG Biometric Security Capabilities Programme: Method, Results and Research challenges", in Proceedings Biometrics Consortium Conference (BCC), (2004)
and was also classified as vulnerability of a biometric system as early as 2009 in Clause 8.3.8.1 of ISO/IEC FDIS 19792 stating: "… Examples of abnormal characteristics could include those with unusually large or small numbers of features. Such characteristics may not be representative of any human biometric characteristic but could be synthesised and copied to an artefact. Alternatively a synthesised characteristic could be injected electrically during a replay attack or planted in the reference database. - feature sets comprising amalgamations of biometric features from 2 or more individuals, e.g. morphed facial images”
The feasibility of such morphing attacks was first shown for face recognition systems in 2014 in the FIDELITY project by M. Ferrara, A. Franco and D. Maltoni in her IJCB paper "The magic passport". In order to measure the vulnerability of biometric recognition systems with respect to morphing attacks, the methodology of morphing attack potential ( MAP) was developed. This methodology is now being standardised in the international standard ISO/IEC 20059 Methodologies to evaluate the resistance of biometric recognition systems to morphing attacks.

Detection of Morphing Attacks

With our research we address the problem of morphing attacks. The work has started only recently. In 2016 the first Morphing Attack Detection (MAD) mechanism was proposed by R. Raghavendra, K. Raja and C. Busch in the BTAS paper "Detecting Morphed Facial Images". Since then we have started numerous efforts to address this research challenge. The MAD state of the art is investigated in the Bologna Online Evaluation Platform (BOEP) and also in the NIST FATE MORPH platform.

Avoiding Morphing Attacks

In order to avoid morphing attacks in the first place, passport applications should be done with live enrolment, meaning the facial photo is captured under supervision in the application process. The European Regulation 2019/1157 has recommended this in recital 32 of the regulation, which reads: "... To this end, Member States could consider collecting biometric identifiers, particularly the facial image, by means of live enrolment by the national authorities issuing identity cards." This is not a strong enforcement, of what is needed. Unfortunately only a few European countries (e.g. Sweden and Norway) have live enrolment in place. Others (e.g. Germany) are starting now (2020) in recognition of the morphing attack problem. But live-enrolment will be fully operational not before 2025. The discussion in the Bundestag was interesting. You can find my position paper on the draft law here.

MAD Publications by our Group

  1. R. Kessler, K. Raja, J. Tapia, C. Busch: "Towards minimizing efforts for Morphing Attacks - Deep embeddings for morphing pair selection and improved Morphing Attack Detection", in PLOS ONE, (2024)
  2. M. Ibsen, R. Nichols, C. Rathgeb, D. Robertson, J. Davis, F. Løvåsdal, K. Raja, R. Jenkins, C. Busch: "Conditional Face Image Manipulation Detection: Combining Algorithm and Human Examiner Decisions", in Proceedings of 12th Workshop on Information Hiding and Multimedia Security (ACM IH&MMSEC 2024), Balona, ES, June 24-25, (2024)
  3. H. Zhang, R. Raghavendra, K. Raja, C. Busch: "Generalized Single-Image-Based Morphing Attack Detection Using Deep Representations from Vision Transformer", in Proceedings of IEEE Computer Society 19th Workshop on Biometrics (CVPRW), Seattle, U.S., June 17, (2024)
  4. M. Ibsen, J. Soler, C. Rathgeb, C. Busch: "TetraLoss: Improving the Robustness of Face Recognition against Morphing Attacks", in Proceedings of the 18th IEEE International Conference on Automatic Face and Gesture Recognition (FG), Istanbul, TK, May 27-31, (2024)
  5. J. Tapia, C. Busch: "AlphaNet: Single Morphing Attack Detection using Multiple Contributors", in Proceedings of IEEE International Workshop on Information Forensics and Security (WIFS), Nürnberg, DE, December 4-7, (2023)
  6. D. Osorio-Roig, M. Ghafourian, C. Rathgeb, R. Vera-Rodriguez, C. Busch, J. Fierrez: "Optimizing Key-Selection for Face-based One-Time Biometrics via Morphing", in Proceedings of IEEE International Workshop on Information Forensics and Security (WIFS), Nürnberg, DE, December 4-7, (2023)
  7. A. Zafar, C. Busch: "Analyzing eyebrow region for morphed image detection", in Proceedings Norwegian Information Security Conference (NISK), Stavanger, NO, November 27-30, (2023)
  8. H. Zhang, R. Raghavendra, K. Raja, C. Busch: "PIPE: Plugging in Identity Prior to Enhance Face Morphing Attack Based on Diffusion Model", in Proceedings Norwegian Information Security Conference (NISK), Stavanger, NO, November 27-30, (2023)
  9. J. Tapia, C. Busch: "Impact of Synthetic Images on Morphing Attack Detection Using a Siamese Network", in Proceedings of 26th Iberoamerican Congress on Pattern Recognition (CIARP), Coimbra, PT, November 27-30, (2023)
  10. J. Tapia, H. Zhang, R. Raghavendra, K. Raja, C. Busch: "Simulating Print/Scan Textures for Morphing Attack Detection", in Proceedings of the 31th European Signal Processing Conference (EUSIPCO 2023), Helsinki, Finland, September 4-8, (2023)
  11. J. Tapia, C. Busch: "Face Feature Visualisation of Single Morphing Attack Detection", in Proceedings of 11th International Workshop on Biometrics and Forensics (IWBF 2023), Barcelona, ES, April 19-20, (2023)
  12. S. Godage, F. Løvåsdal, S. Venkatesh, K. Raja, R. Raghavendra, C. Busch: "Analyzing Human Observer Ability in Morphing Attack Detection - Where Do We Stand?", in IEEE Transactions on Technology and Society Informatics (TTS), (2023)
  13. L. Dargaud, M. Ibsen, J. Tapia, C. Busch: "A Principal Component Analysis-Based Approach for Single Morphing Attack Detection", in Proceedings IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW), (2023)
  14. H. Myhrvold, H. Zhang, J. Tapia, R. Raghavendra, C. Busch: "Explainable Visualization for Morphing Attack Detection",in Proceedings Norwegian Information Security Conference (NISK), Kristiansand, NO, November 28 to December 1, (2022)
  15. K. Raja, G. Gupta, S. Venkatesh, R. Raghavendra, C. Busch: "Towards Generalized Morphing Attack Detection by Learning Residuals", in Elsevier Image and Vision Computing Transactions on Technology and Society (IVC), (2022)
  16. C. Rathgeb, R. Nichols, M. Ibsen, P. Drozdowski, C. Busch: "Crowd-powered Face Manipulation Detection: Fusing Human Examiner Decisions",in Proceedings of International Conference on Image Processing (ICIP), Bordeaux, FR, October 16-19, (2022)
  17. M. Ferrara, A. Franco, D. Maltoni, C. Busch: "Morphing Attack Potential", in Proceedings of 10th International Workshop on Biometrics and Forensics (IWBF 2022), Salzburg, AT, April 20-21, (2022)PDF-Version
  18. R. Nichols, C. Rathgeb, P. Drozdowski, C. Busch: "Psychophysical Evaluation of Human Performance in Detecting Digital Face Image Manipulations", in IEEE Access, (2022)PDF-Version
  19. M. Ibsen, C. Rathgeb, D. Fischer, P. Drozdowski, C. Busch: "Digital Face Manipulation in Biometric Systems", in Springer Handbook of Digital Face Manipulation and Detection: From DeepFakes to Morphing Attacks, (2022)
  20. U. Scherhag, C. Rathgeb, C. Busch: "Face Morphing Attack Detection Methods", in Springer Handbook of Digital Face Manipulation and Detection: From DeepFakes to Morphing Attacks, (2022)
  21. C. Rathgeb, R. Tolosana, R. Vera-Rodriguez, C. Busch: "Handbook of Digital Face Manipulation and Detection", Springer, (2022)
  22. R. Raghavendra, K. Raja, C. Busch: "Algorithmic Fairness in Face Morphing Attack Detection", in Proceedings IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW), (2022)
  23. P. Drozdowski, F. Stockhardt, C. Rathgeb, C. Busch: "Signal-level Fusion for Indexing and Retrieval of Facial Biometric Data", in IET Biometrics, (2022)PDF-Version
  24. J. Tapia, C. Busch: "Single Morphing Attack Detection using Feature Selection and Visualisation based on Mutual Information", in IEEE Access, (2021)
  25. S. Lorenz, U. Scherhag, C. Rathgeb, C. Busch: "Morphing Attack Detection: A Fusion Approach", in Proceedings 24th International Conference on Information Fusion, November 1-4, (2021)
  26. H. Zhang, S. Venkatesh, R. Raghavendra, K. Raja, N. Damer, C. Busch: "MIPGAN - Generating Strong and High Quality Morphing Attacks Using Identity Prior Driven GAN", in IEEE Transactions on Biometrics, Behavior, and Identity Science (TBIOM), (2021)PDF-Version
  27. C. Rathgeb, K. Pöppelman, C. Busch: "Face Morphing Attacks: A Threat to eLearning?", in Proceedings of IEEE Global Engineering Education Conference (EDUCON 2021), Vienna, AT, April 21-23, (2021)
  28. S. Venkatesh, R. Raghavendra, K. Raja, C. Busch: "Face Morphing Attack Generation & Detection: A Comprehensive Survey", in IEEE Transactions on Technology and Society (TTS), (2021)PDF-Version
  29. K. Raja, M. Ferrara, A. Franco, L. Spreeuwers, I. Batskos, F. Wit, M. Gomez-Barrero, U. Scherhag, D. Fischer, S. Venkatesh, J. Singh, G. Li, L. Bergeron, S. Isadskiy, R. Raghavendra, C. Rathgeb, D. Frings, U. Seidel, F. Knopjes, R. Veldhuis, D. Maltoni, C. Busch: "Morphing Attack Detection - Database, Evaluation Platform and Benchmarking", in IEEE Transactions on Information Forensics and Security (TIFS), (2020) PDF-Version
  30. L. Quin, F. Peng, S. Venkatesh, R. Raghavendar, M. Long, C. Busch: "Low Visual Distortion and Robust Morphing Attacks Based on Partial Face Image Manipulation", in IEEE Transactions on Biometrics, Behavior, and Identity Science (TBIOM), (2020) PDF-Version
  31. A. Roettcher, U. Scherhag, C. Busch: "Finding the Suitable Doppelgaenger for a Face Morphing Attack", in Proceedings of International Joint Conference on Biometrics (IJCB 2020), Houston, US, September 28 – October 1, (2020) PDF-Version
  32. S. Venkatesh, K. Raja, R. Raghavendra, C. Busch: "On the Influence of Ageing on Face Morph Attacks: Vulnerability and Detection", in Proceedings of International Joint Conference on Biometrics (IJCB 2020), Houston, US, September 28 – October 1, (2020) PDF-Version
  33. U. Scherhag, J. Kunze, C. Rathgeb, C. Busch: "Face Morph Detection for Unknown Morphing Algorithms and Image Sources: A Multi-Scale Block Local Binary Pattern Fusion Approach", in IET Biometrics, (2020) PDF-Version
  34. S. Venkatesh, R. Raghavendra, K. Raja, C. Busch: "Single Image Face Morphing Attack Detection Using Ensemble of Features", in Proceedings of 23rd International Conference on Information Fusion (FUSION 2020), Virtual Conference, July 6-9, (2020) PDF-Version
  35. U. Scherhag, C. Rathgeb, J. Merkle, C. Busch: "Deep Face Representations for Differential Morphing Attack Detection", in IEEE Transactions on Information Forensics and Security (TIFS), (2020) PDF-Version
  36. S. Venkatesh, H. Zhang, R. Raghavendra, K. Raja, N. Damer, C. Busch: "Can GAN Generated Morphs Threaten Face Recognition Equally as Landmark Based Morphs? - Vulnerability and Detection", in Proceedings of 8th International Workshop on Biometrics and Forensics (IWBF 2020), Porto, PT, April 29 - 30, (2020) PDF-Version
  37. S. Venkatesh, R. Raghavendra, K. Raja, L. Spreeuwers, R. Veldhuis, C. Busch: "Detecting Morphed Face Attacks Using Residual Noise from Deep Multi-scale Context Aggregation Network“, in Proceedings of Winter Conference on Applications of Computer Vision (WACV ’20), Colorado, US, March 1-5, (2020) PDF-Version
  38. J. Merkle, C. Rathgeb, U. Scherhag, C. Busch: "Morphing-Angriffe: Ein Sicherheitsrisiko für Gesichtserkennungssysteme“, in Datenschutz und Datensicherheit (DuD), Vol. 44, no. 1, pp. 26-31, (2020) PDF-Version
  39. J. Singh, S. Venkatesh, K. Raja, R.Raghavendra, C. Busch: "Robust Morph-Detection at Automated Border Control Gate using Deep Decomposed 3D Shape & Diffuse Reflectance", in Proceedings of the 15th International Conference on Signal Image Technology & Internet Based Systems (SITIS 2019), November 26-29, Sorrento - Naples, IT, (2019) PDF-Version
  40. S. Venkatesh, R. Raghavendra, K. Raja, L. Spreeuwers, R. Veldhuis, C. Busch: "Morphed Face Detection Based on Deep Color Residual Noise", in Proceedings of the ninth International Conference on Image Processing Theory, Tools and Applications (IPTA 2019), Istanbul, Turkey, November 6-9, (2019) PDF-Version
  41. C. Busch, S. Caillebotte, U. Seidel, F. Knopjes, D. Maltoni, M. Ferrara, R. Veldhuis, L. Spreeuwers, K. Raja, R. Raghavendra, M. Gomez-Barrero, C. Rathgeb: "Face Morphing Attacks: What needs to be done ", in Proceedings International Conference on Biometrics for Borders (ICBB), Frontex, Warsaw, October 9-10, (2019)PDF-Version
  42. J. Merkle, C. Rathgeb, U. Scherhag, C. Busch, R. Breithaupt: "Face Morphing Detection: Issues and Challenges ", in Proceedings International Conference on Biometrics for Borders (ICBB), Frontex, Warsaw, October 9-10, (2019)PDF-Version
  43. U. Scherhag, L. Debiasi, C. Rathgeb, C. Busch and A. Uhl: "Detection of Face Morphing Attacks based on PRNU Analysis“, in IEEE TBIOM, (2019)PDF-Version
  44. U. Scherhag, C. Rathgeb, J. Merkle, R. Breithaupt, C. Busch: "Face Recognition Systems under Morphing Attacks: A Survey“, in IEEE Access, (2019)PDF-Version
  45. P. Drozdowski, C. Rathgeb, C. Busch: "Turning a Vulnerability into an Asset: Accelerating Facial Identification with Morphing", in Proceedings of the International Conference on Acoustics, Speech and Signal Processing (ICASSP 2019), Brighton, U.K., May 12-17, (2019)PDF-Version
  46. R.Raghavendra, S. Venkatesh, K. Raja, C. Busch: "Towards making Morphing Attack Detection robust using hybrid Scale-Space Colour Texture Features“, in Proceedings of 5th International Conference on Identity, Security and Behaviour Analysis (ISBA 2019), Hyderabad, IN, January 22-24, (2019)PDF-Version
  47. L. Debiasi, C. Rathgeb, U. Scherhag, A. Uhl, C. Busch:"PRNU Variance Analysis for Morphed Face Image Detection", in Proceedings of 9th International Conference on Biometrics: Theory, Applications and Systems (BTAS 2018), Los Angeles, US, October 22-25, (2018)PDF-Version
  48. R.Raghavendra, S. Venkatesh, K. Raja, C. Busch: "Detecting Face Morphing Attacks with Collaborative Representation of Steerable Scale-Space Features“, in Proceedings of 3rd International Conference on Computer Vision and Image Processing (CVIP 2018), Japalpur, IN, September 29 - October 1, (2018)PDF-Version
  49. U. Scherhag, D. Budhrani, M. Gomez-Barrero, C. Busch: "Detecting Morphed Face Images Using Facial Landmarks“, in Proceedings of International Conference on Image and Signal Processing (ICISP 2018), Cherbourg, FR, July 2-4, (2018)PDF-Version
  50. U. Scherhag, C. Rathgeb, C. Busch: "Performance Variation of Morphed Face Image Detection Algorithms across different Datasets“, in Proceedings of 6th International Workshop on Biometrics and Forensics (IWBF 2018), Sassari, IT, June 7-8, (2018)PDF-Version
  51. L. Debiasi, U. Scherhag, C. Rathgeb, A. Uhl, C. Busch: "PRNU-based Detection of Morphed Face Images“, in Proceedings of 6th International Workshop on Biometrics and Forensics (IWBF 2018), Sassari, IT, June 7-8, (2018)PDF-Version
  52. U. Scherhag, C. Rathgeb, C. Busch: "Detection of Morphed Faces from Single Images: a Multi-Algorithm Fusion Approach“, in Proceedings if of the 2nd International Conference on Biometric Engineering and Applications (ICBEA 2018), Amsterdam, The Netherlands, May 16-18, (2018)PDF-Version
  53. U. Scherhag, C. Rathgeb and C. Busch: "Towards Detection of Morphed Face Images in electronic Travel Documents“, in Proceedings of the 13th IAPR International Workshop on Document Analysis Systems (DAS 2018), Vienna, Austria, April 24-27, (2018)PDF-Version
  54. M. Gomez-Barrero, C. Rathgeb, U. Scherhag, C. Busch: "Predicting the Vulnerability of Biometric Systems to Attacks based on Morphed Biometric Samples“, in IET Biometrics, (2018)PDF-Version
  55. C. Rathgeb, C. Busch: "On the Feasibility of Creating Morphed Iris-Codes", in Proceedings of International Joint Conference on Biometrics (IJCB 2017), Denver, Colorado, October 1-4, (2017)PDF-Version
  56. R. Raghavendra, K. Raja, S. Venkatesh, C. Busch: "Face Morphing Versus Face Averaging: Vulnerability and Detection", in Proceedings of International Joint Conference on Biometrics (IJCB 2017), Denver, Colorado, October 1-4, (2017)PDF-Version
  57. U. Scherhag, A. Nautsch, C. Rathgeb, M. Gomez-Barrero, R. Veldhuis, L. Spreeuwers, M. Schils, D. Maltoni, P. Grother, S. Marcel, R. Breithaupt, R. Raghavendra, C. Busch: "Biometric Systems under Morphing Attacks: Assessment of Morphing Techniques and Vulnerability Reporting", in Proceedings of the IEEE 16th International Conference of the Biometrics Special Interest Group (BIOSIG), Darmstadt, September 20-22, (2017)PDF-Version
  58. R. Raghavendra, K. Raja, S. Venkatesh, C. Busch: "Transferable Deep-CNN features for detecting digital and print-scanned morphed face images", in Proceedings of 30th International Conference on Computer Vision and Pattern Recognition Workshop (CVPRW 2017), Honolulu, Hawaii, July 21-26, (2017)PDF-Version
  59. M. Gomez-Barrero, C. Rathgeb, U. Scherhag, C. Busch: "Is Your Biometric System Robust to Morphing Attacks?", in Proceedings of 5th International Workshop on Biometrics and Forensics (IWBF 2017), Coventry, UK, April 4-5, (2017)PDF-Version
  60. U. Scherhag, R. Raghavendra, K. Raja, M. Gomez-Barrero, C. Rathgeb, C. Busch: "On The Vulnerability Of Face Recognition Systems Towards Morphed Face Attacks", in Proceedings of 5th International Workshop on Biometrics and Forensics (IWBF 2017), Coventry, UK, April 4-5, (2017)PDF-Version
  61. R. Raghavendra, K. Raja, C. Busch: "Detecting Morphed Facial Images", in Proceedings of 8th IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS-2016), September 6-9, Niagra Falls, USA, (2016)PDF-Version

Further Information

Talks on MAD

Morphing Attack Databases

top of page