Full Length ArticleA comprehensive overview of biometric fusion
Introduction
Biometrics refers to the automated process of recognizing an individual based on his/her physical or behavioral traits such as face, fingerprints, voice, iris, gait, or signature [1]. These traits are often referred to as biometric modalities or biometric cues. Over the past several years, a number of different biometric modalities [2], [3], [4] have been explored for use in various applications ranging from personal device access systems to border control systems [5].
The general framework of a typical biometric recognition system is summarized in Fig. 1. Here, given some input data (e,g, an image, video or signal), a typical biometric recognition system first performs segmentation or detection, which involves extracting the modality of interest from the input. This is followed by preprocessing, which involves data alignment, noise removal, or data enhancement. Features are extracted from the preprocessed data, which are then used by a classifier for biometric recognition. The recognition process may involve associating an identity with the input data (e.g., biometric identification) or determining if two instances of input data pertain to the same identity (e.g., biometric verification).
Traditionally, biometric recognition systems are unibiometric, which utilize a single biometric cue, and thus may encounter problems due to missing information (e.g., occluded face), poor data quality (e.g. dry fingerprint), overlap between identities (e.g., face images of twins) or limited discriminability (e.g., hand geometry). In such situations, it may be necessary to utilize multiple biometric cues in order to improve the recognition accuracy. For example, a border control system may use both face and fingerprints to establish the identity of an individual [6], [7]. In some cases, a biometric cue could be used alongside traditional user-validation schemes such as passwords/passcodes to verify a user’s identity. For example, many smartphone devices incorporate such a dual-factor authentication scheme [8], [9]. In other applications, multiple sensors could be used to acquire the same biometric modality, thereby allowing the system to operate in different environments. For example, a face recognition system may use a visible spectrum camera as well as a near-infrared camera to image a person’s face, thus facilitating biometric recognition in nighttime environment. The aforementioned examples underscore the need for effective biometric fusion techniques that can consolidate information from multiple sources.
The term multibiometrics has often been used to connote biometric fusion in the literature [10]. In order to develop a multibiometric system, one must consider the following three questions, (i) what to fuse, (ii) when to fuse, and (iii) how to fuse, each of which have been explored in this article.
What to fuse involves selecting the different sources of information to be combined, such as multiple algorithms or multiple modalities. When to fuse is answered by analyzing the different levels of fusion, that is, the various stages in the biometric recognition pipeline at which information can be fused. How to fuse refers to the fusion method that is used to consolidate the multiple sources of information.
Given data from only a single modality (say face only), the performance of a recognition system can often be enhanced by incorporating some ancillary information. Incorporating details such as image quality, subject demographics, soft biometric attributes, and contextual meta-data has shown to improve the performance of recognition systems [72], [131]. While recognition performance is a major metric for evaluating biometric systems, it is important to focus on the security (and privacy) aspect of such systems as well [205]. Information fusion is seen as a viable option for securing the biometric templates in a multibiometric system. Cryptosystems based on multiple modalities have been proposed to securely store biometric templates and prevent access to the original data [11]. Biometric systems are also susceptible to spoof attacks. That is, an adversary can impersonate another person’s identity by presenting a fake or altered biometric trait and gain unauthorized access. Information fusion can play a major role in the detection and deflection of such malicious activities [12]. This paper focuses on the above mentioned aspects of biometric fusion, and thus presents a survey of information fusion techniques along the lines of: (i) biometrics and ancillary information, (ii) spoof (or presentation attack) detection, and (iii) multibiometric cryptosystems.
Section snippets
Multibiometric systems
A multibiometric system can overcome some of the limitations of a unibiometric system by combining information from different sources in a principled manner. The utilization of multiple sources often results in improved recognition performance and enhanced system reliability, since the combined information is likely to be more distinctive to an individual compared to the information obtained from a single source.
Biometrics and ancillary information
Researchers have incorporated ancillary information in the traditional biometrics pipeline in order to improve recognition performance. Ancillary data refers to any additional information that can be provided about a particular biometric sample which might aid in the recognition process. Fig. 5 presents some of the commonly used sources of ancillary information, viz., quality estimates, soft biometric attributes, and contextual information. Ancillary information has also been used to perform
Biometric fusion and presentation attack detection
A biometric system is vulnerable to a number of attacks [153]. One such attack is referred to as a presentation attack where an adversary presents a fake or altered biometric trait to the sensor with the intention of spoofing someone else’s trait; or creating a new virtual identity based on the presented trait; or obfuscating their own trait. Detecting such attacks is essential in improving the security and integrity of the biometric system. In earlier literature, presentation attack was
Multibiometric cryptosystems
Data used by a biometric system can be encrypted using strong cryptographic techniques in order to secure them against external attacks. In addition, biometric matching has to be performed in the encrypted domain to obviate the need to decrypt the data, thereby preventing an adversary from viewing the original data at any time. In multibiometric systems, where data from multiple biometric sources are available, each data piece has to be encrypted. Such systems consisting of multiple biometric
Research challenges and future directions
Biometric fusion has witnessed significant advancements over the past two decades in terms of algorithm development, sources of information being fused, application domains and operational data collected. The literature review in Sections 2–5 suggests that research in biometric fusion has primarily focused on combining multiple sources of information for different problems and designing new fusion algorithms. The questions of what, when, and how to fuse are important for the development of a
Acknowledgments
M. Singh and R. Singh are partially supported through the Infosys CAI at IIIT-Delhi. Ross was supported by the US National Science Foundation under Grant Numbers 1618518 and 1617466 during the writing of this article.
References (216)
- et al.
50 Years of biometric research: accomplishments, challenges, and opportunities
Pattern Recognit. Lett.
(2016) - et al.
Integrated multilevel image fusion and match score fusion of visible and infrared face images for robust face recognition
Pattern Recognit.
(2008) - et al.
Hierarchical fusion of multi-spectral face images for improved recognition performance
Inf. Fusion
(2008) - et al.
A hybrid fingerprint matcher
Pattern Recognit.
(2003) - et al.
Personal authentication using multiple palmprint representation
Pattern Recognit.
(2005) - et al.
Cancelable multi-biometrics: mixing iris-codes based on adaptive bloom filters
Comput. Secur.
(2014) - et al.
Probabilistic recognition of human faces from video
Comput. Vision Image Understanding
(2003) - et al.
Group sparse representation based classification for multi-feature multimodal biometrics
Inf. Fusion
(2016) - et al.
Adaptive fusion of speech and lip information for robust speaker identification
Digit. Signal Process.
(2001) - et al.
Multimodal speaker/speech recognition using lip motion, lip texture and audio
Signal Processing
(2006)
Multiple classifiers in biometrics. part 2: trends and challenges
Inf. Fusion
Adaptive management of multimodal biometrics fusion using ant colony optimization
Inf. Fusion
Information fusion in biometrics
Pattern Recognit. Lett.
Palmprint identification using feature-level fusion
Pattern Recognit.
Score normalization in multimodal biometric systems
Pattern Recognit.
Database, protocols and tools for evaluating score-level fusion algorithms in biometric authentication
Pattern Recognit.
Performance evaluation of score level fusion in multimodal biometric systems
Pattern Recognit.
A comparison of imputation methods for handling missing scores in biometric fusion
Pattern Recognit
Decision-level fusion in fingerprint verification
Pattern Recognit.
Discriminative multimodal biometric authentication based on quality measures
Pattern Recognit.
Fusing multimodal biometrics with quality estimates via a bayesian belief network
Pattern Recognit
Quality-augmented fusion of level-2 and level-3 fingerprint information using DSm theory
Int. J. Approximate Reason.
Qfuse: online learning framework for adaptive biometric system
Pattern Recognit.
An adaptive bimodal recognition framework using sparse coding for face and ear
Pattern Recognit Lett.
On the effectiveness of soft biometrics for increasing face verification rates
Comput. Vision Image Understand.
Soft biometrics for surveillance: an overview
Handbook of statistics
Introduction to Biometrics: A Textbook
Handbook of face Recognition
Handbook of iris Recognition
Handbook of Fingerprint Recognition
Multilevel passcode authentication
US Patent Number US8806610B2
Handbook of multibiometrics
Multibiometric cryptosystems based on feature-level fusion
IEEE Trans. Inf. Forensics Secur.
Combining match scores with liveness values in a fingerprint verification system
IEEE Conference on Biometrics: Theory, Applications and Systems
A protocol for multibiometric data acquisition, storage and dissemination
Technical Report, WVU, Lane Department of Computer Science and Electrical Engineering
Face presentation attack with latex masks in multispectral videos
IEEE Conference on Computer Vision and Pattern Recognition Workshops
RGB-D Face recognition with texture and attribute features
IEEE Trans. Inf. Forensics Secur.
The effect of pixel-level fusion on object tracking in multi-sensor surveillance video
IEEE Conference on Computer Vision and Pattern Recognition
Multibiometric template security using fuzzy vault
IEEE International Conference on Biometrics: Theory, Applications and Systems
Single-sensor multi-instance fingerprint and eigenfinger recognition using weighted score combination methods
Int. J. Biom.
Face recognition in video: Adaptive fusion of multiple matchers
IEEE Conference on Computer Vision and Pattern Recognition Workshops
Face verification via learned representation on feature-rich video frames
IEEE Trans. Inf. Forensics Secur.
On recognizing faces in videos using clustering-based re-ranking and fusion
IEEE Trans. Inf. Forensics Secur.
Fusion of static and dynamic body biometrics for gait recognition
IEEE Trans. Circuits Syst. Video Technol.
Incremental learning for video-based gait recognition with LBP flow
IEEE Trans. Cybern.
Human identification based on gait
Cited by (166)
Deep learning techniques for biometric security: A systematic review of presentation attack detection systems
2024, Engineering Applications of Artificial IntelligenceTwo heads are better than one: Dual systems obtain better performance in facial comparison
2023, Forensic Science InternationalInterval type-2 fuzzy set induced fuzzy rank-level fusion for face recognition
2023, Applied Soft ComputingDistributed deep learning-based signal classification for time–frequency synchronization in wireless networks
2023, Computer CommunicationsA survey of identity recognition via data fusion and feature learning
2023, Information Fusion