Elsevier

Information Fusion

Volume 52, December 2019, Pages 187-205
Information Fusion

Full Length Article
A comprehensive overview of biometric fusion

https://doi.org/10.1016/j.inffus.2018.12.003Get rights and content

Abstract

The performance of a biometric system that relies on a single biometric modality (e.g., fingerprints only) is often stymied by various factors such as poor data quality or limited scalability. Multibiometric systems utilize the principle of fusion to combine information from multiple sources in order to improve recognition accuracy whilst addressing some of the limitations of single-biometric systems. The past two decades have witnessed the development of a large number of biometric fusion schemes. This paper presents an overview of biometric fusion with specific focus on three questions: what to fuse, when to fuse, and how to fuse. A comprehensive review of techniques incorporating ancillary information in the biometric recognition pipeline is also presented. In this regard, the following topics are discussed: (i) incorporating data quality in the biometric recognition pipeline; (ii) combining soft biometric attributes with primary biometric identifiers; (iii) utilizing contextual information to improve biometric recognition accuracy; and (iv) performing continuous authentication using ancillary information. In addition, the use of information fusion principles for presentation attack detection and multibiometric cryptosystems is also discussed. Finally, some of the research challenges in biometric fusion are enumerated. The purpose of this article is to provide readers a comprehensive overview of the role of information fusion in biometrics.

Introduction

Biometrics refers to the automated process of recognizing an individual based on his/her physical or behavioral traits such as face, fingerprints, voice, iris, gait, or signature [1]. These traits are often referred to as biometric modalities or biometric cues. Over the past several years, a number of different biometric modalities [2], [3], [4] have been explored for use in various applications ranging from personal device access systems to border control systems [5].

The general framework of a typical biometric recognition system is summarized in Fig. 1. Here, given some input data (e,g, an image, video or signal), a typical biometric recognition system first performs segmentation or detection, which involves extracting the modality of interest from the input. This is followed by preprocessing, which involves data alignment, noise removal, or data enhancement. Features are extracted from the preprocessed data, which are then used by a classifier for biometric recognition. The recognition process may involve associating an identity with the input data (e.g., biometric identification) or determining if two instances of input data pertain to the same identity (e.g., biometric verification).

Traditionally, biometric recognition systems are unibiometric, which utilize a single biometric cue, and thus may encounter problems due to missing information (e.g., occluded face), poor data quality (e.g. dry fingerprint), overlap between identities (e.g., face images of twins) or limited discriminability (e.g., hand geometry). In such situations, it may be necessary to utilize multiple biometric cues in order to improve the recognition accuracy. For example, a border control system may use both face and fingerprints to establish the identity of an individual [6], [7]. In some cases, a biometric cue could be used alongside traditional user-validation schemes such as passwords/passcodes to verify a user’s identity. For example, many smartphone devices incorporate such a dual-factor authentication scheme [8], [9]. In other applications, multiple sensors could be used to acquire the same biometric modality, thereby allowing the system to operate in different environments. For example, a face recognition system may use a visible spectrum camera as well as a near-infrared camera to image a person’s face, thus facilitating biometric recognition in nighttime environment. The aforementioned examples underscore the need for effective biometric fusion techniques that can consolidate information from multiple sources.

The term multibiometrics has often been used to connote biometric fusion in the literature [10]. In order to develop a multibiometric system, one must consider the following three questions, (i) what to fuse, (ii) when to fuse, and (iii) how to fuse, each of which have been explored in this article.

What to fuse involves selecting the different sources of information to be combined, such as multiple algorithms or multiple modalities. When to fuse is answered by analyzing the different levels of fusion, that is, the various stages in the biometric recognition pipeline at which information can be fused. How to fuse refers to the fusion method that is used to consolidate the multiple sources of information.

Given data from only a single modality (say face only), the performance of a recognition system can often be enhanced by incorporating some ancillary information. Incorporating details such as image quality, subject demographics, soft biometric attributes, and contextual meta-data has shown to improve the performance of recognition systems [72], [131]. While recognition performance is a major metric for evaluating biometric systems, it is important to focus on the security (and privacy) aspect of such systems as well [205]. Information fusion is seen as a viable option for securing the biometric templates in a multibiometric system. Cryptosystems based on multiple modalities have been proposed to securely store biometric templates and prevent access to the original data [11]. Biometric systems are also susceptible to spoof attacks. That is, an adversary can impersonate another person’s identity by presenting a fake or altered biometric trait and gain unauthorized access. Information fusion can play a major role in the detection and deflection of such malicious activities [12]. This paper focuses on the above mentioned aspects of biometric fusion, and thus presents a survey of information fusion techniques along the lines of: (i) biometrics and ancillary information, (ii) spoof (or presentation attack) detection, and (iii) multibiometric cryptosystems.

Section snippets

Multibiometric systems

A multibiometric system can overcome some of the limitations of a unibiometric system by combining information from different sources in a principled manner. The utilization of multiple sources often results in improved recognition performance and enhanced system reliability, since the combined information is likely to be more distinctive to an individual compared to the information obtained from a single source.

Biometrics and ancillary information

Researchers have incorporated ancillary information in the traditional biometrics pipeline in order to improve recognition performance. Ancillary data refers to any additional information that can be provided about a particular biometric sample which might aid in the recognition process. Fig. 5 presents some of the commonly used sources of ancillary information, viz., quality estimates, soft biometric attributes, and contextual information. Ancillary information has also been used to perform

Biometric fusion and presentation attack detection

A biometric system is vulnerable to a number of attacks [153]. One such attack is referred to as a presentation attack where an adversary presents a fake or altered biometric trait to the sensor with the intention of spoofing someone else’s trait; or creating a new virtual identity based on the presented trait; or obfuscating their own trait. Detecting such attacks is essential in improving the security and integrity of the biometric system. In earlier literature, presentation attack was

Multibiometric cryptosystems

Data used by a biometric system can be encrypted using strong cryptographic techniques in order to secure them against external attacks. In addition, biometric matching has to be performed in the encrypted domain to obviate the need to decrypt the data, thereby preventing an adversary from viewing the original data at any time. In multibiometric systems, where data from multiple biometric sources are available, each data piece has to be encrypted. Such systems consisting of multiple biometric

Research challenges and future directions

Biometric fusion has witnessed significant advancements over the past two decades in terms of algorithm development, sources of information being fused, application domains and operational data collected. The literature review in Sections 2–5 suggests that research in biometric fusion has primarily focused on combining multiple sources of information for different problems and designing new fusion algorithms. The questions of what, when, and how to fuse are important for the development of a

Acknowledgments

M. Singh and R. Singh are partially supported through the Infosys CAI at IIIT-Delhi. Ross was supported by the US National Science Foundation under Grant Numbers 1618518 and 1617466 during the writing of this article.

References (216)

  • J. Fierrez et al.

    Multiple classifiers in biometrics. part 2: trends and challenges

    Inf. Fusion

    (2018)
  • A. Kumar et al.

    Adaptive management of multimodal biometrics fusion using ant colony optimization

    Inf. Fusion

    (2016)
  • A. Ross et al.

    Information fusion in biometrics

    Pattern Recognit. Lett.

    (2003)
  • A. Kong et al.

    Palmprint identification using feature-level fusion

    Pattern Recognit.

    (2006)
  • A. Jain et al.

    Score normalization in multimodal biometric systems

    Pattern Recognit.

    (2005)
  • N. Poh et al.

    Database, protocols and tools for evaluating score-level fusion algorithms in biometric authentication

    Pattern Recognit.

    (2006)
  • M. He et al.

    Performance evaluation of score level fusion in multimodal biometric systems

    Pattern Recognit.

    (2010)
  • Y. Ding et al.

    A comparison of imputation methods for handling missing scores in biometric fusion

    Pattern Recognit

    (2012)
  • S. Prabhakar et al.

    Decision-level fusion in fingerprint verification

    Pattern Recognit.

    (2002)
  • J. Fierrez-Aguilar et al.

    Discriminative multimodal biometric authentication based on quality measures

    Pattern Recognit.

    (2005)
  • D.E. Maurer et al.

    Fusing multimodal biometrics with quality estimates via a bayesian belief network

    Pattern Recognit

    (2008)
  • M. Vatsa et al.

    Quality-augmented fusion of level-2 and level-3 fingerprint information using DSm theory

    Int. J. Approximate Reason.

    (2009)
  • S. Bharadwaj et al.

    Qfuse: online learning framework for adaptive biometric system

    Pattern Recognit.

    (2015)
  • Z. Huang et al.

    An adaptive bimodal recognition framework using sparse coding for face and ear

    Pattern Recognit Lett.

    (2015)
  • H. Zhang et al.

    On the effectiveness of soft biometrics for increasing face verification rates

    Comput. Vision Image Understand.

    (2015)
  • D.A. Reid et al.

    Soft biometrics for surveillance: an overview

    Handbook of statistics

    (2013)
  • A. Jain et al.

    Introduction to Biometrics: A Textbook

    (2011)
  • A.K. Jain et al.

    Handbook of face Recognition

    (2011)
  • M.J. Burge et al.

    Handbook of iris Recognition

    (2013)
  • D. Maltoni et al.

    Handbook of Fingerprint Recognition

    (2009)
  • Spain airports implement a multi-biometric solution,...
  • Multi-biometrics: the future of border control,...
  • V. Draluk et al.

    Multilevel passcode authentication

    US Patent Number US8806610B2

    (2014)
  • [In-Depth Look] Samsungs Biometric Technologies Bring Added Security and Convenience to the Galaxy S8,...
  • A. Ross et al.

    Handbook of multibiometrics

    (2006)
  • A. Nagar et al.

    Multibiometric cryptosystems based on feature-level fusion

    IEEE Trans. Inf. Forensics Secur.

    (2012)
  • E. Marasco et al.

    Combining match scores with liveness values in a fingerprint verification system

    IEEE Conference on Biometrics: Theory, Applications and Systems

    (2012)
  • S. Crihalmeanu et al.

    A protocol for multibiometric data acquisition, storage and dissemination

    Technical Report, WVU, Lane Department of Computer Science and Electrical Engineering

    (2007)
  • A. Agarwal et al.

    Face presentation attack with latex masks in multispectral videos

    IEEE Conference on Computer Vision and Pattern Recognition Workshops

    (2017)
  • G. Goswami et al.

    RGB-D Face recognition with texture and attribute features

    IEEE Trans. Inf. Forensics Secur.

    (2014)
  • N. Cvejic et al.

    The effect of pixel-level fusion on object tracking in multi-sensor surveillance video

    IEEE Conference on Computer Vision and Pattern Recognition

    (2007)
  • K. Nandakumar et al.

    Multibiometric template security using fuzzy vault

    IEEE International Conference on Biometrics: Theory, Applications and Systems

    (2008)
  • A. Uhl et al.

    Single-sensor multi-instance fingerprint and eigenfinger recognition using weighted score combination methods

    Int. J. Biom.

    (2009)
  • U. Park et al.

    Face recognition in video: Adaptive fusion of multiple matchers

    IEEE Conference on Computer Vision and Pattern Recognition Workshops

    (2007)
  • G. Goswami et al.

    Face verification via learned representation on feature-rich video frames

    IEEE Trans. Inf. Forensics Secur.

    (2017)
  • H.S. Bhatt et al.

    On recognizing faces in videos using clustering-based re-ranking and fusion

    IEEE Trans. Inf. Forensics Secur.

    (2014)
  • L. Wang et al.

    Fusion of static and dynamic body biometrics for gait recognition

    IEEE Trans. Circuits Syst. Video Technol.

    (2004)
  • M. Hu et al.

    Incremental learning for video-based gait recognition with LBP flow

    IEEE Trans. Cybern.

    (2013)
  • M.S. Nixon et al.

    Human identification based on gait

    (2010)
  • Cited by (166)

    View all citing articles on Scopus
    View full text