Abstract
Although the technical evaluation of biometric identification devices has a history spanning over two decades, it is only now that a general consensus on test and reporting measures and methodologies is developing in the scientific community. By “technical evaluation”, we mean the measurement of the five parameters generally of interest to engineers and physical scientists: false match and false non-match rates, binning error rate, penetration coefficient and transaction times. Additional measures, such as “failure to enroll” or “failure to acquire”, indicative of the percentage of the general population unable to use any particular biometric method, are also important. We have not included in this chapter measures of more interest to social scientists, such as user perception and acceptability. Most researchers now accept the “Receiver Operating Characteristic” (ROC) curve as the appropriate measure of the application-dependent technical performance of any biometric identification device. Further, we now agree that the error rates illustrated in the ROC must be normalized to be independent of the database size and other “accept/reject” decision parameters of the test. This chapter discusses the general approach to application-dependent, decision-policy independent testing and reporting of technical device performance and gives an example of one practical test. System performance prediction based on test results is also discussed.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
W. W. Peterson, and T. G. Birdsall, The Theory of Signal Detectability, Electronic Defense Group, University of Michigan, Technical Report 13, 1954.
W. P. Tanner, and J. A. Swets, “A Decision-Making Theory of Visual Detection,” Psychological Review, Vol. 61, pp. 401–409, 1954.
D. M. Green and J. A. Swets, Signal Detection Theory and Psychophysics, Wiley, 1966.
J. A. Swets (editor), Signal Detection and Recognition by Human Observers, Wiley, 1964.
J. P. Egan, Signal Detection Theory and ROC Analysis, Academic Press, 1975.
J.L. Wayman, “A Dual Channel Approach to Speaker Verification,” Proceedings Speech Research Symposium XII, Rutgers University, June 1992.
J. G. Daugman, “Biometric personal identification system based on iris analysis,” U.S. Patent 5291560, 1994.
S.S. Wilks, Mathematical Statistics, Wiley and Sons, New York, pp. 83, 1962.
J.P. Campbell, address to the 9th Biometric Consortium meeting, Crystal City, VA, April 1997.
J. Williams, Glossary of Biometric Terms, Security Industry Association, 1995.
W. Shen, M. Surette, and R. Khanna, “Evaluation of Automated Biometrics-Based Identification and Verification Systems,” Proceedings IEEE, Vol. 85, pp. 1464–1479, September 1997.
P. Bickel, response to NSA-MSP Problem #97-21, University of California, Berkeley, Department of Statistics, 1998.
C. Franzen, “Convolution Methods for Mathematical Problems in Biometrics,” U.S. Naval Postgraduate School Technical Report, 1998.
J. Williams, “Proposed Standard for Biometric Decidability,” Proceedings CTST’96, pp. 223–234, 1996.
S. Kullback and R. Leibler, “On Information and Sufficiency,” Annals of Mathematical Statistics, Vol.22, pp. 79–86, 1951.
J.L. Wayman, “Error Rate Equations for the General Biometric System,” Automation and Robotics Magazine, Special Issue on Biometric Identification, January 1999 (To appear).
K. James and B. James, NSA SAG Problem 97-25, Dept. of Statistics, UC Berkeley, Monograph, 1998
D. Welsh and K. Sweitzer (of Ride and Show Engineering at Walt Disney World), Presentation, CardTech/SecurTech’ 97, May 21, 1997.
Orkand Corporation, “Personal Identifier Project: Final Report,” State of California Department of Motor Vehicles report DMV88-89, April 1990 (reprinted by the U.S. National Biometric Test Center).
J.L. Hennessy and D.A. Patterson, Computer Architecture: A Quantitative Approach, 2nd ed., Morgan Kaufman, San Francisco, 1996.
M. Abromowitz and I. Stegun, Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables, John Wiley and Sons, New York, 1972.
W. H. Press, B. P. Flannery, S. A. Teukolsky, W. H. Vetterling, Numerical Recipes, 2nd ed., Cambridge University Press, Cambridge, 1988.
P. Bickel, personal correspondence in response to NSA-SAG Problem #97-23, March 1998.
J. L. Wayman, Biometric Identifier Standards Research Final Report, College of Engineering, San Jose State University (sponsored by the Federal Highway Adminstration), October 1997.
B. Atal, “Automatic Recognition of Speakers from Their Voices,” Proceedings IEEE, Vol. 64, pp. 460–475, 1976.
A. Rosenberg, “Automatic Speaker Verification,” Proceedings IEEE, 64, pp. 475–487, 1976.
N. Dixon and T. Martin, Automatic Speech and Speaker Recognition, IEEE Press, NY, 1979.
G. Doddington, “Speaker Recognition: Identifying People by Their Voices”, Proceedings IEEE, Vol. 73, pp. 1651–1664, 1985.
A. Rosenberg and F. Soong, “Recent Research in Automatic Speaker Recognition,” Advances in Speech Signal Processing (Eds. S. Furui and M. Sondhi), Marcel Dekker, 1991.
J. Naik, “Speaker Verification: A Tutorial,” IEEE Communications Magazine, pp. 42–48, 1990.
J.P. Campbell, Jr., “Speaker Recognition: A Tutorial,” Proceedings IEEE, Vol. 85, pp. 1437–1463, September 1997.
J. P. Holmes, et al., “A Performance Evaluation of Biometric Identification Devices,” Sandia National Laboratories, SAND91-0276, June 1991.
F. Bouchier, J. Ahrens, and G. Wells, “Laboratory Evaluation of the IriScan Prototype Biometric Identifier,” Sandia National Laboratories, SAND96-1033, April 1996.
P.J. Phillips, et al., “FERET (Face-Recognition Technology) Recognition Algorithm Development and Test Results,” Army Research Laboratory, ARL-TR-995, October 1996.
P.J. Rauss, et al., “FERET (Face-Recognition Technology) Recognition Algorithms,” Proceedings of ATRWG Science and Technology Conference, July 1996.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1996 Springer Science+Business Media, Inc.
About this chapter
Cite this chapter
Wayman, J.L. (1996). Technical Testing and Evaluation of Biometric Identification Devices. In: Jain, A.K., Bolle, R., Pankanti, S. (eds) Biometrics. Springer, Boston, MA. https://doi.org/10.1007/0-306-47044-6_17
Download citation
DOI: https://doi.org/10.1007/0-306-47044-6_17
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-28539-9
Online ISBN: 978-0-306-47044-8
eBook Packages: Springer Book Archive