Letter to the Editor
A comment on the PCAST report: Skip the “match”/“non-match” stage

https://doi.org/10.1016/j.forsciint.2016.10.018Get rights and content

Highlights

  • Feature-comparison methods produce continuously-valued data.

  • The PCAST report advocates a two-stage procedure:

  • (1) Dichotomise the data into “match” or “non-match”.

  • (2) If “match”, assess correct acceptance and false acceptance rates.

  • A better procedure would directly statistically model the continuously-valued data.

Abstract

This letter comments on the report “Forensic science in criminal courts: Ensuring scientific validity of feature-comparison methods” recently released by the President’s Council of Advisors on Science and Technology (PCAST). The report advocates a procedure for evaluation of forensic evidence that is a two-stage procedure in which the first stage is “match”/“non-match” and the second stage is empirical assessment of sensitivity (correct acceptance) and false alarm (false acceptance) rates. Almost always, quantitative data from feature-comparison methods are continuously-valued and have within-source variability. We explain why a two-stage procedure is not appropriate for this type of data, and recommend use of statistical procedures which are appropriate.

Section snippets

Acknowledgments

This work was funded in-part by a fellowship awarded to Morrison by the Simons Foundation. Morrison, Balding, Dawid, Aitken, Robertson, Pope, Neil, Martire, Gill, Jamieson, de Zoete, and Caliebe would like to thank the Isaac Newton Institute for Mathematical Sciences for its hospitality during the program Probability and Statistics in Forensic Science which was supported by EPSRC Grant Number EP/K032208/1. All opinions expressed are those of the authors/signatories and do not necessarily

References (16)

  • President’s Council of Advisors on Science and Technology

    Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-comparison Methods

    (2016)
  • D.A. Berry et al.

    Statistical inference in crime investigations using deoxyribonucleic acid profiling

    Appl. Stat.

    (1992)
  • I.W. Evett et al.

    An illustration of efficient statistical methods for RFLP analysis in forensic science

    Am. J. Hum. Genet.

    (1993)
  • C.G.G. Aitken et al.

    Statistics and the Evaluation of Forensic Evidence for Forensic Scientist

    (2004)
  • L.A. Foreman et al.

    Interpreting DNA evidence: a review

    Int. Stat. Rev.

    (2003)
  • C.G.G. Aitken

    Statistics in forensic science. Part II. An aid to evaluation of evidence

    Probl. Forensic Sci.

    (2006)
  • D.A. Berry

    Inferences using DNA profiling in forensic identification and paternity cases

    Stat. Sci.

    (1991)
  • J.M. Curran et al.

    Forensic Interpretation of Glass Evidence

    (2000)
There are more references available in the full text version of this article.

Cited by (25)

  • The evidential strength of a combination of corresponding class features in tire examination

    2022, Forensic Science International
    Citation Excerpt :

    Thus, for our case this would entail reporting the expert-specific error rate of a black-box study for a series of tire comparison cases. We agree with Morrison et al. [42] who criticize this approach in general. Also, we think that general error rates are not very relevant in a specific case [43].

  • Forensic Voice Comparison: Overview

    2022, Encyclopedia of Forensic Sciences: Volume 1-4, Third Edition
  • The strange persistence of (source) “identification” claims in forensic literature through descriptivism, diagnosticism and machinism

    2022, Forensic Science International: Synergy
    Citation Excerpt :

    The short answer is: almost everything, which is why these terms should have no place in forensic science,22 despite the fact that they are prominently used throughout official documents, such as the PCAST Report [77]. Longer answers have previously been given by other authors [47,72], making it all the more exasperating that “match”-terminology continues to be highly prevalent in current forensic science literature. A main problem of the term “match” is that it is used to denote two fundamentally different targets: observations on the one hand, and ground truth on the other hand.

View all citing articles on Scopus
View full text