A comment on the PCAST report: Skip the "match"/"non-match" stage

Geoffrey Stewart Morrison; David H Kaye; David J Balding; Duncan Taylor; Philip Dawid; Colin G G Aitken; Simone Gittelson; Grzegorz Zadora; Bernard Robertson; Sheila Willis; Susan Pope; Martin Neil; Kristy A Martire; Amanda Hepler; Richard D Gill; Allan Jamieson; Jacob de Zoete; R Brent Ostrum; Amke Caliebe

doi:10.1016/j.forsciint.2016.10.018

A comment on the PCAST report: Skip the "match"/"non-match" stage

Forensic Sci Int. 2017 Mar:272:e7-e9. doi: 10.1016/j.forsciint.2016.10.018. Epub 2016 Oct 26.

Authors

Geoffrey Stewart Morrison¹, David H Kaye², David J Balding³, Duncan Taylor⁴, Philip Dawid⁵, Colin G G Aitken⁶, Simone Gittelson⁷, Grzegorz Zadora⁸, Bernard Robertson⁹, Sheila Willis¹⁰, Susan Pope¹¹, Martin Neil¹², Kristy A Martire¹³, Amanda Hepler¹⁴, Richard D Gill¹⁵, Allan Jamieson¹⁶, Jacob de Zoete¹⁷, R Brent Ostrum¹⁸, Amke Caliebe¹⁹

Affiliations

¹ Independent Forensic Consultant, Vancouver, British Columbia, Canada; Adjunct Professor, Department of Linguistics, University of Alberta, Edmonton, Alberta, Canada; Simons Foundation Visiting Fellow, Isaac Newton Institute for Mathematical Sciences, Cambridge, England, United Kingdom. Electronic address: geoff-morrison@forensic-evaluation.net.
² Distinguished Professor and Weiss Family Scholar, Penn State Law, Pennsylvania State University, University Park, PA, United States; Regents' Professor Emeritus, Arizona State University College of Law and Department of Life Sciences, Tempe, AZ, United States.
³ Professor of Statistical Genetics, Centre for Systems Genomics, School of Biomedical Sciences, and School of Mathematics & Statistics, University of Melbourne, Melbourne, Victoria, Australia.
⁴ Principal Scientist of Forensic Statistics, Forensic Science South Australia, Adelaide, South Australia, Australia; Associate Professor of Biology, School of Biological Sciences, Flinders University, Adelaide, South Australia, Australia.
⁵ Emeritus Professor of Statistics, Department of Pure Mathematics and Mathematical Statistics, University of Cambridge, Cambridge, England, United Kingdom.
⁶ Professor of Forensic Statistics, School of Mathematics, University of Edinburgh, Edinburgh, Scotland, United Kingdom.
⁷ Forensic Statistician, Statistical Engineering Division, National Institute of Standards and Technology, Gaithersburg, MD, United States.
⁸ Associate Professor, Institute of Forensic Research, Krakow, Poland; Senior Lecturer, Chemometric Research Group, Institute of Chemistry, University of Silesia in Katowice, Poland.
⁹ Barrister, Wellington, New Zealand.
¹⁰ Director General, Forensic Science Ireland, Dublin, Ireland.
¹¹ Independent Consultant, DNA Principal Forensics Ltd., Reading, England, United Kingdom.
¹² Professor of Computer Science and Statistics, Risk Assessment and Decision Analysis Research Group, Department of Computer Science, Queen Mary, University of London, London, England, United Kingdom.
¹³ ARC DECRA Fellow & Senior Lecturer, School of Psychology, University of New South Wales, Sydney, New South Wales, Australia.
¹⁴ Senior Analyst, Innovative Decisions, Inc., Vienna, VA, United States.
¹⁵ Professor of Mathematical Probability, Department of Mathematics, Leiden University, Leiden, The Netherlands.
¹⁶ Director and Consultant Scientist, The Forensic Institute, Glasgow, Scotland, United Kingdom.
¹⁷ PhD Candidate (ABD), Department of Mathematics, University of Amsterdam, Amsterdam, The Netherlands.
¹⁸ Senior Forensic Document Examiner, Forensic Document Examination Section, Science and Engineering Directorate, Canada Border Services Agency, Ottawa, Ontario, Canada.
¹⁹ Forensic Statistician, Institute of Medical Informatics and Statistics, Kiel University, Kiel, Germany.

PMID: 27817943
DOI: 10.1016/j.forsciint.2016.10.018

Abstract

This letter comments on the report "Forensic science in criminal courts: Ensuring scientific validity of feature-comparison methods" recently released by the President's Council of Advisors on Science and Technology (PCAST). The report advocates a procedure for evaluation of forensic evidence that is a two-stage procedure in which the first stage is "match"/"non-match" and the second stage is empirical assessment of sensitivity (correct acceptance) and false alarm (false acceptance) rates. Almost always, quantitative data from feature-comparison methods are continuously-valued and have within-source variability. We explain why a two-stage procedure is not appropriate for this type of data, and recommend use of statistical procedures which are appropriate.

Keywords: False alarm; Forensic statistics; Likelihood ratio; Match/non-match; PCAST report; Sensitivity.

Publication types

Letter