Skip to main content
SpringerLink
Log in

Challenges in Comparative Meta-Analysis of the Accuracy of Multiple Diagnostic Tests

  • Protocol
  • First Online:
Meta-Research

Part of the book series: Methods in Molecular Biology ((MIMB,volume 2345))

Abstract

The rapid increase in diagnostic and screening techniques has urged the need to choose among multiple diagnostic tests. For the majority of diseases, there is more than a single test available, and studies usually compare a subset of these tests. In such cases, a separate meta-analysis of each test cannot provide a reliable answer on the relative accuracy of the multiple available tests. Extensions of standard (hierarchical) meta-analysis to network meta-analysis (NMA) models for the comparison of at least three diagnostic tests have been the subject of methodological research in recent years. NMA can be used to jointly analyze the totality of evidence in order to provide estimates of relative accuracy (sensitivity and specificity ), to compare tests that have not been compared head-to-head, and to obtain a ranking of all competing tests in order to further facilitate the decision-making process.

In this chapter, we illustrate current methodology for meta-analysis of multiple test comparisons, introduce NMA methods of diagnostic tests as an extension to the standard meta-analysis of diagnostic test accuracy (DTA) studies, and present existing approaches to rank tests according to their accuracy, specificity , and sensitivity . We also describe the basic concepts, underlying assumptions, and challenges in NMA of multiple diagnostic tests.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Protocol
USD 49.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 149.00
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Macaskill P GC, Deeks J, Harbord RM, Takwoingi Y. (2010) Chapter 10: Analysing and presenting results. In: Deeks JJ, Bossuyt PM, Gatsonis C (editors), Cochrane handbook for systematic reviews of diagnostic test accuracy version 1.0. The Cochrane Collaboration. http://srdta.cochrane.org/

  2. Reitsma JB, Glas AS, Rutjes AW et al (2005) Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol 58:982–990

    Article  Google Scholar 

  3. Rutter CM, Gatsonis CA (2001) A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Stat Med 20:2865–2884

    Article  CAS  Google Scholar 

  4. Chu H, Cole SR (2006) Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model approach. J Clin Epidemiol 59:1331–1332

    Article  Google Scholar 

  5. Arends LR, Hamza TH, Van Houwelingen JC et al (2008) Bivariate random effects meta-analysis of ROC curves. Med Decis Mak 28:621–638

    Article  CAS  Google Scholar 

  6. Harbord RM, Deeks JJ, Egger M et al (2007) A unification of models for meta-analysis of diagnostic accuracy studies. Biostatistics 8:239–251

    Article  Google Scholar 

  7. Takwoingi Y, Leeflang MMG, Deeks JJ (2013) Empirical evidence of the importance of comparative studies of diagnostic test accuracy. Ann Intern Med 158:544–554

    Article  Google Scholar 

  8. Agency for Healthcare Research and Quality (AHRQ) (2007) Methods reference guide for effectiveness and comparative effectiveness reviews

    Google Scholar 

  9. Canadian Agency for Drugs and Technologies in Health (CADTH) Guidelines for the Economic Evaluation of Health Technologies: Canada. www.cadth.ca/media/pdf/186_EconomicGuidelines_e.pdf

  10. Dias S, Welton NJ, Sutton AJ et al (2013) Evidence synthesis for decision making 1: introduction. Med Decis Mak 33:597–606

    Article  Google Scholar 

  11. Dias S, Welton NJ, Sutton AJ et al (2013) Evidence synthesis for decision making 4: inconsistency in networks of evidence based on randomized controlled trials. Med Decis Mak 33:641–656

    Article  Google Scholar 

  12. Indirect Comparisons Working Group (ISWG) Report of the Indirect Comparisons working Group to the Pharmaceutical Benefits Advisory Committee: Assessing indirect comparisons. www.health.gov.au/internet/main/publishing.nsf/Content/B11E8EF19B358E39CA25754B000A9C07/$File/ICWG%20Report%20FINAL2.pdf

  13. National Institute for Health and Clinical Excellence (NICE) Guide to the Methods of Technology Appraisal. www.nice.org.uk/media/B52/A7/TAMethodsGuideUpdatedJune2008.pdf

  14. Dimou NL, Adam M, Bagos PG (2016) A multivariate method for meta-analysis and comparison of diagnostic tests. Stat Med 35:3509–3523

    Article  Google Scholar 

  15. Hoyer A, Kuss O (2018) Meta-analysis for the comparison of two diagnostic tests-a new approach based on copulas. Stat Med 37:739–748

    Article  Google Scholar 

  16. Hoyer A, Kuss O (2018) Meta-analysis for the comparison of two diagnostic tests to a common gold standard: a generalized linear mixed model approach. Stat Methods Med Res 27:1410–1421

    Article  Google Scholar 

  17. Menten J, Lesaffre E (2015) A general framework for comparative Bayesian meta-analysis of diagnostic studies. BMC Med Res Methodol 15:70

    Article  Google Scholar 

  18. Nyaga VN, Aerts M, Arbyn M (2018) ANOVA model for network meta-analysis of diagnostic test accuracy data. Stat Methods Med Res 27:1766–1784

    Article  Google Scholar 

  19. Nyaga VN, Arbyn M, Aerts M (2018) Beta-binomial analysis of variance model for network meta-analysis of diagnostic test accuracy data. Stat Methods Med Res 27:2554–2566

    Article  Google Scholar 

  20. Trikalinos TA, Hoaglin DC, Small KM et al (2014) Methods for the joint meta-analysis of multiple tests. Res Synth Methods 5:294–312

    Article  Google Scholar 

  21. Rücker G (2018) Network meta-analysis of diagnostic test accuracy studies. In: Biondi-Zoccai G (ed) Diagnostic Meta-Analysis. Springer, Cham

    Google Scholar 

  22. Salanti G (2012) Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Res Synth Methods 3:80–97

    Article  Google Scholar 

  23. Veroniki AA, Vasiliadis HS, Higgins JP et al (2013) Evaluation of inconsistency in networks of interventions. Int J Epidemiol 42:332–345

    Article  Google Scholar 

  24. Takwoingi Y, Partlett C, Riley RD et al (2020) Methods and reporting of systematic reviews of comparative accuracy were deficient: a methodological survey and proposed guidance. J Clin Epidemiol 121:1–14

    Article  Google Scholar 

  25. Mavridis D, Salanti G (2013) A practical introduction to multivariate meta-analysis. Stat Methods Med Res 22:133–158

    Article  Google Scholar 

  26. Riley RD, Price MJ, Jackson D et al (2015) Multivariate meta-analysis using individual participant data. Res Synth Methods 6:157–174

    Article  CAS  Google Scholar 

  27. Riley RD, Thompson JR, Abrams KR (2008) An alternative model for bivariate random-effects meta-analysis when the within-study correlations are unknown. Biostatistics 9:172–186

    Article  Google Scholar 

  28. Koliopoulos G, Nyaga VN, Santesso N et al (2017) Cytology versus HPV testing for cervical cancer screening in the general population. Cochrane Database Syst Rev 8(8):CD008587

    PubMed  Google Scholar 

  29. Rabe-Hesketh S, Skrondal A (2008) Multilevel and longitudinal modeling using Stata. Stata Press, College Station, TX

    Google Scholar 

  30. Chang SM (2012) methods guide for medical test reviews. https://www.ncbi.nlm.nih.gov/books/NBK98241/ (ed)NCBI, Rockville, MD

  31. Trikalinos TA, Balion CM, Coleman CI et al (2012) Meta-analysis of test performance when there is a “gold standard”. In: Chang SM, Matchar DB, Smetana GW, Umscheid CA (eds) Methods guide for medical test reviews. Agency for Healthcare Research and Quality (US), Rockville (MD)

    Google Scholar 

  32. Ma X (2015) Statistical methods for multivariate meta-analysis of diagnostic tests. University of Minessota. https://conservancy.umn.edu/handle/11299/175241

    Google Scholar 

  33. Cheng W (2016) Network meta-analysis of diagnostic accuracy studies. Brown University. https://repository.library.brown.edu/studio/item/bdr:674079/

    Google Scholar 

  34. Owen RK, Cooper NJ, Quinn TJ et al (2018) Network meta-analysis of diagnostic test accuracy studies identifies and ranks the optimal diagnostic tests and thresholds for health care policy and decision-making. J Clin Epidemiol 99:64–74

    Article  Google Scholar 

  35. Lu G, Ades AE (2004) Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med 23:3105–3124

    Article  CAS  Google Scholar 

  36. Higgins JP, Jackson D, Barrett JK et al (2012) Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies. Res Synth Methods 3:98–110

    Article  CAS  Google Scholar 

  37. Lu G, Ades AE (2006) Assessing evidence inconsistency in mixed treatment comparisons. J Am Stat Assoc 101:447–459

    Article  CAS  Google Scholar 

  38. Jansen JP, Trikalinos T, Cappelleri JC et al (2014) Indirect treatment comparison/network meta-analysis study questionnaire to assess relevance and credibility to inform health care decision making: an ISPOR-AMCP-NPC good practice task force report. Value Health 17:157–173

    Article  Google Scholar 

  39. Deutsch R, Mindt M, Xu R et al (2009) Quantifying relative superiority among many binary-valued diagnostic tests in the presence of a gold standard. J Data Sci 7:161–177

    Article  Google Scholar 

Download references

Acknowledgments

GR was funded by DFG—German Research Foundation—grant number RU1747/1-2. YT is funded by a National Institute for Health Research (NIHR) Postdoctoral Fellowship. The views expressed are those of the authors and not necessarily those of the National Health Services (NHS), the NIHR, or the Department of Health and Social Care. We thank Dr. Antonios Athanasiou for his help on preparing the data for our illustrative example.

Author information

Authors and Affiliations

  1. Department of Primary Education, School of Education, University of Ioannina, Ioannina, Greece

    Sofia Tsokani & Dimitris Mavridis

  2. Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Toronto, ON, Canada

    Areti Angeliki Veroniki

  3. Department of Surgery and Cancer, Faculty of Medicine, Institute of Reproductive and Developmental Biology, Imperial College, London, UK

    Areti Angeliki Veroniki

  4. Institute of Medical Biometry and Statistics, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany

    Gerta Rücker

  5. Paris Descartes University, Sorbonne Paris Cité, Faculté de Médecine, Paris, France

    Dimitris Mavridis

  6. Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK

    Yemisi Takwoingi

  7. NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK

    Yemisi Takwoingi

Authors
  1. Areti Angeliki Veroniki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sofia Tsokani
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gerta Rücker
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dimitris Mavridis
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yemisi Takwoingi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Areti Angeliki Veroniki .

Editor information

Editors and Affiliations

  1. Department of Hygiene and Epidemiology University of Ioannina Medical School, Department of Epidemiology and Biostatistics Imperial College London London, UK, Ioannina, Greece

    Evangelos Evangelou

  2. Knowledge Translation Program Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Department of Surgery and Cancer Faculty of Medicine Institute of Reproductive and Developmental Biology Imperial College London, UK ISSN, Toronto, ON, Canada

    Areti Angeliki Veroniki

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Science+Business Media, LLC, part of Springer Nature

About this protocol

Check for updates. Verify currency and authenticity via CrossMark

Cite this protocol

Veroniki, A.A., Tsokani, S., Rücker, G., Mavridis, D., Takwoingi, Y. (2022). Challenges in Comparative Meta-Analysis of the Accuracy of Multiple Diagnostic Tests. In: Evangelou, E., Veroniki, A.A. (eds) Meta-Research. Methods in Molecular Biology, vol 2345. Humana, New York, NY. https://doi.org/10.1007/978-1-0716-1566-9_18

Download citation

  • DOI: https://doi.org/10.1007/978-1-0716-1566-9_18

  • Published:

  • Publisher Name: Humana, New York, NY

  • Print ISBN: 978-1-0716-1565-2

  • Online ISBN: 978-1-0716-1566-9

  • eBook Packages: Springer Protocols

Publish with us

Policies and ethics

Search

Navigation