Abstract
Diagnostic testing is used by biosecurity officers for the detection and identification of plant and animal pathogens, often informing high-consequence decisions such as restricting the entry of trade goods. It is rare that such tests can be considered gold standards; however, uncertainty can be reduced by using the results of other tests, measuring performance on samples of known status and incorporating prior knowledge from expert judgement. This article presents an extension to the methods of Joseph et al. (Am J Epidemiol 141:263–272, 1995), and Dendukuri and Joseph (Biometrics 57:158–167, 2001) for Bayesian estimation in the absence of a gold standard test, which allows for the use of incomplete test data. This extension is demonstrated with a novel application: the case study of myrtle rust from Holliday et al. (Plant Dis 97:828–834, 2013), which involves samples from potential biosecurity risk material on importation pathways to Australia. The samples were tested at two laboratories, and prior estimates for pathway prevalence were obtained by expert elicitation. The Bayesian estimation was based on a model with and without covariances for the test results to assess the assumption of conditional independence. The results show that pathogen prevalence, diagnostic sensitivity and diagnostic specificity can be estimated using all available data even where some samples have been subject to only one of two available tests. The results also indicate the importance of consideration of the assumption of conditional independence. The findings enable diagnostic testing laboratories and decision makers to make use of all test results and to explicitly incorporate prior knowledge to estimate pathogen prevalence and test accuracy.
Similar content being viewed by others
References
Andersen S (1997) Re: “Bayesian estimation of disease prevalence and the parameters of diagnostic tests in the absence of a gold standard”. American Journal of Epidemiology 145:290–291
Aven T (2013) On the meaning and use of the risk appetite concept. Risk Analysis 33:462–468
Beavers DP, Stamey JD (2012) Bayesian sample size determination for binary regression with a misclassified covariate and no gold standard. Computation Statistics and Data Analysis 56:2574–2582
Black MA, Craig BA (2002) Estimating disease prevalence in the absence of a gold standard. Statistics in Medicine 21:2653–2669
Burgman MA, McBride M, Ashton R, Speirs-Bridge A, Flander L, Wintle B, Fidler F, Rumpff L, Twardy C (2011) Expert status and performance. PLoS ONE 6:e22,998
Carnegie AJ, Lidbetter JR, Walker J, Horwood MA, Tesoriero L, Glen M, Priest MJ (2010) Uredo rangelii, a taxon in the guava rust complex, newly recorded on Myrtaceae in Australia. Australasian Plant Pathology 39:463–466
Casella G, George EI (1992) Explaining the Gibbs Sampler. The American Statistician 46:167–174
Coutinho TA, Wingfield MJ, Alfenas AC, Crous PW (1998) Eucalyptus rust: a disease with the potential for serious international implications. Plant Disease 82:819–825
Demissie K, White N, Joseph L, Ernst P (1998) Bayesian estimation of asthma prevalence, and comparison of exercise and questionnaire diagnostics in the absence of a gold standard. Annals of Epidemiology 8:201–208
Dendukuri N, Joseph L (2001) Bayesian approaches to modeling the conditional dependence between multiple diagnostic tests. Biometrics 57:158–167
Dongen SV (2006) Prior specification in Bayesian statistics: three cautionary tales. Journal of Theoretical Biology 242:90–100
Dorny P, Phiri IK, Vercruysse J, Gabriel S, Willingham AL, Brandt J, Victor B, Speybroeck N, Berkvens D (2004) A Bayesian approach for estimating values for prevalence and diagnostic test characteristics of porcine cysticercosis. International Journal for Parasitology 34:569–576
Dunson DB (2001) Commentary: practical advantages of Bayesian analysis of epidemiologic data. American Journal of Epidemiology 153:1222–1226
Enoe C, Georgiadis MP, Johnson WO (2000) Estimation of sensitivity and specificity of diagnostic tests and disease prevalence when the true disease state is unknown. Preventive Veterinary Medicine 45:61–81
Epstein LD, Munoz A, He D (1996) Bayesian imputation of predictive values when covariate information is available and gold standard diagnosis is unavailable. Statistics in Medicine 15:463–476
Ferson S, Ginzburg LR (1996) Different methods are needed to propagate ignorance and variability. Reliability Engineering & System Safety 54(2):133–144
Garber A, Solomon N (1999) Cost-effectiveness of alternative test strategies for the diagnosis of coronary artery disease. Annals of Internal Medicine 130:719–728
Geisser S, Johnson W (1992) Optimal administration of dual screening tests for detecting a characteristic with special reference to low prevalence diseases. Biometrics 48:839–852
Georgiadis MP, Johnson WO, Gardner IA, Singh R (2003) Correlation-adjusted estimation of sensitivity and specificity of two diagnostic tests. Journal of the Royal Statistical Society: Series C (Applied Statistics) 52(1):63–76
Glen M, Alfenas AC, Zauza EAV, Wingfield MJ, Mohammed C (2007) Puccinia psidii: a threat to the Australian environment and economy - a review. Australasian Plant Pathology 36:1–16
Grgurinovic CA, Walsh D, Macbeth F (2006) Eucalyptus rust caused by Puccinia psidii and the threat it poses to Australia. EPPO Bulletin 36:486–489
de Groot JAH, Dendukuri N, Janssen KJM, Reitsma JB, Brophy J, Joseph L, Bossuyt PMM, Moons KGM (2012) Adjusting for partial verification or workup bias in meta-analyses of diagnostic accuracy studies. American Journal of Epidemiology 174:847–853
Hanson T, Johnson WO, Gardner IA (2003) Hierarchical models for estimating herd prevalence and test accuracy in the absence of a gold standard. Journal of agricultural, biological, and environmental statistics 8(2):223–239
Holliday JL, Jones SA, Simpson JA, Glen M, Edwards J, Robinson A, Burgman MA (2013) A novel spore collection device for sampling exposure pathways: a case study of Puccinia psidii. Plant Disease 97:828–834
Johnson WO, Gastwirth JL, Pearson LM (2001) Screening without a “Gold Standard”: the Hui-Walter paradigm revisited. American Journal of Epidemiology 153:921–924
Joseph L (1997) Re: “Bayesian estimation of disease prevalence and the parameters of diagnostic tests in the absence of a gold standard” The first author replies. American Journal of Epidemiology 145:291
Joseph L, Gyorkos TW, Coupal L (1995) Bayesian estimation of disease prevalence and the parameters of diagnostic tests in the absence of a gold standard. American Journal of Epidemiology 141:263–272
Kawanishi T, Uematsu S, Kakishima M, Kagiwada S, Hamamoto H, Horie H, Namba S (2009) First report of rust disease on ohia and the causal fungus, Puccinia psidii, in Japan. Journal of General Plant Pathology 75:428–431
Keefer DL, Bodily SE (1983) Three-point approximations for continuous random variables. Management Science 29:595–609
Krull CR, Waipara NW, Choquenot D, Burns BR, Gormley AM, Stanley MC (2013) Absence of evidence is not evidence of absence: feral pigs as vectors of soil-borne pathogens. Austral Ecology 38:534–542
Langrell SRH, Glen M, Alfenas AC (2008) Molecular diagnosis of Puccinia psidii (guava rust) - a quarantine threat to Australian eucalypt and Myrtaceae biodiversity. Plant Pathology 57:687–701
Lew RA, Levy PS (1989) Estimation of prevalence on the basis of screening tests. Statistics in Medicine 8(10):1225–1230
Lewis F, Sanchez-Vanquez MJ, Torgerson PR (2012) Association between covariates and disease occurrence in the presence of diagnostic error. Epidemiology and Infection 140:1515–1524
Liu J, Chen F, Yu H, Zeng P, Liu L (2014) A two-stage Bayesian method for estimating accuracy and disease prevalence for two dependent dichotomous screening tests when the status of individuals who are negative on both tests is unverified. BMC medical research methodology 14(1):110
Lu Y, Dendukuri N, Schiller I, Joseph L (2010) A Bayesian approach to simultaneously adjusting for verification and reference standard bias in diagnostic test studies. Statistics in Medicine 29:2532–2543
Makowski D, Denis J-B, Ruck L, Penaud A (2008) A Bayesian approach to assess the accuracy of a diagnostic test based on plant disease measurement. Crop Protection 27:1187–1193
Martin TG, Kuhnert PM, Mengersen K, Possingham HP (2005) The power of expert opinion in ecological models using Bayesian methods: impact of grazing on birds. Journal of Applied Ecology 15:266–280
McBride MF, Fidler F, Burgman MA (2012) Evaluating the accuracy and calibration of expert predictions under uncertainty: predicting the outcomes of ecological research. Diversity and Distributions 18:782–794
Mendoza-Blanco JR, Tu XM, Iyengar S (1996) A Bayesian inference on prevalence using a missing-data approach with simulation-based techniques: applications to HIV screening. Statistics in Medicine 15:2161–2176
Mila AL, Yang XB, Carriquiry AL (2003) Bayesian logistic regression of soybean Sclerotinia stem rot prevalence in the U.S. north central region: accounting for uncertainty in parameter estimation. Phytopathology 93:758–764
Mireku E, Simpson JA (2002) Fungal and nematode threats to Australian forests and amenity trees from importation of wood and wood products. Canadian Journal of Plant Pathology 24:117–124
Morris WK, Vesk PA, McCarthy MA (2013) Profiting from pilot studies: analysing mortality using Bayesian models with informative priors. Basic and Applied Ecology 14:81–89
Nauta MJ (2000) Separation of uncertainty and variability in quantitative microbial risk assessment models. International Journal of Food Microbiology 57(1):9–18
Pennello GA (2011) Bayesian analysis of diagnostic test accuracy when disease state is unverified for some subjects. Journal of Biopharmaceutical Statistics 21:954–970
Pollino CA, White AK, Hart BT (2007) Examination of conflicts and improved strategies for the management of an endangered Eucalypt species using Bayesian networks. Ecological Modelling 201:37–59
Poole WK, Flynn PM, Rao AV, Cooley PC (1996) Estimating the prevalence of drug use from self-reports in a cohort for which biologic data are available for a subsample. American Journal of Epidemiology 144:413–420
Rahme E, Joseph L (1998) Estimating the prevalence of a rare disease: adjusted maximum likelihood. Journal of the Royal Statistical Society: Series D (The Statistician) 47(1):149–158
Reitsma JB, Rutjes AW, Khan KS, Coomarasamy A, Bossuyt PM (2009) A review of solutions for diagnostic accuracy studies with an imperfect or missing reference standard. Journal of clinical epidemiology 62(8):797–806
Rubin DB (1988) Using the SIR algorithm to simulate posterior distributions. Bayesian statistics 3(1):395–402
Schiffman M, Wheeler CM, Dasgupta A, Solomon D, Castle PE (2005) A comparison of a prototype PCR assay and Hybrid Capture 2 for detection of carcinogenic human papillomavirus DNA in women with equivocal or mildly abnormal papanicolaou smears. American Journal of Clinical Pathology 124:722–732
Scott AN, Joseph L, Belisle P, Behr MA, Schwartzman K (2008) Bayesian modelling of tuberculosis clustering from DNA fingerprint data. Statistics in Medicine 27:140–156
Simpson JA, Thomas K, Grgurinovic CA (2006) Uredinales species pathogenic on species of Myrtaceae. Australasian Plant Patholology 35:549–562
Speirs-Bridge A, Fidler F, McBride M, Flander L, Cumming G, Burgman M (2010) Reducing overconfidence in the interval judgments of experts. Risk Analysis 30:512–523
Spiegelhalter DJ, Best NG (2003) Bayesian approaches to multiple sources of evidence and uncertainty in complex cost-effectiveness modelling. Statistics in Medicine 22(23):3687–3709
Stubner S (2002) Enumeration of 16S rDNA of Desulfotomaculum lineage 1 in rice field soil by real-time PCR with SybrGreen\(_{{\rm TM}}\) detection. Journal of Microbiological Methods 50:155–164
Toft N, Jørgensen E, (2005) Diagnosing diagnostic tests: evaluating the assumptions underlying the estimation of sensitivity and specificity in the absence of a gold standard. Preventive veterinary medicine 68(1):19–33
Tommerup IC, Alfenas AC, Old KM (2003) Guava rust in Brazil - a threat to eucalyptus and other Myrtaceae. New Zealand Journal of Science 33:420–428
Tu XM, Kowalski J, Jia G (1999) Bayesian analysis of prevalence with covariates using simulation-based techniques: applications to HIV screening. Statistics in Medicine 18:3059–3073
Vettraino AM, Sukno S, Vannini A, Garbelotto M (2010) Diagnostic sensitivity and specificity of different methods used by two laboratories for the detection of Phytophthora ramorum on multiple natural hosts. Plant Pathology 59:289–300
Vidal E, Moreno A, Berolini E, Cambra M (2012) Estimation of the accuracy of two diagnostic methods for the detection of Plum pox virus in nursery blocks by latent class models. Plant Pathology 61:413–422
Vijan S, Hwang EW, Hofer TP, Hayward RA (2001) Which colon cancer screening test? a comparison of costs, effectiveness, and compliance. American Journal of Medicine 111:593–601
Vlems FA, Ladanyi A, Gertler R, Rosenberg R, Diepstra JHS, Rder C, Nekarda H, Molnar B, Tulassay Z, van Muijen GNP, Vogel I (2003) Reliability of quantitative reverse-transcriptase-PCR-based detection of tumour cells in the blood between different laboratories using a standardised protocol. European Journal of Cancer 39:388–396
Acknowledgments
This study was supported by the Australian Centre of Excellence for Risk Analysis (now named the Centre of Excellence for Biosecurity Risk Analysis). We sincerely thank the experts who participated in the elicitation of priors, the biosecurity officers and volunteers who facilitated the collection of samples, and the scientists who performed the laboratory testing. We are also very grateful to Andrew Robinson, Bonnie Wintle, Marissa McBride and Mark Burgman for their advice. Finally, we appreciate the time taken by two anonymous reviewers for providing detailed comments which have greatly improved the manuscript.
Author information
Authors and Affiliations
Corresponding author
Conditional Distributions for Gibbs Sampler
Conditional Distributions for Gibbs Sampler
1.1 Assuming Conditional Independence
Letting \(Y_{\cdot }=\sum Y_{i}\),
1.2 Allowing for Conditional Independence
The following posterior distributions do not have a known form so sampling importance resampling (Rubin 1988) was used to sample the values at each iteration. Letting \(Y_{\cdot }=\sum Y_{i}, u_s=min(S_1,S_2)-S_1S_2\) and \(u_c=min(C_1,C_2)-C_1C_2\),
Rights and permissions
About this article
Cite this article
Clarke, S.J., Jones, S.A. Bayesian Estimation for Diagnostic Testing of Biosecurity Risk Material in the Absence of a Gold Standard when Test Data are Incomplete. JABES 20, 389–408 (2015). https://doi.org/10.1007/s13253-015-0214-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13253-015-0214-5