Skip to main content
Log in

Risk calculators—methods, development, implementation, and validation

  • Review
  • Published:
International Journal of Colorectal Disease Aims and scope Submit manuscript

Abstract

Introduction

A surgical risk calculator (SRC) estimates the probabilities of unfavorable outcomes such as complications or death after a specific surgery. The risk estimates are based on information regarding the patient’s medical history and his current status. They are calculated using risk models derived from the analysis of data from a large number of previous patients in a similar clinical situation.

Methods

This paper discusses several aspects of the SRC development and its implementation into clinical practice: the development of the statistical risk models, their validation and software implementation, the use of the SRC output for shared decision making in clinical settings, and the evaluation of the SRC’s impact on individual patient outcomes as well as on the institution’s quality of care of the clinical institution.

Results

Probably the most elaborate SRC is the ACS NSQIP SRC. A comparable project was started by the German Society for Visceral and General Surgery (DGAV) in the framework of its Study, Documentation, and Quality Center (StuDoQ). It is relevant to consider that the transportability of a SRC from a US American to a German setting is not straightforward.

Conclusions

Risk calculators are important instruments for shared decision making between patients and doctor. Their implementation into clinical practice has to solve technical issues, and it is related to appropriate training of clinicians. There are specific study designs to evaluate the clinical impact of a SCR.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Godolphin W (2003) The role of risk communication in shared decision making. BMJ 327:692–93

    Article  PubMed  PubMed Central  Google Scholar 

  2. Tuckett D, Boulton M, Olson C, Williams A (1985) Meetings between experts. An approach to sharing ideas in medical consultations. Routledge, New York

    Google Scholar 

  3. General Medical Counsel. Consent: patients and doctors making decisions together 2008. Available from: http://www.gmc-uk.org/Consent___English_1015.pdf_48903482.pdf.

  4. Page AE (2015) Safety in surgery: the role of shared decision-making. Patient safetyin surgery 9:24

    Article  Google Scholar 

  5. Boss EF, Mehta N, Nagarajan N, Links A, Benke JR, Berger Z et al (2015) Shared decision making and choice for elective surgical care: a systematic review. Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery

  6. Stacey D, Legare F, Col NF, Bennett CL, Barry MJ, Eden KB et al (2014) Decision aids for people facing health treatment or screening decisions. The Cochrane database of systematic reviews 1:Cd001431

    PubMed  Google Scholar 

  7. Bilimoria KY, Liu Y, Paruch JL, Zhou L, Kmiecik TE, Ko CY, Cohen ME (2013) Development and evaluation of the universal ACS NSQIP surgical risk calculator:a decision aid and informed consent tool for patients and surgeons. J Am Coll Surg 217(5):833–42

    Article  PubMed  PubMed Central  Google Scholar 

  8. Silberzahn R, Uhlmann EL (2015) Many hands make tight work. Nature 526:189–191

    Article  CAS  PubMed  Google Scholar 

  9. Breiman L (2001) Random Forests. In: Machine Learning., p Nr. 45, S. 5-32

    Google Scholar 

  10. Vapnik VN (1998) Statistical learning theory. Wiley, New York

    Google Scholar 

  11. Hastie T, Tibshirani R, Friedman J (2011) The elements of statistical learning: data mining, inference, and prediction. Springer. http://link.springer.com/book/10.1007/978-0-387-77244-8.

  12. Hoerl AE, Kennard RW (1970) Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12(1):55–67

    Article  Google Scholar 

  13. Tibshirani R (1996) Regression shrinkage and selection via the Lasso. J R Stat Soc Ser B 58(1):267–288

    Google Scholar 

  14. Bernard Z (2004) Is combining classifiers better than selecting the best one, Machine learning., pp 255–273

    Google Scholar 

  15. Altman D, Royston P (2000) What do we mean by validating a prognostic model? Stat Med 19:453–473

    Article  CAS  PubMed  Google Scholar 

  16. Krug U et al (2010) Complete remission and early death after intensive chemotherapyin patients aged 60 years or older with acute myeloid leukaemia: a web-basedapplication for prediction of outcomes. Lancet 376:2000–08

    Article  CAS  PubMed  Google Scholar 

  17. Steyerberg E (2009) Clinical prediction models: a practical approach to development, validation, and updating. Springer. http://link.springer.com/book/10.1007/978-0-387-84858-7.

  18. Hosmer DW, Lemeshow S (1980) A goodness-of-fit test for the multiple logistic regression model. Communications in Statistics A10:1043–1069

    Article  Google Scholar 

  19. Brier (1950) Verification of forecasts expressed in terms of probability. Mon Weather Rev 78:1–3

    Article  Google Scholar 

  20. Christakis NA, Lamont EB (2000) Extent and determinants of error in doctors’ prognoses in terminally ill patients: prospective cohort study. BMJ 320(7233):469–72

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. Wegwarth O, Schwartz LM, Woloshin S, Gaissmaier W, Gigerenzer G (2012) Do physicians understand cancer screening statistics? A national survey of primary care physicians in the United States. Ann Intern Med 156(5):340–9

    Article  PubMed  Google Scholar 

  22. Zikmund-Fisher BJ, Fagerlin A, Ubel PA (2010) Risky feelings: why a 6% risk of cancerdoes not always feel like 6%. Patient Educ Couns 81(Suppl):S87–93

    Article  PubMed  Google Scholar 

  23. Zikmund-Fisher BJ (2013) The right tool is what they need, not what we have: ataxonomy of appropriate levels of precision in patient risk communication. Med Care Res Rev 70(1 Suppl):37S–49S

    Article  PubMed  Google Scholar 

  24. Bodemer N, Meder B, Gigerenzer G (2014) Communicating relative risk changes with baseline risk: presentation format and numeracy matter. Med Decis Making 6;34(5):615–626

    Article  Google Scholar 

  25. Gentles SJ, Stacey D, Bennett C, Alshurafa M, Walter SD (2013) Factors explaining the heterogeneity of effects of patient decision aids on knowledge of outcome probabilities: a systematic review sub-analysis. Systematic Reviews 2:95

    Article  PubMed  PubMed Central  Google Scholar 

  26. MRC (2008) http://www.mrc.ac.uk/documents/pdf/complex-interventions-guidance/.

  27. Moore GF, Suzanne A, Mary B, Lyndal B, Chris B, Wendy H, Laurence M, Alicia O’C, Tannaze T, Daniel W, Janis B (2015) Process evaluation of complex interventions: Medical Research Council guidance. BMJ 350:h1258

    Article  PubMed  PubMed Central  Google Scholar 

  28. Murphy AW, Esterman A, Pilotto LS (2006) Cluster randomized controlled trials inprimary care: an introduction. Eur J Gen Pract 12(2):70–3

    Article  PubMed  Google Scholar 

  29. Cohen ME, Bilimoria KY, Ko CY, Hall BL (2009) Development of an American Collegeof Surgeons National Surgery Quality Improvement Program: morbidity and mortalityrisk calculator for colorectal surgery. J Am Coll Surg 208(6):1009–16

    Article  PubMed  Google Scholar 

  30. Kim JC, Yu CS, Lim SB, Kim CW, Park IJ, Yoon YS (2015) Outcomes of ultra-low anteriorresection combined with or without intersphincteric resection in lower rectal cancer patients. Int J Colorectal Dis 30(10):1311–21

    Article  PubMed  Google Scholar 

  31. Lee DS, Ryu JA, Chung CR, Yang J, Jeon K, Suh GY, Lee WY, Park CM (2015) Risk factors foracquisition of multidrug-resistant bacteria in patients with anastomotic leakage aftercolorectal cancer surgery. Int J Colorectal Dis 30(4):497–504

    Article  PubMed  Google Scholar 

  32. Ebinger SM, Warschkow R, Tarantino I, Schmied BM, Marti L (2015) Anastomotic leakageafter curative rectal cancer resection has no impact on long-term survival: a propensity scoreanalysis. Int J Colorectal Dis 30(12):1667–75

    Article  PubMed  Google Scholar 

  33. Rothwell PM, Mehta Z, Howard SC, Gutnikov SA, Warlow CP (2005) From subgroupsto individuals: general principles and the example of carotid endarterectomy. Lancet 365:256–65

    Article  PubMed  Google Scholar 

  34. Green JA, Gonzaga AM, Cohen ED, Spagnoletti CL (2014) Addressing health literacy through clear health communication: a training program for internal medicine residents. Patient Educ Couns 95(1):76–82

    Article  PubMed  Google Scholar 

  35. Justice AC (2000) Assessing the generalizability of prognostic information. Ann Intern Med 1999(130):515–524

    Google Scholar 

  36. Nicholl J (2007) Case-mix adjustment in non-randomized observational evaluations:the constant risk fallacy. J Epidemiol Community Health 61:1010–1013

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ulrich Mansmann.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mansmann, U., Rieger, A., Strahwald, B. et al. Risk calculators—methods, development, implementation, and validation. Int J Colorectal Dis 31, 1111–1116 (2016). https://doi.org/10.1007/s00384-016-2589-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00384-016-2589-3

Keywords

Navigation