Skip to main content
Log in

A Machine-Learning Algorithm to Optimise Automated Adverse Drug Reaction Detection from Clinical Coding

  • Short Communication
  • Published:
Drug Safety Aims and scope Submit manuscript

A Correction to this article was published on 15 April 2019

This article has been updated

Abstract

Introduction

Adverse drug reaction (ADR) detection in hospitals is heavily reliant on spontaneous reporting by clinical staff, with studies in the literature pointing to high rates of underreporting [1]. International Classification of Diseases, 10th Revision (ICD-10) codes have been used in epidemiological studies of ADRs and offer the potential for automated ADR detection systems.

Objective

The aim of this study was to develop an automated ADR detection system based on ICD-10 codes, using machine-learning algorithms to improve accuracy and efficiency.

Methods

For a 12-month period from December 2016 to November 2017, every inpatient episode receiving an ICD-10 code in the range Y40.0–Y59.9 (ADR code) was flagged for review as a potential ADR. Each flagged admission was assessed by an expert pharmacist and, if needed, reviewed at regular ADR committee meetings. For each report, a determination was made about ADR probability and severity. The dataset was randomly split into training and test sets. A machine-learning model using the random forest algorithm was developed on the training set to discriminate between true and false ADR reports. The model was then applied to the test set to assess accuracy using the area under the receiver operating characteristic (AUC).

Results

In the study period, 2917 Y40.0–Y59.9 codes were applied to admissions, resulting in 245 ADR reports after review. These 245 reports accounted for 44.5% of all ADR reporting in our hospital in the study period. A random forest model built on the training set was able to discriminate between true and false reports on the test set with an AUC of 0.803.

Conclusions

Automated ADR detection using ICD-10 coding significantly improved ADR detection in the study period, with improved discrimination between true and false reports by applying a machine-learning model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Change history

  • 15 April 2019

    CK: Principal Investigator. CM and CK were responsible for the study design and conception; all authors were responsible for acquisition and validation of the data; CM was responsible for analysis and interpretation of the data; and all authors contributed to reviewing drafts of the manuscript and approved the final version for publication.

References

  1. Ackroyd-Stolarz S, Hartnell N, MacKinnon NJ. Demystifying medication safety: making sense of the terminology. Res Soc Adm Pharm. 2006;2(2):280–9.

    Article  Google Scholar 

  2. Naranjo, et al. A method for estimating the probability of adverse drug reactions. Clin Pharmacol Ther. 1981(2);30:661–4.

    Article  Google Scholar 

  3. Hazell L, Shakir SAW. Under-reporting of adverse: a systematic review. Drug Saf. 2006;29(5):385–96.

    Article  PubMed  Google Scholar 

  4. Mirbaha F, Shalviri G, Yazdizadeh B, Gholami K, Majdzadeh R. Perceived barriers to reporting adverse drug events in hospitals: a qualitative study using theoretical domains framework approach. Implement Sci. 2015;10(1):1–10.

    Article  Google Scholar 

  5. Bakhsh T, Al-Ghamdi M, Bawazir S, Qureshi N. Barriers, facilitators, strategies, and predictors for reporting adverse drug reactions in three general hospitals in Jeddah, 2013. Br J Med Med Res. 2016;17(4):1–13.

    Article  Google Scholar 

  6. Falconer N, Barras M, Cottrell N. Systematic review of predictive risk models for adverse drug events in hospitalised patients. Br J Clin Pharmacol. 2018;84(5):846–64.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Hohl CM, Karpov A, Reddekopp L, Stausberg J. ICD-10 codes used to identify adverse drug events in administrative data: a systematic review. J Am Med Inform Assoc. 2014;21(3):547–57.

    Article  PubMed  Google Scholar 

  8. World Health Organization. Drugs, medicaments and biological substances causing adverse effects in therapeutic use (Y40-Y59). 2016 [cited 28 November 2018]. http://apps.who.int/classifications/icd10/browse/2016/en#/Y40-Y59. Accessed 28 Nov 2018

  9. Du W, Pearson S-A, Buckley N, Day C, Banks E. Diagnosis-based and external cause-based criteria to identify adverse drug reactions in hospital ICD-coded data: application to an Australia population-based study. Public Heal Res Pract. 2017;27(2):1–6.

    Google Scholar 

  10. Friedman J, Hastie T, Tibshirani R. Regularization paths for generalized linear models via coordinate descent. J Stat Softw. 2010;33(1):128–9.

    Article  Google Scholar 

  11. Karatzoglou A, Meyer D, Hornik K. Support Vector Machines in R. J Stat Softw. 2006;15(9):28.

    Article  Google Scholar 

  12. James G, Witten D, Hastie T, Tibshirani R. An introduction to statistical learning. Berlin: Springer; 2013.

    Book  Google Scholar 

  13. Kubat M, Matwin S, Rosario GE, Rundensteiner EA, Brown DC, Ward MO, et al. Addressing the curse of imbalanced training sets: one-sided selection. Nashville, USA; 1997. p. 179–86. https://arxiv.org/pdf/1609.06570.pdf

  14. Youden WJ. Index for rating diagnostic tests. Cancer. 1950;3(1):32–5.

    Article  CAS  PubMed  Google Scholar 

  15. R Core Team. R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna. 2018 [cited 28 November 2018]. https://www.R-project.org/. Accessed 28 Nov 2018

  16. Meyer D, Dimitriadou E, Hornik K, Weingessel A, Leisch F. e1071: Misc functions of the department of statistics, probability theory group (Formerly: E1071), TU Wien. R package version 1.7-0. 2018 [cited 28 November 2018]. https://CRAN.R-project.org/package=e1071. Accessed 28 Nov 2018

  17. Liaw A, Wiener M. Classification and Regression by randomForest. R News. 2002;2(3):18–22.

    Google Scholar 

  18. Allaire J, Chollet F. keras: R Interface to ‘Keras’. R package version 2.2.4. 2018 [cited 28 November 2018]. https://CRAN.R-project.org/packag=keras. Accessed 28 Nov 2018

  19. Abadi M, et al. TensorFlow: large-scale machine learning on heterogeneous systems, 2015. Software available from http://tensorflow.org. Accessed 28 Nov 2018

  20. Pennington J, Socher R, Manning CD. GloVe: global vectors for word representation. 2014. https://nlp.stanford.edu/pubs/glove.pdf. Accessed 28 Nov 2018

  21. Provisional approval pathway: prescription medicines. Therapeutic goods administration. 2018 [cited 28 November 2018]. https://www.tga.gov.au/provisional-approval-pathway-prescription-medicines. Accessed 28 Nov 2018

  22. Linger M, Martin J. Pharmacovigilance and expedited drug approvals. Aust Prescr. 2018;41(2):50–3.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Contributions

CK: Principal Investigator. CM and CK were responsible for the study design and conception; all authors were responsible for acquisition and validation of the data; CM was responsible for analysis and interpretation of the data; and all authors contributed to reviewing drafts of the manuscript and approved the final version for publication.

Corresponding author

Correspondence to Christopher McMaster.

Ethics declarations

Funding

No sources of funding were used to assist in the preparation of this study.

Conflict of interest

Christopher McMaster, David Liew, Claire Keith, Parnaz Aminian and Albert Frauman have no conflicts of interest that are directly relevant to the content of this study.

Ethical approval

This study was approved by the institutional Human Research Ethics Committee.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

McMaster, C., Liew, D., Keith, C. et al. A Machine-Learning Algorithm to Optimise Automated Adverse Drug Reaction Detection from Clinical Coding. Drug Saf 42, 721–725 (2019). https://doi.org/10.1007/s40264-018-00794-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40264-018-00794-y

Navigation