Influence of educational, audit and feedback, system based, and incentive and penalty interventions to reduce laboratory test utilization: a systematic review

Daniel M. Kobewka; Paul E. Ronksley; Jennifer A. McKay; Alan J. Forster; Carl van Walraven

doi:10.1515/cclm-2014-0778

Publicly Available Published by De Gruyter September 27, 2014

Influence of educational, audit and feedback, system based, and incentive and penalty interventions to reduce laboratory test utilization: a systematic review

Daniel M. Kobewka
Daniel Kobewka MD, FRCPC is a Master’s of science student and general internist at the Ottawa Hospital. He received his MD from the University of Alberta and completed his training in General Internal Medicine at the University of Ottawa. His research interests include quality improvement and end-of-life care.
, Paul E. Ronksley
Paul Ronksley PhD is a post-doctoral fellow in the department of clinical epidemiology at the Ottawa Hospital Research Institute. He received his PhD in Epidemiology from the University of Calgary and is currently funded by a CIHR post-doctoral fellowship award. His research interests are in chronic disease epidemiology and the use of administrative and population-based survey data to study access to care and health care utilization for patients with multi-morbidity.
, Jennifer A. McKay
Jennifer McKay is a Master’s student in the Epidemiology program at the University of Ottawa. Jennifer’s research interests are quality of care, health economics and the funding dynamics of the health care system.
, Alan J. Forster
Alan J. Forster MD, FRCPC, MSc is a general internist and Chief Quality and Performance Officer at the Ottawa Hospital. He is a Professor of Medicine at the University of Ottawa and Senior Scientist in the Clinical Epidemiology Program at the Ottawa Hospital Research Institute. His research focuses on patient safety and quality improvement. He has received several prestigious awards recognizing his work within the field of health services research, including a Career Scientist Award with the Ontario Ministry of Health and Long-Term Care and an Early Research Award and Ontario Research Fund grant from the Ontario Government’s Ministry of Research and Innovation. He has performed seminal work evaluating the incidence of adverse events following discharge from hospital.
and Carl van Walraven
Carl van Walraven MD, FRCPC, MSc is a Senior Scientist with the Ottawa Hospital Research Institute (OHRI) and is the Site Director for the Institute for Clinical Evaluative Sciences uOttawa Site. He is a general internist at the Ottawa Hospital and is a Professor in the Faculties of Medicine and Epidemiology and Community Medicine at the University of Ottawa. Carl’s research focuses on quality improvement, continuity of care, and administrative database research. He has over 150 publications in peer-reviewed journals. Carl has also been the recipient of several prestigious awards recognizing his work within the field of health services research, including the Ontario Ministry of Health Career Scientist Award and the Premier Research Excellence Award.

From the journal Clinical Chemistry and Laboratory Medicine (CCLM)

https://doi.org/10.1515/cclm-2014-0778

Abstract

Laboratory and radiographic tests are often ordered unnecessarily. This excess testing has financial costs and is a burden on patients. We performed a systematic review to determine the effectiveness interventions to reduce test utilization by physicians. The MEDLINE and EMBASE databases were searched for the years 1946 through to September 2013 for English articles that had themes of test utilization and cost containment or optimization. Bibliographies of included papers were scanned to identify other potentially relevant studies. Our search resulted in 3236 articles of which 109 met the inclusion criteria of having an intervention aimed at reducing test utilization with results that could be expressed as a percent reduction in test use relative to the comparator. Each intervention was categorized into one or more non-exclusive category of education, audit and feedback, system based, or incentive or penalty. A rating of study quality was also performed. The percent reductions in test use ranged from a 99.7% reduction to a 27.7% increase in test use. Each category of intervention was effective in reducing test utilization. Heterogeneity between interventions, poor study quality, and limited time horizons makes generalizations difficult and calls into question the validity of results. Very few studies measure any patient safety or quality of care outcomes affected by reduced test use. There are numerous studies that use low investment strategies to reduce test utilization with one time changes in the ordering system. These low investment strategies are the most promising for achievable and durable reductions in inappropriate test use.

Keywords: audit and feedback; education; laboratory utilization; reducing; test use

Introduction

Healthcare spending in has increased in both relative and absolute terms over the past several decades [1, 2]. This increase has been seen in many countries and will be unsustainable if the trend continues at the present rate [3]. Laboratory and radiographic testing is a promising target for reducing spending because many of the tests ordered by physicians are suspected to be unnecessary; in some studies, 95% of tests performed are inappropriate as judged by criteria of redundancy or their probability of adding value to patient care [4].

The perceived prevalence of inappropriate laboratory utilization has prompted many attempts to reduce test utilization. Several systematic and narrative reviews have been published on this topic [5–8]. The most recent review, published in 1998, used a behavioral framework to classify interventions and found that targeting multiple behavioral factors was more successful than targeting a single factor [8]. However, this review only dichotomized study outcomes by statistical significance, making it difficult to understand the utility of these interventions. Other reviews used categories of education, audit and feedback or other but were unable to make generalizations about which strategy is most effective because of differences within each category of intervention, a wide range of effects, and lack of a common measure [5, 6]. To date, no review has quantitatively compared the influence of various interventions on test utilization reduction. In addition, previous reviews have not examined investments required to implement these interventions. For these reasons, we performed a systematic review to determine the effectiveness of all interventions to reduce test utilization by physicians.

Methods

Data sources and searches

We performed a systematic review following a predetermined protocol in accordance with published guidelines for reporting of systematic reviews of randomized controlled trials [9]. We identified all potentially relevant articles published in English by searching Medline (1946 through September 2013) and Embase (1947 through September 2013). Searches were enhanced by systematically scanning bibliographies of identified articles and relevant review articles as well as articles deemed by PubMed to be related to the included studies. To search electronic databases, we used the strategy recommended for systematic reviews of interventional studies [10] and specified two comprehensive search themes (Online-Appendix 1). Theme 1 identified relevant terms related to laboratory utilization. Theme 2 related to optimization and cost containment. We then combined the two datasets using the Boolean operator ‘and’ and limited the intersection to human studies.

Study selection

Relevant articles were selected from the retrieved studies using a two-phase process. Abstracts were first reviewed for eligibility by three researchers (DK, PR, JM). All abstracts reporting on the effect of an intervention on laboratory utilization were selected for full text review. This initial stage was intentionally liberal; we discarded only abstracts that clearly were not reporting inventions aimed at reducing test utilization. Full text articles were then assessed by one reviewer (DK) with verification by two reviewers (PR, JM) to determine if the study met the specified inclusion criteria. Inclusion criteria included: 1) study population had to include physicians; 2) the aim of the intervention was the modification of test utilization; 3) a comparator arm (either standard care or no intervention) was needed; and 4) the study had to quantify laboratory test utilization with and without the intervention so that a percent change could be calculated. All study designs were considered within this review.

Intervention classification

Interventions used to effect physician laboratory ordering practices were categorized into one or more of the following non-exclusive categories:

Educational interventions in which appropriate test ordering (including the distribution of guidelines) was taught to physicians;
Audit and feedback interventions in which physicians were presented their test utilization compared to their previous utilization or peer utilization or the total costs of the tests they ordered;
System-based interventions involving one-time, permanent changes to test ordering processes including: order form modifications; computer order entry systems with rules disallowing test ordering in specific circumstances; and clinical decision support systems (CDSS) in which an interactive computer system forces physicians to integrate previous knowledge about the patient and/or the medical literature into the test ordering process; and
Incentive or penalty interventions in which physicians received rewards or punishments for certain test ordering practices. For each intervention, we also determined whether the targeted physicians were actively engaged during the intervention’s development [8].

Data extraction and quality assessment

For each study, we extracted information on publication year, country of origin, study design (randomized two-arm trials, before and after studies, prospective cohort), the healthcare providers targeted and the tests targeted for reduction. We recorded characteristics of the study intervention(s) along with its duration. We assessed study quality using an adapted version of the Effective Practice and Organization of Care (EPOC) Cochrane guidelines for interventional studies [11]. In particular, we assessed whether: patients and providers were similar across intervention and comparator groups; a randomized control group was used within the study, there was sufficient detail to describe the intervention, there was risk of contamination between the intervention and comparator groups, and whether a time-series analysis was performed. Lastly, we assessed whether studies measured any patient safety outcome that could be affected by reducing test utilization.

Statistical analysis

The unit of analysis was the intervention, with some studies having more than one intervention. The effectiveness of interventions was summarized using the percent relative reduction in test volume for the intervention group relative to the comparator group. This was calculated as:

(# of tests in comparator group)−(# of tests in intervention group)(# of tests in comparator group)

Given the variability in the types of interventions, duration of interventions, targeted tests, and patient populations, we did not conduct a meta-analysis. Instead, a descriptive analysis of the relative reductions was performed using medians and inter-quartile ranges (IQR) to summarize effects across the various intervention types. We also documented when these reductions in a particular study were statistically significant. Results were stratified by categories of interventions (educational, system based, audit/feedback and incentive and penalty) and were visualized using box plots. Stratifications were then conducted by additional interventional factors within each category and by measures of study quality and statistical significance. The effect of physician involvement was also measured across interventions. Finally sensitivity analyses were conducted to assess the effect of interventions that targeted four or more tests.

In order to highlight the interventions with the best return on investment we included a narrative review of interventions with the largest reductions in test utilization as well as the most effective system-based interventions targeting four or more tests. All analyses were performed using Stata version 13.0 (Stata Corp., College Station, TX, USA).

Results

Our search generated 3236 unique citations. Following the initial screening and full-text review, 109 unique studies that included 119 interventions were included in the systematic review (Figure 1) [12–121]. Study characteristics and brief descriptions of each intervention are shown in Table 1. Ninety-three studies (85.3%) used before-after designs, 2 (1.8%) were non-randomized prospective trials, and 14 (12.8%) were randomized controlled trials. Studies were published between 1974 and 2013 with 61 studies (56.0%) performed in North America, 27 (24.8%) in Europe, 11 (10.1%) Australasia, and 10 (9.2%) in other regions. Studies used very limited time horizons with only 15 of 109 studies (13.8%) measuring intervention effects beyond 1 year.

Figure 1

Selection of articles in systematic review.

The search was performed in September of 2013.

Table 1

Study characteristics and description of interventions.

Author Year	Country	Healthcare workers targeted	Site	Intervention	Comparator	Category of Intervention				Targeted tests	Relative reduction of targeted tests
						E	S	A	I
Amukele 2011	USA	Physicians working in surgery, hematology, and internal medicine	Single teaching hospital	The order form was modified with some tests being bundled and others being unbundled. Physicians who were high users were educated about coagulation tests and an algorithm was distributed	Before-after	X	X			PT, PTT, TT and fibrinogen	77.8
Archambault 2011	USA	All physicians	Single teaching hospital	ESR testing was no longer performed	Before-after		X			ESR and CRP	47.5
Attali 2006	Israel	Resident physicians	Single teaching hospital	Educational lecture about excessive and inappropriate testing; tests were unbundled on the request form; a senior physician gave feedback about ordering practices	Before-after	X	X	X		All blood tests	34.5^a
Barazzoni 2002	Switzerland	All physicians seeing patients pre-operatively for elective procedures	6 community hospitals with surgery facilities	All stakeholders were involved in a guideline creation process. The guidelines were then adopted as hospital policy	Before-after	X				All pre-operative tests	26.1^a
Bareford 1990	USA	All medical staff: consultants and junior house staff	Single community hospital	Monthly usage statements were distributed to physicians. Guidelines on appropriate testing were distributed and some tests were canceled if inappropriate	Before-after	X	X	X		CBC, ESR and PT	62.1
Baricchi 2012	Italy	General practitioners	4 primary care clinics	Testing algorithms for 7 common clinical scenarios were developed. Physicians were then educated about the algorithms and told to use them	Before-after	X	X			All blood tests	4.7
Barie 1995	USA	All medical staff in the ICU	Single teaching hospital	Testing pathways were introduced and audit and feedback was done monthly	Before-after	X		X		All blood tests and chest radiographs	34.6
Bates 1999	USA	All physicians	Single teaching hospital	Definitions of a redundant test were created. The computer order entry system created reminders when tests were ordered that were redundant	Randomized trial	X				Electrolytes, urinalysis, digoxin, tobramycin, aminophyline, vancomycin, and gentamycin levels, urine, sputum and stool cultures, Clostridium difficile testing, fibrin split products	0.1
Bates 1997	USA	All physicians	Single teaching hospital	Costs were displayed for each test that was ordered.	Before-after			X		All blood tests and the 35 most common radiologic tests	4.1
Berenholtz 2001	USA	Physicians working in a surgical ICU	Single teaching hospital	Clinical pathways were created for various surgical problems. Appropriate laboratory testing was suggested by the pathway	Before-after	X				All laboratories included in the pathway	–6.8
Berwick 1986	USA	35 internists	Single health management organization	Doctors met to discuss the use of particular tests and journal articles were distributed on the use of these tests Clinicians received individualized feedback on their rates of test ordering compared to their peers and their rank relative to their peers Physicians were ranked by the proportion of tests they ordered that were abnormal. Each physician was then given their rank	Before-after	X				ESR, T4, serum glucose, heterophile antibody testing, ANA, SLE preparation, RF, liver enzymes, cholesterol levels, radiographs of peripheral bones and chest, ECGs	0.2
					Before-after	X					14.2^a
					Before-after	X					–4.1
Boon-Falleur 1995	Belgium	Physicians working on liver a transplant unit	Pediatric liver unit	A rule based decision support application decided on the next day’s laboratory tests based on previous results	Before-after		X			All blood tests	24.9
Buckingham 1994	Scotland	All physicians	All hospitals in a single city	Justification for tests ordered had to be given. Specifics on how this was done are not given	Before-after		X			All biochemical blood tests	1.4
Bunting 2004	Canada	200 physicians who ordered the most tests in 1997	Single province	Feedback on test usage was given four times over a 2-year period	Randomized controlled trial			X		All blood tests	7.8^a
Burnett 1991	Australia	All physicians requesting blood work from a single medical laboratory	Single private medical laboratory	All laboratory tests were unbundled so they had to be ordered individually	Before-after		X			All blood tests	18.8^a
Calderon-Margalit 2005	Israel	All physicians	Single teaching hospital	Several interventions including mandatory consultation prior to ordering, education, restriction or elimination of various tests	Before-after	X	X			All blood tests	19.0^a
Carson 1995	USA	All physicians	Single teaching hospital	An algorithm that automatically restricted fibrin degradation product testing and instead used D-dimer testing for suspected DIC	Before-after		X			D-Dimer and fibrin degradation products	48.3^a
Carter 2002	USA	Physicians sending specimens to the lab	Single teaching hospital	All test requests were reviewed by a more senior physician	Before-after			X		All laboratories that cost >$75 USD	23.0
Chen 2003	USA	All Physicians ordering selected tests	Single teaching hospital	A reminder on the half-life of various anti-epileptic drugs was shown on the computer when repeat levels were requested in a short time period	Before-after	X	X			Anti epileptic drug levels	23.4^a
Chu 2012	Australia	Medical students and residents	Single teaching hospital	Selected tests were restricted and required a senior staff member to sign off on them prior to being ordered	Before-after		X			Coagulation tests, thyroid function tests, ESR and d-dimer	7.5^a
Chu 1996	USA	Physicians taking care of trauma patients	Single teaching hospital	A liberal admission algorithm with extensive testing was compared to physicians only ordering tests they thought necessary	Before-after		X			Hematology, chemistry and coagulation profile, urinalysis, blood gas, blood alcohol level, type and screen	68.3
Cohen 1982	USA	All physicians working on the teaching units	4 clinical teaching units at a single teaching hospital	The cost of all imaging tests ordered were given to each team weekly	Randomized controlled trial			X		CXR, CT scans, US and liver scans	12.9
Cohen 1982	USA	All physicians working on the teaching units	4 clinical teaching units at a single teaching hospital	The cost of all blood tests ordered were given to each team weekly	Randomized controlled trial			X		All blood tests	24.7^a

Davidoff 1989	USA	24 medical interns entering a university training program	Single teaching hospital	Lectures on probability and test characteristics were compared to placebo lectures on economics and cost control	Randomized controlled trial	X				All blood tests	16.5^a
Detsky 1986	Canada	All attending physicians	20 teaching hospitals in a single province	Residents and interns went on strike for 7 days in 1980	Before-after		X			All blood tests	8.3^a
Dickinson 1987	USA	All physicians in the institution	Single teaching hospital	The number of unnecessary tests was measured and shown to department heads	Before-after			X		All blood tests	24.5^a
Dixon 1974	USA	Interns working on the wards	Single teaching hospital	Interns and residents were limited to 8 tests each	Before-after		X			All blood chemistry tests	66.7^a
Dowling 1989	USA	Residents in a family health center	Single family health center	Lectures and guidelines on appropriate testing were distributed. Audit and feedback of test usage was performed repeatedly	Before-after	X		X		CBC and TSH	47.5^a
Durand-Zaleski 1993	France	All physicians at the hospital	Single teaching hospital	The laboratory requisition was changed so that the purpose of each tumor marker was included on the requisition	Before-after		X			Tumor markers	23.9^a
Durieux 2003	France	All physicians at the hospital	Single teaching hospital	The laboratory requisition was changed so that the purpose of each tumor marker was included on the requisition. The purpose for each test, i.e., screening, follow-up or diagnosis was explained on the requisition	Before-after		X	X		CEA, alpha feto-protein, CA19-9	56.5^a
Eccles 2001	England and Scotland	General Practitioners	Multiple primary care clinics	Guidelines on when to order lumbar spine and knee X-rays were circulated along with feedback on the number of requests to each physician	Randomized controlled trial	X		X		Knee and lumbar spine XR	40.8
Eisenberg 1977	USA	Resident physicians	Single community hospital	There was an education program with a weekly lecture; results from a single audit on utility of testing were circulated along with a memorandum from the medical director asking for careful consideration of test usage	Before-after	X				PTT	42.1^a
Emerson 2001	USA	All physicians working at the medical center	Single teaching hospital	Laboratory requisition was redesigned with some tests unbundled and cascades implemented for thyroid function and anemia	Before-after		X			All blood tests	4.7^a
Everett 1985	USA	First year residents	Single teaching hospital	Residents were taught about laboratory utilization using specific cases	Randomized controlled trial	X				All blood tests and microbiology	–12.3
Everett 1983	USA	First year residents training in internal medicine	Single teaching hospital	Audit and feedback of costs was performed weekly, cost education news letters were distributed and costs of tests were put in patient charts. There were one-on-one meetings of faculty with residents to discuss costs	Randomized controlled trial	X		X		All blood tests	15.1^a
Feldkamp 1996	USA	All physicians in health center	Single teaching hospital and 22 satellite clinics	Algorithms were developed for thyroid function testing and implemented by the laboratory	Before-after		X			Thyroid function testing	21.6
Feldman 2013	USA	All providers who order laboratory tests through a computerized order entry system	Single teaching hospital	Cost of each test was displayed when it was ordered	Randomized controlled trial		X	X		61 diagnostic blood tests	8.6^a
Finegan 2005	Canada	All physician trainees in a hospital	Single teaching hospital	Each anesthetist selected tests to be performed preoperatively at their discretion. The comparator was surgery specific protocols that were being used	Before-after					All blood tests	3.6
Fong 2008	Australia	All physicians working in the emergency room	Single teaching hospital	A clinical decision rule was used instead of physician discretion	Before-after		X			CT head	–27.7^a
Fowkes 1986	United Kingdom	All residents working on the medical unit	Single teaching hospital	A guideline on testing was distributed and weekly meetings were held with medical staff to discuss their use of tests in the previous week	Non-randomized controlled trial	X		X		All blood tests	63.4^a
Friedman 2010	USA	Physicians taking care of children post procedure	Single teaching hospital	A set of standard assessment and management plans were developed and implemented after certain procedures	Before-after		X			All blood tests and imaging	15.6
Froom 2012	Israel	All physicians	Regional laboratory serving a single Health Management Organization	Urine microscopy was no longer done automatically instead the physician had to specifically request it	Before-after		X			Urine for dip and microscopy	99.7^a
Gama 1991	United Kingdom	General internists	Single community hospital	Data on personal and peer expenditure of laboratory resources was given to physicians monthly for 12 months	Randomized controlled trial			X		Clinical chemistry	24.6^a
Gama 1992	United Kingdom	Physicians taking care of inpatients	Single community hospital	Data on personal and peer clinical chemistry expenditure per patient was provided to physicians monthly for 12 months	Before-after			X		Clinical chemistry and hematology	27.0^a
Goddard 2011	Ireland	All physicians working in the ICU	Single teaching hospital	All recurring orders for blood work were discontinued; instead each test had to be ordered as a single occurrence	Before-after					CBC, urea, electrolytes, coagulation, LFTs, magnesium, Ca, PO4, albumin, CRP	33.1
Golden 1987	USA	All physicians	Outpatient department of a single teaching hospital	The cost of tests billed to the patient was changed so that ordering individual tests was cheaper than ordering the whole panel	Before-after		X			All blood chemistry test	33.5^a
Gortmaker 1988	USA	All physicians working in the hospital	Single community hospital	Nine staff meetings were held to discuss cost issues. Data on excess usage was sent to all doctors. All physicians reached a consensus on when each test was considered inappropriate	Before-after			X		BUN, CBC, chemistry profile, CK, creatinine, electrolytes, fasting and random blood sugar, hematocrit, PT, SGOT, urinalysis	14.0^a
Grivell 1981	Australia	All physicians caring for inpatients	Single teaching hospital	Physicians received a report every 4 weeks on the type and number of tests ordered. Physicians were also shown where they rank in comparison to their peers	Before-after			X		All blood tests	0.0
Groopman 1992	USA	All medical staff ordering laboratories in the ER	Single teaching hospital	Tests were taken off the standard requisition and had to be written in by hand	Before-after		X			PT/PTT	61.6^a
Gross 1988	USA	All physicians taking care of inpatients	Single teaching hospital	Guidelines on when to order blood cultures were distributed	Before-after	X				Blood cultures	75.0^a
Haschke-Becher 2009	Austria	No doctors were targeted, it was a laboratory intervention	Single teaching hospital	A decision rule was created to determine when the laboratory would process liquor diagnostics	Before-after		X			Liquor diagnostics	28.6^a
Hutton 2009	United Kingdom	All emergency room and medical unit physicians	Single teaching hospital	Requests for CRP from the ER had to be approved by a senior physician. Junior staff were told to limit CRP use. Lastly repeat testing within 24 h was disallowed	Before-after	X	X			CRP	85.0^a
Jelinek 1990	Australia	All physicians	Single teaching hospital	Guidelines on when abdominal films could be ordered were circulated and all requests had to be run by a senior physician	Before-after	X	X			Abdominal plain films	52.2
Kelly 1998	Australia	All physicians in the ER	Single community hospital	Guidelines on who should receive a blood culture were distributed	Before-after	X				Blood cultures	53.0
Kumwilaisak 2008	USA	All physicians working in the surgical ICU	Single teaching hospital	Guidelines regarding when certain blood tests should be used were introduced at a staff meeting and sent out by email. A session was repeated for new residents every month	Before-after	X				All blood tests	20.8^a
Larocque 1994	Canada	Surgeons and surgical house staff	Single teaching hospital	Guidelines on which tests were appropriate for different medical conditions were posted on the wards and distributed as pocket cards	Before-after	X				Pre-op screening tests including blood work, CXR and ECGs	10.1^a
Levick 2013	USA	Clinicians ordering the BNP test	A health network including 2 hospitals and several community health centers	Computer order entry system displayed a warning when a repeat BNP was being ordered	Before-after		X			BNP	22.2^a
Lewandrowski 1994	USA	All physicians working at the hospital	Single teaching hospital	A guideline on appropriate testing was developed. All requests were then reviewed by laboratory staff and the ordering physician was contacted if the request was not in accordance with the guideline	Before-after	X		X		LD-isozymes	99.6
Mancuso 1999	USA	All physicians at the hospital	Single specialized orthopedic surgery hospital	Guidelines were published suggesting selective test ordering preoperatively	Before-after	X				Pre-operative blood work, ECG, CXR,	30.0^a
Martin 1980	USA	Residents and medical students	Single teaching hospital	There was a 1 h lecture on laboratory costs. Money was given to the residents if their laboratory use decreased	Non-randomized controlled trial	X			X	All radiologic and blood tests	5.8^a
				There was a 1-h lecture on laboratory costs. Senior physicians then met weekly with residents, they reviewed cases and suggested changes in test ordering practices.		X		X			–17.8^a
Marton 1985	USA	Residents and medical students	Single teaching hospital	A manual about rational test ordering was given to the trainees	Randomized controlled trial	X				All blood tests	18.2
				Feedback on laboratory usage and costs incurred was shown to trainees along with their usage compared with peers				X			33.3
				A manual about rational test ordering was given to the trainees. They also received feedback on laboratory usage, costs incurred and their usage compared with peers		X		X			40.9^a
May 2006	USA	All docs working on inpatient service	Single teaching hospital	Duplicate orders were deleted and all blood work orders expired in 24 h. Blood work would only be drawn every 4, 6 or 12 h. Future orders could only be for a single occurrence	Before-after		X			All blood tests	11.5^a
Mehari 1997	New Zealand	Physicians working in ICU	Single teaching hospital	A guideline was placed on each patients chart stating when it is appropriate to do each test	Before-after	X				CBC, coagulation tests, Cr, Na, K, urea, calcium, glucose, Mg, PO4, ABGs	21.2
Meng 2006	Canada	All physicians	3 teaching hospitals	A policy was created regarding who is allowed to order troponin testing and how many repeats can be ordered	Before-after		X			TnI	2.9
Merlani 2001	Switzerland	Doctors residents and nurses working in the intensive care unit	Single teaching hospital	A guideline on when arterial blood gases can be done was circulated	Before-after	X		X		ABG	41.5^a
Meyer 2010	USA	Gynecologic oncologists and other physicians	Single teaching hospital	A multidisciplinary team created a guideline and then created a communication plan to disseminate it	Before-after	X				Pap tests	31.2^a
Miyakis 2006	Greece	All physicians working at the hospital	Single teaching hospital	The results of an audit on test use were presented along with strategies for reducing utilization	Before-after	X		X		25 laboratories hematology and chemistry tests	13.5^a
Morris 1992	USA	All physicians working at the hospital	Single teaching hospital	Only one stool sample for parasites was accepted by the laboratory instead of three	Before-after		X			Stool for ova and parasite	33.3
Mutimer 1992	United Kingdom	All physicians working in the liver unit	Liver unit at a single teaching hospital	Laboratory tests to be ordered for the next day were determined by protocol based on patient characteristics and previous testing	Before-after		X			Hematology, biochemistry, immunology, microbiology and cross matching	9.3^a
Nardella 1995	USA	Anesthetists and surgeons	Single teaching hospital	Guidelines about appropriate pre-operative testing were distributed and discussed at staff meetings	Before-after	X				CBC, PT, PTT, bleeding time, BUN, Cr, Glucose ALT AST, electrolytes	61.8^a
Neilson 2004	USA	Residents, physicians and nurse practitioners who used computer order entry	Single teaching hospital	Recurring tests had to be confirmed daily, many tests were limited to a single occurrence and previous results were displayed when ordering tests	Before-after		X			All blood tests and imaging	64.2^a
Nelson 1978	USA	All physicians who work at the hospital	Single community hospital	There was a 20-min grand rounds presentation on B12 testing. Suggestions were given on when to order this test	Before-after	X				B12 and folate	45.3
Nightingale 1994	United Kingdom	Residents working on the liver unit	Liver unit at a single teaching hospital	A computerized protocol was created that suggested tests for the next day based on current laboratory values and patient characteristics	Before-after		X			All blood tests	23.7
Nirel 2011	Israel	All physicians working in the health system	Primary care clinics	An online medical record was introduced so that old laboratory tests could be accessed	Before-after		X			All blood tests	3.6^a
				An online medical record was introduced so that old radiology tests could be accessed	Before-after		X			All radiology tests	12.3
Pageler 2013	USA	Physicians working in pediatric ICU	Single teaching hospital	A rule preventing repeating testing was created in the computer order entry system. It had to be overridden to order repeat testing	Before-after		X			CBC, chemistry and coagulation	37.7^a
Pageler 2009	USA	Not reported	Single teaching hospital	Recurring orders for CXR were no longer allowed and instead had to be re-ordered daily	Before-after		X			Chest XR	10.0
Patel 2005	United Kingdom	All physicians working in the hospital	Single teaching hospital	Urine was not cultured if the urinalysis was normal	Before-after	X				Urine culture	4.0
Pilon 1997	Canada	Physicians and nurses working in the ICU	Single teaching hospital	A guideline about when ABGs should be used was created. It was distributed on pocket cards and taught during education sessions	Before-after	X				ABGs	36.7^a
Pop 1989	Netherland	General practitioners	All physicians using the same laboratory in one city	Two times per year physicians were given a report of tests they had ordered and whether they were appropriate or not. Physicians were also informed of their redundant tests	Before-after			X		BUN, Cr, liver enzymes, WBC, Hgb, ESR, glucose	36.2
Power 1999	Australia	Surgical Residents in pre-op clinic	Single teaching hospital	Published guidelines on test ordering were distributed. A maximum number of tests per patient was instituted and test ordering was reviewed by a consultant	Before-after	X	X	X		CBC, coagulation tests, urea, LFTs, CXR, ECG	37.1
Powles 2009	United Kingdom	All physicians in region using the university laboratory for PSA testing	Single teaching hospital	Prostate-specific antigen was taken off the request form and had to be written in by hand	Before-after		X			PSA	17.3^a
Prat 2009	France	All physicians working in ICU	Single teaching hospital	A guideline was created on when to perform each of the common blood tests. The cost of each test was also shown on the guideline. Teaching was done on how to reduce test utilization	Before-after	X				CXR and daily routine BW	49.0^a
Pysher 1999	USA	All physicians ordering tests at this laboratory	Single teaching hospital	Predefined chemistry panels were eliminated from the requisition forms. Instead each test had to be ordered separately	Before-after		X			All blood chemistry tests	32.7^a
Ramoska 1998	USA	Physicians and residents working in the emergency department	Single teaching hospital	Graphs of laboratory utilization and costs of laboratory tests at the hospital was displayed on a bulletin board	Before-after			X		All blood tests	17.8^a
Ratnaike 1993	Australia	All physicians working in the cardiac care unit	Single teaching hospital	Guidelines were created by the hospital and issued to the cardiology unit. No further details about the guidelines were given	Before-after	X				All blood tests	57.2
Rhyne 1979	USA	Family doctors	The family medicine program associated with one university	A chart audit of test ordering was done and the results were presented to medical staff	Before-after	X		X		Thyroid function tests	36.4^a
Roberts 1993	Canada	All physicians working at the hospital	Single teaching hospital	Serum osmolality was removed from the requisition and blood count differential was not done if white cell count was normal. A protocol was used to determine when electrolytes should be checked. Daily CXRs were no longer allowed	Before-after	X	X			Blood gases, serum osmolality, CBC, Na, K, glucose, CXR, and ECG	18.1^a
Rosenbloom 2005	USA	All physicians working on selected inpatient units	Single teaching hospital	Magnesium could only be ordered once with no recurrence and a screen displaying old results and education on when magnesium should be checked had to be viewed before the order could be entered	Before-after	X	X			Magnesium	55.2^a
Santos 2012	Brazil	All residents and physicians working at the hospital	Single teaching hospital	A meeting was held to discuss evidence for certain tests and the costs of the tests. Patterns of testing and appropriateness of various tests was discussed	Before-after	X				CRP	48.4
Seguin 2002	France	All physicians working in ICU	Single teaching hospital	Costs for each test was shown on the order form	Before-after			X		Electrolytes, CBC, ABG, LFTs, coagulation panel, CXR	18.9
Shalev 2009	Israel	All primary care physicians in Israel	Single health service provider	27 tests were removed from the requisition and 2 were added	Before-after		X			All blood test	4.3^a
Sorita 2011	USA	Residents training in medicine and surgery	Single teaching hospital	There was a 1 h teaching session regarding when to order STAT blood tests and results of an audit were presented. The highest users of STAT tests were given individual feedback	Before-after	X		X		All stat laboratory tests	21.0^a
Spray 2012	USA	Physicians taking care of inpatients	Single community hospital	An electronic protocol for electrolyte replacement and retesting was instituted	Before-after		X			Magnesium, phosphorous and ionized calcium	32.0^a
Stafford 2003	USA	117 primary care providers	Primary care practices affiliated with a single teaching hospital	Audits of individual and group ECG usages patterns were mailed to physicians	Before-after			X		ECG	28.3
Sucov 1998	USA	All physicians working in the hospital	Single teaching hospital	Guidelines were created regarding what tests to order for which patient groups. The guideline was then rolled out with a series of educational sessions. Each department received feedback on its ordering practices but there was no individual feedback	Before-after	X		X		All blood tests	22.0^a
Sussman 1984	USA	All physicians working in the hospital	Single teaching hospital	All recurring orders were eliminated	Before-after		X			CBC, electrolytes, CXR, chemistry panel, FBG, electrocardiogram, PT	20.8
Thompson 1983	USA	All physicians part of the health plan	Single pre-paid health plan	A recommendation was made against screening healthy adults with blood tests and CXR. An educational campaign was then implemented	Before-after	X				All blood chemistry tests and CXR	60.0^a
Tierney 1987	USA	Residents and staff working in the clinic	Single university associated general medicine clinic	All previous results for a test were shown when using the computer order entry system. An option to cancel the current test was then available	Randomized controlled trial		X			All laboratory and imaging and ECGs	8.9^a
Tierney 1990	USA	Physicians working in the clinic	Single university associated general medicine clinic	Cost to the patient of ordering each tests was displayed before the order is confirmed	Randomized controlled trial		X			All blood tests	17.6^a
Toubert 2000	France	All physicians in the hospital	Single teaching hospital	Educational material on thyroid disease was distributed. Physicians were told that any test other than TSH required justification. Tests were still processed even if no justification was provided	Before-after	X				TSH, T3, T4,	39.6
van Wijk 2001	Netherlands	GPs working in practices with electronic medical records	Single health region	Computerized decision support suggested appropriate tests based on basic clinical information	Randomized controlled trial		X			All blood tests	20.3^a
Vardy 2005	Israel	All physicians ordering tests through the HMO	Single health management organization	Lists of appropriate laboratory tests for each clinical condition were created and physicians were educated about them. Each list was then put into the computer order entry system	Before-after	X	X			All blood tests	4.2
Verstappen 2003	Netherlands	Physicians – not specified	5 health regions in the country	Each physician received feedback on their ordering practices regarding upper and lower abdominal pain and cardiovascular conditions compared with their peers. The physicians then met together to discuss how to change their ordering practices	Randomized controlled trial	X		X		Biochemistry and hematology tests, ECGs, sigmoidoscopy and imaging	12.0^a
				Each physician received feedback on their ordering practices regarding COPD, asthma, degenerative joint complaints and non-specific complaints compared with their peers. The physicians then met together to discuss how to change their ordering practices	Randomized controlled trial	X		X		Biochemistry and hematology tests, ECGs, sigmoidoscopy and imaging	8.0
Wagholikar 2011	Australia	All physicians working in the emergency department	Single teaching hospital	Approval from a senior physician was required to order tests in certain circumstances	Before-after		X			CBC and ESR	11.8^a
Winkens 1995	Netherlands	Family physicians	Single regional medical laboratory	Individualized feedback was sent to each physician from a respected internal medicine physician. The feedback included alternative testing strategies	Before-after			X		Cervical smear, ECG, endoscopy, allergy tests, radiographic and ultrasound tests	5.5^a
Winkens 1992	Netherlands	Family physicians	Single regional medical laboratory	Biannual reports were sent on the number and rationality of laboratory testing compared to peers	Before-after	X		X		46 blood tests	25.0^a
Wones 1987	USA	First year internal medicine residents	Single teaching hospital	The group was simply informed that laboratory utilization was being monitored	Randomized controlled trial			X		All laboratory tests	–11.4
				This group received individual feedback on laboratory usage and costs every 2 weeks				X		All laboratory tests	–13.15
				This group received individual feedback on laboratory usage and costs every 2 weeks along with group data for comparison				X		All laboratory tests	–7.27
Zaat 1992	Netherlands	General Practitioners	Two health regions	The laboratory requisition was modified so that it only had 15 tests on it. All other tests had to be written in	Before-after		X			All hematological and chemistry tests	18.2^a
Zaidi 1999	USA	All physicians sending samples to the hospital lab	Single teaching hospital	Guidelines were created from local data and shared with all medical staff. After this all samples not meeting the criteria were rejected	Before-after	X	X			Stool culture	30.9
Zia 2013	USA	All Physicians at the hospital	Single teaching hospital	The test was removed from the laboratory requisition but could still be ordered if written in by hand	Before-after		X			Serum protein electrophoresis and immunofixation	29.6

^aDenotes studies in which a test of statistical significance was performed and found to be significant by the studies’ authors. A, Audit and feedback; E, Educational; I, Incentive or penalty; S, System-based.

Study quality

Study quality is summarized in Online-Appendix 2. Of the 109 studies, 14 (12.8%) had a randomized design and 37 (33.9%) provided evidence that patients were similar between groups. Time-series analysis was performed in only 11 (10.1%) studies. Interventions were described adequately to be reproducible in 88 (80.7%) studies. Test utilization was reported with the patient or the encounter as the unit of analysis in 79/109 (72.5%) studies with the rest using the physician or the group of physicians working that location as the unit of analysis.

Intervention components

Of 119 interventions, 51 (42.9%) had an educational component, 54 (45.4%) had a system-based component, 41 (34.5%) had an audit feedback component, with only one (0.8%) using incentive or penalty. Physicians were involved in the development of interventions in only 17 (14.3%) interventions. Thirty-two (26.9%) interventions were multifaceted, being classified into more than one of the four main intervention categories. A measure of patient safety or quality of care was included in 13 (11.9%) studies.

Intervention effects

The percent relative reductions in laboratory utilization ranged from a +99.7% (a reduction in the test volumes) to –27.7% (an increase in test volume). Table 2 displays the percent relative reductions in test volume for each intervention and their sub-categories. Interventions with an educational component had the highest median relative reduction in test volume at 34.5% (IQR 16.5–49.0) while audit feedback or system-based interventions had a relative reduction of 22.0% (8.6–34.6) and 22.2% (3.6–68.3), respectively (Figure 2). The single incentive and penalty intervention had a 5.8% relative reduction.

Table 2

Percent relative reductions in test utilization by intervention type.^a

	Number of interventions	Median RR (IQR)	Minimum RR	Maximum RR
Educational interventions	51	34.5 (16.5–49.0)	–12.3	99.6
–Δ between intervention and control statistically significant	33	31.2 (18.1–47.5)	4.7	85.0
–Δ between intervention and control statistically insignificant	18	38.3 (8.0–53.0)	–12.3	99.6
Exclusively education intervention	22	30.6 (16.5–48.5)	–12.3	75.0
Exclusively guideline distribution	13	30.0 (20.8–53.0)	4.7	99.6
System-based interventions	54	22.2 (3.6–68.3)	0.1	99.7
–Δ between intervention and control statistically significant	36	19.6 (10.4–36.1)	3.6	99.7
–Δ between intervention and control statistically insignificant	18	24.8 (10.0–37.1)	0.1	77.8
Exclusively system-based intervention	37	21.6 (10.0–32.7)	0.1	99.7
Exclusively clinical decision support	10	22.8 (9.3–23.9)	4.7	85.0
Audit feedback interventions	41	22.0 (8.6–34.6)	–13.1	99.6
–Δ between intervention and control statistically significant	24	23.2 (13.8–34.5)	5.5	63.4
–Δ between intervention and control statistically insignificant	17	18.9 (0–36.2)	–13.1	99.6
Exclusively audit and feedback intervention	20	18.4 (2.1–24.8)	–13.1	36.2
Exclusively cost display	12	18.4 (2.1–28.8)	5.5	99.6
Physicians targeted were involved in creating and implementing the intervention	17	23.0 (8.0–39.6)	3.6	99.6
Physicians targeted were not involved in creating and implementing the intervention	103	22.2 (10.1–36.7)	–27.7	99.7

^aCategories of interventions are not mutually exclusive unless stated.

Figure 2

Box plots of relative reductions in laboratory utilization stratified by intervention type.

1) Interventions with an educational component; 2) Interventions classified as educational only; 3) Interventions classified as educational only that use distribution of a guideline to educate; 4) Interventions with a system-based component; 5) Interventions classified as system-based only; 6) Interventions classified as system-based only that used a clinical decision support tool; 7) Interventions with an audit and feedback component; 8) Interventions classified as audit and feedback only; 9) Interventions classified as audit and feedback only that use cost information display.

Thirty-three (27.7%) of the interventions targeted three or fewer tests, while 86 (72.3%) targeted four or more tests. The median relative reduction for interventions that targeted three or fewer tests was 40.2% (IQR 23.7–52.8) while that targeting four or more tests was 18.8% (IQR 8.2–28.5) (Figure 3). Thirty interventions (25.2%) used multiple interventions; these multifaceted interventions had larger reductions in test use with a median relative reduction in test volume of 32.7% (IQR 15.1–47.5) versus 21.4% (IQR 9.5–33.3) for interventions that were classified into a single category. Interventions for which physicians were engaged during the intervention development had similar median relative reductions as interventions without physician input.

Figure 3

Box plots of relative reductions in laboratory utilization stratified by intervention type among interventions targeting four or more tests.

Finally, we found important differences in the relative reductions across quality indicators (Table 3). Most notably, the effect of interventions was less if a study reported a reduction in test utilization per patient rather than reduction per institution or physician group and studies with a concurrent control group had smaller reductions. Unexpectedly there were larger relative reductions in studies that had a risk of contamination between the experimental and control groups.

Table 3

General descriptions of study quality (n=109).

	Number of studies	Median relative reduction (IQR)
Where patients similar between groups?
Yes	37	19.0 (8.6–37.7)
No	72	24.8 (14.0–41.5)
Were those ordering the tests similar between groups?
Yes	103	23.8 (11.9–40.2)
No	6	11.7 (4.2–25.0)
Was there a concurrent control group?
Yes	27	16.5 (5.8–27.0)
No	82	24.9 (14.0–47.5)
Was the intervention described adequately enough to be replicated?
Yes	88	23.5 (12.4–38.6)
No	21	20.8 (4.7–36.7)
Was there a risk of contamination between experimental and control groups?
Yes	89	24.9 (13.5–40.8)
No	20	15.8 (4.8–29.6)
Were the results reported per patient instead of per institution or per physician?
Yes	79	19.9 (9.1–35.4)
No	30	33.4 (23.4–48.3)
Was a time-series analysis conducted?
Yes	11	26.1 (14.0–55.2)
No	98	23.5 (10.1–37.1)

Interventions with the largest effects

We found extensive heterogeneity in the components that made up each intervention, how interventions were implemented, the study setting, and the tests that were targeted for reduction. Meaningful generalizations across intervention types is therefore difficult; so the interventions with the largest relative reductions in test volume are described here (Table 1). The studies by Froom et al. and Lewandrowski et al. attained the largest relative reductions with both targeting a single test. Froom et al. changed laboratory policy so that urine microscopy was done only if specifically requested by the physician; this resulted in a 99.7% reduction in test use [53]. Lewandrowski et al. distributed guidelines stating when it was appropriate to order LD-isozymes with a laboratory physician notifying the ordering physician when the LD-isozyme request was not compliant with the guideline [69]. This combination of education and feedback led to a 99.6% reduction in LD-isozymes being ordered. The two largest relative reductions among interventions that targeted four or more tests were by Chu et al. and Dixon et al. Chu et al. used a protocol driven algorithm for admitting trauma patients and found a 68.3% reduction in the number of tests ordered per patient [33]. Dixon et al. limited the number of tests that interns could order to eight per patient per day, resulting in a 66.7% reduction in the number of tests ordered per patient [38].

The most successful interventions targeting four or more tests that were purely system based were the study by Chu et al, discussed above, and a study by Neilson et al. Neilson et al. sequentially implemented a number of system-based interventions using a computerized order entry system including: the display of previous laboratory results to physicians ordering tests; the prevention of recurring orders in various circumstances; and the unbundling of tests so that each component had to be ordered individually [82]. This package of interventions resulted in a 64.2% relative reduction in the tests targeted.

Discussion

We summarized 109 studies evaluating the effect of interventions aimed at reducing test utilization by calculating relative reductions to compare interventions. We found that all intervention categories (including education, system based, audit feedback, and incentive penalty) can reduce test utilization. Even though educational interventions had the largest relative reduction in test utilization, the inter-quartile ranges were large and overlapping. The greatest difference in our stratified analysis was between interventions that targeted less than four tests, which had much larger relative reductions than interventions that targeted four or more tests. Consistent with past reviews, we found that interventions using multiple strategies were more effective [8]. Even though all intervention types seem to be effective, we found a large range of effects and much heterogeneity between studies. Many studies did not use appropriate statistical techniques and follow-up was often short. These shortcomings call into question the sustainability and generalizability of many interventions in our review.

Sustainability

Although interventions with an educational component had the greatest median relative reductions, their long-term sustainability is questionable. The resources needed to perpetuate educational strategies (i.e., lectures or individual teaching sessions) are significant and probably need to be continuous to maintain an effect, especially in teaching hospitals where ordering physicians are constantly changing. Even in studies where the educational strategy was low effort (i.e., the distribution of a guideline), multiple reminders and repeated meetings were necessary to maintain the change in behavior [89, 97, 106, 111]. There was only one purely educational intervention that reported results from >1 year after the intervention began but this study did not use time series analysis to determine if utilization returned to baseline during this time [70]. With all but one purely educational interventions having a time horizon of <1 year there is no evidence in our review that educational strategies have long-term sustainability. In contrast there are several examples of system-based interventions which by definition require few resources to maintain, that are shown to maintain their effect for >1 year [14, 47, 53]. The ratio of resource input to test reduction is not measured in the included studies but is an important consideration when deciding to implement a particular intervention.

Generalizability

Many of the interventions have questionable generalizability. There is a wide range of relative reductions among included studies due to heterogeneity at every level. There are differences in how an invention type is implemented, who is target by the intervention, and the pre-existing ordering practices among the targeted physicians. Due to the heterogeneity, similar results cannot be expected even if an intervention is perfectly replicated. Some of the large reductions (>70%) suggest that anomalous ordering is being targeted which further calls into question the generalizability of results. Lastly many of the studies target physicians in training who would likely respond differently than physicians in independent practice.

Quality

The quality of included studies was poor. As expected studies with lower quality had higher relative reductions suggesting that poor quality studies were over estimating the effects of interventions. Surprisingly, very few of the before and after studies used time-series analysis or autocorrelation to analyze results, bringing into question the validity of results.

Limitations

This review has several limitations. Our search strategy used only studies published in English ignoring potentially important studies published in other languages and only searched two databases. Another limitation is that the heterogeneity between interventions makes generalizations about effective strategies difficult. Despite this weakness the relative reduction does tell us which interventions were associated with the largest reductions. This allows these successes to be examined to see if they are applicable to other settings. This review is broad in scope and therefore does not allow detailed examination of particular intervention types to find the elements that lead to success. Reviews of particular types of interventions have been done and explore these factors closely [122–124]. Lastly, our use of relative reductions leaves out absolute reduction and cost savings information that may be important to decision makers.

Conclusions

Our review found that research into interventions to reduce test utilization is ongoing and many strategies can be successful. There are many examples of low investment interventions that work, especially when they target tests that have a high rate of inappropriate use. Due to questionable validity and generalizability of these results, this review should be used as a reference for finding high quality studies with large relative reductions that apply to a setting of interest. Future research needs to determine how low input strategies using computerized order entry can be used to implement system changes, audit and feedback, and education. Database monitoring should be developed so that institutions know when anomalous test use is occurring and which physicians are deviating from the mean. This will allow for targeting of interventions to where they will have the greatest effect. Lastly, further studies in this field must monitor for adverse events caused by reduced testing.

Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.

Financial support: None declared.

Employment or leadership: None declared.

Honorarium: None declared.

Competing interests: The funding organization(s) played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.

Corresponding author: Dr. Daniel M. Kobewka, Departments of Medicine and Epidemiology and Community Medicine, Ottawa Hospital, Civic Campus, 1053 Carling Avenue, Ottawa, ON K1Y 4E9, Canada, E-mail: dkobewka@gmail.com

About the authors

Daniel M. Kobewka

Daniel Kobewka MD, FRCPC is a Master’s of science student and general internist at the Ottawa Hospital. He received his MD from the University of Alberta and completed his training in General Internal Medicine at the University of Ottawa. His research interests include quality improvement and end-of-life care.

Paul E. Ronksley

Paul Ronksley PhD is a post-doctoral fellow in the department of clinical epidemiology at the Ottawa Hospital Research Institute. He received his PhD in Epidemiology from the University of Calgary and is currently funded by a CIHR post-doctoral fellowship award. His research interests are in chronic disease epidemiology and the use of administrative and population-based survey data to study access to care and health care utilization for patients with multi-morbidity.

Jennifer A. McKay

Jennifer McKay is a Master’s student in the Epidemiology program at the University of Ottawa. Jennifer’s research interests are quality of care, health economics and the funding dynamics of the health care system.

Alan J. Forster

Alan J. Forster MD, FRCPC, MSc is a general internist and Chief Quality and Performance Officer at the Ottawa Hospital. He is a Professor of Medicine at the University of Ottawa and Senior Scientist in the Clinical Epidemiology Program at the Ottawa Hospital Research Institute. His research focuses on patient safety and quality improvement. He has received several prestigious awards recognizing his work within the field of health services research, including a Career Scientist Award with the Ontario Ministry of Health and Long-Term Care and an Early Research Award and Ontario Research Fund grant from the Ontario Government’s Ministry of Research and Innovation. He has performed seminal work evaluating the incidence of adverse events following discharge from hospital.

Carl van Walraven

Carl van Walraven MD, FRCPC, MSc is a Senior Scientist with the Ottawa Hospital Research Institute (OHRI) and is the Site Director for the Institute for Clinical Evaluative Sciences uOttawa Site. He is a general internist at the Ottawa Hospital and is a Professor in the Faculties of Medicine and Epidemiology and Community Medicine at the University of Ottawa. Carl’s research focuses on quality improvement, continuity of care, and administrative database research. He has over 150 publications in peer-reviewed journals. Carl has also been the recipient of several prestigious awards recognizing his work within the field of health services research, including the Ontario Ministry of Health Career Scientist Award and the Premier Research Excellence Award.

References

1. National Health Expenditure trends, 1975 to 2013. Ottawa, ON: Canadian Institute for Health Information, 2013.Search in Google Scholar

2. Health at a Glance 2013: OECD Indicators. Paris: OECD Publishing, 2013.Search in Google Scholar

3. Qaseem A, Alguire P, Dallas P, Feinberg LE, Fitzgerald FT, Horwitch C, et al. Appropriate use of screening and diagnostic tests to foster high-value, cost-conscious care. Ann Intern Med 2012;156:147–9.10.7326/0003-4819-156-2-201201170-00011Search in Google Scholar PubMed

4. van Walraven C, Naylor CD. Do we know what inappropriate laboratory utilization is? A systematic review of laboratory clinical audits. J Am Med Assoc 1998;280:9.Search in Google Scholar

5. Grossman RM. A review of physician cost-containment strategies for laboratory testing. Med Care 1983;21:783–802.10.1097/00005650-198308000-00003Search in Google Scholar PubMed

6. Axt-Adam P, van der Wouden JC, van der Does E. Influencing behavior of physicians ordering laboratory tests: a literature study. Med Care 1993;31:784–94.10.1097/00005650-199309000-00003Search in Google Scholar PubMed

7. Young DW. Improving laboratory usage: a review. Postgrad Med 1988;64:283–9.10.1136/pgmj.64.750.283Search in Google Scholar PubMed PubMed Central

8. Solomon DH, Hashimoto H, Daltroy L, Liang MH. Techniques to improve physicians’ use of diagnostic tests: a new conceptual framework. J Am Med Assoc 1998;280:2020–7.10.1001/jama.280.23.2020Search in Google Scholar PubMed

9. Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Br Med J 2009;339:b2535.10.1136/bmj.b2535Search in Google Scholar PubMed PubMed Central

10. Egger M, Smith GD, Altman DG. Principles of and procedures for systematic reviews. Systematic Reviews in Health Care: meta analysis in context, 2nd ed. London: BMJ Publishing Group, 2008:23–42.Search in Google Scholar

11. Cochrane Effective Practice and Organization of Care Review Group. Data collection checklist. Ottawa: Institute of Population Health, University of Ottawa, 2002.Search in Google Scholar

12. Al AM, Zidan A, Hamza AA, Al-Suwaidi R. Evaluation of laboratory investigations in accident and emergency department, Salmaniya Medical Complex. Bahrain Med Bull 2002;24:95–7.Search in Google Scholar

13. Amukele TK, Baird GS, Chandler WL. Reducing the use of coagulation test panels. Blood Coagul Fibrinolysis 2011;22:688–95.10.1097/MBC.0b013e32834b8246Search in Google Scholar PubMed

14. Archambault AJ, Arkin CF, Skelton T, Abel G, Gawoski J. Blood (ASH Annual Meeting Abstracts) 2011;118:Abstract 4774.10.1182/blood.V118.21.4774.4774Search in Google Scholar

15. Attali M, Barel Y, Somin M, Beilinson N, Shankman M, Ackerman A, et al. A cost-effective method for reducing the volume of laboratory tests in a university-associated teaching hospital. Mt Sinai J Med 2006;73:787–94.Search in Google Scholar

16. Barazzoni F, Grilli R, Amicosante AM, Brescianini S, Marca MA, Baggi M, et al. Impact of end user involvement in implementing guidelines on routine pre-operative tests. Int J Qual Health Care 2002;14:321–7.10.1093/intqhc/14.4.321Search in Google Scholar PubMed

17. Bareford D, Hayling A. Inappropriate use of laboratory services: long term combined approach to modify request patterns. Br Med J 1990;301:1305–7.10.1136/bmj.301.6764.1305Search in Google Scholar PubMed PubMed Central

18. Baricchi R, Zini M, Nibali MG, Vezzosi W, Insegnante V, Manfuso C, et al. Using pathology-specific laboratory profiles in clinical pathology to reduce inappropriate test requesting: two completed audit cycles. BMC Health Serv Res 2012;12:187.10.1186/1472-6963-12-187Search in Google Scholar

19. Barie PS, Hydo LJ. Lessons learned: durability and progress of a program for ancillary cost reduction in surgical critical care. J Trauma 1997;43:590–6.10.1097/00005373-199710000-00005Search in Google Scholar

20. Bates DW, Kuperman GJ, Jha A, Teich JM, Orav EJ, Ma’luf N, et al. Does the computerized display of charges affect inpatient ancillary test utilization? Arch Intern Med 1997;157:2501–8.10.1001/archinte.157.21.2501Search in Google Scholar

21. Bates DW, Kuperman GJ, Rittenberg E, Teich JM, Fiskio J, Ma’luf N, et al. A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests. Am J Med 1999;106:144–50.10.1016/S0002-9343(98)00410-0Search in Google Scholar

22. Berenholtz S, Pronovost P, Lipsett P, Dawson P, Dorman T. Assessing the effectiveness of critical pathways on reducing resource utilization in the surgical intensive care unit. Intens Care Med 2001;27:1029–36.10.1007/s001340100952Search in Google Scholar PubMed

23. Berwick DM, Coltin KL. Feedback reduces test use in a health maintenance organization. J Am Med Assoc 1986;255:1450–4.10.1001/jama.1986.03370110072024Search in Google Scholar

24. Boon-Falleur L, Sokal E, Peters M, Ketelslegers JM. A rule-based decision support application for laboratory investigations management. Proc Annu Symp Comput Appl Med Care 1995;314–8.Search in Google Scholar

25. Buckingham K, Russell I, Ross I, Gibson P, Paterson N. The effect of allowing clinical discretion in ordering biochemical tests. Evaluation by complementary methods. Int J Technol Assess 1994;10:695–700.10.1017/S0266462300008278Search in Google Scholar PubMed

26. Bunting PS, Van Walraven C. Effect of a controlled feedback intervention on laboratory test ordering by community physicians. Clin Chem 2004;50:321–6.10.1373/clinchem.2003.025098Search in Google Scholar PubMed

27. Burnett L, Chesher D, Burnett JR. Optimizing the availability of ‘stat’ laboratory tests using Shewhart ‘C’ control charts. Ann Clin Biochem 2002;39:140–4.10.1258/0004563021901892Search in Google Scholar PubMed

28. Calderon-Margalit R, Mor-Yosef S, Mayer M, Adler B, Shapira SC. An administrative intervention to improve the utilization of laboratory tests within a university hospital. Int J Qual Health Care 2005;17:243–8.10.1093/intqhc/mzi025Search in Google Scholar PubMed

29. Carson K, Murillo P, Runge A, Vogel DG, Baum LG. A test-ordering algorithm can reduce laboratory test use. Lab Med 1995;26:277–81.10.1093/labmed/26.4.277Search in Google Scholar

30. Carter E, Bennett BD. Reference test review by pathology house staff: a cost-containment strategy for the clinical laboratory. Clin Leader Manag Rev 2002;16:3–6.Search in Google Scholar

31. Chen P, Tanasijevic MJ, Schoenenberger RA, Fiskio J, Kuperman GJ, Bates DW. A computer-based intervention for improving the appropriateness of antiepileptic drug level monitoring. Am J Clin Pathol 2003;119:432–8.10.1309/A96XU9YKU298HB2RSearch in Google Scholar

32. Chu K, Wagholikar A, Greenslade J. Reducing laboratory test ordering by limiting the range of tests that may be requested by interns and residents on pathology request forms. Emerg Med Australas 2012;24:21.Search in Google Scholar

33. Chu UB, Clevenger FW, Imami ER, Lampard SD, Frykberg ER, Tepas JJ. The impact of selective laboratory evaluation on utilization of laboratory resources and patient care in a level-I trauma center. Am J Surg 1996;172:558–3.10.1016/S0002-9610(96)00234-6Search in Google Scholar

34. Cohen DI, Jones P, Littenberg B, Neuhauser D. Does cost information availability reduce physician test usage? A randomized clinical trial with unexpected findings. Med Care 1982;20: 286–92.10.1097/00005650-198203000-00005Search in Google Scholar

35. Davidoff F, Goodspeed R, Clive J. Changing test ordering behavior. A randomized controlled trial comparing probabilistic reasoning with cost-containment education. Med Care 1989;27:45–58.10.1097/00005650-198901000-00005Search in Google Scholar

36. Detsky AS, McLaughlin JR, Abrams HB, L’Abbe K, Markel FM. Do interns and residents order more tests than attending staff? Results of a house staff strike. Med Care 1986;24: 526–34.10.1097/00005650-198606000-00007Search in Google Scholar

37. Dickinson JC. From process to policy: a generic prescription for test over-utilization in the emergency department. Fam Pract Res J 1987;7:12–21.Search in Google Scholar

38. Dixon RH, Laszlo J. Utilization of clinical chemistry services by medical house-staff. Arch Intern Med 1974;134:1064–7.10.1001/archinte.1974.00320240098012Search in Google Scholar

39. Dowling PT, Alfonsi G, Brown MI, Culpepper L. An education program to reduce unnecessary laboratory tests by residents. Acad Med 1989;64:410–2.10.1097/00001888-198907000-00017Search in Google Scholar

40. Durand-Zaleski I, Rymer JC, Roudot-Thoraval F, Revuz J, Rosa J. Reducing unnecessary laboratory use with new test request form: example of tumour markers. Lancet 1993;342:150–3.10.1016/0140-6736(93)91349-QSearch in Google Scholar

41. Durieux P, Ravaud P, Porcher R, Fulla Y, Manet C-S, Chaussade S. Long-term impact of a restrictive laboratory test ordering form on tumor marker prescriptions. Int J Technol Assess Health Care 2003;19:106–13.10.1017/S0266462303000102Search in Google Scholar

42. Eccles M, Steen N, Grimshaw J, Thomas L, McNamee P, Soutter J, et al. Effect of audit and feedback, and reminder messages on primary-care radiology referrals: a randomised trial. Lancet 2001;357:1406–9.10.1016/S0140-6736(00)04564-5Search in Google Scholar

43. Eisenberg JM. An educational program to modify laboratory use by house staff. J Med Educ 1977;52:578–81.10.1097/00001888-197707000-00006Search in Google Scholar PubMed

44. Emerson JF, Emerson SS. The impact of requisition design on laboratory utilization. Am J Clin Pathol 2001;116:879–84.10.1309/WC83-ERLY-NEDF-471ESearch in Google Scholar PubMed

45. Everett GD. Impact of supervision by medical teachers and in-patient test control programmes on the out-patient test utilization of residents. Med Educ 1985;19:138–42.10.1111/j.1365-2923.1985.tb01154.xSearch in Google Scholar PubMed

46. Everett GD, deBlois CS, Chang PF, Holets T. Effect of cost education, cost audits, and faculty chart review on the use of laboratory services. Arch Intern Med 1983;143:942–4.10.1001/archinte.1983.00350050100019Search in Google Scholar

47. Feldkamp CS, Carey JL. An algorithmic approach to thyroid function testing in a managed care setting: 3-Year experience. Am J Clin Pathol 1996;105:11–6.10.1093/ajcp/105.1.11Search in Google Scholar PubMed

48. Feldman LS, Shihab HM, Thiemann D, Yeh HC, Ardolino M, Mandell S, et al. Impact of providing fee data on laboratory test ordering: a controlled clinical trial. JAMA Intern Med 2013;173:903–8.10.1001/jamainternmed.2013.232Search in Google Scholar PubMed

49. Finegan BA, Rashiq S, McAlister FA, O’Connor P. Selective ordering of preoperative investigations by anesthesiologists reduces the number and cost of tests. Can J Anaesth 2005;52:575–80.10.1007/BF03015765Search in Google Scholar PubMed

50. Fong BW. Common diagnostic tests: use and interpretation. Hawaii Med J 1989;48:7–9.Search in Google Scholar

51. Fowkes FG, Hall R, Jones JH, Scanlon MF, Elder GH, Hobbs DR, et al. Trial of strategy for reducing the use of laboratory tests. Br Med J 1986;292:883–5.10.1136/bmj.292.6524.883Search in Google Scholar PubMed PubMed Central

52. Friedman KG, Rathod RH, Farias M, Graham D, Powell AJ, Fulton DR, et al. Resource utilization after introduction of a standardized clinical assessment and management plan. Congenit Heart Dis 2010;5:374–81.10.1111/j.1747-0803.2010.00434.xSearch in Google Scholar PubMed PubMed Central

53. Froom P, Barak M. Cessation of dipstick urinalysis reflex testing and physician ordering behavior. Am J Clin Pathol 2012;137:486–9.10.1309/AJCPLJFSS62YBAWNSearch in Google Scholar

54. Gama R, Nightingale PG, Broughton PM, Peters M, Bradby GV, Berg J, et al. Feedback of laboratory usage and cost data to clinicians: does it alter requesting behaviour? Ann Clin Biochem 1991;28(Pt 2):143–9.10.1177/000456329102800204Search in Google Scholar

55. Gama R, Nightingale PG, Broughton PM, Peters M, Ratcliffe JG, Bradby GV, et al. Modifying the request behaviour of clinicians. J Clin Pathol 1992;45:248–9.10.1136/jcp.45.3.248Search in Google Scholar

56. Goddard K, Austin SJ. Appropriate regulation of routine laboratory testing can reduce the costs associated with patient stay in intensive care. Crit Care 2011;15:S47–8.10.1186/cc9553Search in Google Scholar

57. Golden WE, Pappas AA, Lavender RC. Financial unbundling reduces outpatient laboratory use. Arch Intern Med 1987;147:1045–8.10.1001/archinte.1987.00370060041008Search in Google Scholar

58. Gortmaker SL, Bickford AF, Mathewson HO, Dumbaugh K, Tirrell PC. A successful experiment to reduce unnecessary laboratory use in a community hospital. Med Care 1988;26:631–42.10.1097/00005650-198806000-00011Search in Google Scholar

59. Grivell AR, Forgie HJ, Fraser CG, Berry MN. Effect of feedback to clinical staff of information on clinical biochemistry requesting patterns. Clin Chem 1981;27:1717–20.10.1093/clinchem/27.10.1717Search in Google Scholar

60. Groopman DS, Powers RD. Effect of ‘standard order’ deletion on emergency department coagulation profile use. Ann Emerg Med 1992;21:524–7.10.1016/S0196-0644(05)82518-1Search in Google Scholar

61. Gross PA, Van Antwerpen CL, Hess WA, Reilly KA. Use and abuse of blood cultures: program to limit use. Am J Infect Control 1988;16:114–7.10.1016/0196-6553(88)90048-XSearch in Google Scholar

62. Haschke-Becher E, Totzke U, Afazel S, Johansson T, Schwarz M, Ladurner G, et al. Clinical decision rules for the use of liquor diagnostics in hospitalized neurology patients reduced costs without affecting clinical outcomes. Int J Technol Assess Health Care 2009;25:208–13.10.1017/S0266462309090266Search in Google Scholar PubMed

63. Hutton HD, Drummond HS, Fryer AA. The rise and fall of C-reactive protein: managing demand within clinical biochemistry. Ann Clin Biochem 2009;46:155–8.10.1258/acb.2008.008126Search in Google Scholar PubMed

64. Jelinek GA, Banham ND. Reducing the use of plain abdominal radiographs in an emergency department. Arch Emerg Med 1990;7:241–5.10.1136/emj.7.4.241Search in Google Scholar PubMed PubMed Central

65. Kelly AM. Rationalising the ordering of blood cultures. Aust Health Rev 1998;21:245–50.10.1071/AH980245Search in Google Scholar PubMed

66. Kumwilaisak K, Noto A, Schmidt UH, Beck CI, Crimi C, Lewandrowski K, et al. Effect of laboratory testing guidelines on the utilization of tests and order entries in a surgical intensive care unit. Crit Care Med 2008;36:2993–9.10.1097/CCM.0b013e31818b3a9dSearch in Google Scholar PubMed

67. Larocque BJ, Maykut RJ. Implementation of guidelines for preoperative laboratory investigations in patients scheduled to undergo elective surgery. Can J Surg 1994;37:397–401.Search in Google Scholar

68. Levick DL, Stern G, Meyerhoefer CD, Levick A, Pucklavage D. Reducing unnecessary testing in a CPOE system through implementation of a targeted CDS intervention. BMC Med Inform Decis 2013;13:43.10.1186/1472-6947-13-43Search in Google Scholar PubMed PubMed Central

69. Lewandrowski K, Bailey E, Dhanak E, Laposata M, Flood J. Mandatory laboratory consultation: how one hospital’s program reduced overuse of cardiac tests. Lab Med 1994;25:460–3.10.1093/labmed/25.7.460Search in Google Scholar

70. Mancuso CA. Impact of new guidelines on physicians’ ordering of preoperative tests. J Gen Intern Med 1999;14:166–72.10.1046/j.1525-1497.1999.00308.xSearch in Google Scholar PubMed PubMed Central

71. Martin AR, Wolf MA, Thibodeau LA, Dzau V, Braunwald E. A trial of two strategies to modify the test-ordering behavior of medical residents. N Engl J Med 1980;303:1330–6.10.1056/NEJM198012043032304Search in Google Scholar PubMed

72. Marton KI, Tul V, Sox HC. Modifying test-ordering behavior in the outpatient medical clinic. A controlled trial of two educational interventions. Arch Intern Med 1985;145:816–21.10.1001/archinte.1985.00360050060009Search in Google Scholar

73. May TA, Clancy M, Critchfield J, Ebeling F, Enriquez A, Gallagher C, et al. Reducing unnecessary inpatient laboratory testing in a teaching hospital. Am J Clin Pathol 2006;126:200–6.10.1309/WP59YM73L6CEGX2FSearch in Google Scholar

74. Mehari SM, Havill JH, Montgomery C. A written guideline implementation can lead to reductions in laboratory testing in an intensive care unit. Anaesth Intensive Care 1997;25:33–7.10.1177/0310057X9702500106Search in Google Scholar PubMed

75. Meng QH, Zhu S, Booth C, Stevens L, Bertsch B, Qureshi M, et al. Impact of the cardiac troponin testing algorithm on excessive and inappropriate troponin test requests. Am J Clin Pathol 2006;126:195–9.10.1309/GK9BFAB1Y5LNBWU1Search in Google Scholar

76. Merlani P, Garnerin P, Diby M, Ferring M, Ricou B. Quality improvement report: linking guideline to regular feedback to increase appropriate requests for clinical tests: blood gas analysis in intensive care. Br Med J 2001;323:620–4.10.1136/bmj.323.7313.620Search in Google Scholar PubMed PubMed Central

77. Meyer L, Schmeler K, Wallbillich J, Urbauer D, Soliman P, Frumovitz M, et al. Clinical practice guidelines decrease unnecessary Pap tests in survivors of gynecologic malignancies. Gynecol Oncol 2011;120:S30–1.10.1016/j.ygyno.2010.12.075Search in Google Scholar

78. Miyakis S, Karamanof G, Liontos M, Mountokalakis TD. Factors contributing to inappropriate ordering of tests in an academic medical department and the effect of an educational feedback strategy. Postgrad Med 2006;82:823–9.10.1136/pgmj.2006.049551Search in Google Scholar PubMed PubMed Central

79. Morris AJ, Wilson ML, Reller LB. Application of rejection criteria for stool ovum and parasite examinations. J Clin Microbiol 1992;30:3213–6.10.1128/jcm.30.12.3213-3216.1992Search in Google Scholar PubMed PubMed Central

80. Mutimer D, McCauley B, Nightingale P, Ryan M, Peters M, Neuberger J. Computerised protocols for laboratory investigation and their effect on use of medical time and resources. J Clin Pathol 1992;45:572–4.10.1136/jcp.45.7.572Search in Google Scholar PubMed PubMed Central

81. Nardella A, Pechet L, Snyder LM. Continuous improvement, quality control, and cost containment in clinical laboratory testing: effects of establishing and implementing guidelines for preoperative tests. Arch Pathol Lab Med 1995;119:518–22.Search in Google Scholar

82. Neilson EG, Johnson KB, Rosenbloom ST, Dupont WD, Talbert D, Giuse DA, et al. The impact of peer management on test-ordering behavior. Ann Intern Med 2004;141:196–204.10.7326/0003-4819-141-3-200408030-00008Search in Google Scholar PubMed

83. Nelson III RB. Teaching technologic restraint. An evaluation of a single session. Eval Health Prof 1978;1:21–8.10.1177/016327877800100302Search in Google Scholar PubMed

84. Nightingale PG, Peters M, Mutimer D, Neuberger JM. Effects of a computerised protocol management system on ordering of clinical tests. Qual Health Care 1994;3:23–8.10.1136/qshc.3.1.23Search in Google Scholar PubMed PubMed Central

85. Nirel N, Rosen B, Sharon A, Samuel H, Cohen AD. The impact of an integrated hospital-community medical information system on quality of care and medical service utilisation in primary-care clinics. Inform Health Soc Ca 2011;36:63–74.10.3109/17538157.2010.535130Search in Google Scholar

86. Pageler N, Longhurst C, Shin A, Adams E, Widen E, Barth R, et al. Using computerized provider order entry (CPOE) to promote sustained optimization of diagnostic radiologic service utilization. Crit Care Med 2009;37:A363.Search in Google Scholar

87. Pageler NM, Franzon D, Longhurst CA, Wood M, Shin AY, Adams ES, et al. Embedding time-limited laboratory orders within computerized provider order entry reduces laboratory utilization. Pediatr Crit Care Med 2013;14:413–9.10.1097/PCC.0b013e318272010cSearch in Google Scholar

88. Patel HD, Livsey SA, Swann RA, Bukhari SS. Can urine dipstick testing for urinary tract infection at point of care reduce laboratory workload? J Clin Pathol 2005;58:951–4.10.1136/jcp.2004.025429Search in Google Scholar

89. Pilon CS, Leathley M, London R, McLean S, Phang PT, Priestley R, et al. Practice guideline for arterial blood gas measurement in the intensive care unit decreases numbers and increases appropriateness of tests. Crit Care Med 1997;25:1308–13.10.1097/00003246-199708000-00016Search in Google Scholar

90. Pop P, Winkens RA. A diagnostic centre for general practitioners: results of individual feedback on diagnostic actions. J R Coll Gen Pract 1989;39:507–8.Search in Google Scholar

91. Power LM, Thackray NM. Reduction of preoperative investigations with the introduction of an anaesthetist-led preoperative assessment clinic. Anaesth Intensive Care 1999;27:481–8.10.1177/0310057X9902700508Search in Google Scholar

92. Powles L, Green J, Taylor E. Can redesigning a laboratory request form reduce the number of inappropriate PSA requests without compromising clinical outcome. Urology 2009;74:S148.10.1016/j.urology.2009.07.748Search in Google Scholar

93. Prat G, Lefevre M, Nowak E, Tonnelier J-M, Renault A, L’Her E, et al. Impact of clinical guidelines to improve appropriateness of laboratory tests and chest radiographs. Intens Care Med 2009;35:1047–53.10.1007/s00134-009-1438-zSearch in Google Scholar

94. Pysher TJ, Bach PR, Lowichik A, Petersen MD, Shields LH. Chemistry test ordering patterns after elimination of predefined multitest chemistry panels in a children’s hospital. Pediatr Dev Pathol 1999;2:446–53.10.1007/s100249900148Search in Google Scholar

95. Ramoska EA. Information sharing can reduce laboratory use by emergency physicians. Am J Emerg Med 1998;16:34–6.10.1016/S0735-6757(98)90061-4Search in Google Scholar

96. Ratnaike S, Hunt D, Eilermann LJ, Hazen R, Deam D. The investigation of chest pain: audit and intervention. Med J Australia 1993;159:666–71.10.5694/j.1326-5377.1993.tb138081.xSearch in Google Scholar

97. Rhyne RL, Gehlbach SH. Effects of an educational feedback strategy on physician utilization of thyroid function panels. J Fam Practice 1979;8:1003–7.Search in Google Scholar

98. Roberts DE, Bell DD, Ostryzniuk T, Dobson K, Oppenheimer L, Martens D, et al. Eliminating needless testing in intensive care – an information-based team management approach. Crit Care Med 1993;21:1452–8.10.1097/00003246-199310000-00011Search in Google Scholar

99. Rosenbloom ST, Chiu K-W, Byrne DW, Talbert DA, Neilson EG, Miller RA. Interventions to regulate ordering of serum magnesium levels: report of an unintended consequence of decision support. J Am Med Informat Assoc 2005;12:546–53.10.1197/jamia.M1811Search in Google Scholar

100. Santos IS, Bensenor IM, Machado JB, Fedeli LM, Lotufo PA. Intervention to reduce C-reactive protein determination requests for acute infections at an emergency department. Emerg Med J 2012;29:965–8.10.1136/emermed-2011-200787Search in Google Scholar

101. Seguin P, Bleichner J, Grolier J, Guillou Y, Malledant Y. Effects of price information on test ordering in an intensive care unit. Intens Care Med 2002;28:332–5.10.1007/s00134-002-1213-xSearch in Google Scholar

102. Shalev V, Chodick G, Heymann AD. Format change of a laboratory test order form affects physician behavior. Int J Med Inform 2009;78:639–44.10.1016/j.ijmedinf.2009.04.011Search in Google Scholar

103. Sorita A, Steinberg D, Leitman M, Luhan P, Burger A, Sivaprasad L. Impact of targeted individual feedback on provider patterns of stat laboratory test ordering frequency. J Hosp Med 2011;6:S135–6.Search in Google Scholar

104. Spray J, Brackbill M, Baker P. Evaluation of effect of electronic decision tree electrolyte replacement protocol on number of electrolyte labs ordered for patients in critical care. Crit Care Med 2012;40:248.10.1097/01.ccm.0000425196.23304.fdSearch in Google Scholar

105. Stafford RS. Feedback intervention to reduce routine electrocardiogram use in primary care. Am Heart J 2003;145:979–85.10.1016/S0002-8703(03)00107-8Search in Google Scholar

106. Sucov A, Bazarian JJ, deLahunta EA, Spillane L. Test ordering guidelines can alter ordering patterns in an academic emergency department. J Emerg Med 1999;17:391–7.10.1016/S0736-4679(99)00017-7Search in Google Scholar

107. Sussman E, Goodwin P, Rosen H. Administrative change and diagnostic test use: the effect of eliminating standing orders. Med Care 1984;22:569–72.10.1097/00005650-198406000-00007Search in Google Scholar

108. Thompson RS, Kirz HL, Gold RA. Changes in physician behavior and cost savings associated with organizational recommendations on the use of ‘routine’ chest X rays and multichannel blood tests. Prev Med 1983;12:385–96.10.1016/0091-7435(83)90247-5Search in Google Scholar

109. Tierney WM, McDonald CJ, Hui SL, Martin DK. Computer predictions of abnormal test results. Effects on outpatient testing. J Am Med Assoc 1988;259:1194–8.10.1001/jama.1988.03720080028023Search in Google Scholar

110. Tierney WM, Miller ME, McDonald CJ. The effect on test ordering of informing physicians of the charges for outpatient diagnostic tests. N Engl J Med 1990;322:1499–504.10.1056/NEJM199005243222105Search in Google Scholar

111. Toubert ME, Chevret S, Cassinat B, Schlageter MH, Beressi JP, Rain JD. From guidelines to hospital practice: reducing inappropriate ordering of thyroid hormone and antibody tests. Eur J Endocrinol 2000;142:605–10.10.1530/eje.0.1420605Search in Google Scholar

112. van Wijk MA, van dL, Mosseveld M, Bohnen AM, van Bemmel JH. Assessment of decision support for blood test ordering in primary care. a randomized trial. Ann Intern Med 2001;134:274–81.10.7326/0003-4819-134-4-200102200-00010Search in Google Scholar

113. Vardy DA, Simon T, Limoni Y, Kuperman O, Rabzon I, Cohen A, et al. The impact of structured laboratory routines in computerized medical records in a primary care service setting. J Med Syst 2005;29:619–26.10.1007/s10916-005-6130-4Search in Google Scholar

114. Verstappen WH, van der Weijden T, Sijbrandij J, Smeele I, Hermsen J, Grimshaw J, et al. Effect of a practice-based strategy on test ordering performance of primary care physicians: a randomized trial. J Am Med Assoc 2003;289:2407–12.10.1001/jama.289.18.2407Search in Google Scholar

115. Wagholikar A, O’Dwyer J, Hansen D, Chu K. Observing effectiveness of pathology ordering controls in emergency departments. Stud Health Tech Informat 2011;168:172–8.Search in Google Scholar

116. Winkens RA, Pop P, Bugter-Maessen A, Grol RP, Kester AD, Beusmans GH, et al. Randomised controlled trial of routine individual feedback to improve rationality and reduce numbers of test requests. Lancet 1995;345:498–502.10.1016/S0140-6736(95)90588-XSearch in Google Scholar

117. Winkens RA, Pop P, Grol RP, Kester AD, Knottnerus JA. Effect of feedback on test ordering behaviour of general practitioners. Br Med J 1992;304:1093–6.10.1136/bmj.304.6834.1093Search in Google Scholar PubMed PubMed Central

118. Wones RG. Failure of low-cost audits with feedback to reduce laboratory test utilization. Med Care 1987;25:78–82.10.1097/00005650-198701000-00009Search in Google Scholar PubMed

119. Zaat JO. Attempts to change test ordering behavior in laboratories. Ann Clin Biochem 1992;17:8–14.Search in Google Scholar

120. Zaidi AK, Macone A, Goldmann AD. Impact of simple screening criteria on utilization of low-yield bacterial stool cultures in a Children’s Hospital. Pediatrics 1999;103: 1189–92.10.1542/peds.103.6.1189Search in Google Scholar PubMed

121. Zia HM, Singh G. Optimization of utilization of serum protein analysis: role of the electronic medical record in promoting consultation by pathology. Am J Clin Pathol 2013;139:793–7.10.1309/AJCP1ZRZ7KLYSLTGSearch in Google Scholar PubMed

122. Roshanov PS, You JJ, Dhaliwal J, Koff D, Mackay JA, Weise-Kelly L, et al. Can computerized clinical decision support systems improve practitioners’ diagnostic test ordering behavior? A decision-maker-researcher partnership systematic review. Implement Sci 2011;6:88.10.1186/1748-5908-6-88Search in Google Scholar PubMed PubMed Central

123. Buntinx F, Winkens R, Grol R, Knottnerus JA. Influencing diagnostic and preventive performance in ambulatory care by feedback and reminders. A review. Fam Pract Res J 1993;10:219–28.10.1093/fampra/10.2.219Search in Google Scholar PubMed

124. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2012;6:CD000259.10.1002/14651858.CD000259.pub3Search in Google Scholar PubMed

Supplemental Material

The online version of this article (DOI: 10.1515/cclm-2014-0778) offers supplementary material, available to authorized users.

Received: 2014-7-29

Accepted: 2014-8-25

Published Online: 2014-9-27

Published in Print: 2015-2-1

Influence of educational, audit and feedback, system based, and incentive and penalty interventions to reduce laboratory test utilization: a systematic review

Abstract

Introduction

Methods

Data sources and searches

Study selection

Intervention classification

Data extraction and quality assessment

Statistical analysis

Results

Study quality

Intervention components

Intervention effects

Interventions with the largest effects

Discussion

Sustainability

Generalizability

Quality

Limitations

Conclusions

About the authors

References

Supplemental Material

Journal and Issue

Articles in the same Issue