http://orcid.org/0000-0001-5032-6357
Figures
Abstract
Background
Poor quality use of medicines (QUM) has adverse outcomes. Governments’ implementation of essential medicines (EM) policies is often suboptimal and there is limited information on which policies are most effective.
Methods
We analysed data on policy implementation from World Health Organisation (WHO) surveys in 2007 and 2011, and QUM data from surveys during 2006–2012 in developing and transitional countries. We compared QUM scores in countries that did or did not implement specific policies and regressed QUM composite scores on the numbers of policies implemented. We compared the ranking of policies in this and two previous studies, one from the same WHO databases (2003–2007) the other from data obtained during country visits in South-East Asia (2010–2015). The rankings of a common set of 17 policies were correlated and we identified those that were consistently highly ranked.
Findings
Fifty-three countries had data on both QUM and policy implementation. Forty policies were associated with effect sizes ranging from +13% to -5%. There was positive correlation between the composite QUM indicator and the number of policies reported implemented: (r) = 0.437 (95% CI 0.188 to 0.632). Comparison of policy rankings between the present and previous studies showed positive correlation with the WHO 2003–7 study: Spearman’s rank correlation coefficient 0.498 (95% CI 0.022 to 0.789). Across the three studies, five policies were in the top five ranked positions 11 out of a possible 15 times: drugs available free at the point of care; a government QUM unit; undergraduate training of prescribers in standard treatment guidelines, antibiotics not available without prescription and generic substitution in the public sector.
Citation: Holloway KA, Ivanovska V, Manikandan S, Jayanthi M, Mohan A, Forte G, et al. (2020) Identifying the most effective essential medicines policies for quality use of medicines: A replicability study using three World Health Organisation data-sets. PLoS ONE 15(2): e0228201. https://doi.org/10.1371/journal.pone.0228201
Editor: Russell Kabir, Anglia Ruskin University, UNITED KINGDOM
Received: August 30, 2019; Accepted: January 9, 2020; Published: February 6, 2020
Copyright: © 2020 Holloway et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All data used for the analyses shown in this article are shown in three supplementary excel files, labelled tables S1 Table (QUM data), S2 Table (Policy data) and S3 Table (study comparison data) corresponding to the descriptions in the manuscript. The data used to compare policy impact on QUM in this study and the previous two published studies is also included (S3 Table).
Funding: No specific funding was sought or provided for this work
Competing interests: All authors have at some stage been involved with, or have worked for WHO. No other potential COI is declared. This does not alter our adherence to PLOS ONE policies on sharing data and materials.
Introduction
Suboptimal (irrational, incorrect, inappropriate) use of medicines is widespread, wasteful, and causes poor patient outcomes including anti-microbial drug resistance [1–9]. Interventions to improve quality use of medicines (QUM) in low/middle-income countries have mostly been small-scale, of limited duration, with small to modest effects [10–11].
Evidence from studies that we conducted in public healthcare sectors in developing and transitional countries suggests that implementation of WHO essential medicines (EM) policies is associated with better quality use (rational use) of medicines (QUM), including more appropriate use of anti-microbial agents [12–14]. The original WHO global data-set [12] covered the period 2003–2007 and there was uncertainty about how well EM policies were executed (based on country self-reports), with simultaneous deployment of multiple policies making it difficult to estimate individual impacts. We accessed a second source of data collected during 2-week visits to countries in South-East Asia during 2010–15, where policy implementation was observed independently [14]. The analyses of these data confirmed several of the findings of the earlier studies [12–13], including a correlation between the total numbers of EM policies implemented and composite measures of QUM. However, it remains unclear which policies are associated with the largest beneficial effects on medicines use.
The aims of the present work were to analyse an updated global WHO data-set (2007–2011), which included some policies not previously evaluated, and to test the consistency of our earlier findings of an increased impact with larger numbers of implemented EM policies. In addition, we wished to assess replicability of findings by correlating the rankings of policies that were common to the three studies to determine whether certain policies were consistently associated with the largest effects.
Methods
The analytical methods used have been described previously [12–14] and are summarised briefly here. QUM data (outcomes) were extracted from independent survey reports contained within the WHO medicines use database for the period 2006–2012 [3] and reported policy implementation data were obtained from WHO policy databases of surveys sent to Ministries of Health in 2007 and 2011 [15–16]. A dataset was created with one set of QUM and policy indicators for each country. Where the same QUM indicator was measured by more than one survey in the same country during 2006–2012, an average value was calculated. Where the same policy was reported differently in 2007 and 2011, the policy information reported from within one year of QUM survey was used or if this was not possible the data were excluded.
Indicators
Thirteen QUM indicators were extracted from the WHO medicines use database [3]. Only surveys using recommended validated measures estimated from at least 600 prescriptions and/or three or more facilities were included [17–18]. The QUM indicators were all expressed as proportions and are described in Table 1, together with the directionality of better (or worse) QUM. Ten indicators were used in the previous analysis of WHO data [12].
Fifty-two indicators of reported policy implementation were extracted (49 from WHO questionnaires sent to Ministries of Health in 2007 and 2011 [19] and 3 from the WHO medicines use database [3]). The selected EM policies included all those that had been associated with better QUM in the previous two studies [12–14]; all were categorised as yes/no variables. Policies were excluded from analysis if there were fewer than six countries reporting implementation or non-implementation of the policy as was done previously [12].
Where policy indicators overlapped, only one was included. Where there was more than one indicator with a time-frame we included the one with the largest sample size. Through this process the number of policy indicators was reduced to 40 (reasons given in Table 2).
Eight policy indicators had not been analyzed in previous studies: availability of essential medicine list (EML) booklets at health facilities; existence of national legislation on drug promotion; prohibition of advertising prescription-only medicines to the public, a national task force to contain antimicrobial resistance and four policies concerning pharmacists—undergraduate training on the EML and standard treatment guidelines (STGs), continuing professional development and whether pharmacists prescribed in primary care (Table 2).
Analyses
As previously [12–14], we did not try to perform head-to-head comparisons of different policies. Countries implemented different combinations of policies, so the impact of a single policy could not be separated from those that were co-implemented.
For each QUM indicator we calculated the mean difference (expressed as a percentage) between countries reporting implementation (or not) of specific policies. For each policy, we estimated the average difference across all 13 QUM indicators, aligning directionality of better use (positive number) and worse use (minus number), and including only those QUM indicators where there were at least three countries with and three without the policy in question [12].
To assess the impact of multiple policies we generated a composite QUM score from 13 QUM indicators, which enabled all countries to be included in the analysis [12]. We calculated how far each country’s value lay above or below the mean value from all countries for each QUM indicator expressed as standard deviation (SD) units. We then calculated the average of the SD unit increments across the thirteen QUM indicators for each country and used linear regression to assess correlation with the number of EM policies that were implemented [12]. We limited these analyses to policies that had a statistically significant association with better QUM in the univariate analyses.
Individual QUM indicators were also regressed on the number of implemented policies to determine whether specific aspects of QUM were influenced by the intensity of policy implementation. The impact of country wealth was assessed by including Gross National Income per capita [20] in multiple linear regression analyses and by repeating the regression analyses for countries with GNIpc above and below the median of USD 2315.
Testing the replicability of findings across three studies
Statistical analysis methods used in the present study were the same as those used in the earlier WHO analysis [12–13] and the SE Asia country visit analysis [14], enabling us to compare findings across three studies [12–14]. For each of the three data-sets we ranked the policies based on their estimated impact from the univariate analyses. We used non-parametric regression analysis to measure the correlation between the ranking of the policies that were common to the three studies [12–14]. We established the overall ordering of policies by calculating the sum of their ranks across the 3 studies. All analyses were done in Excel 2016, using either Epi Info (version 7.2.1.0)- or Stats Direct (version 3.1.20).
All work involved secondary analyses of data collected for other purposes. Data were aggregated at the level of countries or policies, not individuals, so research ethics board approval was not required.
Results
Fifty-three countries had data on both QUM and policy implementation. Regional distribution of countries was Africa (23), Eastern Mediterranean (7), Europe (2), Latin America (2), South-East Asia (11) and Western Pacific (8). On average, data were available from a median of 2 (range 1–30) QUM surveys and 4 QUM indicators (range 1–13) per country. Each QUM indicator was used by a median of 19 countries (range 9–37). Out of a potential 2120 policy responses (40 policies in each of 53 countries), 1787 (84%) were available for analysis. Of fourteen countries reporting policies in both 2007 and 2011, 85 (18%) responses out of a potential 476 policy responses (34 policies [measured in both 2007 and 2011] x 14 countries) were reported differently and of these 54 (11%) were excluded. Supporting information (S1 Table) describes the 13 QUM indicators and 3 policy indicators obtained from the WHO medicines use database, by country together with the survey references. Supporting information (S2 Table) describes information on the reported implementation of 52 policies by country. Supporting information (S3 Table) describes the impact of common policies in this study and the two previously published studies–this being the data used in the replicability analysis.
Strength of associations for individual policies
Table 3 shows the estimates of policy effect on QUM by policy type, comparing results in countries that did, or did not, report implementation. Fig 1 shows these results in order of their estimated effect size.
Bars and numbers represent the estimated mean effect and 95% CI for the mean effect of each policy on a composite measure of QUM. X-axis acronyms: AMR = antimicrobial resistance; EML = Essential Medicines List; QUM = Quality Use of Medicines; STG = Standard Treatment Guideline; OTC = Over-the-Counter; DTC = Drug and Therapeutic Committee; ADR = Adverse Drug Reaction; CME = Continuing Medical Education.
Policies that were statistically significantly associated with 5–10% or higher effects on QUM included: drugs free at the point of care for children less than five years and the poor; generic substitution; a national strategy to contain antimicrobial resistance; a national body dedicated to QUM; booklets of the national essential medicines lists and standard treatment guidelines available at health facilities; not having systemic antibiotics available over-the-counter; an updated national formulary; no prescriber revenue from medicine sales; national legislation on drug promotion; public education; all drugs on the national Essential Medicines List (EML) provided free of charge in a national health or social insurance system; drug and therapeutic committees in hospitals; and undergraduate education of doctors and pharmacists on standard treatment guidelines.
Of the 27 policies that were associated with positive effects, the average estimated effects were: 9.3% (range 7.0 to 13.0)% for economic policies; 7.4% (2.3 to 10.5%) for managerial policies; 7.3% (range 5.6 to 10.2%) for structural policies; 4.9% (range 2.3 to 6.8%) for educational policies; 4.9% (range 1.7 to 8.6%) for regulatory policies, and 4.2% (range 3.2 to 5.1%) for human resource management policies.
Impacts of multiple policies and national wealth
Fig 2 shows a scatter-gram of composite QUM scores versus the number of policies reported implemented. Correlation between the composite QUM indicator and the number of significantly effective policies reported as implemented (out of 18) was moderate (r = 0.437; 95% CI 0.188 to 0.6322) and strengthened when regression was limited to countries with more than two QUM indicators (r = 0.510; 95% CI 0.243 to 0.704). Inclusion of a national wealth measure (GNIpc) in the regression had no effect (r = 0.51; 95% CI 0.243 to 0.704) and the correlation coefficients were similar when analyses were conducted separately for countries with GNIpc levels above (r = 0.55, p = 0.018) and below the group median (r = 0.41, p = 0.048)).
Data is good enough to show better QUM with implementation of more policies, but not to benchmark country performance.
When we examined the impact of multiple policies on individual QUM indicators (Supporting information S4 Table) we found that the percentage of all cases treated with antibiotics was significantly less with implementation of a greater number of policies (r = -0.375; 95% CI -0.624 to -0.059) as was the percentage of upper respiratory tract infection cases treated with antibiotics (r) = -0.554; 95% CI -0.796 to -0.161. Fig 3 shows a scatter-gram of the percentages of upper respiratory tract infection cases treated with antibiotics versus the number of policies reported implemented. The differences in the percentage of upper respiratory tract infection cases treated with antibiotics were large, ranging from 80–100% in countries implementing less than four EM policies to 30–70% in countries implementing more than 15 policies.
Data is good enough to show less antibiotic use in upper respiratory tract infection with implementation of more policies, but not to benchmark country performance.
Replicability of effects across studies
Table 4 summarises the results for 17 policies that were common to the three studies ordered by the sum of the ranks across the three studies. The table also provides the individual study rankings and whether the univariate analyses of effect sizes had 95% CI that excluded zero. We found a significant correlation between the ranking (24 common policies) in the present study (2007–2011) and that found in the previous analysis of WHO global data (2003–2007): rank correlation coefficient Rho = 0.498 (95% CI 0.022 to 0.789). Correlation between the ranking (20 common policies) in the current analysis and that from the SE Asia country visits was weaker: Rho = 0.465 (95% CI -0.020 to 0.773). Nine policies had effect sizes of 4–10% that were statistically significant in two or more of the three studies. Five policies had consistently high positions in the orderings (highlighted in Table 4), appearing in the top 5 ranked positions 11 out of a possible 15 times. They were: medicines free at the point of care; the presence of a government QUM unit, undergraduate training of prescribers in STGs, antibiotics not available without prescription and generic substitution allowed in the public sector. Statistically significant better QUM associated with implementation of more policies was seen in all three studies [12–14].
Discussion
The main findings from the current study of the most recent WHO data-bases were three-fold. Firstly, some essential medicines policies were associated with better QUM. The strongest associations were for: medicines free at the point of care, implementation of STGs and the EML, a national body to promote QUM, a national strategy to contain AMR, disallowing antibiotic availability OTC, generic substitution in the public sector, hospital DTCs, and public education. Secondly, all policy categories had similar overall degrees of association with better QUM. Thirdly, there was a positive correlation between the number of policies that countries reported implementing and their measures of QUM.
The WHO data have significant limitations, notably the reliance on self-report and the variable co-implementation of several policies, making it difficult to discern the true effects of individual policies. In addition, multiple policies and QUM measures make chance associations likely and limit the interpretation of statistical significance testing. In this situation a consistent finding of a relationship between intensity of policy implementation (number of policies) and a composite measure of QUM is important. In this and previous studies [12–14], there were moderate associations between implementation of more EM policies and better QUM, as reflected by both a composite QUM indicator and individual QUM indicators, notably lower antibiotic use in upper respiratory tract infection. The strength of association seen in this study was like those seen in the previous analyses of WHO global data [12] and data from SE Asia [14]. Unlike the previous two studies the association between EM policies and QUM appeared to be weaker in poorer countries than in wealthier ones, although the association was stronger when regression analysis was limited to more robust QUM data (based on more than 2 QUM indicators).
Although analyses of multiple policy exposure are valuable, these analyses have their own limitations. Most importantly, the exposure variable is the number of equally weighted policies and this does not assist in the identification of the most effective policies. With potentially large numbers of policies and co-variates, and modest number of countries, it was not possible to perform multi-variable analyses and conduct comparisons of individual policies. For these reasons, we assessed the replicability of the ordering of policies by estimated effects across the three studies we have completed. The correlations of the rank orders between the present and previous analyses were modest when measured across the full set of 17 policies that were common to each study. However, the five highest ranking policies (Table 4) occupied the top five places on 11 out of a possible 15 occasions.
In a situation defined by weak data we think the replicability we found across three separate studies, using almost identical methods, is the strongest evidence for identifying the most effective essential medicines policies. We are not suggesting that these policies are the only ones that should be considered for implementation. Countries with particular needs may choose from a larger basket of policies. However, five apparently strong policies, each from a different category, represents a minimum investment for countries seeking to improve QUM and optimize the consumption of antimicrobial drugs. The policies are: drugs free at the point of care; existence of a government QUM unit; undergraduate prescriber training in standard treatment guidelines; antibiotics not available over the counter without prescription and generic substitution allowed in the public sector. Because they come from different policy categories it is possible that they have complementary effects, although that couldn’t be tested here.
Comparison with the broader literature
Previous reviews have recommended implementing similar policies to improve QUM [21–22]. Other studies reporting on actual policy effectiveness reported on: prescriber education [10–11]; public education [23–24]; an MOH body dedicated to promoting QUM [25]; hospital drug and therapeutic committees (DTCs) [26]; non-allowance of prescriber revenue from medicine sales [27–29]; non-allowance of antibiotic availability OTC [30], and national legislation and monitoring of drug promotional activities [31]. Greater effectiveness of multi-faceted interventions (which may involve multiple localised policies), as opposed to single-faceted ones, has also been found elsewhere [32–34]. Furthermore, the better QUM seen here with implementation of more policies was large and comparable with intervention effects reported elsewhere [10–11, 32–34]. However, the sustainability of the better QUM achieved with national medicines policy implementation is likely to be much greater than that achieved with the discrete interventions implemented locally.
Limitations
We have extensively discussed the limitations of the WHO data-bases above and in previous reports [12–13]. The policy data used are reliant on self-reports of implementation, which may have been inaccurate, and the apparent effectiveness of individual policies may have been due to co-interventions. The small number of countries, large numbers of possible policy combinations, and other factors, including political will and economic stability, can reduce implementation effectiveness and hamper attempts to estimate the impacts of individual policies or specific policy combinations. Another weakness of the data was the assumption that policies may have remained the same over time. However, the fact that there were only small differences between estimates from countries reporting policy implementation for both 2007 and 2011 suggests that most policies generally remained constant. Furthermore, misreporting and misclassification would likely have weakened any associations seen between policy implementation and QUM.
There were weaknesses in the QUM data. Firstly, they come from surveys published in the literature. While only surveys using standard methodology and indicators [17–18] were used, they were often based on small sample sizes. Secondly, although standard QUM indicators were used, some were probably measured differently across studies. Thirdly, for some countries there were only one or two QUM indicators measured. This gave a less robust picture of overall QUM and was the reason for use of a composite QUM indicator that allowed all countries to be included in the regression analyses. A few countries had outlier QUM estimates based on only 1–2 indicators; this was a possible explanation for the stronger correlation between number of policies implemented and better QUM when the analysis was confined to countries reporting three or more QUM indicators. Finally, the clinical relevance of a composite QUM indicator is not clear, but the component indicators have relevance and we aligned each to ensure that directionality of change was constant. As with uncertainty over policy variables, any inaccuracies of medicine use estimates would likely have weakened any associations seen between policy implementation and QUM.
Our results were limited to the public sector, since there were insufficient QUM data from the private sector. While the private sector may provide most health care in many low and middle-income countries, the findings are still important since many prescribers work in both sectors and many policies are aimed at both the private and public sectors.
Conclusions
In conclusion, repeated analyses of independent data-sets have shown replicability of two principal findings. The first is that five apparently robust essential medicines policies appear to represent the best choices for countries trying to improve medicines use, and the second one is that the implementation of multiple policies increases their effects. In 2016 The Lancet Commission on Essential Medicines identified five crucial areas of essential medicines policy. Three of these: paying for a basket of essential medicines, making essential medicines affordable and promoting quality use of medicines are strongly supported by the findings of this study [1].
Supporting information
S1 Table. Data on quality use of medicines by country.
https://doi.org/10.1371/journal.pone.0228201.s001
(XLSX)
S2 Table. Data on reported policy implementation by country.
https://doi.org/10.1371/journal.pone.0228201.s002
(XLSX)
S4 Table. Linear regression analyses of individual QUM indicators versus number of effective policies (out of 18) countries reported implementing.
https://doi.org/10.1371/journal.pone.0228201.s004
(DOCX)
Acknowledgments
Richard Laing, Professor, Department of Global Health, Boston University School of Public Health, USA, formerly of WHO. He co-coordinated the medicines policy surveys in 2011.
References
- 1. Wirtz VJ, Hogerzeil HV, Gray AL, Bigdeli M, de Joncheere CP, Ewen MA, et al. Essential medicines for universal health coverage. Lancet 2017 Jan 28;389(10067):403–76. pmid:27832874
- 2.
Holloway KA, van Dijk L. The World Medicines Situation (Third Edition), Rational Use of Medicines. WHO/EMP/MIE/2011.2.2. Geneva, World Health Organization, 2011. Available: http://www.who.int/medicines/areas/policy/world_medicines_situation/en/index.html. Accessed 5 July 2019.
- 3.
World Health Organisation. Medicines use in primary care in developing and transitional countries: Fact Book summarizing results from studies reported between 1990 and 2006, WHO/EMP/MAR/2009.3. Geneva, World Health Organisation 2009.
- 4. Pirmohamed M, James S, Meakin S, Green C, Scott AK, Walley TJ, et al. Adverse Drug Reactions as Cause of Admission to Hospital: Prospective Analysis of 18,820 Patients. BMJ 2004; 329: 15–19. pmid:15231615
- 5. Howard RL, Avery AJ, Slavenburg S, Royal S, Pipe G, Lucasson P, et al. Which drugs cause preventable admissions to hospital? A systematic review. Br J Clin Pharmacol. 2006; 63(2): 136–147. pmid:16803468
- 6. Hitchen L. Adverse Drug Reactions Result in 250 000 UK Admissions a Year. BMJ 2006; 332: 1109. pmid:16690649
- 7. Livermore DM. Bacterial resistance: origins, epidemiology, and impact. Clin Infect Dis. 2003; 36(suppl 1), S11–23. pmid:12516026
- 8. Harbarth S, Samore MH. Antimicrobial resistance determinants and future control. Emerg Infect Dis. 2005; 11: 794–801e. pmid:15963271
- 9. Goossens H. Antibiotic consumption and link to resistance. Clin Microbiol Infect. 2009; 15(Suppl 3): 12–5.
- 10. Holloway KA, Ivanovska V, Wagner AK, Vialle-Valentin C, Ross-Degnan D. Have we improved use of medicines in developing and transitional countries and do we know how to? Two decades of evidence. Trop Med Int Health 2013; 18(6): 656–664. pmid:23648177
- 11. Holloway KA, Ivanovska V, Wagner AK, Vialle-Valentin C, Ross-Degnan D. Prescribing for acute childhood infections in developing and transitional countries, 1990–2009. Paediatrics & International Child Health 2014; pmid:24621245
- 12. Holloway KA, Henry D. WHO Essential Medicines Policies and Use in Developing and Transitional Countries: an analysis of reported policy implementation and medicines use surveys. PLoS Med 2014, 11(9): e1001724. pmid:25226527
- 13. Holloway KA, Rosella L, Henry D. The Impact of WHO Essential Medicines Policies on Inappropriate Use of Antibiotics. PLoS One 2016; 11(3): e0152020. pmid:27002977
- 14. Holloway KA, Kotwani A, Batmanabane G, Santoso B, Ratanawijitrasin S, Henry D. “Promoting quality use of medicines in South-East Asia: reports from country situational analyses”. BMC Health Serv Res 2018; 18:526. pmid:29976180
- 15.
World Health Organisation. Country pharmaceutical situations: Fact Book on WHO Level 1 indicators 2007. WHO/EMP/MPC/2010.1. Geneva World Health Organisation 2010.
- 16.
World Health Organisation (2011). Country pharmaceutical profiles. URL: http://www.who.int/medicines/areas/coordination/coordination_assessment/en/ (Accessed 22/9/2018)
- 17.
World Health Organisation. How to investigate drug use in health facilities: selected drug use indicators. WHO/DAP/93.1. Geneva, World Health Organisation 1993.
- 18. Gove S. Integrated management of childhood illness by outpatient health workers: technical basis and overview. The WHO Working Group on Guidelines for Integrated Management of the Sick Child. Bulletin of WHO 1997; 75(Suppl 1): 7–24.
- 19.
World Health Organisation. WHO Operational package for assessing, monitoring and evaluating country pharmaceutical situations: Guide for coordinators and data collectors. WHO/TCM/2007.2. Geneva, World Health Organisation 2007.
- 20.
UN country profile data. URL: http://www.data.un.org (accessed 22/9/2018)
- 21. Laing R, Hogerzeil HV, Ross-Degnan D. Ten recommendations to improve the use of medicines in developing countries. Health Policy Plan. 2001; 16(1): 13–20. pmid:11238425
- 22.
World Health Organisation. Promoting rational use of medicines: core components. WHO Policy Perspectives of Medicines, no.5. Geneva, World Health Organisation 2002.
- 23. Huttner B, Goossens H, Verheij T, Harbath S, for the CHAMP Consortium. Characteristics and outcomes of public campaigns with the aim to improve outpatient antibiotic use in high income countries. Lancet Infect. Dis. 2009; 10(1): 17–31.
- 24. Sabuncu E, David J, Bernède-Bauduin C, Pépin S, Leroy M, Boelle PY, et al. Significant Reduction of Antibiotic Use in the Community after a Nationwide Campaign in France, 2002–2007. PLoS Med 2009; 6(6), e1000084. pmid:19492093
- 25.
Jaffer B. National approach of promote rational use of medicines: the Omani experience. WHO/NGO Technical Briefing Seminar, World Health Assembly, 14th May 2007, Geneva. Directorate of Rational Drug Use Department, MOH, Oman. Available: http://www.who.int/drugresistance/OmanDRDUforWHAMeetingGeneva.pdf. Accessed 5 July 2018.
- 26. Vang C, Tomson G, Kounnavong S, Southammavong T, Phanyanouvong A, Johansson R, et al. Improving the performance of Drug and Therapeutics Committees in hospitals—a quasi-experimental study in Laos. Eur J Clin Pharmacol. 2006; 62: 57–63. pmid:16372173
- 27. Trap B, Hansen EH, Hogerzeil HV. Prescription habits of dispensing and non-dispensing doctors in Zimbabwe. Health Policy Plan. 2002; 17(3):288–295. pmid:12135995
- 28. Trap B, Hansen EH. Treatment of upper respiratory tract infections—a comparative study of dispensing and non-dispensing doctors. J Clin Pharm Ther. 2002; 27(4): 289–299. pmid:12174031
- 29. Park S, Soumerai SB, Adams AS, Finkelstein JA, Jang S Ross-Degnan D. Antibiotic use following a Korean national policy to prohibit medication dispensing by physicians. Health Policy Plan. 2005; 20(5): 302–309. pmid:16000369
- 30. Bavestrello L, Cabello A, Casanova D. Impact of regulatory measures in the trends of community consumption of antibiotics in Chile. Rev Med Chil. 2002; 130(11): 1265–1272. pmid:12587509
- 31.
World Health Organisation. Drug Promotion—what we know, what we have yet to learn–reviews of materials in the WHO/HAI Database on Drug Promotion. EDM Research Series No.32, WHO/EDM/PAR/2004.3, Geneva, World Health Organisation 2004.
- 32. Rowe AK, de Savigny D, Lanata CF, Victora CG. How can we achieve and maintain high-quality performance of health workers in low resource settings? Lancet 2005; 366(9490): 1026–1035. pmid:16168785
- 33. Francke AL, Smit MC, De Veer AJE, Mistiaen P. Factors influencing the implementation of clinical guidelines for health care professionals: a meta-review. BMC Med Inform Decis Mak. 2008; 8: 38. pmid:18789150
- 34. Sketris IS, Ingram EML, Lummis HL. Strategic opportunities for effective optimal prescribing and medication management. Can J Clin Pharmacol. 2009; 16(1): e103–e125. pmid:19182305