Types of studies

We will include randomised controlled trials; non‐randomised controlled trials and controlled before‐and‐after (CBA) studies involving at least two intervention sites and two control sites; and interrupted time series (ITS) and repeated measures (RM) studies with a clearly defined point in time when the intervention occurred and at least three data points before and three after implementation of the intervention.

Types of participants

Healthcare consumers and providers within a large jurisdiction or healthcare system in low‐, middle‐ and high‐ income countries. Jurisdictions can be regional, national or international. We will include studies within organisations, such as health maintenance organisations, only if the organisation is multisited and serves a large population.

Types of interventions

Policies that regulate:

1. who can sell drugs (for instance sales by physicians, pharmacists, other health workers, etc.);

2. where drugs can be sold (for example, pharmacies, other healthcare providers, regular shops, etc.);

3. ownership, location and number of pharmacies or other dispensing places;

4. dispensing behaviour (such as dispensing regulations, regulations of marketing by retailers, financial incentives for pharmacies and other dispensers, generic substitution by pharmacies, separation of prescribing and dispensing functions).

We will define policies as laws, rules or financial or administrative orders made by governments, non‐government organisations or private insurers.

We will exclude interventions that regulate the pricing of drugs (Acosta 2014) or prescribers' behaviour (Sturm 2007) and those related to expanding the role of pharmacists (Evans 2011; Chisholm‐Burns 2010).

Types of outcome measures

We will include studies that report at least one of the following outcomes.

• Drug use (prescribed, dispensed or actually used), such as the number of dispensed doses or the number of dispensed prescriptions

• Health outcomes, such as mortality or any assessment of morbidity

• Healthcare utilisation, such as number of emergency room visits, changes in physician office visits per year, hospitalisation rates, etc.

• Costs (expenditures, including drug costs and prices, other healthcare costs and policy administration costs)

• Adverse effects, such as increased inequalities (between poorer and richer groups), undesirable effects on healthcare providers, increased resource use, or decreased quality of care

Electronic searches

We will search in the following databases.

• The Cochrane Central Register of Controlled Trials (CENTRAL) including the Effective Practice and Organisation of Care (EPOC) Group Register, from 1988 to present at: www.thecochranelibrary.com

• MEDLINE In‐Process & Other Non‐Indexed Citations and MEDLINE Ovid 1946 to Present,

• EMBASE Elsevier, from 1980 to present

• CSA Worldwide Political Science Abstracts, from 1975 to present

• EconLit Proquest , from 1969 to present,

• NHS EED, National Health Services Economic Evaluation Database, from 1988 to present, www.thecochranelibray.com

• System for Information on Grey Literature in Europe (OpenGrey), from 1980 to present, at http://www.opengrey.eu/

• International Network for Rational Use of Drugs (INRUD) from 1989 to present, at www.inrud.org

• PAIS International (Public Affairs Information Service) ProQuest, from 1972 to present

• Worldwide Political Science Abstracts, ProQuest, 1975 to present

• International Clinical Trials Registry Platform (ICTRP), Word Health Organization (WHO) http://www.who.int/ictrp/en/

• ClinicalTrials.gov, US National Institutes of Health (NIH) http://clinicaltrials.gov/

Searching other resources

In addition we will search in the following websites and databases.

• OECD (Organisation for Economic Co‐operation and Development) Publications & Documents www.oecd.org/officialdocuments

• World Bank Documents & Reports: http://documents.worldbank.org/curated/en/docadvancesearch

• World Bank eLibrary: http://elibrary.worldbank.org/action/doSearch

• JOLIS, The Library Network serving the World Bank Group and IMF Global Jolis, online catalogue for the World Bank Country Office http://external.worldbankimflib.org/uhtbin/webcat/#_

• The repository of publications from the WHO Department of Essential Medicines http://apps.who.int/medicinedocs/en/

We present the MEDLINE search strategy in Appendix 1. We will use a modified version of the EPOC search strategy methodology filter to limit the MEDLINE strategy to randomised trials, controlled trials, time series analyses and CBA studies. We will develop the search strategies for the other databases from the MEDLINE strategy.

We will screen the reference lists of all of the relevant reports that we retrieve. We will also search the Science Citation Index for articles citing key references. We will contact the authors of relevant papers, relevant organisations and discussion lists to identify additional studies, including unpublished and ongoing studies.

Selection of studies

Three of the review authors (BP ,TP and CH) will independently assess titles and abstracts identified by the search strategy and the reference lists of relevant reports in duplicate. We will retrieve the full text of any potentially relevant study and two authors (BP, TP and CH) will independently decide whether they should be included using the criteria described above. We will list those studies which, after detailed assessment, are found not to meet the inclusion criteria in a 'Characteristics of excluded studies' table. Two authors will independently extract the data from included studies. We will resolve any disagreements by discussion and, when necessary, through the arbitration of another author.

Data extraction and management

We will extract the following information from included studies using a standardised data extraction form.

• Type of study

• Study setting (country, classified according to World Bank income classification: low‐, middle‐ or high‐income country (World Bank 2013), key features of the healthcare system and concurrent pharmaceutical policies)

• The sponsors of the study

• Characteristics of the participants (organisations, regional or national level policies)

• Characteristic of the policies

• Main outcome measures and study duration

• The results of the main outcome measures

Assessment of risk of bias in included studies

We will assess included studies for risk of bias using the criteria suggested by the Cochrane Effective Practice and Organisation of Care (EPOC) group (EPOC 2015). Two authors will independently assess each study and will reach a consensus assessment.

Measures of treatment effect

For randomised, non‐randomised and CBA studies we will report adjusted relative effects. For dichotomous outcomes we will report, if possible, the relative risk adjusted for baseline differences in the outcome measure (the relative risk postintervention/the relative risk preintervention). For continuous outcomes, we will report, if possible, the relative change adjusted for baseline differences in the outcome measure (the absolute postintervention difference between groups minus the absolute preintervention difference between groups) divided by the postintervention level in the control group).

For ITS and RM studies the results will be reported, if possible, as changes along two dimensions: changes in level and changes in slope. Given that policy changes are often announced some months prior to official implementation (or take some time to be implemented), we will define a transition phase as the six months from the official announcement or formal implementation of the policy change. If the included ITS and RM studies state a different transition phase, we will use that definition. All results will exclude transition‐phase data. Change in level is the immediate effect of the policy and we will measure this as the difference between the fitted values for the first postintervention data point (e.g. after finishing the transition phase) minus the predicted outcome after the intervention based only on the preintervention slope. We will calculate the relative change in level by dividing the change in level by the predicted outcome based only on the preintervention slope and multiplying by 100%. Change in slope is the change in the trend from pre‐ to postintervention that reflects the 'long' term effect of the intervention. Since the interpretation of change in slope could be difficult, we will present the long‐term effects in a way that will be similar to the one we will use to calculate and present the relative immediate effects. We will present the effects after half a year as the difference between the fitted value for the sixth month postintervention data point (half a year after the intervention) minus the predicted outcome six months after the intervention based on the preintervention slope only, divided by the predicted outcome six months after the intervention based on the preintervention slope only and multiplied by 100%. If possible, we will measure the effects after one and two years in the same way. For drug expenditures we will calculate the savings after a half year, and after one and two years as the area between the predicted expenditure curves and the actual expenditure. If studies with an ITS design do not provide an appropriate analysis or reporting of results, but present the data points in a scannable graph or in a table, we will reanalyse the data using methods described in Ramsay 2003, as detailed in the section Dealing with missing data.

Unit of analysis issues

We expect some eligible studies to have a cluster designs (studies in which the unit of allocation is not a person, but is instead a group of people). If we include any study of this design in the review, we will determine whether the data were correctly analysed using the following criteria:

• the analysis was conducted at the same level as the allocation (i.e. at the 'cluster' level); or

• the usual analysis was used but the sample size was reduced to its ‘effective sample size’ or the variance was inflated by the design effect; or

• the analysis was conducted at the level of the individual, but appropriate statistical correction for the clustering was performed (such as generalised estimating equations, mixed models or multilevel models).

If we detect unit of analysis errors we will not attempt to reanalyse the data but will report the results of the study as point estimates of the intervention effect without the presentation of any statistical analysis (P values or confidence intervals) (Higgins 2011)

Dealing with missing data

We will attempt to contact authors to obtain important missing information for studies that were published within the past 10 years. For studies published before 2003, we will use our best judgement to determine the missing information from the available publication (e.g. estimate the numbers corresponding to outcomes that are presented only in graphical form).

If studies with an ITS design do not provide an appropriate analysis or reporting of results but present the data points in a scannable graph or in a table, we will reanalyse the data using methods described in Ramsay 2003. We will use the following segmented time series regression model: Y(t) = B0 + B1*Preslope + B2*Postslope + B3*intervention + e(t), where:

• Y(t) is the outcome in month t;

• Pre‐slope is a continuous variable that indicates the time from the start of the study up to the last point in the preintervention phase and coded constant thereafter;

• Post‐slope is coded 0 up to and including the first point postintervention and coded sequentially from 1 thereafter;

• the intervention is coded 0 for preintervention time points and 1 for postintervention time points.

In this model, B1 estimates the slope of the preintervention data; B2 estimates the slope of the postintervention data; and B3 estimates the change in level of outcome as the difference between the estimated first point postintervention and the extrapolated first point postintervention if the preintervention line was continued into the postintervention phase. We will calculate the difference in slope as B2 minus B1. We will assume the error term (t) to be first order autoregressive. For controlled ITS studies, we will present the difference between the relative changes of the intervention and the control groups. We will calculate confidence intervals (95%) for all effect measures.

Assessment of heterogeneity

For studies reporting similar comparisons and outcome measures, we will assess heterogeneity visually by preparing tables, bubble plots (where the size of the bubble corresponds to the size of the population participating in each study) and box plots (displaying medians, interquartile ranges and ranges) to explore the size of the observed effects in relation to a number of explanatory factors (Subgroup analysis and investigation of heterogeneity). Additionally, we will assess the extent of the heterogeneity in results across comparable studies using forest plots, the I² statistic and the Chi² test.

Assessment of reporting biases

If it is possible, for studies reporting similar comparisons and outcome measures, we will use a funnel plot to visually explore the risk of publication bias, using the population of the jurisdictions included as a proxy for the precision of the estimate and the adjusted risk ratio or risk difference as the treatment effect.

Data synthesis

We will group the studies into the four categories of interventions described above (Types of interventions). We will conduct meta‐analyses only for studies that report similar comparisons (interventions, comparisons and outcome measures that are sufficiently similar that an average effect across those studies would be meaningful).

For randomised, non‐randomised and CBA studies, we will record the number of events (in the case of health outcomes) and the total numbers in each group (for risk ratios), or means and standard deviations in each group (for weighted mean differences, for instance in the case of drug utilisation). We will show all outcome effects with their associated 95% confidence intervals. Anticipating heterogeneity across studies, we will apply a random‐effects model meta‐analysis. We will perform data synthesis using Review Manager version 5.3 (Rev Man 2014)

If it is not possible to synthesize the data from included studies, we will undertake a structured synthesis following the EPOC guidance on this topic (EPOC 2013). For each category of intervention, we will describe the range of effects found in the studies and, if possible, the mechanisms through which the interventions were intended to affect specific outcomes.

We will grade our confidence in the available estimates of effects using a modification of the approach recommended by the GRADE Working Group. When grading the quality of evidence, we will initially grade ITS and RM studies as of ’moderate’ quality, and CBA studies as of ’low’ quality. This reflects our judgement that ITS and RM studies provide more compelling evidence than CBA studies in evaluating policies. We will use the GRADE quality scores high, moderate, low and very low.

Subgroup analysis and investigation of heterogeneity

Given that this review is dealing with a complex health intervention (Shepperd 2009), the following potential effect modifiers will be considered when investigating heterogeneity.

1. Differences in income of countries where the policy was implemented: We will use low‐ and lower‐middle, upper‐middle and high‐income countries, as classified by the World Bank (World Bank 2013). We will group the countries into three categories: low‐income, lower‐ and upper middle‐ and high‐income countries. We expect larger effects of the intervention with the availability of more resources because successful implementation might require information and monitoring systems that are more likely to be adequate in high‐income countries.

2. Differences in disincentives: We will assess whether each policy has no, minor or major disincentives for dispensers to adhere to the policy. We expect smaller effects with greater disincentives for dispensers given that financial disincentives or increased burdens on dispensers are likely to impair successful implementation of policies.

3. Differences in enforcement: We will assess whether the implementation of each policy was monitored and included penalties for non‐adherence, or included some other strategy for ensuring adherence. We expect larger effects with strategies for ensuring adherence given that policies that are not enforced are less likely to be successfully implemented.

We will carry out subgroup analyses only if there are sufficient numbers of studies/data within subgroups.

We will use RevMan to estimate subgroup differences (using the Chi² test for subgroup differences).

Sensitivity analysis

We will assess the robustness of our analyses by performing sensitivity analyses for any comparison that includes studies with a high or uncertain risk of bias, by excluding those studies and recalculating the effect size.