Study designs

In the context of a global pandemic, evidence to inform decisions must be generated rapidly, meaning that methods traditionally used to evaluate the impact of interventions, such as randomised controlled trials (RCTs) or quasi‐experimental studies, while possible, may not be considered feasible, appropriate, timely or ethical. Indeed, in this specific context, simulation models developed to make predictions about the (highly uncertain) future often represent the only available evidence to guide decision‐making. To ensure that we captured all relevant study types, we considered a broad range of empirical studies of any size that provided a quantitative measure of impact, including experimental and quasi‐experimental studies, observational studies, and mathematical modelling studies. Thus, we included the following types of studies:

• Experimental and quasi‐experimental studies, such as

  • RCTs

  • Interrupted time series (ITS) studies

  • Controlled before‐after (CBA) studies and difference‐in‐differences (DiD) studies

  • Instrumental variable (IV) studies

  • Regression discontinuity (RD) studies

• Observational studies, such as

  • Cohort studies

  • Case‐control studies

• Modelling studies, such as

  • Compartmental models (e.g. SEIR‐type models comprising multiple compartments, such as S: susceptible, E: exposed, I: infectious, R: recovered)

  • Bayesian hierarchical models (i.e. models comprising several submodels to integrate observed data as well as uncertainty)

  • Spatial models (i.e. modelling disease transmission spatially)

  • Time‐series models (i.e. models that model the temporal nature of disease transmission using time‐series techniques)

To avoid the inappropriate exclusion of studies, we considered all studies providing a quantitative measure of impact, regardless of whether they were indicated by any of these labels. We considered studies published in peer‐reviewed journals as well as those published on preprint servers. Our rationale for including preprint articles was that in the context of a global pandemic, there may be a scientific as well as moral case for publishing studies at the earliest opportunity to inform the emergency response. We included any studies that had been registered but not yet published (in a peer‐reviewed journal or on a preprint server) as 'ongoing' studies.

We excluded the following types of studies and publications:

• Case reports

• Studies that did not provide a quantitative measure of impact (e.g. studies providing a graphical summary of the number of cases over time in relation to the introduction of control measures, qualitative studies)

• Diagnostic studies (e.g. assessing the sensitivity and specificity of different screening tests in general; we did, however, include studies on the use of screening tests at national borders as a travel‐related control measure)

• Non‐empirical studies (e.g. commentaries, editorials, non‐systematic literature reviews not reporting primary empirical data)

• Systematic reviews (although relevant reviews were used for backward citation searches)

• Conference abstracts

Population

We included studies on human populations (without any age restriction) susceptible to SARS‐CoV‐2/COVID‐19. To be eligible, modelling studies had to use modelling parameters for disease transmission specified to reflect SARS‐CoV‐2/COVID‐19. In the original review, we also included studies on SARS‐CoV‐1/SARS and MERS‐CoV/MERS (Burns 2020).

For this update, we excluded studies:

• not targeting human transmission;

• concerned with humans at risk of developing other infectious diseases, characterised by different transmission properties (e.g.  SARS‐CoV‐1/SARS and MERS‐CoV/MERS, Ebola and viral meningitis, the transmission modes of which are primarily person‐to‐person, rather than airborne); and

• addressing humans at risk of developing other infectious diseases, for which travel‐related control measures do not play a significant role in containing outbreaks (e.g. influenza).

Interventions

We considered travel‐related control measures affecting human travel across national borders. We considered both introduction and implementation, as well as relaxation and de‐implementation of the following measures.

• Closure of national borders to entry or exit, or both, which stop cross‐border travel 

• International travel restrictions or bans, or both, which reduce cross‐border travel. These may include the following specific measures.

  • Denial of entry or exit, or both, on the basis of nationality, travel history, health status or other characteristics

  • Full or partial suspension of cross‐border travel via any or all of land, air and sea

  • Visa requirement or refusal on the basis of nationality, travel history, health status or other characteristics

• Screening at national borders, involving any of the measures listed below, as well as a follow‐up measure, such as testing, self‐isolation or refusal of entry, only for those who screen positive

  • Temperature measurement (e.g. thermography)

  • Health questionnaire (e.g. symptoms, travel history, contact history)

  • Physical examination

  • Testing for current or past infection

• Quarantine or isolation of travellers crossing national borders, including voluntary or government‐mandated quarantine of travellers for different durations and without any follow‐up measures, such as testing at certain days of the quarantine

• Any combination of the above measures

We excluded the following types of interventions.

• Combinations of the above‐mentioned travel‐related control measures with other measures where studies do not provide effect estimates for the travel‐related control measures (e.g. studies providing a combined effect estimate for suspension of cross‐border travel and use of mandatory face masks in the general population) Studies in which the effect of travel‐related control measures cannot be disentangled from the effect of a broader suite of public health measures cannot usefully inform WHO recommendations on whether countries should or should not consider travel‐related control measures to contain the COVID‐19 pandemic.

• All interventions not directly related to travel, including a range of containment and mitigation measures (e.g. community‐based quarantine, personal protective measures, hygiene measures, bans on mass gatherings and other social‐distancing measures).

• All interventions related to movement of animals or goods.

• All interventions concerned with human travel across subnational borders. While subnational measures can potentially inform national travel‐related control measures, these measures are not prioritised by the WHO. As shown in the previous evidence map (Movsisyan 2021), they are also often impossible to disentangle from other subnational measures, such as lockdowns, community quarantine or social distancing recommendations.

• Travel warnings or travel advice issued by the WHO or national governments.

• Studies of interventions solely concerned with the accuracy of tests rather than their implementation as part of an entry and/or exit border control measure.

• Studies of interventions related to international travel but not concerned with cross‐border impacts, i.e. interventions to contain transmission within closed populations that only assessed their effect on these closed populations (e.g. on cruise ships, within detention centres). This exclusion criterion was added post hoc.

• Usual practice (e.g. seasonal changes to travel) or events (e.g. school holidays) affecting travel but not representing travel‐related control measures.

• Cancellation of events affecting international travel but undertaken as a means to prevent mass gatherings (e.g. Hajj, international sporting events, international trade fairs).

We included studies that assessed travel‐related control measures as specified above, targeting populations within one country (e.g. the lockdown of Wuhan, China) if their impact was assessed on the population of other countries (e.g. Australia).  We additionally considered relevant restrictions between mainland China and Hong Kong and Taiwan, given the existence of a hard border and the implementation of travel‐related control measures analogous to those implemented internationally.  

Comparator(s)

We included a range of possible comparators, such as a counterfactual scenario in which the intervention was not implemented, a complete relaxation of the measure, or a partial relaxation of the measure. Likewise, a scenario of no intervention could have been compared against a counterfactual scenario in which an intervention was implemented or relaxed. A relevant study therefore may compare an observed intervention with a simulated scenario of no intervention, while another study may compare simulated stringent interventions with simulated lax interventions, while yet another study may compare an observed intervention with a simulated intervention implemented at an earlier time.

Outcome(s)

Electronic databases

For this update, we ran searches in the following electronic databases.

• Ovid MEDLINE and Epub Ahead of Print, In‐Process & Other Non‐Indexed Citations, Daily and Versions (1946 to 13 November 2020)

• Ovid Embase (1996 to 13 November 2020)

Other searches

We additionally searched the following COVID‐19‐specific databases.

• Cochrane COVID‐19 Study Register (covid-19.cochrane.org), which contains study references from ClinicalTrials.gov, WHO International Clinical Trials Registry Platform (ICTRP), PubMed, medRxiv and other handsearched articles from publishers’ websites.

• WHO 'Global literature on coronavirus disease' database (search.bvsalud.org/global-literature-on-novel-coronavirus-2019-ncov), which primarily contains research (published and/or prepublication) articles indexed in PubMed, Web of Science, Global Index Medicus and Embase. In addition, Lanzhou University (Lanzhou, China) submits citations on a daily basis from the China National Knowledge Infrastructure (CNKI) as well as a number of Chinese journal publishers. Due to high overlap across our sources, we added a filter here to exclude records from MEDLINE and Embase.

In the original review, we also searched the US Center for Disease Control and Prevention (CDC) COVID‐19 Research Articles Downloadable Database, but this resource is no longer available. Instead, the contents of this database are now contained in both the Cochrane COVID‐19 Study Register and the WHO 'Global literature on coronavirus disease' database.

Finally, we conducted backward citation searches of systematic reviews on travel‐related control measures known to us or identified through our searches (see Appendix 2) to identify additional eligible studies. The full search strategy is presented in Appendix 3.

Selection of studies

To harmonise the screening process, we asked all review authors involved with the title and abstract screening to screen an initial set of the same 50 studies, after which we organised a group call to discuss any issues. In the original review, one review author screened all titles and abstracts, while a second review author screened only those excluded by the first review author. For this update, we screened all titles and abstracts in duplicate. The team conducted title and abstract screening using the Rayyan online systematic review software (Ouzzani 2016).

As with the title and abstract screening process, we harmonised the full‐text screening process by asking all review authors involved with full‐text screening to screen an initial same set of 10 studies (Garritty 2020). The team then discussed any open questions or issues in a group call. Subsequently, two review authors working independently each screened the remaining full‐text records in duplicate. The two review authors discussed any discrepancies, and consulted a third review author or the entire author team where necessary until they reached consensus. We recorded reasons for exclusion for all studies excluded at the full‐text screening stage.

Data extraction and management

One review author extracted study characteristics and data from all included main studies using a data extraction form in Microsoft Excel. All extracted data were checked by a second review author. We piloted the data extraction form, using three studies that represented different intervention types and that met the inclusion criteria. Appendix 4 provides the details on the data extraction categories. For studies excluded from the analysis, we extracted descriptive characteristics relating to the PICO elements, as well as a short narrative description of the results. To do so, we used a simplified version of the data extraction form used for the main studies.

In the review protocol (see Appendix 1), we specified that we would consider searching for data from external sources to enhance our understanding of the design features of the travel‐related control measures and the stage of the pandemic at the time these were implemented. However, given the lack of comprehensive reporting and the inconsistency of the information provided across these sources (e.g. discrepancies in how WHO reports described the stage of the pandemic in earlier months), and given that this information was largely not applicable to modelling studies, we decided against using these sources.

Assessment of risk of bias in included studies

One review author rated the risk of bias or the quality of each included study, depending on the type of study, and a second review author checked the judgements. The studies excluded from the review analysis were not further assessed at this stage. The team of review authors involved with assessing risk of bias and quality was largely the same for this update as for the original review. Given that one new review author was involved with this step, at the outset we discussed how to correctly and consistently apply each of the tools to one screening study and two modelling studies before beginning the assessment. These review authors discussed any questions or uncertainties that arose during the process.

Given the broad range of study designs, we applied multiple tools in assessing the risk of bias or quality of included studies, with the same tools applied in the original review and the present update. We had planned to use version 2 of the Cochrane 'Risk of bias' tool for experimental studies (Higgins 2019), and ROBINS‐I for quasi‐experimental and observational intervention studies (Sterne 2016). However, we did not identify any experimental studies. We identified two synthetic control studies, which are generally considered a type of quasi‐experimental study. However, given that ROBINS‐I was not developed for this type of quasi‐experimental study with more sophisticated statistical methods, we assessed these studies with the quality appraisal tool developed for modelling studies, as described below.

To appropriately assess the risk of bias of observational studies evaluating screening at borders, which are more closely related to diagnostic studies than intervention evaluations, we decided post‐protocol to apply the Quality Assessment of Diagnostic Accuracy Studies (QUADAS‐2) tool (Whiting 2011), as also employed in the Cochrane review on screening measures to control COVID‐19 (Viswanathan 2020). This tool comprises four domains: participant selection (i.e. passenger/traveller selection, for our purposes), the index test, the reference standard, and the flow and timing. For each of these domains, using a series of signalling questions, we provided a judgment of ‘low’, ‘unclear’ or ‘high’ risk of bias for each study. Additionally, the tool facilitates a concrete assessment of generalisability through considering how the population, index test and reference standard compares with the aspects of interest in this review. In line with QUADAS‐2 guidance, we considered how best to apply the tool to our specific review question. The tool, including the specifications we applied in making judgements, is outlined in Appendix 5.

As described in the original review (Burns 2020), no validated tool is available for assessing the risk of bias of modelling studies. Following the suggestions by (Egger 2017), we developed a bespoke tool for the assessment of modelling studies and selected criteria from a rapid review of the methodological literature (Philips 2006) and two methodological studies (Caro 2014; Egger 2017). The tool comprises the following domains: (i) model structure, (ii) input data, (iii) validation, (iv) uncertainty and (v) transparency. The individual criteria we applied, in the form of signalling questions, are outlined in Appendix 6. We reported each of the criteria separately, that is, we did not combine multiple criteria into a summary score. This also allows for a distinction between ‘fatal flaw indicators’, notably inappropriate structural assumptions and input parameters, and other aspects of model quality and credibility, such as internal and external model validation (Caro 2014).

Contacting study authors

We contacted study authors to request additional information where unclear or non‐reported aspects precluded the assessment of eligibility or inclusion in the data synthesis.

Data synthesis

Given that observational studies provide a measured estimate of effect whereas modelling studies predict such an effect, we treated these as two separate bodies of evidence in the synthesis (see also 'Assessment of certainty of evidence').

Due to substantial heterogeneity across included studies with regard to the setting, population, intervention and other contextual factors, as well as study methods, and as specified in the protocol (Appendix 1), we decided that data were not sufficiently similar to conduct meta‐analyses. We therefore synthesised the findings narratively and in tabular form, stratified by intervention type and outcome. We adhered to the 'Synthesis without meta‐analysis' (SWiM) in systematic reviews reporting guideline (Campbell 2020).

Part one of the narrative synthesis comprised four steps in moving from the effects reported at the individual study‐level to a summary across studies: (i) we created a study‐by‐study table describing the effects of interventions, as well as potential effect moderators, as estimated in each included study; (ii) we classified the effect direction for each reported intervention effect, following recent guidance (Hilton Boon 2020); (iii) for each intervention category and primary outcome, we subsequently looked across contributing studies to develop the summary of findings, including a description of the proportion of studies predicting a positive, negative or no effect for the intervention; (iv) we abstracted this summary of findings for each intervention category and primary outcome into a concise narrative summary to present, along with the certainty of the evidence, in the 'Summary of findings' table and the 'Results' section of the review, paying particular attention to sources of heterogeneity (see below).

Part two involved determining the direction of effect for each intervention‐outcome pair, which could be a positive effect, no effect, mixed effect, or negative effect. For systematic reviews of public health interventions, a beneficial effect of any size beyond the null is often considered to be potentially relevant. Additionally, for travel‐related control measures, this minimal important difference is highly context‐dependent. For example, the role of international travel in importing cases, and the associated role of travel‐related control measures in containing the pandemic, will be different in countries where community transmission is not occurring compared to countries where community transmission is widespread. Consistent with this perspective, we did not consider the size of the effect in determining effect direction.  

Specifically, we first specified the comparators used in each study (e.g. measure versus no measure or combined measure versus single measure). In determining effect direction, we classified an effect for which a better outcome was observed for the intervention condition than the comparator condition as ‘positive’, and an effect for which a worse outcome was observed for the intervention condition as ‘negative’.  Only studies in which the two conditions reported identical effect estimates were classified as ‘no effect’. Many studies assessed an intervention in multiple countries or examined a range of scenarios related to a specific intervention (e.g. in the context of high‐, moderate‐ and low‐community transmission). Where studies observed consistent effect directions across these conditions, we classified the effect direction as such; where inconsistent effect directions were observed, we classified the effect direction as ‘mixed’.

Assessment of heterogeneity and subgroup analyses

In the absence of meta‐analyses, and given the substantial heterogeneity of included studies, we did not conduct analyses of subgroups. As part of our narrative synthesis, however, we aimed to identify potential sources of heterogeneity that may have influenced intervention effectiveness. Given methodological differences, as well as differences in interventions, contexts and outcomes, for modelling studies we focused on potential moderating factors (e.g. level of community transmission, stringency of intervention, level of travel after relaxation of intervention) that were assessed within a given study. The methods used to assess these potential moderators differed widely across individual studies, however we only considered data that were assessed and clearly reported as part of a formal analysis. Given that observational studies of entry and exit screening measures were relatively homogeneous, for these studies we examined potential moderators across studies (e.g. type of screening, timing of polymerase chain reaction (PCR) testing).

Assessment of certainty of evidence

We used the GRADE approach to assess the certainty of the primary outcomes. One review author collated the evidence for each primary outcome and suggested initial certainty of evidence ratings. These were then further deliberated in a team of review authors and a joint decision for certainty of evidence ratings was made for each primary outcome.

The certainty of evidence is defined in GRADE as the extent to which one can be confident that the true effect of an intervention lies on one side of a specified threshold, or within a chosen range (Hultcrantz 2017). In the original review, as well as in this update, we considered 'difference from the null' as an important threshold, assuming that even small effect sizes may be relevant for population‐level travel‐related control measures, as noted above.

The certainty of evidence rating in GRADE yields four possible levels of evidence: high certainty (i.e. the estimated effect lies close to the true effect), moderate certainty (i.e. the estimated effect is probably close to the true effect), low certainty (i.e. the estimated effect might substantially differ from the true effect), and very low certainty (i.e. the estimated effect is probably substantially different from the true effect).

In accordance with our approach to data synthesis, we rated bodies of evidence from observational and modelling studies separately. In GRADE, evidence from RCTs enters the rating as high certainty, as does evidence from observational studies whose risk of bias has been assessed using ROBINS‐I (Schünemann 2019). Subsequently, five domains are used to downgrade evidence, including study limitations, inconsistency, indirectness, imprecision and publication bias, and three domains are used to upgrade evidence, including plausible confounding, large estimates of effect, and dose‐response relationship. The upgrading applies only when evidence has not been downgraded.

To rate the certainty of evidence from modelling studies, we used the recent guidance developed by the GRADE Working Group (Brozek 2021). As per the guidance, we initially assessed the body of evidence from modelling studies as high certainty and then used the domains described above to assess certainty of model outputs. We then applied the above domains to further downgrade or upgrade certainty of evidence from modelling studies, using tailored interpretations, as specified in the guidance. For example, risk of bias in modelling studies refers to the credibility of the model and its inputs; inconsistency assesses the difference in the results of two or more models; imprecision examines the model point estimate (e.g. predicted event) and the variability of that estimate; indirectness examines model outputs in relation to the prespecified PICO elements of interest; finally, publication bias refers to the likelihood that relevant models have been developed but not made available (Brozek 2021).

To rate the certainty of evidence from observational studies assessing screening at borders in detecting cases (i.e. the proportion of cases detected and the positive predictive value), we used the GRADE guidance for rating the certainty of evidence for diagnostic tests and strategies (Schünemann 2008). In accordance with the guidance, we initially rated the body of evidence from these cross‐sectional studies reporting an appropriate reference standard (e.g. a symptom/exposure‐based screening followed by PCR testing) as high‐certainty evidence. We then applied the five GRADE domains as described above to further downgrade evidence when deemed appropriate.