Elsevier

Vaccine

Volume 36, Issue 21, 17 May 2018, Pages 2946-2952
Vaccine

The influence of incomplete case ascertainment on measures of vaccine efficacy

https://doi.org/10.1016/j.vaccine.2018.04.046Get rights and content

Abstract

Background

Motivated by the unexplained variation in the performance of some vaccines across different settings, we extend previous theoretical work to consider the potential impact of incomplete case ascertainment on measures of vaccine efficacy (VE), which is more likely in subclinical or clinically unimportant infections, such as rotavirus gastroenteritis.

Methods

By simulating the measurement of VE under outbreak conditions using a discrete time stochastic SIR model, we compare three commonly used measures, VERisk, VERate, and VEHazard, calculated respectively based on risk ratio, rate ratio and hazard ratio of disease. We investigate how these measures are influenced by factors such as biological activity, action mechanism of vaccine, proportion of cases ascertained, and underlying force of infection.

Results

Under plausibly low levels of ascertainment, the group with the most infections, and therefore the most missed cases, has the most falsely inflated denominator, producing similar rates in the control and intervention groups. As a result, VERate and VEHazard will underestimate the true VE compared to high case ascertainment scenarios. Furthermore, the extent of underestimation is greater for leaky vaccine models with lower biological protective effects and under conditions which are conducive to high transmission.

Conclusions

This study demonstrates that a biologically active vaccine may produce a low measured VE under a range of epidemiological, vaccine-related and logistical conditions. Low case ascertainment may partly explain the observed heterogeneity in the performance of rotavirus vaccine across different settings, and should be considered in the design and interpretation of future field trials.

Introduction

Evidence that a vaccine can help reduce burden of disease is a prerequisite for introducing the vaccine into a specific region. Because previous rotavirus vaccine candidates have failed in resource-poor settings, the World Health Organization initially delayed prequalification of both the HRV vaccine (GSK, Rotarix) and the PRV (Merck, Rotateq) until supportive clinical trial data was available from resource-poor settings [1]. This motivated our study to better understand the unexplained variation in the performance of some vaccines across different settings, for which we chose to use rotavirus as an illustrative example.

While most rotavirus infections are benign, rotavirus is nonetheless the leading cause of severe gastroenteritis among young children [2], [3], [4]. Implementation of rotavirus vaccine programs have been highly successful in high-income settings with evidence of both direct and herd protection [5], [6]; however, like many other enteric vaccines, rotavirus vaccines have been associated with lower measured vaccine efficacy (VE) in resource-poor settings with the heaviest rotavirus burden [4], [7]. The estimated VE based on the risk ratio of severe disease for RV1 in field trial in Malawi, for example, was only 49% [8]. Notwithstanding differences in the research design across studies, this is lower than estimates from South Africa (54–61%) and considerably lower than that in the US, Australia, Latin America and Europe (80–95%) [9], [10], [11], [12]. Among Aboriginal children in Australia’s Northern Territory with historically very high rates of rotavirus infection [13], lower than expected effectiveness was observed during a widespread outbreak [14].

To understand the impact of vaccination during both pre-licensure vaccine trials and post-licensure evaluation of vaccine programs, the reasons underlying the observed variation in measured VE across settings need to be better understood [1], [4], [15]. There is some evidence that this variation can be partly explained by reduced immune responses to vaccination in resource-poor settings [1], [16], with malnutrition, maternal factors (like breast milk antibodies) and enteric co-infections proposed as mechanisms [17]. Part of the variation in estimated VE may be attributable to incorrect inferences about the biological activity of vaccines based on population studies. The biological activity of a vaccine can be defined as some measure of vaccine-induced reduction in the risk of an individual acquiring infection when exposed to an infective agent. The measured VE of a vaccine might not directly measure its biological activity, since the former is measured in aggregate at the population level rather than at the individual per-exposure level, and can therefore be influenced by disease transmission, which can be dynamically affected by herd immunity and other population level factors [18].

At the population level, VE is usually expressed as a function of the comparative probability of infection among vaccinated and unvaccinated individuals. Different effect measures are not interchangeable in the context of vaccine evaluation, displaying different characteristics depending on the balance of ‘leaky’ versus ‘all-or-nothing’ protective mechanisms [18], [19], [20], [21]. A vaccine which is conceptually ‘leaky’ acts homogenously among vaccine recipients by reducing the probability of transmission of infection per potentially infectious contact; a vaccine that acts in a conceptually ‘all-or-nothing’ manner completely protects some vaccine recipients while providing no protection to the remainder. It has been demonstrated that measuring VE as a function of the cumulative incidence (i.e., 1 – risk ratio) provides a time-invariant measure of the proportion of vaccinated individuals protected for an all-or-nothing vaccine, while VE measured as a function of the proportionate reduction in cases per person-time at risk (i.e., 1 – rate ratio) provides a time-invariant measure of the reduction in the probability of transmission per infectious contact for a leaky vaccine [20], [22].

The importance of both uniform definition and ascertainment of cases has been addressed in the design of vaccine studies [23], recognising that cases of infection may be incompletely ascertained because not all are captured whether by active follow-up or by passive clinical or laboratory surveillance systems [21]. Only a fraction of rotavirus infections result in moderate to severe symptoms requiring medical evaluation, and therefore only a minority are likely to be ascertained as cases in the absence of meticulous monitoring [24]. Under-ascertainment of infection is inevitable for many vaccine preventable diseases, and yet the potential for this to influence VE measures has not been explored. We therefore extend previous theoretical work to consider the potential impact of incomplete ascertainment of infection on the archetypal vaccine models.

We established an epidemiological model that describes rotavirus outbreaks among infants where vaccines are given to half of the study population. By measuring VE as a function of either the risk ratio, rate ratio or hazard ratio, this model approximates typical randomised field trials or non-randomised case-control or cohort studies to evaluate rotavirus vaccines. By investigating how different measures of VE are sensitive not only to various biological activities of vaccine but also different levels of completeness of infection ascertainment, we explore the extent to which low estimates of VE in low-resource settings might be partly explained as artefacts of the statistical models used and analytical factors, rather than true differences in the biological activity of the vaccine.

Section snippets

Stochastic modelling of vaccine trial during rotavirus outbreak

We use discrete-time stochastic models based on a Susceptible-Infected-Recovered structure to simulate the transmission of rotavirus between individuals in a closed population. We make the assumption that outbreaks occur over such a brief period (weeks to months) that births, deaths, inward and outward migrations, and waning immunity are negligible. If the average duration of infectiousness 1/γ is taken to be five days and we assume that the basic reproduction number R0 = 3, then the

Results

While the aggregated number of infected infants increases, the disease dynamic was consistent as the population size increases under both the leaky and all-or-nothing models (Fig. 2), so we fixed our study population size at N = 10,000. In observational studies, an outbreak might not be identified if only a few infections are ascertained. However, with a high force of infection (R0 = 3) the chance of failing to meet this threshold is low (<0.4%, see Table 2) even if the proportion of infections

Discussion

The observed heterogeneity in the performance of a number of vaccines like rotavirus vaccine across different settings are poorly understood but often attributed to reduced immune responses in resource-poor settings [1]. We developed a stochastic epidemiological model to investigate non-immune factors that potentially contribute to this observation. We replicated earlier work, which showed how VE measures are not interchangeable in regard to the vaccine’s leaky and all-or-nothing protective

Conclusion

The measured effect of a vaccine can vary depending on the choice of effect measure and a range of epidemiological, vaccine-related and logistical conditions, including the proportion of infections ascertained as cases. By demonstrating this concept using a stochastic transmission model, we may partly explain the observed heterogeneity in the performance of rotavirus vaccine across different settings, which represents an example that should be generalisable to many other infectious disease

Conflicts of interests

None declared.

Author contributions

Y.W., T.L.S. and J.A.M. developed the model. Y.W. performed analysis and wrote the manuscript; E.S.M. contributed to the interpretation of results; T.L.S. initiated and supervised the project. All authors made substantial contribution to manuscript writing and approved the submission of the manuscript.

Acknowledgement

This work was funded by Telethon Kids Institute. T.L.S. was supported by a National Health and Medical Research Council fellowship [CDF1111657].

References (31)

  • S.A. Madhi et al.

    Effect of human rotavirus vaccine on severe diarrhea in african infants

    N Engl J Med

    (2010)
  • T.L. Snelling et al.

    Rotavirus and the indigenous children of the australian outback: monovalent vaccine effective in a high-burden setting

    Clin Infect Dis

    (2009)
  • J.E. Tate et al.

    Uptake, impact, and effectiveness of rotavirus vaccination in the United States: review of the first 3 years of postlicensure data

    Pediatr Infect Dis J

    (2011)
  • D.C. Payne et al.

    Long-term consistency in rotavirus vaccine protection: RV5 and RV1 vaccine effectiveness in US children, 2012–2013

    Clin Infect Dis

    (2015)
  • A.T. Newall et al.

    Burden of severe rotavirus disease in Australia

    J Paediatr Child Health

    (2006)
  • Cited by (6)

    View full text