The influence of incomplete case ascertainment on measures of vaccine efficacy
Introduction
Evidence that a vaccine can help reduce burden of disease is a prerequisite for introducing the vaccine into a specific region. Because previous rotavirus vaccine candidates have failed in resource-poor settings, the World Health Organization initially delayed prequalification of both the HRV vaccine (GSK, Rotarix) and the PRV (Merck, Rotateq) until supportive clinical trial data was available from resource-poor settings [1]. This motivated our study to better understand the unexplained variation in the performance of some vaccines across different settings, for which we chose to use rotavirus as an illustrative example.
While most rotavirus infections are benign, rotavirus is nonetheless the leading cause of severe gastroenteritis among young children [2], [3], [4]. Implementation of rotavirus vaccine programs have been highly successful in high-income settings with evidence of both direct and herd protection [5], [6]; however, like many other enteric vaccines, rotavirus vaccines have been associated with lower measured vaccine efficacy (VE) in resource-poor settings with the heaviest rotavirus burden [4], [7]. The estimated VE based on the risk ratio of severe disease for RV1 in field trial in Malawi, for example, was only 49% [8]. Notwithstanding differences in the research design across studies, this is lower than estimates from South Africa (54–61%) and considerably lower than that in the US, Australia, Latin America and Europe (80–95%) [9], [10], [11], [12]. Among Aboriginal children in Australia’s Northern Territory with historically very high rates of rotavirus infection [13], lower than expected effectiveness was observed during a widespread outbreak [14].
To understand the impact of vaccination during both pre-licensure vaccine trials and post-licensure evaluation of vaccine programs, the reasons underlying the observed variation in measured VE across settings need to be better understood [1], [4], [15]. There is some evidence that this variation can be partly explained by reduced immune responses to vaccination in resource-poor settings [1], [16], with malnutrition, maternal factors (like breast milk antibodies) and enteric co-infections proposed as mechanisms [17]. Part of the variation in estimated VE may be attributable to incorrect inferences about the biological activity of vaccines based on population studies. The biological activity of a vaccine can be defined as some measure of vaccine-induced reduction in the risk of an individual acquiring infection when exposed to an infective agent. The measured VE of a vaccine might not directly measure its biological activity, since the former is measured in aggregate at the population level rather than at the individual per-exposure level, and can therefore be influenced by disease transmission, which can be dynamically affected by herd immunity and other population level factors [18].
At the population level, VE is usually expressed as a function of the comparative probability of infection among vaccinated and unvaccinated individuals. Different effect measures are not interchangeable in the context of vaccine evaluation, displaying different characteristics depending on the balance of ‘leaky’ versus ‘all-or-nothing’ protective mechanisms [18], [19], [20], [21]. A vaccine which is conceptually ‘leaky’ acts homogenously among vaccine recipients by reducing the probability of transmission of infection per potentially infectious contact; a vaccine that acts in a conceptually ‘all-or-nothing’ manner completely protects some vaccine recipients while providing no protection to the remainder. It has been demonstrated that measuring VE as a function of the cumulative incidence (i.e., 1 – risk ratio) provides a time-invariant measure of the proportion of vaccinated individuals protected for an all-or-nothing vaccine, while VE measured as a function of the proportionate reduction in cases per person-time at risk (i.e., 1 – rate ratio) provides a time-invariant measure of the reduction in the probability of transmission per infectious contact for a leaky vaccine [20], [22].
The importance of both uniform definition and ascertainment of cases has been addressed in the design of vaccine studies [23], recognising that cases of infection may be incompletely ascertained because not all are captured whether by active follow-up or by passive clinical or laboratory surveillance systems [21]. Only a fraction of rotavirus infections result in moderate to severe symptoms requiring medical evaluation, and therefore only a minority are likely to be ascertained as cases in the absence of meticulous monitoring [24]. Under-ascertainment of infection is inevitable for many vaccine preventable diseases, and yet the potential for this to influence VE measures has not been explored. We therefore extend previous theoretical work to consider the potential impact of incomplete ascertainment of infection on the archetypal vaccine models.
We established an epidemiological model that describes rotavirus outbreaks among infants where vaccines are given to half of the study population. By measuring VE as a function of either the risk ratio, rate ratio or hazard ratio, this model approximates typical randomised field trials or non-randomised case-control or cohort studies to evaluate rotavirus vaccines. By investigating how different measures of VE are sensitive not only to various biological activities of vaccine but also different levels of completeness of infection ascertainment, we explore the extent to which low estimates of VE in low-resource settings might be partly explained as artefacts of the statistical models used and analytical factors, rather than true differences in the biological activity of the vaccine.
Section snippets
Stochastic modelling of vaccine trial during rotavirus outbreak
We use discrete-time stochastic models based on a Susceptible-Infected-Recovered structure to simulate the transmission of rotavirus between individuals in a closed population. We make the assumption that outbreaks occur over such a brief period (weeks to months) that births, deaths, inward and outward migrations, and waning immunity are negligible. If the average duration of infectiousness 1/γ is taken to be five days and we assume that the basic reproduction number R0 = 3, then the
Results
While the aggregated number of infected infants increases, the disease dynamic was consistent as the population size increases under both the leaky and all-or-nothing models (Fig. 2), so we fixed our study population size at N = 10,000. In observational studies, an outbreak might not be identified if only a few infections are ascertained. However, with a high force of infection (R0 = 3) the chance of failing to meet this threshold is low (<0.4%, see Table 2) even if the proportion of infections
Discussion
The observed heterogeneity in the performance of a number of vaccines like rotavirus vaccine across different settings are poorly understood but often attributed to reduced immune responses in resource-poor settings [1]. We developed a stochastic epidemiological model to investigate non-immune factors that potentially contribute to this observation. We replicated earlier work, which showed how VE measures are not interchangeable in regard to the vaccine’s leaky and all-or-nothing protective
Conclusion
The measured effect of a vaccine can vary depending on the choice of effect measure and a range of epidemiological, vaccine-related and logistical conditions, including the proportion of infections ascertained as cases. By demonstrating this concept using a stochastic transmission model, we may partly explain the observed heterogeneity in the performance of rotavirus vaccine across different settings, which represents an example that should be generalisable to many other infectious disease
Conflicts of interests
None declared.
Author contributions
Y.W., T.L.S. and J.A.M. developed the model. Y.W. performed analysis and wrote the manuscript; E.S.M. contributed to the interpretation of results; T.L.S. initiated and supervised the project. All authors made substantial contribution to manuscript writing and approved the submission of the manuscript.
Acknowledgement
This work was funded by Telethon Kids Institute. T.L.S. was supported by a National Health and Medical Research Council fellowship [CDF1111657].
References (31)
- et al.
2008 estimate of worldwide rotavirus-associated mortality in children younger than 5 years before the introduction of universal rotavirus vaccination programmes: a systematic review and meta-analysis
Lancet Infect Dis
(2012) - et al.
Impact of high coverage of monovalent human rotavirus vaccine on Emergency Department presentations for rotavirus gastroenteritis
Vaccine
(2015) - et al.
Effectiveness of monovalent human rotavirus vaccine against admission to hospital for acute rotavirus diarrhoea in South African children: a case-control study
Lancet Infect Dis
(2014) - et al.
On the distribution of vaccine protection under heterogeneous response
Math Biosci
(1993) - et al.
Estimation of vaccine efficacy from epidemics of acute infectious agents under vaccine-related heterogeneity
Math Biosci
(1993) - et al.
Performance of rotavirus vaccines in developed and developing countries
Human Vaccines
(2010) - et al.
Global, regional, and national estimates of rotavirus mortality in children <5 years of age, 2000–2013
Clin Infect Dis
(2016) - et al.
Use of formative research in developing a knowledge translation approach to rotavirus vaccine introduction in developing countries
BMC Public Health
(2007) - et al.
Evidence of herd immunity and sustained impact of rotavirus vaccination on the reduction of rotavirus-related medical encounters among infants from 2006 through 2011 in the United States
Pediatr Infect Dis J
(2015) - et al.
Update on rotavirus vaccines
Pediatr Infect Dis J
(2005)
Effect of human rotavirus vaccine on severe diarrhea in african infants
N Engl J Med
Rotavirus and the indigenous children of the australian outback: monovalent vaccine effective in a high-burden setting
Clin Infect Dis
Uptake, impact, and effectiveness of rotavirus vaccination in the United States: review of the first 3 years of postlicensure data
Pediatr Infect Dis J
Long-term consistency in rotavirus vaccine protection: RV5 and RV1 vaccine effectiveness in US children, 2012–2013
Clin Infect Dis
Burden of severe rotavirus disease in Australia
J Paediatr Child Health
Cited by (6)
Role of modelling in COVID-19 policy development
2020, Paediatric Respiratory ReviewsCitation Excerpt :As of 2nd June, 2020, there are ten clinical trials for COVID-19 vaccines in different stages of development and 123 molecules in preclinical evaluations [29]. Modelling can be useful in evaluating the efficacy of vaccines during clinical trials and diminishing biases [30,31]. Models have already shown that an immunity of 60% is desirable to protect those who cannot or choose not to become vaccinated.
Challenges of vaccine effectiveness and waning studies
2019, Clinical Infectious DiseasesAnalyzing Vaccine Trials in Epidemics with Mild and Asymptomatic Infection
2019, American Journal of EpidemiologyEffectiveness of monovalent rotavirus vaccine in the Philippines
2018, Scientific Reports