Influenza vaccine effectiveness by test-negative design – Comparison of inpatient and outpatient settings

doi:10.1016/j.vaccine.2016.02.039

Vaccine

Volume 34, Issue 14, 29 March 2016, Pages 1672-1679

https://doi.org/10.1016/j.vaccine.2016.02.039 Get rights and content

Highlights

•
We found similar VE estimates in test-negative studies done in hospitals versus outpatient clinics.
•
Influenza vaccination coverage was much higher in hospital controls compared to outpatient controls.
•
Influenza virus was more commonly detected among outpatients compared to inpatients, in test-negative studies.

Abstract

Background

Observational studies of influenza vaccine effectiveness (VE) are increasingly using the test-negative design. Studies are typically based in outpatient or inpatient settings, but these two approaches are rarely compared directly. The aim of our study was to assess whether influenza VE estimates differ between inpatient and outpatient settings.

Methods

We searched the literature from Medline, PubMed and Web of Science using a combination of keywords to identify published studies of influenza VE using the test-negative design. Studies assessing any type of influenza vaccine among any population in any setting were considered, while interim studies or re-analyses were excluded. Retrieved articles were reviewed, screened and categorized based on study setting, location and influenza season. We searched for parallel studies in inpatient and outpatient settings that were done in the same influenza season, in the same location, and in the same or similar age groups. For each of the pairs identified, we estimated the difference in VE estimates between settings, and we tested whether the average difference was significant using a paired t-test.

Results

In total 25 pairs of estimates were identified that permitted comparisons between VE estimates in inpatient and outpatient study settings. Within pairs, the prevalence of influenza was generally higher among patients enrolled in the outpatient studies, while influenza vaccination coverage among the test-negative control groups was generally higher in the inpatient studies. There was no heterogeneity in the paired differences in VE, and the pooled difference in VE between inpatient and outpatient studies was −2% (95% confidence interval: −12%, 10%).

Conclusions

We found no differences in VE estimates between inpatient and outpatient settings by studies using the test-negative design. Further research involving direct comparisons of VE estimates from the two settings in the same populations and years would be valuable.

Introduction

Influenza viruses are associated with a substantial disease burden of both medically attended ambulatory care and hospitalizations [1], [2]. Vaccination is the best means of preventing influenza virus infections, but influenza vaccine effectiveness (VE) may differ from year to year and among different populations. Recently, there have been increasing numbers of studies estimating influenza VE using the test-negative design [3]. In this study design, patients are enrolled in outpatient clinics and/or hospitals based on a clinical case definition such as acute respiratory illness (ARI) or other syndromes consistent with influenza virus infections. Patients are then tested for influenza virus, and VE is estimated from the odds ratio comparing the odds of vaccination among patients testing positive for influenza versus those testing negative, adjusting for potential confounding factors. This study design is believed to be valid under a range of scenarios [4], [5]. Importantly, this design is easy to implement in both inpatient and outpatient settings.

Estimates obtained from inpatient and outpatient settings in the same population may be expected to differ for several reasons. First, patients presenting to hospitals may present later in infection, may be older and may be more likely to be co-infected with another respiratory virus. There may therefore be a greater number of false negatives due to reduced viral shedding with time and age [6]. Such reduced sensitivity in case ascertainment can result in attenuation of the odds ratio [3]. In addition, patients at the highest risk of hospitalization, if infected, may be less protected by the vaccine because of poorer VE in people of older age [7] or immunosuppression as a result of chronic underlying conditions [8]. Furthermore, VE estimates between settings may also differ according to vaccine type or brand used. However, few previous studies have directly compared estimates in hospital-based studies with those from outpatient-based studies [9], [10].

The aim of this study was to compare directly the VE estimates obtained from studies based in hospitalized patients with studies that recruited patients in an outpatient setting, using the test-negative design. Because VE can vary from location to location and from year to year, and by age, we intended to focus on comparisons of VE estimates from the same location and influenza season and in the same or similar age groups.

Section snippets

Study search and selection

We previously conducted a review of test-negative studies of influenza VE [3]. All papers identified as meeting the search criteria in that study were also included in the present analysis. The first online searches were updated on 22 July 2015. A second search was done on 28 December 2015. Papers were searched on Medline, PubMed and Web of Science for the following key words:

1.
“Influenza” OR “flu”.
2.
“Vaccine effectiveness” OR “VE”.
3.
“Test-negative” OR “test negative” OR “case-control” OR “case

Results

A total of 85 full-text articles were retrieved from the previous systematic review [3], to which a further 35 articles were added from the updated search to give a total of 120 published test-negative studies. Grouping studies by location, season, age group and VE estimates by type/subtype, we identified 8 study pairs/triplet from a total of 14 publications [9], [10], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23] (Table 1, Fig. 1). These 8 pairs/triplet included VE

Discussion

From 120 articles assessing influenza VE using the test-negative design, we identified 14 publications with suitable information for paired comparison of VE estimates between inpatient and outpatient settings. Based on 25 pairs of VE estimates for 5 countries from 2010 to 2013, despite some absolute differences within many pairs, we found no evidence of substantial statistical difference in the VE estimates in the inpatient study and the outpatient study, with pooled ΔVE = −2% (95%CI: −12%, 10%).

Funding

This project was supported by a commissioned grant from the Health and Medical Research Fund, Food and Health Bureau, Government of the Hong Kong Special Administrative Region (grant no. HK-13-04-04), the Harvard Center for Communicable Disease Dynamics from the National Institute of General Medical Sciences (grant no. U54 GM088558), and a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (Project No. T11-705/14N), The WHO Collaborating Centre for

Acknowledgments

We would like to thank the authors of some of the original articles included in our review who agreed to provide additional information. We also thank Eric Lau and Nan Xia for technical support.

Conflicts of interest

BJC has received research funding from MedImmune Inc. and Sanofi Pasteur, and consults for Crucell NV. The authors report no other potential conflicts of interest.

References (28)

M.L. Jackson et al.
The test-negative design for estimating influenza vaccine effectiveness
Vaccine
(2013)
I.M. Foppa et al.
The case test-negative design for studies of the effectiveness of influenza vaccine
Vaccine
(2013)
A.C. Cheng et al.
Effectiveness of H1N1/09 monovalent and trivalent influenza vaccines against hospitalization with laboratory-confirmed H1N1/09 influenza in Australia: a test-negative case control study
Vaccine
(2011)
A. Levy et al.
Influenza vaccine effectiveness estimates for Western Australia during a period of vaccine and virus strain stability, 2010 to 2012
Vaccine
(2014)
K.M. Neuzil et al.
The effect of influenza on hospitalizations, outpatient visits, and courses of antibiotics in children
N Engl J Med
(2000)
G.O. Emukule et al.
The burden of influenza and RSV among inpatients and outpatients in rural western Kenya, 2009–2012
PLOS ONE
(2014)
S.G. Sullivan et al.
Potential of the test-negative design for measuring influenza vaccine effectiveness: a systematic review
Expert Rev Vaccines
(2014)
F. Carrat et al.
Time lines of infection and disease in human influenza: a review of volunteer challenge studies
Am J Epidemiol
(2008)
A. Gruver et al.
Immunosenescence of ageing
J Pathol
(2007)
Guide CI et al.
(1993)

N. Turner et al.

Effectiveness of seasonal trivalent inactivated influenza vaccine in preventing influenza hospitalisations and primary care visits in Auckland, New Zealand, in 2013

Euro Surveill

(2014)

I. Martínez-Baz et al.

Influenza vaccine effectiveness in preventing inpatient and outpatient cases in a season dominated by vaccine-matched influenza B virus

Hum Vaccines Immunother

(2015)

J. Carpenter et al.

Bootstrap confidence intervals: when, which, what? A practical guide for medical statisticians

Stat Med

(2000)

J.C. Kwong et al.

Vaccine effectiveness against laboratory-confirmed influenza hospitalizations among elderly adults during the 2010–2011 season

Clin Infect Dis

(2013)

Cited by (49)

Real-world effectiveness of seasonal influenza vaccination and age as effect modifier: A systematic review, meta-analysis and meta-regression of test-negative design studies
2024, Vaccine
Under the global risk of epidemic rebound of influenza after COVID-19 outbreak, the study aimed to provide a comprehensive evaluation of the seasonal influenza vaccine effectiveness (IVE) and to explore the potential effect modifiers.
We searched for test-negative design studies with IVE estimates published between January 1, 2017 and December 31, 2022. We estimated pooled IVE using random-effects meta-analysis, and conducted meta-regression with study site, age, sex and comorbidity as explanatory variables.
We identified 2429 publications and included 191 in the meta-analysis. The pooled IVE was 41.4 % (95 % CI: 39.2–43.5 %) against any influenza. For specific strains, the IVE was 55.4 % (95 % CI: 52.7–58.1 %) against A/H1N1, 26.8 % (95 % CI: 23.5–29.9 %) against A/H3N2, 47.2 % (95 % CI: 38.1–54.9 %) against B/Yamagata, and 40.6 % (95 % CI: 23.7–53.7 %) against B/Victoria, and the effectiveness against A/H3N2 was significantly lower than A/H1N1 (p < 0.0001) and B/Yamagata (p < 0.0001). The pooled IVE was 39.2 % (95 % CI: 36.5–41.9 %) in preventing influenza-associated outpatient visit and 43.7 % (95 % CI: 39.7–47.4 %) in preventing influenza-related hospitalization. The IVE against any influenza was 48.6 % (95 % CI: 44.7–52.2 %) for children aged < 18 years, 36.7 % (95 % CI: 31.9–41.1 %) for adults aged 18–64 years, and 30.6 % (95 % CI: 26.2–34.8 %) for elderly aged ≥65 years. Meta-regression revealed that the IVE was associated with the average age of study participants, in which both young adults [relative odds ratio (ROR) = 1.225, 95 % confidence interval (CI): 1.099–1.365, p = 0.0002] and elderly (ROR = 1.245, 95 % CI: 1.083–1.431, p = 0.002) manifested a significantly decreased effectiveness compared with children.
Influenza vaccines provided moderate protection against laboratory-confirmed influenza and related outpatient visit and hospitalization. However, the effectiveness may vary substantially by virus type and age group, suggesting the necessity to tailor vaccination strategies especially for older individuals and against the A/H3N2 strain, and to promote annual immunization and annual analysis of vaccine effectiveness.
Influenza vaccine effectiveness in preventing hospital encounters for laboratory-confirmed infection among Italian adults, 2022/23 season
2023, Vaccine
The effectiveness of seasonal influenza vaccination (SIV) varies from year to year. Interim estimates of vaccine effectiveness (VE) in outpatient settings have suggested that the 2022/23 northern hemisphere SIV was 54 % effective. The main goal of this study was to measure the 2022/23 SIV VE among Italian adults in a hospital setting. The study adopted a retrospective test-negative case-control design and was conducted in a large tertiary hospital (Genoa, Italy) between October 2022 and April 2023. Adult (≥18 years) patients accessing the hospital’s Emergency Department for symptoms ascribable to an acute respiratory infection, for which a reverse-transcription real-time polymerase chain reaction test for the detection of influenza virus was prescribed, were potentially eligible. Of 33,692 referrals assessed, 487 patients were included in the study. A total of 13 % of patients were positive for influenza, most of which (63 %) belonged to the A(H3N2) subtype. SIV VE was 57 % (95 % CI: 11–81 %), 53 % (95 % CI: 2–80 %) and 38 % (95 % CI: −34–74 %) against any influenza, influenza A and A(H3N2), respectively. Although no cases caused by A(H1N1)pdm09 and B strains were observed among vaccinated individuals, VE estimates against the latter were imprecise, owing to their low detection rates. In conclusion, the 2022/23 SIV was moderately effective against hospital encounters for laboratory-confirmed influenza.
Influence of disease attenuation on relative influenza vaccine effectiveness by vaccine type
2022, Vaccine
Benefit conferred by “enhanced” influenza vaccines is often measured by relative vaccine effectiveness, (rVE), which compares disease risk among groups of people who received alternative vaccines. Differences in attenuation of illness severity by vaccine types could manifest as differences in rVE. Using a simulated VE study and cohort of adults aged ≥ 65 years, we examined how rVE varied with assumptions about attenuation of disease severity conferred by standard and enhanced vaccines and how this variation could lead to differing estimates of rVE for prevention of moderate (i.e., outpatient) versus severe (i.e., inpatient) influenza illness. We found that if enhanced vaccines attenuated severe illness more than moderate illness, then rVE observed against severe disease could be higher than rVE observed against moderate disease. Thus, if differences in disease attenuation by vaccine type occurs, estimates of rVE may vary for influenza outcomes of differing levels of severity.
Immune-mediated attenuation of influenza illness after infection: opportunities and challenges
2021, The Lancet Microbe
Citation Excerpt :
A few studies have found higher point estimates of vaccine effectiveness against critical influenza illness (ie, requiring intensive care unit [ICU] admission) versus non-critical illness.19,54 Other studies have not found differences in vaccine effectiveness between inpatient and outpatient settings.17,18 In some studies of patients admitted to hospital with influenza, seasonal vaccination was associated with reduced length of stay and risk of critical illness.16,19,54
Sterilising immunity that blocks infection for life, and thus prevents illness after infection, is the ultimate goal for vaccines. Neither influenza infection nor vaccination provide sterilising immunity. Mutations during influenza viral genome replication result in the emergence of viruses that evade immunity and cause reinfections. Waning of immunity also results in reinfections to homologous influenza viruses. However, immunity might limit the severity of disease after infection or vaccination (ie, immunoattenuation). We provide a comprehensive examination of experimental and observational peer reviewed evidence since 1933, when the first influenza virus was isolated, on whether immunity blocks subsequent infection or attenuates illness. Although an abundance of experimental evidence supports immunoattenuation, clinical evidence is rudimentary and conflicting. To the extent that immunoattenuation occurs, understanding the varied pathways to illness, pathogenesis, clinical manifestations, and correlates of attenuation can improve the design and evaluation of influenza vaccines. By elucidating the mechanisms of immunoattenuation and phenotypes of illness, we clarify ambiguities and identify unmet needs that, if addressed with priority, could strategically improve the design of vaccines for the prevention of influenza.
Variable seasonal influenza vaccine effectiveness across geographical regions, age groups and levels of vaccine antigenic similarity with circulating virus strains: A systematic review and meta-analysis of the evidence from test-negative design studies after the 2009/10 influenza pandemic
2021, Vaccine
We examined the influence of some factors on seasonal influenza vaccine effectiveness (VE) from test-negative design (TND) studies.
We systematically searched for full-text publications of VE against laboratory-confirmed influenza from TND studies in outpatient settings after the 2009/10 influenza pandemic. Two reviewers independently selected and extracted data from the included studies. We calculated pooled adjusted VE across geographical regions, age groups and levels of vaccine antigenic similarity with circulating virus strains, using an inverse variance, random-effects model.
We included 76 full-text articles from 11,931 citations. VE estimates against A(H1N1)pdm09, A(H3N2), influenza B, and all influenza were homogenous and point pooled VE higher in the Southern hemisphere compared with the Northern hemisphere. The difference in pooled VE between the Southern and Northern hemispheres was statistically significant for A(H3N2), influenza B, and all influenza. A consistent pattern was observed in pooled VE across both hemispheres and continents, with the highest point pooled VE being against A(H1N1)pdm09, followed by influenza B, and lowest against A(H3N2). A nearly consistent pattern was observed in pooled VE across age groups in the Northern hemisphere, with pooled VE mostly decreasing with age. Point pooled VE against A(H3N2), influenza B, and all influenza were statistically significantly higher when vaccine was antigenically similar to circulating virus strains compared with when antigenically dissimilar. Similar pattern was observed in the Northern hemisphere, but there was a lack of data from the Southern hemisphere.
Consistent patterns appear to exist in seasonal influenza VE across regions, age groups, and levels of vaccine antigenic similarity with circulating virus strains, with best vaccine performance against A(H1N1)pdm09 and worst against A(H3N2). The evidence highlights the need to consider geographical location, age, and vaccine antigenic similarity with circulating virus strains when designing and evaluating influenza VE studies.
Influenza vaccine effectiveness against influenza-associated hospitalization in children: A systematic review and meta-analysis
2020, Vaccine
Citation Excerpt :
Also, despite the additional data from recent influenza seasons in our meta-analysis, there was significant overlap of influenza seasons (2010–2015) between the studies included in our analysis and those included in the analysis by Belognia et al. [10]. Thus, results concordance may support the view that inpatient and outpatient influenza VE estimates tend to produce similar estimates [49]. However, these findings need to be investigated further, ideally within the same population and during the same influenza seasons.
Vaccination remains the most effective way to prevent influenza infection, albeit vaccine effectiveness (VE) varies by year. Compared to other age groups, children and elderly adults have the highest risk of developing influenza-related complications and requiring hospitalization. During the last years, “test negative design” (TND) studies have been implemented in order to estimate influenza VE. The aim of this systematic review and meta-analysis was to summarize the findings of TND studies reporting influenza VE against laboratory-confirmed influenza-related hospitalization in children aged 6 months to 17 years. We searched the PubMed and Embase databases and identified 2615 non-duplicate studies that required detailed review. Among them, 28 met our inclusion criteria and we performed a random-effects meta-analysis using adjusted VE estimates. In our primary analysis, influenza vaccine offered significant protection against any type influenza-related hospitalization (57.48%; 95% CI 49.46–65.49). When we examined influenza VE per type and strain, VE was higher against H1N1 (74.07%; 95% CI: 54.85–93.30) and influenza B (50.87%; 95% CI: 41.75–59.98), and moderate against H3N2 (40.77%; 95% CI: 25.65–55.89). Notably, influenza vaccination offered higher protection in children who were fully vaccinated (61.79%; 95% CI: 54.45–69.13), compared to those who were partially vaccinated (33.91%; 95% CI: 21.12 – 46.69). Also, influenza VE was high in children less than 5 years old (61.71%; 95% CI: 49.29–74.12) as well as in children 6–17 years old (54.37%; 95% CI: 35.14–73.60). In conclusion, in the pediatric population, influenza vaccination offered significant protection against influenza-related hospitalization and complete annual vaccination should be encouraged.

View all citing articles on Scopus

View full text

Influenza vaccine effectiveness by test-negative design – Comparison of inpatient and outpatient settings

Highlights

Abstract

Background

Methods

Results

Conclusions

Introduction

Section snippets

Study search and selection

Results

Discussion

Funding

Acknowledgments

Vaccine

Vaccine

Vaccine

Vaccine

The effect of influenza on hospitalizations, outpatient visits, and courses of antibiotics in children

N Engl J Med

The burden of influenza and RSV among inpatients and outpatients in rural western Kenya, 2009–2012

PLOS ONE

Potential of the test-negative design for measuring influenza vaccine effectiveness: a systematic review

Expert Rev Vaccines

Time lines of infection and disease in human influenza: a review of volunteer challenge studies

Am J Epidemiol

Immunosenescence of ageing

J Pathol

Effectiveness of seasonal trivalent inactivated influenza vaccine in preventing influenza hospitalisations and primary care visits in Auckland, New Zealand, in 2013

Euro Surveill

Influenza vaccine effectiveness in preventing inpatient and outpatient cases in a season dominated by vaccine-matched influenza B virus

Hum Vaccines Immunother

Bootstrap confidence intervals: when, which, what? A practical guide for medical statisticians

Stat Med

Vaccine effectiveness against laboratory-confirmed influenza hospitalizations among elderly adults during the 2010–2011 season

Clin Infect Dis