Are cleaner cooking solutions clean enough? A systematic review and meta-analysis of particulate and carbon monoxide concentrations and exposures

2.1.1. Population

2.1.2. Interventions

2.1.3. Comparison

2.1.4. Outcome

Titles and abstracts and literature identified from unpublished sources (grey literature and websites) were checked for relevance by two of the review team (N F and R L). The full texts of those potentially meeting the inclusion criteria were screened by four of the review team (N F, R L, K J, and M J) and independently assessed by an additional reviewer (D P). Those clearly not meeting the inclusion criteria were excluded. At each stage of filtering 20% verification was undertaken independently by two reviewers with any discordant decisions being reconciled amongst the review team.

Data extraction was conducted using bespoke pre-piloted forms recording author, year, country, season (where applicable), baseline stove and fuel, intervention stove and fuel, study design, duration of follow up, number of homes assessed, analytical or sampling approach, sampling period, baseline and post-intervention data on kitchen concentrations of and personal exposure to PM and CO (mean and standard deviation, where available). One reviewer (one of N F, R L, K J and M J) completed each data extraction form with 20% verification through double extraction with another reviewer-discrepancies being resolved through discussion.

Quality appraisal was assessed using tools developed by the Department of Public Health and Policy, University of Liverpool (Liverpool Quality Assessment Tools (LQATs)) (Pope et al 2016). The tools have been used extensively across a wide range of health outcomes (Rehfuess et al 2013, Bruce et al 2015, Garn et al 2017, Ronzi et al 2018) and were the quality assessment instrument selected for systematic review chapters informing the WHO guidelines (Bruce et al 2014, Diekman et al 2014, WHO 2014). The LQATs allow appraisal of multiple study designs and assign a star rating for methodological quality relating to reduced risk of bias (selection, response, follow-up and measurement), risk of confounding (balance of randomization and adjustment) and increased accuracy of measurements of exposure and outcome. The star rating can then be used to assess a quality percentage based on the number of awarded stars out of the total available for a specific design. Quality appraisal was undertaken independently by two reviewers (N F and R L) for each included study.

Meta-analyses were conducted for the four distinct HAP outcomes: kitchen PM concentrations (in mg m⁻³), kitchen CO concentrations (in ppm), personal PM exposure (mg m⁻³) and personal CO exposure (ppm). Pooling of data required both estimates (means and standard deviations) of baseline concentrations/exposure and intervention concentrations/exposure. Standard deviations were calculated from confidence intervals or p-values if they were not readily available. Medians and geometric means were infrequently reported by the studies and were not included in the synthesis, despite often skewed distributions. Generic inverse variance meta-analyses using the Cochrane Collaboration's RevMan software version 5 (www.cc-ims.net/RevMan) were used to obtain pooled absolute mean differences (e.g. intervention levels minus baseline levels) and confidence intervals for each outcome. Statistical heterogeneity for each meta-analysis was assessed through the I-squared test with random-effects meta-analyses (Der Simonian and Laird 1986) being used for a test value greater than 20%.

In total seven meta-analyses were possible for each of the four HAP outcomes based on cleaner technology (n = 4) and clean fuel (n = 3) intervention categories, where data was sufficient for pooling. Each meta-analysis was presented in the form of a Forest Plot and was assessed for potential publication bias through measurement of funnel plot asymmetry (this was done graphically and statistically with Begg's (Begg and Mazumdar 1994) and Egger's (Egger et al 1997) hypothesis testing).

Additional quantitative syntheses of intervention impacts were conducted including (a) calculations of percentage reductions in HAP (based on weighted mean reductions of the four HAP outcomes) for each of the seven intervention groups and (b) plots of baseline (no intervention) and intervention levels of the four HAP outcomes against a reference for the WHO iar quality guideline (AQG) level for PM_2.5 (IT1 35 μg m⁻³) and for CO (24 h target of 7 mg m⁻³ converted to a value of 6.11 ppm using the formula ppm= (mg m⁻³ value) × (24.45)/(molecular weight = 28.01), and assuming a temperature of 25 °C and a pressure of 1 atmosphere) for each of the seven intervention groups.

The systematic review process yielded a total of 50 studies (168 estimates) to be included in the evidence synthesis (figure 1). The number of studies included in this review (19 additional studies (59 estimates)) increased by 60% from the previous review by Pope et al (2017) (31 studies (109 estimates)).

Zoom In Zoom Out Reset image size

In terms of the four HAP outcomes, for kitchen concentrations there were 39 studies (73 estimates) with PM_2.5 and 32 studies (59 estimates) with CO; and for personal exposure there were 13 studies (20 estimates) for PM_2.5 and 12 studies (16 estimates) with CO. The study characteristics for each intervention type with information relating to the context of HAP measurement including country, study season, design, duration, parameters measured and quality assessment are presented in tables 1–7.

Results from the meta-analyses are displayed according to the four primary HAP outcomes, stratified by intervention category, in figures 2–5. The impacts on the four HAP outcomes of the cleaner cooking technologies and fuels are described below for each intervention category.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

2.5.1. 'Improved cookstoves' without venting (ICS—no vent)

Nine studies (six in Africa and three in Asia) measured impacts of ICS using standard combustion (i.e. not enhanced by forced draught or gasification), and without venting. These mainly used rocket-type combustion chambers, although some used these designs with a grate, table S1.

2.5.1.1. Kitchen concentrations: particulate matter

Seven studies (12 estimates) measured kitchen PM, six before-and-after designs and one RCT. The weighted mean levels were 0.750 mg m⁻³ at baseline and 0.361 mg m⁻³ post-intervention, with a weighted mean reduction of 51.9%, figure 6. Moderate heterogeneity (I² = 47%) was observed and random effects meta-analysis yielded a pooled effect estimate of −0.23 mg m⁻³ (95% CI: −0.34, −0.12), figure S1(a). Possible publication bias was indicated by both the Begg test (p = 0.099) and the Egger's test (0.005), figure S1(b).

Zoom In Zoom Out Reset image size

2.5.1.2. Kitchen concentrations: carbon monoxide

Eight studies (13 estimates) measured kitchen CO, seven before-and-after designs and one cross-sectional design. Weighted means were 11.1 ppm at baseline and 7.0 ppm post-intervention; weighted mean reduction of 36.1%, figure 6. The pooled estimate in fixed effect meta-analysis (I² = 0%), was −3.3 ppm (95% CI: −4.5, −2.0), figure S2(a). There was no evidence of publication bias from Begg's or Egger's tests (p > 0.1), figure S2(b).

2.5.1.3. Personal exposures: particulate matter and carbon monoxide

No studies provided estimates of personal exposure to particulate matter whereas two studies provided two estimates of exposure to CO (statistically non-significant pooled reduction of −1.7 ppm (95% CI: −4.7, 1.3)).

2.5.2. 'ICS' with venting (ICS—vent)

2.5.3. Complex (multi-facetted) interventions

2.5.4. Advanced combustion ('gasifier') solid fuel stoves

2.6.1. Alcohol based fuels (i.e. ethanol)

2.6.2. LPG

2.6.3. Cooking with electricity

Figures 6–9 show the effects of the cleaner cookstove and clean fuel interventions for the seven intervention categories for the four HAP outcomes and compared against WHO AQGs: Global Update 2005 (AQG) reference levels for PM_2.5 and the 2009 WHO guidelines for indoor air quality: selected pollutants for CO (WHO 2006, 2010).

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

For kitchen concentrations of PM_2.5, all intervention types, and most individual studies, resulted in statistically significant reductions (figure 2) and observed large percentage and absolute reductions, with only a small number reporting minor reductions or even increases. However, it is notable that only interventions involving very clean fuels (particularly for LPG stoves (11 estimates) and electric stoves (six estimates)) consistently achieved mean post-intervention levels near or below the WHO AQG IT-1 level (figure 6). Most of the interventions involving biomass stoves were at much higher PM_2.5 kitchen concentrations than WHO IT-1. Among solid fuel stoves, those interventions with chimneys had lower PM_2.5 concentrations than those without. None of the advanced combustion stoves (gasifiers) achieved WHO IT-1, although these only included stoves with forced draft combustion (the authors were not able to find any field based data from pellet fuel gasifier stoves). Kitchen PM levels from ethanol cookstove interventions were in general much lower than biomass cookstoves, but above the WHO AQG IT-1 PM_2.5 target.

Similar results were observed for studies measuring concentrations (kitchens) of CO, with some notable exceptions. Like PM_2.5, pre-intervention levels of CO showed a great deal of variability with many exceeding the WHO 24 h guideline value (figure 7). Post-intervention kitchen concentrations of CO were significantly reduced for both chimney and non-chimney cleaner biomass stoves, as well as ethanol and LPG clean fuel stove interventions (figure 3). Chimney stoves had the best performance in reducing CO among the solid fuel stoves, and ethanol from the clean fuels. In contrast to the findings for kitchen concentrations of PM (11 out of the 73 cleaner technology and clean fuel interventions achieving the WHO-IT1 guideline level (figure 6)), the majority of all types of interventions achieved CO levels at or below the WHO 24 h guideline (51 of 59 interventions).

For personal exposure to HAP, baseline levels tended to be lower than for kitchen concentrations for both PM_2.5 (figure 8) and CO (figure 9). For personal exposure to PM_2.5, whilst the data was more sparse, only the interventions involving cooking with advanced combustion cookstoves, LPG or electricity resulted in a statistically significant pollutant reduction (figure 4), and only LPG interventions consistently achieved levels below or near the WHO IT-1 AQG (figure 8). All but one of the post intervention levels for LPG were close to or below the WHO IT-1 level of 35 µg m⁻³ for PM_2.5, with a 74% (61%–83%) reduction observed to a post-intervention level of 58 µg m⁻³. Excluding the outlier study with an intervention estimate of 195 µg m⁻³ led to a post intervention average below WHO IT-1 of 33 µg m⁻³ and a pooled reduction of 78.1%. For chimney stoves a 76% (19%–87%) reduction was observed to a post-intervention level of 171 µg m⁻³ and for advanced combustion stoves a 20.3% (increase of 12.5% to reduction of 39.5%) reduction was observed to a post-intervention level of 105 µg m⁻³ (figure 6).

For personal exposure to CO, chimney stoves, advanced combustion biomass stoves, and LPG interventions resulted in statistically significant reductions. All interventions (both cleaner biomass burning technologies and clean fuels) resulted in levels of personal exposure below the WHO 24 h guideline level.

Combustion derived exposure to respirable PM from active and second-hand smoking, HAP and ambient air pollution have been associated with disease outcomes (pneumonia in children and chronic obstructive pulmonary disease, lung cancer, stroke and ischaemic heart disease) through published exposure-response functions (Burnett et al 2014, 2018, HEI 2020). These mostly non-linear functions (not including lung cancer that appears to have a linear exposure-response relationship) highlight that the risk of disease declines more steeply at lower levels of exposure relative to higher levels. For this reason it is the post-intervention levels of PM that are particularly important when considering risk on health rather than absolute or relative reductions of HAP. Whilst post-intervention levels varied substantially between interventions, most of those involving cleaner burning biomass stoves were close to the WHO annual PM_2.5 IT-1 of 35 µg m⁻³. Epidemiological studies and related dose-response curves suggest that to have a meaningful positive impact on the burden of disease from HAP, interventions need to be consistently at or below the interim guideline level (WHO 2006, Burnett et al 2014). Only clean fuels interventions (studies of ethanol, LPG and electricity) reported mean post-intervention concentrations of PM_2.5 close to or at the IT-1 levels (all below 100 µg m⁻³). For the limited number of biomass burning chimney stoves that achieved post intervention levels to the IT-1 target, further examination of these technologies and the circumstances of their use is warranted (e.g. Terrado et al 2005, Young et al 2019), including their impacts on fuel savings and the environment (Berrueta et al 2008, Granderson et al 2009, Johnson et al 2009).

There are a variety of factors that can contribute to reductions in emissions that affect post-intervention levels with a key influence being the use of multiple fuels and stoves in parallel to the intervention to meet different cooking and home energy needs (Ruiz-Mercado and Masera 2015, Quinn et al 2018, Ochieng et al 2020). This fuel and/or stove 'stacking' is common practice and is increasingly being reported in more recent measurement studies. Considering the substantial impact that 'stacking' can have on the level of emissions and exposure in the home, ongoing (and future) efforts to design and promote interventions to reduce HAP and will need to account for these practices to maximize health benefits, for example by understanding and addressing the reasons why people stack and promoting combinations of cleaner interventions. In addition, the effect of lighting, space heating and other combustion sources in and around the home, as well as pollution from other homes and sources in the community, while rarely quantified in evaluation studies, contribute to the observed pre- and post-intervention concentrations and exposures. These factors highlight the need for a comprehensive solution to the HAP problem, with interventions to clean up cooking as well as other household energy uses; this is likely to be more effective if targeted at the community level rather than single households.

This review represents a comprehensive appraisal of the current evidence base for quantified impacts on key health damaging pollutants of all cleaner and clean cooking solutions tested in field based conditions. With a 60% increase in the number of HAP measurement studies within a 3 years period, this meta-analysis draws on a wide range of published and unpublished sources including appraisal of specialist websites, conference proceedings and through direct communication with lead researchers in the clean cooking sector. Given the wealth of quantitative data available representing seven distinct intervention categories with impacts on both concentrations (kitchen) and exposure (cooks and children, where available) to PM_2.5 and CO, a variety of approaches have been used to present results from the review using both meta-analytical representation (e.g. forest plots) as well as referencing impacts for users' needs (i.e. relating values for baseline and post-intervention impacts to WHO AQG values). As one would expect, there is substantial heterogeneity between the included studies in relation to levels of PM_2.5 and CO, intervention type, context, circumstances of intervention use and employed methodology. For this reason the synthesis is presented for both individual studies and pooled across studies, allowing consideration of both general conclusions as well as context- and intervention-specific messages.

Another strength of this review is that it considers kitchen concentrations and exposure measurements from real-life settings in which other stoves, heaters, and lamps may be used concurrently, thus offering a realistic snapshot of HAP levels and exposures in practice. However, it is essential to interpret these measurements as potentially including other background levels of air pollution from household or ambient sources and thus a possible underestimation of the positive effect of cleaner stoves and clean fuels. The observation that clean fuel interventions consistently achieved the WHO IT-1 level for PM_2.5 despite potential stacking of fuels within these households, and potential contributions from PM_2.5 in ambient air, is noteworthy and suggests that it may be possible to achieve even lower concentrations and exposure if cleaner technologies and fuels are used exclusively within the community.

The results of these evaluation studies provide a useful guide to the HAP and exposure reductions achieved by different cooking solutions; however, the actual performance in any given setting will vary according to the contextual and operational conditions in which the stove/fuel combinations are used, including the housing type and condition, user-needs, geographical and climatic conditions, and many other factors (Rehfuess et al 2014, Stanistreet et al 2014, Puzzolo et al 2016). Results should therefore be interpreted with caution, with the overall pooled results for each stove and fuel type seen as a general guide to what is being achieved, while individual studies can provide more detailed and context-specific insights. This concern may be especially important for interpreting the cleanest options, such as LPG and electricity, as the user groups who can afford these fuels for cooking often have substantial socioeconomic and demographic differences from biomass users. At the same time, there is evidence from before and after studies that biomass-using homes transitioning to LPG can achieve low HAP exposures (Johnson et al 2018).

While a greater number of studies are being reported on LPG interventions, a limitation of the current evidence base is the paucity of studies being published on the intervention impacts of advanced combustion stoves and other clean fuels (e.g. alcohol fuels and electricity). There is also a lack of empirical evidence on the direct effects of interventions on personal exposure to PM_2.5. This is likely to partly be the result of an historical lack of suitable technology for measuring personal PM, especially in young children, as well as practical challenges involved including cost, and inconvenience to subjects. Moreover, although there are a few large-scale programmes ongoing that provide evidence on intervention impacts, the majority to date have been studies of relatively small projects and programmes. Given that many of these studies are early efficacy studies typically conducted at the beginning of wider programmes, it is not possible to extrapolate their effectiveness in terms of intervention impacts if implemented at scale over the longer term. The durability of the stove technologies over the longer term needs to be considered in terms of their effectiveness beyond the evaluation periods—there is evidence to suggest, for example, that properly maintained LPG stoves might be more robust in their efficacy to reduce HAP over time than more advanced biomass cookstoves (Mortimer et al 2016, Shen et al 2018). However, given the paucity of evidence further research on long term viability of cleaner cooking options in reducing HAP is needed.

One issue identified through the synthesis of evidence for the review was the presence of potential publication bias identified through statistical funnel plot asymmetry when pooling results. Such bias could have exaggerated observed intervention impacts for reductions in both PM_2.5 and CO due to the pooled evidence being based on studies with larger samples of homes with potentially greater reductions in concentrations. Conversely, estimates derived from the included studies would also include contributions to levels of PM_2.5 and CO from other sources (for example other use of solid fuels and kerosene for heating and lighting or from ambient sources that would have diluted the observed intervention impacts from cleaner cooking).

Another consideration is that most studies summarized measures of PM_2.5 and CO using arithmetic means leading to the syntheses through meta-analyses adopting this metric. Given that the distributions of these pollutants were generally positively skewed, geometric means or medians could have provided more reliable summary measures with use of means potentially leading to exaggerated intervention effects. An important implication of this limitation is that for interventions that achieved levels of kitchen concentrations or personal exposure at or below WHO guideline levels, larger fractions of the study populations would have met these targets than is suggested by the skewed mean estimates. This warrants consideration of a universal reporting structure for future HAP measurement studies (e.g. inclusion of geometric and arithmetic means, stratified by primary and secondary cooking fuel types) to minimize measurement bias when pooling PM_2.5 exposure data in future meta analyses.

There is a need for more studies conducted in the field to strengthen the evidence base for more robust assessment of intervention impacts of clean fuel interventions (more on LPG but in particular ethanol and electricity interventions). Such studies should adopt standardized methods to objectively assess the range of potentially health damaging pollutants, at least assessing concentrations of and exposures to both PM and CO. Monitoring of multiple stoves and other combustion devices in the home and immediate vicinity, and their contributions to levels of HAP and exposure quantified where possible is critical to advancing the evidence-base. Similarly, levels of ambient air pollution resulting from household combustion of neighbouring homes and other sources (which may include garbage burning, burning of crop residue, brick kilns, transport, industry and power generation) should be measured to assess the contribution to household levels of and personal exposure to PM and CO.

At the time of presenting the synthesis for the review, a large observational study was published supporting the main message from the review in relation to the importance of cooking with clean fuels over solid fuels and biomass to achieve reductions in HAP to substantively benefit health when used in everyday settings. In the prospective urban rural epidemiology (PURE)-AIR study, data on kitchen and personal PM_2.5 were collected from 2541 households and 998 individuals from 120 rural communities in Bangladesh, Chile, China, Columbia, India, Pakistan, Tanzania and Zimbabwe (Shupler et al 2020). The authors found that women from households primarily cooking with electricity (48 h mean = 62 µg m⁻³ (range = 50–76)) or gas (48 µg m⁻³ (43–54)), although typically with additional 'stacking' from secondary fuels had significantly lower exposure levels than those using coal (71 µg m⁻³ (57–86)), crop waste (97 µg m⁻³ (73–128)), wood (78 µg m⁻³ (69–89)), dung (146 µg m⁻³ (110–194)) and shrubs/grass (147 µg m⁻³ (109–197)). Whilst exposure to PM_2.5 for women using clean fuels were above the WHO IT-1 guideline level for PM_2.5 of 35 µg m⁻³, the authors note the often high ambient levels of HAP and suggest that 'community-level transition to clean cooking fuels, and potentially emissions reductions from other ambient sources' are required to maximize health gains.

The PURE-AIR study supports the main findings of this analysis, that clean fuel sources are associated with the lowest HAP exposures. In order to bring HAP exposure down to these lower exposure levels across populations, the efforts to characterize effectiveness of interventions and the factors driving contributions to exposure as described above need to be complemented with understanding how those factors can be impacted through policy, pricing, behaviour, and other levers. For example, while venting of stoves is to be encouraged to reduce indoor levels and personal exposures, finding ways to reduce total emissions to the ambient environment (e.g. cleaner combustion technologies or increasing chimney height) is critical to achieving low neighbourhood levels and minimizing exposure of the total population. Similarly, the most effective interventions using clean fuels need to be supported with efforts (e.g. financing mechanisms or comprehensive distribution networks to support continuous household supply of LPG or ethanol) to maximize displacement of traditional technologies. As HAP is influenced by different types of household energy as well as ambient air pollution entering the house, a comprehensive approach towards measuring and addressing HAP is needed. This includes taking heating and lighting into consideration when promoting clean energy use—in addition to cooking.

More high quality and direct feedback of program performance and effectiveness will also help inform decisions and facilitate efforts to maximize exposure reductions. Most critical, the paucity of personal exposure data needs to be addressed. New and more user-friendly instruments for measuring stove usage and exposure are a step in this direction (Chartier et al 2017, Liao et al 2019, Pillarisetti et al 2019, Burrowes et al 2020, Ventrella et al 2020, Wilson et al 2020), though the management and deployment of these devices still need to be supported by technical know-how. This new technology can enable the design of larger scale studies, such as the PURE-AIR and household air pollution intervention network studies (Arku et al 2018, Johnson et al 2020), that include measurements across many countries and/or communities under a harmonized protocol to minimize measurement biases when pooling data across locations in future meta-analyses.

Models to predict HAP exposure have also shown promise to scale-up quantitative exposures to a larger population (Baumgartner et al 2011, Balakrishnan et al 2013, Shupler et al 2018, Hill et al 2019, Sanchez et al 2020, Johnson et al 2021), though additional work is needed to test their performance and reliability across contexts. Biomarkers may also have a role, but need further development to understand how they relate to HAP exposure and health outcomes (Rylance et al 2013, Barr et al 2020).

A public health problem affecting almost half of the world's population demands the most-effective evidence-based cooking solutions are rolled out on a large-scale. Whilst there is not a single 'one size fits all' solution to the HAP problem, a combination of suitable technologies and programmatic approaches can be employed to meet the needs of specific geographical, socio-economic and cultural settings. Innovations in solid fuel stoves to reduce emissions will continue, however, the post-intervention PM levels of advanced combustion stoves presented here suggests that a lot of progress still needs to be made before a 'healthy', scalable biomass stove is available. With a growing evidence base demonstrating (a) generally disappointing post-intervention levels of PM achieved from cleaner biomass burning stoves and (b) the consistent effectiveness in PM reduction for clean fuel interventions such as LPG, ethanol and electricity in achieving emissions consistent with WHO guideline levels for health, greater emphasis needs to be placed on scaling adoption of cleaner fuels for policy, practice and programmes.