http://orcid.org/0000-0001-8498-6214
Figures
Abstract
Prioritizing resources for optimal responses to an ever growing list of existing and emerging infectious diseases represents an important challenge to public health. In the context of climate change, there is increasing anticipated variability in the occurrence of infectious diseases, notably climate-sensitive vector-borne diseases. An essential step in prioritizing efforts is to identify what considerations and concerns to take into account to guide decisions and thus set disease priorities. This study was designed to perform a comprehensive review of criteria for vector-borne disease prioritization, assess their applicability in a context of climate change with a diverse cross-section of stakeholders in order to produce a baseline list of considerations to use in this decision-making context. Differences in stakeholder choices were examined with regards to prioritization of these criteria for research, surveillance and disease prevention and control objectives. A preliminary list of criteria was identified following a review of the literature. Discussions with stakeholders were held to consolidate and validate this list of criteria and examine their effects on disease prioritization. After this validation phase, a total of 21 criteria were retained. A pilot vector-borne disease prioritization exercise was conducted using PROMETHEE to examine the effects of the retained criteria on prioritization in different intervention domains. Overall, concerns expressed by stakeholders for prioritization were well aligned with categories of criteria identified in previous prioritization studies. Weighting by category was consistent between stakeholders overall, though some significant differences were found between public health and non-public health stakeholders. From this exercise, a general model for climate-sensitive vector-borne disease prioritization has been developed that can be used as a starting point for further public health prioritization exercises relating to research, surveillance, and prevention and control interventions in a context of climate change. Multi-stakeholder engagement in prioritization can help broaden the range of criteria taken into account, offer opportunities for early identification of potential challenges and may facilitate acceptability of any resulting decisions.
Citation: Hongoh V, Gosselin P, Michel P, Ravel A, Waaub J-P, Campagna C, et al. (2017) Criteria for the prioritization of public health interventions for climate-sensitive vector-borne diseases in Quebec. PLoS ONE 12(12): e0190049. https://doi.org/10.1371/journal.pone.0190049
Editor: Kristie L. Ebi, University of Washington, UNITED STATES
Received: June 4, 2017; Accepted: December 7, 2017; Published: December 27, 2017
Copyright: © 2017 Hongoh et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The Quebec institute for Public Health research provided facilities for stakeholder meetings. The Public Health Agency of Canada provided funding to cover meeting facility and travel cost of participants for focus group discussions in the context of a separate, concurrent study on West Nile virus. VH is funded by the Canadian Institutes of Health Research, Ouranos Consortium for research in climatology and adaptation to climate change and the Strategic Training Program in Global Health Research, a partnership of the Canadian Institutes of Health Research and the Quebec Population Health Research Network. This project is also nested within the Initiative de Recherche Internationale sur l'Adaptation aux Changements Climatiques - Faire-face Aux Changements Ensemble program funded by the International Development Research Centre, Canadian Institutes of Health Research, Natural Sciences and Engineering Research Council of Canada and Social Sciences and Humanities Research Council from Canada. The funding sources had no involvement in the actual research or preparation of the paper.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Prioritizing resources for optimal response to an ever growing list of existing and emerging infectious disease risks presents an important challenge to public health administrations and their core intervention domains [1,2]. Ongoing global changes such as climate change, large scale land use transformations, increasing global travel and political instability in various regions of the world, contribute to variations in the patterns and occurrence of a number of infectious diseases, notably vector-borne diseases, which are known to be sensitive to weather and climate [3]. Changes in terms of the season of occurrence and the geographical distribution of these diseases are anticipated to increase as weather and climate are known to be drivers of the transmission and distribution of vector-borne diseases [4]. Prioritizing resources between existing and climate sensitive vector-borne diseases is complex but a necessary reality as potentially difficult trade-offs need to be made while taking into account a diversity of viewpoints [5]. For example, how much should be invested in surveillance activities, including vector surveillance and laboratory testing of novel pathogens, for diseases that are not yet endemic in Canada when resources for surveillance of existing diseases are limited? In other words, what non-endemic diseases should we be conducting surveillance for? Adaptation to vector-borne diseases will require both incremental and transformational approaches [6]. This study examines prioritization of climate-sensitive vector-borne diseases as an incremental step in improving management of these diseases within the primary domains of public health activity: research, surveillance, and prevention & control. Prioritization often serves as an initial step in aligning efforts and guiding public health decisions. In Quebec, MCDA methods have been used by the National Public health Institute and the Ministry of health for the prioritisation of Lyme disease surveillance and West nile virus interventions. The prioritization exercises referred to in this study pertain to planning of activities over a 1-5-year time scale while recognizing that these exercises may need to be updated and revisited within that time frame as knowledge evolves. The interventions included in these three domains range from the search for new treatments and diagnostic methods to education and outreach interventions as well as field interventions such as vector control or modifications to the built environment to help reduce the human health impact of infectious diseases. Several disease prioritization exercises have been undertaken in public health and veterinary public health contexts over the last few decades [7–21]. Traditionally, where stakeholders have been involved in these processes, with the exception of the studies by Ng and Sargeant (2012a) and Brookes et al (2014b), experts in public health have been the primary stakeholders included in the process. Prioritization exercises assist in explicitly bringing into focus the issues of concern around the decision problem and systematically evaluating available evidence on these issues. In this way, these exercises help structure reflection and guide decisions around resource allocation in order to ensure effectiveness within organisations and across various levels of government for effective public health delivery [22]. In other words, what are the concerns, what is the state of knowledge around these concerns, what are options going forward and how do these options perform in light of stated concerns. Prioritization exercises have been carried out at various scales from institutional-level [10], country-level [16,23–29] to continent-level [30,31]. Depending on the context, these exercises have been carried out either by institutional representative groups, a cross-section of the expert community concerned, advisory groups, national officials or researchers. Concern over how to prioritize infectious diseases in a context of climate change is more recent and at the the time of the initial review, had been documented only in the prioritization exercise by Cox and colleagues (2013). An evolution in the way prioritization exercises have been carried out over the last few decades can be seen in the examination of the literature [5] and reveals common goals and concerns that have persisted in their undertaking; notably, a push towards a systematic and transparent process and growing awareness of the increasing viewpoints that should be included in such exercises [13,22,25,30]. Refinements to the disease prioritization process over time have sought to separate information on the diseases (criteria measurement) from values pertaining to prioritization concerns (criteria) in order to improve transparency of the process. The explicit and measurable aspect of these exercises is sought by defining explicit criteria on which to evaluate the diseases being prioritized. For example, the measure of whether a disease is increasing or decreasing in the population can be captured as a criterion on the current incidence of human cases in the country; however, other concerns must also be evaluated such as what is the severity of the disease, do effective treatments already exist to limit the disease, is the general population already aware of and adopting effective protective behaviour to prevent the disease? If a disease is increasing in the population but not a severe disease or perhaps effective treatment already exists, should we allocate additional resources to this disease? For instance, Lyme disease is currently increasing in Canada, but relatively effective antibiotic treatment exists, how should this disease be prioritized relative to a new disease not yet endemic to Canada for which no treatment exists such as Dengue? Does the priority of a disease differ whether we are discussing allocation of resources in a research context versus a surveillance or prevention and control context? Criteria and how they are used to evaluate diseases are at the crux of the disease prioritization process. Criteria should represent core considerations or values relating to the prioritization objectives and help explicitly track relative differences between the items being prioritized [32]. Additionally, since health decisions in publicly funded health care systems use tax-paying citizen’s dollars to operate, in the interest of transparency and accountability, it is important to understand what concerns are held by society both to verify acceptability of potential decisions made following prioritization exercises and understand where differences in values or priorities may be present in order to help bridge existing gaps. For example, does society feel that resources should be prioritized for diseases that have more severe effects on vulnerable groups such as the elderly, very young, or pregnant women versus less severe diseases that affect a greater number of individuals? Examining the impact of different prioritization methodologies on the resulting prioritized list of diseases is also an important issue to explore.
In the current study, we identify criteria for the prioritization of vector-borne diseases applicable in a context of climate change in order to construct a general model for disease prioritization. We then examine differences in concerns expressed by stakeholders with regards to prioritization of public health interventions relating to research, surveillance and prevention and control objectives. Stakeholders working in fields both directly and not directly connected to public health were included in the process to broaden the diversity of perspectives and voices considered in the disease prioritization process, as well as assess similarities and differences in values held. While the inclusion of stakeholders from outside of public health is not yet routine in public health decision making, exercises have been conducted to assess the acceptability and ground feasibility of proposed interventions [33], as well as level of public awareness on a subject, such as the level of Lyme disease awareness in Canada [34], which may be useful in guiding future allocation of resources towards educational outreach activities.
The effect of combining current scientific knowledge with stakeholder values on disease prioritization is examined here by means of a pilot prioritization exercise performed with a multicriteria decision aid (MCDA) process using the PROMETHEE (Preference Ranking Organization Method for Enrichment Evaluations) outranking method [35].
Materials and methods
In disease prioritization exercises, criteria are used to systematically take into account concerns relevant to the decision-making context. The use of a participatory multi-stakeholder processes contributes to a broader and transparent selection of decision criteria. Here, a comprehensive review of the disease prioritization literature was conducted to create a synthesis list of the most commonly used prioritization criteria relevant in a context of climate change. These were discussed and validated with a group of stakeholders. The resulting criteria were used in a pilot prioritization exercise using PROMETHEE to examine differences in stakeholder assigned weights and their effect on prioritization under different intervention domains (research, surveillance, prevention and control). These steps are discussed in further details in the following sections and illustrated in a flow chart in Fig 1.
Preliminary criteria identification
A comprehensive review of infectious disease prioritization studies published between 1990 and 2014 was undertaken in Spring 2014 to identify key criteria that should be considered for inclusion in a generic model (Table 1). A keyword search of the literature was carried out using a scientific database (Pubmed) with combinations of the following keywords: “emerging”, “infectious”, “communicable”, “zoonoses”, “disease” and “prioritization”. Titles and abstracts were used to identify potentially relevant articles for further data extraction. Articles published in English or French pertaining to a prioritization exercise of infectious disease related items were retained for review. Additionally, relevant peer reviewed and grey literature referenced by articles retained for data extraction were also included in the review if they met the original inclusion criteria (snowball sampling) [36]. Relevant criteria and their related categories as described by citing articles were extracted from reviewed studies. As climate change may alter the season of occurrence and geographical distribution of climate sensitive vector-borne diseases [4,6], criteria pertaining to elements of the disease transmission process that may be affected by climate change were included. Whether or not conditions for transmission are already present for a disease is an important consideration in prioritization of these diseases. Conditions for disease emergence and transmission in a context of climate change can relate to environmental conditions, vector or reservoir conditions. For example, Anopheles species of mosquito capable of transmitting malaria are present in Canada and thus represent a vector component necessary for endemic transmission of this disease in Canada. Climate change signals are inherently difficult to separate out from the multitude of other driving forces (such as land use patterns, globalization and associated transport of goods, travel, etc.) which may affect disease transmission patterns. As such, many criteria that pertain to general infectious disease prioritization, such as the underlying vulnerability and susceptibility of the population to a disease or current levels of scientific knowledge and treatment availability, are also relevant in assessing climate sensitive vector-borne diseases and therefore were included in the preliminary list. Criteria were organized into categories following the literature review to facilitate criteria discussion with stakeholders and is presented along with the criteria in the results section.
Focus group discussion
Following the literature review, a focus group discussion was held with a small group of stakeholders in Quebec (Canada) in September 2014 to discuss concerns with regards to vector-borne diseases in a context of climate change. Prioritization of diseases was further examined in the context of interventions for research, surveillance as well as prevention and control. Stakeholder invitations were initially sent out to 24 individuals and organizations having previously participated in vector-borne disease consultations by the province. These organizations represent a range of civil, municipal and regional level organizations (including non-governmental environmental rights groups, Sustainable Development, Environment, Wildlife and Parks of Quebec, the Ministry of Agriculture, Fisheries and Food (of Quebec), the Council for the Protection of Sick Patients (of Quebec), municipal representatives from municipalities in the province, representatives from the Health and Welfare Commissioner (of Quebec), academic researchers, representatives from a Quebec seniors rights representative group, regional public health departments and provincial blood donation services) representing the interests of various civil, municipal or regional subsets of the Quebec population. These organizations have all been in existence for over a decade. Stakeholders were selected based on their availability for a concurrent participation in a separate ongoing study on West Nile virus management in Quebec. Implications of this selection are discussed in further detail in the discussion section of this paper. All participating stakeholders gave informed written consent prior to participation in the study. Prior to the stakeholder meeting, participants were invited to reflect on their concerns with regards to managing infectious diseases now and over the course of the next 5 to 10 years in the context of climate change. For example, are diseases with high severity of greater concern than diseases with high incidence or vice versa? What other concerns are of interest, economic concerns, logistic concerns such as treatment availability or knowledge levels of the disease, animal and environmental health impact related concerns? During the focus group discussion, stakeholders began by writing down their prepared concerns. These were then compiled using a modified nominal group technique [37] and discussed with all participants. Following this, the literature identified criteria and their categories were presented in writing and discussed with stakeholders. Stakeholder compiled concerns and literature identified criteria were compared and discussed. Stakeholders were given the opportunity to add additional criteria and clarify or reword the literature identified set in person during the discussion. Measurement scales were also discussed at this time. Following the meeting, stakeholders were given an additional month to reflect on and validate the final list of criteria by means of two rounds of online Delphi review [38]. The online review was conducted using an electronic survey that presented all retained criteria from the in-person discussion and allowed stakeholders to “vote” for the inclusion of individual criteria. Comments on the relevance of the criteria to the prioritization models could also be made at this time. Stakeholders agreed to retain any criteria which received at least one vote from a participating stakeholder. Results from the first online review were compiled and presented to stakeholders to allow further modification before the final validated set was defined. This final set was included in prioritization models pertaining to research, surveillance and prevention and control of vector-borne diseases. This project was reviewed and approved by the Ethical Committee for Health Research of the University of Montreal (Comité d’éthique de la recherche en santé, CERES) (certificate number 14-025-CERES-D).
Criteria weighting
Following validation of the final list of criteria, stakeholders were asked to weight criteria according to their relative importance with regards to research, surveillance and prevention and control objectives. The purpose of this weighting exercise was to translate stakeholder value systems into numerical weights. In order to do this, stakeholders were given a Microsoft Excel spreadsheet tool and asked to distribute 100 points across the list of decision criteria included in the model. The Excel tool included the finalized list of stakeholder validated criteria, desired effect direction of criteria and measurement scales listed by category with replicated sections for each of the three intervention domains (research, surveillance, prevention and control). The desired effect direction of a criterion refers to what the desired state is with respect to that criterion. For instance, with respect to costs, the desired effect direction is generally to prioritize diseases with the highest cost burden, i.e. maximize this criterion; whereas with respect to the general level of knowledge of the public on a disease, the desired effect direction may be to prioritize diseases that the public has little knowledge of, i.e. minimize this criterion. Stakeholders were asked to weight criteria in accordance with perceived importance taking into account their relative importance overall. Weights of zero were permitted for criteria to allow stakeholders to indicate the absence of importance of criteria if applicable. The difference in relative weights assigned to different categories were compared between the three intervention domains (research, surveillance and prevention and control) and Welch’s t-test (unequal variances t-test) were performed in R (version 3.2.2) (R Core Team (2016), Vienna, Austria, http://www.R-project.org) to test for significant differences in category weights.
Pilot prioritization of five diseases
An exploratory prioritization of five potentially climate-sensitive vector-borne diseases, Lyme, West Nile virus, chikungunya, dengue, and malaria was carried out to examine the effects of criteria weightings described in the previous section on disease rankings for each intervention domain. Only Lyme and West Nile virus have shown a local transmission cycle in Quebec in the last 10 years [39,40]; the other three diseases currently manifest themselves as imported cases only, but local cycles may occur in the coming decades due to climate change [41–43]. Lyme expansion in North America has been linked to climate change [39] and while West Nile virus expansion into North America was not directly linked to climate change, its epidemiology has been shown to be directly sensitive to climatic factors [44]. A literature search was conducted in late fall 2014 in Pubmed using each of the five diseases separately and in combination with the 21 identified criteria in order to assess and score disease performance over the criteria. Only articles published in English or French pertaining to the disease and criteria were reviewed. In other words, given the latest literature, what evaluation does each disease obtain on each of the identified criteria using the specified measurement scale. The same disease assessment scores were used for each intervention domain, however, stakeholders weighted each intervention domain separately and weights were shown to vary depending on the domain. Analysis of disease performance and criteria weights was performed with the PROMETHEE method (Preference Ranking Organization Method for Enrichment Evaluations) in visual PROMETHEE software (version 1.4.0.0) (VP Solutions software, Brussels, Belgium, http://www.promethee-gaia.net). This MCDA method requires a set of criteria with corresponding weights and diseases performance scores over each of the criteria in order to produce a ranking of the relative performance of all diseases over the combined criteria. The PROMETHEE II method was used as it provides a complete ranking of results without incomparability [45,46]. PROMETHEE methods are pair-wise comparison methods that are part of the outranking class of decision aid methods enabling comparison of multiple items over multiple criteria [45,46]. In traditional multi-criteria type problems, incomparability can occur when one option performs better on certain criteria while another option performs better on very different criteria making comparison challenging [45,46]. PROMETHEE II was designed to overcome this incomparability without scale effects by calculating a net performance score over all criteria [45,46]. Additionally, a GAIA (Geometrical Analysis for Interactive Aid) visual analysis was also used to graphically explore decision maps. A sensitivity analysis was performed to examine the robustness of rankings for the 1st order given weights expressed by stakeholders.
Results
Literature and stakeholder identified criteria
Following an initial keyword search, the titles and abstracts of 1196 articles were scanned for relevance (Table 1). This resulted in 37 studies which were retained for full text review. Five additional articles referenced within the previous set were also reviewed. From this, 26 studies explicitly reporting prioritization criteria were retained for data extraction. A summary of these studies is shown in Table 2. Studies were primarily from high income countries in North America, Europe and Asia. While prioritization exercises have taken place in developing contexts [47], none were found pertaining explicitly to vector-borne diseases in the reviewed time period. An initial list of 122 criteria was extracted from these studies. The number of criteria used per prioritization exercise ranged from as few as 5 to as many as 57. Reported sources included experts, and lists from previous exercises and the literature.
A number of studies had shared approaches and criteria (e.g.: [7,23,24]; [22,48]; [8,10,27,49]) while other studies used similar concepts, with variations in wording. The most common categories of criteria included: public health impacts, economic or market impacts, animal health impacts (generally pertaining to market impacts but also for animal-welfare), public perception and public health capacity to deal with a disease. These categories were retained in the models and named “Public Health”, “Social Impact”, “Economic”, “Animal and Environmental Health”, “Strategic and Operational” (i.e. logistics). Additionally, a “Risk and Epidemiology” category was added to capture epidemic potential, recent disease trends and proportion of susceptible population. Climate sensitive risk and epidemiology were also included in this category. As climate change will likely alter temperature and precipitation patterns with consequences for animal and vector distribution [4], criteria pertaining to existing conditions for disease transmission were included here. Commonly used prioritization criteria and their frequency were tracked across reviewed studies (see supplementary S1 Table). Recurring relevant criteria were identified and where wording was different but pertaining to the same concept, criteria were combined and synthesized where appropriate into a shorter preliminary list of 20 criteria covering as broad a range of relevant concepts as possible for discussion with stakeholders. This number was chosen in order to present a manageable set for discussion with stakeholders. Retained criteria were then used in a pilot prioritization exercise to examine differences in stakeholder assigned weights under different intervention domains.
Focus group discussion
Twelve stakeholders consented to participate in a discussion held on September 29th, 2014 in Montreal, Quebec, Canada on the topic of perspectives and concerns relevant to disease prioritization in a context of climate change. One third of participants were female. All participants were between the ages of 30 and 65. Stakeholders had backgrounds in microbiology, entomology, biology, medicine, veterinary medicine and patient advocacy and hailed from a mix of both provincial and municipal organizations. Stakeholder discussions revealed coherence between stakeholder identified concerns and the literature constructed list of criteria. Further online validation by stakeholders following the initial meeting, resulted in a finalized list of twenty-one criteria (Table 3). Discussions of appropriate measurement scales and direction of desired effect to assess diseases were also held and are included in Table 3.
Criteria weighting
Weighting of criteria was done individually by stakeholders and returned to the researchers by email in the weeks following the in-person focus group discussion. Ten of the original twelve stakeholders completed the criteria weighting exercise for each of the three intervention domains (research, surveillance or prevention and control). The relative importance of categories was generally similar for stakeholders across domains with no significant differences found between categories (Fig 2). Individual criteria weights were however found to differ within categories and across intervention domains (Fig 3). Stakeholder weights for all criteria are included in the supporting Information S2–S4 Tables. For all three intervention domains, the top three weighted categories were consistently “Public Health”, “Risk and Epidemiology” and “Strategic and Operational” criteria while the bottom three categories were consistently “Animal and Environmental Health”, “Economic” and “Social Impact” criteria. In the top 3, “Public Health” was generally the top weighted category in the subset with “Strategic and Operational” criteria consistently in 3rd place whereas in the bottom 3, the “Animal and Environmental” criteria category was generally top rated while “Social Impact” was either last or tied for last. Despite similarities in the relative importance of categories, differences in individual weights were observed between stakeholders and are reflected in the GAIA visual analysis of projected stakeholder weights (Fig 4). The top weighted criterion varied considerably by individual and by intervention domain though was generally from one of the top 3 weighted categories (i.e. “Public Health”, “Strategic and Operational” or “Risk and Epidemiology” categories). The least weighted criterion also varied considerably by individual and intervention domain however, given the large number of criteria, multiple criteria often shared the lowest value but were not limited in origin to only the least weighted categories (“Animal and Environmental Health”, “Economic” and “Social Impact”). The weights expressed by stakeholders not directly associated with public health organisations (n = 3) were found to vary significantly (though not necessarily together) compared with the rest of the group. This was the case for the weights given to the “Public Health” category for research (p = 0.011) and surveillance (p = 0.016) interventions as well as the weights given to the “Risk and Epidemiology” category for prevention and control (p = 0.035) and research (p = 0.035) interventions. In the previously mentioned cases, the stakeholders not directly associated with public health generally attributed less weight to these categories compared to stakeholders directly working in public health. Conversely, the distribution of weights given to the “Social Impact” category for prevention and control interventions (p = 0.044) and for the “Strategic and Operational” category for research (p = 0.028) interventions were generally found to be higher for stakeholders not directly associated with public health.
The span of stakeholder weights is indicated by the vertical lines with shaped makers indicating the intervention specific group means. Criteria categories are shown along the X axis with average weights by category shown along the Y axis. The differences between the weights given to each intervention domain (research, surveillance and prevention & control) were not found to be significantly different for any of the categories. Criteria category Legend (X axis): PHC: Public Health Criteria; SIC: Social Impact Criteria; REC: Risk and Epidemiology Criteria; AEC: Animal and Environmental Health Criteria; ECC: Economic Criteria; SOC: Strategic and Operational Criteria.
Each line in the graph represents each of the 10 Individual stakeholder’s (S1-S10) weight assignments on all 21 criteria. The relative importance of criteria within each category is seen to vary depending on the intervention domain. For example: the “SOC-03-level of knowledge” criterion received the most weight in the research domain, while the “SOC-01-capacity to detect disease” criterion received the most weight in the surveillance domain. The “SOC-02-Existence of treatment” received the most weight in the prevention and control domain.
Panel A) shows the GAIA map for the research domain, B) shows the surveillance domain and C) the prevention and control domain. In each map, the bold red line represents the group decision axis (i.e. consensus ranking) with the filled circle pointing in the direction of the group ranking. Square markers represent the ranking of the different diseases in k-dimensional space (where k represents the number of criteria) projected onto a 2-dimensional plane. Diseases closest to the group decision axis are prioritized over diseases further away from the decision axis. Stakeholders 1 through 10 are represented by the blue circular markers labelled S1-S10. Stakeholders pointing in the same direction as the group decision axis are most aligned with the group ranking. Stakeholders further away in space from each other and from the group decision axis have more disparate weighting tendencies and hence perspectives. For example, in panel A) S8 shows distinct weight position as compared with the rest of the stakeholders and therefore indicates a different set of values in this context as compared with the rest of the group. Clusters of stakeholders can also be observed occurring in each of the panels and indicate stakeholders with more similar weightings (i.e. perspectives). For example, in panel B) weights by stakeholders S2 and S6 are more similar to each other than to stakeholders S1 and S8.
The criterion “existence of favorable conditions for disease transmission” was weighted as the highest or second highest criterion within the “Risk and Epidemiology” category for the majority of stakeholders. However, across prioritization contexts, its relative importance varied depending on the stakeholder. This suggests that additional concerns (such as disease severity, level of scientific knowledge, current incidence and public risk perception) are given priority depending on the context and stakeholder. Criteria from the “Social Impact”, “Animal and Environmental Health” as well as “Economic” categories were rarely among the top 5 weighted individual criterion for stakeholders, though these do appear among the top 5 for some stakeholders across each of the intervention domains. The small number of stakeholders and difference in perspective between groups of stakeholders is likely driving this observation.
Pilot prioritization of diseases
From the pilot prioritization exercise, West Nile virus and Lyme were ranked 1st and 2nd respectively in the research and prevention and control domains, with this order reversed in the surveillance domain. Dengue, Malaria, and Chikungunya virus were ranked 3rd, 4th and 5th in the research and surveillance domains with Malaria in 3rd and Dengue in 4th in prevention and control domain. Diseases were ranked with and without stakeholder assigned weights across intervention domains to assess dominance. No single disease was found to be dominant. Varying the disease evaluations showed sensitivity of disease rankings to their evaluations. Furthermore, different scales would produce different evaluations with potentially different rankings. As the goal of the current project was not to formally assess local disease priorities (for the creation of an official list of priority diseases), but rather to assess differences in stakeholder perspectives and examine the effect of these on potential rankings, a formal systematic assessment of sensitivity to disease scales was not performed. The retained scales allowed us to distinguish between the relative importance of diseases per criterion. Unweighted uni-criterion analysis revealed criteria for which each of the assessed diseases ranked 1st at least once. For instance, Dengue ranked 1st on the “capacity to detect and diagnose” criterion, Chikungunya ranked 1st on the “level of scientific knowledge of the disease” criterion, Lyme ranked 1st on the “current global trend” criterion, malaria ranked 1st on the “risk perception of the public” criterion and West Nile virus ranked 1st on the “epidemic potential” criterion. Group and individual stakeholder ranking results are shown in Table 4 with corresponding assessment values used (based on context specific data obtained from the literature) shown in the evaluation matrix in Table 5 (supporting references used for disease assessments are provided in supporting information S1 Appendix).
Sensitivity analysis results with weight stability intervals for all criteria by all stakeholders for the research domain are shown in Table 6 in descending order of stability (from least stable to most stable). Five of the twenty-one criteria were found to be very stable as per the size of their stability intervals spanning almost the entire range of possible values from 0–100 for all stakeholders for the 1st order ranking. This indicates that the rank ordering of diseases would not change for any weight value given to these criteria between 0 and 100. These five criteria were “the existence of a vulnerable group”, “potential to increase social inequality”, “ability to infect the environment”, “optimization opportunities” and “reportable disease”. The remaining criteria were found to have relatively small stability intervals (<10 points) for at least one stakeholder indicating high sensitivity to assigned weights by stakeholders. The “current trend”, “cost to individuals” and “general knowledge” criteria were found to be the highly sensitive for 7, 6 and 5 stakeholders respectively. This was closely followed by “existence of favourable conditions”, “disease severity for animals”, “cost to private sector” and “public risk perception” criteria for at least 4 stakeholders. Surveillance and prevention and control sensitivity analysis results were similar and are included in the supplementary material S5 and S6 Tables.
Discussion
The current study solicited Quebec stakeholder perspectives and concerns for vector-borne diseases management under climate change. These concerns were translated into criteria and used to construct prioritization models to rank vector-borne diseases in the context of public health interventions pertaining to research, surveillance and, prevention and control. The prioritization models consisted of 21 criteria and were accompanied by corresponding stakeholder weights that varied depending on the intervention context (research, surveillance or prevention and control). Criteria were distributed across six categories: “public health”, “risk and epidemiology”, “strategic and operational”, “animal and environmental health”, “economic” and “social impact” with the relative importance of criteria within these categories varying depending on the intervention context. A pilot prioritization exercise showed how resulting disease priorities also change depending on the intervention context.
What have we learned from years of prioritization exercises? The use of an initial literature review followed by a stakeholder consultation in our study allowed for the inclusion of a broad range of considerations which were factored into our prioritization models. Our literature review suggests that common criteria and categories recur across studies [7–21]. This may be due to shared learning from previously published studies, but may also be representative of core concerns that are shared across scales and regions which translate into a set of common decision criteria that remain applicable across public health contexts and that can be used as a basis for prioritization models as we have done here. Furthermore, these models were found to be equally applicable in three intervention contexts with associated stakeholder specific weights. While our study included similar categories of criteria to previous prioritization exercises, detailed direct comparisons cannot be made between studies since the prioritization objectives and approaches differed.
What is the value of including stakeholders with a variety of backgrounds in the process? Few studies involve the public in their consultation process, and generally involve only a narrow range of participants in the process. Involvement of a diversity of participants (researchers, government, public health personnel, and non-technical citizens) in the criteria and preference elicitation process can help ensure that a broad set of value perspectives are considered. An attempt at including a broader range of voices in the vector-borne disease prioritization process has been done in the current study. While previous studies have contrasted public and expert rankings [28,29], the current study demonstrated a potential method for how to include these voices in the same consultation exercise in order to provide an opportunity for shared knowledge exchange and discussion of concerns between groups. Including multiple stakeholders with diverse backgrounds helps broaden the perspectives considered and contributes to the inclusion of a representative set of societal concerns and perspectives in the decision analysis process. The weighting process also allows a second measure of where stakeholders stand with regards to these concerns. Weights by non public health related organisations were found to be significantly different from the rest of the group and may in part be explained by differences in perceived responsibility or accountability between these groups [50]. Where differences occur between groups may indicate points of entry for future research into how differences in perspective impact intervention acceptance or uptake down the line. While public health officials are a part of society, their knowledge may push them to dismiss a disease which causes great alarm to the general public (e.g. one with high severity but low risk of transmission in the current context). Understanding what is being factored into each group’s decision making equation is therefore useful in order to inform future interventions and public health actions.
Remaining challenges
From a public policy point of view, ensuring that resulting decisions are well aligned with publicly held values should be of interest and is something that is possible to explore with the current approach. The use of formal prioritization approaches is evolving. Who to include in the process is a consideration that should be regularly revisited by decision makers. The challenge becomes how to process the range of viewpoints consulted during analysis. Should different viewpoints be weighted equally? Should all groups be consulted primarily to elicit the most comprehensive range of viewpoints or are related weightings by different stakeholder groups useful and necessary? These questions should be addressed early on in the prioritization exercise and responses are likely to vary depending on the specific prioritization context. Key participant profiles representing the viewpoints of various societal groups can be constructed and has been used before in environmental decision making processes [51,52]. The consistency of weighting tendencies within stakeholder groups was not assessed in the current study given the small number of participants; however, given a larger sampling, the consistency of weighting trends within groups could be further examined.
This work is still in the early stages of understanding the impact of including broader consultation on these issues and requires further exploration to help guide the minimal set of consultations that should be considered. For certain intervention domains, such as surveillance, public health perspectives may be one of the most important to take into account; however, with regards to publicly funded research interventions, it may be of interest to ensure that resulting decision priorities are coherent with socially held values. While public health officials themselves do represent socially held values, their unique perspective may differ from that of society at large and effort should be made to examine where potential differences may exist. Stakeholders outside of public health, especially non-government stakeholders, offer more than just another view and opinion in this process. They help take into account socially held values, but also ensure that interventions are feasible in the field. Their inclusion is a first step in the communication and knowledge transfer process with these organizations, such as municipalities. With regards to the disease prevention and control domain, understanding and attempting to integrate different perspectives may help pinpoint areas where additional information needs exist. For example, diseases which may pose significant health and economic impacts to livestock populations but few to human populations may fall under the radar of human public health officials. Including stakeholders from other domains such as animal health and environmental health may help to ensure that attention is given to such cases. In this exercise, significant differences were found for the “social impact” and “risk and epidemiology” categories suggesting that important differences may exist between public health experts and the general public with regards to risk perception, knowledge and risk assessment of disease threats. Differences in risk perception between the general public and experts have been observed in previous studies [53–55].
Limitations
This study did not aim to predict which diseases might be sensitive to climate change, but rather set out to identify the primary concerns (i.e. criteria) of a cross-section of society with regards to vector-borne disease management in these contexts. Here “cross section of society” refers not to the number of stakeholders engaged, but rather the diversity of organizations consulted. Furthermore, the pilot prioritization presented was for illustration purposes only and should not be interpreted as a formal assessment of local Quebec priorities. Additional data as well as further discussion with experts and stakeholders is required to validate these findings.
With regards to stakeholders included in this study, these groups were invited based on their representation of a cross-section of society at large and meant to represent perspectives from various interest groups beyond that of public health alone (e.g. patient rights group, senior citizens). Of the 24 invited, only 12 were able to participate due to availability. Of those 12, only 10 further completed the weighting exercise, once again due to time constraints. A larger number of participants with different backgrounds or organizational membership would likely have increased the diversity of perspectives included both in terms of criteria considered and weightings assigned to those criteria. Furthermore, while this same group of stakeholders also participated in a concurrent study on West Nile virus, additional evaluations with different stakeholder groups will be needed to assess whether weightings were strongly influenced by this factor. It should be noted that disease rankings are not driven by stakeholder weightings alone, but rather the combined evaluation of literature evidence with stakeholder weights. The relative stability of the rankings was explored in sensitivity analyses presented the results section (Table 6 and supplemental materials S5 and S6 Tables) and shown to be relatively stable for many of the criteria in the final models.
Future prioritization exercises
The current study offers preliminary models that can be used as a starting point for future prioritization exercises to build on. Thorough reflection should be held on the purpose or objective of the prioritization exercise and who the most representative stakeholders are to include in that context. The approach used in the current study offers an opportunity to identify concerns held by participating stakeholders and provides a method to examine the effect of these differences on disease rankings. The inclusion of diverse perspectives provides opportunities to identify knowledge gaps and data needs that go beyond an assessment of available scientific evidence alone and thus contributes to a more comprehensive process informing decision. In allocating health resources, a formal process for evaluating diverse evidence and interests notably contributes to enhancing and articulating the rigour, transparency, and acceptability of the decision process, in a manner which is compatible with currently held social values. In other words, what, in practice, decision making will gain from this approach includes: 1) a more comprehensive view of the issue–issues that decision makers did not think existed or did not think were important; 2) a sense of perceived risk (above and beyond what the usual experts / specialists are saying); 3) a level of acceptability by using standard criteria to prioritize health risks; 4) a sense of transparency as to how the prioritization results were obtained; 5) a level of rigour to the results given that the prioritization follows a set of transparent and accepted rules. The approach described here is coherent with the vulnerability and adaptation framework described in the 4th IPCC assessment report [56] and more specifically, the impact assessment stage in terms of providing a mechanism with which to narrow a list of diseases to focus on in the context of climate change.
Supporting information
S2 Table. Individual stakeholder weights for all criteria ordered by importance for the research intervention domain.
https://doi.org/10.1371/journal.pone.0190049.s002
(DOCX)
S3 Table. Individual stakeholder weights for all criteria ordered by importance for the surveillance intervention domain.
https://doi.org/10.1371/journal.pone.0190049.s003
(DOCX)
S4 Table. Individual stakeholder weights for all criteria ordered by importance for the prevention and control intervention domain.
https://doi.org/10.1371/journal.pone.0190049.s004
(DOCX)
S5 Table. Weight stability interval sensitivity analysis for all stakeholders for the surveillance domain.
https://doi.org/10.1371/journal.pone.0190049.s005
(DOCX)
S6 Table. Weight stability interval sensitivity analysis for all stakeholders for the prevention & control domain.
https://doi.org/10.1371/journal.pone.0190049.s006
(DOCX)
S1 Appendix. Supporting references used to assess disease scores for the pilot prioritization.
https://doi.org/10.1371/journal.pone.0190049.s007
(DOCX)
Acknowledgments
The authors would like to sincerely thank all stakeholders and experts in Quebec consulted during this project for their participation and contribution as well as the Institut national de santé publique du Québec for the use of their facilities for meeting. The Public Health Agency of Canada provided funding to cover meeting facility and travel cost of participants for focus group discussions in the context of a separate, concurrent study on West Nile virus. VH is funded by the Canadian Institutes of Health Research (CIHR), Ouranos Consortium for research in climatology and adaptation to climate change and the Strategic Training Program in Global Health Research, a partnership of CIHR and the Quebec Population Health Research Network (RRSPQ). This project is also nested within the IRIACC-FACE program funded by IDRC, CIHR, NSERC and SSHRC from Canada. The funding sources had no involvement in the actual research or preparation of the paper.
References
- 1. Ebi KL, Kovats RS, Menne B. An approach for assessing human health vulnerability and public health interventions to adapt to climate change. Environ Health Perspect. 2006;114: 1930–1934. pmid:17185287
- 2. Baltussen R, Niessen L. Priority setting of health interventions: the need for multi-criteria decision analysis. Cost Eff Resour Alloc. 2006;4: 14. pmid:16923181
- 3. Gubler DJ, Reiter P, Ebi KL, Yap W, Nasci R, Patz JA. Climate variability and change in the United States: potential impacts on vector—and rodent-borne diseases. Environ Health Perspect. 2001;109.
- 4. Githeko AK, Lindsay SW, Confalonieri UE, Patz JA. Climate change and vector-borne diseases: a regional analysis. Bull World Health Organ. 2000;78: 1136–1147. pmid:11019462
- 5. Brookes VJ, Del Rio Vilas VJ, Ward MP. Disease prioritization: what is the state of the art? Epidemiol Infect. 2015;FirstView: 1–12. pmid:25876939
- 6.
Smith KR, Woodward A, Campbell-Lendrum D, Chadee DD, Honda Y, Liu Q, et al. Human Health: Impacts, Adaptation, and Co-Benefits. In: Field CB, Barros VR, Dokken DJ, Mach KJ, Mastrandrea MD, Bilir TE, et al., editors. Climate Change 2014: Impacts, Adaptation, and Vulnerability Part A: Global and Sectoral Aspects Contributions of Working Group II to the Firth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge, United Kingdom and New York, NY, USA: Cambridge University Press; 2014. pp. 709–754.
- 7. Doherty J-A. Final report and recommendations from the national notifiable diseases working group. Can Commun Dis Rep. 2006;32.
- 8. Krause G. Prioritisation of infectious diseases in public health. Eurosurveillance. 2008;13.
- 9. Krause G. How can infectious diseases be prioritized in public health? EMBO Rep. 2008;9: S22–S27. pmid:18578019
- 10. Balabanova Y, Gilsdorf A, Buda S, Burger R, Eckmanns T, Gärtner B, et al. Communicable diseases prioritized for surveillance and epidemiological research: results of a standardized prioritization procedure in Germany, 2011. PLoS ONE. 2011;6: e25691. pmid:21991334
- 11.
Institut de Veille Sanitaire. Définition des priorités dans le domaine des zoonoses non alimentaires 2000–2001. Paris, France: Institut de Veille Sanitaire; 2002 p. 40 p.
- 12. Cardoen S, Van Huffer X, Berkvens D, Quoilin S, Ducoffre G, Saegerman C, et al. Evidence-based semiquantitative methodology for prioritization of foodborne zoonoses. Foodborne Pathog Dis. 2009;6: 1083–1096. pmid:19715429
- 13. Havelaar AH, van Rosse F, Bucura C, Toetenel MA, Haagsma JA, Kurowicka D, et al. Prioritizing Emerging Zoonoses in The Netherlands. PLoS ONE. 2010;5: e13965. pmid:21085625
- 14.
Institut de Veille Sanitaire. Définition des priorités dans le domaine des zoonoses non alimentaires 2008–2009. Paris, France: Institut de Veille Sanitaire (InVS); 2010 p. 31 p.
- 15. Gilsdorf A, Krause G. Prioritisation of infectious diseases in public health: feedback on the prioritisation methodology, 15 July 2008 to 15 January 2009. Euro Surveill. 2011;16.
- 16. Ng V, Sargeant JM. A quantitative and novel approach to the prioritization of zoonotic diseases in North America: a public perspective. PLoS ONE. 2012;7: e48519. pmid:23133639
- 17. Cox R, Revie CW, Sanchez J. The Use of Expert Opinion to Assess the Risk of Emergence or Re-Emergence of Infectious Diseases in Canada Associated with Climate Change. PLoS ONE. 2012;7: e41590. pmid:22848536
- 18. Cox R, Sanchez J, Revie CW. Multi-Criteria Decision Analysis Tools for Prioritising Emerging or Re-Emerging Infectious Diseases Associated with Climate Change in Canada. PLoS ONE. 2013;8.
- 19.
Capek I. Définition des priorités dans le domaine des zoonoses non alimentaires 2008–2009. Institut de Veille Sanitaire; 2010.
- 20. Brookes VJ, Hernandez-Jover M, Cowled B, Holyoake PK, Ward MP. Building a picture: Prioritisation of exotic diseases for the pig industry in Australia using multi-criteria decision analysis. Prev Vet Med. 2014;113: 103–117. pmid:24211032
- 21. Brookes VJ, Hernandez-Jover M, Neslo R, Cowled B, Holyoake P, Ward MP. Identifying and measuring stakeholder preferences for disease prioritisation: a case study of the pig industry in Australia. Prev Vet Med. 2014;113: 118–131. pmid:24211106
- 22. Rushdy A O’Mahony M. PHLS overview of communicable diseases 1997: results of a priority setting exercise. Commun Dis Rep CDR Wkly. 1998;8: S1–12.
- 23. Carter A. Establishing goals, techniques and priorities for national communicable disease surveillance. Can J Infect Dis. 1991;2: 37–40. pmid:22451750
- 24. Doherty J-A. Establishing priorities for national communicable disease surveillance. Can J Infect Dis. 2000;11: 21–24. pmid:18159260
- 25. Kadohira M, Hill G, Yoshizaki R, Ota S, Yoshikawa Y. Stakeholder prioritization of zoonoses in Japan with analytic hierarchy process method. Epidemiol Infect. 2015;143: 1477–1485. pmid:25195643
- 26. Del Rio Vilas VJ, Voller F, Montibeller G, Franco LA, Sribhashyam S, Watson E, et al. An integrated process and management tools for ranking multiple emerging threats to animal health. Prev Vet Med. 2013;108: 94–102. pmid:22954461
- 27. Cediel N, Villamil LC, Romero J, Rentaria L, De Meneghi D. setting priorities for surveillance, prevention, and control of zoonoses in Bogota, Colombia. Rev Panam Salud Publica. 2013;33: 316–24. pmid:23764662
- 28. Ng V, Sargeant JM. A stakeholder-informed approach to the identification of criteria for the prioritization of zoonoses in Canada. PLoS ONE. 2012;7: e29752. pmid:22238648
- 29. Ng V, Sargeant JM. A quantitative approach to the prioritization of zoonotic diseases in North America: a health professionals’ perspective. PLoS ONE. 2013;8: e72172. pmid:23991057
- 30. Humblet MF, Vandeputte S, Albert A, Gosset C, Kirschvink N, Haubruge E, et al. Multidisciplinary and evidence-based method for prioritizing diseases of food-producing animals and zoonoses. Emerg Infect Dis. 2012;18. pmid:22469519
- 31.
WHO. The Dubrovnik Pledge on Surveillance and Prioritization of Infectious Diseases. Report on a WHO meeting in Bucharest, Romania 21–23 November, 2003. Copenhagen: WHO Regional Office for Europe; 2003 p. 27 p.
- 32. Belton V, Stewart T. Multiple criteria decision analysis: an integrated approach. 2002;
- 33. Aenishaenslin C, Michel P, Ravel A, Gern L, Waaub J-P, Milord F, et al. Acceptability of tick control interventions to prevent Lyme disease in Switzerland and Canada: a mixed-method study. BMC Public Health. 2016;16: 1–10.
- 34. Aenishaenslin C, Bouchard C, Koffi JK, Pelcat Y, Ogden NH. Evidence of rapid changes in Lyme disease awareness in Canada. Ticks Tick-Borne Dis. 2016;7: 1067–1074. pmid:27665265
- 35. Brans JP, Vincke P. A Preference Ranking Organisation Method: (The PROMETHEE Method for Multiple Criteria Decision-Making). Manag Sci. 1985;31: 647–656.
- 36. Greenhalgh T, Peacock R. Effectiveness and efficienty of search methods in systematic review of complex evidence: audit of primary sources. Br Med J. 2005;33: 1064–1065.
- 37. Jones J, Hunter D. Consensus methods for medical and health services research. BMJ. 1995;311: 376–380. pmid:7640549
- 38. Sawford K, Dhand N, Toribio J-A, Taylor M. The use of a modified Delphi approach to engage stakeholders in zoonotic disease research priority setting. BMC Public Health. 2014;14: 182. pmid:24552445
- 39. Ogden NH, Bouchard C, Kurtenbach K, Margos G, Lindsay LR, Trudel L, et al. Active and passive surveillance and phylogenetic analysis of Borrelia burgdorferi elucidate the process of Lyme disease risk emergence in Canada. Environ Health Perspect. 2010;118: 909–914. pmid:20421192
- 40. Drebot MA, Lindsay R, Barker IK, Buck PA, Fearon M, Hunter F, et al. West Nile virus surveillance and diagnostic: A Canadian perspective. Can J Infect Dis. 2003;14: 105–114. pmid:18159433
- 41. Berrang-Ford L, McLean JD, Gyorkos TW, Ford JD, Ogden NH. Climate change and Malaria in Canada: a systems approach. Interdiscip Perspect Infect Dis. 2009;2009: 13.
- 42. Schwartz KL, Giga A, Boggild AK. Chikungunya fever in Canada: fever and polyarthritis in a returned traveller. Can Med Assoc J. 2014;2014.
- 43. Ogden N, Milka R, Caminade C, Gachon P. Recent and projected future climatic suitability of North America for the Asian tiger mosquito Aedes albopictus. Parasit Vectors. 2014;7: 532. pmid:25441177
- 44. Paz S. Climate change impacts on West Nile virus transmission in a global context. Philos Trans R Soc Lond B Biol Sci. 2015;370. pmid:25688020
- 45.
Brans JP, Mareschal B. PROMETHEE-GAIA. Une méthodologie d’aide à la décision en présence de critères multiples. l’Université de Bruxelles. Paris, France: Ellipses; 2002.
- 46.
Brans J-P, Mareschal B. Promethee Methods. Multiple Criteria Decision Analysis: State of the Art Surveys. Springer New York; 2005. pp. 163–186.
- 47. McGregor S, Henderson KJ, Kaldor JM. How Are Health Research Priorities Set in Low and Middle Income Countries? A Systematic Review of Published Reports. PLoS ONE. 2014;9. pmid:25275315
- 48.
Horby P, Hoa N, Pfeiffer D, Wertheim HL. Drivers of Emerging Zoonotic Infectious Diseases. In: Yamada A, Kahn LH, Kaplan B, Monath TP, Woodall J, Conti L, editors. Confronting Emerging Zoonoses. Springer Japan; 2014. pp. 13–26.
- 49. Pavlin BI, Kool JL, Samo MH, Gerstel L, Working Group on National Notifiable Diseases Prioritization of the Federated States of Micronesia Department of Health and Social Affairs. A standardized process for developing a national notifiable diseases list in a pacific island setting. Asia Pac J Public Health. 2010;22: 279–288. pmid:19666949
- 50. Abelson J, Giacomini M, Lehoux P, Gauvin F-P. Bringing “the public” into health technology assessment and coverage policy decisions: From principles to practice. Health Policy. 2007;82: 37–50. pmid:16996637
- 51. Linkov I, Satterstrom FK, Kiker G, Seager TP, Bridges T, Gardner KH, et al. Multicriteria Decision Analysis: A Comprehensive Decision Approach for Management of Contaminated Sediments. Risk Anal Int J. 2006;26: 61–78. pmid:16492181
- 52. Kiker GA, Bridges TS, Varghese A, Seager TP, Linkov I. Application of multicriteria decision analysis in environmental decision making. Integr Environ Assess Manag. 2005;1: 95–108. pmid:16639891
- 53. Slovic P. Perception of risk. Science. 1987;236: 280–285. pmid:3563507
- 54. Krewski D, Lemyre L, Turner MC, Lee JEC, Dallaire C, Bouchard L, et al. Public Perception of Population Health Risks in Canada: Health Hazards and Sources of Information. Hum Ecol Risk Assess Int J. 2006;12: 626–644.
- 55. Wildavsky A, Dake K. Theories of Risk Perception: Who Fears What and Why? Daedalus. 1990;119: 41–60.
- 56.
Confalonieri UE, Menne B, Akhtar R, Ebi KL, Hauengue M, Kovats RS, et al. Climate Change 2007: Impacts, Adaptation and Vulnerability. Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. In: Parry ML, Canziani OF, Palutikof JP, van der Linden PJ, Hanson CE, editors. 2007: Human health. Cambridge, UK: Cambridge University Press; 2007. pp. 391–431.