Introduction

Most common chronic diseases such as cardiovascular diseases, cancers and diabetes are leading causes of death, accounting for 71% of all deaths and 43% of the global burden [1]. Modifiable behavioural risk factors such as poor eating or food habits increase the risk of chronic diseases [2]. Raising awareness of scientific knowledge about food and nutrition among the public can help improving health overall, and prevent chronic diseases [3].

The dissemination of nutrition information in the media is a potential mean for promoting knowledge about appropriate food choices [4–5]. Media (including radio, television, newspapers, internet and social media) today include a wealth of information about food and nutrition. For example, a recent Google search showed over half a million results related to the terms “nutrition advice”. Media has been shown to have a potential impact on knowledge and awareness of the public of health issues related to the field of nutrition [6–7]. However, this information is often misleading and contradictory [8–10]. According to recent research, the public is regularly exposed to mediocre or poor quality information about nutrition on websites [8] as well as in newspapers [9]. Approximately two-thirds of newspapers related to dietary advice are based on low-quality scientific evidence [10].

Online resources or websites (e.g., Google searches, YouTube) were the most popular source of nutrition information, between 2003 and 2018, among adults [11–12]. However, to the best of our knowledge, there are no online resources that formally evaluate and communicate the veracity of contemporary claims about nutrition based on scientific evidence. Therefore, we have developed a nutrition web-based information resource for the general public named Nutrimedia that provides rigorous evaluations of claims about nutrition. In this article, we describe how we have developed this resource and present an overview of its characteristics, contents and media impact. The research protocol is available in Spanish from the authors upon request.

Methods

Scientific evaluation of nutrition claims

For each claim selected, we developed a scientific report using a systematic and explicit process shown in Fig 1 that includes:

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Nutrimedia evaluation process.

https://doi.org/10.1371/journal.pone.0232393.g001

1. Formulation of structured clinical question.

For each nutrition claim, we formulated a structured clinical question in a PICO (participant, intervention, comparison and outcome(s)) format. We included a maximum of four key public-important outcomes (e.g., death, cancer or cardiovascular events).

2. Identification and selection of the evidence.

To identify and select the best available scientific evidence, we conducted a search prioritizing clinical practice guidelines (CPG) and systematic reviews (SR). We searched on electronic databases (e.g., we searched in MEDLINE and Cochrane Database of Systematic Reviews for SR) (S1 Table), combining at least one MeSH term and free-text terms, with title and/or abstract restrictions, for each PICO component. We also searched grey literature (e.g. Google Scholar), governmental and institutional sources (e.g. World Health Organization, U.S. Department of Agriculture, Spanish Agency for Food Safety and Nutrition) and scientific societies websites (e.g., Sociedad Española de Nutrición Comunitaria).

We included CPGs and/or SRs that used systematic methods to search and identify the evidence in two or more databases (e.g., MEDLINE, EMBASE) and that evaluated the risk of bias of research evidence [13]. When we obtained more than one CPG or SR of similar quality we prioritized the most recent. If not available, we included primary studies prioritising randomized clinical trials.

3. Synthesis of the evidence.

We summarized the following information of each of the documents selected: 1) objectives, 2) methods, 3) main results, and 4) conclusion of the authors (if applicable) (S2 Table).

4. Evaluation of the certainty of the evidence.

We assessed the certainty of the body of evidence (CPG or SR or primary studies) about the effects of nutrition interventions using the GRADE approach [14]. GRADE classifies the certainty of evidence as high, moderate, low, or very low for each outcome of interest within the same clinical question. GRADE categorizes randomized controlled trials (RCTs) as high certainty, whereas observational studies start as low certainty. An available body of RCTs can be downgraded on the basis of five factors: study limitations (risk of bias), indirectness, imprecision, inconsistency of results, and publication bias. In the case of observational studies, one can consider rating up on the basis of three factors: large magnitude of an effect, dose-response gradient, and plausible residual confounding (Table 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Factors of the certainty of the evidence.

https://doi.org/10.1371/journal.pone.0232393.t001

For each evaluation, we generated a Summary of Findings (SoF) table using GRADEPro [15]. SoF tables provide a structured outline of the number of studies and number of participants included for each outcome of interest, certainty (or confidence) of the evidence, and the results both in relative and absolute terms (S3 Table). If the certainty of evidence differed across critical outcomes, the overall certainty of evidence is the lowest certainty of any of the public-important outcomes [16].

5. Conclusion.

We classified the veracity of each claim according to the certainty of the evidence into seven categories: true, probably true, possibly true, possibly false, probably false, false, and uncertain. For example, we assigned true or false, probably true or false, possibly true or false and uncertain conclusion when the overall certainty of the evidence related to a claim was high, moderate, low and very low (or no studies), respectively (Table 2).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. The certainty of the evidence and the veracity of the claims evaluated.

https://doi.org/10.1371/journal.pone.0232393.t002

Results

Nutrimedia was launched in Spanish in November 20, 2017. During the first year and a half, we published 30 evaluations, of which 12 (40%) were questions from the public, 9 (30%) were common beliefs, 7 (23%) were newspaper claims, and 2 (7%) were advertising claims. Most evaluations were related to raw or processed foods (70%; 21/30), followed by diets (20%; 6/30) and supplements (10%; 3/30); and the majority of them were based on SR (80%; 24/30). Nearly half of the claims were classified as “uncertain” (47%; 14/30), followed by “possibly true” (13%; 4/30), probably true (13%; 4/30), “probably false” (10%; 3/30), “possibly false” (7%; 2/30), false (7%; 2/30) and “true” (3%; 1/30) (Table 3). An example of the evaluation of the veracity of a claim is presented in the S4 Table.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Type and veracity of the evaluated claims in https://www.upf.edu/web/nutrimedia until May 20, 2019.

https://doi.org/10.1371/journal.pone.0232393.t003

Nutrimedia has other contents such as “Eating with science” (Comer con ciencia), “To know more” (Para saber más), and “About Nutrimedia” (Sobre Nutrimedia). The “To know more” section provides short videos to promote critical thinking and to facilitate the understanding of some methodological concepts (e.g., what is the GRADE approach and how certainty of the evidence was evaluated; guidelines for the public to interpret information on nutrition; guidelines for journalists to report on nutrition; relevant links in the field of nutrition, and a glossary). In the section “About Nutrimedia”, users can find out who we are, the scientific methodology that we applied to evaluate claims on nutrition, the press releases and the media coverage.

In the section “Ask Nutrimedia”, available from November 20, 2017, to May 5, 2019, the public assessed the interest of 10 questions in an online survey. Of the 12 questions from the public that were evaluated, 6 were the most voted of the 10 closed-ended questions and 6 were posed by users in the survey space for comments and suggestions. Some other questions about nutrition of general interest, most of them posed by the public, were answered by experts in articles and interviews (provided as podcasts) in the “Eating with science” section. The number of online survey respondents was 333 (55.2%, 182/333, general public; and 45.8%, 151/333, health professionals). Fifty-eight respondents (17.4%, 58/333) provided positive (51.7%, 30/58) (e.g., “An excellent idea to combat all the misinformation”; “Thank you for helping me get back to believing in science”) and neutral (5.2%, 3/58%) comments or suggestions (43.1%, 25/58) (e.g., “It would be interesting to receive web updates by email”; “It would be tremendously helpful to give healthy and tasty recipes”).

The number of website users was 47,265 (46,052 new users) since the website was launched (November 20, 2017) and for the first 18 months (until May 20, 2019). The number of page views in this period was 181,360. During this period, Nutrimedia was cited by 94 newspapers, news agencies and newswires included in Factiva Dow Jones database; of these 94 citations, 55 were in Spanish, 33 in Catalan, 4 in English and 2 in Portuguese. According to the data provided by Acceso 360, Nutrimedia was cited 78 times by newspapers and 386 times by websites from Spain and 14 other countries, mostly in Latin America. In addition, the Nutrimedia project and its evaluations were broadcasted on several Spanish radio and television channels. In that period, the estimated economic value of the 78 pieces of news published in printed editions of newspapers was €152,507.55.

Nutrimedia has also been cited by prominent Spanish dieticians and nutritionists. For example, Nutrimedia is referenced in 5 of its 12 recommendations for the general public in the 2018 food-based dietary guideline "Small changes to eating better" by the Public Health Agency of Catalonia [20].

Finally, Nutrimedia was well placed in nutrition searches in Spanish with Google, whose algorithm rates the pages according to the criteria of expertise, authority and trustworthiness [21]; e.g., in a Google search with the terms “meat cancer” (search performed on 2019 September 30, using a logged-out Chrome browser cleared of cookies and previous search history), Nutrimedia was ranked as the third listed information source in a list headed by the World Health Organization.

Discussion

Nutrimedia is the first web-based resource for the general public that evaluates the veracity of media claims about nutrition based on the certainty of evidence, and to communicate the results in plain language and friendly presentation formats. We have evaluated 30 claims and classified nearly half of them as uncertain based on GRADE. During the first year and a half, Nutrimedia had a considerable impact on the media (about one impact per day).

There are some other online resources with different purposes and target populations. “Practice-based evidence in nutrition” provides the latest evidence in practice-based nutrition questions applying the GRADE approach, but this evidence-based decision support system is intended for dieticians/nutritionists and students [21]. “Behind the Headlines”, from the UK National Health Service, analyses critically media claims related to nutrition, but does not assess the certainty of the evidence [22]. Cochrane Nutrition provides nutrition-related Cochrane systematic reviews, but there is often a communication gap between these reviews and the public [23]. To avoid this, Cochrane centers implement alternative ways to make evidence accessible in plain language summaries [24] and blogshots (infographics that summarize the evidence) in English [25], Spanish [26] and other languages. By overcoming these limitations, Nutrimedia is an innovative resource for disseminating quality nutrition information, and making it accessible to the public. Therefore, we believe that Nutrimedia is a pioneering initiative that promotes critical thinking and can have an impact on the food and nutrition literacy of the general public [27].

Nutrimedia has several strengths. The website is user-friendly because no registration is required, the evaluations are accessible within a few clicks and its content is multi-layered and multiformat. For each evaluation, we have applied a rigorous and explicit methodology using GRADE. GRADE approach represents a systematic, explicit and transparent methodological framework for grading the certainty of evidence and it has already been endorsed or used by over 100 organizations, the World Health Organization and the Cochrane Collaboration [28–29] among others. Finally, our project team has extensive knowledge and experience in the fields of nutrition, evidence-based medicine, methods, communication and journalism.

Nutrimedia also has some limitations. Firstly, it is only available in Spanish. Secondly, we used pragmatic search strategies favoring precision over sensitivity [30]. Thirdly, this project has no system yet for monitoring and continuous updating of the evaluations. Finally, analysis of its impact, usefulness, accessibility and understandability is still limited.

Supporting information

Acknowledgments

We would like to thank Darío Lopez, Andrea Juliana Sanabria, Mónica Ballesteros, Carolina Requeijo Lorenzo, Karla Salas Gama, Paulina Fuentes, Laura Martínez García, and Alba Irigoyen for their contribution in this project. We would also like to thank to Victoria Leo for her assistance with the English text.