Who can you trust? Credibility assessment in online health forums

https://doi.org/10.1016/j.hlpt.2013.11.003Get rights and content

Highlights

  • We develop a new model for how information credibility is assessed in online health forums.

  • We identify the different ways that different types of information are assessed in online health forums.

  • We suggest that the ways in which credibility is assessed in online health forums deserves policy attention.

Abstract

As the cost of health care rises governments everywhere are examining how on-line services can replace or augment face-to-face services. Consequently, many health bodies are establishing on-line health forums where patients can share ideas with, or solicit information from, both other patients and health professionals. In the wake of this trend, many on-line forums have arisen which do not have the imprimatur of official government services but are run and managed by private individuals sharing experiences outside of the patient-clinician channel. This phenomenon creates risks and challenges for users who need to evaluate the credibility of unknown and often anonymous contributors to these forums. This paper examines how users assess the credibility of the information in these forums. Five criteria were discovered in the first stage of the work. We then quantitatively tested the relationship between those criteria based on two types of information. Our analysis shows that different criteria are used by participants in online health forums for scientific information and experiential information. We used these novel findings to develop a model for how information credibility is assessed in online health forums. These findings provide important lessons for health promotion bodies considering how to encourage the sharing of valuable health information on-line as well as guidelines for improved tools for health self-management.

Introduction

The Internet is changing the way that people receive medical information, from the traditional doctor–patient model to one where patients supplement or even replace that interaction with a search for information and advice on the Internet.

People appear to be attracted to the anonymous and convenient information seeking that is possible via the Internet. However, unlike in face-to-face consultations with a doctor, there is no guarantee that the information and advice is objective, truthful, or up-to-date, and there is no follow-up to ensure that any information provided is used appropriately.

Information obtained from government or institutional sources typically does describe best practice, and published studies indicate that they are generally perceived to provide information that is highly credible, accurate, current, and complete [1], [2]. However, research into health promotion by government agencies suggest that there are no clear policies and often no clear evidence-base around what is promoted on such sites [3]. Thus, despite some positive attributes, the information from these sites is not always ideal: it is frequently jargon-laden, difficult to comprehend, difficult to incorporate into treatment plans, and its inclusion not fully justified. Political ideologies and the values of individuals running such sites can also effect the type of information published [3].

Thus, many people also seek advice from social media sites, including social networking sites, blogs, and wikis. This type of interaction has been often treated with suspicion by medical professionals because the interactions are typically between patients with no medical training, and there are no formal mechanisms to ensure that advice is accurate, up-to-date, or even truthful.

This suspicion, however, may not be warranted. Some research indicates that social media sites are not a substitute for high quality information, but rather complement those sites by providing emotional support and information about the lived experience of a condition (experiential information) from other patients. However, very few studies have been conducted into this type of behaviour, so it is currently unclear how patients evaluate social media sites. This is even though previous research has identified the potential danger of incorrect information being put into the public domain and the difficulties in assessing the credibility of such information [4].

Online health forums (OHFs) have been chosen for this research, from among all user-centric social media applications. Firstly, OHFs highlight text-based content and are also the most popular tool adopted [5]. In contrast to blogs, OHFs centralise users in one place and better maintain users' anonymity. As opposed to wikis, OHFs enable social interaction and influence between users. To maintain the communication dynamics between users and assess online health information produced by anonymous users, we exclude those OHFs that are explicitly involved with and moderated by medical professionals such as general medical practitioners, specialists, and registered nurses which tend to be more like Q and A and lack emotional exchange. By removing a central authority and medical professionals, we can better answer the research questions:

What types of information are generated by OHF's? How do people assess health information generated by other anonymous end-users in OHFs? Do people use different criteria to assess different types of information?

The remainder of this paper is structured as follows: we start with a review of the key concepts and theories that guide our exploration, and then describe the methodology used to execute this research. We then present the emerging results from two studies, and finally discuss the insights and implications of our findings. We show that fears that online forums provide high risk and under-validated information are generally unfounded. Instead, online health forums appear to be a useful complement to institutional sites, and indeed provide a vital service that has to date been largely ignored by policy makers.

There has been significant work done on what type of information health consumers require [6]. This work suggests that people want information falling under the following categories: Disease entity information (what is the disease?) [7]; Time information (how does it progress?); Self-information (why me?); Self-disease (what will happen) [8]; Self-disease-time (how will my condition change?) [6]. However research suggests that many doctors are unable to satisfy these information needs, either because of a lack of information or a lack of empathy, or just reasons of time and efficiency [9]. “Mutual acceptance, more emotional care, empathy” [9, p. 4] are some of the areas where patients feel the face-to-face provision of health information could be improved, and are cited as reasons for the increased use of On-line Health Forums as a way of satisfy patient needs, not just for different or greater information but for a different style of information communication [10].

Online Health Forums (OHFs) have evolved as a strong Internet presence in the area of both physical and mental health. These may be unstructured discussion groups or may be led by an individual, usually a non-professional, who shares the problem that the group addresses [11]. The prerequisite of the shared medical condition sets OHFs apart from other online communities. The usefulness of the support provided by these groups has been attested to in studies of eating disorders [12] and depression [13]. They have been found to allow patients to form supportive bonds with other people [11] and have been reported as being helpful by many users [14] particularly in providing the sort of information that health consumers want, as discussed in the previous section. Factors working against use have often been technological, such as error prone designs, rather than related to the benefits derived from participation [15].

What drives people to OHFs is the possession of a sense of alienation, feelings of isolation, anxiety about treatment and misconceptions and misinformation [16]. Emotional distress from a medical condition can be alleviated by satisfying informational needs [17], gaining emotional support [18] and engaging social comparison [19]. Thus, informational and emotional supports are appreciated and positively received by OHF users [20], [21].

OHFs, like other user-centric platforms, place end-users under the spotlight. While medical professionals derive their knowledge from the experience of the majority of patients, patients obtain medical knowledge from living with their medical conditions every day. Thus, patients can articulate not only the location, intensity and duration of sensations, but also how they perceive the experience, what they think and why they behave in particular ways [22], [23]. The narrative, personal, discursive and experiential nature of patients′ knowledge challenges the existing credibility criteria used to assess more standard, objective, factual scientific knowledge given by medical professionals. The criteria are particularly tested by the fact that the “wisdom of crowds” can be less accurate than information produced by experts [24].

Consider the following case study:

  • “Little Charly Johns was a trouper.

  • She was only six years old and had cancer – but she fought it with determination. She was in and out of hospital as the disease advanced and retreated.

  • It was tough too for her mother Anna. She joined the Macmillan online cancer forum.

  • There she found support and help from people who knew exactly what she was going through.

  • For two years, Anna kept them updated on Charly's progress.

  • “On the whole she is doing great,” she wrote. “She is happy, lively, giggly and very easily excitable. She is always the first to laugh at anything and the last to stop. Nobody could look at Charly now and have any idea of the things she has endured these past 14 months.”

  • But in November last year, Charly lost her fight for life. On the Macmillan forum there was an outpouring of grief. People wrote poems in Charly's memory. They painted their fingernails pink in accordance with her last wishes – even men.

  • But it was all a lie. Charly did not exist. Neither did Anna.” (BBC News Magazine, The Cruelest of Internet Hoaxes, 1 June, 2012).

This case study highlights why being able to assess the credibility of information contained in Online Health Forums is crucial to users. In the OHF context users are concerned about two things: the quality of information and the credibility of the information source, as the consequence of using the information shared in these forums is significant. Patients risk not just believing incorrect treatment advice, but also the sort of emotional exploitation we see in the case study outlined above.

The perceived quality of information is closely related to how credible [25] or believable we regard it to be and in fact some researchers regard credibility as a subset of quality [26]. Information quality is judged by completeness, accuracy, usefulness and timeliness [25], [27], [28], some of which may be more or less difficult to judge in the online context.

Credibility is a complex factor, with research commonly partitioning the concept into multiple components such as information credibility, source credibility, media credibility and web credibility. Information credibility can be evaluated based on information itself, heuristic cues and interaction [25]. As one heuristic cue, the credibility of the source of the information can be used to signal information credibility, because a credible source is more likely to produce credible information [29]. Cognitive authority, similar to source credibility, specifies a person as a credible source [30].

There are two aspects to source credibility: trustworthiness (an information provider's intention to tell the truth or give unbiased information) and expertise (an information provider's ability and/or qualification to know the truth of a topic) [31]. Source credibility can be judged through surface credibility (simple inspection of superficial features), presumed credibility (stereotyping), reputed credibility (source label such as doctor or professor), and experienced credibility (direct experience with a source over time) [32]. Because the environment of the user-led on-line health forum is anonymous and does not involve professionals, only the first of these, surface credibility, can be realistically assessed – which is part of the difficulty of assessing credibility in this context.

Assessing credibility is impacted by the general factors that affect all aspects of information assimilation. Any assessment of information on the Internet is governed by many factors such as age and gender [33] or personal attributes and cognitive style [34]. In addition, in any cognitive task users tend to utilise the least cognitive effort by taking shortcuts and processing a small amount of information for decision making [35]. Prominence-interpretation theory, for example, argues that prominent aspects of a website dominate the evaluation process [36]. The limited capacity model of message processing [37] reinforces this view, proposing that limited cognitive capacity forces people to select only salient characteristics to encode, store and retrieve messages, particularly in situations where people feel overwhelmed. In these situations, people tend to seek mental shortcuts such as heuristic cues (stereotypes and categories) to minimise the amount of time and cognitive effort needed to process a message [38]. Empirical studies provide some support for these theories: for example, although information consumers may report that they verify all online information, in practice, verification methods that minimise time and cognitive effort tend to be used [39]. Heuristic cues used as part of this process include lists of links, the design of the site, how information is organised, brand names, street addresses and the length of a message [40], [41], [42].

This position has been challenged by dual-process theories, suggesting that people can be motivated to exert cognitive effort. The heuristic-systematic model (HSM) [43] and the Elaboration Likelihood Model (ELM) [44] assert that when individuals′ motivation and ability to process are high, they are inclined to scrutinise carefully all relevant information stemming from the source, message, context, and themselves (e.g. emotions) in an attempt to make an accurate judgment (referred to as the systematic or central route). When either motivation (e.g., personal relevance) or ability to process is low (e.g., insufficient information or knowledge), attitudes can be changed by relatively low-effort processes (referred to as the heuristic or peripheral route). In the health domain, the higher stakes of following the wrong advice and disclosing to the wrong people can motivate users to use more cognitively demanding analytic processes. Nonetheless, without years of systematic medical training, people may not know when to be sceptical and may be more inclined to believe misinformation [45].

Theories of cognition, such as the above, only explain part of the process with regard to how information is selected and assimilated when both quality and credibility is important. They do not explain how information credibility is assessed when users seek satisfactory solutions in specific contexts. This research addresses this question.

Section snippets

Research design and methods

The research design contains two phases of investigation. The first phase was conducted to understand what criteria are used in evaluating user-generated health content and reveal the criteria applied by OHF users. Because credibility assessment of user-generated content in OHFs is barely explored and the traditional strategies may not be suitable, it is crucial to understand the phenomenon within a rich social context. The second phase was conducted to validate the findings from Study 1, and

Discussion

On-line health information comes in a number of forms and users employ multiple strategies in deciding which information to trust or rely on. While the information supplied on official sites is generally seen as evidenced based and scientific, in fact research shows it often reflects government health policy or the particular values of its authors. Consequently, as we have seen in this research, large numbers of people are turning to often anonymous sources on community health forums where they

Conclusion

This paper makes a unique contribution to an understudied area of research by providing insight into how people assess the credibility of user-developed on-line information in the health context. It helps solve the enigma of how people can have the confidence to move away from traditional sources of medical advice to trust the information provided by strangers.

The results show that a different mix of assessment criteria and preferences on each criterion are employed to evaluate the credibility

Ethical approval

Ethics approval was sought and granted by the Engineering Human Ethics Advisory Group, University of Melbourne.

Funding

None.

Competing interests

None declared.

References (77)

  • D. Raphael

    The question of evidence in health promotion

    Health Promot Int

    (2000)
  • F. Lee et al.

    Virtual community informatics: a review and research agenda

    J Inf Technol Theory Appl

    (2003)
  • L.J. Trevena

    A systematic review on communicating with patients about evidence

    J Eval Clin Pract

    (2006)
  • F. Timmins

    A review of the information needs of patients with acute coronary syndromes

    Nurs Crit Care

    (2005)
  • J. Leimeister et al.

    Evaluation of a systematic design for a virtual patient community

    J Comput Mediated Commun

    (2005)
  • Q. Chen et al.

    Internet community group participation: psychosocial benefits for women with breast cancer

    J Comput Mediated Commun

    (2005)
  • G. Castelnuovo

    New and old tools in psychotherapy: the use of technology for the integration of the traditional clinical treatments

    Psychother: Theory Res Pract Training

    (2003)
  • A.A. Celio

    Reducing risk factors for eating disorders: comparison of an Internet- and a classroom-delivered psychoeducational program

    J Consult Clin Psychol

    (2000)
  • K.A. Dyer et al.

    Internet use for web-education on the overlooked areas of grief and loss

    CyberPsychol Behav

    (2000)
  • Wolf, G. The data-driven life. In: New York Times. NY;...
  • J. Grudin

    Groupware and social dynamics: eight challenges for developers

    Commun ACM

    (1994)
  • R.J. McKenna et al.

    Rehabilitation and supportive care of the cancer patient

  • X. Hu

    The prepared patient: information seeking of online support group members before their medical appointments

    J Health Commun: Int Perspect

    (2012)
  • P.A. Thoits

    Mechanisms linking social ties and support to physical and mental health

    J Health Soc Behav

    (2011)
  • L. Festinger

    A theory of social comparison processes

    Hum Relat

    (1954)
  • J. Preece

    Online communities: designing usability, supporting sociability

    (2000)
  • K.B. Wright

    Social support within an on-line cancer community: an assessment of emotional support perceptions of advantages and disadvantages, and motives for using the community from a communication perspective

    J Appl Commun Res

    (2002)
  • E.J. Langer

    Counterclockwise: mindful health and the power of possibility

    (2009)
  • A. Hartzler et al.

    Managing the personal side of health: how patient expertise differs from the expertise of clinicians

    J Med Internet Res

    (2011)
  • L.H. Rector

    Comparison of wikipedia and other encyclopedias for accuracy, breadth, and depth in historical articles

    Ref Serv Rev

    (2008)
  • A.J. Flanagin et al.

    Perceptions of internet information credibility

    J Mass Commun Q

    (2000)
  • Y. Lee

    Crafting rules: context-reflective data quality problem solving

    J Manage Inf Syst

    (2003)
  • A.R. Fragale et al.

    Evolving information credentials: the (Mis)attribution of believable facts to credible sources

    Pers Soc Psychol Bull

    (2004)
  • S.Y. Rieh

    Judgment of information quality and cognitive authority in the web

    J Am Soc Inf Sci Technol

    (2002)
  • M.J. Dutta-Bergman

    The impact of completeness and web use motivation on the credibility of e-health information

    J Commun

    (2004)
  • S. Tseng et al.

    Credibility and computing technology

    Commun ACM

    (1999)
  • Hu J, et al. Demographic prediction based on user's browsing behavioural. In: Proceedings of the international World...
  • S.P. Smith et al.

    Putting yourself in the picture: an evaluation of virtual model technology as an online shopping tool

    Inf Syst Res

    (2013)
  • Cited by (80)

    • Exploring crowdsourced self-care techniques: A study on Parkinson's disease

      2023, International Journal of Human Computer Studies
    • Supporting consumers' informed food choices: Sources, channels, and use of information

      2023, Advances in Food and Nutrition Research
      Citation Excerpt :

      Also, consumers have idiosyncratic views on the relative reliability of different sources of information and apply a range of different criteria to evaluate the reliability of nutrition and health-related information on the web, including quality of the argument, quality of the language, professional appearance of the website, and, if a source is cited, trustworthiness of the source. For user-generated content, user agreement plays a role in how reliable the information is perceived (Lederman, Fan, Smith, & Chang, 2014; Sillence, Briggs, Harris, & Fishwick, 2006). Advertising is a source of food information, but it also has a range of other functions.

    • How online health discussions make people perceive benefits?

      2022, Library and Information Science Research
    • Factors that determine a Patient's willingness to physician selection in online healthcare communities: A trust theory perspective

      2021, Technology in Society
      Citation Excerpt :

      The more services a physician is willing to offer voluntarily and gratis, the more efforts the physician makes. The degree of the physician's effort can represent the physician's benevolence [34,35]. The more benevolent the physician is, the more likely they are to gain the trust of patients, thus promoting the selection of patients.

    • Online health consumer behaviour: What informs user decisions on information quality?

      2021, Computers in Human Behavior Reports
      Citation Excerpt :

      The second category of research on quality OHI has focused on user perception and trust of OHI and its sources. Some of these works addressed for instance, factors that influence user perception of OHI quality on diet-nutrition (Jung et al., 2016), relationship between visual design and user perceptions of OHI website credibility (Sbaffi & Rowley, 2017), and user trust judgments of OHI (Lederman et al., 2014; Rowley et al., 2015). The third category on quality OHI research has looked at evaluation criteria, instruments and methodologies used in assessing quality OHI such as Zhang et al. (2015) where a systematic review of evaluation criteria, instruments and methodologies was conducted and in Oroszlányová et al. (2018) where quality of health web documents was predicted using their characteristics.

    View all citing articles on Scopus
    View full text