Introduction

Diagnostic error is a problem at the forefront of healthcare in the United States [1]. A recent National Academies of Science report concluded that diagnostic error is responsible for approximately 10% of patient deaths and between 6% and 17% of hospital adverse events [2]. Although diagnostic reasoning – and more broadly, clinical reasoning – is essential to patient care, our understanding of what influences it is limited, making it difficult to mitigate the effects of reasoning errors on patient care.

Clinical reasoning can be defined as the integration of clinical information, medical knowledge, and contextual factors to make patient care decisions [3], [4], [5], [6]. A challenging phenomenon in clinical reasoning undoubtedly leading to diagnostic error is context specificity: when a physician arrives at two different diagnoses for two different patients who actually have the same symptoms, findings, and, ultimately, the same diagnosis [7], [8]. In a recent study of think-aloud reflections on reasoning, for instance, physicians note the presence of contextual factors (e.g. patient affect, diagnostic suggestion), which for some physicians seem to create uncertainty and difficulty with closure of the encounter [9]. We sought to build on this work by using a definition of context grounded in educational theory that could be applied to the rich complexity of practice settings in medicine. Doing so enabled us to empirically explore the phenomenon, one that is a source of unwanted variation in patient care. Thus, we defined case context as – going beyond case content – the individual (e.g. sleepiness, burnout, suggesting an incorrect diagnosis), physical (e.g. non-working electronic health record, allotted time for an appointment), and social aspects (e.g. challenging a physician’s credentials) of a patient encounter to include the participants, the setting, and their interactions [10]. In this view, “context is not a fixed set of surrounding conditions, but a wider dynamical process of which the cognition of an individual is only a part” (p. xii) [11].

We approach the effects of context specificity on clinical reasoning using situated cognition theory, which argues that thinking (here, clinical reasoning) is inextricably bound within the context where it happens (complex interactions among patient, physician, and setting evolving over time) [10], [12]. This differs from traditional approaches to clinical reasoning that focus solely on the content of cases (e.g. symptoms, patient history, physical exam results) through mechanisms like illness scripts [13]. Instead, situated cognition recognises the importance of the participants, environment, and interactions therein, as noted in the above definition, offering a useful framework for understanding the effects of context specificity. Using this framework, we group contextual factors into those associated with the physician (e.g. fatigue), the patient (e.g. circuitous history), and the clinical environment (e.g. pressure to multitask [9], [14]; see Figure 1).

Figure 1:

Contextual factors in clinical reasoning.

Another important concept is case specificity, which argues that different content, or different cases (e.g. diagnoses), can lead to different clinical reasoning performance. In explorations of context specificity, something more than case content (i.e. case specificity [15]) is influencing clinical reasoning: namely, what we refer to as contextual factors. While contextual factors can positively influence clinical reasoning, our work to date and the current study focus on understanding diagnostic error and factors that can inhibit it. Recent work suggests that contextual factors can inhibit clinical reasoning in both novice and more experienced physicians [7], [16]. This aligns with work in psychology suggesting that expert performance is not a stable trait, but a shifting, situation-based state [17], [18], [19].

Prior work suggests that one mechanism through which contextual factors may affect physicians is mental effort or cognitive load: constraints on how many information units one’s working memory can hold and process at a time [7], [20], [21]. The assumption is that as a clinical encounter becomes more complex, with the introduction of different contextual factors, the associated cognitive load increases, potentially impairing clinical reasoning performance [7], [21]. Currently, however, relatively little is known about the relationship between cognitive load and clinical reasoning performance. Combining situated cognition’s focus on the interactive elements of the clinical environment with cognitive load theory’s focus on individual cognitive management of those elements offers an opportunity to understand how different inhibiting contextual factors may influence clinical reasoning.

Context specificity is recognised as an important problem in medical education, and has been examined through the lens of prototype theory [22], [23], [24]. These earlier studies isolated particular elements of the case presentation like the language used, the timing of the presentation of a tentative differential diagnosis, and the familiarity of patient characteristics (e.g. name, age) and examined how they affected participants’ (a) determination of a diagnosis and (b) identification of features of a case [23], [24], [25], [26], [27], [28], [29], [30], [31]. While this theoretical approach allowed them to tease apart how particular details of case presentations can affect the choice of leading diagnosis, it did not provide a framework for understanding the overall clinical situation in which this diagnostic choice takes place. Moreover, this work and recent work on contextual factors [14], [32] have looked only at the endpoint of diagnosis or feature identification, not the broader clinical reasoning process (e.g. evidence a physician offers for a given diagnosis) in which these decisions are grounded. Other recent work has taken up a socially situated theoretical model to better understand context specificity, but it has been exploratory and, as such, was conducted with no control group and relatively few participants [7], [14], [32], [33], [34], [35].

Decrements in clinical reasoning can lead to unwanted variance in performance, patient morbidity and mortality, and/or excessive cost [2], so we must find innovative ways to examine and enhance clinical reasoning and context specificity more closely. Thus, the purpose of this study was to investigate both the presence of and the mechanisms behind context specificity, using contextual factors found to be important in our prior work [7], [9], [34]. Because clinical reasoning performance is related to years of professional experience [3] (our proxy for expertise), we also investigated whether experience affected performance. We also controlled for potential ordering effects. We asked:

1. Is there a difference in clinical reasoning performance (as measured by open-ended diagnostic questions) when physicians diagnose cases with and without contextual factors?

2. Is there a difference in self-rated cognitive load (i.e. mental effort) when physicians diagnose cases with and without contextual factors?

We hypothesised that participants would perform better and rate their mental effort as lower in diagnosing cases without inhibiting contextual factors. Further, based on the notion that expertise is a situation-based state (not an invariant trait), we hypothesised that contextual factors would affect physicians equally across experience levels. Finally, because participants regularly see far more than two cases daily, we hypothesised that there would be no ordering effects.

Materials and methods

To explore how context specificity may impair clinical reasoning, we designed two video simulation cases depicting patients with typical presentations of common diseases in primary care practice: new-onset diabetes mellitus and unstable angina. We believe that videos represent the optimal way to conduct this investigation as videos (widely used training tools) ensure all participants receive an identical “stimulus” to fully control both case content (identical content provided) and potentially relevant contextual factors (i.e. to empirically explore what may underpin context specificity). They lasted from just under 4 to 6.5 min and portrayed a clinical interview, a brief physical exam, and still screens of laboratory findings. We consulted with a group of internal medicine physicians to choose commonly encountered cases and contextual factors that are a part of everyday practice in internal medicine (and that tend to emerge with the types of cases we chose) to mitigate the effects of case (i.e. content) specificity [15] and enhance ecological validity. Prior to filming, the cases were screened by a panel of six medical education experts for authenticity and appropriateness (e.g. typicality).

Study participants were quasi-randomly assigned (based on their study day schedules) to one of two conditions: (a) diabetes case with inhibiting contextual factors (low English proficiency and a patient questioning the physician’s credentials); angina case without contextual factors, or (b) angina case with inhibiting contextual factors (misleading diagnostic suggestion, patient reports history circuitously); diabetes case without contextual factors (see Figure 2). Furthermore, because this is early work in this area and because multiple contextual factors are typical across a busy day in practice, we used several contextual factors and two cases. The case content was controlled (i.e. identical presenting symptoms, language and gestures to represent those symptoms, and physical findings) so that it was the same for both diabetes and both angina cases; the only differences were the contextual factors. Also, conditions were balanced to control for potential ordering effects (i.e. whether the contextual factor case came first or second). After watching each case video, participants answered questions about diagnosis and mental effort.

Figure 2:

Study design.

Results

Across all angina cases, the overall mean percentage scores on all PEF items ranged from 43% to 94% (m=70%). The MANCOVA results revealed significant differences between the conditions with and without contextual factors [Pillai’s trace=0.72, F=12.4, df=(6, 29), p<0.001, ηp2=0.72], with no effects for case order or years of experience. Levene’s test indicated equal variances for all dependent variables except the supporting evidence item (F=9.6, p=0.006). Follow-up univariate analyses indicated that participants performed significantly worse on all PEF items except additional history questions in the presence of a contextual factor, with large effect sizes [40] (see Table 2, showing the univariate effects). The ANCOVA of overall mean angina scores revealed significant differences between the conditions with and without contextual factors [F=82.7, df=(1, 34), p<0.001, η²=0.71], with a mean score of 77% without contextual factors [standard deviation (SD)=0.07] and 55% with contextual factors (SD=0.1).

Across all diabetes cases, the mean percentage scores on PEF items ranged from 37% to 84% (m=70%). MANCOVA results revealed no significant differences between the conditions with and without contextual factors [Pillai’s trace=0.33, F=2.3, df=(6, 28), p=0.07, ηp2=0.33] or for either of the covariates. (See Table 3). The ANCOVA of overall mean diabetes PEF score showed no significant differences between conditions [F=0.1, df=(1, 33), p=0.76, ηp2=0.003].

There was no significant difference in self-rated cognitive load with or without contextual factors for the diabetes [F (1, 40)=6.1, p=0.38] or angina [F (1, 40)=1.2, p=0.52] cases. Levene’s test indicated equal variances for both diabetes (F=0.61, p=0.61) and angina (F=0.55, p=0.65). We did, however, observe a trend in cognitive load: self-reported cognitive load was higher in the presence of contextual factors for both diabetes (m=4.85 for no contextual factors and m=5.72 for contextual factors) and angina (m=6.64 for no contextual factors and m=6.95 for contextual factors).

Discussion

In this investigation, we sought to experimentally test the phenomenon of context specificity in a group of experienced internists using a theoretically grounded approach. While context specificity in clinical reasoning has become an important area of study [7], [8], [9], [26], [29], [31], [34], [41], [42], [43], to the best of our knowledge, this is the first study to use a robust socially situated theoretical framework, carefully controlled stimulus, ecologically valid measure of clinical reasoning, and fairly large sample of participants. Using typical presentations of common diagnoses, and common contextual factors, our results suggest that the negative effects of context specificity may in part be dependent on case content, i.e. context specificity holds in our data for the angina, but not diabetes, case (with an aggregated PEF score and with five of six individual PEF measures). While prior context specificity work demonstrated significant correlations between PEF items and cognitive load, those studies did not include control groups without contextual factors [9], [34]. This study extends that work, demonstrating significant performance differences in an angina case across five domains: additional exam actions, problem list, differential diagnosis, leading diagnosis, and supporting evidence. Moreover, this effect held across years of experience, indicating the importance of deliberate practice [17], [19] and context in physicians’ continuing education: understanding case content is not adequate – physicians must carefully practice reasoning with that content across environments.

Although we found context specificity for the angina case, we did not find such an effect for the diabetes case, despite similar score ranges across the cases and our experts’ judgement that both cases were equally common and typical. This finding could be due to inadequate power or to the dose or quality of the contextual factors in the two cases (chosen for their ecological validity with respect to the content area); i.e. perhaps circuitous history poses more or different challenges to clinical reasoning than the agitated non-native English speaker. This could also be evidence of case specificity: some aspect of the content of the diabetes case could make it easier to circumvent contextual effects when compared to the angina case, which has greater acuity and fewer specific diagnostic signs. Thus, neither case content nor contextual factor is the sole predictor of clinical reasoning performance. As others have argued, there is no single cause for diagnostic errors, but a nuanced and complex system of interacting conditions, including context and content [15], [44].

Regarding self-reported cognitive load, while the scores trended in the expected direction for both cases (higher cognitive load with contextual factors), these trends were not statistically significant. This could be due to inadequate power or to problems with our single-item measure of cognitive load (i.e. inadequate sensitivity). This could also indicate that the cause of the performance decline in the angina case is not the result of increased cognitive load, but due to some other set of factors, such as different emotional reactions [45]. Future work in this area could elucidate this.

As the expertise literature would predict, controlling for years of experience did not eliminate the effect of contextual factors [18], [19]. In other words, context specificity effects are not limited to newer physicians. Thus, future support tools should be developed not just for residents, but for attending physicians as well. We did note, however, that more experienced physicians performed significantly better on the additional history and additional exam items. Perhaps these diagnostic tasks become more automated or scripted over the years than others (e.g. leading diagnoses). Again, this resonates with prior work indicating that tasks within clinical reasoning are equally as important as the broader content or context [15], [44], [46].

There are several important study limitations. First, the sample size of 39 participants is only 31% women and, moreover, is relatively small for the statistical test. Yet, this ratio is representative for the participants’ institutions and this is actually a large number of participants for a clinical reasoning study (one requiring 2 h of physician participant time). Second, we only ran two sets of cases, using different contextual factors in each (diabetes and angina), potentially affecting the interpretation (i.e. effects could be the result of a specific type of contextual factor). Yet, based on our prior work, we wanted to include various contextual factors educators and researchers have hypothesised to be important.

This study has several important practical implications. First, and perhaps most centrally for training and continuing education, these results indicate the importance of identifying areas other than content or medical knowledge that contribute to establishing diagnoses as the findings were demonstrated in both residents and attendings. Explicit education in cases with identified contextual factors – perhaps through simulation – could potentially mitigate the negative effects of context specificity. Future work might explore different contextual factors (e.g. appointment length) and their mitigation or elimination. Additionally, the theoretical model and proposed contextual factors could be explored in more authentic settings such as the clinics or wards to better understand context specificity.

Second, this study suggests that we may be underestimating the effect of case specificity on error. Diabetes and angina are two common content areas. While case specificity may explain less variance than individual items within cases in some contexts [15], it appears to have a significant effect on those items when certain contextual factors are introduced.

Third, while serving as an optimal platform for exploring context specificity, the videos themselves may have induced additional cognitive load. We should not assume that clinical reasoning “in person” is the same as clinical reasoning mediated through technology. As such, our findings may have implications for clinical reasoning in technology-enhanced contexts, such as telehealth.

In conclusion, our findings are consistent with expectations of situated cognition theory, which would predict that interactions among different contextual factors and different case content would engender different clinical reasoning challenges. Diagnostic error plagues our healthcare system, and we believe that a work like this can help illuminate the vexing phenomenon of context specificity. Additionally, this work points to the need for interventions to reduce unwanted performance variance. Such interventions could benefit healthcare systems nationwide.