Skip to main content

Process evaluation of an implementation trial to improve the triage, treatment and transfer of stroke patients in emergency departments (T3 trial): a qualitative study

Abstract

Background

The implementation of evidence-based protocols for stroke management in the emergency department (ED) for the appropriate triage, administration of tissue plasminogen activator to eligible patients, management of fever, hyperglycaemia and swallowing, and prompt transfer to a stroke unit were evaluated in an Australian cluster-randomised trial (T3 trial) conducted at 26 emergency departments. There was no reduction in 90-day death or dependency nor improved processes of ED care. We conducted an a priori planned process influential factors that impacted upon protocol uptake.

Methods

Qualitative face-to-face interviews were conducted with purposively selected ED and stroke clinicians from two high- and two low-performing intervention sites about their views on factors that influenced protocol uptake. All Trial State Co-ordinators (n = 3) who supported the implementation at the 13 intervention sites were also interviewed. Data were analysed thematically using normalisation process theory as a sensitising framework to understand key findings, and compared and contrasted between interviewee groups.

Results

Twenty-five ED and stroke clinicians, and three Trial State Co-ordinators were interviewed. Three major themes represented key influences on evidence uptake: (i) Readiness to change: reflected strategies to mobilise and engage clinical teams to foster cognitive participation and collective action; (ii) Fidelity to the protocols: reflected that beliefs about the evidence underpinning the protocols impeded the development of a shared understanding about the applicability of the protocols in the ED context (coherence); and (iii) Boundaries of care: reflected that appraisal (reflexive monitoring) by ED and stroke teams about their respective boundaries of clinical practice impeded uptake of the protocols.

Conclusions

Despite initial high ‘buy-in’ from clinicians, a theoretically informed and comprehensive implementation strategy was unable to overcome system and clinician level barriers. Initiatives to drive change and integrate protocols rested largely with senior nurses who had to overcome contextual factors that fell outside their control, including low medical engagement, beliefs about the supporting evidence and perceptions of professional boundaries. To maximise uptake of evidence and adherence to intervention fidelity in complex clinical settings such as ED cost-effective strategies are needed to overcome these barriers.

Trial registration

Australian New Zealand Clinical Trials Registry (ACTRN12614000939695).

Peer Review reports

Background

Worldwide, stroke is a major cause of mortality and disability [1]. In 2016, there were 13.7 million new stroke cases and 5.5 million deaths from stroke and incidence of stroke are increasing in some countries including east Asia and sub-Saharan Africa [1]. Costs associated with stroke care are substantial. A population-based cost analysis of 32 European countries estimated 60 billion euros, with health care accounting for 27 billion euros (45%) [2]. In Australia, expenditure related to stroke is estimated to cost the economy around $5 billion per annum [3].

Evidence-based stroke care can improve survival following stroke [4]. National clinical guidelines include recommendations for rapid clinical assessment, early diagnosis, and evidence-based management of patients with stroke in the emergency department (ED) [5]. Essential protocol elements include appropriate triage (categories 1 or 2 are recommended in Australia for patients with neurological deterioration) [6]; administration of tissue plasminogen activator (thrombolysis) for eligible patients; prompt transfer to an acute stroke unit; and, as shown in our earlier trial, management of fever, hyperglycaemia and swallowing [4, 5]. Given that there are demonstrated health and economic benefits from implementing evidence-based stroke protocols with an estimated cost saving of $281 million a year, delivering evidence-based care in the ED could have the potential to further improve patient outcomes nationally and internationally [7, 8].

Delivering optimal evidence-based care to patients with stroke in EDs, whilst managing other patients with a broad range of illnesses and injuries of varying severity and clinical urgency is challenging [8, 9]. Hence, providing EDs with support to deliver evidence-based triage, treatment and transfer of patients presenting with acute stroke has the potential to improve multidisciplinary care and patient outcomes.

A prospective, multicentre, parallel-group and cluster-randomised controlled trial (C-RCT) was undertaken to evaluate the effect of a multicomponent evidence-based intervention to improve the Triage, Treatment and Transfer (T3) and 90-day health outcomes of acute stroke patients in 26 Australian EDs across three states and one territory [10]. The intervention (Table 1) was based on one that previously had been successfully used in stroke units (QASC trial) [4] and consisted of multidisciplinary supported, nurse-led evidence-based protocols and an implementation strategy informed by behavioural change theory and the Theoretical Domains Framework [11,12,13]. While there were a priori minimum care elements and a fixed evidence-based implementation strategy, flexible local implementation adaptation was permitted, and regular proactive and reactive implementation support was given to the two nursing clinical champions at each intervention site (one a stroke senior nurse and one a senior emergency nurse) by Trial State Co-ordinators. The ED Directors at each intervention site agreed to support implementation. While the intervention was delivered as planned at the cluster level, there was no statistically significant difference between patients in the intervention and control groups for 90-day death or dependency and no improvements in ED clinician behaviour for key ED stroke care practices [10]

Table 1 T3 Trial intervention components

Process evaluations are conducted following a pragmatic implementation trial such as the T3 Trial and seek to understand the factors that influence intervention uptake [14]. Normalisation process theory (NPT) can be used to understand processes associated with the implementation, embedding and integration of new practices in complex and dynamic health settings [15, 16]. The NPT concepts of coherence (does the new practice make sense?), cognitive participation (how engaged are individuals with practice change), collection action (how were the new practices integrated and use facilitated) and reflexive monitoring (value of and impact of the new practices on the team) focus on individual collective and contextual influences and are useful for exploring and understanding protocol uptake in complex settings [15, 16]. As we had planned a priori to undertake a process evaluation at the end of the trial, the NPT was considered to provide a useful sensitising framework for understanding what facilitated or hindered implementation uptake of the T3 protocols. The aim of this qualitative process evaluation was to understand from the viewpoint of those involved in implementation, the factors that influenced the adoption of evidence-based protocols in the management of acute stroke presentation in the ED and to identify barriers and facilitators to protocol implementation and uptake.

Methods

This qualitative process evaluation was designed prior to commencement of the C-RCT and conducted from mid-August to late September 2017 following the conclusion of the T3 Trial which is detailed elsewhere [10] (see Fig. 1). Frameworks for the design and reporting of the process evaluation of complex interventions developed by the United Kingdom (UK) Medical Research Council [14] and C-RCTs [17] informed the design, conduct and reporting of this study. A qualitative descriptive design [18] was used to interview two groups of participants: ED and stroke clinicians involved in the implementation of the T3 clinical protocols and the Trial State Co-ordinators who were responsible for supporting the intervention sites. Qualitative descriptive design was chosen as it enables a rich description of first-hand experiences or an event that is grounded in the participants’ words. Although the Theoretical Domains Framework was useful for characterising potential barriers to the implementation of the T3 protocols and for informing the development of the intervention strategy [11,12,13], the normalisation process theory (NPT) informed the interview guide and analysis and was chosen because it emphasises the role of individual interactions and collective working in implementation processes.

Fig. 1
figure 1

Overview of T3 Trial with process evaluation

Setting

The T3 Trial enrolled 26 hospitals in three Australian states (New South Wales, Victoria, Queensland) and the Australian Capital Territory that had a 24-h emergency department and dedicated stroke units and randomly allocated 13 EDs to the intervention group and 13 to the control group.

Sampling

Hospitals for process evaluation

For each of the 13 intervention sites, processes of care data for the 13 components of the T3 clinical protocols were obtained from the medical record of each patient trial participant [11] and a score out of 13 allocated for the number of processes of care correctly adhered to. An average score for each of the 13 EDs was calculated and then EDs were ranked from highest to lowest performing and divided into quartiles. Two EDs were randomly selected by an independent statistician from the top and bottom quartiles where the T3 clinical protocol uptake was ‘good’ (two high-performing sites) and ‘poor’ (two low-performing sites).

Participants for process evaluation interviews

From the four intervention hospitals classified as high- and low-performing as described above, purposive maximum variation sampling was used to select participants amongst different disciplines for the interview (Table 2). Participants had to have been either engaged in the direct care of patients with stroke during the T3 Trial and/or had local oversight of the implementation of the T3 clinical protocols. The sample included more nurses as the protocols were designed to be nurse-led with two nurse clinical champions at each site, hence the sample included more nurses. ED Medical Directors, as leaders had a key role in endorsing and influencing the use of the protocols. The three Trial State Co-ordinators, who all had a health background (nursing or psychology), were also approached to be interviewed. Participants provided written consent.

Table 2 Description of participants by group

Data collection

The semi-structured interview guide was designed to facilitate exploration of participants’ views of factors that influenced the introduction, implementation and embedding of the T3 clinical protocols into practice. This comprised open-ended questions about the participant’s role in implementation; their views on whether clinical teams understood what was required of them in relation to implementation and protocol use; their views on barriers and facilitators to protocol uptake including organisational, workplace and individual factors and views on sustainability. NPT concepts [15, 16] informed the interview guide as follows: whether the aims and benefits of the T3 protocols were understood (coherence); actions implemented to facilitate staff awareness and engagement in protocol use (cognitive participation); how protocol uptake was facilitated (collective action); and whether and how the protocols impacted upon roles, responsibilities and care delivery (reflexive monitoring) [15, 16]. Interview schedules for the hospital clinicians and Trial State Co-ordinators contained a core set of similar questions plus additional questions that reflected their different roles. The interview schedule was pilot-tested for meaningfulness and clarity in two mock interviews and minor revisions were made.

Interviews were conducted 12 months after cessation of the intervention. Interviewers were independent from the study investigator team and whilst not blinded to site classification (low- or high-performing site) were unaware of each site’s individual performance data and were blinded to the trial results. Emergency nurses participated in group interviews at each site as it was difficult to schedule individual interviews because of the unpredictable nature of bedside clinical work. Participants were assured of the confidentiality of their and their organisations’ identity through the use of pseudonyms. All interviews were audio-recorded, anonymised and professionally transcribed.

Analysis

Demographic and professional data were summarised using means and counts as appropriate. Analysis of narrative interview data involved three stages. The interview framework was transposed into a matrix that was used to manage data from each of the four research sites. Data were imported into the matrices and partitioned by a professional group, then organised and coded. First, deductive content analysis with open coding was performed guided by, but not limited to, NPT concepts which provided a sensitizing framework for key findings. The second stage, code reduction, was achieved by convergence of codes that related to similar data and was conducted both within and across professional groups and then research sites. All data matrices were cross-checked to identify the stability of codes within the interview framework and across professional groups and research sites [19]. A third analytical step identified a set of themes and sub-themes. To minimise researcher bias, the researchers initially coded transcripts independently and then met to discuss and compare codes and themes. Discrepancies were discussed and resolved. When needed, the researchers consulted the transcripts to ensure consistent interpretation and also linked participant quotes to the themes to illustrate clear links between the agreed themes and the data. The data were judged as saturated when no new issues or insights between sites or between participant groups were reported.

Ethics

Approval was obtained from the Human Ethics Research Committees (HRECs) from the Australian Catholic University (2012 16 N), Sydney Local Health District (Royal Prince Alfred Hospital Zone) (HREC/12/RPAH/32) (NSW Lead Ethics Committee) and other multiple sites.

Results

Characteristics of participants

A total of twenty-eight interviews were completed; 25 clinicians and 3 Trial State Co-ordinators (Table 3). Ten emergency nurses were interviewed in groups comprising of 2–3 participants per session. The duration of the interviews was between 30 and 45 min. Overall, the responses were highly consistent across the four sites and participant groups.

Table 3 Demographic and professional characteristics of participants (n = 28)

Three main themes with sub-themes were identified that captured the major factors to the uptake of the T3 clinical protocols and were linked to NPT concepts (Table 4).

Table 4 Themes and sub-themes

Readiness to change

Mobilising teams

In terms of the benefits of participating in the trial, most participants cited that improved patient outcomes were a motivating factor for practice change. Emergency department nurses and clinical champions, in particular, articulated valuing the opportunity to ‘make a difference’ to patient outcomes. The importance of clinical champions in leading implementation and facilitating staff across all shifts and across the ED and Stroke teams, to ‘pull together’ was emphasized.

[The] clinical champion needs time and ability to drive the protocols daily in ED. Needs to be really tenacious and live and breathe the protocols (Stroke Unit Senior Nurse)

It’s been really good building up relationships with the stroke team because often you get caught in your little silos (Senior Emergency Nurse)

The extent of staff movement and the volume of staff in the ED meant that frequently clinical champions had to Educate clinicians at the point of care with a patient in front of them (Senior Emergency Nurse) to ensure that good coverage of education about the protocols across staff groups. However, clinical champions reported challenges in relation to engaging medical staff and some participants expressed the view that a medical clinical champion was needed as well to help drive change across the multidisciplinary team and to support education about the protocols:

Medical head of department needed to stamp authority, otherwise you struggle to have any impact (Stroke Unit Senior Nurse)

(You) need also medical champions – not just nurse champions (Trial State Co-ordinator)

The lack of medical support resulted in clinical champions feeling solely responsible for implementation, staff education and for mobilising collective interdisciplinary momentum for practice change.

Authority to implement

The T3 clinical protocols were multidisciplinary but designed to be nurse-led. The clinical champions at each hospital were required to drive implementation and to ensure that clinical practice aligned with the protocols. Participants (nursing and medical) stated that the importance of the authority of nursing to implement the protocol and drive practice change was central to whether protocol uptake occurred. While some participants reported that the protocols gave authority to clinical champions when negotiating collaboration over aspects of stroke patient care, other participants felt that clinical champions needed to be ‘more authoritative when implementing evidence’ (Stroke Unit Senior Nurse). Trial State Co-ordinators expressed the belief that clinical champions varied in their ability to be influential, to obtain ongoing buy-in from colleagues and to navigate clinical boundaries and build inter-disciplinary relationships:

It all relates back to team dynamics and also the type of leaders and how people are respected and listened to within the team. I felt that sometimes (a clinical champion) didn’t get the respect and support from the medical lead. It was like the clinical champion was a little bit powerless (Trial State co-ordinator)

Promoting the multidisciplinary protocols as nurse-led was also perceived to hamper a shared understanding of the protocols as multidisciplinary:

… if you’re going to refer to nurse-led changes, then doctors would instantly assume, that’s for nurses, I don’t need to be doing that. I feel like that feeds the silo approach and it would be better to say interdisciplinary protocols (Emergency nurse)

In summary, achieving multidisciplinary education and medical engagement was challenging for clinical champions who were perceived to ‘lack authority’ or have insufficient authority to ensure collective investment in the implementation of the protocols.

Fidelity to protocols

Engaging with the evidence

Participant feedback reflected variable engagement with the research evidence underpinning the protocols, despite the supporting evidence and rationale being presented at multidisciplinary workshops and at individual meetings with stroke, ED and endocrinology medical staff. A number of participants reported that they or others in the ED team expressed beliefs at odds with aspects of the evidence on which the protocols (and supporting national and international guidelines) were predicated. This was especially the case for the efficacy of thrombolysis; management of fever at 37.5 °C and the threshold for treating hyperglycaemia (see Table 1).

Resistance to operationalising the T3 clinical protocol element for treating hyperglycaemia at > 10 mmol/L [180 mg/dL] in patients regardless of diabetic state (which was underpinned by the Australian Diabetes Society guideline for the management of all in-patient hyperglycaemia [regardless of diagnosis] to prevent episodes of major hyperglycaemia) [10, 11] was justified by some clinicians on the basis of evidence from the intensive care setting against ‘tight’ glycaemic control (treatment with insulin to keep the glucose between 4.5 and 6.0 mmol/L [81–108 mg/dL]) [20]:

Most senior staff are aware of work done in intensive care units which show that tight glucose control kills people outside of a study environment. So people essentially ignore it (ED Doctor)

The T3 trial was great and showed benefit but didn’t prove to me that insulin infusions were the way to go for stroke patients. I say this is because many trials had shown that intensive lowering of glycaemia was harmful (Stroke Unit Senior Nurse)

Similarly, for the protocol element pertaining to thrombolysis (Table 1), there was resistance to implementation despite this element being supported by high certainty research evidence. In addition, medical staff views about aspects of the evidence were reported to influence ED nurses and also had the knock-on effect of acting as a barrier to the implementation of all aspects of the protocols:

The entrenched resistance towards thrombolysis from some ED medical staff has an influence on ED nursing staff. (Stroke Unit Senior Nurse)

There is a culture of misunderstanding about risk versus benefits of thrombolysis. These views have filtered to nursing as well (Emergency Nurse)

The ED doctors were so resistant to thrombolysis that this was a barrier to implementing all protocol elements at our site (Stroke Unit Senior Nurse)

Despite multidisciplinary educational workshops and follow-up education, there were multiple comments that the implementation strategy relied on the evidence ‘selling itself’. One participant noted that while this approach may be effective in a specialised setting such as a stroke unit, settings such as the ED where there are rapidly changing and competing clinical priorities as well as unfamiliarity with the evidence for stroke management, more intensive strategies might be needed.

Adherence versus adaptation

All elements of the T3 clinical protocols were expected to be implemented with only minor negotiated changes to align with local hospital policies such as administering insulin subcutaneously rather than intravenously. In contrast, examples of an unacceptable adaptation were not treating fever as per protocol or one-off glycaemic measurement rather than 6-hourly monitoring. While some intervention sites reported embedding the protocols into existing ED stroke pathways, others reported that in the dynamic ED environment focused on triage and assessment it was difficult to follow the protocols at all times, hence full adherence to all elements of the protocols was viewed as ‘sustainable in parts’ reflecting that adherence to the frequency required was challenging in the ED. This led to protocol modifications, and partial adoption of the protocols particularly for the fever and hyperglycaemia protocol elements:

It’s not that the protocol was difficult; there’s a difference between the protocol and then people having the time or effort to actually think about and act on it (ED Doctor)

Little bits of the protocol would be dropped off here and there, like part of the temperature and glycaemia monitoring (Senior Emergency Nurse)

Getting temperature within the hour was more challenging than we realised. Particularly if they’re getting thrombolysis within that first period of time

(Stroke Unit Senior Nurse)

Modifications to the protocols also occurred when there were different recommendations in existing hospital documentation that conflicted with the protocols. This ‘resulted in confusion’ (ED doctor) and led to protocol adaptations:

Doctors are unhappy about giving insulin below 11 [mmol/L (198 mg/dL)]. The biggest challenge in protocol adherence was insulin (Emergency nurse)

To address this, required resources and time to develop new documentation requiring that had to be approved by hospital management and this happened rarely with previous practices often continued that conflicted with the protocols.

In summary, beliefs about the supporting evidence and conflict of the protocols with existing hospital policies and pathways led to low uptake of aspects of the protocols and also protocol modifications and adaptations.

Boundaries of care

Defining professional boundaries

There was a strong view expression that ED was not a place for delivery of the ‘specialist’ care needed for patients presenting with stroke. This was seen as being at odds with the role of ED staff as the first-line providers for patients in need of immediate attention. ED clinicians’ views that stroke management is the ‘specialist care’ meant that delivery of the T3 clinical protocols was not seen as within the boundaries of ED practice:

Stroke is a specialist disease, therefore the patient shouldn’t be in ED and we should have the stroke unit ringing us and saying ‘How dare you still have my patient, get them up to me’ (ED Doctor)

Many comments highlighted that ED priorities are triage and assessment, and that once a diagnosis of stroke is made, the stroke team should take over as the ED has other urgent priorities:

In a busy ED what we can do acutely is set them up for thrombolysis. To be brutal and this is a slightly unfair comment, but the ED sees its job as almost done once we’ve booked the CT and let the stroke team know. I don’t make a lot of excuses for that because there are a lot of other jobs that we need to do and other staff who can provide stroke care (ED Doctor)

Hence, boundaries were constructed through clinicians limiting their responsibilities to their specialist field and these boundaries impacted upon optimal protocol uptake.

Care trajectory

The combination of practical considerations of ED work priorities, beliefs that stroke required a suite of specialist care and the expectation that patients with stroke would be rapidly transferred from ED to a dedicated stroke unit (not borne out by the data [10]) also impeded protocol uptake. Related to professional boundaries, the care trajectory of a patient with acute stroke also influenced adherence to the protocols. Feedback from ED clinicians reflected an assumption that as part of the care trajectory for patients with stroke was timely transfer to the stroke unit (Table 1), then it was not seen as central for ED staff to fully adhere to the protocols for the ongoing monitoring and surveillance elements for fever and hyperglycaemia.

Almost all stroke centres have a very actively engaged stroke team who quickly pull patients out of ED. Therein lies the challenge because you get a loss of interest and engagement from ED (Stroke Unit Senior Nurse)

It would be relatively rare that we’re doing protracted blood sugar or temperature measurements because most of these patients are now going to the ward 60-90 minutes after arrival in ED (ED Doctor)

In addition, once a patient with a stroke was assessed as being stable, then other patients were prioritised.

By definition, ED staff triage everything. So our nursing staff see the patient in the bed next door looks sicker and they're just going to ignore it and that's what they should do, because that's how EDs work (ED doctor)

If a stroke unit bed was unavailable, ED staff expressed a view that this ‘tied up’ ED staff beyond what was perceived to be the appropriate duration of care in ED for patients with stroke: Not our business … . In general, once they’re admitted and waiting for a bed, unless they acutely deteriorate, we would take very little notice of them (ED Doctor).

In summary, the participant views were that the monitoring aspects of the T3 protocols to the required degree once the patient was stabilised was seen as disruptive to ED routine practice which has an emphasis on triage and assessment. Professional boundaries and the established care trajectory for patients with stroke were also used as a justification for not fully enacting the protocols until the patient was transferred from ED.

Discussion

This process evaluation provides important information specific to a contextualised understanding of the uptake of the T3 protocols and contributes knowledge to the broader field of implementation science. The findings are explainable through three themes with sub-themes that have identified, from the perspective of those involved in driving practice change that low medical engagement, beliefs about the supporting evidence held by ED clinicians and perceptions of professional boundaries influenced the implementation and uptake of nurse-led stroke protocols in the ED. These findings fill an important gap in the literature by providing insights into the challenges involved with implementing new and complex interventions for time-critical conditions such as stroke in the ED. The ED is a setting for which there is limited evidence for successful stroke care intervention uptake [21].

The application of the normalisation process theory helped to highlight some important findings. The theme Readiness to change showed that achieving medical engagement was difficult. This was despite multidisciplinary education workshops, individual meetings with medical staff to present the evidence and to clarify implementation responsibilities, and ongoing support and education ‘top-ups.’ Lack of sustained medical engagement and a perception that the clinical champions lacked the authority to lead multidisciplinary change were barriers to ‘collective action’ hampered that effective operationalization and integration of the protocols into everyday practice. Other research reports that for multidisciplinary ‘buy-in’ and effective implementation, a critical mass of champions from a range of disciplines is needed to support multidisciplinary practice change and also that compared with other healthcare staff, medical doctors’ involvement in interprofessional educational activities is low [22,23,24]. Recent research has highlighted the importance of several attributes of clinical champions that could be considered when selecting people to fulfil these roles: influence, ownership, physical presence at the point of change, persuasiveness, grit and participative leadership style [25].

The theme Fidelity to the protocols illustrated the impact of entrenched beliefs about the supporting evidence. Staff acceptance and capacity to fully adhere to protocols is central to intervention fidelity [26, 27]. Although minor protocol adaptations (described above) were permissible, full adherence to the ‘minimum intervention elements’ (Table 1) was expected in order to deliver evidence-based care to improve patient outcomes [4, 10]. However, beliefs about aspects of the evidence held by ED medical staff particularly about thrombolysis and hyperglycaemia management appeared to influence other clinical team members and this acted to mitigate against the full implementation of the protocols.

ED staff reported modifying the protocols because they perceived that full implementation of the protocols was time-consuming in relation to other urgent priorities. This was an unforeseen challenge and may indicate a lack of ‘internalisation’ of the protocols [15] by some clinical team members. Using normalisation process theory concepts, this means that while there was an agreement with the aims and expected benefits of protocol implementation, the value and importance of the supporting evidence and its applicability to the ED setting was not fully understood. Furthermore, using a normaliation process theory lens, it may be that a loss of ‘cognitive participation’ may have occurred after the initial multidisciplinary workshops where the supporting evidence and rationale of the protocols were presented [15, 16]. Overcoming clinician ‘mindlines’ and beliefs about the evidence is a vexing challenge in implementation science [28] and in acute stroke management. This is particularly so in relation to the administration of thrombolysis, a key nursing and medical responsibility [29, 30] and a protocol element with clear evidence of effectiveness [4, 5]. As stated by some interviewees, bias against the use of tissue Plasminogen Activator in some sites, also resulted in low acceptance of other elements of the T3 clinical protocols. It is of concern that beliefs about aspects of the evidence (including international guidelines) supporting these evidence-based protocols can derail or ‘de-legitimate’ all elements of the protocols as well as inhibit the individual and collective action needed to embed and sustain the use of these protocols [15, 16]. From the perspective of normalisation process theory, an individual appraisal by clinicians of the applicability of the evidence to the ED setting worked against the collective action needed to facilitate use and to normalise the use of the protocols in the ED. Other recent research has shown that ED clinicians express low agreement with the evidence supporting tissue Plasminogen Activator use in acute stroke [29, 30]. It is therefore possible that active and ongoing championing by key medical staff might have helped to reinforce the education given about the evidence that was part of the implementation strategy.

Boundaries of care revealed a tension between roles and responsibilities as they related to the different clinical contexts of ED and stroke specialist practice. Boundaries were constructed through clinicians limiting their responsibilities to their specialist field; the routine practicalities of patient transfer between departments and specialties, and through the availability of staff to care for a patient with acute stroke both in the ED and in the stroke unit. These findings are consistent with those from studies highlighting professional role and identity as important factors influencing intervention implementation [31, 32]. There was a strong view that ED was not the place for the delivery of ‘specialist’ stroke care as ED staff are the first-line providers for patients in need of immediate attention. Australian data suggest that ED clinicians primarily attend to patients with fractures, burns and toxic effects of medicinal and non-medicinal substances drug overdoses, with a low volume of stroke presentations [33]. Patients with acute stroke may therefore be regarded by ED clinicians as a lower priority, particularly as they do not present with pain and there is no bedside monitor to measure worsening cerebral infarction. In NPT terms, the assessment and understanding of the value of the T3 protocols for the ED setting (reflexive monitoring) appeared to be negatively impacted by perceptions of professional boundaries and expectations relating to clinical responsibilities at different stages of the patient care trajectory.

Our findings reinforce previous research aimed at implementing new and complex interventions for time-critical conditions in the ED. Results from an evaluation of the implementation of a protocol-based sepsis resuscitation in EDs in the USA highlights issues with performing time-sensitive critical care in the ED with the need for more ED time, resources and nursing staff [34]. A Canadian study evaluating the barriers to implementation of therapeutic hypothermia for cardiac arrest patients in the ED also found that ED staff were reluctant to initiate this time-critical treatment which they considered to be an intensive care unit intervention, with participants describing the ED as ‘treat and go’ [35]. Decreased motivation to adopt changes is not uncommon when staff does not consider the intervention as part of their role [36].

There was also a belief that stroke patients would rapidly transfer out of ED to a stroke unit bed (not borne out by the results of our quantitative analysis [11]). The clinical implications of this were that stroke patients did not receive the full complement of protocol care while waiting for a stroke unit bed. In normalisation process theory terms, a low value was attached to fully enacting the protocols, irrespective of the benefits to patients with stroke [15, 16]. Given that the T3 clinical protocols have resulted in significant reductions in death and dependency following a stroke in a previous trial conducted in the acute care setting [4], further exploration is needed about how to ensure patients with stroke receive the full complement of evidence-based management throughout the care trajectory.

The implementation science implications of our study suggest that in the ED setting, if changes are perceived by ED staff to be related to therapeutic interventions beyond those required for a very acute or resuscitation situation then full implementation is likely to be compromised. The findings of a recent study show that the introduction of guidelines into EDs is likely to be unsuccessful if guidelines interfere with ED workflow [9]. ED workflow may therefore compromise the uptake of evidence-based protocol care. Studies have also demonstrated the inability of staff to prioritise new interventions in the ED where these interventions are considered to be outside the role of ED staff which is focused on hyper-acute patients and maintaining patient workflow [31, 36]. Other stroke implementation trials have also experienced similar difficulties in effecting practice change and protocol uptake and cited workforce shortages and the nature of ED workflow as reasons [10, 29, 30]

Despite the potential for the T3 clinical protocols to significantly improve patient outcomes, in common with the findings of other studies, successful implementation in hospital systems is not always achieved [36]. The extent to which evidence-based practice is adopted and sustained in clinical service environments is variable and dependent on a range of organisational and contextual factors that may be unrelated to the strength of the evidence or the best intentions of clinicians [36]. Where considerable preparatory and ongoing work has occurred, as in our trial, systematically identifying barriers and facilitators, matching to behaviour change techniques [11,12,13] and developing site action plans may also influence evidence adoption. Contextual and organisational pressures within which behavioural change occurs may override these implementation efforts and may not be able to be ameliorated within the context of a trial [36]. A challenge of implementation science research is determining effective implementation strategies to address these complex and unforeseen barriers within the constraints of a trial [37].

Strengths and limitations

The main strength of this process evaluation is the representation from different perspectives of the factors influencing protocol uptake in the emergency department. This study met key recommendations for process evaluation [14, 17], namely that the study was planned a priori to evaluate contextual factors impacting upon the implementation of the T3 Trial protocols from the viewpoint of clinical and support staff (Trial State Co-ordinators) involved in the implementation. However, as the process evaluation was conducted at the completion of the T3 trial, it is possible that participants may not have recalled some details of the implementation process. The researchers were independent of the T3 research team, blind to the main study findings and had the required qualifications to undertake the process evaluation. The use of normalisation process theory to inform the interview schedule, coding and interpretation added depth to the analysis and interpretation of results lending an understanding of factors that supported or constrained normalisation of the intervention into routine practice. We have provided a clear documented audit trail of methods and analysis methods to enable assessment of the rigor of the approach and transferability of findings to other settings. Themes are linked to participant quotes to demonstrate the grounding of the findings in the data.

In common with other studies, there are a few limitations. Data were collected until saturation, but it is possible that more experiences may have been found from additional interviews. However, there was good representation of participants from a range of clinical backgrounds ensuring transferability. Given that the trial included only hospitals with a dedicated stroke team and unit means that these findings may not be generalisable to hospitals with fewer specialist resources for stroke.

Conclusions

A systematic adaptation of a previously successful nurse-led protocol intervention in stroke units and a theoretically informed implementation intervention could not overcome complex individual and contextual factors. These findings add to the body of knowledge about the implementation of research evidence into clinical practice in the ED setting and will help guide the implementation of future interventions for acute stroke care in the ED. In particular, future implementation interventions in the ED might want to consider strategies to address barriers such as beliefs about the evidence, whether clinical champions are invested with sufficient authority to facilitate collective change in a busy and dynamic clinical setting and the impact of professional boundaries on perceptions of clinical responsibility for executing protocols. Alternative models of care for patients presenting with stroke to the ED should be considered to ensure evidence-based treatment of patients with stroke outside of stroke units.

Availability of data and materials

The datasets used and analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

ASU:

Acute stroke unit

ED:

Emergency department

NPT:

Normalisation process theory

TDF:

Theoretical Domains Framework

RCT:

Randomised controlled trial

Thrombolysis:

Tissue plasminogen activator

References

  1. GBD 2016 Stroke Collaborators. Global, regional, and national burden of stroke, 1990–2016: a systematic analysis for the global burden of disease. Lancet Neurol. 2019;18:439–58.

    Article  Google Scholar 

  2. Luengo-Fernandez R, Violato NM, Candio P, Leal J. Economic burden of stroke across Europe: a population-based cost analysis. Eur Stroke J. 2019;5(1):17–25.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Deloitte Access Economics. The economic impact of stroke in Australia. Melbourne: Stroke Foundation; 2013.

    Google Scholar 

  4. Middleton S, McElduff P, Ward J, Grimshaw JM, Dale S, D'Este C, et al. Implementation of evidence-based treatment protocols to manage fever, hyperglycaemia, and swallowing dysfunction in acute stroke (QASC): a cluster randomised controlled trial. Lancet. 2011;378(9804):1699–706.

    Article  PubMed  Google Scholar 

  5. Stroke Foundation. Clinical guidelines for stroke management. Melbourne: Stroke Foundation; 2019.

    Google Scholar 

  6. Australasian College for Emergency Medicine. Guidelines on the implementation of the Australasian triage scale in emergency departments. Victoria: Australasian College for Emergency Medicine; 2000.

    Google Scholar 

  7. Australian Clinical Trials Alliance. Economic evaluation of investigator-initiated clinical trials conducted by networks. Sydney: Australian Commission on Safety and Quality in Health Care; 2017.

    Google Scholar 

  8. Stroke Foundation. National stroke audit – acute services report. Melbourne: Stroke Foundation; 2019.

    Google Scholar 

  9. Kirk JW, Sivertsen DM, Petersen J, Nilsen P, Petersen HV. Barriers and facilitators for implementing a new screening tool in an emergency department: a qualitative study applying the theoretical domains framework. J Clin Nurs. 2016;25(19-20):2786–97.

    Article  PubMed  Google Scholar 

  10. Middleton S, Dale S, Cheung NW, Cadilhac DA, Grimshaw JM, Levi C, et al. Nurse-initiated acute stroke care in emergency departments: the triage, treatment, and transfer implementation cluster randomized controlled trial. Stroke. 2019;50(6):1346–55.

    Article  Google Scholar 

  11. Craig LE, Churilov L, Olenko L, Cadilhac DA, Grimley R, Dale S, et al. Testing a systematic approach to identify and prioritise barriers to successful implementation of a complex healthcare intervention. BMC Med Res Methodol. 2017;17(1):24.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Craig LE, McInnes E, Taylor N, Grimley R, Cadilhac DA, Considine J, et al. Identifying the barriers and enablers for a triage, treatment, and transfer clinical intervention to manage acute stroke patients in the emergency department: a systematic review using the theoretical domains framework (TDF). Implement Sci. 2016;11(1):157.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Craig LE, Taylor N, Grimley R, Cadilhac DA, McInnes E, Phillips R, et al. Development of a theory-informed implementation intervention to improve the triage, treatment and transfer of stroke patients in emergency departments using the theoretical domains framework (TDF): the T 3 trial. Implement Sci. 2017;12(1):88.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258.

    Article  PubMed  PubMed Central  Google Scholar 

  15. 16 CR, Johnson M, Finch T. Implementation, context and complexity. Implement Sci. 2016;11(1):141.

    Article  Google Scholar 

  16. May CR, Cummings A, Girling M, et al. Using normalization process theory in feasibility studies and process evaluations of complex healthcare interventions: a systematic review. Implement Sci. 2018;13(1):80. https://doi.org/10.1186/s13012-018-0758-1.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Grant A, Treweek S, Dreischulte T, Foy R, Guthrie B. Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials. 2013;14(1):15.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Sandelowski M. Whatever happened to qualitative description? Res Nurs Health. 2000;23(4):334–40.

    Article  CAS  PubMed  Google Scholar 

  19. Kietzmann D, Hannig C, Schmidt S. Migrants’ and professionals’ views on culturally sensitive pre-hospital emergency care. Soc Sci Med. 2015;138:234–40.

    Article  PubMed  Google Scholar 

  20. Finfer S, Chittock D, Su S, Blair D, Foster D, Dhingra V, et al. Intensive versus conventional glucose control in critically ill patients. N Engl J Med. 2009;360(13):1283–97.

    Article  PubMed  Google Scholar 

  21. Levi CR, Attia JA, D'Este C, Ryan AE, Henskens F, Kerr E, et al. Cluster randomized trial of thrombolysis implementation support in metropolitan and regional Australian stroke centers: lessons for individual and systems behavior change. J Am Heart Assoc. 2020;9(3):e012732.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Clarke DJ, Forster A. Improving post-stroke recovery: the role of the multidisciplinary health care team. J Multidiscip Healthc. 2015;8:433.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Jones SP, Miller C, Gibson JME, Cook J, Price C, Watkins CL. The impact of education and training interventions for nurses and other health care staff involved in the delivery of stroke care: an integrative review. Nurse Educ Today. 2018;61:249–57.

    Article  PubMed  Google Scholar 

  24. Luker JA, Craig LE, Bennett L, Ellery F, Langhorne P, Wu O, et al. Implementing a complex rehabilitation intervention in a stroke trial: a qualitative process evaluation of AVERT. BMC Med Res Methodol. 2016;16(1):52.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Bonawitz K, Wetmore M, Heisler M, et al. Champions in context: which attributes matter for change efforts in healthcare? Implement Sci. 2020;15:62. https://doi.org/10.1186/s13012-020-01024-9.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Hasson H. Systematic evaluation of implementation fidelity of complex interventions in health and social care. Implement Sci. 2010;5(1):67.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Ibrahim S, Sidani S. Intervention fidelity in interventions: an integrative literature review. Res Theory Nurs Pract. 2016;30(3):258–71.

    Article  PubMed  Google Scholar 

  28. Wieringa S, Greenhalgh T. 10 years of mindlines: a systematic review and commentary. Implement Sci. 2015;10(1):45.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Grady AM, Bryant J, Carey ML, Paul CL, Sanson-Fisher RW, Levi CR. Agreement with evidence for tissue plasminogen activator use among emergency physicians: a cross-sectional survey. BMC Res Notes. 2015;8(1):267.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Hasnain MG, Levi CR, Ryan A, Hubbard IJ, Hall A, Oldmeadow C, et al. Can a multicomponent multidisciplinary implementation package change physicians’ and nurses’ perceptions and practices regarding thrombolysis for acute ischemic stroke? An exploratory analysis of a cluster-randomized trial. Implement Sci. 2019;14(1):98.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Nugus P, Forero R, McCarthy S, Mcdonnell G, Travaglia J, Hilman K, et al. The emergency department “carousel”: an ethnographically-derived model of the dynamics of patient flow. Int Emerg Nurs. 2014;22(1):3–9.

    Article  PubMed  Google Scholar 

  32. Tavender EJ, Bosch M, Gruen RL, Green SE, Knott J, Francis JJ, et al. Understanding practice: the factors that influence management of mild traumatic brain injury in the emergency department-a qualitative study using the theoretical domains framework. Implement Sci. 2014;9(1):8.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Australian Institute of Health and Welfare. Emergency department care 2018-19. Canberra: Australian Institute of Health and Welfare; 2019.

    Google Scholar 

  34. Carlbom DJ, Rubenfeld GD. Barriers to implementing protocol-based sepsis resuscitation in the emergency department—results of a national survey. Crit Care Med. 2007;35(11):2525–32.

    Article  PubMed  Google Scholar 

  35. Toma A, Bensimon CM, Dainty KN, Rubenfeld GD, Morrison LJ, Brooks SC. Perceived barriers to therapeutic hypothermia for patients resuscitated from cardiac arrest: a qualitative study of emergency department and critical care workers. Crit Care Med. 2010;38(2):504–9.

    Article  PubMed  Google Scholar 

  36. Geerligs L, Rankin NM, Shepherd HL, Butow P. Hospital-based interventions: a systematic review of staff-reported barriers and facilitators to implementation processes. Implement Sci. 2018;13(1):36.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Waltz TJ, Powell BJ, Fernández ME, Abadie B, Damschroder LJ. Choosing implementation strategies to address contextual barriers: diversity in recommendations and future directions. Implement Sci. 2019;14(1):42.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors would like to thank all study participants for their time and involvement in the research. Our thanks to Dr. Anne Gardner and Emeritus Professor Glenn Gardner for conducting the evaluation.

Funding

The T3 Trial was funded by a NHMRC Project Grant 1024812 (2012–2017). The following authors received research fellowship funding from the NHMRC: Dominique Cadilhac (co-funded with Heart Foundation: 1063761 and 1154273) and Chris Levi (Practitioner: 1043913). Jeremy Grimshaw holds a Canada Research Chair in Health Knowledge Transfer and Uptake. Non-material support provided by the trial sponsor, Australian Catholic University, to house members of the trial team including authors SM, SD, EM, RP, OF, VS and LC.

Author information

Authors and Affiliations

Authors

Contributions

EM, RP, SM, SD, DC, AG, GG and JG contributed to the design of the study. AG, GG, EM and RP analysed the data. EM drafted the manuscript. All authors critically revised the intellectual content of the manuscript. EM, RP, SM, SD and OF made substantial contributions to the manuscript. The authors read and approved the final manuscript.

Corresponding author

Correspondence to Elizabeth McInnes.

Ethics declarations

Ethics approval and consent to participate

Ethics approval was obtained from the following Human Research Ethics Committees: Australian Catholic University (2012 16 N), Sydney Local Health (Royal Prince Alfred Hospital Zone) (HREC/12/RPAH/32) (New South Wales Lead Ethics Committee); Australian Capital Territory Government Health Directorate (ETH.1.13.009); Uniting Care Health (1231); The Alfred, Melbourne (503/12); Northeast Health, Wangaratta; and South West Healthcare, Warrnambool (5/2012).

Participants provided written informed consent for involvement in the semi-structured interviews.

Consent for publication

Not applicable.

Competing interests

During the duration of the trial and following receipt of funding, SM was appointed to the Research Committee of the National Health and Medical Research Council (NHMRC). Other authors declare that they have no competing interests. The sponsors of the study had no role in the study design, data collection, data analysis, data interpretation or preparation of the manuscript.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

McInnes, E., Dale, S., Craig, L. et al. Process evaluation of an implementation trial to improve the triage, treatment and transfer of stroke patients in emergency departments (T3 trial): a qualitative study. Implementation Sci 15, 99 (2020). https://doi.org/10.1186/s13012-020-01057-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13012-020-01057-0

Keywords