https://orcid.org/0000-0002-4309-0478
Figures
Abstract
Background
Nosocomial infections place a heavy burden on patients and healthcare providers and impact health care institutions financially. Reducing nosocomial infections requires an integrated program of prevention and control using key clinical best care practices. No instrument currently exists that measures these practices in terms of personnel time and material costs.
Objective
To develop and validate an instrument that would measure nosocomial infection control and prevention best care practice costs, including estimates of human and material resources.
Methods
An evaluation of the literature identified four practices essential for the control of pathogens: hand hygiene, hygiene and sanitation, screening and additional precaution. To reflect time, materials and products used in these practices, our team developed a time and motion guide. Iterations of the guide were assessed in a Delphi technique; content validity was established using the content validity index and reliability was assessed using Kruskall Wallis one-way ANOVA of rank test.
Results
Two rounds of Delphi review were required; 88% of invited experts completed the assessment. The final version of the guide contains eight dimensions: Identification [83 items]; Personnel [5 items]; Additional Precautions [1 item]; Hand Hygiene [2 items]; Personal Protective Equipment [14 items]; Screening [4 items]; Cleaning and Disinfection of Patient Care Equipment [33 items]; and Hygiene and Sanitation [24 items]. The content validity index obtained for all dimensions was acceptable (> 80%). Experts statistically agreed on six of the eight dimensions.
Discussion/Conclusion
This study developed and validated a new instrument based on expert opinion, the time and motion guide, for the systematic assessment of costs relating to the human and material resources used in nosocomial infection prevention and control. This guide will prove useful to measure the intensity of the application of prevention and control measures taken before, during and after outbreak periods or during pandemics such as COVID-19.
Citation: Tchouaket Nguemeleu E, Boivin S, Robins S, Sia D, Kilpatrick K, Brousseau S, et al. (2020) Development and validation of a time and motion guide to assess the costs of prevention and control interventions for nosocomial infections: A Delphi method among experts. PLoS ONE 15(11): e0242212. https://doi.org/10.1371/journal.pone.0242212
Editor: Wen-Jun Tu, Chinese Academy of Medical Sciences and Peking Union Medical College, CHINA
Received: May 28, 2020; Accepted: October 28, 2020; Published: November 12, 2020
Copyright: © 2020 Tchouaket Nguemeleu et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files. Additional data cannot be shared publicly because qualitative responses contain answers that may identify participants and institutions. All data can be made available from the Ethics Committee (Comité éthique de recherche du CISSS des Laurentides Saint-Jérôme, Québec, Tél.: 450-431-1020) for researchers who meet the criteria for access to confidential data.
Funding: This research project has been funded by the Fonds de Recherche du Québec-Santé (FRQS, http://www.frqs.gouv.qc.ca/en/) from April 1, 2017, to March 31, 2021, grant number #35124 (ET). Eric Tchouaket received a Junior 1 researcher award from the FRQS. This project was also funded by the Canadian Institutes of Health Research (CIHR, https://cihr-irsc.gc.ca/e/193.html) grant #409071 (ET) and Réseau de Recherche en Interventions en Sciences Infirmières du Québec (RRISIQ, https://rrisiq.com/) grant #26674 (ET). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Abbreviations: BD, Bruno Dubreuil; CBP, Clinical best practice; CDAD, Clostridium difficile associated diarrhea; CINQ, Comité des infections nosocomiales du Québec; CISSS, Integrated Health and Social Services Centre for the Laurentian region, Quebec; CIUSSS, Integrated University Centre for Health and Social Services; CL, Catherine Larouche; CPGNB, Carbapenemase-producing Enterobacteriaceae; CVI, Content validity index; DS, Drissa Sia; ET, Eric Tchouaket; KK, Kelley Kilpatrick; MRSA, Methicillin-resistant Staphylococcus aureus; MSSS, Quebec’s Ministry of Health and Social Services; NI, Nosocomial infection; NIPC, NI Program for Prevention and Control; NP, Natasha Parisien; SB, Sandra Boivin; SBR, Sylvain Brousseau; SR, Stephanie Robins; SPIN, Quebec’s Provincial NI Surveillance Group; TDABC, Time driven activity based costing; VRE, Vancomycin-resistant enterococcus
Introduction
Nosocomial infections (NIs), also known as healthcare associated infections (HCAIs), are infections acquired during an episode of care in a healthcare facility [1]. These infections are directly related to care and are considered among the most common and preventable adverse events [2]. NIs place a significant burden on patients and hospital staff as they result in medical complications, prolonged hospital stays, high rates of morbidity and mortality, and reduced quality of life; they also place a significant burden on healthcare systems as they result in extra costs related to the extended hospitalization or readmission of patients, patient’s care-related expenses and costs involved in limiting further contagion [3–7]. Similar to other nations, Canada has not solved the problem of NIs. In 2013, the Public Health Agency of Canada reported that more than 200,000 patients contract a NI each year, resulting in more than 8,000 deaths [8]. In 2016, the Canadian Patient Safety Institute estimated that one NI occurs in every 41 hospitalizations, at an estimated cost of 281 million dollars, or 41% of the total cost of adverse events [9]. In the province of Québec, the Comité des infections nosocomiales du Québec (CINQ) estimated, based on American studies, that the annual number of NIs reached somewhere between 80,000 and 90,000 patients in 2005, with an estimated mortality rate of between 1 and 10% [10].
To address the issue of NIs, in 2004 the Québec Ministry of Health and Social Services (MSSS) implemented a mandatory program for monitoring NIs in all hospitals [11]. Since then, this program has become a mainstay for healthcare institutions in managing risks, quality, and patient safety. The program currently tracks different pathogens including: methicillin-resistant Staphylococcus aureus (MRSA), Clostridium difficile-associated diarrhea (CDAD), colonization and infection caused by multidrug-resistant bacteria such as vancomycin-resistant enterococci (VRE) and carbapenemase-producing enterobacteria (CPGNB). Along with NI tracking, target 22 of the MSSS 2015–2020 action plan, On the prevention and control of NIs for safe delivery of healthcare in the province of Quebec, is to “evaluate NIPC [Nosocomial Infection Prevention and Control] measures, taking into account the organizational model, the burden of disease, as well as their clinical and epidemiological impacts” [12].
In order to evaluate costs of core NIPC measures, it is helpful to refer to the framework set out by Resar and colleagues of the United States Institute for Healthcare Improvement (IHI) that describes an adverse-events intervention framework of clinical best practices (CBPs) [13]. The authors propose the implementation of “bundles” of evidence-based CBPs in order to ensure the best possible outcomes for: infection prevention, a reduction of complications and the highest level of quality and safety with regards to health care. This framework previously informed Canadian and Quebec healthcare infection prevention and control strategies as well as their respective safety care campaigns [14–16]. Importantly, bundles are defined by the authors as: “A small set of evidence-based interventions for a defined patient segment/population and care setting that, when implemented together, will result in significantly better outcomes than when implemented individually.” Bundles were conceived from a collaborative effort between the IHI and the Voluntary Hospital Association who established a working group to obtain feedback from clinical experts working in 13 hospital intensive care units. The team also undertook an analysis of the medical evidence surrounding clinical processes that were considered to be both the most costly and potentially harmful. From this, design guidelines included 6 key concepts: a) Bundles must have 3–5 interventions and be clinician endorsed; b) Elements in bundles are relatively independent; c) Bundles are specific to a defined patient population; d) Multidisciplinary care teams are involved in the development; and e) Bundle elements should be locally customizable, and descriptive in nature and f) Compliance is measured as an all or none measurement.
According to the IHI, the application of bundled care produces better outcomes and has been shown to bring about monetary savings in the millions. This occurs when bundle elements are measured and accurate is obtained data, either confirming or challenging the assumption that care is being reliably delivered.
Bundles of care, however, do not usually evaluate CBPs that are routinely performed to control NIs. Despite this, four essential and overlapping CBPs are commonly associated with all levels of patient care: a) hand hygiene; b) hygiene and sanitation including the cleaning and disinfecting of surfaces and equipment; c) screening on admission and during hospitalization of carriers and patients who are at risk; and d) application of basic and additional precautions. As has been seen with the COVID-19 pandemic, these four CBPs are very important prevention and control practices to reduce NIs in hospitals [17, 18]. Any bundle designed for improved patient outcomes would be expected to include no less than one of these CBPs and potentially all four. To better understand how these transverse CBPs might impact the costs of care and associated patient outcomes, our team searched for a standardized or systematic way to assess costs.
Studies have reviewed costs and cost-effectiveness related to NIPC management in hospitals. Some studies have assessed the costs and cost benefits of a hand hygiene campaign on the reduction of the incidence of NIs, specifically as they related to MRSA [19], while others focused on the direct costs and cost savings of contact isolation precautions [20–23]. The costs of active surveillance of VRE or MRSA have been estimated in regards to: systematic screening, hygiene procedures, wearing gowns, gloves, single use materials, hydroalcoholic solutions, disinfection procedures, and sanitation [24, 25].
To the best of our knowledge, and after a review of the literature, we found no formal systematic instrument or guide exists that captures the costs required to prevent and control NIs using the four CBPs presented above. Specifically, such a guide should capture, in real time, (i) the time healthcare workers spend on any sequence of actions related to the CBPs of NI prevention and control (hand hygiene, wearing protective barrier apparel, cleaning and disinfecting healthcare equipment and the environment, screening (with or without cultures), additional precautions (by contact, droplet or airborne), isolation, education, training and awareness campaigns); (ii) the materials used for these CBPs; (iii) the products required for these CBPs.
Objective
To fill this gap, this study aimed to develop and validate a time and motion guide that would be used to measure the costs of the four CBPs of NI prevention and control.
Methods
The research team formed a working group to establish the steps required to build the time and motion guide. It was composed of: nurses (SB, NP, CL) in the field of infection prevention and control; a professional expert in hygiene and sanitation (BD); and researchers with expertise in time and motion studies (KK); nursing administration (SB); public health (DS, ET); and, health economics (ET).
Literature review and meetings
Between March 2018 and July 2018, a review of the literature was undertaken in order to appraise and synthesize the data on the costs of the four CBPs, as well as the human resources, material and product costs related to NI prevention and control. We searched the scientific and grey literature using Medline, Cinhal, Pubmed, and Cochrane databases as well as Canadian and Québec government and World Health Organisation (WHO) websites, limiting our search from 2010 to 2018 inclusive. We used the following research strategy and key words: (tool OR instrument OR scale OR measure) AND (“nosocomial infections” OR “hospital acquired infections” OR “health care associated infections”) AND (prevention OR intervention OR treatment OR program OR screening OR assessment OR test OR diagnosis OR “screening tool” OR strategy OR management) AND (cost*). After screening the title and abstract of the articles, we selected and reviewed 22 articles [19–41] and more than ten international and national reports [42–56]. Five meetings of the working group occurred between March and July of 2018. As noted above, the literature was examined to better understand how to measure the costs of CBPs, however a comprehensive cost-accounting tool did not emerge. In order to accurately measure costs, we considered incorporating the methodology from time-driven activity-based costing (TDABC) [57]. TDABC functions by dynamically allocating costs related to the consumption of resources across human-driven processes, with the purpose of adding up costs throughout a supply chain to determine overall expenses. Adapted from a business model, TDABC has been used in healthcare to measure human resource costs related to specific medical conditions such as paediatric surgery, ear nose and throat medicine, dermatology and psychiatry [for a review see 58]. For our purposes, instead of measuring a medical condition we focused on our four CBPs and began breaking down each one in granular fashion, considering the start and end of each process. The activity and location of each CBP was determined. Process maps and all resources used for each step of any CBP. This process revealed that each CBP actually had layers of elements that resulted in the emergence of sub dimensions (later defined as items) that were not previously considered.
Further discussions helped to establish models of workflow of different healthcare staff, and brought to light which CBPs might be used to accomplish individual tasks. In order to capture the time required for CBPs, it was proposed that a chronometer be integrated that could measure at least one, but preferably more than one simultaneously occurring action. This proposal was accepted, and it was decided that external observers would use the guide and chronometer to capture data continuously. These two decisions fall in line with a recent schema of time and motion studies [see 59 for a review]. Members of the team proposed that this detailed and direct continuous observation approach to data collection may result in the tendency of staff to improve their performance [for an example in healthcare see 60]. This point was discussed, and accepted as a limitation of the instrument. Sessions were iterative, with issues being revisited as the guide was developed. By July of 2018, these sessions had proven to be key in helping to establish a consensus between all members as to the necessary components of an algorithm that would serve as a framework for the time and motion guide.
Creation of an algorithm and time and motion guide
To inform our process, the working group first drafted an initial version of the time and motion algorithm (see Fig 1). The algorithm outlines a series of sequential steps observers would follow when recording the time necessary for CBPs and for the materials and products used in daily prevention and control of NIs. The algorithm first included observer and location-specific measures to be captured including: identification of the observer; the date and site where measurements would be taken; the hospital unit (medicine or surgery), and the title of the professional being observed (patient-care attendant, nurse or housekeeping staff). The algorithm was further divided into seven main dimensions: 1) area being evaluated, 2) hand hygiene, 3) personal protective equipment, 4) screening, 5) disinfection of patient care equipment, 6) overall disinfection, hygiene and cleanliness, and 7) additional precautions. Each dimension contained sub-categories of items that described, in a more granular fashion, the time and/or material to capture. This information would provide a more precise measurement of the amount of personnel time and material costs used, and where these resources were spent. For example, the first sub-category of ‘area’ would clarify where (i.e., a private, semi-private or multi-bed room) staff spent time in NIPC actions. In the category of hand-washing, the sub-categories to identify included: when personnel washed their hands (before, during, or after patient intervention) along with the product they used: soap or hydroalcoholic solution. The final and most essential level of the algorithm notes that for each infection and control opportunity, time taken (in seconds) and products used will be systematically recorded.
The algorithm then helped guide the research team to develop the second phase: the creation of the time and motion guide, effectively operationalizing the constructs set out in the research question. The time and motion guide contains items in each of the seven dimensions. The guide was created as a paper questionnaire to be used for the standard measurement of NIPC costs. In order to optimally integrate the chronometer, the questionnaire was incorporated into an online application using the software platform Dataformz (https://www.dataformz.com/en/). This digital version of the questionnaire allows for access from a variety of devices (tablet, phone or computer) in order to capture all actions in real time. It also provides a tool that facilitates nurse observer training and pilot testing province-wide. Users access the Dataformz platform using a secure URL and identify themselves by choosing their name from a list of user names. The dashboard first displays information that is deemed essential, such as the health care facility location, unit, personnel job title, and area being assessed; here users are presented with mandatory fields. At the completion of these fields, users are prompted to ‘submit’ their answers, effectively registering the collected data for that individual in the online database. The nurse observer following this professional then opens pages of the app where appropriate categories of the CBP are measured. For time measurements, users make use of the online chronometer to quantify movements in real time, and this, even if two or more actions are underway, as overlapping actions can be recorded. Recordings of time can be saved or, if there was an error, discarded. Similarly, all materials used in NIPC are listed on the platform as radio buttons or check-all boxes to facilitate data collection. Open text boxes (‘other’) are also provided in sections of the app where the description of a product is not predetermined. When an observation is complete, the user clicks the ‘end questionnaire’ button to submit their record.
Finally, a user manual of the guide was developed to facilitate training of nurse observers and ensure all observations are entered in a standardized fashion. The manual provides a description of the research team and a brief summary of the study’s purpose; the time and motion algorithm is annexed for reference purposes. The user manual describes in detail how to navigate each page of the online platform and how to correctly input observations. For each category of the guide, written explanations are accompanied by screenshots of the interface that users should expect to encounter on the app.
Construction of the time and motion guide questionnaire
The time and motion guide questionnaire to be sent to Delphi panelists included an initial version with 9 dimensions: (1) Identification [11 items]; (2) Personnel [4 items]; (3) Zone [2 items]; (4) Additional Precautions [1 item]; (5) Hand Hygiene [2 items]; (6) Personal Protective Equipment [2 items]; (7) Screening [4 items]; (8) Cleaning and Disinfection of Patient Care Equipment [5 items]; (9) Hygiene and sanitation [5 items]. The questionnaire was designed to elicit both qualitative and quantitative responses for the guide items and dimensions, and to obtain a qualitative evaluation of the web app functioning. Panelists were asked to rate all items in all dimensions of the guide, using a 4-point Likert scale from 1 = “not relevant” to 4 = “very relevant without corrections” as shorter Likert scales have produced stable findings in Delphi studies previously [61, 62]. Panelists were also provided an open-ended question about each item’s relevance, should they have a suggestion for improvement or other opinion.
Content validity of time motion guide
The content validity was assessed using the Delphi Technique among experts. The Delphi Technique is a well described group survey technique widely used in health research that results in the narrowing of a consensus on a topic of public importance [63]. The Delphi process allows for a variety of methodological approaches, including qualitative and quantitative data gathering and analysis that can be revised. It also allows for the investigation of a solution that may not yet exist. Due to its anonymity and iterative process, the Delphi method is considered to be less prone to bias as expert participants have the opportunity to provide their honest opinion or creative solution without feeling the need to agree with dominant personalities [64].
Our team used this systematic and interactive forecasting method as has been done by others in the field of health care [65–67], to obtain consensus from a panel of experts, consulted over two or more rounds, for feedback on the time and motion guide. Experts were invited to two successive rounds of review. After each round, quantitative and qualitative data collected were analysed in order to assess the agreement and consensus. In each step, the panelists received the time and motion guide questionnaire, access to the Dataformz website and the user manual for revision and resubmission. Prior to the second round, panelists received their own feedback along with anonymized qualitative feedback from others (see S1 File for the final questionnaire provided to experts). Once two rounds were complete, the working group addressed any remaining discrepancies and produced a final version of all tools for pilot testing. Fig 2 presents the Delphi process for this study.
Delphi technique: Selection of expert panel and procedure
In order to consider a variety of opinions from participants having lived and/or professional experience in NI prevention and control, 18 experts [as recommended by 68] from two distinct viewpoints were selected: those who incorporated NIPC measures directly in their workday and those who could be considered content experts as a result of their professional training. The first group (12 experts) was composed of professionals that worked directly in infection prevention and control and were considered to have experiential expertise that complemented their professional training. A group of six was chosen from two healthcare facilities that differed in their structure, one being associated with a university and thus serving as a teaching facility, the other not. The time motion guide was analyzed by: a manager in charge of NIPC, two nurses, a microbiologist/infectious disease expert, and two health and sanitation staff. The second group of experts (n = 6) were chosen for their knowledge and expertise in the field, their active involvement in infection prevention and control via their role in government and public policy implementation, or because they were key opinion leaders. This group was made up of two microbiologist/infectious disease experts, two clinical nurse specialists in NIPC and two hygiene and sanitation advisors. We purposely sourced male and female panelists from different geographic regions of Quebec, Canada, to ensure the feedback would be valid for, and representative of, practices across the province. All experts who expressed interest in the study (n = 18) were sent a consent form to be returned by e-mail and all then provided informed consent.
The two rounds of the Delphi survey were conducted between July 2018 and January 2019. All experts were sent the time and motion guide questionnaire and asked to rate each item and dimension between 1 “not relevant”, 2 “relevant with major corrections”, 3 “relevant with few corrections”, and 4 “very relevant without corrections”, based on its relevance for the assessment of costs related to the four CBPs. The content validity was assessed using the content validity index (CVI) [69], which was set as a percentage, counting each item and dimension rated 4 “very relevant without corrections” over the total number of replies by experts for the item or dimension.
Three levels of review were considered to assess the content validity or agreement between experts. First, when the CVI for a dimension (CVI_D) was less than 0.80 (80%), the dimension was considered not acceptable. The dimension was then completely reviewed and revised according to the proposed corrections. The resultant items and dimension were then resubmitted to the same experts in the next Delphi round. When the CVI_D was equal to or greater than 0.80, the dimension was considered acceptable. Second, the CVI for individual items (CVI_I) of all dimensions was assessed. As above, when CVI_I was less than 0.80, the item was considered not acceptable and revised according to the proposed corrections. The item was then resubmitted to the same experts in the second round. When the CVI_I was equal to or greater than 0.80, the item was kept. In a third and final level of review, items that were revised but did not improve to the level of 0.80 following two rounds of Delphi review were revisited by the working group in order to finalize the items and address concerns expressed by the experts in round two.
Reliability of time motion guide
Finally, in order to ascertain reliability, (i.e., how much of the variance in observed ratings was due to a true difference in score, and not measurement error between groups of expert panelists), we performed non-parametric Kruskall-Wallis ANOVA of rank tests for each dimension, with 5% threshold, to demonstrate any effect of group. When the Kruskall-Wallis tests of a dimension was not statistically significant, it meant that all the responses were acceptably constant between the panel experts, or reliable. Otherwise, the dimension and its items were revisited in its entirety and revised according to the proposed corrections by experts. The dimension would then be resubmitted to the same experts in another round. All the revisions were discussed and validated by the research team during working group meetings. Data were analysed using SPSS version 26 (Armonk, NY, USA: IBM Corp). Ethical approval was obtained from the Centre intégré de santé et services sociaux des Laurentides (reference #: 2017MP-364-É and MP-28-2018-002) and the Université du Québec en Outaouais. All participants in the Delphi panel provided written informed consent.
Results
Delphi: Rounds 1 & 2
In an initial round, all 18 experts were invited to complete the time and motion survey using the first version of the algorithm, user manual and web platform; tools were provided in French versions as all panelists were Francophones. Of these, 16 experts (88.9%) completed the questionnaire: 6 from the non-university affiliated hospital (group 1), 5 from the university-affiliated hospital (group 2), and 5 content experts (group 3). For all response rates, see Table 1. For quantitative responses, only one dimension of the guide, (cleaning and disinfection of patient care equipment) met our CVI_D acceptable limit of greater than 80%, at 83.4%. However, only two items within this dimension met a CVI_I of 80%. Reliability for all dimensions was acceptable (i.e., the groups did not significantly differ) with the exception of the dimension hygiene and sanitation, where the effect of expert group was significant, X2(2) = 9.02 p = 0.01. Thus, all the dimensions were reviewed and analyzed based on the comments and suggestions of experts. The guide was reworked and the questionnaire adapted for round 2, with all anonymized responses from the first round provided to experts.
In round 2, all 16 panelists who had completed round 1 were invited to respond to the second version of the questionnaire in order to evaluate the guide using the user manual and web platform. In this round, 14 experts (87.5%) completed the survey with 5 responses each from the university and non-university site, and 4 from content experts. Content validity improved markedly on round 2, with all dimensions surpassing the threshold of 80% (see Table 2). However, one item each from the dimension Identification and Zone as well as three within the dimension of Hygiene and sanitation, had CVIs of below 80%, and were flagged for further review. In this second round, reliability was acceptable for all dimensions except two. In Zone, the Kruskall Wallis test revealed a significant difference between groups of experts: X2(2) = 10.20 p = 0.006, as it did in the dimension Hygiene and Sanitation, X2(2) = 9.95 p = 0.007. Comments were reviewed in order to determine the source of disagreement.
Description of time and motion guide (final version)
The final version of the guide contains eight of the nine original dimensions, with the dimension Zone merged with Hygiene and Sanitation. These eight dimensions include: Identification [83 items]; Personnel [5 items]; Additional Precautions [1 item]; Hand Hygiene [2 items]; Personal Protective Equipment [14 items]; Screening [4 items]; Cleaning and Disinfection of Patient Care Equipment [33 items]; and Hygiene and Sanitation [24 items].
The guide and online platform have been substantially adapted. For example, the first section, Identification, was expanded from 12 items in round 2, to 83 items in the final version. This increased level of detail provides the user with the ability to measure items such as the brand and format of hydroalcoholic solution being used (gel or foam) and the count of disposable items. It also includes newly considered items that have a profound impact on NICP costs, such as staff attendance at NIPC training and information sessions as well as awareness campaigns. Other sections that were vastly revised included both sections that enumerated equipment (either personal protective equipment or patient monitoring equipment), to include detailed lists of all equipment used. The guide includes chronometers at the level of all items that require personnel time; the digital stopwatch can be set to measure time, either sequentially or simultaneously with CBPs, as needed. The complete paper version of the guide can be seen in S2 File. All changes to the time and motion guide were implemented in the final version of the web app and user manual in preparation for pilot testing.
Discussion
The goal of this study was to develop and validate a time and motion guide to facilitate the objective assessment of the costs of four CBPs of NI prevention and control, using a systematic approach. Our process first defined an original set of items chosen from an extensive review of the literature. These were later expanded upon to create a final version of the time and motion guide in three successive waves of review, two of these undertaken by a panel of experts. By using the Delphi technique, preliminary evidence of validity and reliability of the guide were gathered.
Content validity is an important step in the development of questionnaires and quantifies the level of agreement between experts; we decided a priori to evaluate the assessments of our panelists using the CVI, both at the item and dimension level. According to Polit [69] the index is useful in questionnaire development as it provides “a focus on consensus rather than consistency estimates, ease of computation, understandability and ease of communication, provision of both item diagnostic information and scale validity information”. By the end of two rounds of Delphi review, the CVI was deemed acceptable in all dimensions, and only failed to meet the 80% cut-off for six individual items. The Kruskall Wallis tests of reliability revealed disagreement between the groups in two dimensions: Zone, and Hygiene and Sanitation.
However, divergent opinions are an expected outcome of the Delphi process, and have been considered as representing smaller groups who hold a “clustered consensus” within the panel [70]. Therefore, along with the commentary provided by the panelists these items and dimensions were conceptually reviewed and extensively modified. The dimension of Zone was merged within Hygiene and Sanitation. This change reflected not only the poor statistical reliability between expert groups but also the context of sanitation procedures and how they might differ in terms of time and intensity, between physical spaces in clinical settings. For example, the cleaning of an isolation room may require more time or products as all surfaces and equipment are meticulously disinfected or discarded when patients are discharged. This intense terminal cleaning is not comparable to the hygiene and sanitation practices in a common area such as a hallway, where floors and hand railings receive daily disinfection.
The Delphi technique can provide invaluable insight for the development of new solutions; this is especially true within evolving disciplines such as healthcare. For example, Njuangang and colleagues [71], via the Delphi technique, established 11 novel critical success factors and performance measures within hospital maintenance to reduce the risk of maintenance-associated NIs. The ability to identify salient performance measures is also reflected in our study; over two rounds of review, 16 experts expanded and enriched the guide’s functionality and precision to create a tool with 8 dimensions and over 175 items that would assess cost. The strength of our process rested in part on our choice of panelists- a heterogeneous mix of stakeholders who were wholly invested in the innovation of the guide, from its development to its indicators of credibility and quality. From the original group of 18 invitees, 16 members completed the first wave of review, and all but two members completed the second. Those who recused themselves joined our research team in an advisory capacity to further the development and pilot testing of the guide, thus providing expert feedback as part of the working group in the final iteration.
Implications for research and practice
The time and motion guide, in its current version, is an instrument that can effectively allow health care managers and clinicians to assess the costs of nosocomial infection prevention and control practices. This information will complement epidemiological data that tracks the incidence of infection in health care facilities. Importantly, the guide provides a way to measure the cost effectiveness of CBPs in relation to the pathology of NIs. At the time of writing of this manuscript, measuring cost effectiveness in relation to infection control is exceptionally fitting as the world is struggling with the COVID-19 pandemic, where unprecedented infection control and prevention measures are being undertaken.
Aside from cost estimates, the time and motion guide is strategically detailed and can easily be implemented as an online instrument to serve as a training tool for personnel involved in NI control. Along with enumerating, for the user, materials and products used to control the spread of infection, the chronometer of the guide provides an objective measurement of the time taken for CBPs, such as hand washing. While subjectively, healthcare staff may consider their hand hygiene techniques adequate, they may not meet hand hygiene guidelines set out by the WHO that suggest 20–30 seconds for hydro-alcoholic solutions, or 40–60 seconds for soap and water [56]. It could thus serve in training and continuing education. The guide could also help us to better understand how personnel adapt their clinical best care practices to their working conditions. An analysis of the intensity of nosocomial prevention and control measures in terms of costs (time, materials and products) could be undertaken to compare practices and their related costs before, during and after outbreaks or pandemic events such as COVID-19.
Strengths and limitations
A strength of this study is that the time and motion guide represents a new and completely original approach to nosocomial infection prevention and control cost assessment. By adapting the questionnaire to a web-based application, the guide can span geographic separation of health care facilities where it can be tested, and may deliver a framework upon which to structure other health care associated costing tools. The time motion guide is innovative in that it simultaneously measures more than one action at a time, compared to other time motion guides that normally only measure a single action at a time [72, 73]. A further strength of our process is that the Delphi technique was assiduously conceived, administered and reported, an underlying prerequisite for credible conclusions. We attempted to fulfill all sixteen recommendations set out in the Conducting and REporting of DElphi Studies (CREDES) framework [63]. However, this study has limitations. The acceptability of the time and motion guide, and feasibility of use, will have to be pilot-tested by users in different health care facilities and in real-life settings under normal work conditions. Only then will the efficiency and effectiveness of the guide become evident. The study has certain other limitations, notably, that our guide is based on a health care system in one province and country. Health care settings, NI rates and NIPC practices will differ between developed and developing nations, limiting the generalizability of the guide.
Conclusion
We developed and validated a time and motion guide for the measurement of costs of clinical best practices used for nosocomial infection control and prevention in medical and surgical hospital units. Future research should include pilot testing of the guide with a view to how the instrument could be adapted for use in other healthcare settings.
Supporting information
S1 File. Time motion guide questionnaire for experts.
https://doi.org/10.1371/journal.pone.0242212.s001
(DOCX)
Acknowledgments
The authors would like to thank all our funders as well as Patrick Finken for his assistance with the web-based mobile platform.
References
- 1. Allegranzi B, Nejad SB, Combescure C, Graafmans W, Attar H, Donaldson L, et al. Burden of endemic health-care-associated infection in developing countries: systematic review and meta-analysis. The Lancet. 2011;377(9761):228–41. pmid:21146207
- 2. Schwendimann R, Blatter C, Dhaini S, Simon M, Ausserhofer D. The occurrence, types, consequences and preventability of in-hospital adverse events–a scoping review. BMC Health Services Research. 2018;18(1):521. pmid:29973258
- 3. Arefian H, Hagel S, Heublein S, Rissner F, Scherag A, Brunkhorst FM, et al. Extra length of stay and costs because of health care-associated infections at a German university hospital. Am J Infect Control. 2016;44(2):160–6. pmid:26521700
- 4. Bohlouli B, Jackson T, Tonelli M, Hemmelgarn B, Klarenbach S. Health care costs associated with hospital acquired complications in patients with chronic kidney disease. BMC Nephrol 2017;18(375). pmid:29282006
- 5. Cassini A, Plachouras D, Eckmanns T, Abu Sin M, Blank HP, Ducomble T, et al. Burden of Six Healthcare-Associated Infections on European Population Health: Estimating Incidence-Based Disability-Adjusted Life Years through a Population Prevalence-Based Modelling Study. PLoS Med. 2016;13(10):e1002150. pmid:27755545
- 6. McGarry SA, Engemann JJ, Schmader K, Sexton DJ, Kaye KS. Surgical-site infection due to Staphylococcus aureus among elderly patients: mortality, duration of hospitalization, and cost. Infect Control Hosp Epidemiol 2004;25(6):461–7. pmid:15242192
- 7. Serra-Burriel M, Keys M, Campillo-Artero C, Agodi A, Barchitta M, Gikas A, et al. Impact of multi-drug resistant bacteria on economic and clinical outcomes of healthcare-associated infections in adults: Systematic review and meta-analysis. PLOS ONE. 2020;15(1):e0227139. pmid:31923281
- 8.
Public Health Agency of Canada. The Chief Public Health Officer’s report on the state of public health in Canada 2013: infectious disease–the never-ending threat. Ottawa, ON: PHAC. 2013.
- 9.
Chan B, Cochrane D. Measuring Patient Harm in Canadian Hospitals. What can be done to improve patient safety? Canadian Institute for Health Information, Canadian Patient Safety Institute. 2016.
- 10.
Comité d’examen sur la prévention et le contrôle des infections nosocomiales. D’abord, ne pas nuire…Les infections nosocomiales au Québec, un problème majeur de santé, une priorité. Gouvernement du Québec, Ministère de la Santé et des Services sociaux. 2005.
- 11.
MSSS. La prévention et le contrôle des infections nosocomiales: Cadre de référence à l’intention des établissements de santé du Québec. Ministère de la Santé et des Services sociaux-Direction générale de la santé publique; 2006.
- 12.
MSSS. Plan Stratégique du Ministeère de la Santé et des Services sociaux du Québec 2015–2020 (mise à jour 2017). La Direction des communications du ministère de la Santé et des Services sociaux: Ministère de la Santé et des Services sociaux du Québec 2015–2020; 2017 Décembre.
- 13.
Resar R., Griffin FA., Haraden C., Nolan TW. Using Care Bundles to Improve Health Care Quality. IHI Innovation Series white paper. Cambridge, Massachusetts: Institute for Healthcare Improvement (Available on www.IHI.org); 2012.
- 14.
Canadian Patient Safety Institute. Canadian Patient Safety Institute. New approach to controlling superbugs. Safer Healthcare Now! Campaign; Ottawa, ON: CPSI. 2010.
- 15.
Groupe de travail en hygiène et salubrité. Guide de gestion intégrée de la qualité en hygiène et salubrité. Gouvernement du Québec, Ministère de la Santé et des Services sociaux. 2013.
- 16.
Groupe Hygiène et salubrité au regard de la lutte aux infections nosocomiales. Lignes directrices en hygiène et salubrité: analyse et concertation. Québec, QC: Gouvernement du Québec, Ministère de la Santé et des Services sociaux, Direction générale de la coordination, du financement et de l’équipement. 2006.
- 17.
First Affiliated Hospital of Zhejiang University. Handbook of COVID-19 Prevention and Treatment 2020. Available from: https://www.alnap.org/help-library/handbook-of-covid-19-prevention-and-treatment.
- 18.
U.S. Department of Health and Human Sevices, Centers for Disease Control and Prevention. Comprehensive Hospital Preparedness Checklist for Coronavirus Disease 2019 (COVID-19) 2020. Available from: https://www.cdc.gov/coronavirus/2019-ncov/downloads/HCW_Checklist_508.pdf.
- 19. Chun JY, Seo HK, Kim MK, Shin MJ, Kim SY, Kim M, et al. Impact of a hand hygiene campaign in a tertiary hospital in South Korea on the rate of hospital-onset methicillin-resistant Staphylococcus aureus bacteremia and economic evaluation of the campaign. Am J Infect Control. 2016;44(12):1486–91. pmid:27658769
- 20. Roth JA, Hornung-Winter C, Radicke I, Hug BL, Biedert M, Abshagen C, et al. Direct Costs of a Contact Isolation Day: A Prospective Cost Analysis at a Swiss University Hospital. Infect Control Hosp Epidemiol. 2018;39(1):101–3.
- 21. Bessesen MT, Lopez K, Guerin K, Hendrickson K, Williams S, O'Connor-Wright S, et al. Comparison of control strategies for methicillin-resistant Staphylococcus aureus. Am J Infect Control. 2013;41(11):1048–52. pmid:23663857
- 22. Wassenberg MW, Kluytmans JA, Bosboom RW, Buiting AG, van Elzakker EP, Melchers WJ, et al. Rapid diagnostic testing of methicillin-resistant Staphylococcus aureus carriage at different anatomical sites: costs and benefits of less extensive screening regimens. Clinical microbiology and infection: the official publication of the European Society of Clinical Microbiology and Infectious Diseases. 2011;17(11):1704–10. pmid:21595786
- 23. Herr CE, Heckrodt TH, Hofmann FA, Schnettler R, Eikmann TF. Additional costs for preventing the spread of methicillin-resistant Staphylococcus aureus and a strategy for reducing these costs on a surgical ward. Infect Control Hosp Epidemiol. 2003;24(9):673–8. pmid:14510250
- 24. van Rijen MM, Kluytmans JA. Costs and benefits of the MRSA Search and Destroy policy in a Dutch hospital. Eur J Clin Microbiol Infect Dis. 2009;28(10):1245–52. pmid:19618223
- 25. Hassan K, Koh C, Karunaratne D, Hughes C, Giles SN. Financial implications of plans to combat methicillin-resistant Staphylococcus aureus (MRSA) in an orthopaedic department. Ann R Coll Surg Engl. 2007;89(7):668–71.
- 26. Escaut L, Bouam S, Frank-Soltysiak M, Rudant E, Saliba F, Kassis N, et al. Eradication of an outbreak of vancomycin-resistant Enterococcus (VRE): the cost of a failure in the systematic screening. Antimicrob Resist Infect Control. 2013;2(1):18. pmid:23738672
- 27. Chowers M, Carmeli Y, Shitrit P, Elhayany A, Geffen K. Cost Analysis of an Intervention to Prevent Methicillin-Resistant Staphylococcus Aureus (MRSA) Transmission. PLoS One. 2015;10(9):e0138999. pmid:26406889
- 28. Conterno LO, Shymanski J, Ramotar K, Toye B, Zvonar R, Roth V. Impact and cost of infection control measures to reduce nosocomial transmission of extended-spectrum beta-lactamase-producing organisms in a non-outbreak setting. J Hosp Infect. 2007;65(4):354–60.
- 29. Montecalvo MA, Jarvis WR, Uman J, Shay DK, Petrullo C, Horowitz HW, et al. Costs and savings associated with infection control measures that reduced transmission of vancomycin-resistant enterococci in an endemic setting. Infect Control Hosp Epidemiol. 2001;22(7):437–42. pmid:11583213
- 30. Birgand G, Leroy C, Nerome S, Luong Nguyen LB, Lolom I, Armand-Lefevre L, et al. Costs associated with implementation of a strict policy for controlling spread of highly resistant microorganisms in France. BMJ Open. 2016;6(1):e009029. pmid:26826145
- 31. Creamer E, Galvin S, Dolan A, Sherlock O, Dimitrov BD, Fitzgerald-Hughes D, et al. Evaluation of screening risk and nonrisk patients for methicillin-resistant Staphylococcus aureus on admission in an acute care hospital. Am J Infect Control. 2012;40(5):411–5. pmid:21962934
- 32. Gagnaire J, Gagneux-Brunon A, Pouvaret A, Grattard F, Carricajo A, Favier H, et al. Carbapenemase-producing Acinetobacter baumannii: An outbreak report with special highlights on economic burden. Med Mal Infect. 2017;47(4):279–85. pmid:28343727
- 33. Gardam MA, Burrows LL, Kus JV, Brunton J, Low DE, Conly JM, et al. Is surveillance for multidrug-resistant enterobacteriaceae an effective infection control strategy in the absence of an outbreak? The Journal of infectious diseases. 2002;186(12):1754–60. pmid:12447761
- 34. Nelson RE, Samore MH, Jones M, Greene T, Stevens VW, Liu CF, et al. Reducing Time-dependent Bias in Estimates of the Attributable Cost of Health Care-associated Methicillin-resistant Staphylococcus aureus Infections: A Comparison of Three Estimation Strategies. Med Care. 2015;53(9):827–34.
- 35. Scott RD 2nd, Sinkowitz-Cochran R, Wise ME, Baggs J, Goates S, Solomon SL, et al. CDC central-line bloodstream infection prevention efforts produced net benefits of at least $640 Million during 1990–2008. Health Aff (Millwood). 2014;33(6):1040–7. pmid:24889954
- 36. Raschka S, Dempster L, Bryce E. Health economic evaluation of an infection prevention and control program: are quality and patient safety programs worth the investment? Am J Infect Control. 2013;41(9):773–7. pmid:23993762
- 37. Slayton RB, Scott RD, Baggs J, Lessa FC, McDonald LC, Jernigan JA. The cost-benefit of federal investment in preventing Clostridium difficile infections through the use of a multifaceted infection control and antimicrobial stewardship program. Infect Control Hosp Epidemiol. 2015;36(6):681–7. pmid:25783204
- 38. Slover J, Haas JP, Quirno M, Phillips MS, Bosco JA 3rd. Cost-effectiveness of a Staphylococcus aureus screening and decolonization program for high-risk orthopedic patients. J Arthroplasty. 2011;26(3):360–5. pmid:20452175
- 39. Sewell B, Rees E, Thomas I, Ch'ng CL, Isaac M, Berry N. Cost and Impact on Patient Length of Stay of Rapid Molecular Testing for Clostridium difficile. Infect Dis Ther. 2014;3(2):281–93. pmid:25183400
- 40. Barkatali BM, Heywood N, White R, Paton RW. MRSA screening in orthopaedic surgery: clinically valuable and cost effective? A prospective analysis of 8,867 patients. Acta Orthop Belg. 2013;79(4):463–9. pmid:24205779
- 41. Hardy KJ, Szczepura A, Davies R, Bradbury A, Stallard N, Gossain S, et al. A study of the efficacy and cost-effectiveness of MRSA screening and monitoring on surgical wards using a new, rapid molecular test (EMMS). BMC health services research. 2007;7:160 pmid:17915008
- 42.
Institut national de la santé publique du Québec. Surveillance provinciale des diarrhées à Clostridium difficile au Québec: protocole. Montréal, QC: INSPQ [Internet]. 2018. Available from: https://www.inspq.qc.ca/sites/default/files/documents/infectionsnosocomiales/protocole_dacd_2018.pdf.
- 43.
Institut national de la santé publique du Québec. Surveillance provinciale des bactériémies nosocomiales dans les centres hospitaliers de soins aigus du Québec: protocole. Montréal, QC: INSPQ; 2018. 2018.
- 44.
Institut national de la santé publique du Québec. Surveillance provinciale des infections à entérocoque résistant à la vancomycine au Québec: protocole. Montréal, QC: INSPQ. 2018.
- 45.
Institut national de la santé publique du Québec. Surveillance provinciale des bactériémies à Staphylococcus au Québec: protocole. Montréal, QC: INSPQ. 2018.
- 46.
Des soins de santé plus sécuritaires maintenant! Cadres de travail de l’amélioration de la qualité. Trousse en avant! 2015.
- 47.
Ministère de la Santé et des Services sociaux, Plan d'action 2010–2015. Prévention et contrôle des infections nosocomiales 2011 Available from: https://publications.msss.gouv.qc.ca/msss/fichiers/2010/10-209-04.pdf.
- 48.
Ministère de la Santé et des Services sociaux, 2010–2015. Prévention et contrôle des infections nosocomiales, Plan d’action 2010–2015, État d’avancement des travaux, Bilan et faits saillants 2013. Available from: https://publications.msss.gouv.qc.ca/msss/fichiers/2013/13-209-01W.pdf.
- 49.
Agence de la santé publique du Canada. Rapport de l’administrateur en chef de la santé publique sur l’état de la santé publique au Canada, 2013 –Les infections associées aux soins de santé–une diligence raisonnable 2013. Available from: https://www.canada.ca/fr/sante-publique/organisation/publications/rapports-etat-sante-publique-canada-administrateur-chef-sante-publique/rapport-administrateur-chef-sante-publique-etat-sante-publique-canada-2013-maladies-infectieuses-menace-perpetuelle/les-infections-associees-aux-soins-de-sante-une-diligence-raisonnable.html.
- 50.
U.S. Department of Health & Human Services, Centers for Disease Control and Prevention. Guidelines for Environmental Infection Control in Health-Care Facilities, Recommendations of CDC and the Healthcare, Infection Control Practices Advisory Committee (HICPAC) 2003; 52(RR 10):[1–42 pp.]. Available from: https://www.cdc.gov/mmwr/preview/mmwrhtml/rr5210a1.htm.
- 51.
U.S. Department of Health & Human Services, Centers for Disease Control and Prevention. Guideline for Hand Hygiene in Health-Care Settings, Recommendations of the Healthcare Infection Control Practices Advisory Committee and the HICPAC/SHEA/APIC/IDSA Hand Hygiene Task Force 2002; 16(51):[1–44 pp.]. Available from: https://www.cdc.gov/mmwr/preview/mmwrhtml/rr5116a1.htm.
- 52.
Organisation Mondiale de la Santé. Le Guide pédagogique pour la sécurité des patients. 2011.
- 53.
Organisation Mondiale de la Santé. Modèle pour l’auto-évaluation de la promotion et des pratiques d’hygiène des mains au niveau de l’établissement de soins. 2010.
- 54.
World Health Organization. Prévention des infections nosocomiales 2002.
- 55.
U.S. Department of Health & Human Services, Center for Disease Control and Prevention. Hand Hygiene in Healthcare Settings [Available from: https://www.cdc.gov/handhygiene/providers/index.html.
- 56.
World Health Organization. WHO Guidelines on hand hygiene in health care, first global patient safety challenge clean care is safer care 2009. Available from: https://apps.who.int/iris/bitstream/handle/10665/44102/9789241597906_eng.pdf.
- 57. Kaplan RS, Anderson SR. Time-driven activity-based costing. Harvard Business Review. 2004; 82(50):131–138. pmid:15559451
- 58. Keel G, Savage C, Rafiq M, Mazzocato P. Time-driven activity-based costing in health care: A systematic review of the literature. Health Policy. 2017;121(7):755–63. pmid:28535996
- 59. Lopetegui M, Yen P-Y, Lai A, Jeffries J, Embi P, Payne P. Time motion studies in healthcare: what are we talking about? J Biomed Inform. 2014;49:292–9. pmid:24607863
- 60. Eckmanns T, Bessert J, Behnke M, Gastmeier P, Ruden H. Compliance with antiseptic hand rub use in intensive care units: the Hawthorne effect. Infect Control Hosp Epidemiol. 2006;27(9):931–4. pmid:16941318
- 61. Lange T, Kopkow C, Lützner J, Günther K-P, Gravius S, Scharf H-P, et al. Comparison of different rating scales for the use in Delphi studies: different scales lead to different consensus and show different test-retest reliability. BMC Med Res Methodol. 2020;20(1):28. pmid:32041541
- 62. Akins RB, Tolson H, Cole BR. Stability of response characteristics of a Delphi panel: application of bootstrap data expansion. BMC Med Res Methodol. 2005;5:37-. pmid:15670334
- 63. Junger S, Payne SA, Brine J, Radbruch L, Brearley SG. Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: Recommendations based on a methodological systematic review. Palliative medicine. 2017;31(8):684–706. pmid:28190381
- 64. Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C. Using and Reporting the Delphi Method for Selecting Healthcare Quality Indicators: A Systematic Review. PLOS ONE. 2011;6(6):e20476. pmid:21694759
- 65. Knight S, Aggarwal R, Agostini A, Loundou A, Berdah S, Crochet P. Development of an objective assessment tool for total laparoscopic hysterectomy: A Delphi method among experts and evaluation on a virtual reality simulator. PLoS One. 2018;13(1):e0190580. pmid:29293635
- 66. Vogel C, Zwolinsky S, Griffiths C, Hobbs M, Henderson E, Wilkins E. A Delphi study to build consensus on the definition and use of big data in obesity research. International Journal of Obesity. 2019;43(12):2573–86. pmid:30655580
- 67. Havers SM, Martin E, Wilson A, Hall L. Implementation of government-directed policy in the hospital setting: a modified Delphi study. Health Research Policy and Systems. 2019;17(1):91. pmid:31775771
- 68.
Turoff M. The Policy Delphi. (Eds.) HLMT, editor2002.
- 69. Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in nursing & health. 2007;30(4):459–67.
- 70. Birko S, Dove ES, Ozdemir V. Evaluation of Nine Consensus Indices in Delphi Foresight Research and Their Dependency on Delphi Survey Characteristics: A Simulation Study and Debate on Delphi Design and Interpretation. PLoS One. 2015;10(8):e0135162. pmid:26270647
- 71. Njuangang S, Liyanage C, Akintoye A. Application of the Delphi technique in healthcare maintenance. International Journal of Health Care Quality Assurance. 2017;30(8):737–54. pmid:28958204
- 72. Kai Zheng, Guo Michael H, Hanauer David A. Using the time and motion method to study clinical work processes and workflow: methodological inconsistencies and a call for standardized research. J Am Med Inform Assoc. 2011;18:704–10. pmid:21527407
- 73. Ogunfiditimi F., Takis I., Paige V. J., Wyman J. F., Marlow E. Assessing the Productivity of Advanced Practice Providers Using a Time and Motion Study. Journal of healthcare management. 2013;58(3):173–85.