Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Supporting SURgery with GEriatric Co-Management and AI (SURGE-Ahead): A study protocol for the development of a digital geriatrician

  • Christoph Leinert ,

    Roles Conceptualization, Methodology, Project administration, Writing – original draft

    christoph.leinert@uni-ulm.de

    Affiliations Institute for Geriatric Research at AGAPLESION Bethesda Ulm, Ulm University Medical Center, Ulm, Germany, Geriatric Center Ulm, Ulm, Germany

  • Marina Fotteler,

    Roles Conceptualization, Methodology, Project administration, Writing – original draft

    Affiliations Institute for Geriatric Research at AGAPLESION Bethesda Ulm, Ulm University Medical Center, Ulm, Germany, Geriatric Center Ulm, Ulm, Germany, DigiHealth Institute, Neu-Ulm University of Applied Sciences, Neu-Ulm, Germany

  • Thomas Derya Kocar,

    Roles Conceptualization, Methodology, Project administration, Writing – original draft

    Affiliations Institute for Geriatric Research at AGAPLESION Bethesda Ulm, Ulm University Medical Center, Ulm, Germany, Geriatric Center Ulm, Ulm, Germany

  • Dhayana Dallmeier,

    Roles Conceptualization, Methodology, Supervision, Writing – review & editing

    Affiliations Institute for Geriatric Research at AGAPLESION Bethesda Ulm, Ulm University Medical Center, Ulm, Germany, Geriatric Center Ulm, Ulm, Germany, Department of Epidemiology, Boston University School of Public Health, Boston, MA, United States of America

  • Hans A. Kestler,

    Roles Conceptualization, Methodology, Software, Supervision, Writing – review & editing

    Affiliation Institute for Medical Systems Biology, Ulm University, Ulm, Germany

  • Dennis Wolf,

    Roles Software, Writing – original draft

    Affiliation Institute for Medical Systems Biology, Ulm University, Ulm, Germany

  • Florian Gebhard,

    Roles Conceptualization, Investigation, Writing – review & editing

    Affiliation Department for Orthopedic Trauma, Ulm University Medical Center, Ulm, Germany

  • Adriane Uihlein,

    Roles Investigation, Writing – review & editing

    Affiliation Department for Orthopedic Trauma, Ulm University Medical Center, Ulm, Germany

  • Florian Steger,

    Roles Conceptualization, Writing – review & editing

    Affiliation Institute of the History, Philosophy and Ethics of Medicine, Ulm University, Ulm, Germany

  • Reinhold Kilian,

    Roles Conceptualization, Methodology, Writing – review & editing

    Affiliation Department of Psychiatry and Psychiatry II, Section of Health Economics and Health Services Research, Ulm University, Guenzburg, Germany

  • Annabel S. Mueller-Stierlin,

    Roles Writing – review & editing

    Affiliations Department of Psychiatry and Psychiatry II, Section of Health Economics and Health Services Research, Ulm University, Guenzburg, Germany, Institute for Epidemiology and Medical Biometry, Ulm University, Ulm, Germany

  • Christoph W. Michalski,

    Roles Writing – review & editing

    Affiliation Department of General and Visceral Surgery, Ulm University Hospital, Ulm, Germany

  • André Mihaljevic,

    Roles Writing – review & editing

    Affiliation Department of General and Visceral Surgery, Ulm University Hospital, Ulm, Germany

  • Christian Bolenz,

    Roles Writing – review & editing

    Affiliation Department of Urology, Ulm University Hospital, Ulm, Germany

  • Friedemann Zengerling,

    Roles Writing – review & editing

    Affiliation Department of Urology, Ulm University Hospital, Ulm, Germany

  • Elena Leinert,

    Roles Writing – review & editing

    Affiliation Department of Gynecology and Obstetrics, Ulm University Hospital, Ulm, Germany

  • Sabine Schütze,

    Roles Writing – review & editing

    Affiliation Department of Gynecology and Obstetrics, Ulm University Hospital, Ulm, Germany

  • Thomas K. Hoffmann,

    Roles Writing – review & editing

    Affiliation Department of Otorhinolaryngology, Head and Neck Surgery, Ulm University Hospital, Ulm, Germany

  • Graziano Onder,

    Roles Supervision, Writing – review & editing

    Affiliation Department of Cardiovascular, Endocrine-metabolic Diseases and Aging, Instituto Superiore di Sanità, Rome, Italy

  • Karen Andersen-Ranberg,

    Roles Supervision, Writing – review & editing

    Affiliation Geriatric Research Unit, Department of Clinical Research, Faculty of Health, University of Southern Denmark, Odense, Denmark

  • Desmond O’Neill,

    Roles Supervision, Writing – review & editing

    Affiliation Centre for Ageing, Neuroscience and the Humanities, Trinity College Dublin, Dublin, Ireland

  • Martin Wehling,

    Roles Supervision, Writing – review & editing

    Affiliation Working Group FORTA, Faculty of Medicine Mannheim, Ruprecht-Karls-University of Heidelberg, Mannheim, Germany

  • Johannes Schobel,

    Roles Writing – review & editing

    Affiliation DigiHealth Institute, Neu-Ulm University of Applied Sciences, Neu-Ulm, Germany

  • Walter Swoboda,

    Roles Writing – review & editing

    Affiliation DigiHealth Institute, Neu-Ulm University of Applied Sciences, Neu-Ulm, Germany

  • Michael Denkinger,

    Roles Conceptualization, Funding acquisition, Methodology, Project administration, Supervision, Writing – original draft

    Affiliations Institute for Geriatric Research at AGAPLESION Bethesda Ulm, Ulm University Medical Center, Ulm, Germany, Geriatric Center Ulm, Ulm, Germany

  •  [ ... ],
  • for the SURGE-Ahead Study Group

    Membership of the SURGE-Ahead Study Group are provided in the Acknowledgments.

  • [ view all ]
  • [ view less ]

Abstract

Introduction

Geriatric co-management is known to improve treatment of older adults in various clinical settings, however, widespread application of the concept is limited due to restricted resources. Digitalization may offer options to overcome these shortages by providing structured, relevant information and decision support tools for medical professionals. We present the SURGE-Ahead project (Supporting SURgery with GEriatric co-management and Artificial Intelligence) addressing this challenge.

Methods

A digital application with a dashboard-style user interface will be developed, displaying 1) evidence-based recommendations for geriatric co-management and 2) artificial intelligence-enhanced suggestions for continuity of care (COC) decisions. The development and implementation of the SURGE-Ahead application (SAA) will follow the Medical research council framework for complex medical interventions. In the development phase a minimum geriatric data set (MGDS) will be defined that combines parametrized information from the hospital information system with a concise assessment battery and sensor data. Two literature reviews will be conducted to create an evidence base for co-management and COC suggestions that will be used to display guideline-compliant recommendations. Principles of machine learning will be used for further data processing and COC proposals for the postoperative course. In an observational and AI-development study, data will be collected in three surgical departments of a University Hospital (trauma surgery, general and visceral surgery, urology) for AI-training, feasibility testing of the MGDS and identification of co-management needs. Usability will be tested in a workshop with potential users. During a subsequent project phase, the SAA will be tested and evaluated in clinical routine, allowing its further improvement through an iterative process.

Discussion

The outline offers insights into a novel and comprehensive project that combines geriatric co-management with digital support tools to improve inpatient surgical care and continuity of care of older adults.

Trial registration

German clinical trials registry (Deutsches Register für klinische Studien, DRKS00030684), registered on 21st November 2022.

Introduction

Geriatric co-management is defined as the process of shared responsibility and decision making between a medical doctor, e.g. a surgeon, and a geriatrician or a multidisciplinary team aiming to prevent or treat geriatric complications [1]. It has reduced time-to-surgery, hospital and long term mortality, length of stay, and complications in a variety of health care settings across the globe [16]. Research has shown positive implications for different disciplines, such as trauma surgery [4], general surgery [7], oncological surgery [8], urology [9], and gynecology [10]. Despite these advantages, restricted financial resources and a limited number of expert geriatricians and their multidisciplinary teams make a widespread implementation of geriatric co-management in hospitals challenging [11, 12].

A key aspect of geriatric co-management is a comprehensive geriatric assessment (CGA) conducted by a multidisciplinary team. A CGA is a holistic approach for assessing a patient in different domains to determine physical, psychological, social, and functional impairments. Patients that underwent a CGA in the context of geriatric and surgical care are more likely to survive and less likely to be institutionalized [13, 14]. Another important aspect of geriatric co-management is individualized continuity of care (COC) planning. In complex health care systems, knowledge gaps concerning different options of continuous care for older people can lead to misdirection of patients. High level evidence indicates that a targeted discharge management reduces readmission rates, hospital length of stay, costs, and increases patient satisfaction [15]. In contrast, deficits in coordination of care transitions between different health care institutions have been identified as one aspect of iatrogenic disability [16].

In Germany, the amount of geriatric cases has doubled since 1999 and geriatric medicine has been the fastest growing clinical discipline in recent years [17]. Prospectively collected data from the German fragility fracture registry confirmed the benefit of orthogeriatric co-management on mortality [18]. Still, not all patients in need for geriatric care are treated accordingly. For example, in the federal state of Baden-Wuerttemberg less than 5% of geriatric patients, defined by multidimensional screening (“Geriatrie Check BW”), received a geriatric assessment in general practice. Only 22%, 38%, and 2% of geriatric patients with a stroke, femoral fracture, or heart failure, respectively, received treatment in either a department for geriatric rehabilitation or early inpatient geriatric rehabilitation [19].

Structures of geriatric care and options for the continuous care of older patients differ between federal states and regions [20]. Acute medical treatment with the option of early inpatient rehabilitation, day clinic medical treatment, and inpatient or outpatient geriatric rehabilitation can be available as individual or combined services. In Baden-Wuerttemberg, the main destinations after discharge of older people are inpatient acute geriatric care, inpatient acute care in another specialty, a geriatric or a specialized orthopedic or neurologic inpatient or outpatient rehabilitation, day care facilities, long term care facilities including nursing homes, or home with or without additional care.

Orthogeriatric co-management was announced to become a legal requirement for the treatment of older hip fracture patients in Germany by 2025 [21], increasing demand for geriatric expertise. Digital health interventions could help meet this demand [22]. To date, in geriatric medicine, digital technologies have been used to enhance patient assessment, e.g. via sensor-based gait analysis [23], continuous mobility monitoring [24], or digital patient-reported outcome measures for frailty assessment [25]. In the United Kingdom the identification of patients in need of geriatric services is possible using an automated hospital frailty risk score based on health administrative data on medical diagnosis [26]. Processing, analysis, and integration of these new data sources into existing structures is still lacking but has the potential to change common practice [27, 28].

By developing a digital health application, SURGE-Ahead (Supporting SURgery with GEriatric Co-Management and Artificial Intelligence) aims to bring geriatric expertise and support directly to the point of care in surgical departments in an effective and sustainable way.

Methods

The project aims to develop the SURGE-Ahead application (SAA, a dashboard-style user-interface) connected to the hospital information system (HIS) and the laboratory information system (LIS) that will display two core components: (1) evidence-based, easy-to-use clinical co-management recommendations and (2) artificial intelligence (AI)-enhanced continuity of care (COC) proposals. Functionality of these two core components will be based on a minimum geriatric dataset (MGDS) consisting of data from different sources (Fig 1).

thumbnail
Fig 1. Core components of the SURGE-Ahead intervention.

The figure shows the data sources of the minimum geriatric data set (MGDS) describing support of geriatric co-management and continuity of care as the two core components. After processing this input, the SURGE-Ahead application (SAA) will display recommendations for these components as data output.

https://doi.org/10.1371/journal.pone.0287230.g001

The development, implementation, testing, and evaluation of the SURGE-Ahead application is a complex health care intervention. Therefore, SURGE-Ahead is structured according to the guidelines of the Medical Research Council (MRC) for development and evaluation of complex interventions [29, 30]. This development process encompasses four non-linear steps: Development, feasibility/piloting, implementation, and evaluation, with iterative improvements as the project progresses.

The current project focuses on the development of the SAA and will also prepare the implementation and piloting of the application. It is conducted in close cooperation with a multidisciplinary research team including geriatricians, computer scientists, surgeons, ethicists, epidemiologists, public health, and health economics specialists. Additionally, the team is supported by an international advisory board consisting of expert European geriatric researchers. The goals of the development phase are to establish an evidence base, develop a MGDS as input, and prepare evidence-based recommendations as output of the SAA. Furthermore, the digital application and its dashboard-style data user interface, data processing, AI integration, and connection to local hospital information system (HIS) and laboratory information system (LIS) will be programmed. The implementation of the SAA is prepared by analysis of local structures, stakeholder interviews, and functionality and usability tests with potential users. In a final step, AI training data will be gathered, co-management needs will be identified, and MGDS feasibility testing will be conducted in an observational and AI-development study. Ethical implications of the AI-enhanced digital health intervention will be constantly evaluated. The project has been reviewed in a peer-review- process and granted funding by the German Federal Ministry of Education and Research. Sponsor had no influence on study design, collection, management, analysis, or interpretation of data.

Table 1 summarizes the main objectives of the SURGE-Ahead project and Fig 2 shows a logic model describing the development and piloting process.

thumbnail
Fig 2. Logic model of the SURGE-Ahead project.

The figure shows the different resources/inputs and activities as planned work and the outputs and outcomes as intended results of the project. Interactions between main components are highlighted. AI: artificial intelligence, BPMN: Business Process Model and Notation, COC: continuity of care, IT: information technology, MGDS: Minimum Geriatric Dara Set, SAA: SURGE-Ahead application.

https://doi.org/10.1371/journal.pone.0287230.g002

Establishment of an evidence-base

Two major literature reviews, focusing on the two core components of the dashboard will be conducted.

Geriatric co-management.

In a systematic review of clinical guidelines, current recommendations for geriatric co-management of orthogeriatric inpatients will be identified. The review will follow the Preferred Reporting Items for Systematic reviews and Meta-Analyses Extension for Scoping reviews (PRISMA-ScR) and a protocol has been published in the international prospective register of systematic reviews (PROSPERO) [31]. Only evidence-based guidelines published between 2016 and 2021 evaluating the level of evidence (e.g., by using the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) framework) will be included. We chose to focus on orthogeriatric care because the highest-rated evidence in geriatric co-management exists in this field [1].

Continuity of care.

The second core concept is the improvement of continuity of care decisions for older adults discharged from surgery. In a scoping review, eligible predictors and outcome measures of successful continuity of care decisions will be identified to be included in the MGDS and the AI algorithm used to propose the optimal discharge destination for each individual. As the topic and available literature is heterogenous including different study types, we decided to follow a scoping review approach. This methodology offers the opportunity for a broader view on a topic but supports only basic quantitative analysis [32]. The scoping review methodology of the Joanna Biggs Institute (JBI) and the PRISMA-ScR [33, 34] will be followed. A protocol has been published on Open Science Framework (OSF) [35].

Development of the minimum geriatric data set (MGDS)

The MGDS is set up to be the comprehensive input data set of the study. It focuses on two aspects: (1) to provide enough detail to represent relevant aspects of a proper CGA conducted by a multidisciplinary team, (2) to be as concise and short as possible to be conducted in surgical settings with limited time and human resources.

Based on preliminary results from the literature review on geriatric co-management, the following topics and domains relevant to geriatric surgical care have been identified: Activities of daily living (ADL), frailty and multimorbidity, incontinence, mental health (delirium, dementia, depression), mobility and falls, nutrition, pain, polypharmacy, sensory impairment, and social support. These domains have been discussed with an international advisory board. Based on the utility of the assessments for clinical decisions, parsimony of data, and feasibility for the use in the context of a surgical department a first MGDS has been defined. In the observational study, the defined MGDS will be tested for feasibility and adapted, if necessary. The future SAA will be connected to existing IT-systems to include available data from the HIS and LIS (e.g., medications, social anamnesis, International Statistical Classification of Diseases and Related Health Problems (ICD) codes, laboratory results). In addition, we plan to include sensor-data, particularly for mobility and functionality parameters, provided by a six-axis accelerometer (AX6®, Axivity Ltd.). With the collected data, the final results of the conducted reviews, and the information and feedback provided by contributing surgeons and the advisory board the MGDS will be finalized.

Preparation of co-management recommendations

On the basis of the systematic guideline review, evidence-based comprehensible co-management recommendations will be prepared for display as the output data of the SAA. The recommendations will cover identification and treatment of common geriatric syndromes like delirium, immobility or malnutrition. As an example, for patients with delirium or other cognitive impairments, resources and advice for non-pharmacological delirium prevention and management strategies should be provided to the treating medical teams [36, 37].

Furthermore, the SAA will incorporate an automated medication review plan to help manage polypharmacy and avoid potentially inappropriate medications (PIM). Optimizing medication in geriatric patients by deprescribing PIMs is an important aspect of geriatric co-management. Specifically, the reduction of anticholinergic medical burden is an integral part of multicomponent interventions to prevent delirium [38]. Not only over- but also undertreatment is a problem in older patients, e.g., specific osteoporosis treatment is often neglected. To address these issues the dashboard will include an analysis of the patient’s medication based on the Fit for The Aged (FORTA) classification of medication. This will be provided by the external, commercial partner OptiMedis AG® [39].

In addition, data collected during the pilot observational study will help to identify further needs for co-management in the local context, leading to an iterative improvement process. In a final step, the recommendations will be consolidated and approved by the scientific advisory board. The collected recommendations will be prepared for display in the digital dashboard.

Programming of the digital SURGE-Ahead application (SAA)

Dashboard-style user interface.

The front-end of the application will be based on the established Grafana framework, an open-source platform-independent application that allows the visualization of multiple data sources [40]. All graphical dashboard elements in Grafana are independent widgets predefined to display single values or time series such as the patient’s age or change of assessment ratings over time, respectively. Alternatively, Grafana can be extended by custom elements to add new functionalities such as data input. This way, clinical staff will be able to enter patient data (e.g., weight or assessment scores) directly into the dashboard. All data will be stored in a secure Redis database run within the hospital network and visualized in the dashboard applying evidence-based recommendations and rules.

AI for COC-recommendation.

AI development follows the World Health Organization guidelines on ethics and governance of artificial intelligence for health [41]. Based on the data collected in the observation and AI-development study, multiple machine learning models of variant complexity will be implemented using languages such as R, Python, and Julia [42]. Both the expert recommendation of a geriatrician before discharge and follow-up data regarding COC quality will be evaluated as ground truth. A service will include the best-performing model and monitor incoming patient data to create or update COC-proposals. All new data will be used to retrain the model and constantly improve proposal accuracy. The model uncertainty and feature importance (i.e., parameters affecting the decision) will be displayed alongside each recommendation to improve model explainability and trust with the goal of a trustworthy AI according to the European Commission guidelines [43, 44].

Linkage to HIS and LIS.

Data relevant for the MGDS will be identified in the local HIS and LIS and an automated extraction from IT-systems will be prepared for the final SAA.

Preparation of implementation in local context

The departments for trauma surgery, general and visceral surgery, and urology of Ulm University Hospital will participate as the sites of primary implementation. As trauma surgery has the greatest need for geriatric co-management, a special focus will be put on this department. With the exception of the department of trauma surgery, where a liaison-based orthogeriatric co-management is already implemented, none of the other surgical departments have regular geriatric support.

Preparing the implementation of a complex intervention in the local context is an important step in the MRC framework. A lack of thorough preparation is a common reason why complex interventions do not transfer into clinical routine [45]. Local routines and personal preferences of stakeholders and clinical staff have to be incorporated into the development process. Several steps will be taken to ensure participation of local staff and the fit of the intervention to local structures.

Business Process Model and Notation (BPMN 2.0) will be used to model and analyze an exemplary process of an emergency femoral neck fracture admission and surgery. BPMN is a high-level standard to graphically represent the steps in a workflow and particularly useful for complex processes [46, 47]. Institutions or departments, participating personnel, necessary tasks, gateways, and resources will all be included in the BPMN-model, helping to identify the best options for the dashboard implementation.

Expert interviews with surgeons, nurses, social workers, and therapist will be conducted to identify local needs and resources concerning geriatric co-management of older patients. Additionally, personal attitudes regarding ethical implications of the project will be investigated to be incorporated during the dashboard development and implementation as needed.

To further incorporate the views of local stakeholders, consultant and attending surgeons of each department will be involved in the project development and participate in regular workshops and conferences to discuss and consent proceedings of the project with the study group and the international advisory board.

Reducing workload for clinical practitioners is an important requirement for a successful digital application. In this context, not only an efficient technical connection to existing IT systems, but also an intuitive and user-friendly interface, will be important for the success of the technology [48]. A prototype of the SAA will be evaluated and discussed with potential users of the application in a multi-stage, iterative process incorporating imaginary case examples as well as workshops. Usability, and functionality of the SAA will be evaluated with established tools such as the Technical Acceptance Model (TAM) or the Technology Usage Inventory (TUI) as well as self-designed feedback forms and improved accordingly [49, 50].

Observation and AI-development study

Study design.

A one-year prospective observational study with a 9-month recruitment and 3-month follow up period will be conducted in the departments of trauma surgery, general and visceral surgery, and urology starting in February 2023. The dashboard will not be displayed in the study, no intervention is planned. The main objectives are:

  1. Training of the AI. Based on observational data a ground truth for AI training concerning COC proposals will be defined.
  2. Feasibility testing of MGDS. The MGDS will be tested regarding feasibility and quality of data collection in a surgical setting.
  3. Analysis of a standard of care cohort. The collected data will be used to identify specific local needs and resources for geriatric co-management.

Study population.

Patients 70 years or older undergoing an emergency or elective surgery in the departments of trauma surgery, general and visceral surgery, or urology, an Identification of Seniors at Risk (ISAR)-Score of ≥2 and providing informed consent will be included [51]. In case of recruitment barriers patients with an ISAR-Score of ≥1 will be considered as well.

Informed consent will be obtained from all participants by study doctors. If consent is not possible because of cognitive decline, legal representatives will be contacted. Patients with a life expectancy of less than 3 month according to clinical judgement will be excluded. Patients can withdraw their participation at any timepoint. Reasons for discontinuation will be assessed if possible.

Sample size.

Sample size is based on the minimum number of patients needed to train the AI algorithm [52]. Because higher numbers will improve AI performance a recruitment corridor between 170 to 240 patients including an expected dropout rate of 20% will be aimed for. Recruitment will focus on the department of trauma surgery, as this is where the greatest range and variability of possible COC options are covered. In addition, the majority of geriatric trauma patients are emergency cases with underlying geriatric conditions and are more likely to benefit from a subsequent co-management. The following target allocation has been determined: 120–190 participants in trauma surgery, 25 participants in general and visceral surgery, and 25 participants in urology.

Participants timeline.

Preoperative assessments (T0) are divided into compulsory preoperative (T0.1) and optional preoperative (T0.2) assessments. To acknowledge the potential lower resilience of some participants, T0.2 assessments can either be collected pre- or postoperatively, including collateral history. The postoperative interviews and assessments will be performed on days 1, 3, 5 and 7 after surgery (T1-T4). From postoperative day 1 until day 7 the mobility sensor will be attached. If patients are discharged earlier, the postoperative assessments will only be performed until the day of discharge. One to three days before discharge, the discharge assessments will be performed and the expert COC proposal will be documented by a geriatrician. After discharge, final data from the hospital information system data will be recorded, especially adverse events and complications. The medical discharge report will be critically assessed in terms of completeness and quality (time independent) [53]. A telephone follow-up will be performed with participants and general practitioners to evaluate COC decisions on discharge from surgery 90 days (± 7 days) after discharge (T6). Participants’ timeline is displayed in Fig 3.

thumbnail
Fig 3. Participants timeline SURGE-Ahead observation and AI-development study.

HIS: Hospital Information System, LIS: Laboratory Information System, SURG: Surgery.

https://doi.org/10.1371/journal.pone.0287230.g003

Data collection, management and monitoring.

Data at baseline and follow-up visits will be collected via a self-programmed electronic case report form (eCRF) hosted within the hospital network. All assessors and study nurses will be trained in the assessments and eCRF usage and regular monitoring will be performed. 30 patients (10 from each department) will self-complete a paper-based activity diary to evaluate accuracy of sensor data algorithms. Trained scientific data managers will double-check collected data in the eCRF database and query inconsistencies directly with the study nurses. Because of this ongoing monitoring of data quality, a data monitoring committee will not be installed. All data will be stored in a pseudonymized fashion and filed for at least 10 years.

Measurements.

Fig 4 shows all assessments and their schedule throughout the study phase.

thumbnail
Fig 4. Schedule of enrollment and assessments for SURGE-Ahead observation and AI development study.

4-AT: 4 A´s test [57], 5min-MoCA: 5-min Montreal Cognitive Assessment [58], ASA: American Society of Anesthesiologists score, BI: Barthel Index [54], cGFR: calculated glomerular filtration rate with Cockroft Gault formular, CHARMI: Charité Mobility Index [56], CSSRI: Client Sociodemographic and Service Receipt Inventory [59], CFS: Clinical Frailty Scale [60], COC: continuity of care, CRP: c-reactive protein, EQ 5D-5L: EuroQoL 5D Health Questionnaire [61], GTT: Global Trigger Tool [62], HIS: hospital information system, LIS, Laboratory information system, MCV: mean cellular volume, NMS: New Mobility Score [55], NRS-pain: Numeric rating scale pain, NRS-nutrition: nutrition risk scale [63], PHQ-4: patient health questionnaire 4 items, QoL: Quality of Life.

https://doi.org/10.1371/journal.pone.0287230.g004

Primary outcomes on level of care will be assessed in three ways:

  1. An expert COC proposal by a geriatrician based on the collected MGDS data, the medical record and a personal patient contact will be made prospectively before discharge and reevaluated at 90 days follow up. This proposal will be used exclusively for the training of the AI and will have no influence on treatment and discharge decisions.
  2. Any change of level of care (German: ‘Pflegegrad’), use of outpatient nursing services and institutionalization from baseline to discharge and 90 day follow.
  3. Change from baseline to discharge and 90 days follow up of ADL and mobility measured by Barthel index (BI), New Mobility Score (NMS) and Charité Mobility Index (CHARMI) [5456].

Self-reported opinion of participants and general practitioners on COC process will be documented at 90 days follow up to assess appropriateness of COC.

Secondary outcomes are listed in Fig 4.

Statistics.

To identify determinants of COC proposals, regression analyses as well as machine learning approaches will be used. Descriptive statistical methods will be used to highlight specific local needs for geriatric co-management, e.g. delirium prevalence. MGDS feasibility and data quality will be evaluated by analysis of missing data patterns. Subgroup analysis will be done in participants enrolled in trauma surgery as well as for the documented discharge destinations. SAS, R, Python and Stata software will be used for statistical analyses and imputation. Health economic analyses will be conducted from the payer perspective by a preliminary cost-utility analysis with the net-monetary benefit regression approach [6466]. Quality adjusted life years (QALY) will be estimated on the basis of the EQ-5D-5L using the current German value set [6769].

Ethics approval and trial registration.

This study received written ethical consent from the ethics committee of University of Ulm (# 310/22 dated 19th October 2022) (see S3 and S4 Files). Furthermore, the study has been registered in the German clinical trials registry (# DRKS00030684) on 21st November 2022.

Ethical aspects of AI-enhanced health interventions

An important point throughout the project is consideration of ethical concerns and challenges connected to the use of AI in medical care. The choice of suitable postsurgical treatment and care options is an important decision in surgical treatment and the responsibility of the treating consultant surgeon. Therefore, in the context of an AI-enhanced decision support tool, the proposals displayed by the dashboard have to be comprehensible and clearly marked as a suggestion not a decision.

Three requirements are indispensable for an adequate use of such AI-based support tool. Firstly, the treated patients need to be comprehensively informed about the use of such an application and potential risks of such use. The process of informed consent in case of AI-aided medical decision-making should encompass not only information disclosure but also understanding, voluntariness, and competence to decide [70]. Secondly, to comprehensively inform patients, physicians need to have relevant knowledge and understand implications of the use of this technology. Therefore, tailored training of clinical staff in the area of medical AI is required [70]. Thirdly, physicians using the application need to comprehend the rationale behind the AI-proposals. Use of “explainable AI” is recommended by the guidelines of the European Commission to increase trust and acceptance of the AI-generated suggestions for COC [43, 44, 71].

A systematic review concerning ethical aspects of AI-enhanced medical technologies will be conducted. Interviews concerning this topic with local clinical staff as well as patients and caregivers will be conducted, giving them the opportunity to express concerns but also potential chances they might see in the project. These findings will be integrated into the development of the SAA.

Discussion

SURGE-Ahead aims do develop a complex digital health intervention, consisting of a digital version of a CGA presented in the user-friendly SAA that is connected to all relevant data sources and displays high-quality recommendations for geriatric co-management and COC decisions for geriatric patients undergoing a surgical procedure. According to the classification of digital health interventions of the World Health Organization, the programmed application can be categorized addressing clients, healthcare providers, and data services [22]. Before proceeding to a validation study of the dashboard intervention, our application will need to undergo the legal process of the European and German medical device regulation. In a second phase of the project, the clinical impact of the application will be tested at different surgical departments of Ulm University Hospital in a pilot interventional study. In addition to the departments of trauma surgery, general and visceral surgery, and urology, the recruitment is planned to be conducted in the departments of gynecology and ear-nose-throat (ENT) medicine.

The SURGE-Ahead intervention has the potential to change clinical practice. Although most current evidence on geriatric co-management focuses on orthogeriatrics, a CGA and geriatric co-management have also been beneficial in other surgical disciplines and non-emergency admissions [710]. In a recent study, CGA concepts did not work without a geriatrician and a multidisciplinary team, presumably because of the need for expert CGA interpretation and general guidance of the care process [74]. With a concise but comprehensive SAA we aim to partly overcome these issues by providing basic interpretation of MGDS data and AI-enhanced COC support. While for some geriatric patients, the automatically generated advice of the SAA might be sufficient to improve individual care, for others it might work as a differentiated screening tool for complex geriatric needs leading to a resource-oriented allocation of specific geriatric services [11]. While established screening tools have shown deficits in patient selection for geriatric involvement [72, 73], the intervention can offer a low-threshold service that highlights needs and might improve patient selection.

The implementation of geriatric health research knowledge into clinical practice is still insufficient, which has been described as the `know-do’ gap [74]. There are many reasons for this lack of implementation, including the multitude of specialized clinical disciplines treating geriatric patients and a lack of knowledge about geriatric medicine among health professionals in non-geriatric departments [75, 76]. The SAA can help to fill this `know-do’ gap by providing treatment teams with individualized geriatric information on real patient cases.

The SURGE-Ahead intervention does not attempt to replace geriatricians, but it should help to streamline provision of geriatric expertise in non-geriatric settings without initial involvement of a consultant geriatrician. Thereby, it should help to address upcoming shortages in geriatric capacities due to demographic changes [11].

All publications related to the SURGE-Ahead Project will be submitted to peer-reviewed national and international journals after approval from the project publication committee. Authorship eligibility criteria are set up by the publication committee and comply with International Committee of Medical Journal Editors (ICMJE) statements [77].

To conclude, SURGE-Ahead aims to sustainably implement a standardized, digital geriatric clinical decision support system in surgical departments to improve co-management and continuity of care of older patients admitted to these departments, especially in those settings where geriatric expertise is not easily available.

Supporting information

S1 File. Spirit checklist SURGE-Ahead study protocol.

https://doi.org/10.1371/journal.pone.0287230.s001

(PDF)

S2 File. Funding confirmation SURGE-Ahead.

https://doi.org/10.1371/journal.pone.0287230.s002

(PDF)

S3 File. Ethics committee approval SURGE-Ahead German original.

https://doi.org/10.1371/journal.pone.0287230.s003

(PDF)

S4 File. Ethics committee approval SURGE Ahead English translation.

https://doi.org/10.1371/journal.pone.0287230.s004

(PDF)

S5 File. Approved study protocol SURGE-Ahead German original.

https://doi.org/10.1371/journal.pone.0287230.s005

(PDF)

S6 File. Approved study protocol SURGE-Ahead English translation.

https://doi.org/10.1371/journal.pone.0287230.s006

(PDF)

Acknowledgments

The SURGE-Ahead Study Group comprises Anna Ajlani, Karen Andersen-Ranberg, Sophia Andres, Christian Bolenz, Raffael Cintean, Dhayana Dallmeier, Miriam Deniz, Michael Denkinger (Principal Investigator), Colette Dörr-Harim, Anna Lena Flagmeier, Marina Fotteler, Florian Gebhard, Janina Hahn, Daniel Hieber, Thomas Hoffmann, Wolfgang Janni, Hans Kestler, Reinhold Kilian, Thomas Kocar, Christoph Leinert, Elena Leinert, Fabia Mangold, Jörg Meerpohl, Christoph Michalski, André Mihaljevic, Annabel Müller-Stierlin, Nadir Nasir, Graziano Onder, Desmond O’Neill, Marcin Orzechowski, Carlos Pankratz, Johannes Schobel, Fabienne Schochter, Konrad Schütze, Sabine Schütze, Tobias Skuban-Eiseler, Vytautas Stasiunaitis, Florian Steger, Walter Swoboda, Adriane Uihlein, Sebastian Voigt-Radloff, Martin Wehling, Pascal Wendel, Felix Wezel, Dennis Wolf, Friedemann Zengerling, and Marco Zeptner.

References

  1. 1. Van Grootven B, Flamaing J, Dierckx de Casterlé B, Dubois C, Fagard K, Herregods M-C, et al. Effectiveness of in-hospital geriatric co-management: a systematic review and meta-analysis. Age Ageing. 2017;46: 903–910. pmid:28444116
  2. 2. Shahrokni A, Tin AL, Sarraf S, Alexander K, Sun S, Kim SJ, et al. Association of Geriatric Comanagement and 90-Day Postoperative Mortality Among Patients Aged 75 Years and Older With Cancer. JAMA Netw Open. 2020;3: e209265. pmid:32822490
  3. 3. Grigoryan KV, Javedan H, Rudolph JL. Ortho-Geriatric Care Models and Outcomes in Hip Fracture Patients: A Systematic Review and Meta-Analysis. J Orthop Trauma. 2014;28: e49–e55. pmid:23912859
  4. 4. Rapp K, Becker C, Todd C, Rothenbacher D, Schulz C, König H-H, et al. The Association Between Orthogeriatric Co-Management and Mortality Following Hip Fracture. Dtsch Arzteblatt Int. 2020;117: 53–59. pmid:32036854
  5. 5. Pablos-Hernández C, González-Ramírez A, da Casa C, Luis MM, García-Iglesias MA, Julián-Enriquez JM, et al. Time to Surgery Reduction in Hip Fracture Patients on an Integrated Orthogeriatric Unit: A Comparative Study of Three Healthcare Models. Orthop Surg. 2020;12: 457–462. pmid:32167674
  6. 6. Middleton M, Wan B, da Assunçao R. Improving hip fracture outcomes with integrated orthogeriatric care: a comparison between two accepted orthogeriatric models. Age Ageing. 2017;46: 465–470. pmid:27974304
  7. 7. Kmietowicz Z. Emergency laparotomy: lack of geriatrician input leaves frail patients at double risk of death. BMJ. 2020;371: m4437. pmid:33187959
  8. 8. Hempenius L, Slaets JPJ, van Asselt D, de Bock TH, Wiggers T, van Leeuwen BL. Long Term Outcomes of a Geriatric Liaison Intervention in Frail Elderly Cancer Patients. PloS One. 2016;11: e0143364. pmid:26901417
  9. 9. Braude P, Goodman A, Elias T, Babic-Illman G, Challacombe B, Harari D, et al. Evaluation and establishment of a ward-based geriatric liaison service for older urological surgical patients: Proactive care of Older People undergoing Surgery (POPS)-Urology. BJU Int. 2017;120: 123–129. pmid:27167854
  10. 10. Filippova OT, Chi DS, Long Roche K, Sonoda Y, Zivanovic O, Gardner GJ, et al. Geriatric co-management leads to safely performed cytoreductive surgery in older women with advanced stage ovarian cancer treated at a tertiary care cancer center. Gynecol Oncol. 2019;154: 77–82. pmid:31078241
  11. 11. Lester PE, Dharmarajan TS, Weinstein E. The Looming Geriatrician Shortage: Ramifications and Solutions. J Aging Health. 2020;32: 1052–1062. pmid:31583940
  12. 12. Pitkälä KH, Martin FC, Maggi S, Jyväkorpi SK, Strandberg TE. Status of Geriatrics in 22 Countries. J Nutr Health Aging. 2018;22: 627–631. pmid:29717764
  13. 13. Eamer G, Taheri A, Chen SS, Daviduck Q, Chambers T, Shi X, et al. Comprehensive geriatric assessment for older people admitted to a surgical service. Cochrane Database Syst Rev. 2018;1: CD012485. pmid:29385235
  14. 14. Ellis G, Gardner M, Tsiachristas A, Langhorne P, Burke O, Harwood RH, et al. Comprehensive geriatric assessment for older adults admitted to hospital. Cochrane Database Syst Rev. 2017;9: CD006211. pmid:28898390
  15. 15. Gonçalves-Bradley DC, Lannin NA, Clemson L, Cameron ID, Shepperd S. Discharge planning from hospital. Cochrane Database Syst Rev. 2022;2: CD000313. pmid:35199849
  16. 16. Lafont C, Gérard S, Voisin T, Pahor M, Vellas B. Reducing “iatrogenic disability” in the hospitalized frail elderly. J Nutr Health Aging. 2011;15: 645–660. pmid:21968859
  17. 17. Ärzteblatt DÄG Redaktion Deutsches. Geriatrische Versorgung: Der Bedarf wird steigen. In: Deutsches Ärzteblatt [Internet]. 8 Oct 2021 [cited 17 Feb 2022]. Available: https://www.aerzteblatt.de/archiv/221425/Geriatrische-Versorgung-Der-Bedarf-wird-steigen
  18. 18. Schoeneberg C, Aigner R, Pass B, Volland R, Eschbach D, Peiris SE, et al. Effect of time-to-surgery on in-house mortality during orthogeriatric treatment following hip fracture: A retrospective analysis of prospectively collected data from 16,236 patients of the AltersTraumaRegister DGU®. Injury. 2021;52: 554–561. pmid:32951920
  19. 19. Köster I, Ihle P, Schubert I. Geriatrische Versorgungsstrukturen in Baden-Württemberg—Zusammenfassung des Abschlussberichts. AOK, IMVR, Universität Freiburg; 2019 p. 32. Available: https://aok-bw-presse.de/fileadmin/mediathek/dokumente/Geriatrie-BaW%C3%BC_Bericht-Zusammenfassung-2018-fin.pdf
  20. 20. Gogol M. Struktur der geriatrischen Versorgung. In: Lehnert H, editor. DGIM Innere Medizin: herausgegeben von Hendrik Lehnert. Berlin, Heidelberg: Springer; 2018. pp. 1–18. https://doi.org/10.1007/978-3-642-54676-1_444–1
  21. 21. Richtlinie zur Versorgung der hüftgelenknahen Femurfraktur: Erstfassung—Gemeinsamer Bundesausschuss. [cited 17 Feb 2022]. Available: https://www.g-ba.de/beschluesse/4069/
  22. 22. Mehl G, Tamrat T, Labrique A, Orton M, Baker E, Blaschke S, et al. Classification of Digital Health Interventions v 1.0. 2018.
  23. 23. Mellone S, Tacconi C, Chiari L. Validity of a Smartphone-based instrumented Timed Up and Go. Gait Posture. 2012;36: 163–165. pmid:22421189
  24. 24. Jansen C-P, Klenk J, Nerz C, Todd C, Labudek S, Kramer-Gmeiner F, et al. Association between everyday walking activity, objective and perceived risk of falling in older adults. Age Ageing. 2021;50: 1586–1592. pmid:33710267
  25. 25. Alex D, Fauzi AB, Mohan D. Online Multi-Domain Geriatric Health Screening in Urban Community Dwelling Older Malaysians: A Pilot Study. Front Public Health. 2020;8: 612154. pmid:33520920
  26. 26. Gilbert T, Neuburger J, Kraindler J, Keeble E, Smith P, Ariti C, et al. Development and validation of a Hospital Frailty Risk Score focusing on older people in acute care settings using electronic hospital records: an observational study. Lancet Lond Engl. 2018;391: 1775–1782. pmid:29706364
  27. 27. Shahrokni A, Alexander K. What will perioperative geriatric assessment for older cancer patients look like in 2025? Advantages and limitations of new technologies in geriatric assessment. Eur J Surg Oncol J Eur Soc Surg Oncol Br Assoc Surg Oncol. 2020;46: 305–309. pmid:31371102
  28. 28. Fallahzadeh R, Rokni SA, Ghasemzadeh H, Soto-Perez-de-Celis E, Shahrokni A. Digital Health for Geriatric Oncology. JCO Clin Cancer Inform. 2018; 1–12. pmid:30652581
  29. 29. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: Following considerable development in the field since 2006, MRC and NIHR have jointly commissioned an update of this guidance to be published in 2019. Med Res Counc 2019 P 1–39. 2019. Available: https://mrc.ukri.org/documents/pdf/complex-interventions-guidance/
  30. 30. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, et al. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337: a1655. pmid:18824488
  31. 31. Kocar T, Denkinger M, Dallmeier D, Fotteler M, Leinert C. PROSPERO Protocol: Evidence-based recommendations for acute orthogeriatric care: a systematic review of clinical practice guidelines. [cited 21 Feb 2022]. Available: https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=292141
  32. 32. Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Based Heal. 2015/07/03 ed. 2015;13: 141–6. pmid:26134548
  33. 33. Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth. 2020;18: 2119–2126. pmid:33038124
  34. 34. Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med. 2018;169: 467–473. pmid:30178033
  35. 35. Leinert C, Fotteler M, Kocar T, Dhayana Dallmeier, Denkinger M. Predictors and outcomes of interest of continuity of care decisions for older inpatients in acute care settings: a scoping review. 2022 [cited 23 Mar 2022]. Available: https://osf.io/zbv6y
  36. 36. Hshieh TT, Yang T, Gartaganis SL, Yue J, Inouye SK. Hospital Elder Life Program: Systematic Review and Meta-analysis of Effectiveness. Am J Geriatr Psychiatry Off J Am Assoc Geriatr Psychiatry. 2018;26: 1015–1033. pmid:30076080
  37. 37. The American Geriatrics Society Expert Panel on Postoperative Delirium in Older Adults. American Geriatrics Society Abstracted Clinical Practice Guideline for Postoperative Delirium in Older Adults. J Am Geriatr Soc. 2015;63: 142–150. pmid:25495432
  38. 38. Siddiqi N, Harrison JK, Clegg A, Teale EA, Young J, Taylor J, et al. Interventions for preventing delirium in hospitalised non-ICU patients. Cochrane Database Syst Rev. 2016;3: CD005563. pmid:26967259
  39. 39. Rabenberg A, Schulte T, Hildebrandt H, Wehling M. The FORTA (Fit fOR The Aged)-EPI (Epidemiological) Algorithm: Application of an Information Technology Tool for the Epidemiological Assessment of Drug Treatment in Older People. Drugs Aging. 2019;36: 969–978. pmid:31435913
  40. 40. Chakraborty M, Kundan AP. Grafana. In: Chakraborty M, Kundan AP, editors. Monitoring Cloud-Native Applications: Lead Agile Operations Confidently Using Open Source Software. Berkeley, CA: Apress; 2021. pp. 187–240. https://doi.org/10.1007/978-1-4842-6888-9_6
  41. 41. World Health Organization. Ethics and governance of artificial intelligence for health: WHO guidance. World Health Organization; 2021. Available: https://apps.who.int/iris/handle/10665/341996
  42. 42. Bezanson J, Edelman A, Karpinski S, Shah VB. Julia: A Fresh Approach to Numerical Computing. SIAM Rev. 2017;59: 65–98.
  43. 43. Wang D, Yang Q, Abdul A, Lim BY. Designing Theory-Driven User-Centric Explainable AI. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. New York, NY, USA: Association for Computing Machinery; 2019. pp. 1–15. https://doi.org/10.1145/3290605.3300831
  44. 44. Ethics guidelines for trustworthy AI | Shaping Europe’s digital future. [cited 20 Oct 2022]. Available: https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai
  45. 45. Bleijenberg N, de Man-van Ginkel JM, Trappenburg JCA, Ettema RGA, Sino CG, Heim N, et al. Increasing value and reducing waste by optimizing the development of complex interventions: Enriching the development phase of the Medical Research Council (MRC) Framework. Int J Nurs Stud. 2018;79: 86–93. pmid:29220738
  46. 46. Object Management Group. Business Process Model and Notation. In: Object Management Group Business Process Model and Notation [Internet]. 2021 [cited 11 Mar 2022]. Available: https://www.bpmn.org/
  47. 47. Ammenwerth E, Hackl WO. IT-Assisted Process Management in Healthcare. Stud Health Technol Inf. 2020;274: 206–216. pmid:32990675
  48. 48. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82: 581–629. pmid:15595944
  49. 49. Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 1989;13: 319–340.
  50. 50. Kothgassner OD, Felnhofer A, Hauk N, Kastenhofer E, Gomm J, Kryspin-Exner I. Technology Usage Inventory (TUI): Manual. 2013.
  51. 51. McCusker J, Bellavance F, Cardin S, Trépanier S, Verdon J, Ardman O. Detection of older people at increased risk of adverse health outcomes after an emergency visit: the ISAR screening tool. J Am Geriatr Soc. 1999;47: 1229–1237. pmid:10522957
  52. 52. Cover TM. Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition. IEEE Trans Electron Comput. 1965;3: 326–334.
  53. 53. Savvopoulos S, Sampalli T, Harding R, Blackmore G, Janes S, Kumanan K, et al. Development of a quality scoring tool to assess quality of discharge summaries. J Fam Med Prim Care. 2018;7: 394–400. pmid:30090783
  54. 54. Mahoney FI, Barthel DW. Functional evaluation: The Barthel Index: A simple index of independence useful in scoring improvement in the rehabilitation of the chronically ill. Md State Med J. 1965;14: 61–65.
  55. 55. Parker MJ, Palmer CR. A new mobility score for predicting mortality after hip fracture. J Bone Joint Surg Br. 1993;75: 797–798. pmid:8376443
  56. 56. Liebl ME, Elmer N, Schroeder I, Schwedtke C, Baack A, Reisshauer A. Introduction of the Charité Mobility Index (CHARMI)–A Novel Clinical Mobility Assessment for Acute Care Rehabilitation. PLoS ONE. 2016;11: e0169010. pmid:28006023
  57. 57. Bellelli G, Morandi A, Davis DHJ, Mazzola P, Turco R, Gentile S, et al. Validation of the 4AT, a new instrument for rapid delirium screening: a study in 234 hospitalised older people. Age Ageing. 2014;43: 496–502. pmid:24590568
  58. 58. Wong A, Nyenhuis D, Black SE, Law LSN, Lo ESK, Kwan PWL, et al. Montreal Cognitive Assessment 5-Minute Protocol Is a Brief, Valid, Reliable, and Feasible Cognitive Screen for Telephone Administration. Stroke. 2015;46: 1059–1064. pmid:25700290
  59. 59. Roick C, Kilian R, Matschinger H, Bernert S, Mory C, Angermeyer MC. Die deutsche Version des Client Sociodemographic and Service Receipt Inventory. Psychiatr Prax. 2001;28: 84–90. pmid:11605129
  60. 60. Rockwood K, Song X, MacKnight C, Bergman H, Hogan DB, McDowell I. A global clinical measure of fitness and frailty in elderly people. CMAJ. 2005;173. pmid:16129869
  61. 61. Herdman M, Gudex C, Lloyd A, Janssen M, Kind P, Parkin D, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res Int J Qual Life Asp Treat Care Rehabil. 2011;20: 1727–1736. pmid:21479777
  62. 62. IHI Global Trigger Tool for Measuring Adverse Events (Second Edition) | IHI—Institute for Healthcare Improvement. [cited 1 Aug 2022]. Available: https://www.ihi.org/resources/Pages/IHIWhitePapers/IHIGlobalTriggerToolWhitePaper.aspx?PostAuthRed=/resources/_layouts/download.aspx?SourceURL=/resources/Knowledge%20Center%20Assets/IHIWhitePapers%20-%20IHIGlobalTriggerToolforMeasuringAdverseEventsSecondEdition_ab736cac-935b-46e4-b6fe-7f98c45f8a78/IHIGlobalTriggerToolWhitePaper2009.pdf
  63. 63. Kondrup J, Rasmussen HH, Hamberg O, Stanga Z, Ad Hoc ESPEN Working Group. Nutritional risk screening (NRS 2002): a new method based on an analysis of controlled clinical trials. Clin Nutr Edinb Scotl. 2003;22: 321–336. pmid:12765673
  64. 64. Glick HA, Doshi JA, Sonnad SS, Polsky D, Glick HA, Doshi JA, et al. Economic Evaluation in Clinical Trials. Second Edition, Second Edition. Oxford, New York: Oxford University Press; 2014.
  65. 65. Willan AR, Briggs AH. Statistical Analysis of Cost-Effectiveness Data | Wiley. In: Wiley.com [Internet]. [cited 28 Nov 2022]. Available: https://www.wiley.com/en-us/Statistical+Analysis+of+Cost+Effectiveness+Data-p-9780470856260
  66. 66. Salize HJ, Kilian R. Gesundheitsökonomie in der Psychiatrie: Konzepte, Methoden, Analysen. Kohlhammer Verlag; 2010.
  67. 67. Schwandt A, Bergis D, Denkinger M, Gollisch KSC, Sandig D, Stingl H, et al. Risk factors for decline in renal function among young adults with type 1 diabetes. J Diabetes Complications. 2018;32: 940–946. pmid:30055905
  68. 68. Janssen MF, Pickard AS, Golicki D, Gudex C, Niewada M, Scalone L, et al. Measurement properties of the EQ-5D-5L compared to the EQ-5D-3L across eight patient groups: a multi-country study. Qual Life Res Int J Qual Life Asp Treat Care Rehabil. 2013;22: 1717–1727. pmid:23184421
  69. 69. Ludwig K, von der Schulenburg JMG, Greiner W. Valuation of the EQ-5D-5L with composite time trade-off for the German population—an exploratory study. Health Qual Life Outcomes. 2017;15: 39. pmid:28219389
  70. 70. Ursin F, Timmermann C, Orzechowski M, Steger F. Diagnosing Diabetic Retinopathy With Artificial Intelligence: What Information Should Be Included to Ensure Ethical Informed Consent? Front Med. 2021;8. Available: https://www.frontiersin.org/articles/10.3389/fmed.2021.695217 pmid:34368192
  71. 71. Ursin F, Timmermann C, Steger F. Explicability of artificial intelligence in radiology: Is a fifth bioethical principle conceptually necessary? Bioethics. 2022;36: 143–153. pmid:34251687
  72. 72. Gurlit S, Möllmann H. [Preoperative risk identification using the Identification of Seniors at Risk?: Suitability as sole screening tool for inpatient aged risk patients]. Z Gerontol Geriatr. 2018;51: 388–393. pmid:29796867
  73. 73. Singler K, Heppner HJ, Skutetzky A, Sieber C, Christ M, Thiem U. Predictive validity of the identification of seniors at risk screening tool in a German emergency department setting. Gerontology. 2014;60: 413–419. pmid:24969966
  74. 74. Gladman JRF, Conroy SP, Ranhoff AH, Gordon AL. New horizons in the implementation and research of comprehensive geriatric assessment: knowing, doing and the ‘know-do’ gap. Age Ageing. 2016;45: 194–200. pmid:26941353
  75. 75. Roethler C, Adelman T, Parsons V. Assessing emergency nurses’ geriatric knowledge and perceptions of their geriatric care. J Emerg Nurs. 2011;37: 132–137. pmid:21397125
  76. 76. Kotsani M, Kravvariti E, Avgerinou C, Panagiotakis S, Bograkou Tzanetakou K, Antoniadou E, et al. The Relevance and Added Value of Geriatric Medicine (GM): Introducing GM to Non-Geriatricians. J Clin Med. 2021;10: 3018. pmid:34300184
  77. 77. International Committee of Medical Journal Editors. Defining the role of authors and contributors. http://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html. Accessed 1 May 2018. Available: http://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html