Development of a framework for the implementation of synchronous e-mental health: a protocol for a realist synthesis of systematic reviews

David Villarreal-Zegarra; Christoper A. Alarcon-Ruiz; G.J. Melendez-Torres; Roberto Torres-Puente; Juan Ambrosio-Melgarejo; Alejandra B. Romero-Cabrera; Alba Navarro-Flores; Leonardo Albitres-Flores; Ana Lindo-Cavero; Jeff Huarcaya-Victoria

doi:10.12688/f1000research.27150.2

Home Browse Development of a framework for the implementation of synchronous e-mental...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Study Protocol

Revised

Development of a framework for the implementation of synchronous e-mental health: a protocol for a realist synthesis of systematic reviews

[version 2; peer review: 2 approved]

Previously titled: Development of a framework for the implementation of electronic interventions in mental health: a protocol for a meta-synthesis of systematic reviews

David Villarreal-Zegarra^1,2, Christoper A. Alarcon-Ruiz ³, G.J. Melendez-Torres⁴, [...] Roberto Torres-Puente², Juan Ambrosio-Melgarejo², Alejandra B. Romero-Cabrera³, Alba Navarro-Flores^2,5, Leonardo Albitres-Flores^6,7, Ana Lindo-Cavero³, Jeff Huarcaya-Victoria^8,9

David Villarreal-Zegarra^1,2, Christoper A. Alarcon-Ruiz ³, [...] G.J. Melendez-Torres⁴, Roberto Torres-Puente², Juan Ambrosio-Melgarejo², Alejandra B. Romero-Cabrera³, Alba Navarro-Flores^2,5, Leonardo Albitres-Flores^6,7, Ana Lindo-Cavero³, Jeff Huarcaya-Victoria^8,9

PUBLISHED 03 Aug 2021

Author details Author details

¹ Instituto Peruano de Orientación Psicológica, Lima, Peru
² Universidad César Vallejo, Escuela de Medicina, Trujillo, Peru
³ Universidad Científica del Sur, Lima, Peru
⁴ Peninsula Technology Assessment Group, College of Medicine and Health, University of Exeter, Devon, UK
⁵ Facultad de Medicina, Universidad Nacional Federico Villarreal, Lima, Peru
⁶ Facultad de Medicina, Universidad Nacional de Trujillo, Trujillo, Peru
⁷ CRONICAS Center of Excellence in Chronic Diseases, Universidad Peruana Cayetano Heredia, Lima, Peru
⁸ Centro de Investigación en Salud Pública, Facultad de Medicina, Universidad de San Martín de Porres, Lima, Peru
⁹ Departamento de Psiquiatría, Hospital Nacional Guillermo Almenara Irigoyen, Lima, Peru

David Villarreal-Zegarra
Roles: Conceptualization, Funding Acquisition, Investigation, Methodology, Project Administration, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Christoper A. Alarcon-Ruiz
Roles: Conceptualization, Funding Acquisition, Investigation, Methodology, Project Administration, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

G.J. Melendez-Torres
Roles: Conceptualization, Investigation, Methodology, Supervision, Writing – Review & Editing

Roberto Torres-Puente
Roles: Investigation, Writing – Original Draft Preparation, Writing – Review & Editing

Juan Ambrosio-Melgarejo
Roles: Investigation, Writing – Original Draft Preparation, Writing – Review & Editing

Alejandra B. Romero-Cabrera
Roles: Investigation, Writing – Review & Editing

Alba Navarro-Flores
Roles: Investigation, Writing – Review & Editing

Leonardo Albitres-Flores
Roles: Investigation, Writing – Review & Editing

Ana Lindo-Cavero
Roles: Investigation, Writing – Review & Editing

Jeff Huarcaya-Victoria
Roles: Conceptualization, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

This article is included in the Health Services gateway.

Abstract

Background: During the COVID-19 pandemic, it has been necessary to deliver mental health care using technologies (e-mental health). But there have been difficulties in its application. Quantitative systematic reviews such as meta-analysis doesn’t allow us to fully identify and properly describe this subject. Thus, our study has two main objectives: a) "to determine what evidence is available for synchronous e-mental health implementation"; and b) "to develop a framework informed by a realist analysis for the implementation of synchronous e-mental health".
Methods: We will search MEDLINE, EBM Reviews, PsycINFO, EMBASE, SCOPUS, CINAHL Complete, and Web of Science databases from 1st January 2015 to September 2020, with no language restriction. A systematic review with a narrative description and a realist synthesis will be conducted. Primary studies relating to adults with common mental health problems using any type of mobile mental health intervention that includes a synchronic component and communication with a mental health professional will be included. For the analysis, we will make a realist synthesis of the systematic reviews, using a grounded theory approach with an emergent approach to synthesize the information, prioritizing the systematic reviews with a lower risk of bias in the AMSTAR-2 tool. The realist synthesis will be based on the interpretation, integration, and inference of the evaluated elements and the generation of hypotheses to better understand the implementation process of synchronous e-mental health. Finally, we will present the overall assessment in a Summary of Qualitative Findings table.
Conclusion: Our results will allow a better understanding of the facilitator and limitations in implementing e-mental health.

Keywords

Telemedicine, Remote Consultation, Online Systems, Internet-Based Intervention, Mental Health, Mental Disorders, Systematic Reviews as Topic, Qualitative Research

Corresponding author: Christoper A. Alarcon-Ruiz

Competing interests: No competing interests were disclosed.

Grant information: Jeff Huarcaya-Victoria received funding for this study from EsSalud (Premio Kaelin 2020).
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2021 Villarreal-Zegarra D et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Villarreal-Zegarra D, Alarcon-Ruiz CA, Melendez-Torres GJ et al. Development of a framework for the implementation of synchronous e-mental health: a protocol for a realist synthesis of systematic reviews [version 2; peer review: 2 approved]. F1000Research 2021, 9:1282 (https://doi.org/10.12688/f1000research.27150.2) First published: 29 Oct 2020, 9:1282 (https://doi.org/10.12688/f1000research.27150.1) Latest published: 03 Aug 2021, 9:1282 (https://doi.org/10.12688/f1000research.27150.2)

Revised Amendments from Version 1

This new version of the protocol has changed its title, detailing its analysis "a realist synthesis of systematic reviews". Also, we updated several sections of the protocol to describe better the research question and specifies the realist synthesis using a grounded theory approach with an emergent approach to synthesize the information, prioritizing the systematic reviews with a low risk of bias. Additionally, we updated the reference section according to this new version. Finally, we have reviewed and corrected typographical errors and the adequate use of English.

See the authors' detailed response to the review by Roger Hilfiker
See the authors' detailed response to the review by Amy Grove

Introduction

In the wake of the COVID-19 pandemic and after the actions taken by governments (such as social isolation), many mental health problems have increased in patients with COVID-19, patients with psychiatric symptoms, health personnel, and the general population^1,2. As a result, greater interest has been taken in addressing mental health issues during the pandemic^2,3. Studies have identified anxiety disorders, depression, post-traumatic stress disorder, and stress as the most frequent health problems^2–4. As a result, it has been necessary to deliver mental health care through technology (e-mental health) such as internet and mobile devices^5,6 These kind of health care have been very well received and have served to complement or improve the effectiveness of treatments for various chronic diseases, reducing gaps in access to care and providing specialized care in inaccessible places⁷ and it shows great promise in the care of mental health problems^8–10, highlighting the possibilities of care in health systems where resources are limited¹¹.

With the undeniable contribution of e-mental health, it has been important to document the aspects related to its application because, despite its effectiveness, it is known that there have been difficulties in its application¹². Aspects such as adaptability, cost, complexity, external policies and incentives, compatibility or general fit between the e-mental health intervention and the organization¹³ are important to consider to be clear about how and what works in synchronous and electronic interventions in mental health (synchronous e-mental health), and considering its complexity as well. To address this, systematic reviews and meta-analyses have given us knowledge about the effectiveness of e-mental health^14,15. However, qualitative studies are necessary because of their methodology (approach to answer questions about experience, meaning, and perspective) which is useful to address issues of implementation¹⁶. Thus, our study has two main objectives: a) "to determine what evidence is available for synchronous e-mental health implementation"; and b) "to develop a framework informed by a realist analysis for the implementation of synchronous e-mental health".

Methods

Research question

This protocol adheres to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols (PRISMA-P) guidelines¹⁷ A completed PRISMA-P checklist can be found in the Reporting guidelines section¹⁸. The SPIDER framework was used to develop the review question, which is based on describing the Sample (S), Phenomenon of Interest (PI), Design (D), Evaluation (E), and Research type (R)¹⁶:

• Sample: Adults with depression (or major depressive disorder), anxiety (or generalized anxiety disorder), stress (or trauma-related disorders), and/or general mental health problems (unspecified). Participants may be diagnosed through clinical interviews or categorized based on screening assessments (self-reported scales).
• Phenomenon of interest: Any type of e-mental health intervention that includes a synchronous component, communication with a mental health professional (psychiatrist, psychologist, etc.) or a health professional trained in mental health. These interventions include, among others, remote consultation, interactive application, video chats, calls, etc.
• Design: Systematic review.
• Evaluation: All types of outcomes of interest assessed by implementation, economic, qualitative, quantitative, and other studies will be included. For example, a) Health effectiveness outcomes: Depression, anxiety and/or stress symptoms, adherence to treatment, etc; b) Patient outcomes: Quality of life, satisfaction, etc; c) Economic outcomes; d) Damage or adverse effects. Our study seeks to perform a realist synthesis of the evidence, so we focused on different outcomes to use them as input that allows us to evaluate the implementation of synchronous e-mental health. Therefore, we do not perform a quantitative synthesis in any case (i.e., a meta-analysis of effectiveness).
• Research type: Quantitative studies, qualitative studies, mixed methods.

Eligibility criteria

Inclusion criteria: Systematic reviews using at least two databases that synthesized, assessed the quality, and presented sufficient detail of the primary included studies; and reports on inclusion/exclusion criteria¹⁹. Systematic reviews that included primary studies as a unit of analysis focused on a research question. Systematic reviews published in the last five years (since January 1, 2015) without language restrictions. We include this time frame to include only the latest systematic reviews, since in the field of e-health the launch of new technologies makes scientific development dynamic. Reviews must include primary studies relating to adults with common mental health problems: a) Adults with depression (or major depressive disorder), anxiety (or generalized anxiety disorder), stress (or trauma-related disorders) and/or general mental health problems (unspecified); b) Adults attending an outpatient mental health consultation. Reviews must include at least 90% of their primary studies that assess synchronous e-mental health or presenting its results independently only for synchronous e-mental health.

Exclusion criteria: Narrative reviews, scoping reviews, primary studies, opinion/editorial manuscripts, letters to the editor, and reviews of mobile health interventions repositories (i.e., apps stores). Studies in which some of these subjects has participated will also be excluded: a) Adults with some other mental health problem; b) Healthy adults without mental health problems; c) Adults receiving emergency/crisis psychiatric care; d) Interventions that lack a synchronous component (real time information exchange between the user and mental health professional using technologies) or are not sufficiently clear that had a synchronous component.

Information sources

Included databases will be MEDLINE, EBM Reviews, PsycINFO (these three through Ovid), EMBASE (Elsevier), SCOPUS, CINAHL Complete (EBSCOhost), and Web of Science databases, including Science Citation Index Expanded, Social Sciences Citation Index and Conference Proceedings Citation Index (Clarivate Analytics). Articles published in the last five years (after January 1st, 2015) will be included and no language restrictions will be imposed. We will include systematic reviews that report a review of literature using a research strategy in at least two different databases. Later, all references of included studies will be reviewed, and they will be evaluated looking for any additional systematic review that meets the inclusion criteria.

Search strategy

Main search terms to be used are "telemedicine" AND "mental health, anxiety, depression or stress" AND “systematic reviews”. The full search strategy for each database is available in the Extended data section (https://osf.io/c4jua/)¹⁸.

Study records

Data management: The results of the database search will be managed using the Rayyan QCRI free online application to manage the references (eliminate duplicates, and review titles and summaries)²⁰. Full-text review and data extraction will be done using an Excel template (https://osf.io/c4jua/¹⁸).

Selection process: There will be two distinct selection stages. First, a review of the title and abstract and the second a review of the full text. The reviews will be carried out based on pairs of reviewers, who will divide the total number of records. At each stage, there will be a first part where the reviewers will calibrate the accuracy of their reviews and a second part where the actual review will take place. To complete the calibration part, they will make calibration rounds until the discrepancy is less than or equal to 5% of the assigned records. If the percentage of the discrepancy is exceeded, a new round will be performed. Subsequently pairs of reviewers will divide the total of the records identified, and records will be screened independently and in duplicate. Discrepancies will be discussed within pairs with a third reviewer if needed.

In the review by title and abstract, calibration will be done first, 50 records were selected from the total number of records found because of the search strategy. In the full-text review, calibration will first be performed on 10 records that will be selected in the previous stage. The articles excluded in the full-text review stage will be listed and a reason for exclusion will be given for each of them.

Data collection process: For each eligible study, data will be extracted independently and in duplicate on pre-designed data extraction forms. In the event of discrepancies, the reviewers will discuss whether the extracted data should be retained and decide to resolve the discrepancy. If the discrepancy remains, a third reviewer will evaluate any unresolved disagreement between the reviewers on the data extraction and will decide on the value extracted that is in dispute.

Data items

An extraction form was created in an Excel template for the included systematic reviews (https://osf.io/7esgz/). Information will be collected on the author and date of the study, characteristics of the participants, main objective, research questions, inclusion criteria for the systematic review, search date, study selection process, quality assessment (if any), main findings and limitations. Also, the full text of the included article will be extracted along with the tables and supplementary material, to perform the qualitative analysis of the text.

Outcomes and prioritization

Our study aims to develop a framework informed by a realist analysis for the implementation of synchronous e-mental health, using a qualitative strategy to synthesize the information and answer our research question. Therefore, we do not look for specific results such as effectiveness, cost-effectiveness, or other similar ones. Instead, we are interested in identifying the full text of all the studies that answer our research question, so that we can make a realistic synthesis using a grounded theory model with an emergent approach. The develop a framework informed by a realist analysis will be based on the interpretation, integration, and inference of the evaluated elements and the generation of hypotheses to better understand the studied phenomenon (implementation of synchronous e-mental health)^21,22. Priority will be given in the analysis of those studies with the lowest risk of bias assessed. If possible, a realist synthesis will be performed only with those systematic reviews with high quality, to assess whether the results change by selecting only those studies with high quality (low risk of bias).

Risk of bias in individual studies

To assess the quality of the included systematic reviews we will use the "A Measurement Tool to Assess Systematic Reviews-2" (AMSTAR 2), which has sixteen domains. Seven of these domains are considered critical: 1) protocol registered before the start of the review, 2) adequacy of the literature search, 3) justification for the exclusion of individual studies, 4) risk of bias of individual studies included in the review, 5) adequacy of meta-analytic methods, 6) consideration of the risk of bias in interpreting the results of the review, and 7) assessment of the presence and likely impact of publication bias²³.

AMSTAR-2 classifies the quality of systematic reviews into four different categories: high (none or one non-critical weakness), moderate (more than one non-critical weakness), low (one critical weakness with or without non-critical weaknesses), and very low (more than one critical weakness with or without non-critical weaknesses). The quality assessment of each systematic review will be rated by two researchers independently. In case of difference in the overall quality of the systematic reviews, the AMSTAR-2 criteria will be discussed among the researchers to reach a consensus.

Data synthesis

The develop a framework informed by a realist analysis for the implementation of synchronous e-mental health will be carried out using a grounded theory approach with an emergent approach²⁴. The realist synthesis will be based on interpreting, integrating and, inferring the evaluation elements that would allow a better understanding of the synchronous e-mental health implementation process from all the studies included²⁴. To answer the question "what makes the implementation of these interventions work", hypotheses supported by the results of the included studies will be developed and generated through discussion and consensus among researchers²⁴. Two reviewers will follow the three steps established by Thomas and Harden for qualitative syntheses will be followed^25,26. First, the extracted data are freely coded. The reviewers read the full texts of the included articles and code each text fragment that provides information to answer the research question. Second, the codified data are organized and grouped based on the descriptive aspects. The reviewers will review the codes generated in the previous step and group them into codes that are like each other and that allow the description of a part of the research question. Thirdly, the analytical concepts generated in the previous step will be grouped in a way that they are related to each other. The elements that are related to each other will be assumed to be part of a hypothesis that would help to answer the research aim. The sum of all these hypotheses will be referred to as a framework which will attempt to explain the results using a context-linked causality approach represented as context + mechanism = outcome²⁶.

When included studies had been selected, they will be ranked based on the AMSTAR-2 score, with the highest quality studies being assessed first. We will assess all included studies, down to the criterion of theoretical saturation. All qualitative analyses will be performed with NVIVO v.1 software.

Confidence in cumulative evidence

It will be evaluated the approach of Confidence in the Evidence from Reviews of Qualitative research (CERQual) which has four components (Methodological Limitations, Relevance, Coherence and Adequacy data), will be evaluated to contribute to an overall assessment for each hypothesis resulting from the realist synthesis to determine the level of confidence (high, moderate, low, or very low) and present the overall assessment in a Summary of Qualitative Findings (SoQF) table^27,28.

Dissemination plan

We will present the systematic review findings and submit the manuscript to a relevant peer-reviewed journal.

Study status

This systematic review is currently in the data analysis process. The protocol of this systematic review was submitted to PROSPERO registry on 18^th August, 2020 (CRD42020203811).

Data availability

Underlying data

No data are associated with this article.

Extended data

Open Science Framework: PRISMA-P checklist for ‘Development of a framework for the implementation of electronic interventions in mental health: A protocol for a meta-synthesis of systematic reviews’, https://doi.org/10.17605/OSF.IO/3UH4N¹⁸. (Registered on 20^th October 2020: osf.io/tf4b6.)

This project contains the following extended data:

- Full text protocol
- Full search strategy for MEDLINE, EBM Reviews, PsycINFO, EMBASE, SCOPUS, CINAHL Complete, and Web of Science databases
- Excel template for data extraction

Reporting guidelines

Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

Acknowledgments

The authors acknowledge Jackeline Garcia and Juan Barrera, for helping with the initial phases of the information search.

Faculty Opinions recommended

References

1. Vindegaard N, Benros ME: COVID-19 pandemic and mental health consequences: Systematic review of the current evidence. Brain Behav Immun. 2020; 89: 531–542. PubMed Abstract | Publisher Full Text | Free Full Text
2. Talevi D, Socci V, Carai M, et al.: Mental health outcomes of the CoViD-19 pandemic. Riv Psichiatr. 2020; 55(3): 137–44. PubMed Abstract | Publisher Full Text
3. Huarcaya-Victoria J: Consideraciones sobre la salud mental en la pandemia de COVID-19. Rev Peru Med Exp Salud Publica. 2020; 37(2): 327–34. PubMed Abstract | Publisher Full Text
4. Xiong J, Lipsitz O, Nasri F, et al.: Impact of COVID-19 pandemic on mental health in the general population: A systematic review. J Affect Disord. 2020; 277: 55–64. PubMed Abstract | Publisher Full Text | Free Full Text
5. Sullivan AB, Kane A, Roth AJ, et al.: The COVID-19 Crisis: A Mental Health Perspective and Response Using Telemedicine. J Patient Exp. 2020; 7(3): 295–301. PubMed Abstract | Publisher Full Text | Free Full Text
6. Haxhihamza K, Arsova S, Bajraktarov S, et al.: Patient Satisfaction with Use of Telemedicine in University Clinic of Psychiatry: Skopje, North Macedonia During COVID-19 Pandemic. Telemed J E Health. 2020. PubMed Abstract | Publisher Full Text
7. Ruiz EF, Proaño A, Ponce OJ, et al.: [Mobile health for public health in Peru: lessons learned]. Rev Peru Med Exp Salud Publica. 2015; 32(2): 364–72. PubMed Abstract
8. Miranda JJ, Moscoso MG, Toyama M, et al.: Role of mHealth in overcoming the occurrence of post-stroke depression. Acta Neurol Scand. 2018; 137(1): 12–9. PubMed Abstract | Publisher Full Text | Free Full Text
9. Andrews G, Cuijpers P, Craske MG, et al.: Computer Therapy for the Anxiety and Depressive Disorders Is Effective, Acceptable and Practical Health Care: A Meta-Analysis. PLoS One. 2010; 5(10): e13196. PubMed Abstract | Publisher Full Text | Free Full Text
10. Turvey C, Fortney J: The Use of Telemedicine and Mobile Technology to Promote Population Health and Population Management for Psychiatric Disorders. Curr Psychiatry Rep. 2017; 19(11): 88. PubMed Abstract | Publisher Full Text
11. Kaonga NN, Morgan J: Common themes and emerging trends for the use of technology to support mental health and psychosocial well-being in limited resource settings: A review of the literature. Psychiatry Res. 2019; 281: 112594. PubMed Abstract | Publisher Full Text
12. Meurk C, Leung J, Hall W, et al.: Establishing and Governing e-Mental Health Care in Australia: A Systematic Review of Challenges and A Call For Policy-Focussed Research. J Med Internet Res. 2016; 18(1): e10. PubMed Abstract | Publisher Full Text | Free Full Text
13. Ross J, Stevenson F, Lau R, et al.: Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update). Implement Sci. 2016; 11(1): 146. PubMed Abstract | Publisher Full Text | Free Full Text
14. Griffiths KM, Farrer L, Christensen H: The efficacy of internet interventions for depression and anxiety disorders: a review of randomised controlled trials. Med J Aust. 2010; 192(S11): S4–S11. PubMed Abstract
15. Senanayake B, Wickramasinghe SI, Chatfield MD, et al.: Effectiveness of text messaging interventions for the management of depression: A systematic review and meta-analysis. J Telemed Telecare. 2019; 25(9): 513–523. PubMed Abstract | Publisher Full Text
16. Cooke A, Smith D, Booth A: Beyond PICO: The SPIDER Tool for Qualitative Evidence Synthesis. Qual Health Res. 2012; 22(10): 1435–43. PubMed Abstract | Publisher Full Text
17. Moher D, Shamseer L, Clarke M, et al.: Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015; 4(1): 1. PubMed Abstract | Publisher Full Text | Free Full Text
18. Villarreal-Zegarra D, Alarcon-Ruiz CA, Melendez-Torres G, et al.: Development of a framework for the implementation of electronic interventions in mental health: A protocol for a meta-synthesis of systematic reviews. 2020. http://www.doi.org/10.17605/OSF.IO/3UH4N
19. Dissemination CfRa. University of York: Centre for Reviews and Dissemination. [updated 3 June, 2015]. Reference Source
20. Ouzzani M, Hammady H, Fedorowicz Z, et al.: Rayyan—a web and mobile app for systematic reviews. Syst Rev. 2016; 5(1): 210. PubMed Abstract | Publisher Full Text | Free Full Text
21. Husk K, Blockley K, Lovell R, et al.: What approaches to social prescribing work, for whom, and in what circumstances? A realist review. Health Soc Care Community. 2020; 28(2): 309–24. PubMed Abstract | Publisher Full Text | Free Full Text
22. Tan SY, Melendez-Torres GJ, Pang T: Implementation of provider payment system reforms in the age of universal health coverage: a realist review of evidence from Asian developing countries. J Health Serv Res Policy. 2019; 24(4): 279–87. PubMed Abstract | Publisher Full Text
23. Shea BJ, Reeves BC, Wells G, et al.: AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017; 358: j4008. PubMed Abstract | Publisher Full Text | Free Full Text
24. Guetterman TC, Babchuk WA, Howell Smith MC, et al.: Contemporary Approaches to Mixed Methods–Grounded Theory Research: A Field-Based Analysis. J Mix Methods Res. 2019; 13(2): 179–95. Publisher Full Text
25. Thorne S, Jensen L, Kearney MH, et al.: Qualitative metasynthesis: reflections on methodological orientation and ideological agenda. Qual Health Res. 2004; 14(10): 1342–65. PubMed Abstract | Publisher Full Text
26. Rycroft-Malone J, McCormack B, Hutchinson AM, et al.: Realist synthesis: illustrating the method for implementation research. Implement Sci. 2012; 7(1): 33. PubMed Abstract | Publisher Full Text | Free Full Text
27. Group TG-CP: A practical guide for decision makers in health and social care and other users. Reference Source
28. Lewin S, Booth A, Glenton C, et al.: Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series. Implement Sci. 2018; 13(Suppl 1): 2. PubMed Abstract | Publisher Full Text | Free Full Text

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 29 Oct 2020