Development of a framework for the implementation of electronic interventions in mental health: a protocol for a meta-synthesis of systematic reviews

David Villarreal-Zegarra; Christoper A. Alarcon-Ruiz; G.J. Melendez-Torres; Roberto Torres-Puente; Juan Ambrosio-Melgarejo; Alejandra B. Romero-Cabrera; Ana Lindo-Cavero; Jeff Huarcaya-Victoria

doi:10.12688/f1000research.27150.1

Home Browse Development of a framework for the implementation of electronic interventions...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Study Protocol

Development of a framework for the implementation of electronic interventions in mental health: a protocol for a meta-synthesis of systematic reviews

[version 1; peer review: 2 approved with reservations]

David Villarreal-Zegarra^1-3, Christoper A. Alarcon-Ruiz ⁴, G.J. Melendez-Torres⁵, [...] Roberto Torres-Puente¹, Juan Ambrosio-Melgarejo¹, Alejandra B. Romero-Cabrera⁴, Ana Lindo-Cavero⁴, Jeff Huarcaya-Victoria^6,7

David Villarreal-Zegarra^1-3, Christoper A. Alarcon-Ruiz ⁴, [...] G.J. Melendez-Torres⁵, Roberto Torres-Puente¹, Juan Ambrosio-Melgarejo¹, Alejandra B. Romero-Cabrera⁴, Ana Lindo-Cavero⁴, Jeff Huarcaya-Victoria^6,7

PUBLISHED 29 Oct 2020

Author details Author details

¹ CRONICAS Center of Excellence in Chronic Diseases, Universidad Peruana Cayetano Heredia, Lima, Peru
² Instituto de Investigación, Universidad Católica Los Ángeles de Chimbote, Lima, Peru
³ Instituto Peruano de Orientación Psicológica, Lima, Peru
⁴ Universidad Científica del Sur, Lima, Peru
⁵ Peninsula Technology Assessment Group, College of Medicine and Health, University of Exeter, Devon, UK
⁶ Centro de Investigación en Salud Pública, Facultad de Medicina, Universidad de San Martín de Porres, Lima, Peru
⁷ Departamento de Psiquiatría, Hospital Nacional Guillermo Almenara Irigoyen, Lima, Peru

David Villarreal-Zegarra
Roles: Conceptualization, Funding Acquisition, Investigation, Methodology, Project Administration, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Christoper A. Alarcon-Ruiz
Roles: Conceptualization, Funding Acquisition, Investigation, Methodology, Project Administration, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

G.J. Melendez-Torres
Roles: Conceptualization, Investigation, Methodology, Supervision, Writing – Review & Editing

Roberto Torres-Puente
Roles: Investigation, Writing – Original Draft Preparation, Writing – Review & Editing

Juan Ambrosio-Melgarejo
Roles: Investigation, Writing – Original Draft Preparation, Writing – Review & Editing

Alejandra B. Romero-Cabrera
Roles: Investigation, Writing – Review & Editing

Ana Lindo-Cavero
Roles: Investigation, Writing – Review & Editing

Jeff Huarcaya-Victoria
Roles: Conceptualization, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

This article is included in the Health Services gateway.

Abstract

Background: During the COVID-19 pandemic, it has been necessary to incorporate technologies in the care of mental health problems. But there have been difficulties in the application of technology-based interventions in mental health. Some quantitative systematic reviews don’t allow us to fully identify and properly describe this subject. In order to answer the question "how do electronic interventions apply in mental health and what makes the application of any of these interventions work", this study will carry out an overview of systematic reviews, which will make it possible to develop a theoretical framework on the implementation of electronic care in mental health problems.
Methods: We will search MEDLINE, EBM Reviews, PsycINFO, EMBASE, SCOPUS, CINAHL Complete, and Web of Science databases from 1st January 2015 to September 2020, with no language restriction. We will follow a qualitative method approach and include systematic reviews that assess primary studies relating to adults with common mental health problems using any type of mobile mental health intervention that includes a synchronic component and communication with a mental health professional. For the analysis, we will make a meta-synthesis of the systematic reviews, using an emergent grounded theory approach to synthesize the information, prioritizing the systematic reviews with the lowest risk of bias in the AMSTAR-2 tool. The meta-synthesis will be based on interpreting, integrating, and inferring the evaluation elements to understand better the e-health implementation process for patients with mental health problems. Finally, we will present the overall assessment in a Summary of Qualitative Findings table.
Conclusion: Our results will allow a better understanding of the facilitator and limitations in implementing e-health interventions for mental health problems.

Keywords

Telemedicine, Remote Consultation, Online Systems, Internet-Based Intervention, Mental Health, Mental Disorders, Systematic Reviews as Topic, Qualitative Research

Corresponding author: Christoper A. Alarcon-Ruiz

Competing interests: No competing interests were disclosed.

Grant information: Jeff Huarcaya-Victoria received funding for this study from EsSalud (Premio Kaelin 2020).
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2020 Villarreal-Zegarra D et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Villarreal-Zegarra D, Alarcon-Ruiz CA, Melendez-Torres GJ et al. Development of a framework for the implementation of electronic interventions in mental health: a protocol for a meta-synthesis of systematic reviews [version 1; peer review: 2 approved with reservations]. F1000Research 2020, 9:1282 (https://doi.org/10.12688/f1000research.27150.1) First published: 29 Oct 2020, 9:1282 (https://doi.org/10.12688/f1000research.27150.1) Latest published: 03 Aug 2021, 9:1282 (https://doi.org/10.12688/f1000research.27150.2)

Introduction

In the wake of the COVID-19 pandemic and after the actions taken by governments (such as social isolation), many mental health problems have increased in patients with COVID-19, patients with psychiatric symptoms, health personnel, and the general population^1,2. As a result, greater interest has been taken in addressing mental health issues during the pandemic^2,3. An example of this are the studies that identify anxiety disorders, depression, post-traumatic stress disorder, and stress as the most frequent health problems^2–4. To deal with these mental health problems and considering the current context (remote attention), it has been necessary to incorporate the use of technologies in the care^5,6 These technologies have been very well received and have served to complement or improve the effectiveness of treatments for various chronic diseases, reducing gaps in access to care and providing specialized care in inaccessible places⁷. In addition, these interventions using technologies show great promise in the care of mental health problems^8–10, highlighting the possibilities of care using technologies in health systems where resources are limited¹¹.

With the undeniable contribution of the use of technologies in mental health care, it has been important to document the aspects related to the application of interventions using technologies, since despite the effectiveness that these interventions can have, it is known that there have been difficulties in the application of interventions using technology¹². Aspects such as adaptability, cost, complexity, external policies and incentives, compatibility or general fit between the e-health intervention and the organization, etc.¹³ are important to consider in order to be clear about how and what works in these interventions and consider its complexity. To try to answer this there are many studies which have gathered information and synthesized it. An example of this are the systematic reviews and meta-analyses of interventions in mental health care using technologies (the latter also allowed conclusions to be drawn about the effectiveness of the interventions used)^14,15. However, there are certain aspects which this type of review (centered on quantitative studies) does not allow to identify¹⁶. That is why in order to answer the question "how do electronic interventions apply in mental health and what makes the application of any of these interventions work"; this study will carry out an overview of systematic reviews, which will make it possible to develop a theoretical framework on the implementation of electronic care in mental health problems.

Methods

SPIDER question

This protocol adheres to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols (PRISMA-P) guidelines¹⁷ A completed PRISMA-P checklist can be found in the Reporting guidelines section¹⁸. We used the SPIDER framework to develop the review question¹⁶:

Sample: Adults with depression (or major depressive disorder), anxiety (or generalized anxiety disorder), stress (or trauma-related disorders), and/or general mental health problems (unspecified).
Phenomenon of interest: Any type of mobile mental health intervention that includes a synchronic component, communication with a mental health professional (psychiatrist, psychologist, etc.) or a health professional trained in mental health. These interventions include, among others, remote consultation, interactive application, video chats, calls, etc.
Design: Systematic review.
Evaluation: We will include all types of outcomes of interest assessed by implementation studies, economic, qualitative, quantitative and others. For example, a) Health effectiveness outcomes: Depression, anxiety and/or stress symptoms, adherence to treatment, etc; b) Patient outcomes: Quality of life, satisfaction, etc; c) Economic outcomes; d) Damage or adverse effects. Also, we will include all kinds of statistical measures of the effects, if available: Relative risks, odds ratio, risk difference, mean difference and/or number needed to treat.
Research type: Quantitative studies, qualitative studies, mixed methods.

Eligibility criteria

Inclusion criteria: Systematic reviews that report on inclusion/exclusion criteria, conducted an adequate search, synthesized the included studies, assessed the quality of the included studies, and presented sufficient detail on the individual included studies¹⁹. Systematic reviews that included primary studies as a unit of analysis focused on a research question. Systematic reviews that were published in the last five years (since January 1, 2015) without language restrictions. We are including this time frame in order to include only the last updated systematic reviews. Reviews must include primary studies relating to adults with common mental health problems: a) Adults with depression (or major depressive disorder), anxiety (or generalized anxiety disorder), stress (or trauma-related disorders) and/or general mental health problems (unspecified); b) Adults attending an outpatient mental health consultation.

Exclusion criteria: Narrative reviews, scoping reviews, primary studies, opinion/editorial manuscripts, letters to the editor, and reviews of mobile health interventions repositories (i.e. apps stores). Studies in which some of these subjects has participated will also be excluded: a) Adults with some other mental health problem; b) Healthy adults without mental health problems; c) Adults receiving emergency/crisis psychiatric care; d) Interventions that lack a synchronic component (that lack real time information exchange using technologies).

Information sources

Included databases will be MEDLINE (Ovid), EBM Reviews (Ovid), PsycINFO (Ovid), EMBASE (Elsevier), SCOPUS, CINAHL Complete (EBSCOhost), and Web of Science databases, including Science Citation Index Expanded, Social Sciences Citation Index and Conference Proceedings Citation Index (Clarivate Analytics). Articles published in the last five years (after January 1st, 2015) will be included and no language restrictions will be imposed. We will include systematic reviews that report a review of literature using a research strategy in at least two different databases. Later, all references of included studies will be reviewed, and they will be evaluated looking for any additional systematic review that meets the inclusion criteria.

Search strategy

Main search terms to be used are "telemedicine" AND "mental health, anxiety, depression or stress" AND “systematic reviews”. The full search strategy for each database is available in the Extended data section¹⁸.

Study records

Data management: The results of the database search will be managed using the Rayyan QCRI free online application to manage the references (eliminate duplicates, and review titles and summaries)²⁰. Full-text review and data extraction will be done using an Excel template (Extended data¹⁸).

Selection process: There will be two distinct selection stages. First, a review of the title and abstract and the second a review of the full text. The reviews will be carried out based on pairs of reviewers, who will divide the total number of records. At each stage, there will be a first part where the reviewers will calibrate the accuracy of their reviews and a second part where the actual review will take place. To complete the calibration part, they will make calibration rounds until the discrepancy is less than or equal to 5% of the assigned records. If the percentage of the discrepancy is exceeded, a new round will be performed. Subsequently pairs of reviewers will divide the total of the records identified, and records will be screened independently and in duplicate. Discrepancies will be discussed within pairs with a third reviewer if needed.

In the review by title and abstract, first the calibration will be done, 50 records were selected from the total of records found as a result of the search strategy. Each pair had to make an independent review by title and abstract, selecting the studies that met the inclusion criteria. In the selection stage itself, the number of total records will be divided among all pairs of reviewers.

In the full-text review, first, a calibration of 10 records that will select in the previous phase will be performed. Only articles selected by title and abstract will review in full text. The articles excluded in the full-text review stage will be listed and a reason for exclusion will be given for each one of them.

Data collection process: For each eligible study, data will be extracted independently and in duplicate on pre-designed data extraction forms. In the event of discrepancies, the reviewers will discuss whether the extracted data should be retained and decide to resolve the discrepancy. If the discrepancy remains, a third reviewer will evaluate any unresolved disagreement between the reviewers on the data extraction and will decide on the value extracted that is in dispute.

Data items

An extraction form will be created for the included systematic reviews. Information will be collected on the author and date of the study, characteristics of the participants, main objective, research questions, inclusion criteria for the systematic review, search date, study selection process, quality assessment (if any), main findings and limitations. Also, the full text of the included article will be extracted along with the tables and supplementary material, to perform the qualitative analysis of the text.

Outcomes and prioritization

Our study seeks to make a meta-synthesis of the systematic reviews, using a qualitative strategy to synthesize the information and answer our research question. Therefore, we do not look for a specific result such as effectiveness, cost-effectiveness or other similar ones. Instead, we are interested in identifying the full text of all studies that answer our research question, to be able to perform a grounded theory analysis with an emergent approach. Priority will be given in the analysis of those studies with the lowest risk of bias assessed.

Risk of bias in individual studies

To assess the quality of the included systematic reviews we will use the "A Measurement Tool to Assess Systematic Reviews-2" (AMSTAR 2), which has sixteen domains. Seven of these domains are considered critical: 1) protocol registered before the start of the review, 2) adequacy of the literature search, 3) justification for the exclusion of individual studies, 4) risk of bias of individual studies included in the review, 5) adequacy of meta-analytic methods, 6) consideration of the risk of bias in interpreting the results of the review, and 7) assessment of the presence and likely impact of publication bias²¹.

AMSTAR-2 classifies the quality of systematic reviews into four different categories: high (none or one non-critical weakness), moderate (more than one non-critical weakness), low (one critical weakness with or without non-critical weaknesses), and very low (more than one critical weakness with or without non-critical weaknesses). The quality assessment of each systematic review will be rated by two researchers independently. In case of difference in the overall quality of the systematic reviews, the AMSTAR-2 criteria will be discussed among the researchers to reach a consensus.

Data synthesis

The analysis of the data will be carried out using a grounded theory approach with an emergent approach^22,23. The reviewers will follow the three steps established by Thomas and Harden²⁴. First, the extracted data are freely coded. The reviewers read the full texts of the included articles and code each text fragment that provides information to answer the research question. Second, the codified data are organized and grouped based on the descriptive aspects. The reviewers will review the codes generated in the previous step and group them into codes that are like each other and that allow the description of a part of the research question. Third, analytical concepts will generate different groups describing aspects generated in the previous step. The meta-synthesis will be based on interpreting, integrating and, inferring the evaluation elements that would allow a better understanding of the e-health implementation process from all the studies included²⁴. In addition to generate hypotheses supported by the results of the included studies, which will be generated through discussion and consensus among the reviewers²⁴.

When included studies had been selected, they will be ranked based on the AMSTAR-2 score, with the highest quality studies being assessed first. We will assess all included studies, down to the criterion of theoretical saturation. All qualitative analyses will be performed with Atlas.ti v.8 software.

Confidence in cumulative evidence

It will be evaluated the approach of Confidence in the Evidence from Reviews of Qualitative research (CERQual) which has four components (Methodological Limitations, Relevance, Coherence and Adequacy data), and thus contribute to an overall assessment for each systematic review to determine the level of confidence (high, moderate, low, or very low) and present the overall assessment in a Summary of Qualitative Findings (SoQF) table^25,26.

Dissemination plan

We will present the systematic review findings and submit the manuscript to a relevant peer-reviewed journal.

Study status

This systematic review is currently in the first stage (title and abstract screening) in the selection process. The protocol of this systematic review was submitted to PROSPERO registry on 18^th August, 2020 (CRD42020203811).

Data availability

Underlying data

No data are associated with this article.

Extended data

Open Science Framework: PRISMA-P checklist for ‘Development of a framework for the implementation of electronic interventions in mental health: A protocol for a meta-synthesis of systematic reviews’, https://doi.org/10.17605/OSF.IO/3UH4N¹⁸. (Registered on 20^th October 2020: osf.io/tf4b6.)

This project contains the following extended data:

- Full text protocol
- Full search strategy for MEDLINE, EBM Reviews, PsycINFO, EMBASE, SCOPUS, CINAHL Complete, and Web of Science databases
- Excel template for data extraction

Reporting guidelines

Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

Acknowledgments

The authors acknowledge Jackeline Garcia and Juan Barrera, for helping with the initial phases of the information search.

Faculty Opinions recommended

References

1. Vindegaard N, Benros ME: COVID-19 pandemic and mental health consequences: Systematic review of the current evidence. Brain Behav Immun. 2020; 89: 531–542. PubMed Abstract | Publisher Full Text | Free Full Text
2. Talevi D, Socci V, Carai M, et al.: Mental health outcomes of the CoViD-19 pandemic. Riv Psichiatr. 2020; 55(3): 137–44. PubMed Abstract | Publisher Full Text
3. Huarcaya-Victoria J: Consideraciones sobre la salud mental en la pandemia de COVID-19. Rev Peru Med Exp Salud Publica. 2020; 37(2): 327–34. PubMed Abstract | Publisher Full Text
4. Xiong J, Lipsitz O, Nasri F, et al.: Impact of COVID-19 pandemic on mental health in the general population: A systematic review. J Affect Disord. 2020; 277: 55–64. PubMed Abstract | Publisher Full Text | Free Full Text
5. Sullivan AB, Kane A, Roth AJ, et al.: The COVID-19 Crisis: A Mental Health Perspective and Response Using Telemedicine. J Patient Exp. 2020; 7(3): 295–301. PubMed Abstract | Publisher Full Text | Free Full Text
6. Haxhihamza K, Arsova S, Bajraktarov S, et al.: Patient Satisfaction with Use of Telemedicine in University Clinic of Psychiatry: Skopje, North Macedonia During COVID-19 Pandemic. Telemed J E Health. 2020. PubMed Abstract | Publisher Full Text
7. Ruiz EF, Proaño A, Ponce OJ, et al.: [Mobile health for public health in Peru: lessons learned]. Rev Peru Med Exp Salud Publica. 2015; 32(2): 364–72. PubMed Abstract
8. Miranda JJ, Moscoso MG, Toyama M, et al.: Role of mHealth in overcoming the occurrence of post-stroke depression. Acta Neurol Scand. 2018; 137(1): 12–9. PubMed Abstract | Publisher Full Text | Free Full Text
9. Andrews G, Cuijpers P, Craske MG, et al.: Computer Therapy for the Anxiety and Depressive Disorders Is Effective, Acceptable and Practical Health Care: A Meta-Analysis. PLoS One. 2010; 5(10): e13196. PubMed Abstract | Publisher Full Text | Free Full Text
10. Turvey C, Fortney J: The Use of Telemedicine and Mobile Technology to Promote Population Health and Population Management for Psychiatric Disorders. Curr Psychiatry Rep. 2017; 19(11): 88. PubMed Abstract | Publisher Full Text
11. Kaonga NN, Morgan J: Common themes and emerging trends for the use of technology to support mental health and psychosocial well-being in limited resource settings: A review of the literature. Psychiatry Res. 2019; 281: 112594. PubMed Abstract | Publisher Full Text
12. Meurk C, Leung J, Hall W, et al.: Establishing and Governing e-Mental Health Care in Australia: A Systematic Review of Challenges and A Call For Policy-Focussed Research. J Med Internet Res. 2016; 18(1): e10. PubMed Abstract | Publisher Full Text | Free Full Text
13. Ross J, Stevenson F, Lau R, et al.: Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update). Implement Sci. 2016; 11(1): 146. PubMed Abstract | Publisher Full Text | Free Full Text
14. Griffiths KM, Farrer L, Christensen H: The efficacy of internet interventions for depression and anxiety disorders: a review of randomised controlled trials. Med J Aust. 2010; 192(S11): S4–S11. PubMed Abstract
15. Senanayake B, Wickramasinghe SI, Chatfield MD, et al.: Effectiveness of text messaging interventions for the management of depression: A systematic review and meta-analysis. J Telemed Telecare. 2019; 25(9): 513–523. PubMed Abstract | Publisher Full Text
16. Cooke A, Smith D, Booth A: Beyond PICO: The SPIDER Tool for Qualitative Evidence Synthesis. Qual Health Res. 2012; 22(10): 1435–43. PubMed Abstract | Publisher Full Text
17. Moher D, Shamseer L, Clarke M, et al.: Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015; 4(1): 1. PubMed Abstract | Publisher Full Text | Free Full Text
18. Villarreal-Zegarra D, Alarcon-Ruiz CA, Melendez-Torres G, et al.: Development of a framework for the implementation of electronic interventions in mental health: A protocol for a meta-synthesis of systematic reviews. 2020. http://www.doi.org/10.17605/OSF.IO/3UH4N
19. Dissemination CfRa. University of York: Centre for Reviews and Dissemination. [updated 3 June, 2015]. Reference Source
20. Ouzzani M, Hammady H, Fedorowicz Z, et al.: Rayyan—a web and mobile app for systematic reviews. Syst Rev. 2016; 5(1): 210. PubMed Abstract | Publisher Full Text | Free Full Text
21. Shea BJ, Reeves BC, Wells G, et al.: AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017; 358: j4008. PubMed Abstract | Publisher Full Text | Free Full Text
22. Guetterman TC, Babchuk WA, Howell Smith MC, et al.: Contemporary Approaches to Mixed Methods–Grounded Theory Research: A Field-Based Analysis. J Mix Methods Res. 2019; 13(2): 179–95. Publisher Full Text
23. Thorne S, Jensen L, Kearney MH, et al.: Qualitative metasynthesis: reflections on methodological orientation and ideological agenda. Qual Health Res. 2004; 14(10): 1342–65. PubMed Abstract | Publisher Full Text
24. Thomas J, Harden A: Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008; 8: 45. PubMed Abstract | Publisher Full Text | Free Full Text
25. Group TG-CP: A practical guide for decision makers in health and social care and other users. Reference Source
26. Lewin S, Booth A, Glenton C, et al.: Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series. Implement Sci. 2018; 13(Suppl 1): 2. PubMed Abstract | Publisher Full Text | Free Full Text

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 29 Oct 2020

Author details Author details

G.J. Melendez-Torres
Roles: Conceptualization, Investigation, Methodology, Supervision, Writing – Review & Editing

Roberto Torres-Puente
Roles: Investigation, Writing – Original Draft Preparation, Writing – Review & Editing

Juan Ambrosio-Melgarejo
Roles: Investigation, Writing – Original Draft Preparation, Writing – Review & Editing

Alejandra B. Romero-Cabrera
Roles: Investigation, Writing – Review & Editing

Ana Lindo-Cavero
Roles: Investigation, Writing – Review & Editing

Jeff Huarcaya-Victoria
Roles: Conceptualization, Funding Acquisition, Investigation, Methodology, Supervision, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

Jeff Huarcaya-Victoria received funding for this study from EsSalud (Premio Kaelin 2020).
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Article Versions (2)

version 2

Revised

Published: 03 Aug 2021, 9:1282

https://doi.org/10.12688/f1000research.27150.2

version 1

Published: 29 Oct 2020, 9:1282

https://doi.org/10.12688/f1000research.27150.1

© 2020 Villarreal-Zegarra D et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

Villarreal-Zegarra D, Alarcon-Ruiz CA, Melendez-Torres GJ et al. Development of a framework for the implementation of electronic interventions in mental health: a protocol for a meta-synthesis of systematic reviews [version 1; peer review: 2 approved with reservations] F1000Research 2020, 9:1282 (https://doi.org/10.12688/f1000research.27150.1)

NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status: ?

Key to Reviewer Statuses VIEW HIDE

ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions

Version 1

VERSION 1

PUBLISHED 29 Oct 2020

Views

Reviewer Report 18 Jun 2021

Roger Hilfiker, Valais-Wallis School of Health Sciences, HES-SO, University of Applied Sciences and Arts Western Switzerland, Valais, Switzerland

Approved with Reservations

https://doi.org/10.5256/f1000research.29992.r85471

The manuscript reports on a protocol for a systematic review with a meta-synthesis of systematic reviews.

The authors describe a qualitative approach for the analysis of the data with three steps, i.e. coding of each text fragment that provides answers to the research questions, organizing and grouping the codified data, and grouping the aspects.

I have some specific comments:

Research question: Please clarify how you will use the data to answer the question on "what makes the application of any of these interventions work".

Data synthesis: The sentence "In addition to generate hypotheses supported by the results of the included studies, which will be generated through discussion and consensus among the reviewers" is not clear to me.

The authors described the extraction of quantitative data on effectiveness. Please clarify whether these quantitative data are analyzed with the qualitative method or whether quantitative methods are used.

Please describe how the research question on the "what makes the application work" will be answered.

Confidence in cumulative evidence: Wouldn't the research question "what makes the application of any of these interventions work" require a different evaluation of the confidence of the evidence, for example an approach such as GRADE?

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Yes

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Systematic Reviews

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Author Response 03 Aug 2021

Christoper A. Alarcon-Ruiz, Universidad Científica del Sur, Lima, Peru

03 Aug 2021

Author Response

1. Research question: Please clarify how you will use the data to answer the question on "what makes the application of any of these interventions work".
Reply: It was specified ... Continue reading 1. Research question: Please clarify how you will use the data to answer the question on "what makes the application of any of these interventions work".
Reply: It was specified in the subsection of Data synthesis
“The develop a framework informed by a realist analysis for the implementation of synchronous e-mental health will be carried out using a grounded theory approach with an emergent approach (23). The realist synthesis will be based on interpreting, integrating and, inferring the evaluation elements that would allow a better understanding of the synchronous e-mental health implementation process from all the studies included (24). To answer the question "what makes the implementation of these interventions work", hypotheses supported by the results of the included studies will be developed and generated through discussion and consensus among researchers (24). Two reviewers will follow the three steps established by Thomas and Harden for qualitative syntheses will be followed (25). First, the extracted data are freely coded. The reviewers read the full-texts of the included articles and code each text fragment that provides information to answer the research question. Second, the codified data are organized and grouped based on the descriptive aspects. The reviewers will review the codes generated in the previous step and group them into codes that are like each other and that allow the description of a part of the research question. Thirdly, the analytical concepts generated in the previous step will be grouped in a way that they are related to each other. The elements that are related to each other will be assumed to be part of a hypothesis that would help to answer the research aim. The sum of all these hypotheses will be referred to as a framework which will attempt to explain the results using a context-linked causality approach represented as context + mechanism = outcome (26).”

2. Data synthesis: The sentence "In addition to generate hypotheses supported by the results of the included studies, which will be generated through discussion and consensus among the reviewers" is not clear to me.
Reply: We agree with the reviewer, this has been modified in the previous comment.

3. The authors described the extraction of quantitative data on effectiveness. Please clarify whether these quantitative data are analyzed with the qualitative method or whether quantitative methods are used.
Reply: The collection of quantitative data was eliminated, as specified in the method section:
“Our study seeks to perform a realistic synthesis of the evidence, so we focused on different outcomes to use them as input. Therefore, we do not perform a quantitative synthesis in any case (i.e., a meta-analysis of effectiveness).”

4. Please describe how the research question on the "what makes the application work" will be answered.
Reply: A better description of the research objective was made based on the first reviewer.
“Thus, our study has two main objectives: a) "to determine what evidence is available for synchronous e-mental health implementation"; and b) "to develop a framework informed by a realist analysis for the implementation of synchronous e-mental health".”
The process of how to achieve these objectives was answered in the first comment.

5. Confidence in cumulative evidence: Wouldn't the research question "what makes the application of any of these interventions work" require a different evaluation of the confidence of the evidence, for example an approach such as GRADE?
Reply: we will evaluate each key finding using CERQual to assess the methodological limitations, coherence, adequacy, and relevance of findings.
“It will be evaluated the approach of Confidence in the Evidence from Reviews of Qualitative research (CERQual) which has four components (Methodological Limitations, Relevance, Coherence and Adequacy data), will be evaluated to contribute to an overall assessment for each hypothesis resulting from the realist synthesis to determine the level of confidence (high, moderate, low, or very low) and present the overall assessment in a Summary of Qualitative Findings (SoQF) table (27, 28).”
1. Research question: Please clarify how you will use the data to answer the question on "what makes the application of any of these interventions work".
Reply: It was specified in the subsection of Data synthesis
“The develop a framework informed by a realist analysis for the implementation of synchronous e-mental health will be carried out using a grounded theory approach with an emergent approach (23). The realist synthesis will be based on interpreting, integrating and, inferring the evaluation elements that would allow a better understanding of the synchronous e-mental health implementation process from all the studies included (24). To answer the question "what makes the implementation of these interventions work", hypotheses supported by the results of the included studies will be developed and generated through discussion and consensus among researchers (24). Two reviewers will follow the three steps established by Thomas and Harden for qualitative syntheses will be followed (25). First, the extracted data are freely coded. The reviewers read the full-texts of the included articles and code each text fragment that provides information to answer the research question. Second, the codified data are organized and grouped based on the descriptive aspects. The reviewers will review the codes generated in the previous step and group them into codes that are like each other and that allow the description of a part of the research question. Thirdly, the analytical concepts generated in the previous step will be grouped in a way that they are related to each other. The elements that are related to each other will be assumed to be part of a hypothesis that would help to answer the research aim. The sum of all these hypotheses will be referred to as a framework which will attempt to explain the results using a context-linked causality approach represented as context + mechanism = outcome (26).”

2. Data synthesis: The sentence "In addition to generate hypotheses supported by the results of the included studies, which will be generated through discussion and consensus among the reviewers" is not clear to me.
Reply: We agree with the reviewer, this has been modified in the previous comment.

3. The authors described the extraction of quantitative data on effectiveness. Please clarify whether these quantitative data are analyzed with the qualitative method or whether quantitative methods are used.
Reply: The collection of quantitative data was eliminated, as specified in the method section:
“Our study seeks to perform a realistic synthesis of the evidence, so we focused on different outcomes to use them as input. Therefore, we do not perform a quantitative synthesis in any case (i.e., a meta-analysis of effectiveness).”

4. Please describe how the research question on the "what makes the application work" will be answered.
Reply: A better description of the research objective was made based on the first reviewer.
“Thus, our study has two main objectives: a) "to determine what evidence is available for synchronous e-mental health implementation"; and b) "to develop a framework informed by a realist analysis for the implementation of synchronous e-mental health".”
The process of how to achieve these objectives was answered in the first comment.

5. Confidence in cumulative evidence: Wouldn't the research question "what makes the application of any of these interventions work" require a different evaluation of the confidence of the evidence, for example an approach such as GRADE?
Reply: we will evaluate each key finding using CERQual to assess the methodological limitations, coherence, adequacy, and relevance of findings.
“It will be evaluated the approach of Confidence in the Evidence from Reviews of Qualitative research (CERQual) which has four components (Methodological Limitations, Relevance, Coherence and Adequacy data), will be evaluated to contribute to an overall assessment for each hypothesis resulting from the realist synthesis to determine the level of confidence (high, moderate, low, or very low) and present the overall assessment in a Summary of Qualitative Findings (SoQF) table (27, 28).”
Competing Interests: None Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 03 Aug 2021

Christoper A. Alarcon-Ruiz, Universidad Científica del Sur, Lima, Peru

03 Aug 2021

Author Response

1. Research question: Please clarify how you will use the data to answer the question on "what makes the application of any of these interventions work".
Reply: It was specified ... Continue reading 1. Research question: Please clarify how you will use the data to answer the question on "what makes the application of any of these interventions work".
Reply: It was specified in the subsection of Data synthesis
“The develop a framework informed by a realist analysis for the implementation of synchronous e-mental health will be carried out using a grounded theory approach with an emergent approach (23). The realist synthesis will be based on interpreting, integrating and, inferring the evaluation elements that would allow a better understanding of the synchronous e-mental health implementation process from all the studies included (24). To answer the question "what makes the implementation of these interventions work", hypotheses supported by the results of the included studies will be developed and generated through discussion and consensus among researchers (24). Two reviewers will follow the three steps established by Thomas and Harden for qualitative syntheses will be followed (25). First, the extracted data are freely coded. The reviewers read the full-texts of the included articles and code each text fragment that provides information to answer the research question. Second, the codified data are organized and grouped based on the descriptive aspects. The reviewers will review the codes generated in the previous step and group them into codes that are like each other and that allow the description of a part of the research question. Thirdly, the analytical concepts generated in the previous step will be grouped in a way that they are related to each other. The elements that are related to each other will be assumed to be part of a hypothesis that would help to answer the research aim. The sum of all these hypotheses will be referred to as a framework which will attempt to explain the results using a context-linked causality approach represented as context + mechanism = outcome (26).”

2. Data synthesis: The sentence "In addition to generate hypotheses supported by the results of the included studies, which will be generated through discussion and consensus among the reviewers" is not clear to me.
Reply: We agree with the reviewer, this has been modified in the previous comment.

3. The authors described the extraction of quantitative data on effectiveness. Please clarify whether these quantitative data are analyzed with the qualitative method or whether quantitative methods are used.
Reply: The collection of quantitative data was eliminated, as specified in the method section:
“Our study seeks to perform a realistic synthesis of the evidence, so we focused on different outcomes to use them as input. Therefore, we do not perform a quantitative synthesis in any case (i.e., a meta-analysis of effectiveness).”

4. Please describe how the research question on the "what makes the application work" will be answered.
Reply: A better description of the research objective was made based on the first reviewer.
“Thus, our study has two main objectives: a) "to determine what evidence is available for synchronous e-mental health implementation"; and b) "to develop a framework informed by a realist analysis for the implementation of synchronous e-mental health".”
The process of how to achieve these objectives was answered in the first comment.

5. Confidence in cumulative evidence: Wouldn't the research question "what makes the application of any of these interventions work" require a different evaluation of the confidence of the evidence, for example an approach such as GRADE?
Reply: we will evaluate each key finding using CERQual to assess the methodological limitations, coherence, adequacy, and relevance of findings.
“It will be evaluated the approach of Confidence in the Evidence from Reviews of Qualitative research (CERQual) which has four components (Methodological Limitations, Relevance, Coherence and Adequacy data), will be evaluated to contribute to an overall assessment for each hypothesis resulting from the realist synthesis to determine the level of confidence (high, moderate, low, or very low) and present the overall assessment in a Summary of Qualitative Findings (SoQF) table (27, 28).”
1. Research question: Please clarify how you will use the data to answer the question on "what makes the application of any of these interventions work".
Reply: It was specified in the subsection of Data synthesis
“The develop a framework informed by a realist analysis for the implementation of synchronous e-mental health will be carried out using a grounded theory approach with an emergent approach (23). The realist synthesis will be based on interpreting, integrating and, inferring the evaluation elements that would allow a better understanding of the synchronous e-mental health implementation process from all the studies included (24). To answer the question "what makes the implementation of these interventions work", hypotheses supported by the results of the included studies will be developed and generated through discussion and consensus among researchers (24). Two reviewers will follow the three steps established by Thomas and Harden for qualitative syntheses will be followed (25). First, the extracted data are freely coded. The reviewers read the full-texts of the included articles and code each text fragment that provides information to answer the research question. Second, the codified data are organized and grouped based on the descriptive aspects. The reviewers will review the codes generated in the previous step and group them into codes that are like each other and that allow the description of a part of the research question. Thirdly, the analytical concepts generated in the previous step will be grouped in a way that they are related to each other. The elements that are related to each other will be assumed to be part of a hypothesis that would help to answer the research aim. The sum of all these hypotheses will be referred to as a framework which will attempt to explain the results using a context-linked causality approach represented as context + mechanism = outcome (26).”

2. Data synthesis: The sentence "In addition to generate hypotheses supported by the results of the included studies, which will be generated through discussion and consensus among the reviewers" is not clear to me.
Reply: We agree with the reviewer, this has been modified in the previous comment.

3. The authors described the extraction of quantitative data on effectiveness. Please clarify whether these quantitative data are analyzed with the qualitative method or whether quantitative methods are used.
Reply: The collection of quantitative data was eliminated, as specified in the method section:
“Our study seeks to perform a realistic synthesis of the evidence, so we focused on different outcomes to use them as input. Therefore, we do not perform a quantitative synthesis in any case (i.e., a meta-analysis of effectiveness).”

4. Please describe how the research question on the "what makes the application work" will be answered.
Reply: A better description of the research objective was made based on the first reviewer.
“Thus, our study has two main objectives: a) "to determine what evidence is available for synchronous e-mental health implementation"; and b) "to develop a framework informed by a realist analysis for the implementation of synchronous e-mental health".”
The process of how to achieve these objectives was answered in the first comment.

5. Confidence in cumulative evidence: Wouldn't the research question "what makes the application of any of these interventions work" require a different evaluation of the confidence of the evidence, for example an approach such as GRADE?
Reply: we will evaluate each key finding using CERQual to assess the methodological limitations, coherence, adequacy, and relevance of findings.
“It will be evaluated the approach of Confidence in the Evidence from Reviews of Qualitative research (CERQual) which has four components (Methodological Limitations, Relevance, Coherence and Adequacy data), will be evaluated to contribute to an overall assessment for each hypothesis resulting from the realist synthesis to determine the level of confidence (high, moderate, low, or very low) and present the overall assessment in a Summary of Qualitative Findings (SoQF) table (27, 28).”
Competing Interests: None Close
Report a concern

Views

Reviewer Report 11 Feb 2021

Amy Grove, Health Technology Assessment and Implementation Science, Warwick Medical School, University of Warwick, Coventry, UK

Approved with Reservations

https://doi.org/10.5256/f1000research.29992.r78252

Abstract:

"Some quantitative systematic" can you indicate how many rather than using the generic some.
Would the research question be better phrased as 2 questions? or a primary and secondary question?

Abstract:

"Some quantitative systematic" can you indicate how many rather than using the generic some.
Would the research question be better phrased as 2 questions? or a primary and secondary question? i) how question?; ii) what makes question?
Please define what approach you will use "We will follow a qualitative method approach".
Are "incorporate technologies", "implementation of electronic care", "mobile mental health intervention that includes a synchronic component and communication with a mental health professional", "e-health implementation process" and, "synchronic component (that lack real time information exchange using technologies)." all the same thing? I would suggest you select one terminology and use it consistently.

Introduction:

Please remove "To deal with these mental health problems". This is not appropriate terminology.
You talk about "These technologies", can you provide examples?
You say "(centered on quantitative studies) does not allow to identify". Describe what it does not allow; what is missing from existing reviews?

Methods:

The heading, "SPIDER question", needs to be replaced with "research question". The acronym may not be understood by all.
Regarding the sample - do people have to have a certified diagnosis of the condition? What about undiagnosed groups?

Evaluation:

Are you assessing effectiveness? or just implementation of a technology that has shown to be effective elsewhere?
If you are following a "qualitative method approach" why are you including outcomes of "all kinds of statistical measures of the effects"? What will you do with this information?
How do you define "conducted an adequate search"? If you are using a criteria then please list it.
Please provided references for "only the last updated systematic reviews".
The study selection process is overly detailed and repetitive.
You say "An extraction form will be created for the included systematic reviews.", but above you imply it is already done and included in "an Excel template (Extended data)".
"Therefore, we do not look for a specific result such as effectiveness, cost-effectiveness or other similar ones." Then why are you searching for data?
"perform a grounded theory analysis with an emergent approach" - you said earlier "using an emergent grounded theory approach", what is the difference? If no difference, then please use consistent terms.
"Priority will be given in the analysis of those studies with the lowest risk of bias assessed." - Explain why this is the case
In data synthesis you do not state how many "reviewers will follow the three steps".
In the abstract you say you will "develop a theoretical framework" but you do not give detail of how this will be done in the manuscript.
Please also proof read for typographical errors and appropriate use of English.

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Yes

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: HTA and implementation science

CITE

Report a concern

Author Response 03 Aug 2021

Christoper A. Alarcon-Ruiz, Universidad Científica del Sur, Lima, Peru

03 Aug 2021

Author Response

Abstract:
1. "Some quantitative systematic" can you indicate how many rather than using the generic some.
Reply: We specified this concern. Abstract: “Quantitative systematic reviews such as meta-analysis”

2. ... Continue reading Abstract:
1. "Some quantitative systematic" can you indicate how many rather than using the generic some.
Reply: We specified this concern. Abstract: “Quantitative systematic reviews such as meta-analysis”

2. Would the research question be better phrased as 2 questions? or a primary and secondary question? i) how question?; ii) what makes question?
Reply: It was modified in the abstract:
“Thus, our study has two main objectives: a) "to determine what evidence is available for synchronous e-mental health implementation"; and b) "to develop a framework informed by a realistic analysis for the implementation of synchronous e-mental health".”

3. Please define what approach you will use "We will follow a qualitative method approach".
Reply: It was modified in the abstract:
“A systematic review with narrative description and a realist synthesis will be conducted. Primary studies related to adults with common mental health problems using any type of mobile mental health intervention that includes a synchronous component and communication with a mental health professional will be included. For the analysis, we will make a realistic synthesis of the systematic reviews, using a grounded theory approach with an emergent approach to synthesize the information, prioritizing the systematic reviews with a lower risk of bias in the AMSTAR-2 tool. The realist synthesis will be based on the interpretation, integration, and inference of the evaluated elements and the generation of hypotheses to better understand the implementation process of synchronic e-mental health.”

4. Are "incorporate technologies", "implementation of electronic care", "mobile mental health intervention that includes a synchronic component and communication with a mental health professional", "e-health implementation process" and, "synchronic component (that lack real-time information exchange using technologies)." all the same thing? I would suggest you select one terminology and use it consistently.
Reply: We’ll use the term “e-mental health” to refer to mental health care delivery through technology. And we’ll use “synchronous e-mental health” to refer to the delivery of mental health care through technology in real-time.

Introduction:
5. Please remove "To deal with these mental health problems". This is not appropriate terminology.
Reply: We changed it for “As a result, it has been...”

6. You talk about "These technologies", can you provide examples?
Reply: We specified some examples: “such as internet and mobile devices ”

7. You say "(centered on quantitative studies) does not allow to identify". Describe what it does not allow; what is missing from existing reviews?
Reply: Does not provide information about experience, meaning, and perspective, which is important when addressing implementation issues.
“However, qualitative studies are necessary because of their methodology (approach to answer questions about experience, meaning, and perspective) which is useful to address issues of implementation.”

Methods:

8. The heading, "SPIDER question", needs to be replaced with "research question". The acronym may not be understood by all.
Reply: The heading has been modified.

9. Regarding the sample - do people have to have a certified diagnosis of the condition? What about undiagnosed groups?
Reply: It was modified in the methods:
“Sample: Adults with depression (or major depressive disorder), anxiety (or generalized anxiety disorder), stress (or trauma-related disorders), and/or general mental health problems (unspecified). Participants may be diagnosed through clinical interviews or categorized based on screening assessments (self-reported scales).”

Evaluation:
10. Are you assessing effectiveness? or just implementation of a technology that has shown to be effective elsewhere?
Reply: It was modified in the method:
“All types of outcomes of interest assessed by implementation, economic, qualitative, quantitative, and other studies will be included. For example, a) Health effectiveness outcomes: Depression, anxiety and/or stress symptoms, adherence to treatment, etc.; b) Patient outcomes: Quality of life, satisfaction, etc.; c) Economic outcomes; d) Damage or adverse effects. Our study seeks to perform a realist synthesis of the evidence, so we focused on different outcomes to use them as input that allow s us to evaluate the implementation of synchronous e-mental health. Therefore, we do not perform a quantitative synthesis in any case (i.e., a meta-analysis of effectiveness).”

11. If you are following a "qualitative method approach" why are you including outcomes of "all kinds of statistical measures of the effects"? What will you do with this information?
Reply: We agree with the reviewer. The quantitative information was eliminated. It was modified in the method:
“Our study seeks to perform a realist synthesis of the evidence, so we focused on different outcomes to use them as input. Therefore, we do not perform a quantitative synthesis in any case (i.e., a meta-analysis of effectiveness).”

12. How do you define "conducted an adequate search"? If you are using criteria then please list them.
Reply: We agree with your observation. We have specified and listed the criteria we consider to include a systematic review “Systematic reviews using at least two databases that synthesized, assessed the quality, and presented sufficient detail of the primary included studies; and reports on inclusion/exclusion criteria”

13. Please provided references for "only the last updated systematic reviews".
Reply: It was specified in the text:
“We include this time frame to include only the latest systematic reviews since in the field of e-health, the launch of new technologies makes scientific development dynamic. “

14. The study selection process is overly detailed and repetitive.
Reply: We synthesized the study selection:
“In the review by title and abstract, calibration will be done first, 50 records were selected from the total number of records found as a result of the search strategy. In the full-text review, calibration will first be performed on 10 records that will be selected in the previous stage. The articles excluded in the full-text review stage will be listed, and a reason for exclusion will be given for each of them.”

15. You say, "An extraction form will be created for the included systematic reviews.", but above, you imply it is already done and included in "an Excel template (Extended data)".
Reply: It was modified in the subsection of data items:
“An extraction form was created in an excel template for the included systematic reviews (https://osf.io/7esgz/).”

16. "Therefore, we do not look for a specific result such as effectiveness, cost-effectiveness or other similar ones." Then why are you searching for data?
Reply: It was modified in the method:
“Our study seeks to perform a realist synthesis of the evidence, so we focused on different outcomes to use them as input. Therefore, we do not perform a quantitative synthesis in any case (i.e., a meta-analysis of effectiveness).”
The section where we state that we will collect specific quantitative data, such as effect size, has been removed. Studies with efficacy, cost-effectiveness, or other similar outcomes are included because they will help answer our question.

17. "perform a grounded theory analysis with an emergent approach" - you said earlier, "using an emergent grounded theory approach", what is the difference? If no difference, then please use consistent terms.
Reply: It was specified:
“Our study aims to conduct a systematic review of systematic reviews and a realistic synthesis of systematic reviews, using a qualitative strategy to synthesize the information and answer our research question. Therefore, we do not look for specific results such as effectiveness, cost-effectiveness, or other similar ones. Instead, we are interested in identifying the full text of all the studies that answer our research question so that we can make a realistic synthesis using a grounded theory model with an emergent approach. The realistic synthesis will be based on the interpretation, integration, and inference of the evaluated elements and the generation of hypotheses to better understand the studied phenomenon (implementation of synchronous e-mental health) (20, 21).”

18. "Priority will be given in the analysis of those studies with the lowest risk of bias assessed." - Explain why this is the case
Reply: It was specified:
“Priority will be given in the analysis of those studies with the lowest risk of bias assessed. If possible, a realist synthesis will be performed only with those systematic reviews with high quality, to assess whether the results change by selecting only those studies with high quality (low risk of bias).”

19. In data synthesis, you do not state how many "reviewers will follow the three steps".
Reply: We agree with your observation. We have specified the number of reviewers involved in this process. “Two reviewers will follow the three steps established by Thomas and Harden for qualitative syntheses will be followed.”

20. In the abstract, you say you will "develop a theoretical framework" but you do not give detail of how this will be done in the manuscript.
Reply: The authors have identified that the meta-synthesis analysis method was not the most appropriate for the review, as the description is more in line with a realistic synthesis analysis. It was specified in the section on data synthesis.
“The develop a framework informed by a realist analysis for the implementation of synchronous e-mental health will be carried out using a grounded theory approach with an emergent approach (23). The realist synthesis will be based on interpreting, integrating and, inferring the evaluation elements that would allow a better understanding of the synchronic e-mental health implementation process from all the studies included (24). To answer the question "what makes the implementation of these interventions work", hypotheses supported by the results of the included studies will be developed and generated through discussion and consensus among researchers (24). Two reviewers will follow the three steps established by Thomas and Harden for qualitative syntheses will be followed (25). First, the extracted data are freely coded. The reviewers read the full texts of the included articles and code each text fragment that provides information to answer the research question. Second, the codified data are organized and grouped based on the descriptive aspects. The reviewers will review the codes generated in the previous step and group them into codes that are like each other and that allow the description of a part of the research question. Thirdly, the analytical concepts generated in the previous step will be grouped in a way that they are related to each other. The elements that are related to each other will be assumed to be part of a hypothesis that would help to answer the research aim. The sum of all these hypotheses will be referred to as a framework that will attempt to explain the results using a context-linked causality approach represented as context + mechanism = outcome (26). ”

21. Please also proofread for typographical errors and appropriate use of English.
Reply: We accept your observation. We have reviewed and corrected typographical errors and the adequate use of English.
Abstract:
1. "Some quantitative systematic" can you indicate how many rather than using the generic some.
Reply: We specified this concern. Abstract: “Quantitative systematic reviews such as meta-analysis”

2. Would the research question be better phrased as 2 questions? or a primary and secondary question? i) how question?; ii) what makes question?
Reply: It was modified in the abstract:
“Thus, our study has two main objectives: a) "to determine what evidence is available for synchronous e-mental health implementation"; and b) "to develop a framework informed by a realistic analysis for the implementation of synchronous e-mental health".”

3. Please define what approach you will use "We will follow a qualitative method approach".
Reply: It was modified in the abstract:
“A systematic review with narrative description and a realist synthesis will be conducted. Primary studies related to adults with common mental health problems using any type of mobile mental health intervention that includes a synchronous component and communication with a mental health professional will be included. For the analysis, we will make a realistic synthesis of the systematic reviews, using a grounded theory approach with an emergent approach to synthesize the information, prioritizing the systematic reviews with a lower risk of bias in the AMSTAR-2 tool. The realist synthesis will be based on the interpretation, integration, and inference of the evaluated elements and the generation of hypotheses to better understand the implementation process of synchronic e-mental health.”

4. Are "incorporate technologies", "implementation of electronic care", "mobile mental health intervention that includes a synchronic component and communication with a mental health professional", "e-health implementation process" and, "synchronic component (that lack real-time information exchange using technologies)." all the same thing? I would suggest you select one terminology and use it consistently.
Reply: We’ll use the term “e-mental health” to refer to mental health care delivery through technology. And we’ll use “synchronous e-mental health” to refer to the delivery of mental health care through technology in real-time.

Introduction:
5. Please remove "To deal with these mental health problems". This is not appropriate terminology.
Reply: We changed it for “As a result, it has been...”

6. You talk about "These technologies", can you provide examples?
Reply: We specified some examples: “such as internet and mobile devices ”

7. You say "(centered on quantitative studies) does not allow to identify". Describe what it does not allow; what is missing from existing reviews?
Reply: Does not provide information about experience, meaning, and perspective, which is important when addressing implementation issues.
“However, qualitative studies are necessary because of their methodology (approach to answer questions about experience, meaning, and perspective) which is useful to address issues of implementation.”

Methods:

8. The heading, "SPIDER question", needs to be replaced with "research question". The acronym may not be understood by all.
Reply: The heading has been modified.

9. Regarding the sample - do people have to have a certified diagnosis of the condition? What about undiagnosed groups?
Reply: It was modified in the methods:
“Sample: Adults with depression (or major depressive disorder), anxiety (or generalized anxiety disorder), stress (or trauma-related disorders), and/or general mental health problems (unspecified). Participants may be diagnosed through clinical interviews or categorized based on screening assessments (self-reported scales).”

Evaluation:
10. Are you assessing effectiveness? or just implementation of a technology that has shown to be effective elsewhere?
Reply: It was modified in the method:
“All types of outcomes of interest assessed by implementation, economic, qualitative, quantitative, and other studies will be included. For example, a) Health effectiveness outcomes: Depression, anxiety and/or stress symptoms, adherence to treatment, etc.; b) Patient outcomes: Quality of life, satisfaction, etc.; c) Economic outcomes; d) Damage or adverse effects. Our study seeks to perform a realist synthesis of the evidence, so we focused on different outcomes to use them as input that allow s us to evaluate the implementation of synchronous e-mental health. Therefore, we do not perform a quantitative synthesis in any case (i.e., a meta-analysis of effectiveness).”

11. If you are following a "qualitative method approach" why are you including outcomes of "all kinds of statistical measures of the effects"? What will you do with this information?
Reply: We agree with the reviewer. The quantitative information was eliminated. It was modified in the method:
“Our study seeks to perform a realist synthesis of the evidence, so we focused on different outcomes to use them as input. Therefore, we do not perform a quantitative synthesis in any case (i.e., a meta-analysis of effectiveness).”

12. How do you define "conducted an adequate search"? If you are using criteria then please list them.
Reply: We agree with your observation. We have specified and listed the criteria we consider to include a systematic review “Systematic reviews using at least two databases that synthesized, assessed the quality, and presented sufficient detail of the primary included studies; and reports on inclusion/exclusion criteria”

13. Please provided references for "only the last updated systematic reviews".
Reply: It was specified in the text:
“We include this time frame to include only the latest systematic reviews since in the field of e-health, the launch of new technologies makes scientific development dynamic. “

14. The study selection process is overly detailed and repetitive.
Reply: We synthesized the study selection:
“In the review by title and abstract, calibration will be done first, 50 records were selected from the total number of records found as a result of the search strategy. In the full-text review, calibration will first be performed on 10 records that will be selected in the previous stage. The articles excluded in the full-text review stage will be listed, and a reason for exclusion will be given for each of them.”

15. You say, "An extraction form will be created for the included systematic reviews.", but above, you imply it is already done and included in "an Excel template (Extended data)".
Reply: It was modified in the subsection of data items:
“An extraction form was created in an excel template for the included systematic reviews (https://osf.io/7esgz/).”

16. "Therefore, we do not look for a specific result such as effectiveness, cost-effectiveness or other similar ones." Then why are you searching for data?
Reply: It was modified in the method:
“Our study seeks to perform a realist synthesis of the evidence, so we focused on different outcomes to use them as input. Therefore, we do not perform a quantitative synthesis in any case (i.e., a meta-analysis of effectiveness).”
The section where we state that we will collect specific quantitative data, such as effect size, has been removed. Studies with efficacy, cost-effectiveness, or other similar outcomes are included because they will help answer our question.

17. "perform a grounded theory analysis with an emergent approach" - you said earlier, "using an emergent grounded theory approach", what is the difference? If no difference, then please use consistent terms.
Reply: It was specified:
“Our study aims to conduct a systematic review of systematic reviews and a realistic synthesis of systematic reviews, using a qualitative strategy to synthesize the information and answer our research question. Therefore, we do not look for specific results such as effectiveness, cost-effectiveness, or other similar ones. Instead, we are interested in identifying the full text of all the studies that answer our research question so that we can make a realistic synthesis using a grounded theory model with an emergent approach. The realistic synthesis will be based on the interpretation, integration, and inference of the evaluated elements and the generation of hypotheses to better understand the studied phenomenon (implementation of synchronous e-mental health) (20, 21).”

18. "Priority will be given in the analysis of those studies with the lowest risk of bias assessed." - Explain why this is the case
Reply: It was specified:
“Priority will be given in the analysis of those studies with the lowest risk of bias assessed. If possible, a realist synthesis will be performed only with those systematic reviews with high quality, to assess whether the results change by selecting only those studies with high quality (low risk of bias).”

19. In data synthesis, you do not state how many "reviewers will follow the three steps".
Reply: We agree with your observation. We have specified the number of reviewers involved in this process. “Two reviewers will follow the three steps established by Thomas and Harden for qualitative syntheses will be followed.”

20. In the abstract, you say you will "develop a theoretical framework" but you do not give detail of how this will be done in the manuscript.
Reply: The authors have identified that the meta-synthesis analysis method was not the most appropriate for the review, as the description is more in line with a realistic synthesis analysis. It was specified in the section on data synthesis.
“The develop a framework informed by a realist analysis for the implementation of synchronous e-mental health will be carried out using a grounded theory approach with an emergent approach (23). The realist synthesis will be based on interpreting, integrating and, inferring the evaluation elements that would allow a better understanding of the synchronic e-mental health implementation process from all the studies included (24). To answer the question "what makes the implementation of these interventions work", hypotheses supported by the results of the included studies will be developed and generated through discussion and consensus among researchers (24). Two reviewers will follow the three steps established by Thomas and Harden for qualitative syntheses will be followed (25). First, the extracted data are freely coded. The reviewers read the full texts of the included articles and code each text fragment that provides information to answer the research question. Second, the codified data are organized and grouped based on the descriptive aspects. The reviewers will review the codes generated in the previous step and group them into codes that are like each other and that allow the description of a part of the research question. Thirdly, the analytical concepts generated in the previous step will be grouped in a way that they are related to each other. The elements that are related to each other will be assumed to be part of a hypothesis that would help to answer the research aim. The sum of all these hypotheses will be referred to as a framework that will attempt to explain the results using a context-linked causality approach represented as context + mechanism = outcome (26). ”

21. Please also proofread for typographical errors and appropriate use of English.
Reply: We accept your observation. We have reviewed and corrected typographical errors and the adequate use of English.
Competing Interests: None Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 03 Aug 2021

Christoper A. Alarcon-Ruiz, Universidad Científica del Sur, Lima, Peru

03 Aug 2021

Author Response

Abstract:
1. "Some quantitative systematic" can you indicate how many rather than using the generic some.
Reply: We specified this concern. Abstract: “Quantitative systematic reviews such as meta-analysis”

2. ... Continue reading Abstract:
1. "Some quantitative systematic" can you indicate how many rather than using the generic some.
Reply: We specified this concern. Abstract: “Quantitative systematic reviews such as meta-analysis”

2. Would the research question be better phrased as 2 questions? or a primary and secondary question? i) how question?; ii) what makes question?
Reply: It was modified in the abstract:
“Thus, our study has two main objectives: a) "to determine what evidence is available for synchronous e-mental health implementation"; and b) "to develop a framework informed by a realistic analysis for the implementation of synchronous e-mental health".”

3. Please define what approach you will use "We will follow a qualitative method approach".
Reply: It was modified in the abstract:
“A systematic review with narrative description and a realist synthesis will be conducted. Primary studies related to adults with common mental health problems using any type of mobile mental health intervention that includes a synchronous component and communication with a mental health professional will be included. For the analysis, we will make a realistic synthesis of the systematic reviews, using a grounded theory approach with an emergent approach to synthesize the information, prioritizing the systematic reviews with a lower risk of bias in the AMSTAR-2 tool. The realist synthesis will be based on the interpretation, integration, and inference of the evaluated elements and the generation of hypotheses to better understand the implementation process of synchronic e-mental health.”

4. Are "incorporate technologies", "implementation of electronic care", "mobile mental health intervention that includes a synchronic component and communication with a mental health professional", "e-health implementation process" and, "synchronic component (that lack real-time information exchange using technologies)." all the same thing? I would suggest you select one terminology and use it consistently.
Reply: We’ll use the term “e-mental health” to refer to mental health care delivery through technology. And we’ll use “synchronous e-mental health” to refer to the delivery of mental health care through technology in real-time.

Introduction:
5. Please remove "To deal with these mental health problems". This is not appropriate terminology.
Reply: We changed it for “As a result, it has been...”

6. You talk about "These technologies", can you provide examples?
Reply: We specified some examples: “such as internet and mobile devices ”

7. You say "(centered on quantitative studies) does not allow to identify". Describe what it does not allow; what is missing from existing reviews?
Reply: Does not provide information about experience, meaning, and perspective, which is important when addressing implementation issues.
“However, qualitative studies are necessary because of their methodology (approach to answer questions about experience, meaning, and perspective) which is useful to address issues of implementation.”

Methods:

8. The heading, "SPIDER question", needs to be replaced with "research question". The acronym may not be understood by all.
Reply: The heading has been modified.

9. Regarding the sample - do people have to have a certified diagnosis of the condition? What about undiagnosed groups?
Reply: It was modified in the methods:
“Sample: Adults with depression (or major depressive disorder), anxiety (or generalized anxiety disorder), stress (or trauma-related disorders), and/or general mental health problems (unspecified). Participants may be diagnosed through clinical interviews or categorized based on screening assessments (self-reported scales).”

Evaluation:
10. Are you assessing effectiveness? or just implementation of a technology that has shown to be effective elsewhere?
Reply: It was modified in the method:
“All types of outcomes of interest assessed by implementation, economic, qualitative, quantitative, and other studies will be included. For example, a) Health effectiveness outcomes: Depression, anxiety and/or stress symptoms, adherence to treatment, etc.; b) Patient outcomes: Quality of life, satisfaction, etc.; c) Economic outcomes; d) Damage or adverse effects. Our study seeks to perform a realist synthesis of the evidence, so we focused on different outcomes to use them as input that allow s us to evaluate the implementation of synchronous e-mental health. Therefore, we do not perform a quantitative synthesis in any case (i.e., a meta-analysis of effectiveness).”

11. If you are following a "qualitative method approach" why are you including outcomes of "all kinds of statistical measures of the effects"? What will you do with this information?
Reply: We agree with the reviewer. The quantitative information was eliminated. It was modified in the method:
“Our study seeks to perform a realist synthesis of the evidence, so we focused on different outcomes to use them as input. Therefore, we do not perform a quantitative synthesis in any case (i.e., a meta-analysis of effectiveness).”

12. How do you define "conducted an adequate search"? If you are using criteria then please list them.
Reply: We agree with your observation. We have specified and listed the criteria we consider to include a systematic review “Systematic reviews using at least two databases that synthesized, assessed the quality, and presented sufficient detail of the primary included studies; and reports on inclusion/exclusion criteria”

13. Please provided references for "only the last updated systematic reviews".
Reply: It was specified in the text:
“We include this time frame to include only the latest systematic reviews since in the field of e-health, the launch of new technologies makes scientific development dynamic. “

14. The study selection process is overly detailed and repetitive.
Reply: We synthesized the study selection:
“In the review by title and abstract, calibration will be done first, 50 records were selected from the total number of records found as a result of the search strategy. In the full-text review, calibration will first be performed on 10 records that will be selected in the previous stage. The articles excluded in the full-text review stage will be listed, and a reason for exclusion will be given for each of them.”

15. You say, "An extraction form will be created for the included systematic reviews.", but above, you imply it is already done and included in "an Excel template (Extended data)".
Reply: It was modified in the subsection of data items:
“An extraction form was created in an excel template for the included systematic reviews (https://osf.io/7esgz/).”

16. "Therefore, we do not look for a specific result such as effectiveness, cost-effectiveness or other similar ones." Then why are you searching for data?
Reply: It was modified in the method:
“Our study seeks to perform a realist synthesis of the evidence, so we focused on different outcomes to use them as input. Therefore, we do not perform a quantitative synthesis in any case (i.e., a meta-analysis of effectiveness).”
The section where we state that we will collect specific quantitative data, such as effect size, has been removed. Studies with efficacy, cost-effectiveness, or other similar outcomes are included because they will help answer our question.

17. "perform a grounded theory analysis with an emergent approach" - you said earlier, "using an emergent grounded theory approach", what is the difference? If no difference, then please use consistent terms.
Reply: It was specified:
“Our study aims to conduct a systematic review of systematic reviews and a realistic synthesis of systematic reviews, using a qualitative strategy to synthesize the information and answer our research question. Therefore, we do not look for specific results such as effectiveness, cost-effectiveness, or other similar ones. Instead, we are interested in identifying the full text of all the studies that answer our research question so that we can make a realistic synthesis using a grounded theory model with an emergent approach. The realistic synthesis will be based on the interpretation, integration, and inference of the evaluated elements and the generation of hypotheses to better understand the studied phenomenon (implementation of synchronous e-mental health) (20, 21).”

18. "Priority will be given in the analysis of those studies with the lowest risk of bias assessed." - Explain why this is the case
Reply: It was specified:
“Priority will be given in the analysis of those studies with the lowest risk of bias assessed. If possible, a realist synthesis will be performed only with those systematic reviews with high quality, to assess whether the results change by selecting only those studies with high quality (low risk of bias).”

19. In data synthesis, you do not state how many "reviewers will follow the three steps".
Reply: We agree with your observation. We have specified the number of reviewers involved in this process. “Two reviewers will follow the three steps established by Thomas and Harden for qualitative syntheses will be followed.”

20. In the abstract, you say you will "develop a theoretical framework" but you do not give detail of how this will be done in the manuscript.
Reply: The authors have identified that the meta-synthesis analysis method was not the most appropriate for the review, as the description is more in line with a realistic synthesis analysis. It was specified in the section on data synthesis.
“The develop a framework informed by a realist analysis for the implementation of synchronous e-mental health will be carried out using a grounded theory approach with an emergent approach (23). The realist synthesis will be based on interpreting, integrating and, inferring the evaluation elements that would allow a better understanding of the synchronic e-mental health implementation process from all the studies included (24). To answer the question "what makes the implementation of these interventions work", hypotheses supported by the results of the included studies will be developed and generated through discussion and consensus among researchers (24). Two reviewers will follow the three steps established by Thomas and Harden for qualitative syntheses will be followed (25). First, the extracted data are freely coded. The reviewers read the full texts of the included articles and code each text fragment that provides information to answer the research question. Second, the codified data are organized and grouped based on the descriptive aspects. The reviewers will review the codes generated in the previous step and group them into codes that are like each other and that allow the description of a part of the research question. Thirdly, the analytical concepts generated in the previous step will be grouped in a way that they are related to each other. The elements that are related to each other will be assumed to be part of a hypothesis that would help to answer the research aim. The sum of all these hypotheses will be referred to as a framework that will attempt to explain the results using a context-linked causality approach represented as context + mechanism = outcome (26). ”

21. Please also proofread for typographical errors and appropriate use of English.
Reply: We accept your observation. We have reviewed and corrected typographical errors and the adequate use of English.
Abstract:
1. "Some quantitative systematic" can you indicate how many rather than using the generic some.
Reply: We specified this concern. Abstract: “Quantitative systematic reviews such as meta-analysis”

2. Would the research question be better phrased as 2 questions? or a primary and secondary question? i) how question?; ii) what makes question?
Reply: It was modified in the abstract:
“Thus, our study has two main objectives: a) "to determine what evidence is available for synchronous e-mental health implementation"; and b) "to develop a framework informed by a realistic analysis for the implementation of synchronous e-mental health".”

3. Please define what approach you will use "We will follow a qualitative method approach".
Reply: It was modified in the abstract:
“A systematic review with narrative description and a realist synthesis will be conducted. Primary studies related to adults with common mental health problems using any type of mobile mental health intervention that includes a synchronous component and communication with a mental health professional will be included. For the analysis, we will make a realistic synthesis of the systematic reviews, using a grounded theory approach with an emergent approach to synthesize the information, prioritizing the systematic reviews with a lower risk of bias in the AMSTAR-2 tool. The realist synthesis will be based on the interpretation, integration, and inference of the evaluated elements and the generation of hypotheses to better understand the implementation process of synchronic e-mental health.”

4. Are "incorporate technologies", "implementation of electronic care", "mobile mental health intervention that includes a synchronic component and communication with a mental health professional", "e-health implementation process" and, "synchronic component (that lack real-time information exchange using technologies)." all the same thing? I would suggest you select one terminology and use it consistently.
Reply: We’ll use the term “e-mental health” to refer to mental health care delivery through technology. And we’ll use “synchronous e-mental health” to refer to the delivery of mental health care through technology in real-time.

Introduction:
5. Please remove "To deal with these mental health problems". This is not appropriate terminology.
Reply: We changed it for “As a result, it has been...”

6. You talk about "These technologies", can you provide examples?
Reply: We specified some examples: “such as internet and mobile devices ”

7. You say "(centered on quantitative studies) does not allow to identify". Describe what it does not allow; what is missing from existing reviews?
Reply: Does not provide information about experience, meaning, and perspective, which is important when addressing implementation issues.
“However, qualitative studies are necessary because of their methodology (approach to answer questions about experience, meaning, and perspective) which is useful to address issues of implementation.”

Methods:

8. The heading, "SPIDER question", needs to be replaced with "research question". The acronym may not be understood by all.
Reply: The heading has been modified.

9. Regarding the sample - do people have to have a certified diagnosis of the condition? What about undiagnosed groups?
Reply: It was modified in the methods:
“Sample: Adults with depression (or major depressive disorder), anxiety (or generalized anxiety disorder), stress (or trauma-related disorders), and/or general mental health problems (unspecified). Participants may be diagnosed through clinical interviews or categorized based on screening assessments (self-reported scales).”

Evaluation:
10. Are you assessing effectiveness? or just implementation of a technology that has shown to be effective elsewhere?
Reply: It was modified in the method:
“All types of outcomes of interest assessed by implementation, economic, qualitative, quantitative, and other studies will be included. For example, a) Health effectiveness outcomes: Depression, anxiety and/or stress symptoms, adherence to treatment, etc.; b) Patient outcomes: Quality of life, satisfaction, etc.; c) Economic outcomes; d) Damage or adverse effects. Our study seeks to perform a realist synthesis of the evidence, so we focused on different outcomes to use them as input that allow s us to evaluate the implementation of synchronous e-mental health. Therefore, we do not perform a quantitative synthesis in any case (i.e., a meta-analysis of effectiveness).”

11. If you are following a "qualitative method approach" why are you including outcomes of "all kinds of statistical measures of the effects"? What will you do with this information?
Reply: We agree with the reviewer. The quantitative information was eliminated. It was modified in the method:
“Our study seeks to perform a realist synthesis of the evidence, so we focused on different outcomes to use them as input. Therefore, we do not perform a quantitative synthesis in any case (i.e., a meta-analysis of effectiveness).”

12. How do you define "conducted an adequate search"? If you are using criteria then please list them.
Reply: We agree with your observation. We have specified and listed the criteria we consider to include a systematic review “Systematic reviews using at least two databases that synthesized, assessed the quality, and presented sufficient detail of the primary included studies; and reports on inclusion/exclusion criteria”

13. Please provided references for "only the last updated systematic reviews".
Reply: It was specified in the text:
“We include this time frame to include only the latest systematic reviews since in the field of e-health, the launch of new technologies makes scientific development dynamic. “

14. The study selection process is overly detailed and repetitive.
Reply: We synthesized the study selection:
“In the review by title and abstract, calibration will be done first, 50 records were selected from the total number of records found as a result of the search strategy. In the full-text review, calibration will first be performed on 10 records that will be selected in the previous stage. The articles excluded in the full-text review stage will be listed, and a reason for exclusion will be given for each of them.”

15. You say, "An extraction form will be created for the included systematic reviews.", but above, you imply it is already done and included in "an Excel template (Extended data)".
Reply: It was modified in the subsection of data items:
“An extraction form was created in an excel template for the included systematic reviews (https://osf.io/7esgz/).”

16. "Therefore, we do not look for a specific result such as effectiveness, cost-effectiveness or other similar ones." Then why are you searching for data?
Reply: It was modified in the method:
“Our study seeks to perform a realist synthesis of the evidence, so we focused on different outcomes to use them as input. Therefore, we do not perform a quantitative synthesis in any case (i.e., a meta-analysis of effectiveness).”
The section where we state that we will collect specific quantitative data, such as effect size, has been removed. Studies with efficacy, cost-effectiveness, or other similar outcomes are included because they will help answer our question.

17. "perform a grounded theory analysis with an emergent approach" - you said earlier, "using an emergent grounded theory approach", what is the difference? If no difference, then please use consistent terms.
Reply: It was specified:
“Our study aims to conduct a systematic review of systematic reviews and a realistic synthesis of systematic reviews, using a qualitative strategy to synthesize the information and answer our research question. Therefore, we do not look for specific results such as effectiveness, cost-effectiveness, or other similar ones. Instead, we are interested in identifying the full text of all the studies that answer our research question so that we can make a realistic synthesis using a grounded theory model with an emergent approach. The realistic synthesis will be based on the interpretation, integration, and inference of the evaluated elements and the generation of hypotheses to better understand the studied phenomenon (implementation of synchronous e-mental health) (20, 21).”

18. "Priority will be given in the analysis of those studies with the lowest risk of bias assessed." - Explain why this is the case
Reply: It was specified:
“Priority will be given in the analysis of those studies with the lowest risk of bias assessed. If possible, a realist synthesis will be performed only with those systematic reviews with high quality, to assess whether the results change by selecting only those studies with high quality (low risk of bias).”

19. In data synthesis, you do not state how many "reviewers will follow the three steps".
Reply: We agree with your observation. We have specified the number of reviewers involved in this process. “Two reviewers will follow the three steps established by Thomas and Harden for qualitative syntheses will be followed.”

20. In the abstract, you say you will "develop a theoretical framework" but you do not give detail of how this will be done in the manuscript.
Reply: The authors have identified that the meta-synthesis analysis method was not the most appropriate for the review, as the description is more in line with a realistic synthesis analysis. It was specified in the section on data synthesis.
“The develop a framework informed by a realist analysis for the implementation of synchronous e-mental health will be carried out using a grounded theory approach with an emergent approach (23). The realist synthesis will be based on interpreting, integrating and, inferring the evaluation elements that would allow a better understanding of the synchronic e-mental health implementation process from all the studies included (24). To answer the question "what makes the implementation of these interventions work", hypotheses supported by the results of the included studies will be developed and generated through discussion and consensus among researchers (24). Two reviewers will follow the three steps established by Thomas and Harden for qualitative syntheses will be followed (25). First, the extracted data are freely coded. The reviewers read the full texts of the included articles and code each text fragment that provides information to answer the research question. Second, the codified data are organized and grouped based on the descriptive aspects. The reviewers will review the codes generated in the previous step and group them into codes that are like each other and that allow the description of a part of the research question. Thirdly, the analytical concepts generated in the previous step will be grouped in a way that they are related to each other. The elements that are related to each other will be assumed to be part of a hypothesis that would help to answer the research aim. The sum of all these hypotheses will be referred to as a framework that will attempt to explain the results using a context-linked causality approach represented as context + mechanism = outcome (26). ”

21. Please also proofread for typographical errors and appropriate use of English.
Reply: We accept your observation. We have reviewed and corrected typographical errors and the adequate use of English.
Competing Interests: None Close
Report a concern

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 29 Oct 2020

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 2 (revision) 03 Aug 21	read	read
Version 1 29 Oct 20	read	read

Amy Grove, University of Warwick, Coventry, UK
Roger Hilfiker, University of Applied Sciences and Arts Western Switzerland, Valais, Switzerland

Comments on this article

All Comments(0)

Add a comment

Browse by related subjects

Back to all reports

Reviewer Report

3 Views

07 Sep 2021 | for Version 2

Roger Hilfiker, Valais-Wallis School of Health Sciences, HES-SO, University of Applied Sciences and Arts Western Switzerland, Valais, Switzerland

3 Views Cite this report Responses(0)

Approved

Thank you very much for the changes made.

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Systematic Reviews

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

6 Views

17 Aug 2021 | for Version 2

Amy Grove, Health Technology Assessment and Implementation Science, Warwick Medical School, University of Warwick, Coventry, UK

6 Views Cite this report Responses(0)

Approved

I am satisfied with the revisions.

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

HTA and implementation science

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

14 Views

18 Jun 2021 | for Version 1

Roger Hilfiker, Valais-Wallis School of Health Sciences, HES-SO, University of Applied Sciences and Arts Western Switzerland, Valais, Switzerland

14 Views Cite this report Responses(1)

Approved With Reservations

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Yes

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Systematic Reviews

Respond to this report

Responses (1)

Author Response

03 Aug 2021

Christoper A. Alarcon-Ruiz, Universidad Científica del Sur, Lima, Peru

1. Research question: Please clarify how you will use the data to answer the question on "what makes the application of any of these interventions work".
Reply: It was specified in the subsection of Data synthesis
“The develop a framework informed by a realist analysis for the implementation of synchronous e-mental health will be carried out using a grounded theory approach with an emergent approach (23). The realist synthesis will be based on interpreting, integrating and, inferring the evaluation elements that would allow a better understanding of the synchronous e-mental health implementation process from all the studies included (24). To answer the question "what makes the implementation of these interventions work", hypotheses supported by the results of the included studies will be developed and generated through discussion and consensus among researchers (24). Two reviewers will follow the three steps established by Thomas and Harden for qualitative syntheses will be followed (25). First, the extracted data are freely coded. The reviewers read the full-texts of the included articles and code each text fragment that provides information to answer the research question. Second, the codified data are organized and grouped based on the descriptive aspects. The reviewers will review the codes generated in the previous step and group them into codes that are like each other and that allow the description of a part of the research question. Thirdly, the analytical concepts generated in the previous step will be grouped in a way that they are related to each other. The elements that are related to each other will be assumed to be part of a hypothesis that would help to answer the research aim. The sum of all these hypotheses will be referred to as a framework which will attempt to explain the results using a context-linked causality approach represented as context + mechanism = outcome (26).”

2. Data synthesis: The sentence "In addition to generate hypotheses supported by the results of the included studies, which will be generated through discussion and consensus among the reviewers" is not clear to me.
Reply: We agree with the reviewer, this has been modified in the previous comment.

3. The authors described the extraction of quantitative data on effectiveness. Please clarify whether these quantitative data are analyzed with the qualitative method or whether quantitative methods are used.
Reply: The collection of quantitative data was eliminated, as specified in the method section:
“Our study seeks to perform a realistic synthesis of the evidence, so we focused on different outcomes to use them as input. Therefore, we do not perform a quantitative synthesis in any case (i.e., a meta-analysis of effectiveness).”

4. Please describe how the research question on the "what makes the application work" will be answered.
Reply: A better description of the research objective was made based on the first reviewer.
“Thus, our study has two main objectives: a) "to determine what evidence is available for synchronous e-mental health implementation"; and b) "to develop a framework informed by a realist analysis for the implementation of synchronous e-mental health".”
The process of how to achieve these objectives was answered in the first comment.

5. Confidence in cumulative evidence: Wouldn't the research question "what makes the application of any of these interventions work" require a different evaluation of the confidence of the evidence, for example an approach such as GRADE?
Reply: we will evaluate each key finding using CERQual to assess the methodological limitations, coherence, adequacy, and relevance of findings.
“It will be evaluated the approach of Confidence in the Evidence from Reviews of Qualitative research (CERQual) which has four components (Methodological Limitations, Relevance, Coherence and Adequacy data), will be evaluated to contribute to an overall assessment for each hypothesis resulting from the realist synthesis to determine the level of confidence (high, moderate, low, or very low) and present the overall assessment in a Summary of Qualitative Findings (SoQF) table (27, 28).”

View more View less

Competing Interests

None

Back to all reports

Reviewer Report

32 Views

11 Feb 2021 | for Version 1

Amy Grove, Health Technology Assessment and Implementation Science, Warwick Medical School, University of Warwick, Coventry, UK

32 Views Cite this report Responses(1)

Approved With Reservations

Abstract:

"Some quantitative systematic" can you indicate how many rather than using the generic some.
Would the research question be better phrased as 2 questions? or a primary and secondary question? i) how question?; ii) what makes question?
Please define what approach you will use "We will follow a qualitative method approach".
Are "incorporate technologies", "implementation of electronic care", "mobile mental health intervention that includes a synchronic component and communication with a mental health professional", "e-health implementation process" and, "synchronic component (that lack real time information exchange using technologies)." all the same thing? I would suggest you select one terminology and use it consistently.

Introduction:

Please remove "To deal with these mental health problems". This is not appropriate terminology.
You talk about "These technologies", can you provide examples?
You say "(centered on quantitative studies) does not allow to identify". Describe what it does not allow; what is missing from existing reviews?

Methods:

The heading, "SPIDER question", needs to be replaced with "research question". The acronym may not be understood by all.
Regarding the sample - do people have to have a certified diagnosis of the condition? What about undiagnosed groups?

Evaluation:

Are you assessing effectiveness? or just implementation of a technology that has shown to be effective elsewhere?
If you are following a "qualitative method approach" why are you including outcomes of "all kinds of statistical measures of the effects"? What will you do with this information?
How do you define "conducted an adequate search"? If you are using a criteria then please list it.
Please provided references for "only the last updated systematic reviews".
The study selection process is overly detailed and repetitive.
You say "An extraction form will be created for the included systematic reviews.", but above you imply it is already done and included in "an Excel template (Extended data)".
"Therefore, we do not look for a specific result such as effectiveness, cost-effectiveness or other similar ones." Then why are you searching for data?
"perform a grounded theory analysis with an emergent approach" - you said earlier "using an emergent grounded theory approach", what is the difference? If no difference, then please use consistent terms.
"Priority will be given in the analysis of those studies with the lowest risk of bias assessed." - Explain why this is the case
In data synthesis you do not state how many "reviewers will follow the three steps".
In the abstract you say you will "develop a theoretical framework" but you do not give detail of how this will be done in the manuscript.
Please also proof read for typographical errors and appropriate use of English.

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Yes

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

HTA and implementation science

Respond to this report

Responses (1)

Author Response

03 Aug 2021

Christoper A. Alarcon-Ruiz, Universidad Científica del Sur, Lima, Peru

Abstract:
1. "Some quantitative systematic" can you indicate how many rather than using the generic some.
Reply: We specified this concern. Abstract: “Quantitative systematic reviews such as meta-analysis”

2. Would the research question be better phrased as 2 questions? or a primary and secondary question? i) how question?; ii) what makes question?
Reply: It was modified in the abstract:
“Thus, our study has two main objectives: a) "to determine what evidence is available for synchronous e-mental health implementation"; and b) "to develop a framework informed by a realistic analysis for the implementation of synchronous e-mental health".”

3. Please define what approach you will use "We will follow a qualitative method approach".
Reply: It was modified in the abstract:
“A systematic review with narrative description and a realist synthesis will be conducted. Primary studies related to adults with common mental health problems using any type of mobile mental health intervention that includes a synchronous component and communication with a mental health professional will be included. For the analysis, we will make a realistic synthesis of the systematic reviews, using a grounded theory approach with an emergent approach to synthesize the information, prioritizing the systematic reviews with a lower risk of bias in the AMSTAR-2 tool. The realist synthesis will be based on the interpretation, integration, and inference of the evaluated elements and the generation of hypotheses to better understand the implementation process of synchronic e-mental health.”

4. Are "incorporate technologies", "implementation of electronic care", "mobile mental health intervention that includes a synchronic component and communication with a mental health professional", "e-health implementation process" and, "synchronic component (that lack real-time information exchange using technologies)." all the same thing? I would suggest you select one terminology and use it consistently.
Reply: We’ll use the term “e-mental health” to refer to mental health care delivery through technology. And we’ll use “synchronous e-mental health” to refer to the delivery of mental health care through technology in real-time.

Introduction:
5. Please remove "To deal with these mental health problems". This is not appropriate terminology.
Reply: We changed it for “As a result, it has been...”

6. You talk about "These technologies", can you provide examples?
Reply: We specified some examples: “such as internet and mobile devices ”

7. You say "(centered on quantitative studies) does not allow to identify". Describe what it does not allow; what is missing from existing reviews?
Reply: Does not provide information about experience, meaning, and perspective, which is important when addressing implementation issues.
“However, qualitative studies are necessary because of their methodology (approach to answer questions about experience, meaning, and perspective) which is useful to address issues of implementation.”

Methods:

8. The heading, "SPIDER question", needs to be replaced with "research question". The acronym may not be understood by all.
Reply: The heading has been modified.

9. Regarding the sample - do people have to have a certified diagnosis of the condition? What about undiagnosed groups?
Reply: It was modified in the methods:
“Sample: Adults with depression (or major depressive disorder), anxiety (or generalized anxiety disorder), stress (or trauma-related disorders), and/or general mental health problems (unspecified). Participants may be diagnosed through clinical interviews or categorized based on screening assessments (self-reported scales).”

Evaluation:
10. Are you assessing effectiveness? or just implementation of a technology that has shown to be effective elsewhere?
Reply: It was modified in the method:
“All types of outcomes of interest assessed by implementation, economic, qualitative, quantitative, and other studies will be included. For example, a) Health effectiveness outcomes: Depression, anxiety and/or stress symptoms, adherence to treatment, etc.; b) Patient outcomes: Quality of life, satisfaction, etc.; c) Economic outcomes; d) Damage or adverse effects. Our study seeks to perform a realist synthesis of the evidence, so we focused on different outcomes to use them as input that allow s us to evaluate the implementation of synchronous e-mental health. Therefore, we do not perform a quantitative synthesis in any case (i.e., a meta-analysis of effectiveness).”

11. If you are following a "qualitative method approach" why are you including outcomes of "all kinds of statistical measures of the effects"? What will you do with this information?
Reply: We agree with the reviewer. The quantitative information was eliminated. It was modified in the method:
“Our study seeks to perform a realist synthesis of the evidence, so we focused on different outcomes to use them as input. Therefore, we do not perform a quantitative synthesis in any case (i.e., a meta-analysis of effectiveness).”

12. How do you define "conducted an adequate search"? If you are using criteria then please list them.
Reply: We agree with your observation. We have specified and listed the criteria we consider to include a systematic review “Systematic reviews using at least two databases that synthesized, assessed the quality, and presented sufficient detail of the primary included studies; and reports on inclusion/exclusion criteria”

13. Please provided references for "only the last updated systematic reviews".
Reply: It was specified in the text:
“We include this time frame to include only the latest systematic reviews since in the field of e-health, the launch of new technologies makes scientific development dynamic. “

14. The study selection process is overly detailed and repetitive.
Reply: We synthesized the study selection:
“In the review by title and abstract, calibration will be done first, 50 records were selected from the total number of records found as a result of the search strategy. In the full-text review, calibration will first be performed on 10 records that will be selected in the previous stage. The articles excluded in the full-text review stage will be listed, and a reason for exclusion will be given for each of them.”

15. You say, "An extraction form will be created for the included systematic reviews.", but above, you imply it is already done and included in "an Excel template (Extended data)".
Reply: It was modified in the subsection of data items:
“An extraction form was created in an excel template for the included systematic reviews (https://osf.io/7esgz/).”

16. "Therefore, we do not look for a specific result such as effectiveness, cost-effectiveness or other similar ones." Then why are you searching for data?
Reply: It was modified in the method:
“Our study seeks to perform a realist synthesis of the evidence, so we focused on different outcomes to use them as input. Therefore, we do not perform a quantitative synthesis in any case (i.e., a meta-analysis of effectiveness).”
The section where we state that we will collect specific quantitative data, such as effect size, has been removed. Studies with efficacy, cost-effectiveness, or other similar outcomes are included because they will help answer our question.

17. "perform a grounded theory analysis with an emergent approach" - you said earlier, "using an emergent grounded theory approach", what is the difference? If no difference, then please use consistent terms.
Reply: It was specified:
“Our study aims to conduct a systematic review of systematic reviews and a realistic synthesis of systematic reviews, using a qualitative strategy to synthesize the information and answer our research question. Therefore, we do not look for specific results such as effectiveness, cost-effectiveness, or other similar ones. Instead, we are interested in identifying the full text of all the studies that answer our research question so that we can make a realistic synthesis using a grounded theory model with an emergent approach. The realistic synthesis will be based on the interpretation, integration, and inference of the evaluated elements and the generation of hypotheses to better understand the studied phenomenon (implementation of synchronous e-mental health) (20, 21).”

18. "Priority will be given in the analysis of those studies with the lowest risk of bias assessed." - Explain why this is the case
Reply: It was specified:
“Priority will be given in the analysis of those studies with the lowest risk of bias assessed. If possible, a realist synthesis will be performed only with those systematic reviews with high quality, to assess whether the results change by selecting only those studies with high quality (low risk of bias).”

19. In data synthesis, you do not state how many "reviewers will follow the three steps".
Reply: We agree with your observation. We have specified the number of reviewers involved in this process. “Two reviewers will follow the three steps established by Thomas and Harden for qualitative syntheses will be followed.”

20. In the abstract, you say you will "develop a theoretical framework" but you do not give detail of how this will be done in the manuscript.
Reply: The authors have identified that the meta-synthesis analysis method was not the most appropriate for the review, as the description is more in line with a realistic synthesis analysis. It was specified in the section on data synthesis.
“The develop a framework informed by a realist analysis for the implementation of synchronous e-mental health will be carried out using a grounded theory approach with an emergent approach (23). The realist synthesis will be based on interpreting, integrating and, inferring the evaluation elements that would allow a better understanding of the synchronic e-mental health implementation process from all the studies included (24). To answer the question "what makes the implementation of these interventions work", hypotheses supported by the results of the included studies will be developed and generated through discussion and consensus among researchers (24). Two reviewers will follow the three steps established by Thomas and Harden for qualitative syntheses will be followed (25). First, the extracted data are freely coded. The reviewers read the full texts of the included articles and code each text fragment that provides information to answer the research question. Second, the codified data are organized and grouped based on the descriptive aspects. The reviewers will review the codes generated in the previous step and group them into codes that are like each other and that allow the description of a part of the research question. Thirdly, the analytical concepts generated in the previous step will be grouped in a way that they are related to each other. The elements that are related to each other will be assumed to be part of a hypothesis that would help to answer the research aim. The sum of all these hypotheses will be referred to as a framework that will attempt to explain the results using a context-linked causality approach represented as context + mechanism = outcome (26). ”

21. Please also proofread for typographical errors and appropriate use of English.
Reply: We accept your observation. We have reviewed and corrected typographical errors and the adequate use of English.

View more View less

Competing Interests

None

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] 1. Vindegaard N, Benros ME: COVID-19 pandemic and mental health consequences: Systematic review of the current evidence. Brain Behav Immun. 2020; 89: 531–542. PubMed Abstract | Publisher Full Text | Free Full Text

[2] 2. Talevi D, Socci V, Carai M, et al.: Mental health outcomes of the CoViD-19 pandemic. Riv Psichiatr. 2020; 55(3): 137–44. PubMed Abstract | Publisher Full Text

[3] 3. Huarcaya-Victoria J: Consideraciones sobre la salud mental en la pandemia de COVID-19. Rev Peru Med Exp Salud Publica. 2020; 37(2): 327–34. PubMed Abstract | Publisher Full Text

[4] 4. Xiong J, Lipsitz O, Nasri F, et al.: Impact of COVID-19 pandemic on mental health in the general population: A systematic review. J Affect Disord. 2020; 277: 55–64. PubMed Abstract | Publisher Full Text | Free Full Text

[5] 5. Sullivan AB, Kane A, Roth AJ, et al.: The COVID-19 Crisis: A Mental Health Perspective and Response Using Telemedicine. J Patient Exp. 2020; 7(3): 295–301. PubMed Abstract | Publisher Full Text | Free Full Text

[6] 6. Haxhihamza K, Arsova S, Bajraktarov S, et al.: Patient Satisfaction with Use of Telemedicine in University Clinic of Psychiatry: Skopje, North Macedonia During COVID-19 Pandemic. Telemed J E Health. 2020. PubMed Abstract | Publisher Full Text

[7] 7. Ruiz EF, Proaño A, Ponce OJ, et al.: [Mobile health for public health in Peru: lessons learned]. Rev Peru Med Exp Salud Publica. 2015; 32(2): 364–72. PubMed Abstract

[8] 8. Miranda JJ, Moscoso MG, Toyama M, et al.: Role of mHealth in overcoming the occurrence of post-stroke depression. Acta Neurol Scand. 2018; 137(1): 12–9. PubMed Abstract | Publisher Full Text | Free Full Text

[9] 9. Andrews G, Cuijpers P, Craske MG, et al.: Computer Therapy for the Anxiety and Depressive Disorders Is Effective, Acceptable and Practical Health Care: A Meta-Analysis. PLoS One. 2010; 5(10): e13196. PubMed Abstract | Publisher Full Text | Free Full Text

[10] 10. Turvey C, Fortney J: The Use of Telemedicine and Mobile Technology to Promote Population Health and Population Management for Psychiatric Disorders. Curr Psychiatry Rep. 2017; 19(11): 88. PubMed Abstract | Publisher Full Text

[11] 11. Kaonga NN, Morgan J: Common themes and emerging trends for the use of technology to support mental health and psychosocial well-being in limited resource settings: A review of the literature. Psychiatry Res. 2019; 281: 112594. PubMed Abstract | Publisher Full Text

[12] 12. Meurk C, Leung J, Hall W, et al.: Establishing and Governing e-Mental Health Care in Australia: A Systematic Review of Challenges and A Call For Policy-Focussed Research. J Med Internet Res. 2016; 18(1): e10. PubMed Abstract | Publisher Full Text | Free Full Text

[13] 13. Ross J, Stevenson F, Lau R, et al.: Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update). Implement Sci. 2016; 11(1): 146. PubMed Abstract | Publisher Full Text | Free Full Text

[14] 14. Griffiths KM, Farrer L, Christensen H: The efficacy of internet interventions for depression and anxiety disorders: a review of randomised controlled trials. Med J Aust. 2010; 192(S11): S4–S11. PubMed Abstract

[15] 15. Senanayake B, Wickramasinghe SI, Chatfield MD, et al.: Effectiveness of text messaging interventions for the management of depression: A systematic review and meta-analysis. J Telemed Telecare. 2019; 25(9): 513–523. PubMed Abstract | Publisher Full Text

[16] 16. Cooke A, Smith D, Booth A: Beyond PICO: The SPIDER Tool for Qualitative Evidence Synthesis. Qual Health Res. 2012; 22(10): 1435–43. PubMed Abstract | Publisher Full Text

[17] 17. Moher D, Shamseer L, Clarke M, et al.: Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015; 4(1): 1. PubMed Abstract | Publisher Full Text | Free Full Text

[18] 18. Villarreal-Zegarra D, Alarcon-Ruiz CA, Melendez-Torres G, et al.: Development of a framework for the implementation of electronic interventions in mental health: A protocol for a meta-synthesis of systematic reviews. 2020. http://www.doi.org/10.17605/OSF.IO/3UH4N

[19] 19. Dissemination CfRa. University of York: Centre for Reviews and Dissemination. [updated 3 June, 2015]. Reference Source

[20] 20. Ouzzani M, Hammady H, Fedorowicz Z, et al.: Rayyan—a web and mobile app for systematic reviews. Syst Rev. 2016; 5(1): 210. PubMed Abstract | Publisher Full Text | Free Full Text

[21] 21. Shea BJ, Reeves BC, Wells G, et al.: AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017; 358: j4008. PubMed Abstract | Publisher Full Text | Free Full Text

[22] 22. Guetterman TC, Babchuk WA, Howell Smith MC, et al.: Contemporary Approaches to Mixed Methods–Grounded Theory Research: A Field-Based Analysis. J Mix Methods Res. 2019; 13(2): 179–95. Publisher Full Text

[23] 23. Thorne S, Jensen L, Kearney MH, et al.: Qualitative metasynthesis: reflections on methodological orientation and ideological agenda. Qual Health Res. 2004; 14(10): 1342–65. PubMed Abstract | Publisher Full Text

[24] 24. Thomas J, Harden A: Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008; 8: 45. PubMed Abstract | Publisher Full Text | Free Full Text

[25] 25. Group TG-CP: A practical guide for decision makers in health and social care and other users. Reference Source

[26] 26. Lewin S, Booth A, Glenton C, et al.: Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series. Implement Sci. 2018; 13(Suppl 1): 2. PubMed Abstract | Publisher Full Text | Free Full Text

Development of a framework for the implementation of electronic interventions in mental health: a protocol for a meta-synthesis of systematic reviews

Abstract

Keywords

Introduction

Methods

SPIDER question

Eligibility criteria

Information sources

Search strategy

Study records

Data items

Outcomes and prioritization

Risk of bias in individual studies

Data synthesis

Confidence in cumulative evidence

Dissemination plan

Study status

Data availability

Underlying data

Extended data

Reporting guidelines

Acknowledgments

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

Competing Interests Policy

Stay Updated