Abstract
Implementation science is an emerging field of research with considerable penetration in physical medicine and less in the fields of mental health and social services. There remains a lack of consensus on methodological approaches to the study of implementation processes and tests of implementation strategies. This paper addresses the need for methods development through a structured review that describes design elements in nine studies testing implementation strategies for evidence-based interventions addressing mental health problems of children in child welfare and child mental health settings. Randomized trial designs were dominant with considerable use of mixed method designs in the nine studies published since 2005. The findings are discussed in reference to the limitations of randomized designs in implementation science and the potential for use of alternative designs.
Similar content being viewed by others
References
Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (in press). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research.
Atkins, M. S., Frazier, S. L., Leathers, S. J., Graczyk, P. A., Talbott, E., Jakobsons, L., et al. (2008). Teacher key opinion leaders and mental health consultation in low-income urban schools. Journal of Consulting and Clinical Psychology, 76(5), 905–908.
Barrett-Connor, E., Grady, D., & Stefanick, M. (2004). The rise and fall of menopausal hormone therapy. Annual Review of Public Health, 26, 115–140.
Barwick, M. A., Peters, J., & Boydell, K. (2009). Getting to uptake: Do communities of practice support the implementation of evidence-based practice? Journal of the Canadian Academy of Child and Adolescent Psychiatry, 18(1), 16–29.
Bert, S. C., Farris, J. R., & Borkowski, J. G. (2008). Parent training: Implementation strategies for adventures in parenting. Journal of Primary Prevention, 29(3), 243–261.
Berwick, D. (2008). The science of improvement. JAMA, 299(10), 1182–1184.
Brown, C. H., & Liao, J. (1999). Principles for designing randomized preventive trials in mental health: An emerging developmental epidemiologic paradigm. American Journal of Community Psychology, 27, 677–714.
Brown, C., Ten Have, T., Jo, B., Dagne, G., Wyman, P., Muthén, B., et al. (2009). Adaptive designs for randomized trials in public health. Annual Review of Public Health, 30, 1–25.
Brown, C. H., Wang, W., Kellam, S. G., Muthén, B. O., Petras, H., Toyinbo, P., et al. (2008). Methods for testing theory and evaluating impact in randomized field trials: Intent-to-treat analyses for integrating the perspectives of person, place, and time. Drug and Alcohol Dependence, S95, S74–S104.
Brown, C. H., Wyman, P. A., Guo, J., & Peña, J. (2006). Dynamic wait-listed designs for randomized trials: New designs for prevention of youth suicide. Clinical Trials, 3, 259–271.
Cable, G. (2001). Enhancing causal interpretations of quality improvement interventions. Quality in Health Care, 10(3), 179–186.
Chamberlain, P., Brown, C. H., Saldana, L., Reid, J., Wang, W., Marsenich, L., et al. (2008). Engaging and recruiting counties in an experiment on implementing evidence-based practice in California. Administration and Policy in Mental Health, 35(4), 250–260.
Collins, L., Murphy, S., Nair, V., & Strecher, V. (2005). A strategy for optimizing and evaluating behavioral interventions. Annals of Behavioral Medicine, 30(1), 65–73.
Concato, J., Shah, N., & Horwitz, R. (2000). Randomized, controlled trials, observational studies, and the hierarchy of research designs. The New England Journal of Medicine, 342(25), 1887–1892.
Cook, T., Shadish, W., & Wong, V. (2008). Three conditions under which experiments and observational studies produce comparable causal estimates: New findings from within-study comparisons. Journal of Policy Analysis and Management, 27(4), 724–750.
Culliton, B. (2006). Extracting knowledge from science: A conversation with Elias Zerhouni. Health Affairs, 25(3), w94–w103.
Duan, N., Braslow, J., Weisz, J., & Wells, K. (2001). Fidelity, adherence, and robustness of interventions. Psychiatric Services, 52(4), 413.
Duan, N., Wells, K., Braslow, J., & Weisz, J. (2002). Randomized encouragement trial: A paradigm for public health intervention evaluation. Unpublished Manuscript.
Flay, B. (1986). Efficacy and effectiveness trials (and other phases of research) in the development of health promotion programs. Preventive Medicine, 15(5), 451–474.
Glasgow, R., Magid, D., Beck, A., Ritzwoller, D., & Estabrooks, P. (2005). Practical clinical trials for translating research to practice: Design and measurement recommendations. Medical Care, 43(6), 551–557.
Glisson, C., & Schoenwald, S. K. (2005). The ARC organizational and community intervention strategy for implementing evidence-based children’s mental health treatments. Mental Health Services Research, 7(4), 243–259.
Green, P., & Krieger, A. (1993). Conjoint analysis with product-positioning applications. Handbooks in Operations Research and Management Science, 5, 467–515.
Green, P., Krieger, A., & Wind, Y. (2001). Thirty years of conjoint analysis: Reflections and prospects. Interfaces, 31(3), 56–73.
Herschell, A. D., McNeil, C. B., Urquiza, A. J., McGrath, J. M., Zebell, N. M., Timmer, S. G., et al. (2009). Evaluation of a treatment manual and workshops for disseminating parent–child interaction therapy. Administration and Policy in Mental Health and Mental Health Services Research, 36(1), 63–81.
Lavori, P., Dawson, R., & Rush, A. (2000). Flexible treatment strategies in chronic disease: Clinical and research implications. Biological Psychiatry, 48(6), 605–614.
Lavori, P., Rush, A., Wisniewski, S., Alpert, J., Fava, M., Kupfer, D., et al. (2001). Strengthening clinical effectiveness trials: Equipoise-stratified randomization. Biological Psychiatry, 50(10), 792–801.
Lochman, J. E., Boxmeyer, C., Powell, N., Qu, L., Wells, K., & Windle, M. (2009). Dissemination of the Coping Power Program: Importance of intensity of counselor training. Journal of Consulting and Clinical Psychology, 77(3), 397–409.
Murphy, S., Lynch, K., Oslin, D., McKay, J., & Ten Have, T. (2007). Developing adaptive treatment strategies in substance abuse research. Drug and Alcohol Dependence, 88, S24–S30.
Murphy, S., Oslin, D., & Ten-Have, T. (2008, June). Convergence: Five committees, one dialogue. Paper presented at the NIMH conference, Crystal City, VA.
National Research Council and Institute of Medicine (Ed.). (2009). Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. Washington, DC: National Academy Press.
Palinkas, L. A., Aarons, G. A., Chorpita, B. F., Hoagwood, K., Landsverk, J., & Weisz, J. R. (2009). Cultural exchange and the implementation of evidence-based practices: Two case studies. Research on Social Work Practice, 19(5), 602–612.
Palinkas, L. A., Schoenwald, S. K., Hoagwood, K., Landsverk, J., Chorpita, B. F., & Weisz, J. R. (2008). An ethnographic study of implementation of evidence-based treatments in child mental health: First steps. Psychiatric Services, 59(7), 738–746.
Pawson, R., & Tilley, N. (1997). Realistic evaluation. London: Sage Publications Ltd.
Poduska, J., Kellam, S. G., Brown, C. H., Ford, C., Keegan, N., Windham, A., et al. (2009). A group randomized control trial of a classroom-based intervention aimed at preventing early risk factors for drug abuse: Integrating effectiveness and implementation research. Implementation Science, 4(56), 1–11.
Proctor, E., Landsverk, J., Aarons, G., Chambers, D., Glisson, C., & Mittman, B. (2009). Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health and Mental Health Services Research, 36(1), 24–34.
Schwartz, D., & Lellouch, J. (2009). Explanatory and pragmatic attitudes in therapeutical trials. Journal of Clinical Epidemiology, 62(5), 499–505.
Shortell, S. M. (2004). Increasing value: A research agenda for addressing the managerial and organizational challenges facing health care delivery in the United States. Medical Care Research and Review, 61, 12S–30S.
Strecher, V., McClure, J., Alexander, G., Chakraborty, B., Nair, V., Konkel, J., et al. (2008). Web-based smoking cessation components and tailoring depth: Results of a randomized trial. American Journal of Preventive Medicine, 34(5), 373–381.
Ten Have, T., Coyne, J., Salzer, M., & Katz, I. (2003a). Research to improve the quality of care for depression: Alternatives to the simple randomized clinical trial* 1. General Hospital Psychiatry, 25(2), 115–123.
Ten Have, T., Joffe, M., & Cary, M. (2003b). Causal logistic models for non-compliance under randomized treatment with univariate binary response. Statistics in Medicine, 22(8), 1255–1283.
Tunis, S., Stryer, D., & Clancy, C. (2003). Practical clinical trials: Increasing the value of clinical research for decision making in clinical and health policy. JAMA, 290(12), 1624–1632.
Walshe, K. (2007). Understanding what works—and why—in quality improvement: The need for theory-driven evaluation. International Journal for Quality in Health Care 19, 57–59.
West, S., Duan, N., Pequegnat, W., Gaist, P., Des Jarlais, D., Holtgrave, D., et al. (2008). Alternatives to the randomized controlled trial. American Journal of Public Health, 98(8), 1359–1366.
Westfall, J., Mold, J., & Fagnan, L. (2007). Practice-based research—“Blue Highways” on the NIH roadmap. JAMA, 297(4), 403–406.
Zerhouni, E. (2003). The NIH roadmap. Science (Washington), 302(5642), 63–72.
Zerhouni, E. (2005). Translational and clinical science—time for a new vision. New England Journal of Medicine, 353(15), 1621–1623.
Acknowledgments
Support for this manuscript was provided by the following grants: P30 MH074678, DA019984, MH040859, DA024370, P30 MH068579, and MH080916.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Landsverk, J., Brown, C.H., Rolls Reutz, J. et al. Design Elements in Implementation Research: A Structured Review of Child Welfare and Child Mental Health Studies. Adm Policy Ment Health 38, 54–63 (2011). https://doi.org/10.1007/s10488-010-0315-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10488-010-0315-y