Abstract
Despite the increased knowledge of the contextual determinants of implementation, the mechanisms leading to implementation success are not well understood. Explicating and testing program theories underlying implementation strategies designed through participatory processes involving intervention users with the expert knowledge of the practice context can lead to a better understanding of strategies that work in a real-world implementation. The purpose of this study was to identify facilitating and interfering contexts and mechanisms in implementing a pilot parenting intervention, Behavior Checker, as a part of routine service delivery in a rural primary care facility in the USA. This study was an organizational case study informed by the realist evaluation paradigm. Multiple methods of data collection were used, including key informant interviews with organizational representatives, a focus group and surveys with healthcare staff, and direct observation. The results indicated that the following mechanisms were central to facilitating and interfering with Behavior Checker implementation in the participating clinic: (1) staff knowledge and confidence; (2) complexity compression; (3) perceptions of sustainability; (4) workflow efficiency; (5) a systematic method of prompting intervention delivery and documentation; (6) capability for ongoing performance evaluation; (7) perception of fit; and (8) innovation-specific and general organizational resources. Understanding mechanisms can contribute to identifying strategies that support successful implementation and sustainable adoption of interventions that may benefit society.
Similar content being viewed by others
References
Barth, R. P., & Liggett-Creel, K. (2014). Common components of parenting programs for children birth to eight years of age involved with child welfare services. Children and Youth Services Review, 40, 6–12.
Cifuentes, M., Davis, M., Fernald, D., Gunn, R., Dickinson, P., & Cohen, D. J. (2015). Electronic health record challenges, workarounds, and solutions observed in practices integrating behavioral health and primary care. The Journal of the American Board of Family Medicine, 28(Supplement 1), S63–S72. https://doi.org/10.3122/jabfm.2015.S1.150133
Creswell, J. W., Klassen, A. C., Plano Clark, V. L., & Smith, K. C. (2011). Best practices for mixed methods research in the health sciences. Bethesda (maryland), 2013, 541–545.
Crilly, N. (2006). Graphic elicitation: Using research diagrams as interview stimuli. Qualitative Research, 6(3), 341–366. https://doi.org/10.1177/1468794106065007
Dalkin, S. M., Greenhalgh, J., Jones, D., Cunningham, B., & Lhussier, M. (2015). What’s in a mechanism? Development of a key concept in realist evaluation. Implementation Science. https://doi.org/10.1186/s13012-015-0237-x
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4(1), 50. https://doi.org/10.1186/1748-5908-4-50
De Souza, D. E. (2013). Elaborating the context-mechanism-outcome configuration (CMOc) in realist evaluation: A critical realist perspective. Evaluation, 19(2), 141–154. https://doi.org/10.1177/1356389013485194
Ebenso, B., Manzano, A., Uzochukwu, B., Etiaba, E., Huss, R., Ensor, T., Newell, J., Onwujekwe, O., Ezumah, N., Hicks, J., & Mirzoev, T. (2019). Dealing with context in logic model development: Reflections from a realist evaluation of a community health worker programme in Nigeria. Evaluation and Program Planning, 73, 97–110. https://doi.org/10.1016/j.evalprogplan.2018.12.002
Eldredge, L. K. B., Markham, C. M., Ruiter, R. A., Fernández, M. E., Kok, G., & Parcel, G. S. (2016). Planning health promotion programs: An intervention mapping approach. Wiley.
Evans, J., Grudniewicz, A., Baker, G., & Wodchis, W. P. (2016). Organizational context and capabilities for integrating care: A framework for improvement. International Journal of Integrated Care. https://doi.org/10.5334/ijic.2416
Evans, S., & Scarborough, H. (2014). Supporting knowledge translation through collaborative translational research initiatives: ‘Bridging’ versus ‘blurring’ boundary spanning approaches in the UK CLAHRC initiative. Social Science Medicine. https://doi.org/10.1016/j.socscimed.2014.01.025
Farley, K., Thompson, C., Hanbury, A., & Chambers, D. (2013). Exploring the feasibility of conjoint analysis as a tool for prioritizing innovations for implementation. Implementation Science, 8(1), 56. https://doi.org/10.1186/1748-5908-8-56
Farmer, T., Robinson, K., Elliott, S. J., & Eyles, J. (2006). Developing and implementing a triangulation protocol for qualitative health research. Qualitative Health Research, 16(3), 377–394.
Fernandez, M. E., ten Hoor, G. A., van Lieshout, S., Rodriguez, S. A., Beidas, R. S., Parcel, G., Ruiter, R. A. C., Markham, C. M., & Kok, G. (2019). Implementation mapping: Using intervention mapping to develop implementation strategies. Frontiers in Public Health. https://doi.org/10.3389/fpubh.2019.00158
Fulton, B. D., Scheffler, R. M., Sparkes, S. P., Auh, E. Y., Vujicic, M., & Soucat, A. (2011). Health workforce skill mix and task shifting in low income countries: A review of recent evidence. Human Resources for Health, 9(1), 1–11. https://doi.org/10.1186/1478-4491-9-1
Green, A. E., Fettes, D. L., & Aarons, G. A. (2012). A concept mapping approach to guide and understand dissemination and implementation. The Journal of Behavioral Health Services & Research, 39(4), 362–373. https://doi.org/10.1007/s11414-012-9291-1
Han, X., & Ku, L. (2019). Enhancing staffing in rural community health centers can help improve behavioral health care. Health Affairs, 38(12), 2061–2068. https://doi.org/10.1377/hlthaff.2019.00823
Hopf, Y., Francis, J., Helms, P., Haughney, J., & Bond, C. (2016). Core requirements for successful data linkage: an example of a triangulation method. British Medical Journal Open, 6(10), e011879.
Jetelina, K. K., Woodson, T. T., Gunn, R., Muller, B., Clark, K. D., DeVoe, J. E., Balasubramanian, B. A., & Cohen, D. J. (2018). Evaluation of an electronic health record (EHR) tool for integrated behavioral health in primary care. The Journal of the American Board of Family Medicine, 31(5), 712–723. https://doi.org/10.3122/jabfm.2018.05.180041
Kazi, M. (2003). Realist evaluation for practice. British Journal of Social Work, 33(6), 803–818. https://doi.org/10.4135/9781849209762
Krichbaum, K., Diemert, C., Jacox, L., Jones, A., Koenig, P., Mueller, C., & Disch, J. (2007). Complexity compression: Nurses under fire. Nursing Forum, 42(2), 86–94. https://doi.org/10.1111/j.1744-6198.2007.00071.x
Krichbaum, K. E., Peden-McAlpine, C., Diemert, C., Koenig, P., Mueller, C., & Savik, K. (2011). Designing a measure of complexity compression in registered nurses. Western Journal of Nursing Research, 33(1), 7–25. https://doi.org/10.1177/0193945910383877
Leeuw, F. L. (2003). Reconstructing program theories: Methods available and problems to be solved. American Journal of Evaluation, 24(1), 5–20. https://doi.org/10.1177/109821400302400102
Leslie, L. K., Mehus, C. J., Hawkins, J. D., Boat, T., McCabe, M. A., Barkin, S., Perrin, E. C., Metzler, C. W., Prado, G., Tait, V. F., Brown, R., & Beardslee, W. (2016). Primary health care: Potential home for family-focused preventive interventions. American Journal of Preventive Medicine, 51(4), S106–S118. https://doi.org/10.1016/j.amepre.2016.05.014
Lewis, C. C., Boyd, M. R., Walsh-Bailey, C., Lyon, A. R., Beidas, R., Mittman, B., Aarons, G. A., Weiner, B. J., & Chambers, D. A. (2020). A systematic review of empirical studies examining mechanisms of implementation in health. Implementation Science. https://doi.org/10.1186/s13012-020-00983-3
Maier, C. B., & Aiken, L. H. (2016). Task shifting from physicians to nurses in primary care in 39 countries: A cross-country comparative study. European Journal of Public Health, 26(6), 927–934. https://doi.org/10.1093/eurpub/ckw098
Maluka, S., Kamuzora, P., SanSebastián, M., Byskov, J., Ndawi, B., Olsen, Ø. E., & Hurtig, A. K. (2011). Implementing accountability for reasonableness framework at district level in Tanzania: A realist evaluation. Implementation Science, 6(1), 1–15. https://doi.org/10.1186/1748-5908-6-11
Moon, D. J., & Ballard, A. J. (2020). The comprehensive framework for integrated healthcare implementation: a realist evaluation of positive parenting in rural primary care in the US. Journal of Integrated Care. https://doi.org/10.1108/JICA-06-2020-0039
Moon, D. J., Damman, J. L., & Romero, A. (2020). The effects of primary care-based parenting interventions on parenting and child behavioral outcomes: A systematic review. Trauma, Violence, & Abuse, 21(4), 706–724. https://doi.org/10.1177/1524838018774424
Moon, D. J., Lauer, S. J., & Unell, B. (2021). Behavior Checker® staff training for positive parenting in primary care: Changes in the knowledge, attitudes, and confidence. Journal of Child and Family Studies, 30(4), 932–940. https://doi.org/10.1007/s10826-021-01917-3
O’Cathain, A., Murphy, E., & Nicholl, J. (2010). Three techniques for integrating data in mixed methods studies. BMJ, 341, c4587.
Pawson, R., & Tilley, N. (1997). Realistic evaluation. Sage Publications.
Perrin, E. C., Leslie, L. K., & Boat, T. (2016). Parenting as primary prevention. JAMA Pediatrics, 170(7), 637–638. https://doi.org/10.1001/jamapediatrics.2016.0225
Powell, B. J., Beidas, R. S., Lewis, C. C., Aarons, G. A., McMillen, J. C., Proctor, E. K., & Mandell, D. S. (2017). Methods to improve the selection and tailoring of implementation strategies. The Journal of Behavioral Health Services & Research, 44(2), 177–194. https://doi.org/10.1007/s11414-015-9475-6
Powell, B. J., Fernandez, M. E., Williams, N. J., Aarons, G. A., Beidas, R. S., Lewis, C. C., McHugh, S. M., & Weiner, B. J. (2019). Enhancing the impact of implementation strategies in healthcare: A research agenda. Frontiers in Public Health. https://doi.org/10.3389/fpubh.2019.00003
Powell, B. J., McMillen, J. C., Proctor, E. K., Carpenter, C. R., Griffey, R. T., Bunger, A. C., Glass, J. E., & York, J. L. (2011). A compilation of strategies for implementing clinical innovations in health and mental health. Medical Care Research and Review, 69(2), 123–157. https://doi.org/10.1177/1077558711430690
Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, L. J., Smith, J. L., Matthieu, M. M., Proctor, E. K., & Kirchner, J. E. (2015). A refined compilation of implementation strategies: Results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science, 10(1), 1–14. https://doi.org/10.1186/s13012-015-0209-1
Proctor, E. K., Landsverk, J., Aarons, G., Chambers, D., Glisson, C., & Mittman, B. (2009). Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health and Mental Health Services Research, 36(1), 24–34. https://doi.org/10.1007/s10488-008-0197-4
Proctor, E. K., Powell, B. J., & McMillen, J. C. (2013). Implementation strategies: Recommendations for specifying and reporting. Implementation Science, 8(1), 1–11. https://doi.org/10.1186/1748-5908-8-139
Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38(2), 65–76. https://doi.org/10.1007/s10488-010-0319-7
Pronovost, P. J., Berenholtz, S. M., & Needham, D. M. (2008). Translating evidence into practice: A model for large scale knowledge translation. BMJ. https://doi.org/10.1136/bmj.a1714
Rogers, P. J., Petrosino, A., Huebner, T. A., & Hacsi, T. A. (2000). Program theory evaluation: Practice, promise, and problems. New Directions for Evaluation., 2000(87), 5–13. https://doi.org/10.1002/ev.1177
Salmona, M., Lieber, E., & Kaczynski, D. (2019). Qualitative and mixed methods data analysis using Dedoose: A practical approach for research across the social sciences. Sage Publications.
Salter, K. L., & Kothari, A. (2014). Using realist evaluation to open the black box of knowledge translation: a state-of-the-art review. Implementation Science. https://doi.org/10.1186/s13012-014-0115-y
Scriven, M. (1994). The fine line between evaluation and explanation. Evaluation Practice, 15(1), 75–77. https://doi.org/10.1177/109821409401500108
Shah, R., Kennedy, S., Clark, M. D., Bauer, S. C., & Schwartz, A. (2016). Primary care-based interventions to promote positive parenting behaviors: A meta-analysis. Pediatrics. https://doi.org/10.1542/peds.2015-3393
Smith, R. C., Laird-Fick, H., D’Mello, D., Dwamena, F. C., Romain, A., Olson, J., Kent, K., Blackman, K., Solomon, D., & Spoolstra, M. (2014). Addressing mental health issues in primary care: An initial curriculum for medical residents. Patient Education and Counseling, 94(1), 33–42. https://doi.org/10.1016/j.pec.2013.09.010
Unell, B., & Wyckoff, J. (2016). Behavior Checker: Pediatric telephone and office prescriptions for managing common behavioral issues (1st Edn.). Raised with Love and Limits Foundation.
Waltz, T. J., Powell, B. J., Fernández, M. E., Abadie, B., & Damschroder, L. J. (2019). Choosing implementation strategies to address contextual barriers: Diversity in recommendations and future directions. Implementation Science, 14(1), 42. https://doi.org/10.1186/s13012-019-0892-4
Webb, H. (2012). A realistic evaluation exploring the implementation of the Social and Emotional Aspects of Learning (SEAL) programme across a whole-school context. Unpublished doctoral dissertation. University of Birmingham.
Woodson, T. T., Gunn, R., Clark, K. D., Balasubramanian, B. A., Jetelina, K. K., Muller, B., Miller, B. F., Burdick, T. E., & Cohen, D. J. (2018). Designing health information technology tools for behavioural health clinicians integrated within US-based primary care teams. Journal of Innovation in Health Informatics, 25(3), 148–168.
Wyckoff, J., & Unell, B. (2019). Discipline with love and limits: Practical solutions to over 100 common childhood behavior problems. Da Capo Press Hachette Book Group, Inc.
Yin, R. K. (2003). Case study research: design and methods. Sage Publications.
Funding
This study was funded by the Doris Duke Charitable Foundation and the New York Community Trust Robert and Ellen Popper Scholarship Fund.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
We have no known conflict of interest to disclose.
Rights and permissions
About this article
Cite this article
Moon, D.J., Johnson-Motoyama, M. & Nichols, C.B. Contexts and Mechanisms for Implementation Success: A Realist Evaluation of Behavior Checker in Integrated Primary Care. Glob Implement Res Appl 1, 160–171 (2021). https://doi.org/10.1007/s43477-021-00018-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s43477-021-00018-z