Skip to main content
SpringerLink
Log in

Addressing quality in mixed methods research: a review and recommendations for a future agenda

  • Published:
Quality & Quantity Aims and scope Submit manuscript

Abstract

Quality is one of the most debated topics in the recent history of mixed methods research. A growing number of authors are currently discussing how the quality of mixed methods research should be conceptualised and operationalised, with the ultimate aim of promoting well designed and properly implemented mixed methods studies. These authors argue that mixed methods research has a number of unique features with respect to monomethod research and, as such, should be appraised according to its own set of quality criteria. Based on a systematic search of the literature, this review (1) examines the features and trends of the literature on the quality of mixed methods research published until February 2016; (2) provides a metasummary of the most prevalent quality criteria suggested in this literature; and (3) proposes a number of recommendations for future discussion and research on this topic. The review concludes that (1) publications on the quality of mixed methods research are increasingly more prevalent and elaborated; (2) a shared set of core quality criteria for appraising mixed methods research can be identified across publications; and (3) future work on this topic should focus on increasing the number of empirical publications on quality, achieving greater consistency in quality terminology, and reaching an agreement on core quality criteria.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Brown, K.M., Elliott, S.J., Leatherdale, S.T., Robertson-Wilson, J.: Searching for rigour in the reporting of mixed methods population health research: a methodological review. Health Educ. Res. 30(6), 811–839 (2015)

    Article  Google Scholar 

  • Bryman, A.: Integrating quantitative and qualitative research: how is it done? Qual. Res. 6(1), 97–113 (2006a)

    Article  Google Scholar 

  • Bryman, A.: Paradigm peace and implications for quality. Int. J. Soc. Res. Method 9(2), 111–126 (2006b)

    Article  Google Scholar 

  • Bryman, A.: The research question in social research: what is its role? Int. J. Soc. Res. Method 10(1), 5–20 (2007)

    Article  Google Scholar 

  • Bryman, A.: June 1989 and beyond: Julia Brannen’s contribution to mixed methods research. Int. J. Soc. Res. Method 17(2), 121–131 (2014)

    Article  Google Scholar 

  • Bryman, A., Becker, S., Sempik, J.: Quality criteria for quantitative, qualitative and mixed methods research: a view from social policy. Int. J. Soc. Res. Method 11(4), 261–276 (2008)

    Article  Google Scholar 

  • Burrows, T.J.: A preliminary rubric design to evaluate mixed methods research. Doctoral Dissertation, Virginia Tech (2013)

  • Canales, G.: Transformative, mixed methods checklist for psychological research with mexican americans. J. Mixed Method Res. 7(1), 6–21 (2013)

    Article  Google Scholar 

  • Caracelli, V.J., Riggin, L.J.C.: Mixed-method evaluation: developing quality criteria through concept mapping. Eval. Pract. 15(2), 139–152 (1994)

    Article  Google Scholar 

  • Cheek, J.: It depends: possible impacts of moving the field of mixed methods research toward best practice guidelines. In: Hesse-Biber, S., Johnson, R.B. (eds.) The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry, pp. 624–636. Oxford University Press, New York (2015)

    Google Scholar 

  • Collins, K.M.T.: Validity in multimethod and mixed research. In: Hesse-Biber, S., Johnson, R.B. (eds.) The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry, pp. 240–256. Oxford University Press, New York (2015)

    Google Scholar 

  • Collins, K.M.T., Onwuegbuzie, A.J., Jiao, Q.G.: A mixed methods investigation of mixed methods sampling designs in social and health science research. J. Mixed Method Res. 1(3), 267–294 (2007)

    Article  Google Scholar 

  • Collins, K.M.T., Onwuegbuzie, A.J., Johnson, B.: Securing a place at the table: a review and extension of legitimation criteria for the conduct of mixed research. Am. Behav. Sci. 56(6), 849–865 (2012)

    Article  Google Scholar 

  • Creswell, J.W.: A Concise Introduction to Mixed Methods Research. Sage, Thousand Oaks (2015a)

    Google Scholar 

  • Creswell, J.W.: Revisiting mixed methods and advancing scientific practices. In: Hesse-Biber, S., Johnson, R.B. (eds.) The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry, pp. 57–71. Oxford University Press, New York (2015b)

    Google Scholar 

  • Creswell, J.W., Plano-Clark, V.L.: Designing and Conducting Mixed Methods Research, 2nd edn. Sage, Thousand Oaks (2011)

    Google Scholar 

  • Creswell, J.W., Tashakkori, A.: Editorial: developing publishable mixed methods manuscripts. J. Mixed Method Res. 1(2), 107–111 (2007)

    Article  Google Scholar 

  • Creswell, J.W., Klassen, A.C., Plano Clark, V.L., Smith, K.C.: Best Practices for Mixed Methods Research in the Health Sciences. National Institutes of Health, Washington, DC (2011)

    Book  Google Scholar 

  • Curry, L.A., Nunez-Smith, M.: Mixed Methods in Health Sciences Research: A Practical Primer. Sage, Thousand Oaks (2015)

    Google Scholar 

  • Dahlberg, B., Wittink, M.N., Gallo, J.J.: Funding and publishing integrated studies: Writing effective mixed methods manuscripts and grant proposals. In: Tashakkori, A., Teddlie, C. (eds.) The SAGE Handbook of Mixed Methods in Social and Behavioral Research, pp. 775–802. Sage, Thousand Oaks (2010)

    Chapter  Google Scholar 

  • Dellinger, A.B., Leech, N.L.: Toward a unified validation framework in mixed methods research. J. Mixed Method Res. 1(4), 309–332 (2007)

    Article  Google Scholar 

  • Giddings, L.S., Grant, B.M.: From rigour to trustworthiness: validating mixed methods. In: Andrew, S., Halcomb, E.J. (eds.) Mixed Methods Research for Nursing and the Health Sciences, pp. 119–134. Wiley-Blackwell, Oxford (2009)

    Google Scholar 

  • Greene, J.C.: Mixed Methods in Social Inquiry. Jossey-Bass, San Francisco (2007)

    Google Scholar 

  • Hadi, M.A., Alldred, D.P., Closs, S.J., Briggs, M.: Mixed-methods research in pharmacy practice: recommendations for quality reporting (part 2). Int. J. Pharm. Pract. 22(1), 96–100 (2014)

    Article  Google Scholar 

  • Halcomb, E.J., Hickman, L.: Mixed methods research. Nurs. Stand. 29(32), 41–47 (2015)

    Article  Google Scholar 

  • Hammersley, M.: The issue of quality in qualitative research. Int. J. Res. Method Educ. 30(3), 287–305 (2007)

    Article  Google Scholar 

  • Hesse-Biber, S., Johnson, R.B.: Editorial: coming at things differently: future directions of possible engagement with mixed methods research. J. Mixed Method Res. 7(2), 103–109 (2013)

    Article  Google Scholar 

  • Heyvaert, M., Hannes, K., Maes, B., Onghena, P.: Critical appraisal of mixed methods studies. J. Mixed Method Res. 7(4), 302–327 (2013)

    Article  Google Scholar 

  • Ihantola, E.-M., Kihn, L.-A.: Threats to validity and reliability in mixed methods accounting research. Qual. Res. Acc. Manag. 8(1), 39–58 (2011)

    Article  Google Scholar 

  • Ivankova, N.V.: Implementing quality criteria in designing and conducting a sequential QUAN → QUAL mixed methods study of student engagement with learning applied research methods online. J. Mixed Method Res. 8(1), 25–51 (2014)

    Article  Google Scholar 

  • Ivankova, N.V.: Mixed Methods Applications in Action Research. Sage, Thousand Oaks (2015)

    Google Scholar 

  • Ivankova, N.V., Kawamura, Y.: Emerging trends in the utilization of integrated designs in the social, behavioral, and health sciences. In: Tashakkori, A., Teddlie, C. (eds.) The SAGE Handbook of Mixed Methods in Social and Behavioral Research, pp. 581–612. Sage, Thousand Oaks (2010)

    Chapter  Google Scholar 

  • Johnson, R.B., Onwuegbuzie, A.J., Turner, L.A.: Towards a definition of mixed methods research. J. Mixed Method Res. 1(2), 112–133 (2007)

    Article  Google Scholar 

  • Lavelle, E., Vuk, J., Barber, C.: Twelve tips for getting started using mixed methods in medical education research. Med. Teach. 35(4), 272–276 (2013)

    Article  Google Scholar 

  • Leech, N.L.: Interviews with the early developers of mixed methods research. In: Tashakkori, A., Teddlie, C. (eds.) The SAGE Handbook of Mixed Methods in Social and Behavioral Research, pp. 253–272. Sage, Thousand Oaks (2010)

    Chapter  Google Scholar 

  • Leech, N.L.: Writing mixed research reports. Am. Behav. Sci. 56(6), 866–881 (2012)

    Article  Google Scholar 

  • Leech, N.L., Dellinger, A.B.: Validity: Mixed methods. In: Chapelle, C.A. (ed.) The Encyclopedia of Applied Linguistics, pp. 1–7. Wiley-Blackwell, Oxford (2013)

    Google Scholar 

  • Leech, N.L., Onwuegbuzie, A.J.: A typology of mixed methods research designs. Qual. Quant. 43(2), 265–275 (2009)

    Article  Google Scholar 

  • Mertens, D.M.: Editorial: publishing mixed methods research. J. Mixed Method Res. 5(1), 3–6 (2011)

    Article  Google Scholar 

  • Mertens, D.M.: Mixed methods. In: Trainor, A.A., Graue, E. (eds.) Reviewing Qualitative Research in the Social Sciences, pp. 139–150. Routledge, New York (2013)

    Google Scholar 

  • Mertens, D.M.: Research and Evaluation in Education and Psychology, 4th edn. Sage, Thousand Oaks (2015)

    Google Scholar 

  • Mertens, D.M., Bazeley, P., Bowleg, L., Fielding, N., Maxwell, J., Molina-Azorin, J.F., Niglas, K.: The future of mixed methods: a five year projection to 2020. MMIRA (2016). https://mmira.wildapricot.org/resources/Documents/MMIRAtaskforcereportJan2016final.pdf

  • Molina-Azorin, J.F.: The use and added value of mixed methods in management research. J. Mixed Method Res. 5(1), 7–24 (2011)

    Article  Google Scholar 

  • Morse, J.M., Niehaus, L.: Mixed Method Design: Principles and Procedures. Left Coast Press, Walnut Creek (2009)

    Google Scholar 

  • Nastasi, B.K., Hitchcock, J.H., Brown, L.M.: An inclusive framework for conceptualizing mixed methods design typologies: Moving toward fully integrated synergistic research models. In: Tashakkori, A., Teddlie, C. (eds.) The SAGE Handbook of Mixed Methods in Social and Behavioral Research, pp. 305–338. Sage, Thousand Oaks (2010)

    Chapter  Google Scholar 

  • O’Cathain, A.: Assessing the quality of mixed methods research: Toward a comprehensive framework. In: Tashakkori, A., Teddlie, C. (eds.) The SAGE Handbook of Mixed Methods in Social and Behavioral Research, pp. 531–555. Sage, Thousand Oaks (2010)

    Chapter  Google Scholar 

  • O’Cathain, A., Murphy, E., Nicholl, J.: The quality of mixed methods studies in health services research. J. Health Serv. Res. Policy 13(2), 92–98 (2008)

    Article  Google Scholar 

  • Onwuegbuzie, A.J., Corrigan, J.A.: Improving the quality of mixed research reports in the field of human resource development and beyond: a call for rigor as an ethical practice. Hum. Resour. Dev. Q. 25(3), 273–299 (2014)

    Article  Google Scholar 

  • Onwuegbuzie, A.J., Johnson, R.K.: The validity issue in mixed research. Res. Sch. 13(1), 48–63 (2006)

    Google Scholar 

  • Onwuegbuzie, A.J., Poth, C.: Editors’ afterword: toward evidence-based guidelines for reviewing mixed methods research manuscripts submitted to journals. Int. J. Qual. Method 15(1), 1–13 (2016)

    Article  Google Scholar 

  • Onwuegbuzie, A.J., Johnson, R.B., Collins, K.M.T.: Assessing legitimation in mixed research: a new framework. Qual. Quant. 45(6), 1253–1271 (2011)

    Article  Google Scholar 

  • Papadimitriou, A., Ivankova, N.V., Hurtado, S.: Addressing challenges of conducting quality mixed methods studies in higher education. In: Tight, M. (ed.) Theory and Method in Higher Education Research, pp. 133–153. Bingley, Emerald (2013)

    Chapter  Google Scholar 

  • Plano-Clark, V.L., Ivankova, N.V.: Mixed Methods Research: A Guide to the Field. Sage, Thousand Oaks (2016)

    Google Scholar 

  • Plano-Clark, V.L., Huddleston-Casas, C.A., Churchill, S.L., Green, D.O., Garrett, A.L.: Mixed methods approaches in family science research. J. Fam. Issues 29(11), 1543–1566 (2008)

    Article  Google Scholar 

  • Pluye, P., Gagnon, M.-P., Griffiths, F., Johnson-Lafleur, J.: A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in Mixed Studies Reviews. Int. J. Nurs. Stud. 46(4), 529–546 (2009)

    Article  Google Scholar 

  • Ridenour, C.S., Newman, I.: Mixed methods research: exploring the interactive continuum. Southern Illinois University Press, Illinois (2008)

    Google Scholar 

  • Ritchie, J., Spencer, L., O’Connor, W.: Carrying out qualitative analysis. In: Ritchie, J., Lewis, J. (eds.) Qualitative Research Practice: A Guide for Social Science Students and Researchers, pp. 219–262. Sage, London (2003)

    Google Scholar 

  • Sale, J.E.M., Brazil, K.: A strategy to identify critical appraisal criteria for primary mixed-method studies. Qual. Quant. 38(4), 351–365 (2004)

    Article  Google Scholar 

  • Sandelowski, M., Barroso, J.: Handbook for Synthesizing Qualitative Research. Springer, New York (2007)

    Google Scholar 

  • Schifferdecker, K.E., Reed, V.A.: Using mixed methods research in medical education: basic guidelines for researchers. Med. Educ. 43(7), 637–644 (2009)

    Article  Google Scholar 

  • Seale, C.: The Quality of Qualitative Research. Sage, London (1999)

    Book  Google Scholar 

  • Tashakkori, A., Teddlie, C.: Quality of inferences in mixed methods research: calling for an integrative framework. In: Bergman, M.M. (ed.) Advances in Mixed Methods Research, pp. 101–119. Sage, London (2008)

    Google Scholar 

  • Tashakkori, A., Teddlie, C.: Epilogue: Current developments and emerging trends in integrated research methodology. In: Tashakkori, A., Teddlie, C. (eds.) The SAGE Handbook of Mixed Methods in Social and Behavioral Research, pp. 803–826. Sage, Thousand Oaks (2010)

    Chapter  Google Scholar 

  • Teddlie, C., Tashakkori, A.: Major issues and controversies in the use of mixed methods in the social and behavioral sciences. In: Tashakkori, A., Teddlie, C. (eds.) Handbook of Mixed Methods in Social and Behavioral Research, pp. 3–50. Sage, Thousand Oaks (2003)

    Google Scholar 

  • Teddlie, C., Tashakkori, A.: Foundations of Mixed Methods Research: Integrating Quantitative and Qualitative Approaches in the Social and Behavioral Sciences. Sage, Thousand Oaks (2009)

    Google Scholar 

  • Teddlie, C., Tashakkori, A.: Overview of contemporary issues in mixed methods research. In: Tashakkori, A., Teddlie, C. (eds.) The SAGE Handbook of Mixed Methods in Social and Behavioral Research, pp. 1–41. Sage, Thousand Oaks (2010)

    Google Scholar 

  • Venkatesh, V., Brown, S.A., Bala, H.: Bridging the qualitative-quantitative divide: guidelines for conducting mixed methods research in Information Systems. MIS Q. 37(1), 21–54 (2013)

    Article  Google Scholar 

  • Wisdom, J.P., Cavaleri, M.A., Onwuegbuzie, A.J., Green, C.A.: Methodological reporting in qualitative, quantitative, and mixed methods health services research articles. Health Serv. Res. 47(2), 721–745 (2012)

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank Marie-Hélène Paré for her valuable comments on the manuscript.

Author information

Authors and Affiliations

  1. Department of Psychology and Education, Universitat Oberta de Catalunya, Rambla del Poblenou, 156, 08018, Barcelona, Spain

    Sergi Fàbregues

  2. Department of Management, University of Alicante, Campus de San Vicente, Ap. 99, 03080, Alicante, Spain

    José F. Molina-Azorín

Authors
  1. Sergi Fàbregues
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. José F. Molina-Azorín
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sergi Fàbregues.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fàbregues, S., Molina-Azorín, J.F. Addressing quality in mixed methods research: a review and recommendations for a future agenda. Qual Quant 51, 2847–2863 (2017). https://doi.org/10.1007/s11135-016-0449-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11135-016-0449-4

Keywords

Search

Navigation