Skip to main content
SpringerLink
Log in

Qualitative research on software development: a longitudinal case study methodology

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

This paper reports the use of a qualitative methodology for conducting longitudinal case study research on software development. We provide a detailed description and explanation of appropriate methods of qualitative data collection and analysis that can be utilized by other researchers in the software engineering field. Our aim is to illustrate the utility of longitudinal case study research, as a complement to existing methodologies for studying software development, so as to enable the community to develop a fuller and richer understanding of this complex, multi-dimensional phenomenon. We discuss the insights gained and lessons learned from applying a longitudinal qualitative approach to an empirical case study of a software development project in a large multi-national organization. We evaluate the methodology used to emphasize its strengths and to address the criticisms traditionally made of qualitative research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Allison I, Merali Y (2007) Software process improvement as emergent change: a structurational analysis. Inf Softw Technol 49(6):668–681

    Article  Google Scholar 

  • Benbasat I, Goldstein DK, Mead M (1987) The case research strategy in studies of information systems. Manage Inf Syst Q 11(3):369–386

    Google Scholar 

  • Boehm B (2006) A view of 20th and 21st century software engineering. In: Rombach D, Soffa ML (eds) Proceedings of the 28th International Conference on Software Engineering. ACM, New York, pp 12–29

    Google Scholar 

  • Bratthall L, Jørgensen M (2002) Can you trust a single data source exploratory software engineering case study? Empire Software Eng 7(1):9–26

    Article  MATH  Google Scholar 

  • Braun V, Clarke V (2006) Using thematic analysis in psychology. Qual Res Psychol 3(2):77–101

    Article  Google Scholar 

  • Bryman A, Bell E (2007) Business research methods, 2nd edn. Oxford University Press, Oxford

    Google Scholar 

  • Cavaye ALM (1996) Case study research: a multi-faceted research approach for IS. Inf Syst J 6(3):227–242

    Article  Google Scholar 

  • Charette RN (2005) Why software fails. IEEE Spectr 42(9):42–49

    Article  Google Scholar 

  • Ciborra CU, Lanzara GF (1994) Formative contexts and information technology: understanding the dynamics of innovation in organizations. Acc Manag Inf Technol 4(2):61–86

    Google Scholar 

  • Coleman G, O’Connor R (2007) Using grounded theory to understand software process improvement: a study of Irish software product companies. Inf Softw Technol 49(6):654–667

    Article  Google Scholar 

  • Constantinides P, Barrett M (2006) Negotiating ICT development and use: the case of a telemedicine system in the healthcare region of Crete. Inf Organ 16(1):27–55

    Article  Google Scholar 

  • Darke P, Shanks G, Broadbent M (1998) Successfully completing case study research: Combining rigour, relevance and pragmatism. Inf Syst J 8(4):273–289

    Article  Google Scholar 

  • Dittrich Y, John M, Singer J, Tessem B (2007) Editorial for the special issue on qualitative software engineering research. Inf Softw Technol 49(6):531–539

    Article  Google Scholar 

  • Dittrich Y, Rönkkö K, Eriksson J, Hansson C, Lindeberg O (2008) Cooperative method development: combining qualitative empirical research with method, technique and process improvement. Empire Software Eng 13(3):231–260

    Article  Google Scholar 

  • Doolin B (1996) Alternative views of case research in information systems. Aust J Inf Syst 3(2):21–29

    Google Scholar 

  • Easterbrook SM, Singer J, Storey M-A, Damian D (2008) Selecting empirical methods for software engineering research. In: Shull F, Singer J, Sjøberg DIK (eds) Guide to advanced empirical software engineering. Springer, London, pp 285–311

    Chapter  Google Scholar 

  • El Emam K, Koru AG (2008) A replicated survey of IT software project failures. IEEE Softw 25(5):84–90

    Article  Google Scholar 

  • Flyvbjerg B (2006) Five misunderstandings about case-study research. Qual Inq 12(2):219–245

    Article  Google Scholar 

  • Galliers RD, Swan JA (2000) There’s more to information systems development than structured approaches: information requirements analysis as a socially mediated process. Requir Eng 5(2):74–82

    Article  Google Scholar 

  • Gasson S (1999) A social action model of situated information systems design. Data Base Adv Inf Syst 30(2):82–97

    Google Scholar 

  • Geertz C (1973) Thick description: toward an interpretative theory of culture. In: The interpretation of cultures: selected essays. Basic Books, New York

  • Giddens A (1984) The constitution of society: outline of the theory of structuration. University of California Press, Berkeley

    Google Scholar 

  • Glaser BG, Strauss A (1967) The discovery of grounded theory: strategies for qualitative research. Aldine, Chicago

    Google Scholar 

  • Glass RL, Vessey I, Ramesh V (2002) Research in software engineering: an analysis of the literature. Inf Softw Technol 44(8):491–506

    Article  Google Scholar 

  • Golden-Biddle K, Locke K (1993) Appealing work: an investigation of how ethnographic texts convince. Organ Sci 4(4):595–616

    Article  Google Scholar 

  • Guba EG (1990) The alternative paradigm dialog. In: Guba EG (ed) The paradigm dialog. Sage, London, pp 17–27

    Google Scholar 

  • Harrison R, Badoo N, Barry E, Biffl S, Parra A, Winter B, Wuest J (1999) Directions and methodologies for empirical software engineering research. Empire Software Eng 4(4):405–410

    Article  Google Scholar 

  • Höst M, Runeson P (2007) Checklists for software engineering case study research. In: Proceedings of the First International Symposium on Empirical Software Engineering and Measurement (ESEM’07). IEEE Computer Society, Washington DC, pp 479–481

  • Klein HK, Myers MD (1999) A set of principles for conducting and evaluating interpretive field studies in information systems. Manage Inf Syst Q 23(1):67–94

    Google Scholar 

  • Langley A (1999) Strategies for theorizing from process data. Acad Manage Rev 24(4):691–710

    MathSciNet  Google Scholar 

  • Langley A, Truax J (1994) A process study of new technology adoption in smaller manufacturing firms. J Manag Stud 31(5):619–652

    Article  Google Scholar 

  • Lanzara GF (1999) Between transient constructs and persistent structures: designing systems in action. J Strateg Inf Syst 8(4):331–349

    Article  Google Scholar 

  • Latour B (1987) Science in action: how to follow scientists and engineers through society. Harvard University Press, Cambridge

    Google Scholar 

  • Leonard-Barton D (1990) A dual methodology for case studies: synergistic use of a longitudinal single site with replicated multiple sites. Organ Sci 1(3):248–266

    Article  Google Scholar 

  • Lethbridge TC, Singer J, Sim SE (2005) Studying software engineers: data collection techniques for software field studies. Empire Software Eng 10(3):311–341

    Article  Google Scholar 

  • Madsen S, Kautz K, Vidgen R (2006) A framework for understanding how a unique and local IS development method emerges in practice. Eur J Inf Syst 15(2):225–238

    Article  Google Scholar 

  • Markus ML, Lee AS (1999) Special issue on intensive research in information systems: using qualitative, interpretive, and case methods to study information technology. Manage Inf Syst Q 23(1):35–38

    Google Scholar 

  • Markus ML, Robey D (1988) Information technology and organizational change: causal structure in theory and research. Manag Sci 34(5):583–598

    Article  Google Scholar 

  • McLeod L (2008) Understanding IS development and acquisition: a process approach. PhD thesis (http://hdl.handle.net/10292/644), Auckland University of Technology

  • McLeod L, MacDonell S (in press) Factors that affect software systems development project outcomes: a survey of research. ACM Comput Surv

  • Miller J (2008) Triangulation as a basis for knowledge discovery in software engineering. Empire Software Eng 13(2):223–228

    Article  Google Scholar 

  • Mingers J (2001) Combining IS research methods: towards a pluralist methodology. Inf Syst Res 12(3):240–259

    Article  Google Scholar 

  • Myers MD, Newman M (2007) The qualitative interview in IS research: examining the craft. Inf Organ 17(1):2–26

    Article  Google Scholar 

  • Nandhakumar J, Avison DE (1999) The fiction of methodical development: a field study of information systems development. Inf Technol People 12(2):176–191

    Article  Google Scholar 

  • Nandhakumar J, Jones M (1997) Too close for comfort? Distance and engagement in interpretive information systems research. Inf Syst J 7(2):109–131

    Article  Google Scholar 

  • Pentland BT (1999) Building process theory with narrative: from description to explanation. Acad Manage Rev 24(4):711–724

    Google Scholar 

  • Perry DE, Porter AA, Votta LG (2000) Empirical studies of software engineering: a roadmap. In: Proceedings of the Conference on the Future of Software Engineering. ACM, New York, NY, pp 345–355

  • Perry DE, Sim SE, Easterbrook SM (2004) Case studies for software engineers. In: Proceedings of the 26th International Conference on Software Engineering (ICSE’04). IEEE Computer Society, Washington DC, pp 736–738

  • Perry DE, Sim SE, Easterbrook SM (2005) Case studies for software engineers. In: Proceedings of the 29th Annual IEEE/NASA Software Engineering Workshop—Tutorial Notes. IEEE Computer Society, Los Alamitos CA, pp 96–159

  • Pettigrew AM (1990) Longitudinal field research on change: theory and practice. Organ Sci 1(3):267–292

    Article  Google Scholar 

  • Pettigrew AM (1997) What is processual analysis? Scand J Manag 13(4):337–348

    Article  Google Scholar 

  • Robinson H, Segal J, Sharp H (2007) Ethnographically-informed empirical studies of software practice. Inf Softw Technol 49(6):540–551

    Article  Google Scholar 

  • Rönkkö K, Lindeberg O, Dittrich Y (2002) ‘Bad practice’ or ‘bad methods’: are software engineering and ethnographic discourses incompatible? In: Proceedings of the 2002 International Symposium on Empirical Software Engineering (ISESE’02). IEEE Computer Society, Washington DC, pp 204–210

  • Royal Academy of Engineering (2004) The challenges of complex IT projects. Royal Academy of Engineering, London

    Google Scholar 

  • Runeson P, Höst M (2009a) Guidelines for conducting and reporting case study research in software engineering. Empire Software Eng 14(2):131–164

    Article  Google Scholar 

  • Runeson P, Höst M (2009b) Tutorial: case studies in software engineering. In: Bomarius F, Oivo M, Jaring P, Abrahamsson P (eds) Product-focused software process improvement (Proceedings of the 10th International Conference, PROFES 2009, Oulu, Finland, June 15–17). Springer, Berlin, pp 441–442

    Google Scholar 

  • Schultze U, Orlikowski WJ (2004) A practice perspective on technology-mediated network relations: the use of Internet-based self-serve technologies. Inf Syst Res 15(1):87–106

    Article  Google Scholar 

  • Seaman C (1999) Qualitative methods in empirical studies of software engineering. IEEE Trans Softw Eng 25(4):557–572

    Article  Google Scholar 

  • Segal J, Grinyer A, Sharp H (2005) The type of evidence produced by empirical software engineers. In: Proceedings of the Workshop on Realising Evidence-Based Software Engineering (REBSE’05). ACM, New York, NY

  • Sieber JE (2001) Protecting research subjects, employees and researchers: implications for software engineering. Empire Software Eng 6(4):329–341

    Article  MATH  Google Scholar 

  • Sim SE, Singer J, Storey M-A (2001) Beg, borrow, or steal: using multidisciplinary approaches in empirical software engineering research. Empire Software Eng 6(1):85–93

    Article  MATH  Google Scholar 

  • Singer J, Vinson NG (2002) Ethical issues in empirical studies of software engineering. IEEE Trans Softw Eng 28(12):1171–1180

    Article  Google Scholar 

  • Sjøberg DIK, Dybå T, Jørgensen M (2007) The future of empirical methods in software engineering research. In: Briand LC, Wolf AL (eds) Future of software engineering (FOSE’07). IEEE Computer Society, Los Alamitos, pp 358–378

    Chapter  Google Scholar 

  • Star SL (1989) The structure of ill-structured solutions: boundary objects and heterogeneous distributed problem solving. In: Gasser L, Huhns MN (eds) Distributed artificial intelligence (vol. 2). Pitman Publishing, London, pp 37–54

    Google Scholar 

  • Subrahmanian E, Monarch I, Konda S, Granger H, Milliken R, Westerberg A, The N-Dim Group (2003) Boundary objects and prototypes at the interfaces of engineering design. Comput Support Coop Work 12(2):185–203

    Article  Google Scholar 

  • Tolich M, Davidson C (1999) Starting fieldwork: an introduction to qualitative research in New Zealand. Oxford University Press, Auckland

    Google Scholar 

  • Voss C, Tsikriktsis N, Frohlich M (2002) Case research in operations management. Int J Oper Prod Manage 22(2):195–219

    Article  Google Scholar 

  • Wainer J, Barsottini C (2007) Empirical research in CSCW—a review of the ACM/CSCW conferences from 1998 to 2004. J Braz Comput Soc 13(3):27–36

    Article  Google Scholar 

  • Wallace L, Keil M (2004) Software project risks and their effect on outcomes. Commun ACM 47(4):68–73

    Article  Google Scholar 

  • Walsham G (1993) Interpreting information systems in organizations. Wiley, Chichester

    Google Scholar 

  • Walsham G (1995) Interpretive case studies in IS research: nature and method. Eur J Inf Syst 4(2):74–81

    Article  Google Scholar 

  • Walsham G (2006) Doing interpretive research. Eur J Inf Syst 15:320–330

    Article  Google Scholar 

  • Walsham G, Sahay S (1999) GIS for district-level administration in India: problems and opportunities. Manage Inf Syst Q 23(1):39–66

    Google Scholar 

  • Westrup C (1993) Information systems methodologies in use. J Inf Technol 8(4):267–275

    Article  Google Scholar 

  • Whittle A (2005) Preaching and practising ‘flexibility’: Implications for theories of subjectivity at work. Hum Relat 58(10):1301–1322

    Article  Google Scholar 

  • Wynekoop JL, Russo NL (1997) Studying system development methodologies: an examination of research methods. Inf Syst J 7(1):47–65

    Article  Google Scholar 

  • Yin RK (2003) Case study research: design and methods, 3rd edn. Sage, Thousand Oaks

    Google Scholar 

  • Zuboff S (1988) In the age of the smart machine. Basic Books, New York

    Google Scholar 

Download references

Acknowledgments

This research was funded through a Top Achiever Doctoral Scholarship by the Tertiary Education Commission of New Zealand. We would like to acknowledge the support of AlphaCo and SoftCo.

Author information

Authors and Affiliations

  1. Auckland University of Technology, Private Bag 92006, Auckland, 1142, New Zealand

    Laurie McLeod, Stephen G. MacDonell & Bill Doolin

Authors
  1. Laurie McLeod
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stephen G. MacDonell
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bill Doolin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stephen G. MacDonell.

Additional information

Editors: Carolyn Seaman, Jonathan Sillito, Rafael Prikladnicki, Tore Dybå and Kari Rönkkö

Rights and permissions

Reprints and permissions

About this article

Cite this article

McLeod, L., MacDonell, S.G. & Doolin, B. Qualitative research on software development: a longitudinal case study methodology. Empir Software Eng 16, 430–459 (2011). https://doi.org/10.1007/s10664-010-9153-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10664-010-9153-5

Keywords

Search

Navigation