Farm tourism experiences in travel reviews: A cross-comparison of three alternative methods for data analysis

https://doi.org/10.1016/j.jbusres.2011.09.018Get rights and content

Abstract

This paper demonstrates three alternative approaches for mining consumer sentiment from large amounts of qualitative data found in online travel reviews. Manual content coding, corpus-based semantic analysis, and stance-shift analysis represent methods varying greatly in both process and output. For illustration purposes, they are applied in an exploratory study focused on consumers' reaction to farm stays in order to demonstrate how large volumes of qualitative data can be analyzed quantitatively in a relatively efficient and reliable way. A total of 800 narratives describing farm stay experiences and representing four national settings (Australia, Italy, UK, and USA) was collected. The results reveal that each method provides unique insights of what a farm stay vacation evokes, helpful to farm entrepreneurs wishing to develop a tourism business. The findings indicate universal values as recurrent key drivers of customer satisfaction closely relate to rural experiences. Comparing the national datasets, local differences are evident, highlighting regional variations in terms of service products and consumer preferences. From a methodological viewpoint, all three methods produce reliable results by evidence of the similarities across all three analyses. The current study shows manual content coding, corpus-based semantic method, and stance-shift analysis can capture the peculiarities of rural experiences in different national settings.

Introduction

Visitor experiences and customer satisfaction are complex phenomena to measure and analyse. They involve a diverse array of moments of truth, all influenced by the visitors' unique expectations and evaluations (Crotts & Pan 2007). Crotts, Pan, and Raschid (2008) highlight numerous methods to capture consumer likes and dislikes linked to customer loyalty. Though surveys and guest comment cards are efficient, these methods produce generally vague assessments of a guest experience, providing few insights about each attribute's importance to the consumer. Alternatively, open-ended questions evoke richer, more personally meaningful responses (Crotts and Pan, 2007, Pritchard and Havitz, 2006), but these responses can lose their richness and subtle details when summarized for analysis.

The use and popularity of electronic word of mouth have increased over the last few years (Mack, Blose, & Pan 2008), affording researchers access to large amounts of qualitative data for analysing consumer opinions about a tourist destination (Pan, McLauren, & Crotts 2007). Marketing researchers gain insights into consumer experience in a less costly, time-consuming, or intrusive way than focus groups and personal interviews (Kozinets 2002), but the data's sheer volume can be overwhelming.

Among marketing researchers, no single approach prevails as the best method for analysing such qualitative data. Woodside, Sood, and Miller (2008) purport the storytelling method, to create narrative interpretation maps to gain insights about consumer experiences. Kwortnik and Ross (2007) combine the analysis of online forums with consumers' ethnographic and introspective vacation-planning tasks, while Pan et al. (2007) rely on word frequencies and semantic network analysis to investigate consumer opinions related to a tourist destination. Mason and Davis, 2007, Crotts et al., 2009 propose stance-shift analysis as an alternative approach to quantitative content analysis focusing on key language pattern identification. The research presented here adds to this discussion by comparing manual content coding, corpus-based semantic analysis, and stance-shift analysis as applied to a single qualitative dataset. Each method is evaluated based on ability to produce relevant marketing insights efficiently and reliably.

To demonstrate each method, an exploratory study was conducted on consumer reactions to farm stays. This topic was chosen because of the sector's potential to influence rural economies, and the unique and under-researched nature of the product itself. Like bed and breakfasts and small boutique inns of decades past, regional differences exist; however, relatively little is known about cross-national understanding about consumers' values and reactions to farm stay experiences (Ollenburg 2008). The current study attempts to fill this void through detailing the elements and subtle details of both positive and negative guest experiences reported across a variety of national settings.

The paper begins with a brief review of the farm tourism literature, focusing on the customer experience. Next, the study of farm stay guests' online travel reviews provides additional insights about consumers' likes and dislikes employing manual coding of content, corpus-based semantic analysis, and stance-shift analysis. Finally, the paper discusses the potential benefits these three methods offer to glean useful marketing insights in a reliable and efficient way.

Section snippets

Farm entrepreneurs, tourism experiences and customer preferences

The family farm is not only a home, but also a business. The responsibilities of running a rural farm are driven by the cyclical nature of planting and harvesting crops, and the daily responsibilities of caring for livestock (Trussell & Shaw 2007). Opening a working farm to visitors offers a secondary revenue source, but only if the farm's capacity and market demand are sufficient to offset the increased costs (Wilson 2007). Well-designed and well-managed farm stay enterprises potentially

Data collection

Three methods were applied individually to one large qualitative database. This strategy provides broad-based information about farm tourism entrepreneurs, and demonstrates each method's advantages and disadvantages. The narratives were collected with permission from TripAdvisor.com using the travel blog website's search engine. Following Busby and Rendle, 2000, Roberts and Hall, 2001, Weaver and Fennell, 1997, three keywords were identified that reflect the industry: ‘farm stay’,

Discussion and conclusions

This study's purpose is two-fold. First, the results assess consumer sentiments to farm stay experiences across four national settings. Study findings provide meaningful insights about national markets. Second, and most importantly, the study demonstrates three alternative methods to analyze large volumes of qualitative data quantitatively in a relatively efficient and reliable way.

Study results reveal farm stay vacations evoke both universal and nation specific reactions – both positive and

References (42)

  • C.A. Albacete-Sáez et al.

    Service quality measurement in rural accommodation

    Ann Tourism Res

    (2006)
  • J.C. Crotts et al.

    Measuring guest satisfaction and competitive position in the hospitality and tourism industry, An application of stance-shift analysis to travel blog narratives

    J Travel Res

    (2009)
  • J.C. Crotts et al.

    A survey method for identifying key drivers of customer delight

    Int J Contemp Hospit Manag

    (2008)
  • B. Davis et al.

    Stance shift analysis

  • L. Davidson et al.

    I spy with my little eye: A comparison of manual versus computer-aided analysis of data gathered by projective techniques

    Qual Market Res Int J

    (2010)
  • G. Gibbs

    Analyzing qualitative data

    (2007)
  • R.M. Hernández Maestro et al.

    Small-business owners' knowledge and rural tourism establishment performance

    J Travel Res

    (2009)
  • M.B. Holbrook

    More on content analysis in consumer research

    J Consum Res

    (1977)
  • M.B. Holbrook et al.

    Assessing the role of emotions as mediators of consumer responses to advertising

    J Consum Res

    (1987)
  • S. Hsu et al.

    Storytelling research of consumers' self reports of urban tourism

    J Bus Res

    (2009)
  • A. Kaplan

    Content analysis and the theory of signs

    Philos Sci

    (1943)
  • Cited by (60)

    • How social bots can influence public opinion more effectively: Right connection strategy

      2024, Physica A: Statistical Mechanics and its Applications
    • Exploring destination loyalty: Application of social media analytics in a nature-based tourism setting

      2021, Journal of Destination Marketing and Management
      Citation Excerpt :

      This transformation is based on term occurrences, term frequency, and inverse document frequency values. Computer-assisted sentiment analysis has unique advantages such as outperforming manual content coding analyses in terms of efficiency and reliability of the results (Capriello et al., 2013), and also significantly lowers cost, time, and labor compared to traditional methods like surveys or focus groups (Chiu, Chiu, Sung, & Hsieh, 2015). Sentiment analysis is premised on the idea that the content of a review is based either on opinions, personal feelings, beliefs, and judgment about entities or events (i.e. subjective), or is based on facts, evidence, and measurable observations (i.e. objective) (Feldman, 2013).

    • Comparative clustering of destination attractions for different origin markets with network and spatial analyses of online reviews

      2019, Tourism Management
      Citation Excerpt :

      Both the volume and favorability of UGC information affects tourists' perceptions of a destination and can effectively counter information propagated by induced destination agents (Stepchenkova and Zhan, 2013). Research has shown that UGC has been successfully applied to study such aspects of tourism and hospitality operations as service quality and satisfaction (e.g., Au, Buhalis, & Law, 2009, pp. 73–85; Jeong & Jeon, 2008; Ye, Li, Wang, & Law, 2014), destination image and reputation (e.g., Choi, Lehto, & Morrison, 2007; Lim, Chung, & Weaver, 2012), experiences and behavior (e.g., Capriello, Mason, Davis, & Crotts, 2013; Crotts, Mason, & Davis, 2009), eWOM (e.g., Arsal, Woosnam, Baldwin, & Backman, 2010; Lee, Law, & Murphy, 2011), and tourists’ mobility patterns (Hawelka et al., 2014; Liu, Sui, Kang, & Gao, 2014). Attractions are those things a destination can boast of.

    View all citing articles on Scopus
    1

    Tel.: + 1 704 814 0886.

    2

    Tel.: + 1 7045367629; fax: + 1 7046983961.

    3

    Tel.:+1 843 953 6916; fax: + 1 843 953 5697.

    View full text