Having expectations of information systems benefits that match received benefits: does it really matter?

Abstract

A study was conducted to examine the effect of implementing a new system on its users, specifically, the relationship between pre-implementation expectations and their perceived benefits based on post-implementation experience. Disconfirmation theory was used as the theoretical basis; this predicts that unrealistically high expectations will result in lower levels of perceived benefit than those associated with realistic expectations (i.e. where expectations match experience). Support was found for this prediction, refuting the predictions of dissonance theory. In addition to examining expectations of system use generally, six expectation categories were examined to identify the critical categories where managers should keep expectations from becoming unrealistically high. Significant relationships were found for three expectation categories: system usefulness, ease of use, and information quality. The results indicate that creating and maintaining realistic expectations of future system benefits really does matter.

Introduction

User expectations are potentially an important factor affecting the perceived benefits arising from use of new information systems (IS) [28]. Users continue to become more demanding in terms of what they want from their systems and managers struggle to determine how to establish realistic expectations. The technological advances in recent years and the over stimulation provided by the enthusiastic media have together created higher user expectations. Technologically competent users are now expecting appropriate state-of-the-art technology in their applications while lower competence users are demanding user-friendly applications with simple user interfaces [30]. To build support for new projects, managers may oversell the benefits of future systems, thereby either inadvertently or deliberately creating high expectations that may go unfulfilled.

Therefore, it is no surprise that managing user expectations has become difficult but more essential. Managers who develop effective strategies for this can potentially enhance the success of the implementation of new IS. And this is certainly not a new problem. In the late 1980s, user expectations came to the forefront of the significant problems facing IS managers [26], [27]. Consistent with this, Conrath and Mignen’s [10] survey of EDP/MIS managers found that user expectations of EDP/MIS products and services were ranked second in a list of 33 items affecting user satisfaction.

What are effective strategies to manage user expectations? What is the impact of having high expectations? A recent meta-analysis of factors that influence end-user satisfaction found several studies that examined the association between expectations and user satisfaction. A moderate positive effect size was found. This suggests that creating high expectations can result in having higher user satisfaction. One very significant limitation of almost all of these studies, as pointed out by the authors, is that the studies were cross-sectional (i.e. measured at one point in time). Expectations were measured after the fact. After the system was introduced, users were asked to recall what expectations they had held before the system was introduced. This method can introduce recollection errors [25], and it does not assess the match between expectation and experience. Szajna and Scamell [42], the only longitudinal study in the meta-analysis, explained the positive association they found in their laboratory experiment using cognitive dissonance theory. Mahmood et al. [28] conclude their meta-analysis by calling for more focused research via longitudinal studies at pre-implementation and post-implementation time periods.

As far as we were able to determine, there are only two field-based studies that have examined the effect of expectations on perceived user benefits by measuring expectations before implementation of an IS and measuring views of actual use after implementation. Ginzberg did this in 1981 [14] and Marcolin did this in 1994 (an unpublished Ph.D. thesis [29]). In both of these studies, these researchers found that unrealistically high expectations led to lower levels of user satisfaction. Marcolin grounded her work in disconfirmation theory and the results supported it (Ginzberg did not theoretically ground his work).

Managing expectations is important, but what should managers do? One set of findings suggests that high expectations are good; another set of findings suggests that managers should avoid creating unrealistically high expectations. This study was intended to investigate the impact of the realism of expectations on perceptions of the benefits of system use and break down this impact in order to offer advice to managers and provide guidance for future research. First, the study examines the effects that unrealistically high expectations have on measures of the perceived net benefit of systems. Next, the paper looks at expectations at a more detailed level by examining individual expectation categories.