Electronic performance appraisals: The effects of e-mail communication on peer ratings in actual and simulated environments

https://doi.org/10.1016/j.obhdp.2005.07.001Get rights and content

Abstract

Across three empirical studies, this paper explores the effect of using e-mail as a communication medium (versus pen-and-paper) when conducting performance appraisals of peers. The notion put forth by Media Richness Theory that paper-form and e-mail media should be considered identical for conveying this information was theoretically challenged and differences were empirically supported. Using two different settings, results demonstrate that evaluators offered more negative appraisals of their peers when using e-mail than when using traditional paper-form methods. Reduced feelings of social obligation in the e-mail condition were found to mediate this relationship, indicating that social psychological processes can influence the effects of different media on peer ratings.

Section snippets

Theoretical perspectives comparing communication media

Much previous work, dominated by Media Richness Theory (see Daft & Lengel, 1986) has focused on communication media tools and has compared their features and capabilities. Generally, these studies conclude that face-to-face communication should be more comprehensive than voice-only communication (such as communicating via telephone), which should in turn be more comprehensive than text-only communication such as either paper-text letters or e-mail (considered equivalent by these theories),

E-Mail versus paper-based performance appraisals

Research on how e-mail appraisals compare to face-to-face interaction has revealed distinct differences; unfortunately, the research comparing e-mail appraisals to other written forms (i.e., the focus of the presented research) is limited, and conclusions are mixed at best. A wealth of previous research has demonstrated that people tend to give more negative appraisals when communicating in a computer mediated mode than when communicating face-to-face (Herbert and Vorauer, 2003, Siegel et al.,

Study 1 method

Participants were 73 fulltime graduate level business students who completed the study as part of a class assignment. All participants were informed that they were to provide a performance review evaluating each of their teammates’ performance during a simulated negotiation. The experimental design had one manipulation—they either completed their performance review on-line via e-mail (n = 36) or by traditional paper-form (n = 37). All participants were randomly assigned to one of the two conditions.

Study 3

Two items of concern remained based on the methods employed by the first two studies, so an entirely new design was implemented as Study 3. First, in Studies 1 and 2, we did not actually know where the participants completed the appraisal task, and so the online-versus-paper context may have spuriously overlapped with anonymity in the study as it was executed, as a function of location. In other words, if it was the case that those in the e-mail condition systematically filled out their forms

General discussion

The three studies presented a clear pattern of results suggesting that the communication media used for performance appraisals of peers does indeed matter. Specifically, e-mail seemed to result in more negative ratings overall, as well as a lessened sense of social obligation, than did paper-form ratings. In demonstrating this, we contribute empirical evidence to a small but growing stream of literature that describes the potential differences resulting from the use of various communication

References (61)

  • J.R. Carlson et al.

    Channel expansion theory and the experiential nature of media richness perceptions

    Academy of Management Journal

    (1999)
  • R.L. Daft et al.

    Organizational information requirements, media richness and structural design

    Management Science

    (1986)
  • G. DeSanctis et al.

    Capturing the complexity in advanced technology use: Adaptive Structuration Theory

    Organization Science

    (1994)
  • V.U. Druskat et al.

    Effects and timing of developmental peer appraisals in self-managing work groups

    Journal of Applied Psychology

    (1999)
  • V. Dubrovsky et al.

    The equalization phenomenon: Status effects in computer-mediated and face-to-face decision groups

    Human Computer Interaction

    (1991)
  • M.C. Fiol

    Intraorganizational cognition and interpretation

  • J. Fulk et al.

    A social influence model of technology use

    (1990)
  • T.L. Griffith et al.

    Distinguishing between the forest and the trees: Media, features, and methodology in electronic communication research

    Organization Science

    (1994)
  • B.G. Herbert et al.

    Seeing through the screen: Is evaluative feedback communicated more effectively in face-to-face or computer-mediated exchanges

    Computers in Human Behavior

    (2003)
  • A. Joinson

    Social desirability, anonymity, and Internet-based questionnaires

    Behavior Research Methods, Instruments, & Computers

    (1999)
  • J.R. Katzenbach et al.

    The wisdom of teams: Creating the high performance organization

    (1993)
  • D.A. Kenny et al.

    Data analysis in social psychology

  • S. Kiesler et al.

    Response effects in the electronic survey

    Public Opinion Quarterly

    (1986)
  • W.C. King et al.

    A quasi-experimental assessment of the effect of computerizing noncognitive paper-and-pen measurements: A test of measurement equivalence

    Journal of Applied Psychology

    (1995)
  • R.C. King et al.

    Media appropriateness: Effects of experience on communication media choice

    Decision Sciences

    (1997)
  • E.M. Landry

    Scrolling around the new organization: The potential for conflict in the on-line environment

    Negotiation Journal

    (2000)
  • G.J. Lautenschlager et al.

    Computer administration of questions: More desirable or more social desirability?

    Journal of Applied Psychology

    (1990)
  • A.S. Lee

    Electronic mail as a medium for rich communication: An empirical investigation using hermeneutic interpretation

    MIS Quarterly

    (1994)
  • M. Markus

    Electronic mail as the medium of managerial choice

    Organization Science

    (1994)
  • C.L. Martin et al.

    Some effects of computerized interviewing on job applicant responses

    Journal of Applied Psychology

    (1989)
  • Cited by (50)

    • Emoticons in Business Communication: Is the :) Worth it?

      2016, Emotions, Technology, and Design
    • The bias in the bias: Comparative optimism as a function of event social undesirability

      2014, Organizational Behavior and Human Decision Processes
      Citation Excerpt :

      Socially desirable responding is an important issue in several organizational settings. Examples include the accuracy of personality tests in personnel selection (Bing, Kluemper, Davison, Taylor, & Novicevic, 2011; Komar, Brown, Komar, & Robie, 2008) and the honesty of performance appraisals (Kurtzberg, Naquin, & Belkin, 2005). Unfortunately, disentangling faking behavior from genuine responses after the fact is difficult (Shoss & Strube, 2011).

    • Developing Digital Communication Competency in the Business Classroom

      2022, Business and Professional Communication Quarterly
    View all citing articles on Scopus
    View full text