Elsevier

Computers & Education

Volume 59, Issue 2, September 2012, Pages 710-721
Computers & Education

The effect of emotional feedback on behavioral intention to use computer based assessment

https://doi.org/10.1016/j.compedu.2012.03.003Get rights and content

Abstract

This study introduces emotional feedback as a construct in an acceptance model. It explores the effect of emotional feedback on behavioral intention to use Computer Based Assessment (CBA). A female Embodied Conversational Agent (ECA) with empathetic encouragement behavior was displayed as emotional feedback. More specifically, this research aims at investigating the effect of Emotional Feedback on Behavioral Intention to Use a CBA system, Perceived Playfulness, Perceived Usefulness, Perceived Ease of Use, Content and Facilitating Conditions. An appropriate survey questionnaire was completed by 134 students. Results demonstrate that Emotional Feedback has a direct effect on Behavioral Intention to Use a CBA system and on other crucial determinants of Behavioral Intention. Finally, the proposed acceptance model for computer based assessment extended with the Emotional Feedback variable explains approximately 52% of the variance of Behavioral Intention.

Highlights

► Emotional feedback regarding Computer Based Assessment's acceptance is examined. ► Εmotional feedback was a female agent with empathetic encouragement behavior. ► Emotional feedback has a direct effect on Behavioral Intention to Use. ► Emotional feedback defines also Playfulness, Usefulness and Ease of Use. ► The proposed model explains 52% of the variance of Behavioral Intention.

Introduction

Previous studies showed learners' preference on CBA procedures (Croft et al., 2001, Ricketts & Wilks, 2002; Sambell, Sambell, & Sexton, 1999). The main reasons that learners are pleased with CBA are the following: 1) learners are able to take the assessment anywhere and anytime through a computer. 2) They are also able to take the test as many times as they wish as long as this service is provided. 3) They feel more assured regarding the results' accuracy and fairness since the computer does not care who the test taker is. 4) They are able to see their results as soon as they complete the assessment (Bocij & Greasley, 1999; Cassady & Gridley, 2005). 5) CBA provides them immediate feedback to identify their strengths and weaknesses (Crippen & Brooks, 2002; Gretes & Green, 2000).

Instantaneous or immediate feedback is very important to learners. Educators may use feedback strategies to help learners during the CBA. Feedback could support learners to create knowledge and abilities. Feedback in education has to do with the educator's responses to the learner's actions, thoughts, emotions, needs, attitudes, wills, intentions etc. (Economides, 2006a). Feedback is also useful to guide and support learners during CBA (Thelwall, 2000). It may also try to improve the learner's strengths, performance, and to reduce his/her weaknesses (Economides, 2006a, Wilson et al., 2011).

A case of immediate feedback is emotional feedback. Emotional feedback has been developed to regulate learners' emotional states towards learning. Prior studies showed that positive emotions enhance problem solving and decision making, leading to cognitive processing that is not only flexible, innovative, and creative, but also thorough and efficient (Isen, 2001; Isen, Daubman, & Nowicki, 1987). On the other hand, negative emotions have been shown to impede performance on learning tasks (Izard, 1984). Moreover, previous studies regarding emotional feedback investigated the implementation and evaluation of emotional regulation strategies concerning the management of learners' emotions and behaviors (Beale & Creed, 2009; Burleson & Picard, 2007; D′ Mello et al., 2008; Robison, McQuiggan, & Lester, 2010).

However, to the best of our knowledge previous studies did not connect the use of emotional feedback with the acceptance and the intention to use a CBA. Thus, the goal of this paper is to identify the effect of emotional feedback on learners' behavioral intention and on other crucial variables regarding CBA acceptance. Based on previous studies on technology acceptance and especially towards learning and assessment systems' acceptance, we developed a research questionnaire to evaluate the effect of emotional feedback on CBA's acceptance through a causal model.

Section snippets

Computer based assessment acceptance

Computer Based Assessment Acceptance Model (CBAAM) has been proposed regarding the acceptance of a CBA (Terzis & Economides, 2011). CBAAM have adopted variables from previous studies in order to define Behavioral Intention to Use a CBA system. Particularly, it has adopted Perceived Usefulness (PU) and Perceived Ease of Use (PEOU) from Technology Acceptance Model (TAM) (Davis, 1989). Furthermore, from Unified Theory of Acceptance and Use of Technology (UTUAT), it used Facilitating Conditions

Perceived playfulness

Csikszentimihalyi's (1975) and Deci and Ryan's (1985) studies inspired Moon and Kim to extend TAM with the Perceived Playfulness construct. Perceived Playfulness is an individual's beliefs formed by the interaction with the system (Moon & Kim, 2001). The main dimensions that describe Perceived Playfulness (PP) are the following three:

  • (a)

    Concentration: Whether the user is concentrated on the activity.

  • (b)

    Curiosity: Whether the user's cognitive curiosity is aroused (Malone, 1981a, Malone, 1981b).

  • (c)

Design of the ECA

Previous studies showed that a higher realism of an agent is connected to a higher positive learner's attitude regarding ECA (Yee et al., 2007). Moreover, other studies showed better effectiveness of female agents regarding frustration than male agents (Hone, 2005). Furthermore, previous studies showed a variety regarding the expression of empathy through an ECA. Some researchers used only sentences, such as “It seems you did not like this question so much” (Hone, 2005; Prendinger & Ishizuka,

Results

The technique of partial least-squares (PLS) analysis was used to analyze the measurement and the structural model. Many studies on technology acceptance on learning systems used PLS analysis (eg. Han, 2003; Hsu, Chen, Chiu, & Ju, 2007; Van Raaij & Schepers, 2008; Zhang, Zhao, & Tan, 2008; Yi & Hwang, 2003). Regarding the sample size, the minimum recommended value is defined by the two following guidelines: (a) 10 times larger than the number of items for the most complex construct; (b) 10

Discussion

The main aim of this study was to investigate the effects of Emotional Feedback on Behavioral Intention to use a CBA, and on other crucial determinants of Behavioral Intention to use a CBA system.

Prior knowledge regarding the technology acceptance and emotional feedback was used and customized for CBA. The results demonstrate that Emotional Feedback has a direct effect on Behavioral Intention to Use, on Perceived Playfulness, on Perceived Usefulness, on Perceived Ease of use, on Content and on

Conclusions

This research aimed to investigate the impact of the emotional feedback on the Behavioral Intention to Use a computer based assessment. Data supports the measurement and the structural model. Our study indicates that Emotional Feedback is a significant determinant for Behavioral Intention to Use a CBA, Perceived Playfulness, Perceived Usefulness, Perceived Ease of Use, Content and Facilitating Conditions.

Researchers and tutors will find the study useful since it introduces for first time the

References (84)

  • C. Ong et al.

    Gender differences in perceptions and relationships among dominants of e-learning acceptance

    Computers in Human Behaviour

    (2006)
  • H. Shih

    Using a cognitive-motivation-control view to assess the adoption intention for Web-based learning

    Computers & Education

    (2008)
  • T. Teo

    Modelling technology acceptance in education: a study of pre-service teachers

    Computers & Education

    (2009)
  • V. Terzis et al.

    The acceptance and use of computer based assessment

    Computers & Education

    (2011)
  • M. Thelwall

    Computer-based assessment: a versatile educational tool

    Computers & Education

    (2000)
  • E.M. Van Raaij et al.

    The acceptance and use of a virtual learning environment in China

    Computers & Education

    (2008)
  • Y. Wang

    Assessment of learner satisfaction with asynchronous electronic learning systems

    Information and Management

    (2003)
  • K. Wilson et al.

    Improving student performance in a first-year geography course: examining the importance of computer-assisted formative assessment

    Computers & Education

    (2011)
  • M.Y. Yi et al.

    Predicting the use of web-based information systems: self-efficacy, enjoyment, learning goal orientation, and the technology adoption model

    International Journal of Human Computer Studies

    (2003)
  • S. Zhang et al.

    Extending TAM for online learning systems: an intrinsic motivation perspective

    Tsinghua Science and Technology

    (2008)
  • R. Agarwal et al.

    Are individual differences germane to the acceptance of new information technologies?

    Decision Sciences

    (1999)
  • R. Agarwal et al.

    Time flies when you’re having fun: cognitive absorption and beliefs about information technology usage

    MIS Quarterly

    (2000)
  • S. Alexander

    E-learning developments and experiences

    Education + Training

    (2001)
  • D. Barclay et al.

    The partial least squares approach to causal modelling: personal computer adoption and use as an illustration

    Technology Studies

    (1995)
  • G. Beekman et al.

    Tomorrow's technology and you

    (2007)
  • K.-I. Benţa et al.

    Towards an affective aware home

  • P. Bocij et al.

    Can computer-based testing achieve quality and efficiency in assessment?

    International Journal of Educational Technology

    (1999)
  • W. Burleson et al.

    Gender-specific approaches to developing emotionally intelligent learning companions

    Intelligent Systems

    (2007)
  • J.C. Cassady et al.

    The effects of online formative and summative assessment on test anxiety and performance

    Journal of Technology, Learning and Assessment

    (2005)
  • J. Cassell et al.

    Is it self-administration if the computer gives you encouraging looks?

  • A.W. Chickering et al.

    Seven principles for good practice in undergraduate education

    AAHE Bulletin

    (1987)
  • W.W. Chin

    The partial least squares approach to structural equation Modeling

  • J. Cohen

    Statistical power analysis for the behavioural sciences

    (1988)
  • D.R. Compeau et al.

    Computer self-efficacy: development of a measure and initial test

    MIS Quarterly

    (1995)
  • D. Compeau et al.

    Social cognitive theory and individual reactions to computing technology: a longitudinal study

    MIS Quarterly

    (1999)
  • K.J. Crippen et al.

    An analysis of student learning at a testing web site emphasizing descriptive chemistry

    Journal of Computers in Mathematics and Science Teaching

    (2002)
  • M. Csikszentmihalyi

    Beyond boredom and anxiety

    (1975)
  • S. D'Mello et al.

    AutoTutor detects and responds to learners affective and cognitive states

  • F.D. Davis

    Perceived usefulness, perceived ease of use, and user acceptance of information technology

    MIS Quarterly

    (1989)
  • M.H. Davis

    Empathy: A social psychological approach

    (1996)
  • E.L. Deci et al.

    Intrinsic motivation and self-determination in human behavior

    (1985)
  • M.J. Den Uyl et al.

    The FaceReader: online facial expression recognition

    Proceedings of Measuring Behaviour

    (2005)
  • Cited by (72)

    • Making conversations with chatbots more personalized

      2021, Computers in Human Behavior
      Citation Excerpt :

      This study focused on examining the behaviour of introverts-extraverts because these are the two most examined dimensions of personality in the study of technology adoption and usage. In particular, some studies have found that extraversion has a positive impact on behavioural intention to use new technology (Terzis, Moridis, & Economides, 2012; Wang et al., 2012; Zmud, 1979). Alternatively, the trait of openness represents the trait that a person is open to experiencing a variety of activities, is broad minded and independent.

    View all citing articles on Scopus
    1

    Tel.: +30 2310 891768; fax: +30 2310 891292.

    2

    Tel.: +30 2310 891799; fax: +30 2310 891292.

    View full text