Elsevier

Computers in Human Behavior

Volume 75, October 2017, Pages 42-49
Computers in Human Behavior

Full length article
Contributions of mixed reality in a calligraphy learning task: Effects of supplementary visual feedback and expertise on cognitive load, user experience and gestural performance

https://doi.org/10.1016/j.chb.2017.05.006Get rights and content

Highlights

  • We investigated the effects of two visual feedback types in a calligraphy learning task.

  • We investigated the effects of level of expertise in a calligraphy learning task.

  • We measured cognitive load, user experience and gestural performance.

  • Cognitive load and gestural performance differ between two visual feedback types.

  • Gestural performance differ between novices and experts.

Abstract

The learning of handwriting and calligraphy can be improved by supplementary sensory feedback. Emerging technologies such as mixed reality devices can implement appropriate visual, auditory, or proprioceptive feedback. Yet there have been few studies on supplementary visual feedback. Our study was designed to fill this gap, by comparing the effects of two visual feedback types implemented on a graphic tablet with touchscreen (PenWidthFB versus ColoredVelocityFB feedback), in two groups with different levels of expertise in calligraphy (novice versus expert). We collected measures of cognitive load, user experience and gestural performance. Results showed that 1) there was no significant difference in cognitive load between experts and novices, but PenWidthFB feedback created a higher cognitive load than ColoredVelocityFB feedback, 2) for the user experience, there were no obvious differences between experts and novices, and between the two feedback types, 3) concerning the gestural performance, the experts were faster than the novices, and the applied pressure with ColoredVelocityFB feedback was higher than with PenWidthFB feedback.

Introduction

Handwriting is based on the effective processing of feedback. Feedback is a flow of information regarding a writer's movements (Schmidt & Lee, 2005). Two types of feedback are naturally present in handwriting (Paillard, 1990): visual feedback and proprioceptive feedback. In addition to these “intrinsic” types of feedback, there is “extrinsic” feedback. Unlike intrinsic feedback, which is the internal information perceived in the outcome of a gesture, extrinsic feedback - also known as supplementary sensory feedback (Danna & Velay, 2015) - comes from an external source, such as a human or virtual coach. Supplementary sensory feedback provides visual, auditory, or proprioceptive information that complements some intrinsic feedback. Although visual feedback is the most appropriate type of feedback for the perception of morphokinetic information (i.e., the design of the letters) and topokinetic information (i.e., the spatial arrangement of the text in the graphic space such as the space between letters and words and the placing of punctuation), there have been fewer studies into visual feedback than there have been into auditory and proprioceptive feedback. The well-known drawback of visual feedback is the danger of creating cognitive overload either by adding an intentional sharing task (Weil & Amundson, 1994) or by modifying the nature of the task (Gonzalez et al., 2011).

Recently, Danna and Velay (2015) proposed three solutions for adding visual feedback without causing cognitive overload: 1) introducing feedback after, and not during, the execution of the gesture (Porter & van Galen, 1992), 2) changing in real time the color of the ink associated with a kinetic variable of the movement, and 3) reducing the written record until it disappears completely. These three solutions can be implemented using mixed reality technologies. Mixed reality connects the physical and the virtual worlds (Milgram & Kishino, 1994) and refers to augmented reality and augmented virtuality. Augmented virtuality enriches the virtual world with real objects (George, Michel, Serna, & Bisognin, 2014), while augmented reality presents the user in real time with virtual entities in addition to the components of the real environment (Caudell & Mizell, 1992). The graphic tablet with a touchscreen is an example of an augmented reality device capable of integrating Danna and Velay's (2015) three solutions for implementing visual feedback.

Our aim in the present study was to examine the effects of supplementary visual feedback via a graphic tablet on cognitive load, user experience and gestural performance, for two groups with different levels of expertise in calligraphy.

Section snippets

Related work

This section describes previous work using mixed reality within the context of handwriting learning. More precisely, it focuses on the various factors that impact learning (in relation to aspects such as cognitive load, user experience, and overall performance).

Aims and hypotheses

Three main observations can be distilled from the previous sections: 1) even though visual feedback is the most appropriate type of feedback for the perception of morphokinetic and topokinetic information, few studies have looked at visual supplementary feedback (Danna & Velay, 2015); 2) the effects of expertise on cognitive load, user experience and task effectiveness have mainly been studied in relation to technology-enhanced learning, especially in science, technology, engineering, and

Participants

Seventy-six participants (39 women and 37 men) who had volunteered to take part in this study were assigned to one of two feedback types (ColoredVelocityFB versus PenWidthFB) and to one of two levels of expertise in calligraphy (expert versus novice). The level of expertise was determined according to a score computed from participants' answers to nine questions about drawing and calligraphy: novices obtained a score between 0 and 1.67 (i.e., <1.96 which is the mean score), while experts

Results

We ran Levene's test for homogeneity of variance on all the dimensions. The data were analyzed using a between-groups analysis of variance when homoscedasticity was met, while the nonparametric Mann–Whitney U test was used when homoscedasticity was not met.

Discussion

The purpose of our study was to examine the effects of supplementary visual feedback via a graphic tablet on cognitive load, user experience and gestural performance, for two groups with different levels of expertise in calligraphy. This was an original study for three reasons. First, there have been fewer studies into visual feedback than into auditory and proprioceptive feedback (Danna & Velay, 2015) even though visual feedback is the most suitable type of feedback for the perception of

Conclusion

The present study yielded knowledge about the effects of two supplementary visual feedback types implemented on a graphic tablet with touchscreen for subjects with two levels of expertise in calligraphy (novices versus experts). Participants performed a simple task corresponding to a “minimum” gesture in calligraphy, in which they had to trace several series of straight vertical lines either with PenWidthFB feedback or with ColoredVelocityFB feedback. “ColoredVelocityFB” feedback calculates the

Funding

This study was part of the DESCRIPT research project funded by the Picardy regional authority and by the French National Center for Scientific Research.

Conflict of interest

The authors declare that they have no conflicts of interest.

Author note

Contributions of mixed reality in a calligraphy learning task: effects of supplementary visual feedback and expertise on cognitive load, user experience and gestural performance.

References (47)

  • L. Chanquoy et al.

    La charge cognitive Théorie et application

    (2007)
  • A. Covaci et al.

    Third person view and guidance for more natural motor behaviour in immersive basketball playing

  • J. Danna et al.

    Basic and supplementary sensory feedback in handwriting

    Frontiers in Psychology

    (2015)
  • K.A. Ericsson

    Recent advances in expertise research: A commentary on the contributions to the special issue

    Applied Cognitive Psychology

    (2005)
  • D. Feygin et al.

    Haptic guidance: Experimental evaluation of a haptic training method for a perceptual motor skill

  • W.T. Fitch et al.

    Sonifying the body electric: Superiority of an auditory display over a visual display in a complex, multivariate system

  • R. Frenoy et al.

    Adaptive training environment without prior knowledge: Modeling feedback selection as a multi-armed bandit problem

  • R. Frenoy et al.

    CalliSmart: An adaptive informed environment for intelligent calligraphy training

  • L. Fricoteaux et al.

    Adaptation de retours immersifs en environnement virtuel informé basée sur une observation incertaine de l'utilisateur. Application à la formation au transport fluvial

    Revue D'Intelligence Artificielle

    (2015)
  • S. George et al.

    Évaluation de l’impact d’un jeu sérieux en réalité mixte

    STICEF

    (2014)
  • C. Gonzalez et al.

    Is tracing or copying better when learning to repro-duce a pattern?

    Experimental Brain Research

    (2011)
  • S.G. Hart et al.

    Development of NASA-TLX (task load Index): Results of empirical and theoretical research

  • M. Hassenzahl

    The thing and I: Understanding the relationship between user and product

  • Cited by (29)

    • Does modifying visual feedback facilitate learning to write new pseudoletters?

      2023, Human Movement Science
      Citation Excerpt :

      Supplementary FB makes it possible to modify the basic mechanisms that are involved in the motor control of handwriting (for a review, see Danna & Velay, 2015), by providing additional information that a writer can access in real-time (concurrent FB), or by providing new information not naturally accessible either during or after execution (terminal FB, Sigrist, Rauter, Riener, & Wolf, 2013). For example, during execution, it is possible to correct handwriting movements with haptic guidance (Bluteau, Coquillart, Payan, & Gentaz, 2008), to provide additional auditory FB with handwriting sonification (e.g., Danna et al., 2015; Ecalle, Boisson, Labat, Versace, & Magnan, 2021), or to change the digital trace on the screen in order to add customized visual information (e.g., Loup-Escande et al., 2017). After execution, it is also possible to provide additional visual FB on the product (e.g., on the shape, Bonneton-Botté et al., 2020) or on the process of handwriting (e.g., Søvik & Teulings, 1983).

    • Training birdsong recognition using virtual reality

      2021, Virtual Reality and Intelligent Hardware
    • AI technologies for education: Recent research &amp; future directions

      2021, Computers and Education: Artificial Intelligence
      Citation Excerpt :

      •Personalized learning systems or environments (PLS/E) (Bahçeci & Gürol, 2016; Griol et al., 2014; Köse, 2017; Montalvo et al., 2018; Samarakou et al., 2018; Santos & Notargiacomo, 2018; Xu & Wang, 2006; Walkington & Bernacki, 2019); •Visualizations (Keshav et al., 2017; Leony, Munoz-Merino, Pardo, & Kloos, 2013; Lou-Escande, Frenoy, Poplimont, Thouvenin Gapenne, & Megalakaki, 2017) Only one study focused solely on chatbot in education and it was not directly linked to learning outcomes.

    • The state of immersive technology research: A literature analysis

      2018, Computers in Human Behavior
      Citation Excerpt :

      As shown in Table 9, learning performance, behavioral changes, pain reduction, and continuance intention were commonly cited as response variables. In the field of education, researchers found that the use of immersive technologies enhanced learning processes, student engagement, and outcomes (Cheng & Tsai, 2014; Frank & Kapila, 2017; Ibáñez, Di-Serio, Villarán-Molina, & Delgado-Kloos, 2016; Loup-Escande et al., 2017; Yoon, Elinich, Wang, Steinmeier, & Tucker, 2012). Researchers who view immersive technologies as persuasive tools argue that the use of immersive technologies can motivate users to change their behaviors, and they thus focus on user behavior as a consequence of immersive technology use.

    View all citing articles on Scopus
    View full text