Full length articleContributions of mixed reality in a calligraphy learning task: Effects of supplementary visual feedback and expertise on cognitive load, user experience and gestural performance
Introduction
Handwriting is based on the effective processing of feedback. Feedback is a flow of information regarding a writer's movements (Schmidt & Lee, 2005). Two types of feedback are naturally present in handwriting (Paillard, 1990): visual feedback and proprioceptive feedback. In addition to these “intrinsic” types of feedback, there is “extrinsic” feedback. Unlike intrinsic feedback, which is the internal information perceived in the outcome of a gesture, extrinsic feedback - also known as supplementary sensory feedback (Danna & Velay, 2015) - comes from an external source, such as a human or virtual coach. Supplementary sensory feedback provides visual, auditory, or proprioceptive information that complements some intrinsic feedback. Although visual feedback is the most appropriate type of feedback for the perception of morphokinetic information (i.e., the design of the letters) and topokinetic information (i.e., the spatial arrangement of the text in the graphic space such as the space between letters and words and the placing of punctuation), there have been fewer studies into visual feedback than there have been into auditory and proprioceptive feedback. The well-known drawback of visual feedback is the danger of creating cognitive overload either by adding an intentional sharing task (Weil & Amundson, 1994) or by modifying the nature of the task (Gonzalez et al., 2011).
Recently, Danna and Velay (2015) proposed three solutions for adding visual feedback without causing cognitive overload: 1) introducing feedback after, and not during, the execution of the gesture (Porter & van Galen, 1992), 2) changing in real time the color of the ink associated with a kinetic variable of the movement, and 3) reducing the written record until it disappears completely. These three solutions can be implemented using mixed reality technologies. Mixed reality connects the physical and the virtual worlds (Milgram & Kishino, 1994) and refers to augmented reality and augmented virtuality. Augmented virtuality enriches the virtual world with real objects (George, Michel, Serna, & Bisognin, 2014), while augmented reality presents the user in real time with virtual entities in addition to the components of the real environment (Caudell & Mizell, 1992). The graphic tablet with a touchscreen is an example of an augmented reality device capable of integrating Danna and Velay's (2015) three solutions for implementing visual feedback.
Our aim in the present study was to examine the effects of supplementary visual feedback via a graphic tablet on cognitive load, user experience and gestural performance, for two groups with different levels of expertise in calligraphy.
Section snippets
Related work
This section describes previous work using mixed reality within the context of handwriting learning. More precisely, it focuses on the various factors that impact learning (in relation to aspects such as cognitive load, user experience, and overall performance).
Aims and hypotheses
Three main observations can be distilled from the previous sections: 1) even though visual feedback is the most appropriate type of feedback for the perception of morphokinetic and topokinetic information, few studies have looked at visual supplementary feedback (Danna & Velay, 2015); 2) the effects of expertise on cognitive load, user experience and task effectiveness have mainly been studied in relation to technology-enhanced learning, especially in science, technology, engineering, and
Participants
Seventy-six participants (39 women and 37 men) who had volunteered to take part in this study were assigned to one of two feedback types (ColoredVelocityFB versus PenWidthFB) and to one of two levels of expertise in calligraphy (expert versus novice). The level of expertise was determined according to a score computed from participants' answers to nine questions about drawing and calligraphy: novices obtained a score between 0 and 1.67 (i.e., <1.96 which is the mean score), while experts
Results
We ran Levene's test for homogeneity of variance on all the dimensions. The data were analyzed using a between-groups analysis of variance when homoscedasticity was met, while the nonparametric Mann–Whitney U test was used when homoscedasticity was not met.
Discussion
The purpose of our study was to examine the effects of supplementary visual feedback via a graphic tablet on cognitive load, user experience and gestural performance, for two groups with different levels of expertise in calligraphy. This was an original study for three reasons. First, there have been fewer studies into visual feedback than into auditory and proprioceptive feedback (Danna & Velay, 2015) even though visual feedback is the most suitable type of feedback for the perception of
Conclusion
The present study yielded knowledge about the effects of two supplementary visual feedback types implemented on a graphic tablet with touchscreen for subjects with two levels of expertise in calligraphy (novices versus experts). Participants performed a simple task corresponding to a “minimum” gesture in calligraphy, in which they had to trace several series of straight vertical lines either with PenWidthFB feedback or with ColoredVelocityFB feedback. “ColoredVelocityFB” feedback calculates the
Funding
This study was part of the DESCRIPT research project funded by the Picardy regional authority and by the French National Center for Scientific Research.
Conflict of interest
The authors declare that they have no conflicts of interest.
Author note
Contributions of mixed reality in a calligraphy learning task: effects of supplementary visual feedback and expertise on cognitive load, user experience and gestural performance.
References (47)
- et al.
The effect of real-time auditory feedback on learning new characters
Human Movement Science
(2015) - et al.
GULLIVER: A decision-making system based on user observation for an adaptive training in informed virtual environments
Engineering Applications of Artificial Intelligence
(2014) - et al.
Création et validation d’une version française du questionnaire AttrakDiff pour l'évaluation de l’expérience utilisateur des systèmes interactifs
Revue européenne de psychologie appliquée
(2015) - et al.
Immediate vs. postponed visual feedback in practicing a handwriting task
Human Movement Science
(1992) - et al.
Handwriting process and product characteristics of children diagnosed with developmental coordination disorder
Human Movement Science
(2008) - et al.
Real-time feedback of handwriting in a teaching program
Acta Psychologica
(1983) - et al.
The control of pen pressure in handwriting: A subtle point
Human Movement Science
(1991) - et al.
Conceptual and operational definition of personal innovativeness in the domain of IT
Information Systems Research
(1998) - et al.
Increasing viscosity and inertia using a robotically controlled pen improves handwriting in children
Journal of Child Neurology
(2009) - et al.
Augmented reality: An application of heads-up display technology to manual manufacturing processes
La charge cognitive Théorie et application
Third person view and guidance for more natural motor behaviour in immersive basketball playing
Basic and supplementary sensory feedback in handwriting
Frontiers in Psychology
Recent advances in expertise research: A commentary on the contributions to the special issue
Applied Cognitive Psychology
Haptic guidance: Experimental evaluation of a haptic training method for a perceptual motor skill
Sonifying the body electric: Superiority of an auditory display over a visual display in a complex, multivariate system
Adaptive training environment without prior knowledge: Modeling feedback selection as a multi-armed bandit problem
CalliSmart: An adaptive informed environment for intelligent calligraphy training
Adaptation de retours immersifs en environnement virtuel informé basée sur une observation incertaine de l'utilisateur. Application à la formation au transport fluvial
Revue D'Intelligence Artificielle
Évaluation de l’impact d’un jeu sérieux en réalité mixte
STICEF
Is tracing or copying better when learning to repro-duce a pattern?
Experimental Brain Research
Development of NASA-TLX (task load Index): Results of empirical and theoretical research
The thing and I: Understanding the relationship between user and product
Cited by (29)
Does modifying visual feedback facilitate learning to write new pseudoletters?
2023, Human Movement ScienceCitation Excerpt :Supplementary FB makes it possible to modify the basic mechanisms that are involved in the motor control of handwriting (for a review, see Danna & Velay, 2015), by providing additional information that a writer can access in real-time (concurrent FB), or by providing new information not naturally accessible either during or after execution (terminal FB, Sigrist, Rauter, Riener, & Wolf, 2013). For example, during execution, it is possible to correct handwriting movements with haptic guidance (Bluteau, Coquillart, Payan, & Gentaz, 2008), to provide additional auditory FB with handwriting sonification (e.g., Danna et al., 2015; Ecalle, Boisson, Labat, Versace, & Magnan, 2021), or to change the digital trace on the screen in order to add customized visual information (e.g., Loup-Escande et al., 2017). After execution, it is also possible to provide additional visual FB on the product (e.g., on the shape, Bonneton-Botté et al., 2020) or on the process of handwriting (e.g., Søvik & Teulings, 1983).
Training birdsong recognition using virtual reality
2021, Virtual Reality and Intelligent HardwareAI technologies for education: Recent research & future directions
2021, Computers and Education: Artificial IntelligenceCitation Excerpt :•Personalized learning systems or environments (PLS/E) (Bahçeci & Gürol, 2016; Griol et al., 2014; Köse, 2017; Montalvo et al., 2018; Samarakou et al., 2018; Santos & Notargiacomo, 2018; Xu & Wang, 2006; Walkington & Bernacki, 2019); •Visualizations (Keshav et al., 2017; Leony, Munoz-Merino, Pardo, & Kloos, 2013; Lou-Escande, Frenoy, Poplimont, Thouvenin Gapenne, & Megalakaki, 2017) Only one study focused solely on chatbot in education and it was not directly linked to learning outcomes.
The state of immersive technology research: A literature analysis
2018, Computers in Human BehaviorCitation Excerpt :As shown in Table 9, learning performance, behavioral changes, pain reduction, and continuance intention were commonly cited as response variables. In the field of education, researchers found that the use of immersive technologies enhanced learning processes, student engagement, and outcomes (Cheng & Tsai, 2014; Frank & Kapila, 2017; Ibáñez, Di-Serio, Villarán-Molina, & Delgado-Kloos, 2016; Loup-Escande et al., 2017; Yoon, Elinich, Wang, Steinmeier, & Tucker, 2012). Researchers who view immersive technologies as persuasive tools argue that the use of immersive technologies can motivate users to change their behaviors, and they thus focus on user behavior as a consequence of immersive technology use.
Decentralized Internet-based calligraphy education leads the internal development of aesthetic education in schools
2024, Applied Mathematics and Nonlinear SciencesTheory-Driven Design in Metaverse Virtual Reality Learning Environments: Two Illustrative Cases
2023, IEEE Transactions on Learning Technologies