Mixed-reality learning environments: Integrating mobile interfaces with laboratory test-beds
Introduction
The role that laboratories play in providing valuable learning experiences is well-established (Feisel & Rosa, 2005). In addition to reinforcing concepts introduced by conventional instruction, hands-on laboratory work has been particularly effective at promoting design, collaboration, and social communication skills, which are as desirable in graduates as is a rigorous theoretical background. Although they have enabled learners to put theory into practice, laboratories constantly seek to deliver the most engaging and cost-effective educational experiences possible using the prevailing technology. This is because laboratories require sophisticated laboratory-grade hardware and software to allow for effective interaction with test-beds and visualization of abstract concepts, such as the effects that certain parameters have on the behavior of a system. Thus, limited funding can often reduce access to facilities and inhibit the quality of in-lab activities. Moreover, learners must often focus efforts on installing, calibrating, and troubleshooting equipment at the expense of experimenting and building deep conceptual understanding.
We are living at a time when laptops and desktops are being replaced by smartphones and tablets as the primary personal computers (Bonnington, 2015, Gillett, 2012). In such a rapidly evolving technological landscape, the equipment used in laboratory education is no longer up to date with the expectations of learners, who have become accustomed to mobile, high-quality experiences with interactive media. To meet this challenge, virtual and remote laboratories have been made available on mobile devices, allowing learners to interact with simulated or real experiments from anywhere and at any time (Maiti & Tripathy, 2012). In addition to accessibility and mobility, researchers believe that implementations of mobile remote experiments will better engage and motivate learners (da Silva, Rochadel, Marcelino, Gruber, & Bilessimo, 2013). However, the mobile-based access to virtual and remote laboratories does not address the challenges facing traditional hands-on laboratories, where there have been few efforts for the comprehensive integration of mobile devices and where research into their potential educational and operational benefits remains largely unexplored.
In this paper, a new laboratory approach is presented wherein mobile devices such as smartphones and tablets are employed in roles that go beyond those of traditional graphical interfaces; rather, the devices become responsible for aspects of measurement, estimation, and control of laboratory test-beds. By integrating vision-based control, AR, and touchscreen interaction, mobile devices can provide enhanced interactive experiences with laboratory equipment that may deepen conceptual understanding and improve learner engagement. Specifically, when the camera of a mobile device is pointed towards a laboratory test-bed, the device augments live video with graphics that learners can directly manipulate to control test-beds and perform experiments. Thus, an intimate connection is made between the digital media running on the interface and the physical dynamics of the test-bed. The term mixed-reality learning environment (MRLE) has been used to refer to such systems (Chang, Lee, Wang, & Chen, 2010). To demonstrate the proposed approach, the development of an MRLE is presented that uses a mobile interface and a motor test-bed to teach concepts of dynamic systems and control (i.e., damping, stability, and the effects of pole locations on system response). To assist learners' understanding of these concepts, the mobile interface captures a live video of the test-bed with its camera and projects a 3D virtual motor arm that appears attached with the test-bed, providing a visual aid that learners can use to observe phenomena. Moreover, learners intuitively command the angular position of the motor arm by tapping and dragging on the touchscreen to manipulate the virtual arm. Additional interactive media, such as real-time plots of the system response and an interactive pole-zero plot, engage students by allowing them to conduct inquiry-based investigations with the motor test-bed from their personal devices. For example, by tapping on the interactive pole-zero plot, the learner relocates the closed-loop poles of the system, altering the system's dynamics so that the learner can examine the resulting effects on the system response.
The following is the supplementary data related to this article:
The novelty of the work presented in this paper lies both in the technological development and in the research conducted. Specifically, a novel approach is proposed for integrating the capabilities of mobile devices with those of laboratory test-beds in the development of learning environments for teaching dynamic systems and control concepts. In addition to providing interactive mixed-reality visualizations that enhance student engagement, these learning environments reduce the need for significant amounts of laboratory-grade equipment, thus improving the cost-effectiveness of the traditional laboratory. In some cases, these benefits provided by the mobile devices may even facilitate the development of more portable laboratory platforms that utilize learners' devices in various aspects of their operation. To validate the proposed MRLE approach in terms of its educational effectiveness and associated user experiences, a user study was conducted using the developed platform with two classes of undergraduate students. The objective of the study was to examine whether student participants demonstrate significant improvement in content knowledge or report having significantly beneficial experiences after using the MRLE platform compared to before using the MRLE platform and compared to student participants exposed to the content using traditional classroom and hands-on laboratory techniques.
The paper is organized as follows. In Section 2, we present a review of prior applications that utilize mobile devices as tools in laboratory education. Next, Section 3 provides a detailed overview of the proposed approach, outlines the development of an illustrative platform, and describes the design of evaluation conducted with undergraduate students. Section 4 presents the results of evaluation. Then, Section 5 discusses some observations and rationale for the technical approach and evaluation methodology as well as the evaluation results validating the hypotheses concerning the educational impact and user experience associated with the proposed MRLE approach. Finally, Section 6 offers some concluding remarks and future directions of this work.
Section snippets
Background
For many educational institutions, limited resources have made maintaining up-to-date laboratory facilities a serious burden. Moreover, in many areas of the world where there is lack of infrastructure, educators, and finances, access to quality education is simply not available. These issues have attracted research into providing novel technological solutions for science and engineering education that leverage the ubiquitous presence of mobile devices in people's lives. In particular,
Methodology
The applications discussed in Section 2 demonstrate how the sensing, storage, computation, and communication (SSCC) capabilities of mobile devices have been leveraged to provide learners with the information, tools, data, visualizations, measurements, or remote access needed for effective laboratory learning. For some applications, the SSCC capabilities of devices can be integrated further to provide learners with immersive mixed-reality experiences that can enhance their interactions with
Assessment results
To investigate the educational effectiveness of the proposed system, four scores are assigned to each of the participants' pre- and post-assessments, one for each of the three topic areas being tested (i.e., system damping, stability, and poles), and one overall score. Each assessment item was classified as belonging to one or more of the three topic areas. Specifically, damping, stability, poles were included in 8, 5, and 15 assessment items, respectively. Table 1 shows the average percentage
Benefits and drawbacks of MRLE
During the development and evaluation of the MRLE platform, a variety of notable benefits and drawbacks of the proposed MRLE approach were observed vis-à-vis the conventional hands-on, virtual, and remote laboratories. As an extension of traditional hands-on laboratories, the MRLE approach shares some similarities to hands-on laboratories, such as the need for real laboratory equipment that must be purchased and maintained. Moreover, platforms implementing the MRLE approach require learners to
Conclusions
Laboratories must constantly adapt their styles and strategies to changes in the economic and technological landscape. In this paper, a novel approach was proposed in which the hardware and software of mobile devices and the physical dynamics of laboratory test-beds are blended to create mobile mixed-reality learning environments. By developing mobile applications that incorporate interactive plots and AR, unique cost-effective and stimulating hands-on educational experiences can be provided
Acknowledgments
This work is supported in part by the National Science Foundation awards RET Site [grant numbers EEC-1132482 and EEC-1542286], DRK-12 [grant number DRL-1417769], ITEST [grant number DRL-1614085], and GK-12 Fellows DGE [grant number 0741714], and NY Space Grant Consortium [grant number 48240-7887].
References (39)
- et al.
Improving the authentic learning experience by integrating robots into the mixed-reality environment
Computers & Education
(2010) - et al.
Process and learning outcomes from remotely-operated, simulated, and hands-on student laboratories
Computers & Education
(2011) - et al.
Mathematics and geometry education with collaborative augmented reality
Computers & Graphics
(2003) A comprehensive and modular laboratory for control systems design and implementation
(1995)In less than two years, a smartphone could be your only computer
(2015)- et al.
Using three-dimensional models to teach molecular structures in high school chemistry
Journal of Science Education and Technology
(1995) - de Lima, J.P.C., Rochadel, W., Silva, A.M., Simão, J.P.S., da Silva, J.B., Alves, J.B.M. (2014). Application of remote...
- et al.
The role of the laboratory in undergraduate engineering education
Journal of Engineering Education
(2005) - Frank, J.A., Brill, A., Kapila, V. (2016) Interactive mobile interface with augmented reality for learning digital...
- Frank, J.A., Gómez, J.A.D., Kapila, V. (2015). Using tablets in the vision-based control of a ball and beam test-bed....
Development of mobile interfaces to interact with automatic control experiments [Focus on education]
IEEE Control Systems
Why tablets will become our primary computing device
Pose estimation from corresponding point data
IEEE Transactions on Systems, Man, and Cybernetics
Multiple view geometry in computer vision
iPads in the science laboratory: Experience in designing and implementing a paperless chemistry laboratory course
Journal of STEM Education: Innovations & Research
Making it real: Exploring the potential of augmented reality for teaching primary school science
Virtual Reality
Video analysis of projectile motion using tablet computers as experimental tools
Physics Education
Classical experiments revisited: Smartphones and tablet PCs as experimental tools in acoustics and optics
Physics Education
Cited by (79)
Simulation platforms in initial teacher education: Past practice informing future potentiality
2022, Computers and EducationVirtual Reality to Teach Students in Laboratories: A Bibliometric and Network Analysis
2024, Journal of Chemical EducationTheory-Driven Design in Metaverse Virtual Reality Learning Environments: Two Illustrative Cases
2023, IEEE Transactions on Learning TechnologiesHeterogeneity in Extended Reality Influences Procedural Knowledge Gain and Operation Training
2023, IEEE Transactions on Learning Technologies