Klen Čopič Pucihar
Paul Coulton
Abstract
Despite considerable progress in mobile augmented reality (AR) over recent years, there are few commercial entertainment systems utilizing this exciting technology. To help understand why, we shall review the state of the art in mobile AR solutions, in particular sensor-based, marker-based, and markerless solutions through a design lens of existing and future entertainment services. The majority of mobile AR that users are currently likely to encounter principally utilize sensor-based or marker-based solutions. In sensor-based systems, the poor accuracy of the sensor measurements results in relatively crude augmentation, whereas in marker-based systems, the requirement to physically augment our environment with fiducial markers limits the opportunity for wide-scale deployment. While the alternative online markerless systems overcome these limitations, they are sensitive to environmental conditions (i.e. light conditions), are computationally more expensive, and present greater complexity of implementation, particularly in terms of their system-initialization requirements. To simplify the operation of online markerless systems, a novel, fully autonomous map initialization method based on accelerometer data is also presented; when compared with alternative move-matching techniques, it is simpler to implement, more robust, faster, and less computationally expensive. Finally, we highlight that while there are many technical challenges remaining to make mobile AR development easier, we also acknowledge that because of the nature of AR, it is often difficult to assess the experience that mobile AR will provide to users without resorting to complex system implementations. We address this by presenting a method of creating low-fidelity prototypes for mobile AR entertainment systems.
- Bradski, G. and Kaehler, A., Learning Open CV: Computer Vision with the OpenCV Library. O'Reilly, 2008. Google Scholar
Digital Library
- Burnett, D. and Coulton, P., Using Infra-Red Beacons as Unobtrusive Markers for Mobile Augmented Reality. MobileHCI 2011 Workshop: Mobile Augmented Reality: Design Issues & Opportunities, Stocholm, Sweeden, 2011.Google Scholar
- Chehimi, F. and Coulton, P., Motion controlled mobile 3D multiplayer gaming. Proceedings of the 2008 International Conference in Advances on Computer Entertainment Technology - ACE '08, New York, ACM Press, 267--267. Google ScholarDigital Library
- Chehimi, F., Coulton, P. and Edwards, R., 3D Motion Control of Connected Augmented Virtuality on Mobile Phones. Proceedings of 2008 International Symposium on Ubiquitous Virtual Reality. 67--70. Google ScholarDigital Library
- Chen, D., Tsai, S., Hsu, C.-h. and Singh, J.P. Mobile Augmented Reality for Books on a Shelf. Multimedia and Expo (ICME), 2011. Google ScholarDigital Library
- Coulton, P., Bamford, W., Experimenting Through Mobile ‘Apps’ and ‘App Stores’. International Journal of Mobile Human Computer Interaction (IJMHCI), 3(4), 55--70. Google ScholarDigital Library
- Čopič Pucihar, K. and Coulton, P., Estimating Scale using Depth From Focus for Mobile Augmented Reality. Procceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems 2011, Pisa, Italy, (2011). Google ScholarDigital Library
- Čopič Pucihar, K., Coulton, P. and Hutchinson, D., Utilizing Sensor Fusion in Markerless Mobile Augmented Reality. Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services. 663--666. Google ScholarDigital Library
- Čopič Pucihar, K., Coulton, P., Towards Collaboratively Mapped Multi- View Mobile Augmented Reality. MobileHCI 2011 Workshop: Mobile Augmented Reality: Design Issues & Opportunities, Stocholm, Sweeden, 2011.Google Scholar
- Gilbertson, P., Coulton, P., Chehimi, F. and Vajk, T., Using tilt as an interface to contro no-button 3D mobile games. ACM Computers in Entertainment, 6 (3). 1--13. Google ScholarDigital Library
- Gustafsson, A., Bichard, J., Brunnberg, L., Juhlin, O. and Combetto, M., Believable environments: generating interactive storytelling in vast location-based pervasive games. Proceedings of the 2006 ACM SIGCHI international conference on Advances in computer entertainment technology, 2006. Google ScholarDigital Library
- Hagbi, N., Bergig, O., El-Sana, J. and Billinghurst, M., Shape recognition and pose estimation for mobile augmented reality. Proceedings of the 8th IEEE International Symposium on Mixed and Augmented Reality, (2009), IEEE Computer Society, 65--71. Google ScholarDigital Library
- Hartley, R. and Zisserman, A., Multiple View Geometry in Computer Vision. Cambridge University Press, 2004. Google ScholarDigital Library
- Kelley, J.F., An iterative design methodology for user-friendly natural language office information applications. ACM Transactions on Information Systems (TOIS), 2 (1). 26--41. Google ScholarDigital Library
- Klein, G. and Murray, D., Parallel Tracking and Mapping on a Camera Phone. Proceedings of the 8th IEEE International Symposium on Mixed and Augmented Reality, (Orlando, 2009). Google ScholarDigital Library
- Kurz, D., and Benhimane, S., Inertial sensor-aligned visual feature descriptors. Proceedings IEEE Conference on Computer Vision and Pattern Recognition, 2011. Google ScholarDigital Library
- Kurz, D., and Benhimane, S., Gravity-Aware Handheld Augmented Reality. Proceedings of the 8th IEEE International Symposium on Mixed and Augmented Reality, (2011), pp. 111--120. Google ScholarDigital Library
- Lee, W., Park, Y., Lepetit, V. and Woo, W., Point-and-shoot for ubiquitous tagging on mobile phones. Proceedings of the 9th IEEE International Symposium on Mixed and Augmented Reality, 57--64.Google Scholar
- Lucas, B. D., and Kanade, T., “An Iterative Image Registration Technique with an Application to Stereo Vision”, in Proceedings of the 7th International Joint Conference on Artificial Intelligence, 1981, pp. 674--679. Google ScholarDigital Library
- Mulloni, A., Seichter, H., and Schmalstieg, D., Enhancing Handheld Navigation Systems with Augmented Reality, MobileHCI 2011 Workshop: Mobile Augmented Reality: Design Issues & Opportunities, Stocholm, Sweeden, 2011.Google Scholar
- J. Park, S. You, and U. Neumann. Natural feature tracking for ex- tendible robust augmented realities. Proceedings International Workshop on Augmented Reality, 1998. Google ScholarDigital Library
- Rashid, O., Mullins, I. and Coulton, P., Extending cyberspace: location based games using cellular phones. Computers in Entertainment (CIE), 1--18. Google ScholarDigital Library
- Shi, J. and Tomasi, C., Good features to track, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1994, pp. 593--600.Google Scholar
- Qualcomm, Vuforia Extends Image Recognition to the Cloud, http://www.qualcomm.com/media/releases/2012/06/27/vuforia-extends-image-recognition-cloud (accessed on: 11. Oct. 2012).Google Scholar
- Wagner, D., Reitmayr, G., Mulloni, A., Drummond, T. and Schmalstieg, D., Pose tracking from natural features on mobile phones. Proceedings of the 7th IEEE International Symposium on Mixed and Augmented Reality, (Washington, DC, USA, 2008), 125--134. Google ScholarDigital Library
- Wagner, D. and Schmalstieg, D., History and Future of Tracking for Mobile Phone Augmented Reality. Proceedings of the 2009 International Symposium on Ubiquitous Virtual Reality, (GIST, Guangju, Korea, 2009), 7--10. Google ScholarDigital Library
Recommendations
Augmented reality technologies, systems and applications
This paper surveys the current state-of-the-art of technology, systems and applications in Augmented Reality. It describes work performed by many different research groups, the purpose behind each new Augmented Reality system, and the difficulties and ...
Haptics in Augmented Reality
ICMCS '99: Proceedings of the IEEE International Conference on Multimedia Computing and Systems - Volume 2An augmented reality system merges synthetic sensory information into a user's perception of a three-dimensional environment. An important performance goal for an augmented reality system is that the user perceives a single seamless environment. In most ...
Augmented reality as perceptual reality
VSMM'06: Proceedings of the 12th international conference on Interactive Technologies and Sociotechnical SystemsAs shown in Paul Milgram et al’s Reality-Virtuality Continuum (1994), Augmented Reality occupies a very unique status in the spectrum of Mixed Reality. Unlike Virtual Reality, which is completely made up of the virtual and has been the most important ...
Comments