Seita Kayukawa
Keita Higuchi
ABSTRACT
We present an assistive suitcase system, BBeep, for supporting blind people when walking through crowded environments. BBeep uses pre-emptive sound notifications to help clear a path by alerting both the user and nearby pedestrians about the potential risk of collision. BBeep triggers notifications by tracking pedestrians, predicting their future position in real-time, and provides sound notifications only when it anticipates a future collision. We investigate how different types and timings of sound affect nearby pedestrian behavior. In our experiments, we found that sound emission timing has a significant impact on nearby pedestrian trajectories when compared to different sound types. Based on these findings, we performed a real-world user study at an international airport, where blind participants navigated with the suitcase in crowded areas. We observed that the proposed system significantly reduces the number of imminent collisions.
Supplemental Material
Available for Download
zip
paper052pvc.zip (1.2 KB)
Preview video captions
- Dragan Ahmetovic, Cole Gleason, Chengxiong Ruan, Kris Kitani, Hironobu Takagi, and Chieko Asakawa. 2016. NavCog: a navigational cognitive assistant for the blind. In Proc. ACM Conference on HumanComputer Interaction with Mobile Devices and Services (MobileHCI '16). ACM, 90--99. Google Scholar
Digital Library
- Tomohiro Amemiya, Jun Yamashita, Koichi Hirota, and Michitaka Hirose. 2004. Virtual leading blocks for the deaf-blind: A real-time way-finder by verbal-nonverbal hybrid interface and high-density RFID tag space. In Proc. IEEE Conference on Virtual Reality (VR '04). IEEE, 165--287. Google ScholarDigital Library
- Mauro Avila Soto, Markus Funk, Matthias Hoppe, Robin Boldt, Katrin Wolf, and Niels Henze. 2017. DroneNavigator: Using Leashed and Free-Floating Quadcopters to Navigate Visually Impaired Travelers. In Proc. ACM SIGACCESS Conference on Computers and accessibility (ASSETS '17). ACM, 300--304. Google ScholarDigital Library
- JM Bengamin. 1973. The new C-5 laser cane for the blind. In Conference on the electronic prosthetics, 1973.Google Scholar
- BB Blasch, SJ LaGrow, and WR De l'Aune. 1996. Three aspects of coverage provided by the long cane: Object, surface, and foot-placement preview. Journal of Visual Impairment and Blindness 90 (1996), 295-- 301.Google ScholarCross Ref
- GP Boucek Jr, James E Veitengruber, and Wayne D Smith. 1977. Aircraft alerting systems criteria study. Volume II. Human factors guidelines for aircraft alerting systems. Technical Report. BOEING COMMERCIAL AIRPLANE CO SEATTLE WA.Google Scholar
- Michael Brock and Per Ola Kristensson. 2013. Supporting blind navigation using depth sensing and sonification. In Proc. ACM International Joint Conference on Pervasive and ubiquitous computing adjunct publication (UbiComp '13). ACM, 255--258. Google ScholarDigital Library
- Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2017. Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields. In Proc. IEEE International Conference on Computer Vision and Pattern Recognition (CVPR '17). IEEE, 1302--1310.Google ScholarCross Ref
- Ping-Ling Chen, Wafaa Saleh, and Chih-Wei Pai. 2017. Texting and walking: a controlled field study of crossing behaviours and inattentional blindness in Taiwan. Behaviour & Information Technology 36, 4 (2017), 435--445. Google ScholarDigital Library
- Sakmongkon Chumkamon, Peranitti Tuvaphanthaphiphat, and Phongsak Keeratiwintakorn. 2008. A blind navigation system using RFID for indoor environments. In Proc. IEEE International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON '08), Vol. 2. IEEE, 765--768.Google ScholarCross Ref
- Karen Duarte, José Cecílio, and Pedro Furtado. 2014. Easily guiding of blind: Providing information and navigation-smartnav. In Wireless Internet. Springer, 129--134.Google Scholar
- Judy Edworthy, Sarah Loxley, and Ian Dennis. 1991. Improving auditory warning design: Relationship between warning sound parameters and perceived urgency. Human factors 33, 2 (1991), 205--231.Google Scholar
- Navid Fallah, Ilias Apostolopoulos, Kostas Bekris, and Eelke Folmer. 2012. The user as a sensor: navigating users with visual impairments in indoor spaces using tactile landmarks. In Proc. ACM CHI Conference on Human Factors in Computing Systems (CHI '12). ACM, 425--432. Google ScholarDigital Library
- Navid Fallah, Ilias Apostolopoulos, Kostas Bekris, and Eelke Folmer. 2013. Indoor human navigation systems: A survey. Interacting with Computers 25, 1 (2013), 21--33.Google Scholar
- José Faria, Sérgio Lopes, Hugo Fernandes, Paulo Martins, and João Barroso. 2010. Electronic white cane for blind people navigation assistance. In Proc. IEEE Conference on World Automation Congress (WAC '10). IEEE, 1--7.Google Scholar
- Vítor Filipe, Filipe Fernandes, Hugo Fernandes, António Sousa, Hugo Paredes, and João Barroso. 2012. Blind navigation support system based on Microsoft Kinect. Procedia Computer Science 14 (2012), 94-- 101.Google ScholarCross Ref
- Masahiro Furukawa, Hiromi Yoshikawa, Taku Hachisu, Shogo Fukushima, and Hiroyuki Kajimoto. 2011. "Vection field" for pedestrian traffic control. In Proc. ACM International Conference on Augmented Human (AH '11). ACM, 19. Google ScholarDigital Library
- Thomas Gallagher, Elyse Wise, Binghao Li, Andrew G Dempster, Chris Rizos, and Euan Ramsey-Stewart. 2012. Indoor positioning system based on sensor fusion for the blind and visually impaired. In Proc. IEEE International Conference on Indoor Positioning and Indoor Navigation (IPIN '12). IEEE, 1--9.Google ScholarCross Ref
- Christian Gonzalez, Bridget A Lewis, Daniel M Roberts, Stephanie M Pratt, and Carryl L Baldwin. 2012. Perceived urgency and annoyance of auditory alerts in a driving context. 56, 1 (2012), 1684--1687.Google Scholar
- Ellen C Haas and Judy Edworthy. 1996. Designing urgency into auditory warnings using pitch, speed and loudness. Computing & Control Engineering Journal 7, 4 (1996), 193--198.Google ScholarCross Ref
- Juan David Hincapié-Ramos and Pourang Irani. 2013. CrashAlert: enhancing peripheral alertness for eyes-busy mobile interaction while walking. In Proc. ACM CHI Conference on Human Factors in Computing Systems (CHI '13). ACM, 3385--3388. Google ScholarDigital Library
- Hsieh-Chang Huang, Ching-Tang Hsieh, and Cheng-Hsiang Yeh. 2015. An indoor obstacle detection system using depth information and region growth. Sensors 15, 10 (2015), 27116--27141.Google ScholarCross Ref
- Andreas Hub, Joachim Diepstraten, and Thomas Ertl. 2004. Design and development of an indoor navigation and object identification system for the blind. In Proc. ACM SIGACCESS Conference on Computers and accessibility (ASSETS '04). ACM, 147--152. Google ScholarDigital Library
- Kiyohide Ito, Makoto Okamoto, Junichi Akita, Tetsuo Ono, Ikuko Gyobu, Tomohito Takagi, Takahiro Hoshi, and Yu Mishima. 2005. CyARM: an alternative aid device for blind persons. In Proc. ACM CHI Extended Abstracts on Human Factors in Computing Systems (CHI '05). ACM, 1483--1488. Google ScholarDigital Library
- Beomjoon Kim and Joelle Pineau. 2016. Socially Adaptive Path Planning in Human Environments Using Inverse Reinforcement Learning. International Journal of Social Robotics 8, 1 (2016), 51--66.Google ScholarCross Ref
- Jee-Eun Kim, Masahiro Bessho, Shinsuke Kobayashi, Noboru Koshizuka, and Ken Sakamura. 2016. Navigating visually impaired travelers in a large train station using smartphone and bluetooth low energy. In Proc. ACM Annual Symposium on Applied Computing (AC '16). ACM, 604--611. Google ScholarDigital Library
- Kris M. Kitani, Brian D. Ziebart, James Andrew Bagnell, and Martial Hebert. 2012. Activity Forecasting. In Proc. IEEE European Conference on Computer Vision (ECCV '12), Andrew Fitzgibbon, Svetlana Lazebnik, Pietro Perona, Yoichi Sato, and Cordelia Schmid (Eds.). Springer Berlin Heidelberg, 201--214. Google ScholarDigital Library
- Vladimir Kulyukin, Chaitanya Gharpure, John Nicholson, and Grayson Osborne. 2006. Robot-assisted wayfinding for the visually impaired in structured indoor environments. Autonomous Robots 21, 1 (2006), 29--41. Google ScholarDigital Library
- Bing Li, J Pablo Munoz, Xuejian Rong, Jizhong Xiao, Yingli Tian, and Aries Arditi. 2016. ISANA: wearable context-aware indoor assistive navigation with obstacle avoidance for the blind. In Proc. IEEE European Conference on Computer Vision (ECCV '16). Springer, 448--462.Google ScholarCross Ref
- Shachar Maidenbaum, Shlomi Hanassy, Sami Abboud, Galit Buchs, Daniel-Robert Chebat, Shelly Levy-Tzedek, and Amir Amedi. 2014. The "EyeCane", a new electronic travel aid for the blind: Technology, behavior & swift learning. Restorative neurology and neuroscience 32, 6 (2014), 813--824.Google Scholar
- Roberto Manduchi, Sri Kurniawan, and Homayoun Bagherinia. 2010. Blind guidance using mobile computer vision: A usability study. In Proc. ACM SIGACCESS Conference on Computers and accessibility (ASSETS '10). ACM, 241--242. Google ScholarDigital Library
- Dawn C Marshall, John D Lee, and P Albert Austria. 2007. Alerts for in-vehicle information systems: Annoyance, urgency, and appropriateness. Human factors 49, 1 (2007), 145--157.Google Scholar
- Edward Marshall and Sue Baker. 1995. Alarms in nuclear power plant control rooms: current approaches and future design. In Human factors in alarm design. Taylor & Francis, Inc., 183--191. Google ScholarDigital Library
- Christina Meredith and Judy Edworthy. 1995. Are there too many alarms in the intensive care unit? An overview of the problems. Journal of advanced nursing 21, 1 (1995), 15--20.Google ScholarCross Ref
- Madoka Nakajima and Shinichiro Haruyama. 2012. Indoor navigation system for visually impaired people using visible light communication and compensated geomagnetic sensing. In Proc. IEEE International Conference on Communications in China (ICCC '12). IEEE, 524--529.Google ScholarCross Ref
- Helen Petrie, Valerie Johnson, Thomas Strothotte, Andreas Raab, Steffi Fritz, and Rainer Michel. 1996. MoBIC: Designing a travel aid for blind and elderly people. The Journal of Navigation 49, 1 (1996), 45--52.Google ScholarCross Ref
- Huy-Hieu Pham, Thi-Lan Le, and Nicolas Vuillerme. 2016. Real-time obstacle detection system in indoor environment for the visually impaired using microsoft kinect sensor. Journal of Sensors 2016 (2016).Google Scholar
- Ioannis Politis, Stephen A Brewster, and Frank Pollick. 2014. Evaluating multimodal driver displays under varying situational urgency. In Proc. ACM CHI Conference on Human Factors in Computing Systems (CHI '14). ACM, 4067--4076. Google ScholarDigital Library
- Lisa Ran, Sumi Helal, and Steve Moore. 2004. Drishti: an integrated indoor/outdoor blind navigation system and service. In Proc. the Second IEEE Annual Conference on Pervasive Computing and Communications (PerCom '04). IEEE, 23--30. Google ScholarDigital Library
- Joseph Redmon and Ali Farhadi. 2017. YOLO9000: Better, Faster, Stronger. In Proc. IEEE International Conference on Computer Vision and Pattern Recognition (CVPR '17). IEEE, 6517--6525.Google Scholar
- Shadan Sadeghian Borojeni, Susanne CJ Boll, Wilko Heuten, Heinrich H Bülthoff, and Lewis Chuang. 2018. Feel the Movement: Real Motion Influences Responses to Take-over Requests in Highly Automated Vehicles. In Proc. ACM CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, 246. Google ScholarDigital Library
- Daisuke Sato, Uran Oh, Kakuya Naito, Hironobu Takagi, Kris Kitani, and Chieko Asakawa. 2017. Navcog3: An evaluation of a smartphonebased blind indoor navigation assistant with semantic features in a large-scale environment. In Proc. ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17). ACM, 270--279. Google ScholarDigital Library
- Shraga Shoval, Johann Borenstein, and Yoram Koren. 1994. Mobile robot obstacle avoidance in a computerized travel aid for the blind. In Proc. IEEE International Conference on Robotics and Automation. IEEE, 2023--2028.Google ScholarCross Ref
- Shraga Shoval, Johann Borenstein, and Yoram Koren. 1998. The Navbelt-A computerized travel aid for the blind based on mobile robotics technology. IEEE Transactions on Biomedical Engineering 45, 11 (1998), 1376--1386.Google ScholarCross Ref
- Yu-Chih Tung and Kang G Shin. 2018. Use of Phone Sensors to Enhance Distracted Pedestrians' Safety. IEEE Transactions on Mobile Computing 17, 6 (2018), 1469--1482.Google ScholarCross Ref
- Iwan Ulrich and Johann Borenstein. 2001. The GuideCane-applying mobile robot technologies to assist the visually impaired. IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans 31, 2 (2001), 131--136. Google ScholarDigital Library
- William R Wiener, Richard L Welsh, and Bruce B Blasch. 2010. Foundations of orientation and mobility. Vol. 1. American Foundation for the Blind.Google Scholar
- Takuma Yagi, Karttikeya Mangalam, Ryo Yonetani, and Yoichi Sato. 2018. Future Person Localization in First-Person Videos. In Proc. IEEE International Conference on Computer Vision and Pattern Recognition (CVPR '18).Google ScholarCross Ref
- Cang Ye, Soonhac Hong, Xiangfei Qian, and Wei Wu. 2016. Co-robotic cane: A new robotic navigation aid for the visually impaired. IEEE Systems, Man, and Cybernetics Magazine 2, 2 (2016), 33--42.Google ScholarCross Ref
- Limin Zeng, Markus Simros, and Gerhard Weber. 2017. Camera-based mobile electronic travel aids support for cognitive mapping of unknown spaces. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '17). ACM, 8. Google ScholarDigital Library
- Xucong Zhang, Yusuke Sugano, Mario Fritz, and Andreas Bulling. 2017. Mpiigaze: Real-world dataset and deep appearance-based gaze estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence (2017).Google Scholar
Index Terms
- BBeep: A Sonic Collision Avoidance System for Blind Travellers and Nearby Pedestrians
Recommendations
CaBot: Designing and Evaluating an Autonomous Navigation Robot for Blind People
ASSETS '19: Proceedings of the 21st International ACM SIGACCESS Conference on Computers and AccessibilityNavigation robots have the potential to overcome some of the limitations of traditional navigation aids for blind people, specially in unfamiliar environments. In this paper, we present the design of CaBot (Carry-on roBot), an autonomous suitcase-shaped ...
Guiding Blind Pedestrians in Public Spaces by Understanding Walking Behavior of Nearby Pedestrians
We present a guiding system to help blind people walk in public spaces while making their walking seamless with nearby pedestrians. Blind users carry a rolling suitcase-shaped system that has two RGBD Cameras, an inertial measurement unit (IMU) sensor, ...
A biologically inspired method for robot navigation in a cluttered environment
The problem of wheeled mobile robot (WMR) navigation toward an unknown target in a cluttered environment has been considered. The biologically inspired navigation algorithm is the equiangular navigation guidance (ENG) law combined with a local obstacle ...
Comments