Ruilin Liu
ABSTRACT
Remote driving brings human operators with sophisticated perceptual and cognitive skills into an over-the-network control loop, with the hope of addressing the challenging aspects of vehicular autonomy based exclusively on artificial intelligence (AI). This paper studies the human behavior in a remote driving setup, i.e., how human remote drivers perform and assess their workload under the state-of-the-art network conditions. To explore this, we build a scaled remote driving prototype and conduct a controlled human study with varying network delays based on current commercial LTE network technology. The study demonstrates that remote driving over LTE is not immediately feasible, primarily caused by network delay variability rather than delay magnitude. In addition, our findings indicate that the negative effects of remote driving over LTE can be mitigated by a video frame arrangement strategy that regulates delay magnitude to achieve a smoother display.
- A Aashto. 2001. Policy on geometric design of highways and streets. American Association of State Highway and Transportation Officials, Washington, DC 3 (2001), 3--2.Google Scholar
- Evan Ackerman. 2015. Ford Developing Cross Country Automotive Remote Control. http://spectrum.ieee.org/cars-that-think/transportation/advanced-cars/ford-developing-cross-country-automotive-remote-control. (Feb. 2015).Google Scholar
- Evan Ackerman. 2016. Fatal Tesla self-driving car crash reminds us that robots aren't perfect. IEEE-Spectrum 1 (2016).Google Scholar
- Nicola Bui, Foivos Michelinakis, and Joerg Widmer. 2017. Fine-grained LTE Radio Link Estimation for Mobile Phones. In The 18th International Symposium on a World of Wireless Mobile and Multimedia Networks (WoWMoM 2017).Google Scholar
- Jessie YC Chen, Ellen C Haas, and Michael J Barnes. 2007. Human performance issues and user interface design for teleoperated robots. Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on 37, 6 (2007), 1231--1245. Google ScholarDigital Library
- Frederic Chucholowski, Stefan Büchner, Johannes Reicheneder, and Markus Lienkamp. 2013. Prediction methods for teleoperated road vehicles. In Conference on Future Automotive Technology-Focus Electromobility, Garching: Bayern Innovativ GmbH.Google Scholar
- Douglas W Cunningham, Astros Chatziastros, Markus Von der Heyde, and Heinrich H Bülthoff. 2001. Driving in the future: Temporal visuomotor adaptation and generalization. Journal of Vision 1, 2 (2001), 3.Google ScholarCross Ref
- Alex Davies. 2017. Nissan's Path to Self-driving Cars? Humans in Call Centers. https://www.wired.com/2017/01/nissans-self-driving-teleoperation/. (2017).Google Scholar
- James Davis, Christopher Smyth, and Kaleb McDowell. 2010. The effects of time lag on driving performance and a possible mitigation. Robotics, IEEE Transactions on 26, 3 (2010), 590--593. Google ScholarDigital Library
- Stephen R Ellis, Bernard D Adelstein, S Baumeler, GJ Jense, and Richard H Jacoby. 1999. Sensor spatial distortion, visual latency, and update rate effects on 3D tracking in virtual environments. In Virtual Reality, 1999. Proceedings., IEEE. IEEE, 218--221. Google ScholarDigital Library
- Guizzo Erico. 2013. How Google's self-driving car works. IEEE Spectrum 18 (2013).Google Scholar
- Darrell Etherington. 2017. Starsky Robotics' autonomous transport trucks also give drivers remote control. https://techcrunch.com/2017/02/28/starsky-robotics-autonomous-transport-trucks-also-give-drivers-remote-control/. (Feb. 2017).Google Scholar
- Peter Fairley. 2017. Self-driving cars have a bicycle problem {News}. IEEE Spectrum 54, 3 (2017), 12--13. Google ScholarDigital Library
- Franz Faul, Edgar Erdfelder, Albert-Georg Lang, and Axel Buchner. 2007. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods 39, 2 (01 May 2007), 175--191.Google Scholar
- FierceWireless. 2014a. 3G/4G wireless network latency: How do Verizon, AT&T, Sprint and T-Mobile compare? http://www.fiercewireless.com/special-reports/3g4g-wireless-network-latency-how-did-verizon-att-sprint-and-t-mobile-compa-3. (Aug. 2014).Google Scholar
- FierceWireless. 2014b. Mosaik Solutions shows quarter-by-quarter LTE coverage development in the U.S. http://www.fiercewireless.com/story/mosaik-solutions-shows-quarter-quarter-lte-coverage-development-us/2614-11-21/. (Nov. 2014).Google Scholar
- Sebastian Gnatzig, Frederic Chucholowski, Tito Tang, and Markus Lienkamp. 2013. A System Design for Teleoperated Road Vehicles. In ICINCO (2). 231--238.Google Scholar
- Andrĺęs Gonzĺćlez. 2017. New mobile networks set to enable remote driving, Telefonica says. http://in.reuters.com/article/telecoms-mobileworld-remote-driving-idINKBN1662H7http. (2017).Google Scholar
- Mark Harris. 2015. Uber could be first to test completely driverless cars in public. IEEE Spectrum 14 (2015), 2015.Google Scholar
- Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Advances in psychology 52 (1988), 139--183.Google Scholar
- Amin Hosseini and Markus Lienkamp. 2016a. Enhancing telepresence during the teleoperation of road vehicles using HMD-based mixed reality. In Intelligent Vehicles Symposium (IV), 2016 IEEE. IEEE, 1366--1373.Google ScholarCross Ref
- Amin Hosseini and Markus Lienkamp. 2016b. Predictive safety based on track-before-detect for teleoperated driving through communication time delay. In Intelligent Vehicles Symposium (IV), 2016 IEEE. IEEE, 165--172.Google ScholarCross Ref
- Amin Hosseini, Florian Richthammer, and Markus Lienkamp. 2016. Predictive haptic feedback for safe lateral control of teleoperated road vehicles in urban areas. In Vehicular Technology Conference (VTC Spring), 2016 IEEE 83rd. IEEE, 1--7.Google ScholarCross Ref
- Amin Hosseini, Thomas Wiedemann, and Markus Lienkamp. 2014. Interactive path planning for teleoperated road vehicles in urban environments. In Intelligent Transportation Systems (ITSC), 2014 IEEE 17th International Conference on. IEEE, 400--405.Google ScholarCross Ref
- Junxian Huang, Feng Qian, Alexandre Gerber, Z Morley Mao, Subhabrata Sen, and Oliver Spatscheck. 2012. A close examination of performance and power characteristics of 4G LTE networks. In Proceedings of the 10th international conference on Mobile systems, applications, and services. ACM, 225--238. Google ScholarDigital Library
- Huawei. 2017. Huawei Demonstrates 5G-based Remote Driving with China Mobile and SAIC Motor. http://www.huawei.com/en/news/2017/6/5G-based-Remote-Driving. (2017).Google Scholar
- Audrius Jurgelionis, J Laulajainen, Matti Hirvonen, and Alf Inge Wang. 2011. An empirical study of netem network emulation functionalities. In Computer Communications and Networks (ICCCN), 2011 Proceedings of 20th International Conference on. IEEE, 1--6.Google ScholarCross Ref
- Jennifer Kay and Charles Thorpe. 1995. Semi-Autonomous Vehicle Teleoperation Across Low-Bandwidth or High-Delay Transmission Links. In International Joint Conference on Artificial Intelligence.Google Scholar
- Will Knight. 2013. Driverless Cars Are Further Away Than You Think. http://www.technologyreview.com/featuredstory/520431/driverless-cars-are-further-away-than-you-think/. (Oct. 2013).Google Scholar
- J.C. Lane, C.R. Carignan, B.R. Sullivan, D.L. Akin, T. Hunt, and R. Cohen. 2002. Effects of time delay on telerobotic control of neutral buoyancy vehicles. In Robotics and Automation, 2002. Proceedings. ICRA '02. IEEE International Conference on, Vol. 3. 2874--2879.Google Scholar
- Yuanjie Li, Chunyi Peng, Zengwen Yuan, Jiayao Li, Haotian Deng, and Tao Wang. 2016. Mobileinsight: extracting and analyzing cellular network information on smartphones. In Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking. ACM, 202--215. Google ScholarDigital Library
- Jason P. Luck, Patricia L. McDermott, Laurel Allender, and Deborah C. Russell. 2006. An Investigation of Real World Control of Robotic Assets Under Communication Latency. In Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-robot Interaction (HRI '06). ACM, New York, NY, USA, 202--209. Google ScholarDigital Library
- Erik Nygren, Ramesh K Sitaraman, and Jennifer Sun. 2010. The akamai network: a platform for high-performance internet applications. ACM SIGOPS Operating Systems Review 44, 3 (2010), 2--19. Google ScholarDigital Library
- Nicolas Oros and Jeffrey L Krichmar. 2013. Smartphone Based Robotics: Powerful, Flexible and Inexpensive Robots for Hobbyists, Educators, Students and Researchers. IEEE Robotics & Automation Magazine (2013).Google Scholar
- Bob Ricks, Curtis W Nielsen, Michael Goodrich, and others. 2004. Ecological displays for robot interaction: A new perspective. In Intelligent Robots and Systems, 2004.(IROS 2004). Proceedings. 2004 IEEE/RSJ International Conference on, Vol. 3. IEEE, 2855--2860.Google ScholarCross Ref
- Sascha Segan. 2013. Fastest Mobile Networks 2013. http://www.pcmag.com/article2/0,2817,2420334,00.asp. (June 2013).Google Scholar
- TATA Consultancy Services. 2017. WANem The Wide Area Network emulator. http://wanem.sourceforge.net/. (2017).Google Scholar
- Thomas B Sheridan. 1993. Space teleoperation through time delay: review and prognosis. Robotics and Automation, IEEE Transactions on 9, 5 (1993), 592--606.Google Scholar
- Thomas B Sheridan and William R Ferrell. 1963. Remote manipulative control with transmission delay. IEEE Transactions on Human Factors in Electronics 4, 1 (1963), 25--29.Google ScholarCross Ref
- Tito Tang, Pascal Vetter, Simon Finkl, Korbinian Figel, and Markus Lienkamp. 2014. Teleoperated Road Vehicles--The" Free Corridor" as a Safety Strategy Approach. In Applied Mechanics and Materials, Vol. 490. Trans Tech Publ, 1399--1409.Google ScholarCross Ref
- Steve Vozar and Dawn M Tilbury. 2014. Driver modeling for teleoperation with time delay. IFAC Proceedings Volumes 47, 3 (2014), 3551--3556.Google ScholarCross Ref
- Youtube. 2017. Live encoder settings, bitrates and resolutions. https://support.google.com/youtube/answer/2853702?hl=en. (2017).Google Scholar
Index Terms
- Investigating Remote Driving over the LTE Network
Recommendations
Driving from a Distance: Challenges and Guidelines for Autonomous Vehicle Teleoperation Interfaces
CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing SystemsAutonomous vehicle (AV) technologies are rapidly evolving with the vision of having self-driving cars moving safely with no human input. However, it is clear that at least in the near and foreseeable future, AVs will not be able to resolve all road ...
Autonomous Driving: Investigating the Feasibility of Bimodal Take-Over Requests
Autonomous vehicles will need de-escalation strategies to compensate when reaching system limitations. Car-driver handovers can be considered one possible method to deal with system boundaries. The authors suggest a bimodal auditory and visual handover ...
A cost efficient multi remote driver selection for remote operated vehicles
Highlights- We propose a novel multi driver remote driving for autonomous vehicles.
- We ...
AbstractTechnological, social and economic forces are changing the notion of mobility as culture is exerting strong pressure on demand for a “shared,” “reliable” and “on-demand” economy. In an era of the next mobility revolution, autonomous ...
Comments