Adam Eric Leeper
Leila Takayama
ABSTRACT
Human-in-the loop robotic systems have the potential to handle complex tasks in unstructured environments, by combining the cognitive skills of a human operator with autonomous tools and behaviors. Along these lines, we present a system for remote human-in-the-loop grasp execution. An operator uses a computer interface to visualize a physical robot and its surroundings, and a point-and-click mouse interface to command the robot. We implemented and analyzed four different strategies for performing grasping tasks, ranging from direct, real-time operator control of the end-effector pose, to autonomous motion and grasp planning that is simply adjusted or confirmed by the operator. Our controlled experiment (N=48) results indicate that people were able to successfully grasp more objects and caused fewer unwanted collisions when using the strategies with more autonomous assistance. We used an untethered robot over wireless communications, making our strategies applicable for remote, human-in-the-loop robotic applications.
Supplemental Material
- A. Bettini, P. Marayong, S. Lang, A. Okamura, and G. Hager. Vision-Assisted Control for Manip. Using Virtual Fixtures. IEEE Trans. on Robotics, 20(6), 2004. Google Scholar
Digital Library
- M. Ciocarlie, K. Hsiao, E. Jones, S. Chitta, R. Rusu, and I. Sucan. Towards reliable grasping and manipulation in household environments. In ISER, 2010.Google Scholar
- H. Das. Kinematic Control and Visual Display of Redundant Teleoperators. PhD thesis, MIT, 1989.Google Scholar
- W. Griffin, W. Provancher, and M. Cutkosky. Feedback Strategies for Telemanipulation with Shared Control of Object Handling Forces. Presence: Teleoperators and Virtual Environments, 14(6):720--731, Dec. 2005. Google ScholarDigital Library
- S. Hart. Nasa-task load index (nasa-tlx); 20 years later. In HFES, 2006.Google ScholarCross Ref
- S. Hayati and S. Venkataraman. Design and implementation of a robot control system with traded and shared control capability. In ICRA, 1989.Google ScholarCross Ref
- G. Hirzinger, B. Brunner, J. Dietrich, and J. Heindl. Sensor-Based Space Robotics--ROTEX and Its Telerobotic Features. IEEE Transactions on Robotics and Automation, 9(5), 1993.Google Scholar
- E. Horvitz. Principles of mixed-initiative user interfaces. In CHI, 1999. Google ScholarDigital Library
- K. Hsiao, S. Chitta, M. Ciocarlie, and E. Jones. Contact-reactive grasping of objects with partial shape information. In IROS, 2010.Google ScholarCross Ref
- J. Lee and K. See. Trust in automation: Designing for appropriate reliance. Human Factors: The Journal of the Human Factors and Erg. Society, 46(1):50--80, 2004.Google ScholarCross Ref
- P. Maes. Agents that reduce work and information overload. Communications of the ACM, pages 30--41, 1994. Google ScholarDigital Library
- M. Maybury. Intelligent user interfaces: An introduction. In IUI, pages 3--4, 1999. Google ScholarDigital Library
- P. Michelman and P. Allen. Shared autonomy in a robot hand teleoperation system. In IROS, 1994.Google ScholarCross Ref
- M. Oda, N. Inaba, Y. Takano, S. Nishida, M. Kayashi, and Y. Sugano. Onboard local compensation on ETS-W space robot teleoperation. In IEEE/ASME Intl. Conf. on Advanced Intelligent Mechatronics, 1999.Google ScholarCross Ref
- M. K. O'Malley, A. Gupta, M. Gen, and Y. Li. Shared Control in Haptic Systems for Performance Enhancement and Training. Journal of Dynamic Systems, Measurement, and Control, 128(1), 2006.Google Scholar
- H. Pongrac, A. Peer, B. Farber, and M. Buss. Effects of varied human movement control on task performance and feeling of telepresence. In EuroHaptics, 2008. Google ScholarDigital Library
- J. Rotter. Generalized expectancies of internal versus external control for reinforcements. Psychological Monographs, 80, 1966.Google Scholar
- S. Schneider and R. Cannon. Experimental object-level strategic control with cooperating manipulators. Intl. Journal of Robotics Research, 12(4), 1993.Google ScholarCross Ref
- T. Sheridan. Telerobotics, Automation, and Human Supervisory Control. MIT Press, Cambridge, MA, 1992. Google ScholarDigital Library
- E. You and K. Hauser. Assisted Teleoperation Strategies for Aggressively Controlling a Robot Arm with 2D Input. In RSS, 2011.Google ScholarCross Ref
Index Terms
- Strategies for human-in-the-loop robotic grasping
Recommendations
Intuitive interaction for robotic grasping
WASA '12: Proceedings of the Workshop at SIGGRAPH AsiaTo explore an intuitive human-robot interface for service robots, this work develops effective and user-friendly approaches which combine the real time remote vision-based teleoperation and autonomous grasping. In the first approach, hand gestures are ...
On Visual Servoing to Improve Performance of Robotic Grasping
CRV '15: Proceedings of the 2015 12th Conference on Computer and Robot VisionWe introduce image based visual servoing (IBVS) into a shared autonomy grasping system to improve its performance. Visual servoing is a technique that uses visual input to control a dynamic system, such as a robot. Autonomous grasp planning is used to ...
Vocal Interaction with a 7-DOF Robotic Arm for Object Detection, Learning and Grasping
HRI '16: The Eleventh ACM/IEEE International Conference on Human Robot InteractionThis work presents preliminary results on the de- velopment of a system for multimodal interaction between a user and a robotic arm for human-robot cooperation tasks. In particular the system allows the user to ask the robot to grasp objects lying on a ...
Comments