Elsevier

Robotics and Autonomous Systems

Volume 107, September 2018, Pages 221-235
Robotics and Autonomous Systems

Aerial manipulation—A literature survey

https://doi.org/10.1016/j.robot.2018.06.012Get rights and content

Highlights

  • Comprehensive survey on unmanned aerial manipulator applications.

  • Overview of UAV platforms and manipulation/interaction mechanisms.

  • Overview of missions and operational scenarios.

  • Overview of the design, modeling, estimation/control of unmanned aerial manipulators.

  • Identification of current shortcoming and gaps and suggesting future directions.

Abstract

This paper presents a literature survey on aerial manipulation. First of all, an extensive study of aerial vehicles and manipulation/interaction mechanisms in aerial manipulation is presented. Various combinations of aerial vehicles and manipulators and their applications in different missions are discussed. Next, two main modeling methods and a detailed investigation of existing estimation and control techniques in aerial manipulation are explained. Finally the shortcomings of current aerial manipulation research are highlighted and a number of directions for future research are suggested.

Introduction

In recent years, a significant growth in Unmanned Aerial Vehicle (UAV) industry has been realized. As an example, in the United States only, there were approximately a million UAV or “drone” gifts for Christmas 2015 [1]. To this date, UAVs have been used in applications such as remote sensing of agricultural products [2], forest fire monitoring [3], search and rescue [4], border monitoring [5], transmission line inspection [6], and plant assets inspection [7]. Fully functional UAVs for plant inspection have appeared as recently as 2010 for UK onshore oil refineries [8]. In 2012, the supermajor oil and gas company, British Petroleum, established research teams to develop the necessary technologies to use UAVs for oil pipeline inspection in Prudhoe, Alaska [9] and over the course of only a few years, the technology has matured to become the standard practice for onshore and offshore platforms [7]. The above achievements have benefited various industries tremendously; however, an important common shortcoming in the mentioned applications is that the UAV is employed to merely sense, monitor and “see” the environment, but physical interaction with the environment is strictly avoided. Motivated by this, researchers in the last few years have begun examining applications in which a UAV is required to perform perching, grasping, and manipulation [[10], [11], [12], [13], [14], [15]]. This new area of research, usually known as aerial manipulation, encourages physical interaction of the UAV with its surrounding environment and enables UAVs to perform a whole new set of missions.

Aerial manipulation falls within a well-studied broad research category known as mobile manipulation. However, most of the research carried out in mobile manipulation focuses on ground robots. The main distinct challenges in the aerial manipulation problem are:

  • 1.

    Unlike ground robots, UAVs do not have a stable base and therefore forces and torques generated by the presence and movement of the manipulation mechanism and/or the payload directly affect the vehicle’s position, attitude and even its stability;

  • 2.

    Unlike ground robots, the performance of UAVs’ propulsion system vary in close vicinity of the ground and/or walls;

  • 3.

    UAVs are often underactuated platforms with highly nonlinear coupled dynamics, introducing further complications into their control design; and

  • 4.

    UAVs usually have stringent payload weight constraints and therefore cannot accommodate industrial dexterous robotic manipulators.

The above challenges encourage the development of a new research theme for the aerial manipulation problem.

An aerial manipulation system, viz. Unmanned Aerial Manipulator (UAM) hereafter, consists of two subsystems, namely a UAV and an interaction/manipulation mechanism (such as a robotic manipulator or a rigid tool) employed to physically interact with the environment. A rich amount of research literature and a number of review papers have been published on either of the above subsystems. As an example, in [16], a comprehensive survey of control algorithms for UAVs was presented. In that work, a number of schemes such as Proportional–Derivative–Integral (PID), Linear Quadratic Regulator (LQR), H, sliding mode variable structure, backstepping, and adaptive control along with their advantages and drawbacks in the control of UAVs with Vertical Take-Off and Landing (VTOL) capabilities, e.g. the quadcopter, were discussed. A detailed review of motion planning and trajectory planning algorithms for UAV guidance was also presented in [17]. Later, a review of path planning algorithms in the presence of disturbances and uncertainty was given in [18]. Also, a comprehensive literature survey on manipulation and grasping in robotic manipulation was given in [19]. While the above works summarize a broad body of literature on UAVs and robotic manipulators, they do not specifically discuss UAMs. In fact, to the best of authors’ knowledge, there is no published review paper on aerial manipulation including mission scenarios, mathematical modeling, and control schemes used in UAMs.

Aerial manipulation is a new field of research. Some of the pioneering works in this area appeared in the first years of the current decade [[10], [11], [20], [21], [22]] where the manipulation usually consisted of a gripper rigidly attached to a UAV body or was based on tethered configurations. Over the course of a few years, aerial manipulation has considerably evolved and more recent works, e.g. [[14], [15], [23], [24], [25], [26]], address challenging problems such as valve turning (see Fig. 1) and pick-and-place by several Degrees-of- Freedom (DoF) robotic manipulators. The authors believe that the coming years will bring further advancement in aerial manipulation and will enable more practical and reliable UAMs in a variety of applications.

The rest of this paper is organized as follows. Section 2 describes the most commonly used UAVs and manipulation/interaction mechanisms in aerial manipulation systems. Section 3 thoroughly studies missions and scenarios realized, to this date, in the area of aerial manipulation. Two main modeling methodologies and various control schemes in aerial manipulation are presented in Sections 4 Modeling of a UAM, 5 Estimation and control of UAM, respectively. Conclusions and directions for future research are presented in Section 6.

Section snippets

The physical subsystems of UAMs

In general, an aerial manipulation system contains two main physical subsystems, a UAV platform and a manipulation mechanism, with the necessary sensors and control systems for its autonomous or semi-autonomous functionality. In this section, we describe the most common subsystems of a UAM as well as the possible sensory configurations considered for various applications.

Missions and operational scenarios

In this section, the missions and operational scenarios considered for UAMs are discussed.

Modeling of a UAM

In this section, kinematic and dynamic modeling of a UAM and its interaction with the environment are discussed. For the sake of simplicity, a UAM consisting of a quadcopter and a manipulator with nm DoF is considered. The UAM model presented in this section is similar to that in [23] and can be used for other aerial platforms (such as tri-rotors or ducted-fan vehicles) with minor modification. Consider the schematic view of a UAM in Fig. 7, where FI is an inertial coordinate frame, a frame FV

State and parameter estimation

Accurate and efficient estimation of the state and parameters of UAMs is an important element of autonomous aerial manipulation missions. However, dynamic modeling of a UAM results in coupled and highly nonlinear equations of motion and therefore conventional estimation algorithms are generally difficult to apply for UAM applications. Among the limited works in this context, a least square problem was formulated in [11], using control input and acceleration measurements, to estimate the mass,

Discussion and future directions

The area of aerial manipulation was studied in this paper. The findings are summarized as follows:

  • Various types of UAV platforms have been used in aerial manipulation applications. Strong hovering capability is the main characteristic that such vehicles should possess and therefore fixed-wing UAVs are not appropriate platforms for aerial manipulation applications. Among vehicles with hovering capability, quadcopters are the most popular ones, followed by helicopters. Other

Acknowledgment

This research was financially sponsored through a Discovery Grant # 203060-12 from the Natural Sciences and Engineering Research Council of Canada, NSERC.

Hossein Bonyan Khamseh received his B.Sc. and M.Sc. in aerospace engineering from Amirkabir University and Shahid Beheshti University in 2008 and 2010, respectively. He is currently pursuing his Ph.D. at Mechanical and Industrial Engineering Department of Ryerson University, Toronto, Canada. His current research interests include aerial manipulation, aerial robotics, estimation, control, and visual servoing. To this date, He has authored and co-authored more than 20 conference and journal

References (165)

  • M.T. Connolly, The use of multi rotor remotely operated aerial vehicles (ROAVs) as a method of close visually...
  • BP Alaska: Unmanned Aerial Vehicle (UAV) Pilot Testing, 2012. [Online]. Available:...
  • MichaelN. et al.

    Cooperative manipulation and transportation with aerial robots

    Auton. Robots

    (2011)
  • D. Mellinger, Q. Lindsey, M. Shomin, V. Kumar, Design, modeling, estimation and control for aerial grasping and...
  • J. Thomas, G. Loianno, K. Sreenath, V. Kumar, Toward image based visual servoing for aerial grasping and perching, in:...
  • G. Gioioso, M. Ryll, D. Prattichizzo, H.H. Bülthoff, A. Franchi, Turning a near-hovering controlled quadrotor into a 3D...
  • S. Kim, S. Choi, H.J. Kim, Aerial manipulation using a quadrotor with a two dof robotic arm, in: Proc. IEEE/RSJ...
  • OrsagM. et al.

    Hybrid adaptive control for aerial manipulation

    J. Intell. Rob. Syst.

    (2014)
  • HuaM.-D. et al.

    Introduction to feedback control of underactuated VTOLvehicles: A review of basic control design ideas and principles

    IEEE Control Syst.

    (2013)
  • GoerzenC. et al.

    A survey of motion planning algorithms from the perspective of autonomous UAV guidance

    J. Intell. Robot. Syst.

    (2010)
  • DadkhahN. et al.

    Survey of motion planning literature in the presence of uncertainty: Considerations for UAV guidance

    J. Intell. Robot. Syst.

    (2012)
  • A. Bicchi, V. Kumar, Robotic grasping and contact: A review, in: Proc. IEEE International Conference on Robotics and...
  • P.E.I. Pounds, D.R. Bersak, A.M. Dollar, Grasping from the air: Hovering capture and load stability, in: Proc. IEEE...
  • Q. Lindsey, D. Mellinger, V. Kumar, Construction of cubic structures with quadrotor teams, in: Proc. Robotics: Science...
  • FinkJ. et al.

    Planning and control for cooperative manipulation and transportation with aerial robots

    Int. J. Robot. Res.

    (2011)
  • K. Baizid, et al., Experiments on behavioral coordinated control of an unmanned aerial vehicle manipulator system, in:...
  • G. Garimella, M. Kobilarov, Towards model-predictive control for aerial pick-and-place, in: Proc. IEEE International...
  • C. Korpela, M. Orsag, P. Oh, Towards valve turning using a dual-arm aerial manipulator, in: Proc. IEEE/RSJ...
  • M. Orsag, C. Korpela, S. Bogdan, P. Oh, Valve turning using a dual-arm aerial manipulator, in: Proc. International...
  • AlexanderL.M. et al.

    Economics of transmission-line maintenance and repair by helicopter

    Trans. Am. Inst. Electr. Eng. Part III

    (1953)
  • AlexanderL.M. et al.

    Maintenance of transmission lines by helicopter

    Electr. Eng.

    (1953)
  • CooperH.G. et al.

    Helicopter use on fire suppression in the pacific northwest

    J. For.

    (1959)
  • PrebbleM.L. et al.

    The situation with respect to forest entomology and pathology in Canada, 1943 to 1953

    For. Chronicle

    (1954)
  • Little Jr.E.L.

    The helicopter for tropical botanical field work

    Ecology

    (1956)
  • NovakL.R.

    The evolution of the McDonnell model 120 crane helicopter

    J. Am. Helicopter Soc.

    (1960)
  • B. Woodley, H. Jones, E. Frew, L.D.S. Rock, A contestant in the 1997 international aerial robotics competition....
  • ShakerniaO. et al.

    Landing an unmanned air vehicle: Vision based motion estimation and nonlinear control

    Asian J. Control

    (1999)
  • C.S. Sharp, O. Shakernia, S.S. Sastry, A vision system for landing an unmanned aerial vehicle, in: Proc. IEEE...
  • O. Shakernia, R. Vidal, C.S. Sharp, Y. Ma, S. Sastry, Multiple view motion estimation and control for landing an...
  • J. Kimmett, J. Valasek, J. Junkins, Autonomous aerial refueling utilizing a vision based navigation system, in: Proc....
  • J. Kimmett, J. Valasek, J.L. Junkins, Vision based controller for autonomous aerial refueling, in: Proc. International...
  • ValasekJ. et al.

    Vision-based sensor and navigation system for autonomous air refueling

    J. Guid. Control Dyn.

    (2005)
  • C.M. Korpela, T.W. Danko, P.Y. Oh, Designing a system for mobile manipulation from an unmanned aerial vehicle, in:...
  • C. Korpela, et al., Flight stability in aerial redundant manipulators, in: Proc. IEEE International Conference on...
  • G. Heredia, et al., Control of a multirotor outdoor aerial manipulator, in: Proc. IEEE/RSJ International Conference on...
  • KobilarovM.

    Nonlinear trajectory control of multi-body aerial manipulators

    J. Intell. Robot. Syst.

    (2014)
  • G. Jiang, R. Voyles, Hexrotor UAV platform enabling dextrous aerial mobile manipulation, in: Proc. International Micro...
  • G. Arleo, F. Caccavale, G. Muscio, F. Pierri, Control of quadrotor aerial vehicles equipped with a robotic arm, in:...
  • H.-N. Nguyen, D. Lee, Hybrid force/motion control and internal dynamics of quadrotors for tool operation, in: Proc....
  • M. Orsag, C. Korpela, S. Bogdan, P. Oh, Lyapunov based model reference adaptive control for aerial manipulation, in:...
  • Cited by (0)

    Hossein Bonyan Khamseh received his B.Sc. and M.Sc. in aerospace engineering from Amirkabir University and Shahid Beheshti University in 2008 and 2010, respectively. He is currently pursuing his Ph.D. at Mechanical and Industrial Engineering Department of Ryerson University, Toronto, Canada. His current research interests include aerial manipulation, aerial robotics, estimation, control, and visual servoing. To this date, He has authored and co-authored more than 20 conference and journal papers and a book chapter. He has also served a number of roles including conference/journal reviewer and co-chair, guest editor, invited speaker, and instructor in various capacities.

    Farrokh Janabi-Sharifi (S’91–M’95–SM’02) is a Professor of Mechanical–Industrial Engineering and the Director of Robotics, Mechatronics and Manufacturing Automation Laboratory (RMAL) at Ryerson University, Toronto, Canada. He received the Ph.D. degree in Electrical and Computer Engineering from the University of Waterloo, Canada in 1995. He was NSERC postdoctoral fellow and instructor in the Center for Intelligent Machines and Department of Electrical–Computer Engineering of McGill University (1995–1997). His research interests span over optomechatronic systems with the focus on image-guided control and planning of robots. He has been a visiting professor in KAIST, Taejon, Korea; IRISA–INRIA, Rennes, France; and TUM, Munich, Germany. He has also been Organizer and/or Co-organizer of several international conferences on optomechatronic systems control. He was General Chair of 2010 International Symposium on Optomechatronic Technologies, Toronto, Canada. He currently serves as the Associate Editor of several journals including International Journal of Optomechatronics and technical editor of IEEE/ASME Transactions on Mechatronics.

    Abdelkader Abdessameud is an Assistant Professor at the school of science, engineering and technology (SSET), Pennsylvania State University, Harrisburg. He received a Ph.D. degree in Electrical and Computer Engineering (Robotics and Control) at the University of Western Ontario (UWO), Canada, in 2010. Prior joining Penn State Harrisburg, he held several post-doctoral positions at the UWO and Ryerson University, Toronto. From 2001 to 2007, he was affiliated to the University of Boumerdes, Algeria, as a full-time Instructor (Maître Assistant Chargé de Cours) and Research Assistant. His research interests lie in the broad area of robotics and control with applications to unmanned aerial vehicles, aerial manipulation, attitude estimation and control, and the cooperative control of multi-agent systems.

    View full text