Abstract
Multisensor data fusion is the process of combining observations from a number of different sensors to provide a robust and complete description of an environment or process of interest. Data fusion finds wide application in many areas of robotics such as object recognition, environment mapping, and localization.
This chapter has three parts: methods, architectures, and applications. Most current data fusion methods employ probabilistic descriptions of observations and processes and use Bayes’ rule to combine this information. This chapter surveys the main probabilistic modeling and fusion techniques including grid-based models, Kalman filtering, and sequential Monte Carlo techniques. This chapter also briefly reviews a number of nonprobabilistic data fusion methods. Data fusion systems are often complex combinations of sensor devices, processing, and fusion algorithms. This chapter provides an overview of key principles in data fusion architectures from both a hardware and algorithmic viewpoint. The applications of data fusion are pervasive in robotics and underly the core problem of sensing, estimation, and perception. We highlight two example applications that bring out these features. The first describes a navigation or self-tracking application for an autonomous vehicle. The second describes an application in mapping and environment modeling.
The essential algorithmic tools of data fusion are reasonably well established. However, the development and use of these tools in realistic robotics applications is still developing.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Abbreviations
- 2-D:
-
two-dimensional
- 3-D:
-
three-dimensional
- 4-D:
-
four-dimensional
- ASN:
-
active sensor network
- CAN:
-
controller area network
- CCI:
-
control command interpreter
- COTS:
-
commercial off-the-shelf
- COV:
-
characteristic output vector
- DDF:
-
decentralized data fusion
- DFRA:
-
distributed field robot architecture
- EKF:
-
extended Kalman filter
- GPS:
-
global positioning system
- GUI:
-
graphical user interface
- GV:
-
ground vehicle
- HO:
-
human operator
- ILLS:
-
instrumented logical sensor system
- IR:
-
infrared
- JDL:
-
joint directors of laboratories
- LSS:
-
logical sensor system
- MC:
-
Monte Carlo
- MESSIE:
-
multi expert system for scene interpretation and evaluation
- RCS:
-
real-time control system
- SAR:
-
synthetic aperture radar
- SFX:
-
sensor fusion effect
- SIR:
-
sampling importance resampling
- SMC:
-
sequential Monte Carlo
- UAV:
-
fusing air vehicle
References
S. Thrun, W. Burgard, D. Fox: Probabilistic Robotics (MIT Press, Cambridge 2005)
J.O. Berger: Statistical Decision Theory and Bayesian Analysis (Springer, Berlin, Heidelberg 1985)
A. Elfes: Sonar-based real-world mapping and navigation, IEEE Trans. Robotics Autom. 3(3), 249–265 (1987)
L. Matthies, A. Elfes: Integration of sonar and stereo range data using a grid-based representation, Proc. IEEE Int. Conf. Robotics Autom. (ICRA) (1988) pp. 727–733
L.D. Stone, C.A. Barlow, T.L. Corwin: Bayesian Multiple Target Tracking (Artech House, Norwood 1999)
Y. Bar-Shalom: Multi-Target Multi-Sensor Tracking (Artec House, Norwood 1990)
Y. Bar-Shalom, T.E. Fortmann: Tracking and Data Association (Academic, New York 1988)
P.S. Maybeck: Stochastic Models, Estimaton and Control (Academic, New York 1979)
W. Sorensen: Special issue on the applications of the Kalman filter, IEEE Trans. Autom. Control 28(3), 254–255 (1983)
B.D.O. Anderson, J.B. Moore: Optimal Filtering (Prentice Hall, Englewood Cliffs 1979)
J. Manyika, H.F. Durrant-Whyte: Data Fusion and Sensor Management: An Information-Theoretic Approach (Ellis Horwood, New York 1994)
S. Sukkarieh, E. Nettleton, J.H. Kim, M. Ridley, A. Goktogan, H. Durrant-Whyte: The ANSER project: Data fusion across multiple uninhabited air vehicles, Int. J. Robotics Res. 22(7), 505–539 (2003)
R.E. Moore: Interval Analysis (Prentice Hall, Englewood Cliffs 1966)
D. Dubois, H. Prade: Fuzzy Sets and Systems: Theory and Applications (Academic, New York 1980)
S. Blackman, R. Popoli: Design and Analysis of Modern Tracking Systems (Artec House, Boston 1999)
D. Pagac, E.M. Nebot, H. Durrant-Whyte: An evidential approach to map-building for autonomous vehicles, IEEE Trans. Robotics Autom. 14(4), 623–629 (1998)
D. Hall, J. Llinas: Handbook of Multisensor Data Fusion (CRC, Boca Raton 2001)
E.L. Waltz, J. Llinas: Sensor Fusion (Artec House, Boston 1991)
M. Kam, Z. Zhu, P. Kalata: Sensor fusion for mobile robot navigation, Proceedings IEEE 85, 108–119 (1997)
H. Carvalho, W. Heinzelman, A. Murphy, C. Coelho: A general data fusion architecture, Proc. 6th Int. Conf. Inf. Fusion, Cairns (2003)
A. Makarenko: A Decentralized Architecture for Active Sensor Networks, Ph.D. Thesis (University of Sydney, Sydney 2004)
M. Dekhil, T. Henderson: Instrumented logical sensors systems, Int. J. Robotics Res. 17(4), 402–417 (1998)
J.A. Profeta: Safety-critical systems built with COTS, Computer 29(11), 54–60 (1996)
H. Hu, J.M. Brady, F. Du, P. Probert: Distributed real-time control of a mobile robot, J. Intell. Autom. Soft Comput. 1(1), 63–83 (1995)
S.A. Schneider, V. Chen, G. Pardo: ControlShell: A real-time software framework, AIAA Conf. Intell. Robotics Field Fact. Serv. Space (1994)
D. Simon, B. Espiau, E. Castillo, K. Kapellos: Computer-aided design of a generic robot controller handling reactivity and real-time issues, IEEE Trans. Control Syst. Technol. 4(1), 213–229 (1993)
C. Giraud, B. Jouvencel: Sensor selection in a fusion process: a fuzzy approach, Proc. IEEE Int. Conf. Multisens. Fusion Integr., Las Vegas (1994) pp. 599–606
R. Kapur, T.W. Williams, E.F. Miller: System testing and reliability techniques for avoiding failure, Computer 29(11), 28–30 (1996)
K.H. Kim, C. Subbaraman: Fault-tolerant real-time objects, Communication ACM 40(1), 75–82 (1997)
D.B. Stewart, P.K. Khosla: Mechanisms for detecting and handling timing errors, Communication ACM 40(1), 87–93 (1997)
G. Weller, F. Groen, L. Hertzberger: A sensor processing model incorporating error detection and recovery. In: Traditional and Non-Traditional Robotic Sensors, ed. by T. Henderson (Springer, Berlin, Heidelberg 1990) pp. 351–363
R.R. Brooks, S. Iyengar: Averaging Algorithm for Multi-Dimensional Redundant Sensor Arrays: Resolving Sensor Inconsistencies, Tech. Rep. (Louisiana State University, Baton Rouge 1993)
T.C. Henderson, M. Dekhil: Visual Target Based Wall Pose Estimation, Tech. Rep. UUCS-97-010 (University of Utah, Salt Lake City 1997)
S. Iyengar, D. Jayasimha, D. Nadig: A versatile architecture for the distributed sensor integration problem, Computer 43, 175–185 (1994)
D. Nadig, S. Iyengar, D. Jayasimha: A new architecture for distributed sensor integration, Proc. IEEE Southeastcon (1993)
L. Prasad, S. Iyengar, R.L. Kashyap, R.N. Madan: Functional characterization of fault tolerant integration in distributed sensor networks, IEEE Trans. Syst. Man Cybern. 25, 1082–1087 (1991)
L. Prasad, S. Iyengar, R. Rao, R. Kashyap: Fault-tolerence sensor integration using multiresolution decomposition, Am. Phys. Soc. 49(4), 3452–3461 (1994)
H.F. Durrant-Whyte: Integration, Coordination, and Control of Multi-Sensor Robot Systems (Kluwer, Boston 1987)
F. Groen, P. Antonissen, G. Weller: Model based robot vision, IEEE Instrum. Meas. Technol. Conf. (1993) pp. 584–588
R. Joshi, A.C. Sanderson: Model-based multisensor data fusion: A minimal representation approach, Proc. IEEE Int. Conf. Robotics Autom. (ICRA) (1994)
A.J. Briggs, B.R. Donald: Automatic sensor configuration for task-directed planning, Proc. IEEE Int. Conf. Robotics Autom. (ICRA) (1994) pp. 1345–1350
G. Hager: Task Directed Sensor Fusion and Planning (Kluwer, Boston 1990)
G. Hager, M. Mintz: Computational methods for task-directed sensor data fusion and sensor planning, Int. J. Robotics Res. 10(4), 285–313 (1991)
B. Donald: On information invariants in robotics, Artif. Intell. 72, 217–304 (1995)
V. Braitenberg: Vehicles: Experiments in Synthetic Psychology (MIT Press, Cambridge 1984)
R.A. Brooks: A robust layered control system for a mobile robot, IEEE Trans. Robotics Autom. 2(1), 14–23 (1986)
K.P. Valavanis, A.L. Nelson, L. Doitsidis, M. Long, R.R. Murphy: Validation of a Distributed Field Robot Architecture Integrated with a Matlab Based Control Theoretic Environment: A Case Study of Fuzzy Logic Based Robot Navigation, CRASAR Tech. Rep. 25 (University of South Florida, Tampa 2004)
R.R. Murphy: Introduction to AI Robotics (MIT Press, Cambridge 2000)
S. Lee: Sensor fusion and planning with perception-action network, Proc. IEEE Conf. Multisens. Fusion Integr. Intell. Syst., Washington (1996)
S. Lee, S. Ro: Uncertainty self-management with perception net based geometric data fusion, Proc. IEEE Conf. Robotics Autom. (ICRA), Albuquerque (1997)
B.A. Draper, A.R. Hanson, S. Buluswar, E.M. Riseman: Information acquisition and fusion in the mobile perception laboratory, Proc. SPIE Sens. Fusion VI (1993)
S.S. Shafer, A. Stentz, C.E. Thorpe: An architecture for sensor fusion in a mobile robot, Proc. IEEE Int. Conf. Robotics Autom. (ICRA) (1986) pp. 2002–2007
S. Nagata, M. Sekiguchi, K. Asakawa: Mobile robot control by a structured hierarchical neural network, IEEE Control Syst. Mag. 10(3), 69–76 (1990)
M. Pachter, P. Chandler: Challenges of autonomous control, IEEE Control Syst. Mag. 18(4), 92–97 (1998)
R. Joshi, A.C. Sanderson: Multisensor Fusion (World Scientific, Singapore 1999)
V. Berge-Cherfaoui, B. Vachon: Dynamic configuration of mobile robot perceptual system, Proc. IEEE Conf. Multisens. Fusion Integr. Intell. Syst., Las Vegas (1994)
V. Clement, G. Giraudon, S. Houzelle, F. Sandakly: Interpretation of Remotely Sensed Images in a Context of Multisensor Fusion Using a Multi-Specialist Architecture, Rapp. Rech.: No. 1768 (INRIA, Sophia-Antipolis 1992)
J. Albus: RCS: A cognitive architecture for intelligent multi-agent systems, Proc. IFAC Symp. Intell. Auton. Veh., Lisbon (2004)
R. Camden, B. Bodt, S. Schipani, J. Bornstein, R. Phelps, T. Runyon, F. French: Autonomous Mobility Technology Assessment, Interim Rep., ARL-MR 565 (Army Research Laboratory, Washington 2003)
T. Queeney, E. Woods: A generic architecture for real-time multisensor fusion tracking algorithm development and evaluation, Proc. SPIE Sens. Fusion VII, Vol. 2355 (1994) pp. 33–42
T. Henderson, E. Shilcrat: Logical sensor systems, J. Robotics Syst. 1(2), 169–193 (1984)
T. Henderson, C. Hansen, B. Bhanu: The specification of distributed sensing and control, J. Robotics Syst. 2(4), 387–396 (1985)
J.D. Elliott: Multisensor Fusion within an Encapsulated Logical Device Architecture, Master's Thesis (University of Waterloo, Waterloo 2001)
M.D. Naish: Elsa: An Intelligent Multisensor Integration Architecture for Industrial Grading Tasks, Master's Thesis (University of Western Ontario, London 1998)
A. Makarenko, A. Brooks, S. Williams, H. Durrant-Whyte, B. Grocholsky: A decentralized architecture for active sensor networks, Proc. IEEE Int. Conf. Robotics Autom. (ICRA), New Orleans (2004) pp. 1097–1102
B. Grocholsky, A. Makarenko, H. Durrant-Whyte: Information-theoretic coordinated control of multiple sensor platforms, Proc. IEEE Int. Conf. Robotics Autom. (ICRA), Taipei (2003) pp. 1521–1527
R. Gregor, M. Lützeler, M. Pellkofer, K.-H. Siedersberger, E. Dickmanns: EMS-Vision: A perceptual system for autonomous vehicles, IEEE Trans. Intell. Transp. Syst. 3(1), 48–59 (2002)
B. Rao, H. Durrant-Whyte, A. Sheen: A fully decentralized multi-sensor system for tracking and surveillance, Int. J. Robotics Res. 12(1), 20–44 (1993)
B. Upcroft: Non-gaussian state estimation in an outdoor decentralised sensor network, Proc. IEEE Conf. Decis. Control (CDC) (2006)
S. Kumar, F. Ramos, B. Upcroft, H. Durrant-Whyte: A statistical framework for natural feature representation, Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), Edmonton (2005) pp. 1–6
S. Gao, Y. Zhong, W. Li: Random weighting method for multisensor data fusion, IEEE Sens. J. 11(9), 1955–1961 (2011)
M. Lhuillier: Incremental fusion of structure-from-motion and GPS using constrained bundle adjustments, IEEE Trans. Pattern Anal. Mach. Intell. 34(12), 2489–2495 (2012)
S. Yu, L. Tranchevent, X. Liu, W. Glanzel, J. Suykens, B. DeMoor, Y. Moreau: Optimized data fusion for kernel k-means clustering, IEEE Trans. Pattern Anal. Mach. Intell. 34(5), 1031–1039 (2012)
K. Kolev: Fast joint estimation of silhouettes and dense 3D geometry from multiple images, IEEE Trans. Pattern Anal. Mach. Intell. 34(3), 493–505 (2012)
C. Loy: Incremental activity modeling in multiple disjoint cameras, IEEE Trans. Pattern Anal. Mach. Intell. 34(9), 1799–1813 (2012)
N. Poh, J. Kittler: A unified framework for biometric expert fusion incorporating quality measures, IEEE Trans. Pattern Anal. Mach. Intell. 34(1), 3–18 (2012)
M.-F. Weng, Y.-Y. Chuang: Cross-domain multicue fusion for concept-based video indexing, IEEE Trans. Pattern Anal. Mach. Intell. 34(10), 1927–1941 (2012)
M. Hwangbo, J.-S. Kim, T. Kanade: Gyro-aided feature tracking for a moving camera: fusion, auto-calibration and GPU implementation, Intl. J. Robotics Res. 30(14), 1755–1774 (2011)
H. Seraji, N. Serrano: A multisensor decision fusion system for terrain safety assessment, IEEE Trans. Robotics 25(1), 99–108 (2009)
H. Himberg, Y. Motai, A. Bradley: Interpolation volume calibration: a multisensor calibration technique for electromagnetic trackers, IEEE Trans. Robotics 28(5), 1120–1130 (2012)
K. Zhou, S.I. Roumeliotis: Optimal motion strategies for range-Only constrained multisensor target tracking, IEEE Trans. Robotics 24(5), 1168–1185 (2008)
N.R. Ahmed, E.M. Sample, M. Campbell: Bayesian multicategorical soft data fusion for human–robot collaboration, IEEE Trans. Robotics PP(99), 1–18 (2012)
H. Frigui, L. Zhang, P.D. Gader: Context-dependent multisensor fusion and its application to land mine detection, IEEE Trans. Geosci. Remote Sens. 48(6), 2528–2543 (2010)
J.G. Garcia, A. Robertson, J.G. Ortega, R. Johansson: Sensor fusion for compliant robot motion control, IEEE Trans. Robotics 24(2), 430–441 (2008)
R. Heliot, B. Espiau: Multisensor input for CPG-based sensory—motor coordination, IEEE Trans. Robotics 24(1), 191–195 (2008)
S. Liu, R.X. Gao, D. John, J.W. Staudenmayer, P.S. Freedson: Multisensor data fusion for physical activity assessment, IEEE Trans. Bio-Med. Eng. 59(3), 687–696 (2012)
A. Martinelli: Vision and IMU data fusion: closed-Form solutions for attitude, speed, absolute scale, and bias determination, IEEE Trans. Robotics 28(1), 44–60 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Video-References
Video-References
- :
-
AnnieWay available from http://handbookofrobotics.org/view-chapter/35/videodetails/132
- :
-
Application of visual odometry for sewer inspection robots available from http://handbookofrobotics.org/view-chapter/35/videodetails/638
- :
-
Multisensor remote surface inspection available from http://handbookofrobotics.org/view-chapter/35/videodetails/639
Rights and permissions
Copyright information
© 2016 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Durrant-Whyte, H., Henderson, T.C. (2016). Multisensor Data Fusion. In: Siciliano, B., Khatib, O. (eds) Springer Handbook of Robotics. Springer Handbooks. Springer, Cham. https://doi.org/10.1007/978-3-319-32552-1_35
Download citation
DOI: https://doi.org/10.1007/978-3-319-32552-1_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-32550-7
Online ISBN: 978-3-319-32552-1
eBook Packages: EngineeringEngineering (R0)