Abstract
RGB-D cameras are novel sensing systems that capture RGB images along with per-pixel depth information. In this paper we investigate how such cameras can be used in the context of robotics, specifically for building dense 3D maps of indoor environments. Such maps have applications in robot navigation, manipulation, semantic mapping, and telepresence. We present RGB-D Mapping, a full 3D mapping system that utilizes a novel joint optimization algorithm combining visual features and shape-based alignment. Visual and depth information are also combined for view-based loop closure detection, followed by pose optimization to achieve globally consistent maps.We evaluate RGB-D Mapping on two large indoor environments, and show that it effectively combines the visual and shape information available from RGB-D cameras.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Akbarzadeh, A., Frahm, J.M., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Merrell, P., Phelps, M., Sinha, S., Talton, B., Wang, L., Yang, Q., Stewenius, H., Yang, R., Welch, G., Towles, H., Nistér, D., Pollefeys, M.: Towards urban 3D reconstruction from video. In: Proc. of the Third International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT) (2006)
Besl, P.J., McKay, N.D.: A method for registration of 3-d shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 14(2) (1992)
Canesta, http://www.canesta.com/
Chen, Y., Medioni, G.: Object modeling by registration of multiple range images. Image Vision Comput. 10(3), 145–155 (1992)
Clemente, L., Davison, A., Reid, I., Neira, J., Tardós, J.: Mapping large loops with a single hand-held camera. In: Proc. of Robotics: Science and Systems, RSS (2007)
Debevec, P., Taylor, C.J., Malik, J.: Modeling and rendering architecture from photographs: A hybrid geometryand image-based approach. In: SIGGRAPH (1996)
Furukawa, Y., Curless, B., Seitz, S., Szeliski, R.: Reconstructing building interiors from images. In: Proc. of the International Conference on Computer Vision, ICCV (2009)
Furukawa, Y., Ponce, J.: Patch-based multi-view stereo software (PMVS), http://grail.cs.washington.edu/software/pmvs/
Grisetti, G., Grzonka, S., Stachniss, C., Pfaff, P., Burgard, W.: Estimation of accurate maximum likelihood maps in 3D. In: Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS (2007)
Grisetti, G., Stachniss, C., Grzonka, S., Burgard, W.: A tree parameterization for efficiently computing maximum likelihood maps using gradient descent. In: Proc. of Robotics: Science and Systems, RSS (2007)
Horn, B.K.P.: Closed-form solution of absolute orientation using unit quaternions. J. Opt. Soc. Am. A 4(4), 629–642 (1987)
Mesa Imaging, http://www.mesa-imaging.ch/
Johnson, A., Kang, S.B.: Registration and integration of textured 3-d data. In: International Conference on Recent Advances in 3-D Digital Imaging and Modeling (3DIM 1997), pp. 234–241 (May 1997)
Kim, Y.M., Theobalt, C., Diebel, J., Kosecka, J., Micusik, B., Thrun, S.: Multi-view image and ToF sensor fusion for dense 3D reconstruction. In: Workshop on 3-D Digital Imaging and Modeling (3DIM) (2009)
Konolige, K.: Projected texture stereo. In: Proc. of the IEEE International Conference on Robotics & Automation, ICRA (2010)
Konolige, K., Agrawal, M.: FrameSLAM: From bundle adjustment to real-time visual mapping. IEEE Transactions on Robotics 25(5) (2008)
Krainin, M., Henry, P., Ren, X., Fox, D.: Manipulator and object tracking for in hand 3D object modeling. Technical Report UW-CSE-10-09-01, University of Washington (2010), http://www.cs.washington.edu/ai/Mobile_Robotics/projects/hand_tracking/
Lowe, D.: Discriminative image features from scale-invariant keypoints. International Journal of Computer Vision 60(2) (2004)
May, S., Dröschel, D., Holz, D., Fuchs, E., Malis, S., Nüchter, A., Hertzberg, J.: Three-dimensional mapping with time-of-flight cameras. Journal of Field Robotics (JFR) 26(11-12) (2009)
Microsoft (2010), http://www.xbox.com/en-US/kinect
Newman, P., Sibley, G., Smith, M., Cummins, M., Harrison, A., Mei, C., Posner, I., Shade, R., Schröter, D., Murphy, L., Churchill, W., Cole, D., Reid, I.: Navigating, recognising and describing urban spaces with vision and laser. International Journal of Robotics Research (IJRR) 28(11-12) (2009)
Nister, D.: An efficient solution to the five-point relative pose problem. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 26(6), 756–777 (2004)
Pfister, H., Zwicker, M., van Baar, J., Gross, M.: Surfels: Surface elements as rendering primitives. In: ACM Transactions on Graphics (Proc. of SIGGRAPH) (2000)
Pollefeys, M., Nister, D., Frahm, J.-M., Akbarzadeh, A., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Kim, S.-J., Merrell, P., Salmi, C., Sinha, S., Talton, B., Wang, L., Yang, Q., Stewenius, H., Yang, R., Welch, G., Towles, H.: Detailed real-time urban 3D reconstruction from video. International Journal of Computer Vision 72(2), 143–167 (2008)
PrimeSense, http://www.primesense.com/
Ramos, F., Fox, D., Durrant-Whyte, H.: CRF-matching: Conditional random fields for feature-based scan matching. In: Proc. of Robotics: Science and Systems, RSS (2007)
Rusinkiewicz, S., Levoy, M.: Efficient variants of the ICP algorithm. In: Third International Conference on 3D Digital Imaging and Modeling (2001)
Segal, A., Haehnel, D., Thrun, S.: Generalized-ICP. In: Proc. of Robotics: Science and Systems, RSS (2009)
Sharp, G.C., Lee, S.W., Wehe, D.K.: ICP registration using invariant features. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 24(1), 90–102 (2002)
Snavely, N., Seitz, S., Szeliski, R.: Photo tourism: Exploring photo collections in 3D. In: ACM Transactions on Graphics (Proc. of SIGGRAPH) (2006)
Thrun, S., Burgard, W., Fox, D.: A real-time algorithm for mobile robot mapping with applications to multi-robot and 3D mapping. In: Proc. of the IEEE International Conference on Robotics & Automation, ICRA (2000)
Triebel, R., Burgard, W.: Improving simultaneous mapping and localization in 3D using global constraints. In: Proc. of the National Conference on Artificial Intelligence, AAAI (2005)
Wu, C.: SiftGPU: A GPU implementation of scale invariant feature transform, SIFT (2007), http://cs.unc.edu/~ccwu/siftgpu
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag GmbH Berlin Heidelberg
About this chapter
Cite this chapter
Henry, P., Krainin, M., Herbst, E., Ren, X., Fox, D. (2014). RGB-D Mapping: Using Depth Cameras for Dense 3D Modeling of Indoor Environments. In: Khatib, O., Kumar, V., Sukhatme, G. (eds) Experimental Robotics. Springer Tracts in Advanced Robotics, vol 79. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28572-1_33
Download citation
DOI: https://doi.org/10.1007/978-3-642-28572-1_33
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-28571-4
Online ISBN: 978-3-642-28572-1
eBook Packages: EngineeringEngineering (R0)