Skip to main content
SpringerLink
Log in

Interactive building and augmentation of piecewise planar environments using the intersection lines

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

This paper describes a method for online interactive building of piecewise planar environments for immediate use in augmented reality. This system combines user interaction from a camera–mouse and automated tracking/reconstruction methods to recover planar structures of the scene that are relevant for the augmentation task. An important contribution of our algorithm is that the process of tracking and reconstructing planar structures is decomposed into three steps—tracking, computation of the intersection lines of the planes, reconstruction—that can each be visually assessed by the user, making the interactive modeling procedure really robust and accurate with intuitive interaction. Videos illustrating our system both on synthetic and long real-size experiments are available at http://www.loria.fr/~gsimon/vc.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Arulampalam, S., Maskell, S., Gordon, N., Clapp, T.: A tutorial on particle filters for on-line non-linear/non-Gaussian Bayesian tracking. IEEE Trans. Signal Process. 50(2), 174–188 (2002)

    Article  Google Scholar 

  2. Bunnun, P., Mayol-Cuevas, W.W.: OutlinAR: An assisted interactive model building system with reduced computational effort. In: IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 61–64. IEEE Computer Society, Los Alamitos (2008)

    Chapter  Google Scholar 

  3. Chekhlov, D., Gee, A.P., Calway, A., Mayol-Cuevas, W.: Ninja on a plane: Automatic discovery of physical planes for augmented reality using visual slam. In: Proceedings of the 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, ISMAR ’07, Washington, DC, USA, pp. 1–4. IEEE Computer Society, Los Alamitos (2007)

    Chapter  Google Scholar 

  4. Chekhlov, D., Pupilli, M., Mayol-Cuevas, W., Calway, A.: Real-time and robust monocular slam using predictive multi-resolution descriptors. In: 2nd International Symposium on Visual Computing, November 2006 (2006)

    Google Scholar 

  5. Davison, A.J., Reid, I.D., Molton, N.D., Stasse, O.: MonoSLAM: Real-time single camera SLAM. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 1052–1067 (2007)

    Article  Google Scholar 

  6. Faugeras, O., Lustman, F.: Motion and structure from motion in a piecewise planar environment. Rapport de recherche 856. INRIA (1988)

  7. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Technical Note 213, Artificial Intelligence Center, SRI International, Menlo Park, CA, March 1980

  8. Freeman, R., Steed, A.: Interactive modelling and tracking for mixed and augmented reality. In: Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST ’06, New York, NY, USA, pp. 61–64. ACM, New York (2006)

    Chapter  Google Scholar 

  9. Gee, A.P., Chekhlov, D., Calway, A., Mayol-Cuevas, W.W.: Discovering higher level structure in visual slam. IEEE Trans. Robot. 24(5), 980–990 (2008)

    Article  Google Scholar 

  10. Harris, C., Stephens, M.: A Combined Corner and Edge Detector. In: Proceedings of 4th Alvey Conference, Cambridge, August 1988 (1988)

    Google Scholar 

  11. Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000). ISBN: 0521623049

    MATH  Google Scholar 

  12. Huynh, D.N.T., Raveendran, K., Xu, Y., Spreen, K., MacIntyre, B.: Art of defense: A collaborative hand-held augmented reality board game. In: Sandbox ’09: Proceedings of the 2009 ACM SIGGRAPH Symposium on Video Games, New York, NY, USA, pp. 135–142. ACM, New York (2009)

    Chapter  Google Scholar 

  13. Jin, H., Favaro, P., Soatto, S.: A semi-direct approach to structure from motion. Vis. Comput. 19(6), 377–394 (2003)

    Article  Google Scholar 

  14. Johansson, B.: View synthesis and 3d reconstruction of piecewise planar scenes using intersection lines between the planes. In: ICCV, pp. 54–59 (1999)

    Google Scholar 

  15. Julier, S., Baillot, Y., Lanzagorta, M., Brown, D., Rosenblum, L.: Bars: Battlefield augmented reality system. In: NATO Symposium on Information Processing Techniques for Military Systems, Istanbul, Turkey, October 2000, pp. 9–11 (2000)

    Google Scholar 

  16. Kato, H., Billinghurst, M.: Marker tracking and HMD calibration for a video-based augmented reality conferencing system. In: Proceedings of the 2nd International Workshop on Augmented Reality, San Francisco (1999)

    Google Scholar 

  17. Klein, G., Murray, D.: Parallel tracking and mapping for small AR workspaces. In: Proc. Sixth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR’07), Nara, Japan, November 2007 (2007)

    Google Scholar 

  18. Klein, G., Murray, D.: Improving the agility of keyframe-based SLAM. In: Proc. Eur. Conf. Computer Vision (ECCV’08) (2008)

    Google Scholar 

  19. Klein, G., Murray, D.: Parallel Tracking and Mapping on a Camera Phone. In: Proc. Eighth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR’09), Orlando, USA, October 2009 (2009)

    Google Scholar 

  20. Lepetit, V., Vachetti, L., Thalmann, D., Fua, P.: Fully automated and stable registration for augmented reality applications. In: Proceedings of International Symposium on Mixed and Augmented Reality, Tokyo, June 2003, pp. 93–101 (2003)

    Chapter  Google Scholar 

  21. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  22. Noury, N., Sur, F., Berger, M.-O.: Determining point correspondences between two views under geometric constraint and photometric consistency. Research Report RR-7246, INRIA, 04 (2010)

  23. Rabin, J., Delon, J., Gousseau, Y., Moisan, L.: MAC-RANSAC: A robust algorithm for the recognition of multiple objects. In: Fifth International Symposium on 3D Data Processing, Visualization and Transmission (2010)

    Google Scholar 

  24. Schmid, C., Zisserman, A.: Automatic line matching across views. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 666–671 (1997)

    Google Scholar 

  25. Simon, G., Berger, M.-O.: Detection of the intersection lines in multiplanar environments: application to real-time estimation of the camera-scene geometry. In: 19th International Conference on Pattern Recognition—ICPR, pp. 1–4, Tampa United States, IEEE (2008)

  26. Sinha, S., Steedly, D., Szeliski, R.: Piecewise planar stereo for image-based rendering. In: Proc. Int. Conf. Computer Vision (ICCV’09), pp. 1881–1888 (2009)

    Google Scholar 

  27. Thormählen, T.: Zuverlässige Schätzung der Kamerabewegung aus einer Bildfolge. Ph.D. Thesis, University of Hannover. Related software ‘Voodoo Camera Tracker’ can be downloaded from http://www.digilab.uni-hannover.de (2006)

  28. Toldo, R., Fusiello, A.: Robust multiple structures estimation with j-linkage. In: Proceedings of the 10th European Conference on Computer Vision: Part I, ECCV ’08, pp. 537–547. Springer, Berlin (2008)

    Google Scholar 

  29. Tomasi, C., Kanade, T.: Detection and tracking of point features. Technical Report CMU-CS-91-132, Carnegie Mellon University, April 1991

  30. van den Hengel, A., Hill, R., Ward, B., Dick, A.: In situ image-based modeling. In: IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 107–110 (2009)

    Chapter  Google Scholar 

  31. Vigueras, F., Berger, M.-O., Simon, G.: Iterative multi-planar camera calibration: Improving stability using model selection. In: Vision, Video and Graphics (VVG’03), Bath, UK, July 2003 (2003)

    Google Scholar 

  32. Vincent, E., Laganière, R.: Detecting planar homographies in an image pair. In: 2nd International Symposium on Image and Signal Processing and Analysis, Pula, Croatia, June 2001 (2001)

    Google Scholar 

  33. Xu, G., Terai, J., Shum, H.: A linear algorithm for camera self-calibration, motion and structure recovery for multi-planar scenes from two perspective images. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Hilton Head Island, South Carolina (USA) (2000)

    Google Scholar 

  34. Xu, L., Oja, E., Kultanen, P.: A new curve detection method: Randomized Hough Transform (RHT). Pattern Recognit. Lett. 11, 331–338 (1990)

    Article  MATH  Google Scholar 

  35. Zuliani, M., Kenney, C.S., Manjunath, B.S.: The multiransac algorithm and its application to detect planar homographies. In: IEEE International Conference on Image Processing (2005)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. LORIA, Nancy University, Campus Scientifique, BP 239, 54506, Vandœuvre-les-Nancy, France

    Gilles Simon

  2. LORIA, INRIA Nancy-Grand Est, 615 rue du Jardin Botanique, 54600, Villers-lès-Nancy, France

    Marie-Odile Berger

Authors
  1. Gilles Simon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marie-Odile Berger
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gilles Simon.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Simon, G., Berger, MO. Interactive building and augmentation of piecewise planar environments using the intersection lines. Vis Comput 27, 827–841 (2011). https://doi.org/10.1007/s00371-011-0556-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-011-0556-0

Keywords

Search

Navigation