Skip to main content
SpringerLink
Log in

Line-based initialization method for mobile augmented reality in aircraft assembly

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

A fast line-based initialization method for mobile augmented reality in aircraft assembly is proposed in this paper. The challenge of recognition and pose estimation of aircraft structural parts in monocular images on mobile devices is mainly addressed. Rather than local feature points, straight lines are extracted and utilized here since they are more suitable for aircraft structural parts, which usually have a solid color and lack textures to generate invariant descriptors. The geometric relationship between 3D world lines and their projections on the camera image are built to estimate the relative 6-DOF pose including rotation and translation, respectively. The inertial sensors of mobile devices are used to provide partial rotation information directly, and the rest is obtained through a voting process. To shrink the search space and, therefore, improve the speed and robustness of initialization, a color-based ROI mask and a vertical lines extraction method are proposed. To determine the translation, the color mask is first used to roughly approximate the real pose and then a possible-correspondence map of lines is built by parallelism alignment combined with the rotation already obtained. This paper describes the details of our method and also shows its effectiveness with experiments. Moreover, the proposed method is especially suitable for but not limited to the aircraft assembly field.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Álvarez, H., Borro, D.: Junction assisted 3d pose retrieval of untextured 3d models in monocular images. Computer Vision Image Underst. 117(10), 1204–1214 (2013)

    Article  Google Scholar 

  2. Bay, H., Ess, A., Tuytelaars, T.: Speeded-up robust features (surf). Computer Vision Image Underst. 110(3), 34659 (2008)

    Article  Google Scholar 

  3. Bleser, G., Wuest, H., Stricker, D.: Online camera pose estimation in partially known and dynamic scenes. In: Proceedings of ISMAR06, pp. 56–65 (2006)

  4. Carmichael, O., Hebert, M.: Shape-based recognition of wiry objects. IEEE Trans. Pattern Anal. Mach. Intell. 26(12), 15371552 (2004)

    Article  Google Scholar 

  5. David, P., DeMenthon, D.: Object recognition in high clutter images using line features. In: Tenth IEEE International Conference on Computer Vision, pp. 1581–1588 (2005)

  6. Gioi, R.V., Jakubowicz, J., Morel, J., Randall, G.: Lsd: a fast line segment detector with a false detection control. IEEE Trans. Pattern Anal. Mach. Intell. 4, 722–732 (2008)

    Google Scholar 

  7. Han, P., Zhao, G.: Cad-based 3d objects recognition in monocular images for mobile augmented reality. Computers Graph. 50, 36–46 (2015)

    Article  Google Scholar 

  8. Harris, C., Stennett, C.: Rapid a video-rate object tracker. University of Oxford, Oxford (1990)

    Book  Google Scholar 

  9. Kim, G., Hebert, M., Park, S.: Preliminary development of a line feature-based object recognition system for textureless indoor objects. Recent Progr. Robot. Viable Robot. Serv. Hum. 370, 255–268 (2008)

    Article  Google Scholar 

  10. Kim, K., Park, N., Woo, W.: Vision-based all-in-one solution for augmented reality and its storytelling applications. Visual Computer 30(4), 417–429 (2014)

    Article  Google Scholar 

  11. Kotake, D., Satoh, K., Uchiyama, S.: A fast initialization method for edge-based registration using an inclination constraint. In: IEEE International Symposium on Mixed and Augmented Reality, pp. 1–10(2007)

  12. Kurz, D., Benhimane, S.: Handheld augmented reality involving gravity measurements. Computers Graph. 36(7), 866–883 (2012)

    Article  Google Scholar 

  13. Liarokapis, F., Macan, L., Malone, G., Rebolledo-Mendez, G., Freitas, S.D.: Multimodal augmented reality tangible gaming. Visual Computer 25(12), 1109–1120 (2009)

    Article  Google Scholar 

  14. Lowe, D.: Distinctive image feature from scale invariant keypoints. Int. J. Computer Vision 60(2), 91110 (2004)

    Article  Google Scholar 

  15. Lu, Z., Li, Y., Wang, J., Chen, P., Chen, F.: 3d object recognition using line structure. In: 3rd International Conference on Digital Image Processing, pp. 800–916. International Society for Optics and Photonics (2011)

  16. Metaio: Edge-based initialization (2015). http://www.metaio.com/

  17. Mikolajczyk, K., Zisserman, A., Schmid, C.: Shape recognition with edge-based features. In: Proceedings of British Machine Vision Conference (2003)

  18. Mirzaei, M., Ghorshi, S., Mortazavi, M.: Audio-visual speech recognition techniques in augmented reality environments. Visual Computer 30(3), 245–257 (2014)

    Article  Google Scholar 

  19. Petr, H.: Augmented reality system for virtual training of parts assembly. Procedia Eng. 100, 699–706 (2015)

    Article  Google Scholar 

  20. Pished, B., Dima, D., Andrew, C., Walterio, M.: Integrating 3d object detection, modelling and tracking on a mobile phone. In: 2012 IEEE International Symposium on Mixed and Augmented Reality, pp. 273–274. IEEE (2012)

  21. Qiu, Z., Wei, H.: Line segment based man-made object recognition using invariance. In: International Joint Conference on Artificial Intelligence, pp. 460–464 (2009)

  22. Rafael, R., Jordan, H., James, O.: Augmented reality-based manual assembly support with visual features for different degrees of difficulty. Int. J. Human-Computer Interact. 31(5), 337–349 (2015)

    Article  Google Scholar 

  23. Ronald, A., Billinghurst, M., Klinker, G.: Special section on mobile augmented reality. Computers Graph. 35(4), 7–8 (2011)

    Google Scholar 

  24. Shahrokni, A., Vacchetti, L., Lepetit, V., Fua, P.: Polyhedral object detection and pose estimation for augmented reality applications. In: Proceedings of Computer Animation, p. 6569 (2002)

  25. Smith, A.: Color gamut transform pairs. Acm Siggraph Computer Graphics 12(3), 12–19 (1978)

    Article  Google Scholar 

  26. Truong, H., Sukhan, L., Jang, S.: Model-based recognition of 3d objects using intersecting lines. In: IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, 2008, pp. 656–660 (2008)

  27. Vuforia: Smart terrain (2014). https://www.qualcomm.com/products/vuforia

  28. Zhang, L., Xu, C., Lee, K., Koch, R.: Robust and efficient pose estimation from line correspondences. In: Computer Vision–ACCV, pp. 217–230. Springer, New York (2013)

Download references

Acknowledgments

This research has been partly funded by the National Natural Science Foundation of China (2015AA7045038).

Author information

Authors and Affiliations

  1. Beijing Key Laboratory of Aircraft Assembly and Robotics Equipment Technology, Beihang University, Beijing, China

    Pengfei Han

  2. State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China

    Gang Zhao

Authors
  1. Pengfei Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gang Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gang Zhao.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Han, P., Zhao, G. Line-based initialization method for mobile augmented reality in aircraft assembly. Vis Comput 33, 1185–1196 (2017). https://doi.org/10.1007/s00371-016-1281-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-016-1281-5

Keywords

Search

Navigation