Skip to main content
SpringerLink
Log in

High-precision calibration of wide-angle fisheye lens with radial distortion projection ellipse constraint (RDPEC)

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

This paper presents a novel technique for wide-angle fisheye lens calibration which requires neither metric information nor particular reference pattern. First, the fisheye imaging model with the interior Orientation parameters (IOPs)—principal point (u0,v0), focal length f, aspect ratio λ and radial distortion coefficients (k1, k2), is established. Then, upon the fisheye imaging model and the parameter dependency between f and (k1, k2), the radial distortion projection ellipse constraint (RDPEC) for space lines in fisheye image is mathematically formulated to build a non-linear calibration model for high-precision estimation of the IOPs. In this step, parameter initialization based on the geometry of fisheye image outline ellipse (FIOE) is discussed as well. Finally, initial IOPs are further optimized though least square technique by taking the projection ellipse arcs of space lines in fisheye image as observation. The proposed calibration technique was tested on two kinds of fisheye images: (a) simulated image with a set of ground-truth IOPs, (b) internet images with unknown IOPs. Experimental results show that the calibration parameters in this paper are in the best agreement with the fisheye imaging model, compared with the ground-truth parameters and the parameters estimated by two state-of-the-art literature. Compared to that by a state-of-the-art CNN and the well-known software DXO, the proposed technique can enable a high-quality correction of fisheye images in different regions. This makes it very useful in application scenarios containing space lines, such as urban panorama surveillance, auto-parking and, robot navigation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Gurtner, A., Greer, D.G., Glassock, R., et al.: Investigation of fish-eye lenses for small-UAV aerial photography. IEEE Trans. Geosci. Remote Sens. 47(3), 709–721 (2009)

    Article  Google Scholar 

  2. Li, S., Hai, Y.: Easy calibration of a blind-spot-free fisheye camera system using a scene of a parking space. IEEE Trans. Intell. Transp. Syst. 12(1), 232–242 (2011)

    Article  Google Scholar 

  3. Naruse T., Kaneko T., Yamashita A., et al.: 3-D measurement of objects in water using fish-eye stereo camera. In: Proceedings of the 2012 19th IEEE international conference on image processing. Orlando, FL, USA, 30 Sept.–3 Oct, pp. 2773–2776 (2013).

  4. Delibasis, K.K., Plagianakos, V.P., Maglogiannis, I.: Estimation of robot position and orientation using a stationary fisheye camera. Int. J. Artif. Intell. Tools. 24(06), 1560004 (2015)

    Article  Google Scholar 

  5. Cumhur, S.: Comparison and calibration of mobile phone fisheye lens and regular fisheye lens via equidistant model. J. Sens. 2016(1), 1–11 (2016)

    Google Scholar 

  6. Arfaoui, A., Thibault, S.: Fisheye lens calibration using virtual grid. Appl. Opt. 52(12), 2577–2583 (2013)

    Article  Google Scholar 

  7. Aghayari S., Saadatseresht M., Omidalizarandi M., et al.: Geometric calibration of full spherical panoramic RICOH-theta camera. ISPRS Ann. Photogram. Remote Sens. Spat. Inform. Sci. IV-1/W1, 237–245 (2017).

  8. Huang, M.Y., Wu, J.: Fish-eye lens calibration by ideal projection ellipse constraints. J. Image Graph. 24(11), 1972–1984 (2019)

    Google Scholar 

  9. Kannala, J., Brandt, S.S.: A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses. IEEE Trans. Pattern Anal. Mach. Intell. 28(8), 1335–1340 (2006)

    Article  Google Scholar 

  10. Kanatani, K.: Calibration of ultrawide fisheye lens cameras by eigenvalue minimization. IEEE Trans. Pattern Anal. Mach. Intell. 35(4), 813–822 (2013)

    Article  Google Scholar 

  11. Ying X., Hu Z., Zha H.: Fisheye lenses calibration using straight-line spherical perspective projection constraint. In: Computer vision: ACCV 2006, 7th Asian conference on computer vision, Hyderabad, India, January 13–16, Proceedings, Part II. Springer, New York. pp. 61–70 (2006).

  12. Huang, F., Wang, Y., Shen, X., et al.: Method for calibrating the fisheye distortion center. Appl. Opt. 51(34), 8169–8176 (2012)

    Article  Google Scholar 

  13. Melo R., Antunes M., et al.: Unsupervised intrinsic calibration from a single frame using a plumb-line approach. In: IEEE international conference on computer vision. Sydney, Australia, Dec.3–Dec.6, pp. 537–544 (2013).

  14. Zhang, M., Yao, J., Xia, M., et al.: Line-based multi-label energy optimization for fisheye image rectification and calibration. Comput. Vis. Pattern Recogn. 7, 4137–4145 (2015)

    Google Scholar 

  15. Chan S., Zhou X., Huang C., et al.: An improved method for fisheye camera calibration and distortion correction. In: International conference on advanced robotics and mechatronics. Macau, China, Aug 18–20, pp. 579–584 (2016).

  16. Ramalingam, S., Sturm, P.: A unifying model for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 39(7), 1309–1319 (2017)

    Article  Google Scholar 

  17. LeCun, Y., Bengio, Y., Hinton, G.E.: Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  18. Rong J., Huang S., Shang Z., et al.: Radial lens distortion correction using convolutional neural networks trained with synthesized images. In: Asian conference on computer vision, pp. 35–49 (2016)

  19. Yang, S., Lin, C., Liao, K., et al.: Unsupervised fisheye image correction through bidirectional loss with geometric prior: ScienceDirect. J. Vis. Commun. Image Represen. 66, 102692 (2019)

    Article  Google Scholar 

  20. Yin X., Wang X., Yu J., et al.: FishEye RecNet: a multi-context collaborative deep network for fisheye image rectification. In: European conference on computer vision. Springer, Cham, pp. 475–490 (2018)

  21. Pan Z., Władysław S.: Deep learning for rectification of radial distortion image. In: Photonics applications in astronomy, communications, industry, and high-energy physics experiments, Wilga, Poland, pp. 111760H (2019).

  22. Yang S., Lin C., Liao K., et al.: Progressively complementary network for fisheye image rectification using appearance flow. In: IEEE/CVF conference on computer vision and pattern recognition (2021)

  23. Deng, J., Dong, W., Socher R, et al.: Imagenet: a large-scale hierarchical image database. In: Proceedings of the IEEE computer vision and pattern recognition, pp. 248–255 (2009)

  24. Zhou, B., Zhao, H., Puig, X., et al.: Semantic understanding of scenes through the ADE20K dataset. Int. J. Comput. Vis. 127(3), 302–321 (2016)

    Article  Google Scholar 

  25. Zhou, B., Lapedriza, A., Khosla, A., et al.: Places: a 10 million image database for scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40, 1452–1464 (2018)

    Article  Google Scholar 

  26. Ray, A., Srivastava, D.C.: Non-linear least squares ellipse fitting using the genetic algorithm with applications to strain analysis. J. Struct. Geol. 30(12), 1593–1602 (2008)

    Article  Google Scholar 

  27. Danijela P., Miomir S.: The q-Levenberg-Marquardt method for unconstrained nonlinear optimization. In: Center for applied mathematics and electronic, Belgrade, pp. 1–5 (2021).

  28. Lu, W., Tan, J.: Detection of incomplete ellipse in images with strong noise by iterative randomized Hough transform (IRHT). Pattern Recogn. 41(4), 1268–1279 (2008)

    Article  Google Scholar 

  29. Courbon, J., Mezouar, Y., Eck, L., et al.: A generic fisheye camera model for robotic applications. In: IEEE/RSJ international conference on intelligent robots and systems. IEEE (2007)

  30. Bouguet, J. Y.: Camera calibration toolbox for Matlab. http://www.vision.caltech.edu/bouguetj/calib_doc/. Accessed on 14th Oct 2015.

  31. SOFTONIC INTERNATIONAL S.A. Copyright SOFTONIC INTERNATIONAL S.A. © 1997–2022: All rights reserved. https://dxo-optics-pro.en.softonic.com/

Download references

Acknowledgements

This work was partly supported by National Natural Science Foundation of China (41761087), Ningbo Science and Technology Innovation Project (2020Z019) and Innovation Project of Guangxi Graduate Education (2018YJCXB62).

Author information

Authors and Affiliations

  1. School of Electronic Engineering and Automation, Guilin University of Electronic Technology, Guilin, 541004, China

    Mingyi Huang, Jun Wu, Peng Zhiyong & Xuemei Zhao

  2. Guangxi Key Laboratory for Optoelectronics Information Processing, Guilin, 541004, China

    Peng Zhiyong

Authors
  1. Mingyi Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jun Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peng Zhiyong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xuemei Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jun Wu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, M., Wu, J., Zhiyong, P. et al. High-precision calibration of wide-angle fisheye lens with radial distortion projection ellipse constraint (RDPEC). Machine Vision and Applications 33, 44 (2022). https://doi.org/10.1007/s00138-022-01296-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-022-01296-9

Keywords

Search

Navigation