Skip to main content
SpringerLink
Log in

3D matching by combining CAD model and computer vision for autonomous bin picking

  • Original Paper
  • Published:
International Journal on Interactive Design and Manufacturing (IJIDeM) Aims and scope Submit manuscript

Abstract

Since today, most of the manufacturing companies have the operated CAD model, so CAD-based object autonomous bin picking which using 6DOF Manipulator may be a good option that can save time and increases productivity for an assembly line. This research aims to present an effectively autonomous method that can increases productivity as well as respond quickly of changing items based on customer demand for an assembly line which using 6DOF Manipulator by combining CAD data and computer vision system. Firstly, The 3D CAD model of grasped object is projected onto six different 2D planes, then combining six views to form the final pointcloud. Secondly, a voting scheme is used to estimated the 3D pose of object which is obtained by a 3D camera. For tuning a precision of an estimation such as surface normal, angle and location of an object, Iterative closest point (ICP) algorithm is applied. Before doing experiments, the recognition algorithm is verified through the simulation program. Through implement experiments, the system proved that it is stable, good precision and applicable in production line where mass product is produced. Moreover, the developed system allows non-expert users with basic knowledge about CAD drawing and image processing can generate a pose of an object from CAD model and transmit data to a manipulator for the bin picking task.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Hardin, W.: Vision enables freestyle bin-picking, Vision System Design (2007).

  2. Golnabia, H., Asadpour, A.: Design and application of industrial machine vision systems. Robot. Comput. Integr. Manuf. 23(6), 630–637 (2007)

    Article  Google Scholar 

  3. Ulrich, M., Wiedemann, C., Steger, C.: CAD-based recognition of 3D objects in monocular images. In: Proc. IEEE International Conference on Robotics and Automation, pp. 1191–1198 (2009)

  4. Casado, F., Lapido, Y.L., Losada, D. P., Alonso, A.S.: Pose estimation and object tracking using 2D image. In: 27th International Conference on Flexible Automation and Intelligent Manufacturing, 27–30 June, Modena, Italia, pp. 63–71 (2017)

  5. Collet, A., Berenson, D., Srinivasa, S.S., Ferguson, D.: Object recognition and full pose registration from a single image for robotic manipulation. In: IEEE International Conference on Robotics and Automation, pp. 48–55 (2009)

  6. Ban, K., Warashina, F., Kanno, I., Kumiya, H.: Industrial intelligent robot. FANUC Tech. Rev. 16(2), 29–34 (2003)

    Google Scholar 

  7. Xu, J.: Rapid 3D surface profile measurement of industrial parts using two-level structured light pat-terns. Opt. Lasers. Eng 49, 907–914 (2011)

    Article  Google Scholar 

  8. Bousmalis, K., Irpan, A., Wohlhart Y. P., Bai, M., Kelcey, M., Kalakrishnan, M., Downs, L., Ibarz, J., Pastor, P., Konolige, K.S., Levine, V. Vanhoucke, Using simulation and domain adaptation to improve efficiency of deep robotic grasping. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), Australia, pp. 4243–4250 (2018)

  9. Mahler, J., Liang, J., Niyaz, S., Laskey, M., Doan, R., Liu, X., Ojea, J.A., Goldberg, K. Dex-net 2.0: deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics. In: Robotics: Science and Systems (2017)

  10. Tuan-Tang, L., Chyi-Yeu, L.: Bin-picking for planar objects based on a deep learning network: a case study of USB packs. Sens. Actuators 9, 3602–3618 (2019)

    Google Scholar 

  11. Iglesias, A.T., Pastor-López, I., Urquijo, B.S., García-Bringas, P.: Effective bin picking approach by combining deep learning and point cloud processing techniques. In: de la Cal E.A., Villar Flecha J.R., Quintián H., Corchado E. (eds) Hybrid Artificial Intelligent Systems. HAIS 2020. Lecture Notes in Computer Science, vol 12344. Springer, Cham

  12. Benjamin, J., Tevon, W., Remi, G., Konrad, A.: Pose estimation and bin picking for deformable products. IFAC-PapersOnLine 52(30), 361–366 (2019)

    Article  Google Scholar 

  13. Chun-Tse, L., Cheng-Han, T., Jen-Yuan, C.: A CAD-free random bin picking system for fast changeover on multiple objects. In: ASME 2020 29th Conference on Information Storage and Processing Systems (2020)

  14. Chen, J., Fujinami, T., Li, E.: Deep bin picking with reinforcement learning. In: Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden, PMLR 80, 2018, pp. 1–8

  15. Wu, G., Bin, L., Qibing, Z., Huang, M., Guo, Y.: Using color and 3D geometry features to segment fruit point cloud and improve fruit recognition accuracy. Comput. Electron. Agric. 174, 1–8 (2020)

    Google Scholar 

  16. Aldoma, A. et al. CAD-model recognition and 6DOF pose estimation using 3D cues. In: Proc. IEEE Int. Conf. Comput. Vis. pp. 585–592 (2011)

  17. Song, K.T., Wu, C.H., Jiang, S.Y.: CAD-based pose estimation design for random bin picking using a RGB-D camera. J. Intell. Robot. Syst. 87, 455–470 (2017)

    Article  Google Scholar 

  18. Li, M., Hashimoto, K.: Fast and robust pose estimation algorithm for bin picking using point pair feature. In: 24th International Conference on Pattern Recognition (ICPR), pp. 1604–1609 (2018)

  19. Hanh, L.D., Tu, H.B.: Computer vision for industrial robot in planar bin picking application. Adv. Sci. Technol. Eng. Syst. J. 5(6), 1244–1249 (2020)

    Article  Google Scholar 

  20. Robotic Bin Picking Made Possible with the New FANUC iRVision 3D Area Sensor, (2015), http://robot.fanucamerica.com/robot-applications/Bin-Picking-Robot-with-3D-Area-Sensor-Vision-System.aspx

  21. Pick-itTM Robotic Picking, (2015), http://www.intermodalics.eu/pick-it-3d

  22. Szilvśi-Nagy, M., Mátyási, G.: Analysis of STL files. Math. Comput. Model. 38(7–9), 945–960 (2003)

    Article  MathSciNet  Google Scholar 

  23. Hallad, S.A., Banapurmath, N.R., Patil, A.Y., Hunashyal, A.M., Shettar, A.S.: Studies on the effect of multi-walled carbon nanotube–reinforced polymer-based nano-composites using finite element analysis software tool. Proc. Inst. Mech. Eng. Part N J. Nanomater. Nanoeng. Nanosyst. 230(4), 200–212 (2016)

    Google Scholar 

  24. Vishalagoud, S.P., Farheen, B., Kurahatti, R.V., Arun, Y.P., Raju, G.U., Afzal, A., Manzoore, E., Soudagar, M., Ravinder Kumar, C., Ahamed, S.: A study of sound pressure level (SPL) inside the truck cabin for new acoustic materials: an experimental and FEA approach. Alex Eng J 60(6), 5949–5976 (2021)

    Article  Google Scholar 

  25. Dhaduti, S.C., Sarganachari, S.G., Patil, A.Y., Yunus, K.T.M.: Prediction of injection molding parameters for symmetric spur gear. J. Mol. Model. 26(11), 302 (2020). https://doi.org/10.1007/s00894-020-04560-9

    Article  Google Scholar 

  26. Hallad, S.A., Patil, A.Y., BanapurmathNR, H.A.M., Shettar, A.S., Ayachit, N.H.: Experimental and numerical validation on the utilization of polymer based nano-composites for structural applications using FEA software tool. Mater. Focus 6(6), 685–690 (2017)

    Article  Google Scholar 

  27. Katz, S., Tal, A., Basri, R.: Direct visibility of point sets. In: Proceeding ACM SIGGRAPH Conference on Computer Graphic, pp. 1–8 (2007)

  28. Hanh, L.D., Duc, L.M.: Planar object recognition for bin picking application. In: Proceedings of 5th NAFOSTED Conference on Information and Computer Science (NICS), Ho Chi Minh, pp. 213–218 (2018)

  29. Rusu, R.B.: Semantic 3D object maps for everyday manipulation in human living environments. Künstl Intell. 24, 345–348 (2010)

    Article  Google Scholar 

  30. Drost, B., Ulrich, M., Navab, N., Ilic, S.: Model globally, match locally. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition, pp. 998–1005 (2010)

  31. Bullet Physics SDK, Real-time collision detection and multi-physics simulation for VR, games, visual effects, robotics, machine learning etc, https://opensource.google/projects/bullet3

  32. Kverneland, K. O.: The ISO System of Limits and Fits—Tolerances and Deviations. (2007)

Download references

Acknowledgements

We acknowledgment the support of time and facilities from Ho Chi Minh City University of Technology (HCMUT), VNU-HCM for this study

Author information

Authors and Affiliations

  1. Mechanical Department, Ho Chi Minh City University of Technology, Vietnam National University, Ho Chi Minh City, 70000, Vietnam

    Le Duc Hanh & Khuong Thanh Gia Hieu

Authors
  1. Le Duc Hanh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Khuong Thanh Gia Hieu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Le Duc Hanh.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hanh, L.D., Hieu, K.T.G. 3D matching by combining CAD model and computer vision for autonomous bin picking. Int J Interact Des Manuf 15, 239–247 (2021). https://doi.org/10.1007/s12008-021-00762-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12008-021-00762-4

Keywords

Search

Navigation