Abstract
Computer vision has been gaining interest in a wide range of research areas in recent years, from medical to industrial robotics. The architecture, engineering and construction and facility management sector ranks as one of the most intensive fields where vision-based systems/methods are used to facilitate decision making processes during the construction phase. Construction sites make efficient monitoring extremely tedious and difficult due to clutter and disorder. Extensive research has been carried out to investigate the potential to utilise computer vision for assisting on-site managerial tasks. This paper reviews studies on computer vision in the past decade, with a focus on state-of-the-art methods in a typical vision-based scheme, and discusses challenges associated with their application. This research aims to guide practitioners to successfully find suitable approaches for a particular project.
Similar content being viewed by others
References
Lowe D (2015) The Computer Vision Industry. https://www.cs.ubc.ca/~lowe/vision.html. Accessed 11 Nov 2019
Forsyth DA, Ponce J (2002) Computer vision: a modern approach. Prentice Hall Professional Technical Reference, Upper Saddle River
Yang J et al (2015) Construction performance monitoring via still images, time-lapse photos, and video streams: now, tomorrow, and the future. Adv Eng Inform 29(2):211–224
Han KK, Golparvar-Fard M (2017) Potential of big visual data and building information modeling for construction performance analytics: an exploratory study. Autom Constr 73:184–198
Wang D, Dai F, Ning X (2015) Risk assessment of work-related musculoskeletal disorders in construction: state-of-the-art review. J Constr Eng Manag 141(6):04015008
Seo J et al (2015) Computer vision techniques for construction safety and health monitoring. Adv Eng Inform 29(2):239–251
Teizer J (2015) Status quo and open challenges in vision-based sensing and tracking of temporary resources on infrastructure construction sites. Adv Eng Inform 29(2):225–238
Ranaweera K, Ruwanpura J, Fernando S (2013) Automated real-time monitoring system to measure shift production of tunnel construction projects. J Comput Civ Eng 27(1):68–77
Rebolj D et al (2017) Point cloud quality requirements for Scan-vs-BIM based automated construction progress monitoring. Autom Constr 84:323–334
Fang Q et al (2018) Detecting non-hardhat-use by a deep learning method from far-field surveillance videos. Autom Constr 85:1–9
Fang W et al (2018) Automated detection of workers and heavy equipment on construction sites: a convolutional neural network approach. Adv Eng Inform 37:139–149
Luo X, Li H, Yang X, Yu Y, Cao D (2019) Capturing and understanding workers’ activities in far-field surveillance videos with deep action recognition and Bayesian nonparametric learning. Comput Aided Civ Infrastruct Eng 34(4):333–351
Azar ER, Dickinson S, McCabe B (2013) Server-customer interaction tracker: computer vision-based system to estimate dirt-loading cycles. J Constr Eng Manag 139(7):785–794
Luo X et al (2018) Towards efficient and objective work sampling: recognizing workers’ activities in site surveillance videos with two-stream convolutional networks. Autom Constr 94:360–370
Mneymneh BE, Abbas M, Khoury H (2019) Vision-based framework for intelligent monitoring of hardhat wearing on construction sites. J Comput Civ Eng 33(2):04018066
Wu Y et al (2010) Object recognition in construction-site images using 3D CAD-based filtering. J Comput Civ Eng 24(1):56–64
Kim J, Chi S, Seo J (2018) Interaction analysis for vision-based activity identification of earthmoving excavators and dump trucks. Autom Constr 87:297–308
Hui L, Park M-W, Brilakis I (2015) Automated brick counting for façade construction progress estimation. J Comput Civ Eng 29(6):04014091
Bae H, Golparvar-Fard M, White J (2015) Image-based localization and content authoring in structure-from-motion point cloud models for real-time field reporting applications. J Comput Civ Eng 29(4):B4014008
Park M-W, Elsafty N, Zhu Z (2015) Hardhat-wearing detection for enhancing on-site safety of construction workers. J Constr Eng Manag 141(9):04015024
Zhu Z, Ren X, Chen Z (2016) Visual tracking of construction jobsite workforce and equipment with particle filtering. J Comput Civ Eng 30(6):04016023
Gong J, Caldas CH, Gordon C (2011) Learning and classifying actions of construction workers and equipment using Bag-of-Video-Feature-Words and Bayesian network models. Adv Eng Inform 25(4):771–782
Yang J, Shi Z, Wu Z (2016) Vision-based action recognition of construction workers using dense trajectories. Adv Eng Inform 30(3):327–336
Yuan C, Li S, Cai H (2017) Vision-based excavator detection and tracking using hybrid kinematic shapes and key nodes. J Comput Civ Eng 31(1):04016038
Konstantinou E, Brilakis I (2018) Matching construction workers across views for automated 3D vision tracking on-site. J Constr Eng Manag 144(7):04018061
Soltani MM, Zhu Z, Hammad A (2018) Framework for location data fusion and pose estimation of excavators using stereo vision. J Comput Civ Eng 32(6):04018045
Dai F, Lu M (2013) Three-dimensional modeling of site elements by analytically processing image data contained in site photos. J Constr Eng Manag 139(7):881–894
Golparvar-Fard M, Peña-Mora F, Savarese S (2011) Integrated sequential as-built and as-planned representation with D4AR tools in support of decision-making tasks in the AEC/FM industry. J Constr Eng Manag 137(12):1099–1116
Park M-W, Koch C, Brilakis I (2012) Three-dimensional tracking of construction resources using an on-site camera system. J Comput Civ Eng 26(4):541–549
Kim H, Kim H (2018) 3D reconstruction of a concrete mixer truck for training object detectors. Autom Constr 88:23–30
Chi S, Caldas CH (2012) Image-based safety assessment: automated spatial safety risk identification of earthmoving and surface mining activities. J Constr Eng Manag 138(3):341–351
Son H, Kim C (2010) 3D structural component recognition and modeling method using color and 3D data for construction progress monitoring. Autom Constr 19(7):844–854
Han S, Lee S (2013) A vision-based motion capture and recognition framework for behavior-based safety management. Autom Constr 35:131–141
Seo J et al (2015) Motion data-driven biomechanical analysis during construction tasks on sites. J Comput Civ Eng 29(4):B4014005
Guo H et al (2018) Image-and-skeleton-based parameterized approach to real-time identification of construction workers’ unsafe behaviors. J Constr Eng Manag 144(6):04018042
Han S, Lee S, Peña-Mora F (2014) Comparative study of motion features for similarity-based modeling and classification of unsafe actions in construction. J Comput Civ Eng 28(5):A4014005
Han S, Lee S, Peña-Mora F (2013) Vision-based detection of unsafe actions of a construction worker: case study of ladder climbing. J Comput Civ Eng 27(6):635–644
Turkan Y et al (2013) Toward automated earned value tracking using 3D imaging tools. J Constr Eng Manag 139(4):423–433
Rausch C et al (2017) Optimum assembly planning for modular construction components. J Comput Civ Eng 31(1):04016039
Chen J et al (2017) Principal axes descriptor for automated construction-equipment classification from point clouds. J Comput Civ Eng 31(2):04016058
Sharif M-M et al (2017) Automated model-based finding of 3D objects in cluttered construction point cloud models. Comput Aided Civ Infrastruct Eng 32(11):893–908
Wang Q, Cheng JCP, Sohn H (2017) Automated estimation of reinforced precast concrete rebar positions using colored laser scan data. Comput Aided Civ Infrastruct Eng 32(9):787–802
Rausch C et al (2017) Kinematics chain based dimensional variation analysis of construction assemblies using building information models and 3D point clouds. Autom Constr 75:33–44
Teizer J, Allread BS, Mantripragada U (2010) Automating the blind spot measurement of construction equipment. Autom Constr 19(4):491–501
Roh S, Aziz Z, Peña-Mora F (2011) An object-based 3D walk-through model for interior construction progress monitoring. Autom Constr 20(1):66–75
Jeelani I, Han K, Albert A (2018) Automating and scaling personalized safety training using eye-tracking data. Autom Constr 93:63–77
Wang Z, Li H, Zhang X (2019) Construction waste recycling robot for nails and screws: computer vision technology and neural network approach. Autom Constr 97:220–228
Azar ER (2017) Semantic annotation of videos from equipment-intensive construction operations by shot recognition and probabilistic reasoning. J Comput Civ Eng 31(5):04017042
Golparvar-Fard M, Heydarian A, Niebles JC (2013) Vision-based action recognition of earthmoving equipment using spatio-temporal features and support vector machine classifiers. Adv Eng Inform 27(4):652–663
Khosrowpour A, Niebles JC, Golparvar-Fard M (2014) Vision-based workface assessment using depth images for activity analysis of interior construction operations. Autom Constr 48:74–87
Yang J et al (2014) Vision-based tower crane tracking for understanding construction activity. J Comput Civ Eng 28(1):103–112
Irizarry J, Costa DB (2016) Exploratory study of potential applications of unmanned aerial systems for construction management tasks. J Manag Eng 32(3):05016001
Bang S, Kim H, Kim H (2017) UAV-based automatic generation of high-resolution panorama at a construction site with a focus on preprocessing for image stitching. Autom Constr 84:70–80
Golparvar-Fard M, Peña-Mora F, Savarese S (2009) D4AR–a 4-dimensional augmented reality model for automating construction progress monitoring data collection, processing and communication. J Inf Technol Constr 14(13):129–153
Ergan S et al (2008) Technological assessment and process implications of field data capture technologies for construction and facility/infrastructure management. Electron J Inf Technol Constr 13:134–154
Cho YK, Gai M (2014) Projection-recognition-projection method for automatic object recognition and registration for dynamic heavy equipment operations. J Comput Civ Eng 28(5):A4014002
Kim H, Kim K, Kim H (2016) Data-driven scene parsing method for recognizing construction site objects in the whole image. Autom Constr 71:271–282
Chen J, Fang Y, Cho YK (2017) Real-time 3D crane workspace update using a hybrid visualization approach. J Comput Civ Eng 31(5):04017049
Azar ER, McCabe B (2012) Automated visual recognition of dump trucks in construction videos. J Comput Civ Eng 26(6):769–781
Ding L et al (2018) A deep hybrid learning model to detect unsafe behavior: integrating convolution neural networks and long short-term memory. Autom Constr 86:118–124
Kim C, Son H, Kim C (2013) Automated construction progress measurement using a 4D building information model and 3D data. Autom Constr 31:75–82
Park M-W, Brilakis I (2012) Construction worker detection in video frames for initializing vision trackers. Autom Constr 28:15–25
Park M-W, Brilakis I (2016) Continuous localization of construction workers via integration of detection and tracking. Autom Constr 72:129–142
Hamledari H, McCabe B, Davari S (2017) Automated computer vision-based detection of components of under-construction indoor partitions. Autom Constr 74:78–94
Canny J (1987) A computational approach to edge detection. In: Fischler MA, Firschein O (eds) Readings in computer vision. Morgan Kaufmann, San Francisco, pp 184–203
Sobel I, Feldman G (1968) A 3 × 3 isotropic gradient operator for image processing. Pattern Classif Scene Anal 271–272
Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Machine Intell 24(5):603–619
Harris CG, Stephens M (1988) A combined corner and edge detector. In: Alvey vision conference. Citeseer
Smith SM, Brady JM (1997) SUSAN—a new approach to low level image processing. Int J Comput Vis 23(1):45–78
Rosten E, Drummond T (2006) Machine learning for high-speed corner detection. In: European conference on computer vision. Springer
Lowe DG (1999) Object recognition from local scale-invariant features. In: Proceedings of the seventh IEEE international conference on computer vision
Wu H, Zhao J (2018) An intelligent vision-based approach for helmet identification for work safety. Comput Ind 100:267–277
Stauffer C, Grimson WEL (1999) Adaptive background mixture models for real-time tracking. In: Proceedings. 1999 IEEE computer society conference on computer vision and pattern recognition (Cat. No PR00149). IEEE
Chen J, Fang Y, Cho YK (2018) Performance evaluation of 3D descriptors for object recognition in construction applications. Autom Constr 86:44–52
Zhu Z, Davari K (2015) Comparison of local visual feature detectors and descriptors for the registration of 3D building scenes. J Comput Civ Eng 29(5):04014071
Kolar Z, Chen H, Luo X (2018) Transfer learning and deep convolutional neural networks for safety guardrail detection in 2D images. Autom Constr 89:58–70
Han K, Degol J, Golparvar-Fard M (2018) Geometry- and appearance-based reasoning of construction progress monitoring. J Constr Eng Manag 144(2):04017110
Chen H et al (2019) A proactive workers’ safety risk evaluation framework based on position and posture data fusion. Autom Constr 98:275–288
Fang W et al (2018) Falls from heights: a computer vision-based approach for safety harness detection. Autom Constr 91:53–61
Kim H et al (2018) Detecting construction equipment using a region-based fully convolutional network and transfer learning. J Comput Civ Eng 32(2):04017082
Fang Q et al (2018) Computer vision aided inspection on falling prevention measures for steeplejacks in an aerial environment. Autom Constr 93:148–164
Kim D et al (2019) Remote proximity monitoring between mobile construction resources using camera-mounted UAVs. Autom Constr 99:168–182
Soltani MM, Zhu Z, Hammad A (2017) Skeleton estimation of excavator by detecting its parts. Autom Constr 82:1–15
Chi S, Caldas CH (2011) Automated object identification using optical video cameras on construction sites. Comput Aided Civ Infrastruct Eng 26(5):368–380
Bai Y, Huan J, Kim S (2012) Measuring bridge construction efficiency using the wireless real-time video monitoring system. J Manag Eng 28(2):120–126
Memarzadeh M, Golparvar-Fard M, Niebles JC (2013) Automated 2D detection of construction equipment and workers from site video streams using histograms of oriented gradients and colors. Autom Constr 32:24–37
Kim K, Kim H, Kim H (2017) Image-based construction hazard avoidance system using augmented reality in wearable device. Autom Constr 83:390–403
Gouveia LTd et al (2011) Entropy-based approach to analyze and classify mineral aggregates. J Comput Civ Eng 25(1):75–84
Son H, Kim C, Kim C (2012) Automated color model–based concrete detection in construction-site images by using machine learning algorithms. J Comput Civ Eng 26(3):421–433
Kim J, Chi S (2017) Adaptive detector and tracker on construction sites using functional integration and online learning. J Comput Civ Eng 31(5):04017026
Rezazadeh Azar E, McCabe B (2012) Part based model and spatial–temporal reasoning to recognize hydraulic excavators in construction images and videos. Autom Construct 24:194–202
Comaniciu D, Ramesh V, Meer P (2003) Kernel-based object tracking. IEEE Trans Pattern Anal Mach Intell 5:564–575
Park M-W, Makhmalbaf A, Brilakis I (2011) Comparative study of vision tracking methods for tracking of construction site resources. Autom Constr 20(7):905–915
Zhu Z et al (2016) Predicting movements of onsite workers and mobile equipment for enhancing construction site safety. Autom Constr 68:95–101
Bügler M et al (2017) Fusion of photogrammetry and video analysis for productivity assessment of earthwork processes. Comput Aided Civ Infrastruct Eng 32(2):107–123
Kim H, Kim K, Kim H (2016) Vision-based object-centric safety assessment using fuzzy inference: monitoring struck-by accidents with moving objects. J Comput Civ Eng 30(4):04015075
Lee Y-J, Park M-W (2019) 3D tracking of multiple onsite workers based on stereo vision. Autom Constr 98:146–159
Xiao B, Zhu Z (2018) Two-dimensional visual tracking in construction scenarios: a comparative study. J Comput Civ Eng 32(3):04018006
Camera Calibrator App, in Computer Vision Toolbox™. 2013, MathWorks®. p. The Camera Calibrator app allows you to estimate camera intrinsics, extrinsics, and lens distortion parameters. You can use these camera parameters for various computer vision applications. These applications include removing the effects of lens distortion from an image, measuring planar objects, or reconstructing 3-D scenes from multiple cameras
Bouguet J-Y (2004) Camera Calibration Toolbox for Matlab. http://www.vision.caltech.edu/bouguetj/calib_doc/. Accessed 15 Nov 2019
Khoury H et al (2015) Infrastructureless approach for ubiquitous user location tracking in construction environments. Autom Constr 56:47–66
Rodriguez-Gonzalvez P et al (2014) Image-based modeling of built environment from an unmanned aerial system. Autom Constr 48:44–52
Son H, Kim C, Cho YK (2017) Automated schedule updates using as-built data and a 4D building information model. J Manag Eng 33(4):04017012
Pučko Z, Šuman N, Rebolj D (2018) Automated continuous construction progress monitoring using multiple workplace real time 3D scans. Adv Eng Inform 38:27–40
Turkan Y et al (2012) Automated progress tracking using 4D schedule and 3D sensing technologies. Autom Constr 22:414–421
Golparvar-Fard M et al (2009) Visualization of construction progress monitoring with 4D simulation model overlaid on time-lapsed photographs. J Comput Civ Eng 23(6):391–404
Aggarwal JK, Ryoo MS (2011) Human activity analysis: a review. ACM Comput Surv CSUR 43(3):16
Luo X et al (2018) Recognizing diverse construction activities in site images via relevance networks of construction-related objects detected by convolutional neural networks. J Comput Civ Eng 32(3):04018012
Luo H et al (2018) Convolutional neural networks: computer vision-based workforce activity assessment in construction. Autom Constr 94:282–289
Kong L et al (2018) Quantifying the physical intensity of construction workers, a mechanical energy approach. Adv Eng Inform 38:404–419
Zhang H, Yan X, Li H (2018) Ergonomic posture recognition using 3D view-invariant features from single ordinary camera. Autom Constr 94:1–10
Ray SJ, Teizer J (2013) Computing 3D blind spots of construction equipment: implementation and evaluation of an automated measurement and visualization method utilizing range point cloud data. Autom Constr 36:95–107
Ibrahim YM et al (2009) Towards automated progress assessment of workpackage components in construction projects using computer vision. Adv Eng Inform 23(1):93–103
Zhang X et al (2009) Automating progress measurement of construction projects. Autom Constr 18(3):294–301
Han KK, Golparvar-Fard M (2015) Appearance-based material classification for monitoring of operation-level construction progress using 4D BIM and site photologs. Autom Constr 53:44–57
Dimitrov A, Golparvar-Fard M (2014) Vision-based material recognition for automated monitoring of construction progress and generating building information modeling from unordered site image collections. Adv Eng Inform 28(1):37–49
Kim M-K et al (2015) A framework for dimensional and surface quality assessment of precast concrete elements using BIM and 3D laser scanning. Autom Constr 49:225–238
Tang P, Huber D, Akinci B (2011) Characterization of laser scanners and algorithms for detecting flatness defects on concrete surfaces. J Comput Civ Eng 25(1):31–42
Golparvar-Fard M, Peña-Mora F, Savarese S (2015) Automated progress monitoring using unordered daily construction photographs and IFC-based building information models. J Comput Civ Eng 29(1):04014025
Wang C, Cho YK (2015) Smart scanning and near real-time 3D surface modeling of dynamic construction equipment from a point cloud. Autom Constr 49:239–249
Azar ER (2016) Construction equipment identification using marker-based recognition and an active zoom camera. J Comput Civ Eng 30(3):04015033
Asadi K et al (2018) Vision-based integrated mobile robotic system for real-time applications in construction. Autom Constr 96:470–482
Shahandashti SM et al (2011) Data-fusion approaches and applications for construction engineering. J Constr Eng Manag 137(10):863–869
Soltani MM, Zhu Z, Hammad A (2016) Automated annotation for visual recognition of construction resources using synthetic images. Autom Constr 62:14–23
Liu K, Golparvar-Fard M (2015) Crowdsourcing construction activity analysis from jobsite video streams. J Constr Eng Manag 141(11):04015035
Han KK, Cline D, Golparvar-Fard M (2015) Formalized knowledge of construction sequencing for visual monitoring of work-in-progress via incomplete point clouds and low-LoD 4D BIMs. Adv Eng Inform 29(4):889–901
Gong J, Caldas CH (2010) Computer vision-based video interpretation model for automated productivity analysis of construction operations. J Comput Civ Eng 24(3):252–263
Gong J, Caldas CH (2011) An object recognition, tracking, and contextual reasoning-based video interpretation method for rapid productivity analysis of construction operations. Autom Constr 20(8):1211–1226
Brilakis I, Park M-W, Jog G (2011) Automated vision tracking of project related entities. Adv Eng Inform 25(4):713–724
Jog GM, Brilakis IK, Angelides DC (2011) Testing in harsh conditions: tracking resources on construction sites with machine vision. Autom Constr 20(4):328–337
Funding
Funding was provided by Australian Research Council (Grant No. LP180100222).
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of interest
On behalf of all authors, the corresponding author states that there is no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Xu, S., Wang, J., Shou, W. et al. Computer Vision Techniques in Construction: A Critical Review. Arch Computat Methods Eng 28, 3383–3397 (2021). https://doi.org/10.1007/s11831-020-09504-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11831-020-09504-3