ReviewReview of image-based analysis and applications in construction
Introduction
The construction industry is the main contributor to developing and maintaining the civil infrastructure, with billions of dollars spent every year. However, the construction industry has been deemed to be relatively conservative in terms of adopting data-driven technology innovations to improve safety and productivity [1]. In contrast to other industries, productivity in construction has not improved over the past decades [2,3]. The U.S. construction-sector productivity is lower today than it was in 1968 [4]. Improving construction productivity worldwide, however, has the potential to increase the added value by 1.6 trillion dollars, which corresponds to a global GDP increase of 2% [5]. The construction industry is currently one of the least digitized industries in the world according to MGI's digitization index [6]. For example, it is estimated that current site managers consume almost half their time manually collecting and processing progress monitoring data before making a decision [7]. Hence there is a clear need for “automation” to achieve maximum efficiency. Among the most promising automation technologies is Image-based analysis as it offers not only a faster and more consistent way to obtain more in-depth information than obtained from manual visual analysis.
Computer vision and image-based learning techniques have recently matured to allow for computers to automatically analyze images and videos. Computer vision techniques have been introduced since the 1960s [8] and have attracted various researchers in multiple other fields because images are easy to collect and, in some cases, readily available (e.g., security CCTV cameras). Currently, many researchers are investigating the use of advanced image-based methods in the construction industry with potential applications to improve workers' safety, monitor activities progress and facilitate project management, assess structural damages (e.g., crack detection) for structural health monitoring and rehabilitation purposes, among others.
A major area image-based analysis can address is construction site safety. The construction process itself is very challenging due to the extensive interactions among workers, equipment, site spaces, materials, and the environment, which exposes participants to multiple hazardous situations on a daily basis, making them prone to injuries or even deaths. Injuries in the construction industry constituted 7% of non-fatal injuries and 14% of workplace deaths in the United States in 2018 [9]. In Canada, incidents in the construction industry have constituted around 10% of lost-time claims and 20% of workplace fatalities over the past three years [10]. In Great Britain, 54,000 injuries are happening to construction workers each year, the second-highest number of injuries among all industries [11]. Such accidents have cost the British economy 1.2 Billion pounds in 2017/18 [11]. It is estimated that 80% of accidents on construction sites can be attributed to the workers' behavior [12]. This includes not wearing protective equipment such as hardhats and safety harness, being struck by vehicles and construction equipment, maintaining unhealthy postures, and others. Hence, monitoring the construction site to limit such unsafe behavior is of extreme importance. However, the current monitoring methods are inefficient as they depend on manual observations and data collection.
Another area that could potentially benefit from image-based analysis technology is productivity and progress monitoring. Out of every ten construction projects, five are behind schedule and six are overbudget [13]. Among the major reasons for such poor performance is the flawed progress monitoring and reporting which leads to improper management and decision making [14]. Traditional worksite monitoring practices are conducted manually through visual observations and using paper-based drawings or, at best, tablets that require specific views of drawings to be manually generated and rendered in advance for each inspection task [15]. This process is very time-consuming, given thousands of elements and hundreds of construction activities on the site, as well as prone to errors. For example, large constructions, such as highways, require traveling between the different work areas to gather information or attend specific issues which translates to a misuse of time and transportation costs which makes the monitoring efforts infrequent and untimely enough for effective decision making. In contrast, image-based methods can be used to provide real-time information about the status of various operations throughout the construction site through documenting work progress as well as worker's and equipment's behavior and productivity. This allows project managers to identify delays in the construction plans and make better and more timely decisions concerning the project progress moving forward.
Image-based analysis is not only useful during construction, but also after construction as well as all infrastructure components need to be continuously monitored to ensure they are physically fit and viable for usage. Even after the structure is fully constructed, frequent monitoring is required to help with decision making and resource allocation regarding the structure's rehabilitation and upgrades. Traditional methods involve sending inspectors to assess the asset conditions and file reports that estimate any required maintenance work. While visual inspections are widely used for preliminary and regular inspections because of its effectiveness in detecting external defects such as cracks and spalling which could be evaluated as symptoms of potential structural degradation, subjectivity and inconsistency are inevitable in visual inspections as different inspectors may evaluate the same structure differently [16]. Furthermore, some parts of the structure are not accessible for inspectors, making an overall assessment sometimes impractical [16]. An inspection site visit typically takes 4 h to be completed and for each hour spent in the field for inspection, additional three hours are spent in the office to generate the reports [17]. As an example of how time-consuming inspection is (using current methods) Ontario ministry of education has issued a bid in 2010 seeking a company to inspect a total of 4800 schools over a five-year period [18]. Errors and gaps produced by in manual assessments not only waste time and money, but can also lead to catastrophic events, such as the collapse of I-35 W highway bridge in Minneapolis, MN which caused 13 deaths and 145 injuries [19]. or the failure of Oroville dam spillways which took place in February 2017, forcing around 200,000 Californians to evacuate [20,21].
With image-based analysis being a potential key tool in enhancing the construction industry, this paper reviews previous works regarding the use of image-based analysis in the construction industry, following the research methodology in Fig. 1. It highlights current research trends and innovative works, challenges and limitations, and future research potential. The first step of the research technology (section 2) presents the approach used in collecting the journal papers to be analyzed. The second step (section 3) is presenting a general overview of the findings, including a thorough explanation of the most commonly used computer vision techniques. Finally, the papers were further analyzed according to their different applications to highlight notable achievements and major research gaps (sections 4, 5).
Section snippets
Survey methodology
An extensive literature review was conducted using the Scopus journal database as it automatically aggregates the search results from multiple academic databases, including: the American Society of Civil Engineers (ASCE) library, Elsevier (ScienceDirect), Wiley, IeeeXplore, and others. Only high-level keywords ((“Computer vision” OR “Image*” OR “Video*”) AND (“Construction”)) were used so that a wide search area can be covered. To highlight recent technical advances, the search was only limited
General survey results
Analyzing the 100 papers found in the construction literature related to image-based analysis, Fig. 2 illustrates an overview analysis of the main research areas and their applications. Main applications of image-based analysis in construction can be grouped into three main categories; (1) construction safety; (2) progress monitoring; and (3) damage assessment. A number of related review papers were also investigated (right-hand-side of Fig. 2).
Before undergoing further analysis, the number of
Applications of image-based analysis in construction
A perfectly automated inspection or monitoring framework consists of two main modules; automated data acquisition using UAVs or stationary cameras (e.g. CCTV), and automated decision making through data processing using image-based analysis techniques [8]. While the rapid growth in the drone industry has made the first module a reality, the analysis module and turning images into decision-support information remains a challenge. Towards this goal, the following three subsections review the
Summary and concluding remarks
This paper presents an overview of major image-based learning techniques utilized and their applications in the construction industry. Most applications are in the fields of site safety, progress monitoring, and damage assessment. The reviewed articles on image-based analysis applications in construction represent groundbreaking achievements in terms of using computer vision technology to enhance construction productivity and safety. To maximize the benefit from computer vision technology,
Data availability statement
All data generated or analyzed during the study are included in the published paper
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper
References (153)
- et al.
Convolutional neural networks: computer vision-based workforce activity assessment in construction
Autom. Constr.
(2018) - et al.
Vision-based action recognition of construction workers using dense trajectories
Adv. Eng. Inform.
(2016) - et al.
Proactive behavior-based safety management for construction safety improvement
Saf. Sci.
(2015) - et al.
Construction performance monitoring via still images, time-lapse photos, and video streams: now, tomorrow, and the future
Adv. Eng. Inform.
(2015) - et al.
A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure
Adv. Eng. Inform.
(2015) - et al.
A deep hybrid learning model to detect unsafe behavior: integrating convolution neural networks and long short-term memory
Autom. Constr.
(2018) - et al.
Predicting movements of onsite workers and mobile equipment for enhancing construction site safety
Autom. Constr.
(2016) - et al.
Comparison of deep convolutional neural networks and edge detectors for image-based crack detection in concrete
Constr. Build. Mater.
(2018) - et al.
Automated hardhat detection for construction safety applications
Procedia Eng.
(2017) - et al.
Detecting non-hardhat-use by a deep learning method from far-field surveillance videos
Autom. Constr.
(2018)
Automatic detection of hardhats worn by construction personnel: a deep learning approach and benchmark dataset
Autom. Constr.
Recognizing people’s identity in construction sites with computer vision: a spatial and temporal attention pooling network
Adv. Eng. Inform.
An automatic and non-invasive physical fatigue assessment method for construction workers
Autom. Constr.
Detection of construction workers under varying poses and changing background in image sequences via very deep residual networks
Autom. Constr.
A deep learning-based approach for mitigating falls from height with computer vision: convolutional neural network
Adv. Eng. Inform.
Transfer learning and deep convolutional neural networks for safety guardrail detection in 2D images
Autom. Constr.
Ergonomic posture recognition using 3D view-invariant features from single ordinary camera
Autom. Constr.
A deep learning-based method for detecting non-certified work on construction sites
Adv. Eng. Inform.
Falls from heights: a computer vision-based approach for safety harness detection
Autom. Constr.
Construction worker’s awkward posture recognition through supervised motion tensor decomposition
Autom. Constr.
Full body pose estimation of construction equipment using computer vision and deep learning techniques
Autom. Constr.
Combining inverse photogrammetry and BIM for automated labeling of construction site images for machine learning
Autom. Constr.
Two-step long short-term memory method for identifying construction activities through positional and attentional cues
Autom. Constr.
A vision-based marker-less pose estimation system for articulated construction robots
Autom. Constr.
Action recognition of earthmoving excavators based on sequential pattern analysis of visual features and operation cycles
Autom. Constr.
End-to-end vision-based detection, tracking and activity analysis of earthmoving equipment filmed at ground level
Autom. Constr.
3D tracking of multiple onsite workers based on stereo vision
Autom. Constr.
Vision-based detection and visualization of dynamic workspaces
Autom. Constr.
Interior construction state recognition with 4D BIM registered image sequences
Autom. Constr.
Automated detection of workers and heavy equipment on construction sites: a convolutional neural network approach
Adv. Eng. Inform.
Towards efficient and objective work sampling: recognizing workers’ activities in site surveillance videos with two-stream convolutional networks
Autom. Constr.
Automatic matching of construction onsite resources under camera views
Autom. Constr.
Automated computer vision-based detection of components of under-construction indoor partitions
Autom. Constr.
Integrated detection and tracking of workforce and equipment from construction jobsite videos
Autom. Constr.
Vision-based action recognition of construction workers using dense trajectories
Adv. Eng. Inform.
Computer vision application programming for settlement monitoring in a drainage tunnel
Autom. Constr.
Computer vision-based concrete crack detection using U-net fully convolutional networks
Autom. Constr.
A hybrid SURF-DIC algorithm to estimate local displacements in structures using low-cost conventional cameras
Eng. Fail. Anal.
Underground sewer pipe condition assessment based on convolutional neural networks
Autom. Constr.
Autonomous concrete crack detection using deep fully convolutional neural network
Autom. Constr.
KPMG Report: Construction Industry Slow to Adopt New Technology
The Construction Industry Has a Productivity Problem — And here's how to Solve it
Reinventing Construction: A Route to Higher Productivity
Digital Globalization: The New Era of Global Flows
Automatic indoor construction process monitoring for tiles based on BIM and computer vision
J. Constr. Eng. Manag.
“Advances in Computer Vision-Based Civil Infrastructure Inspection and Monitoring.” Engineering, Chinese Academy of Engineering, 5(american socie)
National Census of Fatal Occupational Injuries in 2018
National Work Injury, Disease and Fatality Statistics
Construction statistics in Great Britain
LCI National Webinar: preliminary findings from national project performance research
Cited by (71)
Applications of existing and emerging construction safety technologies
2024, Automation in ConstructionA survey on crack detection in concrete surface using image processing and machine learning
2024, Journal of Building Pathology and RehabilitationExploring the adoption of technology against delays in construction projects
2024, Engineering, Construction and Architectural ManagementBrain-regulated learning for classifying on-site hazards with small datasets
2024, Computer-Aided Civil and Infrastructure Engineering