Review
Review of image-based analysis and applications in construction

https://doi.org/10.1016/j.autcon.2020.103516Get rights and content

Highlights

  • Reviewed 100 recent articles (2015–2020) on the use of image-based analysis technology in construction.

  • The researched papers show three main research trends: Construction Safety, Progress Monitoring, and Damage Assessment.

  • Artificial Neural Networks (ANNs) is the most commonly used technique for image-based analysis models, especially after 2017.

  • Technical challenges and research gaps are identified.

  • Developing applications that provide high-level understanding of the overall scene is an area of future research potential.

Abstract

Image-based analysis techniques offer a robust way to solve engineering problems due to the availability of visual data (e.g., surveillance cameras). Hence, research efforts have focused on applying Image-based techniques in the construction industry to improve the safety and productivity of construction operations as well as the resilience and sustainability of the construction assets. This paper explores the state-of-the-art in Image-based analysis techniques and their applications in construction. Over 100 journal papers were retrieved from the Scopus database for an in-depth review of major applications, benefits, and areas of future research potential. Accordingly, Three main research directions were identified that utilize image-based technologies: (1) construction safety; (2) progress monitoring; and (3) damage assessment. It is observed that most research efforts focused on object detection (e.g., hardhats, defects) for safety inspection and repair planning. Potential future developments include integrating object detection with quantification and sizing techniques to develop more comprehensive applications.

Introduction

The construction industry is the main contributor to developing and maintaining the civil infrastructure, with billions of dollars spent every year. However, the construction industry has been deemed to be relatively conservative in terms of adopting data-driven technology innovations to improve safety and productivity [1]. In contrast to other industries, productivity in construction has not improved over the past decades [2,3]. The U.S. construction-sector productivity is lower today than it was in 1968 [4]. Improving construction productivity worldwide, however, has the potential to increase the added value by 1.6 trillion dollars, which corresponds to a global GDP increase of 2% [5]. The construction industry is currently one of the least digitized industries in the world according to MGI's digitization index [6]. For example, it is estimated that current site managers consume almost half their time manually collecting and processing progress monitoring data before making a decision [7]. Hence there is a clear need for “automation” to achieve maximum efficiency. Among the most promising automation technologies is Image-based analysis as it offers not only a faster and more consistent way to obtain more in-depth information than obtained from manual visual analysis.

Computer vision and image-based learning techniques have recently matured to allow for computers to automatically analyze images and videos. Computer vision techniques have been introduced since the 1960s [8] and have attracted various researchers in multiple other fields because images are easy to collect and, in some cases, readily available (e.g., security CCTV cameras). Currently, many researchers are investigating the use of advanced image-based methods in the construction industry with potential applications to improve workers' safety, monitor activities progress and facilitate project management, assess structural damages (e.g., crack detection) for structural health monitoring and rehabilitation purposes, among others.

A major area image-based analysis can address is construction site safety. The construction process itself is very challenging due to the extensive interactions among workers, equipment, site spaces, materials, and the environment, which exposes participants to multiple hazardous situations on a daily basis, making them prone to injuries or even deaths. Injuries in the construction industry constituted 7% of non-fatal injuries and 14% of workplace deaths in the United States in 2018 [9]. In Canada, incidents in the construction industry have constituted around 10% of lost-time claims and 20% of workplace fatalities over the past three years [10]. In Great Britain, 54,000 injuries are happening to construction workers each year, the second-highest number of injuries among all industries [11]. Such accidents have cost the British economy 1.2 Billion pounds in 2017/18 [11]. It is estimated that 80% of accidents on construction sites can be attributed to the workers' behavior [12]. This includes not wearing protective equipment such as hardhats and safety harness, being struck by vehicles and construction equipment, maintaining unhealthy postures, and others. Hence, monitoring the construction site to limit such unsafe behavior is of extreme importance. However, the current monitoring methods are inefficient as they depend on manual observations and data collection.

Another area that could potentially benefit from image-based analysis technology is productivity and progress monitoring. Out of every ten construction projects, five are behind schedule and six are overbudget [13]. Among the major reasons for such poor performance is the flawed progress monitoring and reporting which leads to improper management and decision making [14]. Traditional worksite monitoring practices are conducted manually through visual observations and using paper-based drawings or, at best, tablets that require specific views of drawings to be manually generated and rendered in advance for each inspection task [15]. This process is very time-consuming, given thousands of elements and hundreds of construction activities on the site, as well as prone to errors. For example, large constructions, such as highways, require traveling between the different work areas to gather information or attend specific issues which translates to a misuse of time and transportation costs which makes the monitoring efforts infrequent and untimely enough for effective decision making. In contrast, image-based methods can be used to provide real-time information about the status of various operations throughout the construction site through documenting work progress as well as worker's and equipment's behavior and productivity. This allows project managers to identify delays in the construction plans and make better and more timely decisions concerning the project progress moving forward.

Image-based analysis is not only useful during construction, but also after construction as well as all infrastructure components need to be continuously monitored to ensure they are physically fit and viable for usage. Even after the structure is fully constructed, frequent monitoring is required to help with decision making and resource allocation regarding the structure's rehabilitation and upgrades. Traditional methods involve sending inspectors to assess the asset conditions and file reports that estimate any required maintenance work. While visual inspections are widely used for preliminary and regular inspections because of its effectiveness in detecting external defects such as cracks and spalling which could be evaluated as symptoms of potential structural degradation, subjectivity and inconsistency are inevitable in visual inspections as different inspectors may evaluate the same structure differently [16]. Furthermore, some parts of the structure are not accessible for inspectors, making an overall assessment sometimes impractical [16]. An inspection site visit typically takes 4 h to be completed and for each hour spent in the field for inspection, additional three hours are spent in the office to generate the reports [17]. As an example of how time-consuming inspection is (using current methods) Ontario ministry of education has issued a bid in 2010 seeking a company to inspect a total of 4800 schools over a five-year period [18]. Errors and gaps produced by in manual assessments not only waste time and money, but can also lead to catastrophic events, such as the collapse of I-35 W highway bridge in Minneapolis, MN which caused 13 deaths and 145 injuries [19]. or the failure of Oroville dam spillways which took place in February 2017, forcing around 200,000 Californians to evacuate [20,21].

With image-based analysis being a potential key tool in enhancing the construction industry, this paper reviews previous works regarding the use of image-based analysis in the construction industry, following the research methodology in Fig. 1. It highlights current research trends and innovative works, challenges and limitations, and future research potential. The first step of the research technology (section 2) presents the approach used in collecting the journal papers to be analyzed. The second step (section 3) is presenting a general overview of the findings, including a thorough explanation of the most commonly used computer vision techniques. Finally, the papers were further analyzed according to their different applications to highlight notable achievements and major research gaps (sections 4, 5).

Section snippets

Survey methodology

An extensive literature review was conducted using the Scopus journal database as it automatically aggregates the search results from multiple academic databases, including: the American Society of Civil Engineers (ASCE) library, Elsevier (ScienceDirect), Wiley, IeeeXplore, and others. Only high-level keywords ((“Computer vision” OR “Image*” OR “Video*”) AND (“Construction”)) were used so that a wide search area can be covered. To highlight recent technical advances, the search was only limited

General survey results

Analyzing the 100 papers found in the construction literature related to image-based analysis, Fig. 2 illustrates an overview analysis of the main research areas and their applications. Main applications of image-based analysis in construction can be grouped into three main categories; (1) construction safety; (2) progress monitoring; and (3) damage assessment. A number of related review papers were also investigated (right-hand-side of Fig. 2).

Before undergoing further analysis, the number of

Applications of image-based analysis in construction

A perfectly automated inspection or monitoring framework consists of two main modules; automated data acquisition using UAVs or stationary cameras (e.g. CCTV), and automated decision making through data processing using image-based analysis techniques [8]. While the rapid growth in the drone industry has made the first module a reality, the analysis module and turning images into decision-support information remains a challenge. Towards this goal, the following three subsections review the

Summary and concluding remarks

This paper presents an overview of major image-based learning techniques utilized and their applications in the construction industry. Most applications are in the fields of site safety, progress monitoring, and damage assessment. The reviewed articles on image-based analysis applications in construction represent groundbreaking achievements in terms of using computer vision technology to enhance construction productivity and safety. To maximize the benefit from computer vision technology,

Data availability statement

All data generated or analyzed during the study are included in the published paper

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper

References (153)

  • J. Wu et al.

    Automatic detection of hardhats worn by construction personnel: a deep learning approach and benchmark dataset

    Autom. Constr.

    (2019)
  • R. Wei et al.

    Recognizing people’s identity in construction sites with computer vision: a spatial and temporal attention pooling network

    Adv. Eng. Inform.

    (2019)
  • Y. Yu et al.

    An automatic and non-invasive physical fatigue assessment method for construction workers

    Autom. Constr.

    (2019)
  • H. Son et al.

    Detection of construction workers under varying poses and changing background in image sequences via very deep residual networks

    Autom. Constr.

    (2019)
  • W. Fang et al.

    A deep learning-based approach for mitigating falls from height with computer vision: convolutional neural network

    Adv. Eng. Inform.

    (2019)
  • Z. Kolar et al.

    Transfer learning and deep convolutional neural networks for safety guardrail detection in 2D images

    Autom. Constr.

    (2018)
  • H. Zhang et al.

    Ergonomic posture recognition using 3D view-invariant features from single ordinary camera

    Autom. Constr.

    (2018)
  • Q. Fang et al.

    A deep learning-based method for detecting non-certified work on construction sites

    Adv. Eng. Inform.

    (2018)
  • W. Fang et al.

    Falls from heights: a computer vision-based approach for safety harness detection

    Autom. Constr.

    (2018)
  • J. Chen et al.

    Construction worker’s awkward posture recognition through supervised motion tensor decomposition

    Autom. Constr.

    (2017)
  • H. Luo et al.

    Full body pose estimation of construction equipment using computer vision and deep learning techniques

    Autom. Constr.

    (2020)
  • A. Braun et al.

    Combining inverse photogrammetry and BIM for automated labeling of construction site images for machine learning

    Autom. Constr.

    (2019)
  • J. Cai et al.

    Two-step long short-term memory method for identifying construction activities through positional and attentional cues

    Autom. Constr.

    (2019)
  • C.J. Liang et al.

    A vision-based marker-less pose estimation system for articulated construction robots

    Autom. Constr.

    (2019)
  • J. Kim et al.

    Action recognition of earthmoving excavators based on sequential pattern analysis of visual features and operation cycles

    Autom. Constr.

    (2019)
  • D. Roberts et al.

    End-to-end vision-based detection, tracking and activity analysis of earthmoving equipment filmed at ground level

    Autom. Constr.

    (2019)
  • Y.J. Lee et al.

    3D tracking of multiple onsite workers based on stereo vision

    Autom. Constr.

    (2019)
  • X. Luo et al.

    Vision-based detection and visualization of dynamic workspaces

    Autom. Constr.

    (2019)
  • C. Kropp et al.

    Interior construction state recognition with 4D BIM registered image sequences

    Autom. Constr.

    (2018)
  • W. Fang et al.

    Automated detection of workers and heavy equipment on construction sites: a convolutional neural network approach

    Adv. Eng. Inform.

    (2018)
  • X. Luo et al.

    Towards efficient and objective work sampling: recognizing workers’ activities in site surveillance videos with two-stream convolutional networks

    Autom. Constr.

    (2018)
  • B. Zhang et al.

    Automatic matching of construction onsite resources under camera views

    Autom. Constr.

    (2018)
  • H. Hamledari et al.

    Automated computer vision-based detection of components of under-construction indoor partitions

    Autom. Constr.

    (2017)
  • Z. Zhu et al.

    Integrated detection and tracking of workforce and equipment from construction jobsite videos

    Autom. Constr.

    (2017)
  • J. Yang et al.

    Vision-based action recognition of construction workers using dense trajectories

    Adv. Eng. Inform.

    (2016)
  • I.-H. Chen et al.

    Computer vision application programming for settlement monitoring in a drainage tunnel

    Autom. Constr.

    (2020)
  • Z. Liu et al.

    Computer vision-based concrete crack detection using U-net fully convolutional networks

    Autom. Constr.

    (2019)
  • C. Cabo et al.

    A hybrid SURF-DIC algorithm to estimate local displacements in structures using low-cost conventional cameras

    Eng. Fail. Anal.

    (2019)
  • S.I. Hassan et al.

    Underground sewer pipe condition assessment based on convolutional neural networks

    Autom. Constr.

    (2019)
  • C.V. Dung et al.

    Autonomous concrete crack detection using deep fully convolutional neural network

    Autom. Constr.

    (2019)
  • H. Busta

    KPMG Report: Construction Industry Slow to Adopt New Technology

  • J. Woetzel et al.

    The Construction Industry Has a Productivity Problem — And here's how to Solve it

  • F. Barbosa et al.

    Reinventing Construction: A Route to Higher Productivity

    (2017)
  • J. Manyika et al.

    Digital Globalization: The New Era of Global Flows

    (2016)
  • H. Deng et al.

    Automatic indoor construction process monitoring for tiles based on BIM and computer vision

    J. Constr. Eng. Manag.

    (2020)
  • B.F. Spencer et al.

    “Advances in Computer Vision-Based Civil Infrastructure Inspection and Monitoring.” Engineering, Chinese Academy of Engineering, 5(american socie)

    (2019)
  • US Bureau of Labor Statistics

    National Census of Fatal Occupational Injuries in 2018

  • Association of Workers'’ Compensation Boards of Canada

    National Work Injury, Disease and Fatality Statistics

  • Health and Safety Executive

    Construction statistics in Great Britain

  • M. Bevan et al.

    LCI National Webinar: preliminary findings from national project performance research

  • Cited by (71)

    • Exploring the adoption of technology against delays in construction projects

      2024, Engineering, Construction and Architectural Management
    • Brain-regulated learning for classifying on-site hazards with small datasets

      2024, Computer-Aided Civil and Infrastructure Engineering
    View all citing articles on Scopus
    View full text