RCC Structural Deformation and Damage Quantification Using Unmanned Aerial Vehicle Image Correlation Technique

Abstract

Reinforced cement concrete (RCC) is universally acknowledged as a low-cost, rigid, and high-strength construction material. Major structures like buildings, bridges, dams, etc., are made of RCC and subjected to repetitive loading during their service life for which structural performance deteriorates with time. Bridges and high-rise structures, being above ground level, are hard to equip with the contact mechanical methods to inspect strains and displacements for structural health monitoring (SHM). A non-contact, optical and computer vision based full field measuring technique called digital image correlation (DIC) technique was developed in the recent past to specifically evaluate bridge decks. Generally, optical images of structure in field conditions are not acquired precisely perpendicular to the object, which instinctively affects the deformation results obtained during loading conditions. An unmanned aerial vehicle (UAV) equipped with DIC vision-based technique acts as a rapid and cost-effective tool to quantify the serviceability of bridges by measuring strains and displacements at inaccessible locations. In this study, a non-contact unmanned aerial vehicle image correlation (UAVIC) technique is used on a scaled bridge girder and a contact method of measuring deformations with a dial gauge. Both investigations are correlated for accuracy assessment, and it is understood that results in laboratory conditions are 90% accurate. Similarly, the UAVIC technique is also performed on a rail over the bridge in the field conditions to understand the feasibility of the proposed method and evaluate damage quantification of it.

1. Introduction

Reinforced cement concrete (RCC) structures are designed for a specific serviceability life span. Many of the high-rise structures are made up of RCC. During its functionality, RCC structure tends to deteriorate on repetitive loading and other environmental conditions. Infrastructure development resembles a country’s overall growth, and as a part of the transportation infrastructure, bridges act as a major connectivity link in highways and railways. Bridges are expensive to construct and maintain, and it is also evident that various factors influence their serviceability [1]. Ninety percent of bridges are built in unavoidable conditions, either on water bodies or on rail tracks, and they are critical zones in a transportation network. There is a need for periodic monitoring of RCC structures for damage analysis and maintaining it for further commencement to service. As per the guidelines issued by the Federal Highway Administration inspection standards, bridges need to be inspected every 2–4 years based on the age of the structure [2]. Structures being at ground level are easy to inspect for their service life with the available wide range of non-destructive testing (NDT) techniques. Structures above ground level i.e., more than 5 m are hard to inspect for serviceability, as the height of the structure poses safety risks to inspectors and engineers [3]. Bridge performance evaluation to quantify service life based on deterioration at critical regions are challenging and are typically carried out with the assistance of snooper trucks and contact NDT methods. During situations of natural calamities, it is hard to check the damage to infrastructure through traditional methods like visual and in-contact NDT methods.

The first non-contact damage quantification for structure and property is performed by using high-resolution satellite data sets. The usage of in-contact sensors for vibration observations has given a great advantage to real-time monitoring of the structure connected to data acquisition platforms. These in-contact sensors are highly sensitive to handle and also vulnerable in external loading and harsh environmental conditions [4]. These sensors are highly capable of identifying defects at their installed locations, but they have demonstrated ineffectiveness in monitoring the entire structure at the same time [5]. Infrastructure failures are mostly associated with improper identification and maintenance of the damage zone in a structure [6]. Visual inspection, assessment and documentation of repairs are the most common methodology followed by inspectors and engineers [7]. Although it is a quite simple and common process, the machinery involved in inspection makes it more expensive, and the occurrence of accidents and manual errors are high. In addition, human vision inspection can perceive limited observations at a time compared to computer vision techniques [3]. Photogrammetry concepts clubbed with computer vision are developed by researchers to identify the critical zones by analyzing the digital image for crack properties, spalling and damage in a structure.

With the advent of digital image correlation (DIC) in the recent past, many civil engineering applications like structural health monitoring (SHM) are simplified [8]. The DIC vision technique is a remote sensing and photogrammetric method that can perceive the deformation of an object by analyzing the prior and post-loading digital image [9]. The availability of numerous and diversified image processing algorithms has facilitated the researcher to choose according to the requirement of an application [10]. Algorithms, such as pattern recognition, have made locating the cracks in a structure and monitoring easy [11,12]. The DIC technique is a rapid, reliable, cost-effective and non-contact NDT method. DIC methods have paved the way for new cost-effective procedures for SHM in major areas, such as bridge monitoring and vibrations in wind turbines [13,14]. It has the capability to visually document a vast area in a stipulated time without missing any fine details of the structure.

DIC is versatile in processing 2D and 3D datasets for obtaining displacements and deformation measurements in a structure [15]. The data acquisition platforms, such as digital cameras and terrestrial laser scanners are fixed on tripods that remain immovable facing perpendicular (90 ± 7) to the object in a particular location [10]. The data obtained from different angles of the target object produces oblique images that abruptly end up yielding inaccurate displacement results [16,17]. In spite of its impressive accuracy and unique capabilities for investigating the structural strains under dynamic phenomena, DIC is limited to only a few applications in SHM due to a fixed platform [18]. The immovable platform and acquisition of near perpendicular images are not permitting the DIC to be performed on inaccessible locations of a bridge [19]. In the process of obtaining near perpendicular images, inspectors use snooper trucks or other elevated platforms that are aligned to the structural members which also require extra effort, time and money. There have been a few ongoing research on DIC deformation evaluation and comparison studies on structures using UAV, but they are limited to laboratory conditions. The practical applicability of drone DIC in structural health monitoring applications is still a critical topic of research [20,21].

2. Literature Review

There are various remote sensing (RS) structural inspection techniques that can be used in monitoring and damage assessment of a bridge based on the level of importance and accuracy. The majority of remote sensing inspections are done through imaging sensors; different kinds of imaging sensors can perceive different kinds of information. Red Green Blue (RGB) or visible spectrum imaging sensor is employed for damage quantification [22]. Similarly, multispectral imaging sensors for damage assessment [23], hyperspectral imaging sensors for concrete properties [24], light detection and ranging (LiDAR) sensors or laser scanning for determining the crack depth [25] and near infrared (NIR) or thermal imaging sensors for identifying the moisture ingress and delamination of bridge deck [26]. These techniques are classified depending upon the type of data acquisition platform at varying altitudes and imaging sensors employed.

The level of accuracy relies on the operational altitude of sensors and the resolution of imaging data. The data acquisition platforms as shown in Figure 1 are categorized based on high accuracy and low accuracy. Inspectors and engineers chose the data acquisition platform based on the type of monitoring, budget allocated and time span.

(i) High accuracy:

(a)

Terrestrial Cameras: terrestrial cameras are mounted on a tripod and they have the capability to continuously acquire the data for longer periods. Different sensors, such as visible range (RGB), laser, thermal and multispectral can be mounted on the same data acquisition platform. Terrestrial cameras and laser scanners to produce 3D models through point clouds can be operated even in varied climatic conditions.

(b)

Robot: Robotics can be operated directly on top of the bridge deck to produce high-resolution data sets using sensors, such as RGB, laser, ground penetrating radar (GPR) and multispectral. GPR has a unique capability in exploring the internal core condition of the RCC bridge deck to inspect the further serviceability of the structure.

(c)

Unmanned Aerial Vehicle (UAV): UAV/drone operational altitude is low and restricted to height of 400 ft (120 m) as per the guidelines issued by the Director General of Civil Aviation (DGCA), so it can be operated very near to the target object. UAV can acquire high resolution digital, multispectral and thermal imaging datasets along with laser point cloud data to inspect the bridge.

(ii) Low accuracy:

(a)

Unmanned/Manned aircraft: Aircraft flying range is above 1 km to 10 km and flies at supersonic speeds. The data acquired at specified speed and altitude generates coarse resolution images of the structure that can actually be used for preparing a rough estimate of a damaged structure during natural calamity. Only high-resolution visible range RGB and Multispectral imaging data acquired through aircraft is considered for damage assessment of a bridge.

(b)

Satellite: Satellites are operated at higher altitudes of 10–1000 km, and the revisit time on a particular location is more than 5 days, which may not be available at desired dates. Data sets obtained are of coarser resolution and may contain cloud cover that makes the bridge monitoring a hard task [27].

All remote sensing acquisition systems are well designed to assess the condition above the bridge deck. Although there are certain advantages and disadvantages to each data acquisition system, UAV monitoring systems stand out as they can be operated even below the bridge deck and also at inaccessible locations.

Objective

There is a need for a rapid, risk-free and cost-effective method to monitor and document the damage of a bridge by ensuring minimal impacts on traffic flow. Development of mobile DIC that can hover through inaccessible locations and obtain the near perpendicular images for investigating the structural health. DIC mounted on a UAV can cost-effectively solve the issue of obtaining near perpendicular image structure. The main objective of this study is to evaluate DIC studies using UAV in laboratory conditions and real-time field conditions on an active service bridge. The remote sensing setup would also facilitate the inspectors to obtain high resolution spatial and temporal images remotely to inspect and actively alarm the engineers.

3. Methodology

The methodology is divided into two phases, such as a feasibility study of UAVIC inside the laboratory and field implementation on a bridge. Initially, the lab studies included three RCC beam specimens, their deformation studies upon loading with a standalone digital camera setup, and a UAV mounted camera setup. The obtained results are compared, and deflection accuracy assessment studies are conducted. Second, the entire proposed UAVIC configuration is tested on an existing serviceable bridge, and deformation studies are performed.

3.1. UAVIC Studies in Laboratory Conditions

The above Figure 2 depicts the flow chart of the methodology followed to perform the study.

An experimental setup along image acquisition platforms, such as digital camera and UAV procedures is described here. RCC beams labelled and speckled are tested with increasing displacement levels that correspond to a load setup. The methodology discusses image processing methods and dial gauge setup for deformation and crack identification.

MATLAB-based open source Ncorr V1.2 software for DIC analysis is used in MATLAB for measuring displacements and strains.

3.1.1. Specimen Preparation

Three concrete specimens of M25 grade strength is casted for conducting the study. Three specimens are prepared by spraying white paint as a background for the application of random speckles with black paint on the surface at the centre portion of the beam as shown in Figure 3. To achieve accurate results, it is important to have a considerable quantity of black speckles with random shapes and sizes [28].

3.1.2. Loading Test

A three-point bending test conducted on specimens to measure displacements and strains. Three-point bending means placing the concrete specimen across the support span on either end of the material and bringing down a point load to the centre of the span and bending the material until failure while recording applied force and displacements. To relate the deformation to the corresponding load, it is necessary to record the readings and their labelling. Each load increment of 5 kN, the deflection is noted on dial gauge and images of digital camera and drone camera are also obtained at the same time.

3.1.3. Image Acquisition

A series of digital images are acquired during load implications at different load levels to process in DIC [29]. Image acquisition during loading condition is done simultaneously with DSLR camera and drone camera as shown in Figure 4. Dial gauge is also placed prior to loading at the bottom centre of the beam to measure the deformation that occurred due to load implication. Similarly, images for the rest of the beams are also acquired at specific load levels. The camera images are acquired using a Nikon D5600 (Nikon, Tokyo, Japan) bearing a CMOS sensor of 24.2 MP with a focal length of 18 mm and 1920 × 1080 resolution, and it has 60 fps video recording capabilities. The drone image acquisition is achieved by employing DJI Phantom4 Pro V2.0 (DJI, Shenzhen, China) that is equipped with CMOS 20MP sensor with a focal length of 8.8 mm and resolution of 5472 × 3648 pixels, and it has 60 fps video recording capabilities. The UAV is a quadcopter with a designed flight time of 30 min and a maximum weight of 1375 g. It is a relatively cheaper and commonly used UAV in many engineering applications, due to its onboard sensor capabilities. Both the cameras are set at 3 m away from the target beam for imaging.

3.1.4. Image Correlation Using Ncorr via MATLAB

It is a crucial stage at which displacement and strain fields are studied and determined from the image correlation algorithms. In this study MATLAB, version R2018a (MathWorks, MA, USA) and Ncorr software version 1.2.2 (Georgia Institute of Technology, Atlanta, GA, USA) is used. The work flow in Ncorr software is described below. The entire set of digital camera images and drone camera images for each specimen need to be processed in the software. Thirty digital camera images and 30 drone camera images of each specimen are processed separately for deflection investigations. Altogether, 270 images are processed.

3.1.5. Setting Images

The reference image is the initial one taken before application of load as shown in Figure 5a, then the current image (deformed image) is uploaded as shown in Figure 5b. All the images should be in the same format as the reference image to minimize initial processing. Region of interest drawn on the surface for which displacements and strains are to be determined and minimizes computational time. The same image processing tools and algorithms were used for processing the images obtained from UAV as shown in Figure 6 prior to loading and after loading.

3.1.6. Setting of DIC Parameters

DIC parameters need to be set up on ROI for displacement measurements in the Ncoor V1.2 image processing software. Subset radius is chosen to cover the portion of speckle pattern which should be small and at least covers more than 3 speckles that does not result in noisy displacement data as shown in Figure 7. Subset spacing is chosen in such a way that it should increase the computational speed. Multithreading is done for speeding up the computation and accurate results. In this study, an optimal subset size of 80, subset spacing of 15 and 4 number of threads are set as DIC parameters. Subset size ranges between 10 and 200. Subset spacing ranges between 0 and 20 are adopted.

3.1.7. DIC Analysis

To perform image analysis a contiguous region is selected for processing and seeds are placed in Ncoor V1.2 software. The seed placement process gives initial assessment for DIC analysis from which neighboring pixels are computed for correlation analysis in the series of digital images. DIC analysis for the strain computations are also performed based on the displacements occurring on the beam. The selection of strain radius is similar to that of subset radius, which is desired to be the smallest that does not result in noisy strain data. In this study, a strain radius of 10 is selected as shown in Figure 8. The strain anomaly is given on the beam facade based on the load distribution.

3.1.8. Crack Identification and Feature Extraction

Reinforced cement concrete upon loading exhibits minimum elastic properties and tends to bend a certain limit. Beyond the elastic limit, RCC exhibits cracks upon further excessive loading of the beam. Micro cracks may be caused during shrinkage, creep and heat of hydration. The cause of cracks in concrete structures are of three basic types, flexural, shear and flexural-shear [30]. These cracks produced during and after construction may cause failure to the structure. Cracks will have a much longer major axis (length) compared to the minor axis (width), resulting in a ratio greater than 1, while a ratio closer to 1 represents surface defects. Properties of selected cracks are investigated by a set of image processing tools available in MATLAB with a designated alphabet for each crack. The methodology of crack identification and parameter estimation is given in Figure 9. The acquired 2D images are converted to grey scale images and edge detection technique is performed for crack identification. The identified cracks on the beam are quantified for crack width and length by performing segmentation and morphological operations on the grey scale images. The result for crack width and length of laboratory tested beam is given in Section 4.4; similarly, bridge crack investigations are given in Section 4.6.

3.2. UAVIC Evaluation Studies on Bridge

Bridges are the most important RCC structures in the transportation system for connecting inaccessible locations. Due to its elevation, monitoring the bridges also became a herculean task in terms of safety and traffic mobility. The proposed and tested UAVIC is deployed to inspect the fully functional RCC Bridge available in the vicinity of the institution. The UAVIC study workflow on the bridge is given in Figure 10. The selected structure is rail over bridge (ROB) aged 21 years, with a length of 540 m and height 7 m and heavy weight freight on it. An automated process based on digital imaging and processing was used to determine structural damages of spalled concrete, crack initiation and propagation, crack width measurement and monitoring. Digital images of ROB are acquired by flying UAV on the center span and away from railway power lines. The span of the bridge deck is 11 m in length from center to center of support and 7 m in height. Images are obtained in the broad daylight under sunny conditions for detailed information of the bridge. There are two sets of reflective reference points attached on the supports of the bridge deck.

3.2.1. Bridge Image Acquisition

A triangular scale at the center of the bridge deck is attached for reference length to perform DIC studies on the bridge. The irregularity and other dust particles on the surface of the bridge beam acted as a speckle pattern for conducting DIC studies. UAV images facing the center of the bridge deck prior to vehicle passage as shown in Figure 11a and after vehicle passage are acquired as shown in Figure 11b. Similarly, different kinds of vehicles with varying loading condition images are acquired during the passage at a specified point.

3.2.2. UAVIC Studies

The pre-processing is done for every UAV acquired bridge image by referencing with fixed points and ortho-rectifying them. These images are stacked on one another exactly at referencing points and post–processed for DIC studies. The images are processed for deflection and deformation investigations of the bridge deck. The vehicle during the passage generates an impact load on the deck slab. Images of prior and during the vehicle loading are given to Ncorr V1.2 software, and the above-mentioned processing is done for all the images with different loading conditions. The strains and displacements are recorded from the investigations, and results are presented.

3.2.3. Damage Quantification

Damage analysis of the beam done by investigating the present structural condition of the bridge components. The entire stretch of the bridge is investigated for occurrence of cracks by flying the UAV. Crucial aspects, such as column and beam joints, beneath the deck slab, sides of the slab supporting beams and growth of organic matter are inspected. The damage structural images are analyzed for crack width and length investigations, and damage is ascertained by the results obtained. Few damaged portions of the bridge are shown in Figure 12 in alphabetical order and described: (a) crack in the joint and damage of beam, (b) crack in the deck of the bridge, (c) damage and crack in the pier, (d) spalling of beam and (e) crack in the beam and slab joint.

Excessive loading and poor maintenance of bridge structures causes cracking, which affects the structure’s performance [31]. According to Sheerin [32], longitudinal and ver-tical cracks, which are commonly found in bridges, are classified as moderate or severe. The majority of the cracks detected in the study also fall into this category. In RCC constructions, fracture widths greater than 0.3mm cause rebar corrosion, expand the crack and cause structural damage. The maximum crack width allowed under service loads, as defined by several standard codes, is 0.3mm; any crack width greater than that should be treated [33]. The damage index is calculated based on the loading situation, crack width, and crack length [34,35]. The deterioration of bridge in terms of spalling and chemical attack is also considered for damage analysis [36,37].

4. Results and Discussions

4.1. Displacements Investigations

Results from RCC beam specimens tested for two-point bending have been evaluated. The image analysis performed on the DSLR images and UAV images of a single beam in the Ncorr V1.2 software is yielded with the below outputs. The maximum load has been recorded as 112 kN at which the failure has occurred. After the failure load, the displacement increases rapidly and the load decreases. Figure 13a depicts displacement at failure load in the longitudinal direction (U-Displacement) is 5.3640 mm, which indicates compression in the deformed image. The median value was found to be −0.0599 mm. The following Figure 13b depicts displacement at failure load in the lateral direction (V-Displacement) is −3.563 mm which is negative indicates tension. The median value has found to be −6.4953 mm. The results of the given beam are of a single deformed image, and each image yields different displacements according to the varying load values.

The following Figure 14a depicts displacement at failure load in the longitudinal direction (U-Displacement) is 5.8320, which indicates compression in the deformed image. The median value was found to be −0.0630 mm. The following Figure 14b depicts displacement at failure load in the lateral direction (V-Displacement) is −4.113 mm which is negative indicates tension. The median value has found to be −6.992 mm.

4.2. Strain Investigations

The DIC analysis also helps in determining the strains occurred with the beam. There are more strains in the longitudinal directions compared to the lateral directions. Figure 15a depicts strain in X-direction is 0.0126 mm. The median value has found to be 0.0025 mm. Figure 15b depicts strain in Y-direction is 0.1321 mm. The median value was found to be 0.0020 mm.

4.3. Load-Displacement Plots

Displacements obtained from DIC analysis are compared with the experimentally recorded dial gauge values at varying load values. Displacement values of DSLR images are obtained separately from UAV images and plotted. Small variation in the investigations is observed in dial gauge to DSLR displacements, with an accuracy of 95%, whereas the variations are higher in investigations observed form dial gauge to UAV displacements, with an accuracy of 88%. A sudden spike is observed in UAVIC results at some intervals due to occurrence of vibrations and magnetic attractions in the drone. Figure 16a displays a load-displacement graph of beam-A with failure load as 88 kN and maximum displacement as 4.82 mm.

Similarly, beam-2 graph as shown in Figure 16b indicated failure load as 109 kN and maximum displacement as 5.35 mm, and beam-3 as shown in Figure 16c signifies failure load as 116 kN and maximum displacement as 6.45 mm, respectively. A good result has been observed between the two studies, with a deviation of 12% of experimental values.

4.4. Crack Detection and Parametric Analysis on the Beam

The cracks formed during the failure load is investigated for its parameters, such as crack length and width. Each crack in the beam is analyzed and designated with a nomenclature. The major crack on the beam is investigated with image analysis, and results are presented in

Table 1. Length and width of crack in beam.

Beam	Length in mm	Width in mm
A	153	0.937
B	214	1.282
C	178	1.147

.

4.5. UAVIC Investigation on Bridge

The proposed method is validated in the laboratory conditions and investigated in the field conditions under controlled climate. Image analysis of different vehicle-loading conditions is performed in the DIC Ncorr software. The investigated displacement records are presented in

Table 2. Vehicle load type and displacement values from UAVIC.

Vehicle Passage	1	2	3	4	5	6	7
Deformation (mm)	2.82	1.47	3.22	3.18	1.30	2.63	0.89

.

Figure 17a depicts displacement at failure load in the longitudinal direction (U-Displacement) is 2.82 mm, which indicates compression in the deformed image. The median value was found to be −0.0519 mm. Figure 17b depicts that displacement at failure load in the lateral direction (V-Displacement) is −4.1928 mm, which indicates tension. The median value was found to be −3.2359 mm.

4.6. Crack Detection and Parametric Analysis on the ROB

The image analysis similar to beam crack properties investigations is carried out on the bridge element images acquired by UAV. A few damaged portions were put on to the show in the UAVIC studies in the previous section of damage quantification. The designated alphabetical order of damaged portions is investigated for crack properties and given in

Table 3. UAVIC captured bridge elements crack properties.

Bridge Component	Crack Width in mm	Crack Length in mm
(A) Crack in the joint and damage of beam	1.82	583
(B) Crack in the deck of the bridge	2.42	467
(C) Damage and crack in the pier	0.96	328
(E) Crack in the beam and slab joint	3.28	524

.

5. Conclusions

From this study, it is understood that RCC beams undergo ductile failure with lateral displacements as the steel material has high elasticity of modulus that increases rigidity and moment of inertia so that it can withstand heavy loads and undergo larger deflections before failure. The maximum surface crack width evaluated from DIC analysis is found to be 0.147 mm for RCC beam. The load displacement curve declares that dial gauge and digital camera DIC and UAVIC investigations results are similar, with a variation of 12%. UAVIC investigation shows small spikes in the displacement recordings, it is due to magnetic sensitivity of sensors and vibrations in the UAV. While conducting the study, authors observed a small drift in the UAV due the availability of heavy machinery like Universal Testing Machine (UTM) attractions in the laboratory conditions. In field conditions, while monitoring the bridge, the wind pressure caused the drift in the drone, gimbal played a major role in locking the view of target point. The UAVIC has proved to be a preliminary assessment tool at inaccessible locations in RCC structural monitoring. Based on the deterioration studies it is understood that the crack width in the bridge exceeded the permissible limits, and it should be treated with fibers [38,39]. Usage of self-healing concrete in the wide opened cracks also prevents future deterioration [36]. In terms of concrete spalling, the severity level is categorized as 1, and near maintenance is required [40]. The investigated bridge needs immediate and periodic maintenance for enhancing the serviceability of the structure. For obtaining the better results, bridge monitoring studies can only be conducted on highly sunny days with low windy conditions in the field. Rather than a quadcopter, an octocopter would yield better results for its better stability and improved hovering time at in-situ conditions. The dust on the bridge beam acted as a speckle pattern; in exceptional condition speckles should be drawn on the beam to conduct the study. As a future scope of study, to stabilize the UAVIC images a laser referencing system by placing the lasers at four corners of the target would yield high-precision results.