Automated assessment of wind damage to windows of buildings at a city scale based on oblique photography, deep learning and CFD
Introduction
Damage to non-structural components of buildings by strong wind storms not only causes direct economic losses but also has adverse impacts on building functions, which leads to indirect economic losses (i.e., the economic losses caused by the business interruption due to repair works), especially in wind-prone cities [1]. Windows are one of the non-structural components most vulnerable to wind damage in buildings. If wind-induced window damage in buildings could be predicted accurately and rapidly at the city scale, buildings with a high risk of window damage could be identified and strengthened prior to strong wind events, thereby reducing direct and indirect economic losses.
Most previous studies have been limited to the simulation of wind-induced window damage in virtual building clusters [2,3]. These studies assumed the building configuration and window distribution, and then compared the wind loads with the resistance of the windows to perform damage simulations for virtual buildings and analyze the window damage patterns. In contrast, few studies have proposed efficient simulation methods for wind damage to windows in a real built environment, as such simulations face the challenge of obtaining window information with high efficiency. Specifically, to calculate or model the wind load on windows, the semantic information (e.g., location and size) of each window in the real built environment must be obtained. However, real urban areas contain a large number of buildings with various windows, making it difficult to use manual or experience-based methods to expeditiously and accurately collect the semantic information for all of the windows. This study aims to address this problem.
In recent years, the rapid development of unmanned aerial vehicle (UAV) technology has provided new ways to obtain urban information [4,5]. Because oblique aerial images taken by UAVs contain abundant information about urban buildings, methods for establishing 3-D models of urban buildings based on oblique photography technology have been studied by numerous researchers. For example, Yalcin & Selcuk utilized oblique photography to construct highly realistic 3-D building cluster models with texture information [6]. Li et al. proposed a texture-reconstruction method based on oblique photography to improve the local texture precision of buildings [7]. Xu et al. mapped regional earthquake simulation results to a 3-D urban model established through oblique photography to display the dynamic responses of buildings in earthquakes with high fidelity [8]. Although existing methods can construct highly realistic models of building clusters with texture information, the windows in these models are presented in the form of maps and do not contain the semantic information required for wind damage assessment. Hence, it is necessary to combine oblique photography with an image semantic segmentation algorithm to collect window semantic information efficiently for real building clusters.
Image semantic segmentation refers to technology that classifies pixels in an image and divides the pixels into suitable categories (e.g., background, people, or cars) [[9], [10], [11]]. For example, the semantic segmentation network proposed by Ronneberger et al., U-Net, has achieved good results in tasks in the medical field [9]. Badrinarayanan et al. proposed a deep convolutional semantic segmentation network called Segnet, which can be used in fields such as automatic driving and indoor navigation [11]. Martinović et al. proposed a three-layered approach to conduct semantic segmentation of building facades [12]. Affara et al. extracted the image features of building facades based on the particle filtering method and classified the extracted features using a support vector machine [13]. However, the methods proposed in the above studies can only be used to obtain the pixel coordinates (pixel position in the image) of windows, rather than the world coordinates (position in the real world) that can be directly used in window damage assessments. In this study, an automated method that combines oblique photography with image semantic segmentation technology is introduced to obtain the world coordinates of windows for buildings at a city scale.
Additionally, empirical wind pressure values given by design codes or wind-tunnel-test-based databases have been employed in previous studies to perform window damage assessments [2,3,14]. Such methods cannot rationally consider the influence of building configurations or the interactions between buildings. In contrast, recent developments in computational fluid dynamics (CFD) [15,16] may provide a tool for remedying these defects.
To address these challenges, this study proposes a high-efficiency automated simulation method for the wind damage to building windows at the city scale using oblique photography, deep learning, and CFD technologies. The framework of the proposed assessment method is introduced in Section 2. The details of the underlying methodologies used in the framework are presented in Sections 3 Oblique-photography-based building façade image segmentation, 4 Deep-learning-based window recognition, 5 CFD-based assessment of window damage. Finally, a real-world community in Shenzhen, China is considered as a case study in Section 6 to showcase the workflow and demonstrate the reliability and feasibility of the proposed method. Conclusions are presented in Section 7.
Section snippets
Framework
The framework of the proposed assessment method is presented in Fig. 1 and consists of three modules.
Oblique-photography-based building façade image segmentation
Three steps are required to complete the oblique-photography-based façade image segmentation: (1) matching aerial images to building façades, (2) coordinate transformation, and (3) extracting and rectifying the building façade images. The details of these steps are described below.
Model architecture
The Pix2Pix model [17] is a typical deep learning network based on conditional generative adversarial networks [23] that consists of two parts: a generator and a discriminator. In this study, the generator generates the target image according to the input building façade image, while the discriminator determines the authenticity of the generated image, i.e., distinguishes whether the image is generated by the generator or manually annotated.
The generator architecture of the Pix2Pix model is
CFD-based assessment of window damage
The 3-D CFD model for the target building cluster can be established easily through automated methods based on the GIS data [27,28]. It should be noted that, on one hand, the wind speed and direction in an urban area will change continuously over time during a typhoon; on the other hand, the damage to a window in a specific building will affect the internal pressure acting on the remaining windows of the building. Consequently, the window damage of building clusters is sensitive to the
Study area
A case study of a community (Fig. 13) in northwest Shenzhen, China, was performed to illustrate the implementation of the proposed evaluation framework. Shenzhen is frequently subject to typhoons, given its location along the southeast coast of China [31]. There are 34 buildings in the considered community, and the tallest has a height of 28 m. A fixed-wing UAV equipped with five cameras was used to capture images of the community and construct the oblique photography database. The longitude,
Conclusions
An automated method for evaluating the wind damage to building windows at the city scale was proposed in this study based on oblique photography, deep learning, and computational fluid dynamics technologies. As a case study, the window damage as well as the direct economic losses and repair times were simulated using the proposed method for a community in Shenzhen, China, under Typhoon Hato with different Beaufort scales. The main conclusions are as follows:
- (1)
Using the proposed building façade
CRediT author statement
Donglian Gu: Methodology, Data curation, Formal analysis, Validation, Investigation, Writing- Original draft, Writing- Review & editing. Wang Chen: Methodology, Data curation, Formal analysis, Validation, Writing- Original draft. Xinzheng Lu: Conceptualization, Methodology, Writing- Review & editing, Supervision, Funding acquisition.
Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
Funding: This work was supported by the National Natural Science Foundation of China (No. 52121005); and the Tencent Foundation through the Xplorer Prize. The authors also appreciate Beijing PARATERA Tech Co., Ltd. for providing the computational hardware and software that were used in this work.
References (35)
- et al.
Reliability-based modeling of typhoon induced wind vulnerability for residential buildings in Japan
J. Wind Eng. Ind. Aerod.
(2014) - et al.
3D city modelling with oblique photogrammetry method
Procedia Technol.
(2015) - et al.
Photo-realistic visualization of seismic dynamic responses of urban building clusters based on oblique aerial photography
Adv. Eng. Inf.
(2020) - et al.
CFD simulation for pedestrian wind comfort and wind safety in urban areas: general decision framework and case study for the Eindhoven University campus
Environ. Model. Software
(2012) - et al.
Wind comfort assessment by means of large eddy simulation with lattice Boltzmann method in full scale city area
Build. Environ.
(2018) - et al.
Automated structural design of shear wall residential buildings using generative adversarial networks
Autom. ConStruct.
(2021) - et al.
Automated regional seismic damage assessment of buildings using an unmanned aerial vehicle and a convolutional neural network
Autom. ConStruct.
(2020) Status of land cover classification accuracy assessment
Remote Sens. Environ.
(2002)- et al.
The JCSS probabilistic model code
Struct. Saf.
(1997) - et al.
Observation of Typhoon Hato based on the 356-m high meteorological gradient tower at Shenzhen
J. Wind Eng. Ind. Aerod.
(2020)
AIJ guidelines for practical applications of CFD to pedestrian wind environment around buildings
J. Wind Eng. Ind. Aerod.
Wind-induced damage to buildings and disaster risk reduction
Research on the loss of group residential buildings under fierce winds
Nat. Hazards
UAV for 3D mapping applications: a review
Appl. Geomat.
High resolution imagery collection utilizing unmanned aerial vehicles (UAVs) for post-disaster studies
A novel OpenMVS-based texture reconstruction method based on the fully automatic plane segmentation for 3D mesh models
Rem. Sens.
U-net: convolutional networks for biomedical image segmentation
Cited by (10)
Automated design of cable-net structures with multi-objective state
2024, Automation in ConstructionPrediction on the seismic performance limits of reinforced concrete columns based on machine learning method
2024, Soil Dynamics and Earthquake EngineeringCIM-powered physics-based assessment of wind damages to building clusters considering trees
2023, Developments in the Built EnvironmentAutomated UAV image-to-BIM registration for building façade inspection using improved generalised Hough transform
2023, Automation in ConstructionA computational framework for the simulation of wind effects on buildings in a cityscape
2023, Journal of Wind Engineering and Industrial Aerodynamics