Large-scale urban point cloud labeling and reconstruction
Introduction
Three dimensional (3D) modeling of urban buildings from point clouds has long been an active research topic in photogrammetry, GIS and remote sensing communities. Many methods have been developed from airborne laser scanning (ALS) point cloud-based building reconstruction (Lafarge and Mallet, 2012, Lin et al., 2013) to terrestrial laser scanning (TLS) or mobile laser scanning (MLS) point cloud-based street-side modeling (Frueh et al., 2005, Nan et al., 2010). Due to the large number of object categories and many overlapping or closely neighboring objects in large-scale urban scenes, automatic 3D urban reconstruction remains an open problem (Musialski et al., 2013).
In this paper, we propose a novel framework for ALS point cloud classification and building reconstruction. Fig. 1 illustrates the main process of labeling point clouds and modeling the segmented building points. A Rectified Linear Units Neural Network (ReLu-NN) is introduced to classify the point clouds. Before the training process, we compute the point-based feature descriptors and encode them by using the sparse autoencoder for obtaining discriminative features. The features are then taken as the input layer of the ReLu-NN. In the training process, we utilize the ReLu instead of the traditional sigmoid as the activation function of the network. The segmentation method based on the Euclidean distance (Rusu, 2009) divides building points from the classification results. Since the segmented building point clouds are often noisy, a structure-aware repairing method is introduced to consolidate them. Finally, lightweight buildings are reconstructed from the point cloud by using the contouring method (Zhou and Neumann, 2010).
The main contributions of our approach are threefold:
- (i)
An urban point cloud labeling and building modeling approach is presented. This approach provides a complete description of the process from the input of the ALS point clouds to the building reconstruction.
- (ii)
The ReLu-NN introduced can easily classify unorganized point clouds without rasterizing the data. Most of the parameters in the network are learned, and thus the intensive parameter tuning cost is significantly reduced. The training does not need a large number of training samples, and it achieves high labeling performances even when the scenes are complex.
- (iii)
The data consolidation and modeling algorithms generate a high generalization and compact representations of urban buildings with complex roof components.
Section snippets
Point cloud classification
Feature representation is a key step for point cloud classification. Much work focuses on point-based features and local neighbourhood features (Vosselman, 2013, Xu et al., 2014, Martinovic et al., 2015, Li et al., 2017a, Li et al., 2017b). Li et al. (2016) present a three-step point cloud parsing framework to formulate and solve a joint classification and segmentation problem from TLS point clouds. They first compute the feature per 3D point, then a linear SVM classifier is trained, and the
Construction of feature descriptors for each point
The architectures of tree crowns are generally complex while structures of building facades and roofs are smooth. In this paper, we use a method similar to our previous one (Li et al., 2017b) to obtain the features from different datasets.
For the TLS point cloud, the full descriptor per point pi is:
where is a 119-dimensional vector; RGBi is the mean RGB colors of pi as seen in the camera images; LABi is the LAB values
Building reconstruction
After the class building is determined, we use the distance-based segmentation approach (Rusu, 2009) to divide the building points from the point cloud. Reconstruction of 3D buildings from the segmented building point cloud is a challenge due to the poor-quality data, contamination with missing data, noise and outliers. To overcome this problem, we first consolidate the segmented point clouds, and enhance the quality of the data through removing the noise, outliers and recovering the regions
Experimental results
To validate the performance of our method, we carry out both qualitative and quantitative evaluations on various large-scale point clouds.
Discussions
The presented ReLu-NN still utilizes hand-crafted features for each modality independently and combine them in a heuristic manner. It often fails to consider the consistency and complementary information among features adequately. Currently, the features learned by the deep learning approaches can obtain high-quality image classification results. However, most of the state-of-the-art deep learning-based methods like Qi et al. (2017) and Liu et al. (2017) are hard to apply to parse large-scale
Conclusions
In this paper, we have presented a framework for urban scene labeling and building modeling from outdoor point clouds. The ReLu-NN is applied to parse each point in the point cloud into one semantic class. It can accurately label the points from the point clouds of complex urban environments. Experimental results show that the ReLu-NN parses the scenes efficiently and effectively, and it is capable of recognizing different class objects with similar shapes. Compared with other methods, our
References (43)
- et al.
Segmentation of terrestrial laser scanning data using geometry and image information
ISPRS .J Photogramm. Rem. Sens.
(2013) - et al.
3D terrestrial lidar data classification of complex natural scenes using a multi-scale dimensionality criterion: applications in geomorphology
ISPRS J. Photogramm. Rem. Sens.
(2012) - et al.
Fusion of airborne laserscanning point clouds and images for supervised and unsupervised scene classification
ISPRS J. Photogramm. Rem. Sens.
(2014) - et al.
3D building reconstruction from ALS data using unambiguous decomposition into elementary structures
ISPRS J. Photogramm. Rem. Sens.
(2016) - et al.
A hierarchical methodology for urban facade parsing from TLS point clouds
ISPRS J. Photogramm. Rem. Sens.
(2017) - et al.
Knowledge based reconstruction of building models from terrestrial laser scanning data
ISPRS J. Photogramm. Rem. Sens.
(2009) - et al.
Multiple-entity based classification of airborne laser scanning data in urban areas
ISPRS J. Photogramm. Rem. Sens.
(2014) - et al.
A methodology for automated segmentation and reconstruction of urban 3-D buildings from ALS point clouds
IEEE J. Sel. Top. Appl. Earth Obs. Rem. Sens.
(2014) - et al.
A comprehensive automated 3D approach for building extraction, reconstruction, and regularization from airborne laser scanning point clouds
Sensors
(2008) - et al.
Complete classification of raw LIDAR data and 3D reconstruction of buildings
Pattern Anal. Appl.
(2006)
Data processing algorithms for generating textured 3D building facade meshes from laser scans and camera images
Int. J. Comput. Vis.
Deep sparse rectifier neural networks
Aistats
Deep learning-based tree classification using mobile LiDAR data
Rem. Sens. Lett.
Digital photogrammetric camera evaluation–generation of digital elevation models
Photogrammetrie-Fernerkundung-Geoinformation
Edge-aware point set resampling
ACM Trans. Graph.
Consolidation of unorganized point clouds for surface reconstruction
ACM Trans. Graph.
Dense semantic 3D reconstruction
IEEE Trans. Pattern Anal. Mach. Intell.
Creating large-scale city models from 3D-point clouds: a robust approach with hybrid representation
Int. J. Comput. Vis.
Deep learning
Nature
Classification of urban point clouds: a robust supervised approach with automatically generating training data
IEEE J. Sel. Top. Appl. Earth Obs. Rem. Sens.
Cited by (76)
Conservation and management of Chinese classical royal garden heritages based on 3D digitalization - A case study of Jianxin courtyard in Jingyi garden in fragrant hills
2022, Journal of Cultural HeritageCitation Excerpt :of these cultural sites. [18,19]. The digital collection and archiving of Chinese traditional gardens has made progress; however, there are many important elements of artificial and natural surface features in Chinese royal gardens, amongst which there are many forms and complex classification objects, including water bodies, rocks, plants, buildings, topography, historical information, etc., which causes the scene to have diverse targets, complex morphological structures, target occlusion and overlapping, big spatial density differences, etc., resulting in common problems with the fine classification of 3D point clouds of this kind of cultural heritage [5,20-23]. Fine classification of 3D point clouds is the process of identifying and extracting artificial and natural surface features from disordered point clouds, and it is the basis of subsequent applications, such as digital ground model generation and 3D reconstruction of complex scenes [1,24,25].
DCPLD-Net: A diffusion coupled convolution neural network for real-time power transmission lines detection from UAV-Borne LiDAR data
2022, International Journal of Applied Earth Observation and GeoinformationCitation Excerpt :Recent methods, such as the PCCN (Zhang et al., 2019) and Superpoint Graph (Landrieu and Simonovsky, 2018) try to improve the global and local context integration by extending the convolution operator to high-dimensional lattice or graph to handle the unordered raw point clouds. Zhang et al.(2018) encode the point-based feature descriptors with a sparse autoencoder to obtain features and process those features using the proposed ReLu-NN for point cloud classification. Computing the features of non-object points will bring more computation costs and increase the difficulty of detection.
A new weakly supervised approach for ALS point cloud semantic segmentation
2022, ISPRS Journal of Photogrammetry and Remote SensingCitation Excerpt :The 3D coordinates and associated attributes (e.g., laser reflectance and return count information) are usually contained in ALS point clouds. To fully interpret a complex geographical scene, the key step is to acquire semantic information as a valuable cue utilized in various remote sensing applications, such as land cover surveys (Yan et al., 2015), forest monitoring (Yao et al., 2012; Polewski et al., 2015), change detection (Okyay et al., 2019), and 3D mapping (Fan et al., 2014; Zhang et al., 2018). Semantic segmentation, or classification, which usually assigns a label to each point, is an indispensable solution for point cloud parsing.
Data acquisition for urban building energy modeling: A review
2022, Building and EnvironmentCitation Excerpt :As LiDAR is efficient in object scanning and capable of providing sufficient point clouds, its data completeness is high, reaching 94%–100% [67]. In addition, its data accuracy is also good, which could be more than 85%, as advanced algorithms have been proposed to classify point clouds [64,67,109,110]. Similar to LiDAR, oblique photogrammetry data also have high completeness due to the sufficient point clouds provided.
Semantic Segmentation and Roof Reconstruction of Urban Buildings Based on LiDAR Point Clouds
2024, ISPRS International Journal of Geo-Information