Line segment extraction for large scale unorganized point clouds

doi:10.1016/j.isprsjprs.2014.12.027

ISPRS Journal of Photogrammetry and Remote Sensing

Volume 102, April 2015, Pages 172-183

https://doi.org/10.1016/j.isprsjprs.2014.12.027 Get rights and content

Abstract

Line segment detection in images is already a well-investigated topic, although it has received considerably less attention in 3D point clouds. Benefiting from current LiDAR devices, large-scale point clouds are becoming increasingly common. Most human-made objects have flat surfaces. Line segments that occur where pairs of planes intersect give important information regarding the geometric content of point clouds, which is especially useful for automatic building reconstruction and segmentation. This paper proposes a novel method that is capable of accurately extracting plane intersection line segments from large-scale raw scan points. The 3D line-support region, namely, a point set near a straight linear structure, is extracted simultaneously. The 3D line-support region is fitted by our Line-Segment-Half-Planes (LSHP) structure, which provides a geometric constraint for a line segment, making the line segment more reliable and accurate. We demonstrate our method on the point clouds of large-scale, complex, real-world scenes acquired by LiDAR devices. We also demonstrate the application of 3D line-support regions and their LSHP structures on urban scene abstraction.

Introduction

Benefiting from the advances in sensor technology for both airborne and ground-based mobile laser scanning, dense points clouds have become increasingly common, and the need for new approaches to address these point clouds has become increasingly important. As the common feature in man-made objects, straight linear structures play an important role in a variety of applications, such as: road extraction (Yang et al., 2013); building outline extraction (Baillard et al., 1999); localization (Borges et al., 2010), city model building (Lafarge and Mallet, 2012); calibration (Moghadam et al., 2013); line-based visualization (Chen and Wang, 2011); and more. This paper emphasizes straight line segment extraction for point clouds, whereas most of the existing work concentrates on 2D line segment detection in a single image (Ballard, 1981, Burns et al., 1986, Von Gioi et al., 2010) and 3D line segment reconstruction in multi-view images (Baillard et al., 1999, Woo et al., 2009, Jain et al., 2010). Only a few papers consider point clouds (Lu et al., 2008, Moghadam et al., 2013).

A large number of dense point clouds have been obtained by current scanners; the RIEGL VMX-450 scanner, for example, can yield 1.1 million range measurements per second. Therefore, one of the biggest challenges is to find an efficient way to address the voluminous data. Unorganized point clouds lack normal vector and connectivity information, making the problem even more challenging.

Our method is designed to cope with line segment extraction for large-scale unorganized point clouds from the real word. A line segment here is defined as the intersection of two half-planes. To extract the line segment, we take into account the point region that is near the straight linear structure. Such a region is designated as a “3D line-support region.” The word “3D” is used to distinguish the region from the concept of a “line-support region,” which has proved to be a robust descriptor to extract line segments in images.

The key idea of our method is to first convert a point cloud into a collection of shaded images by non-photorealistic rendering with different viewpoints; then the LSD algorithm (Von Gioi et al., 2010) is applied to these images to extract the 2D line-support regions. These 2D line-support regions are then back-projected into the original point cloud as 3D line-support regions, with each region containing roughly one line segment. Next, to maintain accuracy, each 3D line-support region is fitted by our Line-Segment-Half-Planes (LSHP) structure. Finally, the 3D line-support regions and their LSHP structures are refined as the output.

Fig. 1 presents a result of our method. Given an unorganized 3D raw scan point cloud as the input (Fig. 1(a)), our method extracts the 3D line-support region and LSHP structures as the output, where the line segments are drawn in black and the attached half-planes are represented by colored 3D rectangles (Fig. 1(b)). As a result, the LSHP structure provides an abstraction of a point cloud, and the vegetation in the input is filtered.

Section snippets

2D line segment detection for a single image

Image line segment detection has been studied over several decades. The traditional methods combine the Canny edge detector (Canny, 1986) and Hough transform (Ballard, 1981). These methods are generally slow and produce a significant number of false detections. Recently, an efficient line segment detector with false detection control (designated LSD) was presented by Von Gioi et al. (2010). LSD follows the method proposed by Burns et al. (1986). First, the image is partitioned into a collection

Overview

This section first introduces the concepts of 3D line-support regions and Line-Segment-Half-Planes (LSHP) structures, and then provides an overview of our approach.

3D line-support region extraction

2D line-support region extraction in images has already been well investigated. The traditional methods group the edge pixels with similar gradient directions into line segments. However, although there are several methods that can extract sharp feature points in 3D space, there is not, to our best knowledge, a method to group these feature points into line segments, especially for scenes containing vegetation and other non-manifold objects.

To cope with this problem, we take the 2D line-support

LSHP modeling

Each 3D line-support region is fitted by an LSHP structure. The LSHP structure is used to validate the 3D line-support region and ‘provide a geometric constraint for line segments.

It is difficult to directly fit the LSHP structure for a 3D line-support region in 3D space. Instead, we can take advantage of the corresponding 2D line-support regions to ascertain an optimal projection direction of the 3D line-support region (Fig. 6(a)). By projecting the 3D line-support region along the projection

Refinement of 3D line-support regions and LSHP structures

Because the 3D line-support regions are obtained from multi-view images, there are many overlaps. It is necessary to combine the 3D line-support regions and LSHP structures that share the same line segment. At the same time, the boundaries of the 3D line-support regions also need to be refined in 3D space.

Environment

The method was evaluated using typical street-scene LiDAR point clouds acquired by a RIGEL VMX-450 MLS system; the average density of the point clouds used is approximately 2500 points/m². We also tested our method on the classical “sharp sphere” point cloud data.

Information on the test point clouds is summarized in Table 1. The first column of the table is the name of the data, the second column is the figure number of the point clouds as presented in this paper, the third column is the number

Application

In this section, we illustrate the effectiveness of 3D line-support regions and their relative LSHP structures in urban scene abstraction.

Today, with the widespread use of point clouds acquired by current devices, it is efficient to create large-scale dense point clouds from large outside environments. However, to fully model the entire scene is time-consuming with the current technology. For other applications, such browsing the data on the Internet via a browser, the bandwidth is the major

Conclusion and future work

In this paper, we have proposed an effective line segment extraction method that is capable of accurately extracting line segments from large-scale unorganized raw scan point clouds. The 3D line-support regions are also extracted at the same time. These 3D line-support regions are fitted by our Line-Segment-Half-Planes (LSHP) structure, which provides a geometric constraint for line segments, making the line segments more reliable and accurate.

The proposed method was tested on raw scan point

Acknowledgments

This work was supported by an NSFC Grant (Project No. 61371144) and an NSERC Discovery Grant. The authors would like to thank the anonymous reviewers for their valuable comments.

References (32)

D.H. Ballard
Generalizing the hough transform to detect arbitrary shapes
Pattern Recogn.
(1981)
M.L. Fredman
On computing the length of longest increasing subsequences
Discr. Math.
(1975)
B. Yang et al.
Semi-automated extraction and delineation of 3d roads of street scene from mobile laser scanning point clouds
ISPRS J. Photogramm. Remote Sens.
(2013)
E. Altantsetseg et al.
Feature line extraction from unorganized noisy point clouds using truncated fourier series
Vis. Comput.
(2013)
Baillard, C., Schmid, C., Zisserman, A., Fitzgibbon, A., England, O.O., 1999. Automatic line matching and 3d...
P. Borges et al.
Vision-based localization using an edge map extracted from 3d laser range data
Boucheny, C., 2009. Interactive Scientific Visualization of Large Datasets: Towards a Perceptive-Based Approach (Ph.D....
J.B. Burns et al.
Extracting straight lines
IEEE Trans. Pattern Anal. Mach. Intell.
(1986)
J. Canny
A computational approach to edge detection
IEEE Trans. Pattern Anal. Mach. Intell.
(1986)
D. Ceylan et al.
Factored Facade Acquisition Using Symmetric Line Arrangements

T. Chen et al.

3d line segment detection for unorganized point clouds from multi-view stereo

J. Daniels et al.

Robust smooth feature extraction from point clouds

A. Desolneux et al.

Meaningful alignments

Int. J. Comput. Vis.

(2000)

E. Eisemann et al.

Flash photography enhancement via intrinsic relighting

ACM Trans. Graph.

(2004)

S. Heuel et al.

Matching, reconstructing and grouping 3d lines from multiple views using uncertain projective geometry

A. Jain et al.

Exploiting global connectivity constraints for reconstruction of 3d line segments from images

Cited by (117)

Accurate and complete line segment extraction for large-scale point clouds
2024, International Journal of Applied Earth Observation and Geoinformation
Line segment extraction from point clouds is critical for analyzing and understanding large-scale scenes. The main challenge is to generate line segments accurately as well as completely. However, state-of-the-art approaches continue to struggle with this issue. In this paper, we propose a novel method for effectively generating line segments from large-scale point clouds. To this end, we design a weighted centroid displacement scheme for identifying comprehensive feature points. Then, we employ an $L_{1}$ -median optimization to refine the identified feature points to perceive geometric edges on the underlying surface accurately. Following that, we generate complete and concise line segments from the refined feature points by designing three geometric operators: clustering, exclusion, and assimilation. The clustering operator generates the initial line segments based on optimized feature points, and the exclusion operator and the assimilation operator ensure the completeness and continuity of these line segments. We evaluate our approach on various scene point clouds, such as TLS, MLS, and ALS data. Extensive experimental results show that our method can outperform the competing approaches in terms of both accuracy and efficiency.
BoundED: Neural boundary and edge detection in 3D point clouds via local neighborhood statistics
2023, ISPRS Journal of Photogrammetry and Remote Sensing
Extracting high-level structural information from 3D point clouds is challenging but essential for tasks like urban planning or autonomous driving requiring an advanced understanding of the scene at hand. Existing approaches are still not able to produce high-quality results consistently while being fast enough to be deployed in scenarios requiring interactivity. We propose to utilize a novel set of features describing the local neighborhood on a per-point basis via first and second order statistics as input for a simple and compact classification network to distinguish between non-edge, sharp-edge, and boundary points in the given data. Leveraging this feature embedding enables our algorithm to outperform the state-of-the-art technique PCEDNet in terms of quality and processing time while additionally allowing for the detection of boundaries in the processed point clouds.
Robust and Accurate Feature Detection on Point Clouds
2023, CAD Computer Aided Design
Geometric feature detection on surfaces is a crucial task for the characterization and understanding of geometry shapes. In this paper, we present a robust and reliable approach for accurately capturing local surface variations at different feature sizes within point clouds. To this end, we define a bilateral weighted centroid projection-based metric to quantify surface deviations. Based on the metric, we propose a structure-to-detail feature perception algorithm to accurately locate geometric features of varying sizes. Additionally, we use tensor analysis to extract boundary features. We evaluate our method on various object- and scene-level point clouds, demonstrating its superiority and versatility over existing techniques. Computational results show that our method is capable of identifying a wide range of geometric characteristics within point clouds, including complicated structures, rich textures, fine details, shallow curves, and geometric boundaries. We also validate the effectiveness of our approach on several downstream applications, including segmentation, surface reconstruction, and feature line extraction.
WireframeNet: A novel method for wireframe generation from point cloud
2023, Computers and Graphics (Pergamon)
Generating wireframe from point clouds is a challenging task. To make this process easier, we introduce WireframeNet, a deep neural network that transforms point clouds into wireframes. The network inputs a set of disordered points and outputs a complete wireframe structure. We use the insight of the medial axis transform to filter the original point cloud, then predict a set of edge points by learning the geometric transformation, and finally analyze the connectivity between the edge points to construct the complete wireframe structure. We train and evaluate publicly available wireframe datasets and compare the results quantitatively and qualitatively with traditional and other deep learning-based methods. Extensive experiments have demonstrated the robust and efficient performance of our proposed WireframeNet for the task of wireframe structure extraction from point clouds.
Automated extraction of geometric primitives with solid lines from unstructured point clouds for creating digital buildings models
2023, Automation in Construction
Point clouds produced by laser scanners are an invaluable source of data for reconstructing multi-dimensional digital models that reflect the as-is conditions of built facilities. However, previous studies aimed to reconstruct models by overlaying the dataset on top of ground-truth reference models to manually adjust the accuracy of the output. Therefore, this paper describes the extraction of geometric primitives with solid lines—the simplest form of objectified data that computer-aided design systems can handle—from unorganized data points and creation of digital models of built facilities in a form of floor plan. The geometric primitives are extracted from 3D points by hybridizing machine learning algorithms, which are mean-shift clustering, non-convex hull, and random sample and consensus (RANSAC). This paper provides a solution for creating a new form of as-built model with high accuracy and robustness from scratch without the involvement of ground-truth solutions or manual adjustments.
Geometric feature enhanced line segment extraction from large-scale point clouds with hierarchical topological optimization
2022, International Journal of Applied Earth Observation and Geoinformation
Citation Excerpt :
Borges et al. (2010) firstly aggregated point clouds into clusters using the local features and then projected clusters into 2D to get two endpoints of the plane intersecting line and the depth discontinuity edges in point clouds. Lin et al. (2015) firstly used the EDL algorithm to project the point clouds into images from multiple perspectives. 3D line segments are then extracted by back-projecting 2D line segments to the 3D LSHP structure.
As the most common primitives, line segments play an essential role in the vectorized reconstruction of artificial scenes. In this paper, a geometric feature enhanced line extraction method with Hierarchical Topological Optimization is proposed to extract line segments from large-scale point clouds. Firstly, 3D projection plane regions are extracted from point clouds using region growing and merging; Secondly, the point clouds in the detected planar region are projected onto 2D grids for efficiently geometry features enhanced line detection. Specifically, the edge feature index of each point inside each projection grid is calculated and fused to represent the edge-geometry enhanced feature. The likelihood of edge presenting on the projection grids is produced using a pre-trained convolutional neural network that combines multi-scale edge locating outputs. Then, line segments on the projected edge maps are extracted by the MCMLSD algorithm and further back-projected into 3D space to form the 3D line segment candidates. Thirdly, the hierarchical topological relationships between the contour and line segments are used to optimize the candidate 3D line segments. The optimization process consists of merging perceptually accurate 3D line segments, closing the contour line based on the contour and merging plane intersection lines based on line segments. The point clouds of seven large-scale outdoor scenes from the Semantic 3D and WHU-TLS public datasets are selected for qualitative and quantitative evaluations. Experiments show that the proposed approach can efficiently extract line segments that represent the geometric characteristics of the scenes comprehensively and accurately. Compared with the SOTA line segments extraction algorithms (i.e., Lu et al. (2019), Zhang et al. (2020a)), the proposed method filters out major miscellaneous/broken lines and extract more complete line segments. The $Completeness$ and $Correctness$ of the line segments extraction results reach 86% on average compared with manual labeled ground truth ( $d_{l}$ , $d_{s}$ : 0.5,0.5), at an average processing speed of 25,000 points per second.

View all citing articles on Scopus

View full text

Line segment extraction for large scale unorganized point clouds

Abstract

Introduction

Section snippets

2D line segment detection for a single image

Overview

3D line-support region extraction

LSHP modeling

Refinement of 3D line-support regions and LSHP structures

Environment

Application

Conclusion and future work

Acknowledgments

Pattern Recogn.

Discr. Math.

ISPRS J. Photogramm. Remote Sens.

Feature line extraction from unorganized noisy point clouds using truncated fourier series

Vis. Comput.

Vision-based localization using an edge map extracted from 3d laser range data

Extracting straight lines

IEEE Trans. Pattern Anal. Mach. Intell.

A computational approach to edge detection