Abstract
The creation of high-quality semantically parsed 3D models for dense metropolitan areas is a fundamental urban modeling problem. Although recent advances in acquisition techniques and processing algorithms have resulted in large-scale imagery or 3D polygonal reconstructions, such data-sources are typically noisy, and incomplete, with no semantic structure. In this paper, we present an automatic data fusion technique that produces high-quality structured models of city blocks. From coarse polygonal meshes, street-level imagery, and GIS footprints, we formulate a binary integer program that globally balances sources of error to produce semantically parsed mass models with associated facade elements. We demonstrate our system on four city regions of varying complexity; our examples typically contain densely built urban blocks spanning hundreds of buildings. In our largest example, we produce a structured model of 37 city blocks spanning a total of 1, 011 buildings at a scale and quality previously impossible to achieve automatically.
Supplemental Material
Available for Download
Supplemental material.
- Lama Affara, Liangliang Nan, Bernard Ghanem, and Peter Wonka. 2016. Large Scale Asset Extraction for Urban Images. ECCV (2016), 437--452.Google Scholar
- Oswin Aichholzer, Franz Aurenhammer, David Alberts, and Bernd Gärtner. 1996. A novel type of skeleton for polygons. In The Journal of Universal Computer Science. Springer, 752--761.Google Scholar
- Dragomir Anguelov, Carole Dulong, Daniel Filip, Christian Frueh, Stéphane Lafon, Richard Lyon, Abhijit Ogale, Luc Vincent, and Josh Weaver. 2010. Google street view: Capturing the world at street level. Computer 43, 6 (2010), 32--38. Google ScholarDigital Library
- Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla. 2017. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. IEEE TPAMI (2017).Google Scholar
- Therese Biedl, Stefan Huber, and Peter Palfrader. 2016. Planar matchings for weighted straight skeletons. International Journal of Computational Geometry & Applications 26, 03n04 (2016), 211--229. Google ScholarDigital Library
- Claus Brenner. 2005. Building reconstruction from images and laser scanning. International Journal of Applied Earth Observation and Geoinformation 6, 3 (2005), 187--198.Google ScholarCross Ref
- Gabriel J. Brostow, Jamie Shotton, Julien Fauqueur, and Roberto Cipolla. 2008. Segmentation and Recognition Using Structure from Motion Point Clouds. ECCV (2008), 44--57. Google ScholarDigital Library
- Duygu Ceylan, Minh Dang, Niloy J. Mitra, Boris Neubert, and Mark Pauly. 2016. Discovering Structured Variations Via Template Matching. CGF (01 2016). Google ScholarDigital Library
- Duygu Ceylan, Niloy J. Mitra, Youyi Zheng, and Mark Pauly. 2013. Coupled Structure-from-Motion and 3D Symmetry Detection for Urban Facades. ACM TOG (2013), 15. Google ScholarDigital Library
- John W. Chinneck. 2008. Feasibility and Infeasibility in Optimization: Algorithms and Computational Methods. Springer. Google ScholarDigital Library
- Andrea Cohen, Alexander G Schwing, and Marc Pollefeys. 2014. Efficient structured parsing of facades using dynamic programming. IEEE CVPR (2014), 3206--3213. Google ScholarDigital Library
- Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. 2016. The Cityscapes Dataset for Semantic Urban Scene Understanding. In IEEE CVPR.Google Scholar
- Dengxin Dai, Mukta Prasad, Gerhard Schmitt, and Luc Van Gool. 2012. Learning domain knowledge for facade labelling. ECCV (2012), 710--723. Google ScholarDigital Library
- Paul E Debevec, Camillo J Taylor, and Jitendra Malik. 1996. Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach. ACM SIGGRAPH (1996), 11--20. Google ScholarDigital Library
- Anthony R Dick, Philip HS Torr, and Roberto Cipolla. 2004. Modelling and interpretation of architecture from several images. IJCV 60, 2 (2004), 111--134. Google ScholarDigital Library
- David Eigen and Rob Fergus. 2015. Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. IEEE ICCV (2015), 2650--2658. Google ScholarDigital Library
- David Eppstein and Jeff Erickson. 1999. Raising roofs, crashing cycles, and playing pool: Applications of a data structure for finding pairwise interactions. Discrete & Computational Geometry 22, 4 (1999), 569--592.Google ScholarCross Ref
- Tian Fang, Zhexi Wang, Honghui Zhang, and Long Quan. 2013. Image-based modeling of unwrappable facades. IEEE TVCG 19, 10 (2013), 1720--1731.Google Scholar
- K. Fukunaga and L. Hostetler. 1975. The estimation of the gradient of a density function, with applications in pattern recognition. IEEE TIT 21, 1 (January 1975), 32--40. Google ScholarDigital Library
- Yasutaka Furukawa and Jean Ponce. 2010. Accurate, dense, and robust multiview stereopsis. IEEE PAMI 32, 8 (2010), 1362--1376. Google ScholarDigital Library
- Ignacio Garcia-Dorado, Ilke Demir, and Daniel G Aliaga. 2013. Automatic urban modeling using volumetric reconstruction with surface graph cuts. Computers & Graphics 37, 7 (2013), 896--910. Google ScholarDigital Library
- Aleksey Golovinskiy, Vladimir G Kim, and Thomas Funkhouser. 2009. Shape-based recognition of 3D point clouds in urban environments. IEEE ICCV (2009), 2154--2161.Google Scholar
- Gurobi. 2016. Gurobi Optimizer Reference Manual. (2016). http://www.gurobi.comGoogle Scholar
- Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe: Convolutional Architecture for Fast Feature Embedding. arXiv preprint arXiv:1408.5093 (2014).Google Scholar
- Haiyong Jiang, Liangliang Nan, Dong-Ming Yan, Weiming Dong, Xiaopeng Zhang, and Peter Wonka. 2016. Automatic constraint detection for 2D layout regularization. IEEE TVCG 22, 8 (2016), 1933--1944. Google ScholarDigital Library
- Michael Kazhdan, Matthew Bolitho, and Hugues Hoppe. 2006. Poisson Surface Reconstruction. SGP (2006), 61--70. Google ScholarDigital Library
- Michael Kazhdan and Hugues Hoppe. 2013. Screened poisson surface reconstruction. ACM TOG 32, 3 (2013), 29. Google ScholarDigital Library
- Tom Kelly and Peter Wonka. 2011. Interactive architectural modeling with procedural extrusions. ACM TOG 30, 2 (2011), 14. Google ScholarDigital Library
- Alex Kendall, Vijay Badrinarayanan, and Roberto Cipolla. 2015. Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding. arXiv preprint arXiv.1511.02680 (2015).Google Scholar
- Mateusz Kozinski, Raghudeep Gadde, Sergey Zagoruyko, Guillaume Obozinski, and Renaud Marlet. 2015. A MRF shape prior for facade parsing with occlusions. IEEE CVPR (2015), 2820--2828.Google Scholar
- Florent Lafarge, Xavier Descombes, Josiane Zerubia, and Marc Pierrot-Deseilligny. 2010. Structural approach for building reconstruction from a single DSM. IEEE TPAMI 32, 1 (2010), 135--147. Google ScholarDigital Library
- Robert G Laycock and AM Day. 2003. Automatically generating large urban environments based on the footprint data of buildings. ACM SMA (2003), 346--351. Google ScholarDigital Library
- Minglei Li, Peter Wonka, and Liangliang Nan. 2016. Manhattan-world urban reconstruction from point clouds. ECCV (2016), 54--69.Google Scholar
- Yangyan Li, Xiaokun Wu, Yiorgos Chrysathou, Andrei Sharf, Daniel Cohen-Or, and Niloy J Mitra. 2011a. Globfit: Consistently fitting primitives by discovering global relations. ACM SIGGRAPH 30, 4 (2011), 52. Google ScholarDigital Library
- Yangyan Li, Qian Zheng, Andrei Sharf, Daniel Cohen-Or, Baoquan Chen, and Niloy J Mitra. 2011b. 2D-3D fusion for layer decomposition of urban facades. IEEE ICCV (2011), 882--889. Google ScholarDigital Library
- Hui Lin, Jizhou Gao, Yu Zhou, Guiliang Lu, Mao Ye, Chenxi Zhang, Ligang Liu, and Ruigang Yang. 2013. Semantic decomposition and reconstruction of residential scenes from LiDAR data. ACM SIGGRAPH 32, 4 (2013), 66. Google ScholarDigital Library
- Andelo Martinović, Markus Mathias, Julien Weissenberg, and Luc Van Gool. 2012. A three-layered approach to facade parsing. ECCV (2012), 416--429. Google ScholarDigital Library
- Bogdan C Matei, Harpreet S Sawhney, Supun Samarasekera, Janet Kim, and Rakesh Kumar. 2008. Building segmentation for densely built urban regions using aerial lidar data. IEEE CVPR (2008), 1--8.Google Scholar
- Peter Miller et al. 2017. Buildings - OpenStreetMap Wiki. (2017). Retrieved August 8, 2017 from http://wiki.openstreetmap.org/wiki/BuildingsGoogle Scholar
- Aron Monszpart, Nicolas Mellado, Gabriel J Brostow, and Niloy J Mitra. 2015. RAPter: rebuilding man-made scenes with regular arrangements of planes. ACM SIGGRAPH 34, 4 (2015), 103--1. Google ScholarDigital Library
- Pascal Müller, Peter Wonka, Simon Haegler, Andreas Ulmer, and Luc Van Gool. 2006. Procedural modeling of buildings. ACM SIGGRAPH 25, 3 (2006), 614--623. Google ScholarDigital Library
- Pascal Müller, Gang Zeng, Peter Wonka, and Luc Van Gool. 2007. Image-based procedural modeling of facades. ACM SIGGRAPH 26, 3 (2007), 85. Google ScholarDigital Library
- Przemyslaw Musialski, Peter Wonka, Daniel G Aliaga, Michael Wimmer, L v Gool, and Werner Purgathofer. 2013. A survey of urban reconstruction. CGF 32, 6 (2013), 146--177. Google ScholarDigital Library
- Liangliang Nan, Caigui Jiang, Bernard Ghanem, and Peter Wonka. 2015. Template assembly for detailed urban reconstruction. CGF Eurographics 34, 2 (2015), 217--228. Google ScholarDigital Library
- Liangliang Nan, Andrei Sharf, Hao Zhang, Daniel Cohen-Or, and Baoquan Chen. 2010. Smartboxes for interactive urban reconstruction. ACM SIGGRAPH 29, 4 (2010), 93. Google ScholarDigital Library
- Gen Nishida, Ignacio Garcia-Dorado, Daniel G Aliaga, Bedrich Benes, and Adrien Bousseau. 2016. Interactive sketching of urban procedural models. ACM SIGGRAPH 35, 4 (2016), 130. Google ScholarDigital Library
- Charalambos Poullis and Suya You. 2009. Automatic reconstruction of cities from remote sensor data. IEEE CVPR (2009), 2775--2782.Google Scholar
- Hayko Riemenschneider, Ulrich Krispel, Wolfgang Thaller, Michael Donoser, Sven Havemann, Dieter Fellner, and Horst Bischof. 2012. Irregular lattices for complex shape grammar facade parsing. IEEE CVPR (2012), 1640--1647. Google ScholarDigital Library
- David Salinas, Florent Lafarge, and Pierre Alliez. 2015. Structure-Aware Mesh Decimation. CGF 34, 6 (2015), 211--227. Google ScholarDigital Library
- Chao-Hui Shen, Shi-Sheng Huang, Hongbo Fu, and Shi-Min Hu. 2011. Adaptive partitioning of urban facades. ACM SIGGRAPH Asia 30, 6 (2011), 184. Google ScholarDigital Library
- Noah Snavely, Steven M Seitz, and Richard Szeliski. 2006. Photo tourism: exploring photo collections in 3D. ACM SIGGRAPH 25, 3 (2006), 835--846. Google ScholarDigital Library
- Olivier Teboul, Iasonas Kokkinos, Loic Simon, Panagiotis Koutsourakis, and Nikos Paragios. 2013. Parsing facades with shape grammars and reinforcement learning. IEEE TPAMI 35, 7 (2013), 1744--1756. Google ScholarDigital Library
- Radim Tylecek. 2012. The CMP facade database. Technical Report. Tech. rep., CTU-CMP-2012-24, Czech Technical University.Google Scholar
- Carlos A Vanegas, Daniel G Aliaga, and Bedřich Beneš. 2010. Building reconstruction using manhattan-world grammars. IEEE CVPR (2010), 358--365.Google Scholar
- Yannick Verdie, Florent Lafarge, and Pierre Alliez. 2015. LOD generation for urban scenes. Technical Report 3. 30:1--30:14 pages.Google Scholar
- Jianxiong Xiao, Tian Fang, Ping Tan, Peng Zhao, Eyal Ofek, and Long Quan. 2008. Image-based façade modeling. ACM SIGGRAPH Asia 27, 5 (2008), 161.Google Scholar
- Chao Yang, Tian Han, Long Quan, and Chiew-Lan Tai. 2012. Parsing façade with rank-one approximation. IEEE CVPR (2012), 1720--1727. Google ScholarDigital Library
- Qian Zheng, Andrei Sharf, Guowei Wan, Yangyan Li, Niloy J Mitra, Daniel Cohen-Or, and Baoquan Chen. 2010. Non-local scan consolidation for 3D urban scenes. ACM SIGGRAPH 29, 4 (2010), 94. Google ScholarDigital Library
- Qian-Yi Zhou and Ulrich Neumann. 2010. 2.5D dual contouring: a robust approach to creating building models from aerial lidar point clouds. ECCV (2010), 115--128. Google ScholarDigital Library
Index Terms
BigSUR: large-scale structured urban reconstruction
Recommendations
Automatic Deep Inference of Procedural Cities from Global-scale Spatial Data
Recent advances in big spatial data acquisition and deep learning allow novel algorithms that were not possible several years ago. We introduce a novel inverse procedural modeling algorithm for urban areas that addresses the problem of spatial data ...
Proceduralization of Buildings at City Scale
3DV '14: Proceedings of the 2014 2nd International Conference on 3D Vision - Volume 01We present a framework for the conversion of existing 3D unstructured urban models into a compact procedural representation that enables model synthesis, querying, and simplification of large urban areas. During the de-instancing phase, a dissimilarity-...
Procedural Urban Modeling in Practice
Film and game studios can no longer meet audience demand for visual content by increasing production budgets. Instead they are turning to procedural modeling, particularly for modeling cities. The authors review procedural modeling, examine the ...
Comments