Kangxue Yin
Hui Huang
Skip Abstract Section
Abstract
We introduce P2P-NET, a general-purpose deep neural network which learns geometric transformations between point-based shape representations from two domains, e.g., meso-skeletons and surfaces, partial and complete scans, etc. The architecture of the P2P-NET is that of a bi-directional point displacement network, which transforms a source point set to a prediction of the target point set with the same cardinality, and vice versa, by applying point-wise displacement vectors learned from data. P2P-NET is trained on paired shapes from the source and target domains, but without relying on point-to-point correspondences between the source and target point sets. The training loss combines two uni-directional geometric losses, each enforcing a shape-wise similarity between the predicted and the target point sets, and a cross-regularization term to encourage consistency between displacement vectors going in opposite directions. We develop and present several different applications enabled by our general-purpose bidirectional P2P-NET to highlight the effectiveness, versatility, and potential of our network in solving a variety of point-based shape transformation problems.
- Marc Alexa, Johannes Behr, Daniel Cohen-Or, Shachar Fleishman, David Levin, and Claudio T Silva. 2003. Computing and rendering point set surfaces. IEEE Trans. Vis. & Comp. Graphics 9, 1 (2003), 3--15. Google Scholar
Digital Library
- Matthew Berger, Andrea Tagliasacchi, Lee M. Seversky, Pierre Alliez, Gaël Guennebaud, Joshua A. Levine, Andrei Sharf, and Claudio T. Silva. 2017. A Survey of Surface Reconstruction from Point Clouds. Computer Graphics Forum 36, 1 (2017), 301--329. Google ScholarDigital Library
- Sema Berkiten, Maciej Halber, Justin Solomon, Chongyang Ma, Hao Li, and Szymon Rusinkiewicz. 2017. Learning detail transfer based on geometric features. In Computer Graphics Forum (Eurographics), Vol. 36. 361--373. Google ScholarDigital Library
- Jeannette Bohg, Javier Romero, Alexander Herzog, and Stefan Schaal. 2014. Robot arm pose estimation through pixel-wise part classification. In Proc. of ICRA. IEEE, 3143--3150. https://github.com/jbohg/render_kinectGoogle ScholarCross Ref
- Arunkumar Byravan and Dieter Fox. 2016. SE3-Nets: Learning Rigid Body Motion using Deep Neural Networks. In Proc. of ICRA.Google Scholar
- Junjie Cao, Andrea Tagliasacchi, Matt Olson, Hao Zhang, and Zhinxun Su. 2010. Point cloud skeletons via laplacian based contraction. In Proc. IEEE Int. Conf. on Shape Modeling and Applications. 187--197. Google ScholarDigital Library
- Jonathan C Carr, Richard K Beatson, Jon B Cherrie, Tim J Mitchell, W Richard Fright, Bruce C McCallum, and Tim R Evans. 2001. Reconstruction and representation of 3D objects with radial basis functions. In ACM Trans. on Graph (SIGGRAPH). 67--76. Google ScholarDigital Library
- Massimiliano Corsini, Paolo Cignoni, and Roberto Scopigno. 2012. Efficient and flexible sampling with blue noise properties of triangular meshes. IEEE Trans. Vis. & Comp. Graphics 18, 6 (2012), 914--924. Google ScholarDigital Library
- Angela Dai, Charles Ruizhongtai Qi, and Matthias Nießner. 2017. Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesis. In Proc. of CVPR.Google ScholarCross Ref
- Haoqiang Fan, Hao Su, and Leonidas Guibas. 2017. A point set generation network for 3d object reconstruction from a single image. In Proc. of CVPR.Google ScholarCross Ref
- Matheus Gadelha, Subhransu Maji, and Rui Wang. 2017. Shape Generation using Spatially Partitioned Point Clouds. In Proc. of BMVC.Google ScholarCross Ref
- Markus Gross and Hanspeter Pfister. 2007. Point-Based Graphics. Morgan Kaufmann. Google ScholarDigital Library
- Michael Gschwandtner, Roland Kwitt, Andreas Uhl, and Wolfgang Pree. 2011. BlenSor: blender sensor simulation toolbox. Advances in visual computing (2011), 199--208. Google ScholarDigital Library
- Paul Guerrero, Yanir Kleiman, Maks Ovsjanikov, and Niloy J Mitra. 2017. PCP-NET: Learning Local Shape Properties from Raw Point Clouds. arXiv preprint arXiv:1710.04954 (2017).Google Scholar
- Ruizhen Hu, Wenchao Li, Oliver van Kaick, Hui Huang, Melinos Averkiou, Daniel Cohen-Or, and Hao Zhang. 2017. Co-Locating Style-Defining Elements on 3D Shapes. ACM Trans. on Graph 36, 3 (2017), 33:1--33:15. Google ScholarDigital Library
- Hui Huang, Dan Li, Hao Zhang, Uri Ascher, and Daniel Cohen-Or. 2009. Consolidation of unorganized point clouds for surface reconstruction. In ACM Trans. on Graph (SIGGRAPH Asia), Vol. 28. 176:1--176:7. Google ScholarDigital Library
- Hui Huang, Shihao Wu, Daniel Cohen-Or, Minglun Gong, Hao Zhang, Guiqing Li, and Baoquan Chen. 2013a. L1-medial skeleton of point cloud. ACM Trans. on Graph (SIGGRAPH) 32, 4 (2013), 65:1--65:8. Google ScholarDigital Library
- Hui Huang, Shihao Wu, Minglun Gong, Daniel Cohen-Or, Uri Ascher, and Hao Zhang. 2013b. Edge-aware point set resampling. ACM Trans. on Graph 32, 1 (2013), 9:1--9:12. Google ScholarDigital Library
- Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-image translation with conditional adversarial networks. In Proc. of CVPR.Google ScholarCross Ref
- Max Jaderberg, Karen Simonyan, Andrew Zisserman, et al. 2015. Spatial transformer networks. In Proc. of NIPS. 2017--2025. Google ScholarDigital Library
- Michael Kazhdan and Hugues Hoppe. 2013. Screened Poisson surface reconstruction. ACM Trans. on Graph 32, 3 (2013), 29:1--29:13. Google ScholarDigital Library
- Chen-Hsuan Lin, Chen Kong, and Simon Lucey. 2018. Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction. In Proc. of AAAI.Google Scholar
- Yaron Lipman, Daniel Cohen-Or, David Levin, and Hillel Tal-Ezer. 2007. Parameterization-free projection for geometry reconstruction. In ACM Trans. on Graph (SIGGRAPH), Vol. 26. 22:1--22:6. Google ScholarDigital Library
- Ming-Yu Liu, Thomas Breuel, and Jan Kautz. 2017. Unsupervised Image-to-Image Translation Networks. In Proc. of NIPS. 700--708.Google Scholar
- Zhaoliang Lun, Matheus Gadelha, Evangelos Kalogerakis, Subhransu Maji, and Rui Wang. 2017. 3D Shape Reconstruction from Sketches via Multi-view Convolutional Networks. In Proc. of 3DV.Google ScholarCross Ref
- Niloy J Mitra and An Nguyen. 2003. Estimating surface normals in noisy point cloud data. In Symp. on Geom. Proc. 322--328. Google ScholarDigital Library
- Matthias Müller, Richard Keiser, Andrew Nealen, Mark Pauly, Markus Gross, and Marc Alexa. 2004. Point based animation of elastic, plastic and melting objects. In Symp. on Computer Animation. 141--151. Google ScholarDigital Library
- Mark Pauly, Richard Keiser, Leif P Kobbelt, and Markus Gross. 2003. Shape modeling with point-sampled geometry. ACM Trans. on Graph (SIGGRAPH) 22, 3 (2003), 641--650. Google ScholarDigital Library
- Reinhold Preiner, Oliver Mattausch, Murat Arikan, Renato Pajarola, and Michael Wimmer. 2014. Continuous Projection for Fast L1 Reconstruction. ACM Trans. on Graph 33, 4 (July 2014), 47:1--47:13. Google ScholarDigital Library
- Charles R. Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas. 2017a. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. In Proc. of CVPR.Google Scholar
- Charles R. Qi, Li Yi, Hao Su, and Leonidas J. Guibas. 2017b. PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metrie Space. In Proc. of NIPS.Google Scholar
- Minhyuk Sung, Hao Su, Vladimir G Kim, Siddhartha Chaudhuri, and Leonidas Guibas. 2017. ComplementMe: Weakly-Supervised Component Suggestions for 3D Modeling. ACM Trans. on Graph (SIGGRAPH Asia) 36, 6 (2017), 226:1--226:12. Google ScholarDigital Library
- Andrea Tagliasacchi, Thomas Delame, Michela Spagnuolo, Nina Amenta, and Alexandru Telea. 2016. 3D Skeletons. In Eurographics State of the Art Report.Google Scholar
- Andrea Tagliasacchi, Hao Zhang, and Daniel Cohen-Or. 2009. Curve skeleton extraction from incomplete point cloud. In ACM Trans. on Graph (SIGGRAPH Asia), Vol. 28. 71:1--71:9. Google ScholarDigital Library
- Shihao Wu, Hui Huang, Minglun Gong, Matthias Zwicker, and Daniel Cohen-Or. 2015a. Deep Points Consolidation. ACM Trans. on Graph 34, 6 (2015), 176:1--176:13. Google ScholarDigital Library
- Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 2015b. 3d shapenets: A deep representation for volumetric shapes. In Proc. of CVPR. 1912--1920.Google Scholar
- Zili Yi, Hao Zhang, Ping Tan, and Minglun Gong. 2017. DualGAN: Unsupervised Dual Learning for Image-to-image Translation. In Proc. of ICCV.Google ScholarCross Ref
- Tinghui Zhou, Shubham Tulsiani, Weilun Sun, Jitendra Malik, and Alexei A Efros. 2016. View synthesis by appearance flow. In Proc. Euro. Conf. on Comp. Vis. 286--301.Google ScholarCross Ref
- Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proc. of ICCV.Google ScholarCross Ref
Index Terms
- P2P-NET: bidirectional point displacement net for shape transform
Recommendations
SDM-NET: deep generative network for structured deformable mesh
We introduce SDM-NET, a deep generative neural network which produces structured deformable meshes. Specifically, the network is trained to generate a spatial arrangement of closed, deformable mesh parts, which respects the global part structure of a ...
Virtual Restoration of Ancient Wooden Ships Through Non-rigid 3D Shape Assembly with Ruled-Surface FFD
AbstractIn recent years, 3D data has been widely used in archaeology and in the field of conservation and restoration of cultural properties. Virtual restoration, which reconstructs the original state in virtual space, is one of the promising applications ...
MSSPA-GC: Multi-Scale Shape Prior Adaptation with 3D Graph Convolutions for Category-Level Object Pose Estimation
AbstractCategory-level object pose estimation aims to predict the 6D object pose and size of arbitrary objects from known categories. It remains a challenge due to the large intra-class shape variation. Recently, the introduction of the shape prior ...
Graphical abstractDisplay Omitted
Highlights- A novel shape prior adaptation method for category-level object pose estimation.
- A 3D-GCN to learn multi-scale pose-sensitive features from observed point clouds.
- A MLP network to learn multi-scale shape-aware features from prior ...
Comments