Ilya Makarov
ABSTRACT
We study depth map reconstruction for a specific task of fast rough depth approximation having sparse depth samples obtained from low-cost depth sensors or SLAM algorithms. We propose a model interpolating downsampled semi-dense depth values and then processing super-resolution. We study our method in comparison with the state-of-the-art approaches transferring RGB information to depth. It appears that the proposed approach can be used to approximately estimate high-resolution depth maps.
- Daniel J Butler, Jonas Wulff, Garrett B Stanley, and Michael J Black. 2012. A naturalistic open source movie for optical flow evaluation. In European Conference on Computer Vision. Springer, 611--625. Google Scholar
Digital Library
- David Eigen and Rob Fergus. 2015. Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. In Proceedings of the IEEE International Conference on Computer Vision. 2650--2658. Google ScholarDigital Library
- C. Cadena et al. 2016. Multi-modal Auto-Encoders as Joint Estimators for Robotics Scene Understanding.. In Robotics: Science and Systems.Google Scholar
- Kai-Lung Hua, Kai-Han Lo, and Yu-Chiang Frank Frank Wang. 2016. Extended guided filtering for depth map upsampling. IEEE MultiMedia 23, 2 (2016), 72--83.Google ScholarCross Ref
- Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016. Perceptual losses for real-time style transfer and super-resolution. In European Conference on Computer Vision. Springer, 694--711.Google ScholarCross Ref
- Yevhen Kuznietsov, Jörg Stückler, and Bastian Leibe. 2017. Semi-supervised deep learning for monocular depth map prediction. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition. 6647--6655.Google ScholarCross Ref
- Iro Laina, Christian Rupprecht, Vasileios Belagiannis, Federico Tombari, and Nassir Navab. 2016. Deeper depth prediction with fully convolutional residual networks. In 3D Vision (3DV), 2016 Fourth International Conference on. IEEE, 239--248.Google Scholar
- Yiyi Liao, Lichao Huang, Yue Wang, Sarath Kodagoda, Yinan Yu, and Yong Liu. 2017. Parse geometry from a line: Monocular depth estimation with partial laser observation. In Robotics and Automation (ICRA), 2017 IEEE International Conference on. IEEE, 5059--5066.Google ScholarCross Ref
- Wei Liu, Yijun Li, Xiaogang Chen, Jie Yang, Qiang Wu, and Jingyi Yu. 2015. Robust High Quality Image Guided Depth Upsampling. CoRR abs/1506.05187 (2015). arXiv:1506.05187 http://arxiv.org/abs/1506.05187Google Scholar
- Fangchang Ma and Sertac Karaman. 2017. Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image. CoRR abs/1709.07492 (2017). arXiv:1709.07492 http://arxiv.org/abs/1709.07492Google Scholar
- Ilya Makarov, Vladimir Aliev, and Olga Gerasimova. 2017a. Semi-Dense Depth Interpolation using Deep Convolutional Neural Networks. In Proceedings of the 2017 ACM on Multimedia Conference. ACM, 1407--1415. Google ScholarDigital Library
- Ilya Makarov, Vladimir Aliev, Olga Gerasimova, and Pavel Polyakov. 2017b. Depth Map Interpolation Using Perceptual Loss. In Mixed and Augmented Reality (ISMAR-Adjunct), 2017 IEEE International Symposium on. IEEE, 93--94.Google ScholarCross Ref
- Pushmeet Kohli Nathan Silberman, Derek Hoiem and Rob Fergus. 2012. Indoor Segmentation and Support Inference from RGBD Images. In ECCV.Google Scholar
- Min Ni, Jianjun Lei, Runmin Cong, Kaifu Zheng, Bo Peng, and Xiaoting Fan. 2017. Color-Guided Depth Map Super Resolution Using Convolutional Neural Network. IEEE ACCESS 5 (2017), 26666--26672.Google ScholarCross Ref
- German Ros, Laura Sellart, Joanna Materzynska, David Vazquez, and Antonio M. Lopez. 2016. The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
- Daniel Scharstein and Chris Pal. 2007. Learning conditional random fields for stereo. In CVPR, 2007. CVPR'07. IEEE Conference on. IEEE, 1--8.Google ScholarCross Ref
- Nick Schneider, Lukas Schneider, Peter Pinggera, Uwe Franke, Marc Pollefeys, and Christoph Stiller. 2016. Semantically guided depth upsampling. In German Conference on Pattern Recognition. Springer, 37--48.Google ScholarCross Ref
- Benjamin Ummenhofer, Huizhong Zhou, Jonas Uhrig, Nikolaus Mayer, Eddy Ilg, Alexey Dosovitskiy, and Thomas Brox. 2017. Demon: Depth and motion network for learning monocular stereo. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Vol. 5.Google ScholarCross Ref
- Yi Xiao, Xiang Cao, Xianyi Zhu, Renzhi Yang, and Yan Zheng. 2018. Joint convolutional neural pyramid for depth map super-resolution. arXiv preprint arXiv:1801.00968 (2018).Google Scholar
- Qingxiong Yang, Ruigang Yang, James Davis, and David Nistér. 2007. Spatial-depth super resolution for range images. In Computer Vision and Pattern Recognition, 2007. CVPR'07. IEEE Conference on. IEEE, 1--8.Google ScholarCross Ref
Recommendations
Fast Semi-dense Depth Map Estimation
RETech'18: Proceedings of the 2018 ACM Workshop on Multimedia for Real Estate TechWe consider the problem of depth reconstruction from downsampled sparse depth values. We compare our approach with semi-dense depth map interpolation and direct RGB-to-Depth reconstruction solutions on several datasets, including Matterport 3D dataset ...
Fast Depth Reconstruction Using Deep Convolutional Neural Networks
Advances in Computational IntelligenceAbstractIn this paper, we study depth reconstruction via RGB-based, Sparse-Depth, and RGBd approaches. We showed that combination of RGB and Sparse Depth approach in RGBd scenario provides the best results. We also proved that the models performance can ...
Single Depth Map Super-resolution with Local Self-similarity
ICVIP '18: Proceedings of the 2018 2nd International Conference on Video and Image ProcessingConsumer depth sensors such as time-of-flight camera or Kinect have gained significant popularity in recently. However, the captured depth maps suffer from limited spatial resolution and a variety of noise, making such depth maps difficult to be ...
Comments