Wenjing Wang
Skip Abstract Section
Abstract
With the progress of Mars exploration, numerous Mars image data are being collected and need to be analyzed. However, due to the severe train-test gap and quality distortion of Martian data, the performance of existing computer vision models is unsatisfactory. In this article, we introduce a semi-supervised framework for machine vision on Mars and try to resolve two specific tasks: classification and segmentation. Contrastive learning is a powerful representation learning technique. However, there is too much information overlap between Martian data samples, leading to a contradiction between contrastive learning and Martian data. Our key idea is to reconcile this contradiction with the help of annotations and further take advantage of unlabeled data to improve performance. For classification, we propose to ignore inner-class pairs on labeled data as well as neglect negative pairs on unlabeled data, forming supervised inter-class contrastive learning and unsupervised similarity learning. For segmentation, we extend supervised inter-class contrastive learning into an element-wise mode and use online pseudo labels for supervision on unlabeled areas. Experimental results show that our learning strategies can improve the classification and segmentation models by a large margin and outperform state-of-the-art approaches.
- [1] . 2021. Machine learning based path planning for improved rover navigation. In IEEE Aerospace Conference.Google Scholar
Cross Ref
- [2] . 2020. Integrating machine learning for planetary science: Perspectives for the next decade. arXiv (2020).Google Scholar
- [3] . 2019. S4L: Self-supervised semi-supervised learning. In IEEE International Conference on Computer Vision.Google Scholar
- [4] . 2016. Computational imaging for VLBI image reconstruction. In IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- [5] . 2019. Learning imbalanced datasets with label-distribution-aware margin loss. In Conference on Neural Information Processing Systems.Google Scholar
- [6] . 2020. Unsupervised learning of visual features by contrasting cluster assignments. In Conference on Neural Information Processing Systems.Google Scholar
- [7] . 2008. Automatic detection of dust devils and clouds on mars. Machine Vision and Applications 19, 5–6 (2008), 467–482.Google ScholarDigital Library
- [8] . 2007. Oasis: Onboard autonomous science investigation system for opportunistic rover science. Journal of Field Robotics 24, 5 (2007), 379–397.Google ScholarDigital Library
- [9] . 2018. Encoder-decoder with atrous separable convolution for semantic image segmentation. In European Conference on Computer Vision.Google ScholarDigital Library
- [10] . 2020. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning.Google Scholar
- [11] . 2020. Improved baselines with momentum contrastive learning. arXiv (2020).Google Scholar
- [12] . 2019. Class-balanced loss based on effective number of samples. In IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- [13] . 2007. Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Transactions on Image Processing 16, 8 (2007), 2080–2095.Google ScholarCross Ref
- [14] . 2009. ImageNet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- [15] . 2017. Improved regularization of convolutional neural networks with cutout. arXiv (2017).Google Scholar
- [16] . 2016. Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 38, 2 (2016), 295–307.Google ScholarDigital Library
- [17] . 2020. COSMIC: Content-based onboard summarization to monitor infrequent change. In IEEE Aerospace Conference.Google ScholarCross Ref
- [18] . 2019. Data augmentation using GANs. arXiv (2019).Google Scholar
- [19] . 2019. Machine-learning-driven new geologic discoveries at mars rover landing sites: Jezero and NE syrtis. Earth and Space Science Open Archive (2019), 23.Google Scholar
- [20] . 2009. Automated targeting for the MER rovers. In IEEE International Conference on Space Mission Challenges for Information Technology.Google Scholar
- [21] . 2020. Momentum contrast for unsupervised visual representation learning. In IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- [22] . 2016. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- [23] . 2021. Multi-task learning-based all-in-one collaboration framework for degraded image super-resolution. ACM Transactions on Multimedia Computing, Communications, and Applications 17, 1 (2021), 1–21.Google ScholarDigital Library
- [24] . 1997. A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Transactions on Image Processing 6, 7 (1997), 965–976.Google ScholarDigital Library
- [25] . 2020. Decoupling representation and classifier for long-tailed recognition. In International Conference on Learning Representations.Google Scholar
- [26] . 2020. Supervised contrastive learning. In Conference on Neural Information Processing Systems.Google Scholar
- [27] . 2015. Adam: A method for stochastic optimization. In International Conference on Learning Representations.Google Scholar
- [28] . 2012. ImageNet classification with deep convolutional neural networks. In Conference on Neural Information Processing Systems.Google ScholarDigital Library
- [29] . 2013. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In International Conference on Machine Learning Workshops.Google Scholar
- [30] . 2017. Focal loss for dense object detection. In IEEE International Conference on Computer Vision.Google ScholarCross Ref
- [31] . 2021. Bootstrapping semantic segmentation with regional contrast. arXiv (2021).Google Scholar
- [32] . 2017. Learning from weak and noisy labels for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 39, 3 (2017), 486–500.Google ScholarDigital Library
- [33] . 2021. Machine learning for mars exploration. arXiv (2021).Google Scholar
- [34] . 2021. ClassMix: Segmentation-based data augmentation for semi-supervised learning. In IEEE Winter Conference on Applications of Computer Vision.Google Scholar
- [35] . 2020. Semi-supervised semantic segmentation with cross-consistency training. In IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- [36] . 2019. Machine learning for predicting thermal power consumption of the mars express spacecraft. IEEE Aerospace and Electronic Systems Magazine 34, 7 (2019), 46–60.Google ScholarCross Ref
- [37] . 2020. Designing network design spaces. In IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- [38] . 2016. SPOC: Deep learning-based terrain classification for mars rover missions. In AIAA SPACE. .Google ScholarCross Ref
- [39] . 2018. MobileNetV2: Inverted residuals and linear bottlenecks. In IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- [40] . 2015. FaceNet: A unified embedding for face recognition and clustering. In IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- [41] . 2018. Identifying exoplanets with deep learning: A five-planet resonant chain around kepler-80 and an eighth planet around kepler-90. Astronomical Journal 155, 2 (2018), 94.Google ScholarCross Ref
- [42] . 2015. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations.Google Scholar
- [43] . 2014. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research 15, 56 (2014), 1929–1958.Google ScholarDigital Library
- [44] . 2017. Revisiting unreasonable effectiveness of data in deep learning era. In IEEE International Conference on Computer Vision.Google ScholarCross Ref
- [45] . 2021. AI4MARS: A dataset for terrain-aware autonomous driving on mars. In IEEE Conference on Computer Vision and Pattern Recognition Workshops.Google ScholarCross Ref
- [46] . 2017. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In Conference on Neural Information Processing Systems.Google Scholar
- [47] . 2018. Representation learning with contrastive predictive coding. arXiv (2018).Google Scholar
- [48] . 2018. Deep mars: CNN classification of mars imagery for the PDS imaging atlas. In Conference on Innovative Applications of Artificial Intelligence.Google ScholarCross Ref
- [49] . 2018. Exoplanet biosignatures: Future directions. Astrobiology 18, 6 (2018), 779–824.Google ScholarCross Ref
- [50] . 2016. A discriminative feature learning approach for deep face recognition. In European Conference on Computer Vision.Google ScholarCross Ref
- [51] . 2020. DoMars16k: A diverse dataset for weakly supervised geomorphologic analysis on mars. Remote Sensing 12, 23 (2020), 3981.Google ScholarCross Ref
- [52] . 2020. Depth image denoising using nuclear norm and learning graph model. ACM Transactions on Multimedia Computing, Communications, and Applications 16, 4 (2020), 1–17.Google ScholarDigital Library
- [53] . 2019. CutMix: Regularization strategy to train strong classifiers with localizable features. In IEEE International Conference on Computer Vision.Google ScholarCross Ref
- [54] . 2018. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations.Google Scholar
- [55] . 2022. Semi-supervised contrastive learning with similarity co-calibration. IEEE Transactions on Multimedia (2022). https://ieeexplore.ieee.org/document/9732218.Google Scholar
- [56] . 2021. Contrastive learning for label efficient semantic segmentation. In IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- [57] . 2020. BBN: Bilateral-branch network with cumulative learning for long-tailed visual recognition. In IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
Index Terms
- Semi-supervised Learning for Mars Imagery Classification and Segmentation
Recommendations
Semi-supervised multi-label image classification based on nearest neighbor editing
Semi-supervised multi-label classification has been applied to many real-world applications such as image classification, document classification and so on. In semi-supervised learning, unlabeled samples are added to the training set for enhancing the ...
Semi-supervised Bi-dictionary Learning Using Smooth Representation-Based Label Propagation
CYBERC '15: Proceedings of the 2015 International Conference on Cyber-Enabled Distributed Computing and Knowledge DiscoveryDue to heavy clutters and occlusions of complex background, natural images contain complex features in data structure which often cause errors in image classification. In this paper, we propose semi-supervised bi-dictionary learning for image ...
Inductive Semi-supervised Multi-Label Learning with Co-Training
KDD '17: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data MiningIn multi-label learning, each training example is associated with multiple class labels and the task is to learn a mapping from the feature space to the power set of label space. It is generally demanding and time-consuming to obtain labels for training ...
Comments