Abstract
Architecture design is a crucial step for neural-network-based methods, and it requires years of experience and extensive work. Encouragingly, with recently proposed neural architecture search (NAS), the architecture design process could be automated. In particular, differentiable architecture search (DARTS) reduces the time cost of search to a couple of GPU days. However, due to the inconsistency between the architecture search and evaluation of DARTS, its performance has yet to be improved. We propose two strategies to narrow the search/evaluation gap: firstly, rectify the operation with the highest confidence; secondly, prune the operation with the lowest confidence iteratively. Experiments show that our method achieves 2.46%/2.48% (test error, Strategy 1 or 2) on CIFAR-10 and 16.48%/16.15% (test error, Strategy 1 or 2) on CIFAR-100 at a low cost of 11 or 8 (Strategy 1 or 2) GPU hours, and outperforms state-of-the-art algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cai, H., Zhu, L., Han, S.: ProxylessNAS: direct neural architecture search on target task and hardware. In: International Conference on Learning Representations (2019)
Chen, X., Xie, L., Wu, J., Tian, Q.: Progressive differentiable architecture search: bridging the depth gap between search and evaluation. In: International Conference on Computer Vision (2019)
Liu, H., Simonyan, K., Yang, Y.: DARTS: differentiable architecture search. In: International Conference on Learning Representations (2019)
Pham, H., Guan, M.Y., Zoph, B., Le, Q.V., Dean, J.: Efficient neural architecture search via parameter sharing. In: International Conference on Machine Learning, pp. 4092–4101 (2018)
Real, E., Aggarwal, A., Huang, Y., Le, Q.V.: Regularized evolution for image classifier architecture search. In: AAAI Conference on Artificial Intelligence (2019)
Xu, Y., et al.: PC-DARTS: partial channel connections for memory-efficient architecture search. In: International Conference on Learning Representations (2020)
Zoph, B., Le, Q.V.: Neural architecture search with reinforcement learning. In: International Conference on Learning Representations (2017)
Acknowledgments
This work is supported by the National Natural Science Foundation of China under Grant 61673029. This work is also a research achievement of Key Laboratory of Science, Technology, and Standard in Press Industry (Key Laboratory of Intelligent Press Media Technology).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, Y., Zhou, Y., Wang, Y., Tang, Z. (2020). PD-DARTS: Progressive Discretization Differentiable Architecture Search. In: Lu, Y., Vincent, N., Yuen, P.C., Zheng, WS., Cheriet, F., Suen, C.Y. (eds) Pattern Recognition and Artificial Intelligence. ICPRAI 2020. Lecture Notes in Computer Science(), vol 12068. Springer, Cham. https://doi.org/10.1007/978-3-030-59830-3_26
Download citation
DOI: https://doi.org/10.1007/978-3-030-59830-3_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59829-7
Online ISBN: 978-3-030-59830-3
eBook Packages: Computer ScienceComputer Science (R0)