Jyotirmoy Deshmukh
Abstract
Many problems in the design and analysis of cyber-physical systems (CPS) reduce to the following optimization problem: given a CPS which transforms continuous-time input traces in Rm to continuous-time output traces in Rn and a cost function over output traces, find an input trace which minimizes the cost. Cyber-physical systems are typically so complex that solving the optimization problem analytically by examining the system dynamics is not feasible. We consider a black-box approach, where the optimization is performed by testing the input-output behaviour of the CPS.
We provide a unified, tool-supported methodology for CPS testing and optimization. Our tool is the first CPS testing tool that supports Bayesian optimization. It is also the first to employ fully automated dimensionality reduction techniques. We demonstrate the potential of our tool by running experiments on multiple industrial case studies. We compare the effectiveness of Bayesian optimization to state-of-the-art testing techniques based on CMA-ES and Simulated Annealing.
- H. Abbas and G. Fainekos. 2012. Convergence proofs for Simulated Annealing falsification of safety properties. In Communication, Control, and Computing (Allerton), 2012 50th Annual Allerton Conference on. IEEE, 1594--1601.Google Scholar
- T. Akazaki. 2016. Falsification of Conditional Safety Properties for Cyber-Physical Systems with Gaussian Process Regression. 439--446.Google Scholar
- R. Alur. 2015. Principles of Cyber-Physical Systems. The MIT Press. Google Scholar
- R. Alur, T. Feder, and T. A. Henzinger. 1996. The benefits of relaxing punctuality. J. ACM 43, 1 (1996), 116--146. Google ScholarDigital Library
- Y. Annpureddy, C. Liu, G. E. Fainekos, and S. Sankaranarayanan. 2011. S-TaLiRo: A tool for temporal logic falsification for hybrid systems. In TACAS 11 (Lecture Notes in Computer Science), Vol. 6605. Springer, 254--257. Google ScholarDigital Library
- S. Bansal, R. Calandra, T. Xiao, S. Levine, and C. Tomlin. 2017. Goal-driven dynamics learning via Bayesian optimization. CoRR abs/1703.09260 (2017).Google Scholar
- M. Branicky. 1995. Studies in hybrid systems: modeling, analysis, and control. Ph.D. thesis, Massachusetts Institute of Technology. Google ScholarDigital Library
- E. Brochu, V. M. Cora, and N. de Freitas. 2010. A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. CoRR abs/1012.2599 (2010).Google Scholar
- A. D. Bull. 2011. Convergence rates of efficient global optimization algorithms. J. Mach. Learn. Res. 12 (Nov. 2011), 2879--2904. Google ScholarDigital Library
- J. Deshmukh, X. Jin, J. Kapinski, and O. Maler. 2015. Stochastic local search for falsification of hybrid systems. In ATVA. Springer, 500--517.Google Scholar
- A. Donzé. 2010. Breach, A Toolbox for Verification and Parameter Synthesis of Hybrid Systems. Springer, 167--170. Google ScholarDigital Library
- A. Donzé and O. Maler. 2010. Robust Satisfaction of Temporal Logic over Real-Valued Signals. Springer, 92--106. Google ScholarDigital Library
- T. Dreossi, T. Dang, A. Donzé, J. Kapinski, X. Jin, and J. V. Deshmukh. 2015. Efficient Guiding Strategies for Testing of Temporal Properties of Hybrid Systems. Springer International Publishing, 127--142.Google Scholar
- B. Fabien. 1998. Some tools for the direct solution of optimal control problems. Advances in Engineering Software 29, 1 (1998), 45--61. Google ScholarDigital Library
- G. Fainekos. 2015. Automotive control design bug-finding with the S-TaLiRo tool. In ACC 2015. 4096.Google ScholarCross Ref
- S. Grünewälder, J.-Y. Audibert, M. Opper, and J. Shawe-Taylor. 2010. Regret Bounds for Gaussian Process Bandit Problems. In AISTATS 2010. 273--280.Google Scholar
- N. Hansen. 2016. The CMA Evolution Strategy: A tutorial. CoRR abs/1604.00772 (2016).Google Scholar
- M. Huang, K. Zaseck, K. Butts, and I. Kolmanovsky. 2016. Rate-based model predictive controller for diesel engine air path: Design and experimental evaluation. IEEE Trans. on Control Systems Technology 99 (2016), 1--14.Google Scholar
- X. Jin, J. V. Deshmukh, J. Kapinski, K. Ueda, and K. Butts. 2014. Powertrain control verification benchmark. In HSCC’14. ACM, 253--262. Google ScholarDigital Library
- W. B. Johnson and J. Lindenstrauss. 1984. Extensions of Lipschitz mappings into a Hilbert space. Contemporary Mathematics 26 (1984), 189--206.Google ScholarCross Ref
- D. R. Jones, C. D. Perttunen, and B. E. Stuckman. 1993. Lipschitzian optimization without the Lipschitz constant. Journal of Optimization Theory and Applications 79, 1 (1993), 157--181.Google ScholarDigital Library
- S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. 1983. Optimization by simulated annealing. Science 220, 4598 (1983), 671--680.Google Scholar
- D. Lizotte, T. Wang, M. Bowling, and D. Schuurmans. 2007. Automatic gait optimization with Gaussian process regression. In IJCAI 07. 944--949. Google ScholarDigital Library
- N. Mahendran, Z. Wang, F. Hamze, and N. D. Freitas. 2012. Adaptive MCMC with Bayesian optimization. In AISTATS-12), N. D. Lawrence and M. A. Girolami (Eds.), Vol. 22. 751--760.Google Scholar
- O. Maler and D. Nickovic. 2013. Monitoring properties of analog and mixed-signal circuits. STTT 15, 3 (2013), 247--268.Google ScholarDigital Library
- A. Marco, P. Hennig, J. Bohg, S. Schaal, and S. Trimpe. 2016. Automatic LQR tuning based on Gaussian process global optimization. In ICRA 16. 270--277.Google Scholar
- Mathworks. 2017. Simulink—Simulation and model-based design. https://www.mathworks.com/products/simulink.html.Google Scholar
- W. Messner and D. Tilbury. 2017. Control tutorials for MATLAB and Simulink. http://ctms.engin.umich.edu/CTMS/index.php?aux=Basics_Simulink% #27.Google Scholar
- M. J. D. Powell. 2009. The BOBYQA algorithm for bound constrained optimization without derivatives. Technical Report DAMTP2009/NA06. University of Cambridge.Google Scholar
- C. E. Rasmussen and C. K. I. Williams. 2006. Gaussian processes for machine learning. MIT Press. Google ScholarDigital Library
- S. Sankaranarayanan and G. Fainekos. 2012. Falsification of temporal properties of hybrid systems using the cross-entropy method. In HSCC 12. ACM, 125--134. Google ScholarDigital Library
- P. Tabuada. 2009. Verification and Control of Hybrid Systems - A Symbolic Approach. Springer. Google ScholarDigital Library
- Z. Wang, F. Hutter, M. Zoghi, D. Matheson, and N. De Freitas. 2016. Bayesian optimization in a billion dimensions via random embeddings. Journal of Artificial Intelligence Research 55, 1 (2016), 361--387. Google ScholarCross Ref
- Y. Xue and P. Bogdan. 2017. Constructing compact causal mathematical models for complex dynamics. In ICCPS’17. ACM, 97--107. Google ScholarDigital Library
Index Terms
- Testing Cyber-Physical Systems through Bayesian Optimization
Recommendations
Black-box optimization benchmarking of IPOP-saACM-ES and BIPOP-saACM-ES on the BBOB-2012 noiseless testbed
GECCO '12: Proceedings of the 14th annual conference companion on Genetic and evolutionary computationIn this paper, we study the performance of IPOP-saACM-ES and BIPOP-saACM-ES, recently proposed self-adaptive surrogate-assisted Covariance Matrix Adaptation Evolution Strategies. Both algorithms were tested using restarts till a total number of function ...
A Survey on Testing for Cyber Physical System
ICTSS 2015: Proceedings of the 27th IFIP WG 6.1 International Conference on Testing Software and Systems - Volume 9447Cyber Physical Systems CPS bridge the cyber-world of computing and communications with the physical world and require development of secure and reliable software. It asserts a big challenge not only on testing and verifying the correctness of all ...
Digital-twin-based testing for cyber–physical systems: A systematic literature review
Abstract Context:Cyber–physical systems present a challenge to testers, bringing complexity and scale to safety-critical and collaborative environments. Digital twins enhance these systems through data-driven and simulation based ...
Comments