Joel Dyer
Michael Wooldridge
ABSTRACT
Agent-based modelling (ABMing) is a promising approach to modelling and reasoning about complex systems such as financial markets. However, the application of ABMs in practice is often impeded by the models’ complexity and the ensuing difficulty of performing parameter inference and optimisation tasks. This in turn has motivated efforts directed towards the construction of differentiable ABMs, enabled by recently developed effective auto-differentiation frameworks, as a strategy for addressing these challenges.
In this paper, we discuss and present experiments that demonstrate how differentiable programming may be used to implement and calibrate heterogeneous ABMs in finance. We begin by considering in more detail the difficulties inherent in constructing gradients for discrete ABMs. Secondly, we illustrate solutions to these difficulties, by using a discrete agent-based market simulation model as a case study. Finally, we show through numerical experiments how our differentiable implementation of this discrete ABM enables the use of powerful tools from probabilistic machine learning and conditional generative modelling to perform robust parameter inferences and uncertainty quantification, in a simulation-efficient manner.
- Philipp Andelfinger. 2021. Differentiable Agent-Based Simulation for Gradient-Guided Simulation-Based Optimization. arXiv:2103.12476 [cs, eess] (March 2021). arxiv:2103.12476 [cs, eess]Google Scholar
- Gaurav Arya, Moritz Schauer, Frank Schäfer, and Christopher Rackauckas. 2022. Automatic differentiation of programs with discrete randomness. Advances in Neural Information Processing Systems 35 (2022), 10435–10447.Google Scholar
- Yuanlu Bai, Henry Lam, Tucker Balch, and Svitlana Vyetrenko. 2022. Efficient Calibration of Multi-Agent Simulation Models from Output Series with Bayesian Optimization. In Proceedings of the Third ACM International Conference on AI in Finance. 437–445.Google ScholarDigital Library
- Atilim Gunes Baydin, Barak A Pearlmutter, Alexey Andreyevich Radul, and Jeffrey Mark Siskind. 2018. Automatic differentiation in machine learning: a survey. Journal of Marchine Learning Research 18 (2018), 1–43.Google ScholarDigital Library
- Yoshua Bengio, Nicholas Léonard, and Aaron Courville. 2013. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432 (2013).Google Scholar
- Jeff Bezanson, Alan Edelman, Stefan Karpinski, and Viral B Shah. 2017. Julia: A Fresh Approach to Numerical Computing. SIAM review 59, 1 (2017), 65–98.Google ScholarDigital Library
- Pier Giovanni Bissiri, Chris Holmes, and Stephen G Walker. 2016. A general framework for updating belief distributions. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 78, 5 (2016), 1103.Google ScholarCross Ref
- Badr-Eddine Cherief-Abdellatif and Pierre Alquier. 2020. MMD-Bayes: Robust Bayesian Estimation via Maximum Mean Discrepancy. In Proceedings of The 2nd Symposium on Advances in Approximate Bayesian Inference(Proceedings of Machine Learning Research, Vol. 118), Cheng Zhang, Francisco Ruiz, Thang Bui, Adji Bousso Dieng, and Dawen Liang (Eds.). PMLR, 1–21.Google Scholar
- Ayush Chopra, Alexander Rodríguez, Jayakumar Subramanian, Arnau Quera-Bofarull, Balaji Krishnamurthy, B. Aditya Prakash, and Ramesh Raskar. 2023. Differentiable Agent-Based Epidemiology. In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems(AAMAS ’23). International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 1848–1857.Google ScholarDigital Library
- Rama Cont. 2007. Volatility clustering in financial markets: empirical facts and agent-based models. Long memory in economics (2007), 289–309.Google Scholar
- Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. 2017. Density estimation using Real NVP. In International Conference on Learning Representations. https://openreview.net/forum?id=HkpbnH9lxGoogle Scholar
- Joel Dyer. 2022. Likelihood-free Bayesian inference for dynamic, stochastic simulators in the social sciences. Ph. D. Dissertation. University of Oxford.Google Scholar
- Joel Dyer, Patrick Cannon, J Doyne Farmer, and Sebastian Schmon. 2022. Black-box Bayesian inference for economic agent-based models. arXiv preprint arXiv:2202.00625 (2022).Google Scholar
- Joel Dyer, Patrick Cannon, J Doyne Farmer, and Sebastian M Schmon. 2022. Calibrating Agent-based Models to Microdata with Graph Neural Networks. In ICML 2022 Workshop AI for Agent-Based Modelling.Google Scholar
- Joel Dyer, Patrick Cannon, and Sebastian M Schmon. 2023. Approximate Bayesian Computation with Path Signatures. arxiv:2106.12555 [stat.ME]Google Scholar
- Reiner Franke. 2009. Applying the method of simulated moments to estimate a small agent-based asset pricing model. Journal of Empirical Finance 16, 5 (2009), 804–815.Google ScholarCross Ref
- Christian Gourieroux, Alain Monfort, and Eric Renault. 1993. Indirect inference. Journal of applied econometrics 8, S1 (1993), S85–S118.Google ScholarCross Ref
- Jakob Grazzini, Matteo G. Richiardi, and Mike Tsionas. 2017. Bayesian estimation of agent-based models. Journal of Economic Dynamics and Control 77 (2017), 26–47. https://doi.org/10.1016/j.jedc.2017.01.014Google ScholarCross Ref
- Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. 2012. A kernel two-sample test. The Journal of Machine Learning Research 13, 1 (2012), 723–773.Google ScholarDigital Library
- Geoffrey Grimmett and David Stirzaker. 2001. Probability and random processes. Oxford university press.Google Scholar
- Stanislao Gualdi, Marco Tarzia, Francesco Zamponi, and Jean-Philippe Bouchaud. 2015. Tipping points in macroeconomic agent-based models. Journal of Economic Dynamics and Control 50 (2015), 29–61.Google ScholarCross Ref
- Eric Jang, Shixiang Gu, and Ben Poole. 2016. Categorical Reparameterization with Gumbel-Softmax. In International Conference on Learning Representations.Google Scholar
- Diederik P Kingma and Max Welling. 2013. Auto-encoding variational Bayes. arXiv preprint arXiv:1312.6114 (2013).Google Scholar
- Jeremias Knoblauch, Jack Jewson, and Theodoros Damoulas. 2022. An optimization-centric view on Bayes’ rule: Reviewing and generalizing variational inference. Journal of Machine Learning Research 23, 132 (2022), 1–109.Google Scholar
- Ignacio N. Lobato and Carlos Velasco. 2000. Long Memory in Stock-Market Trading Volume. Journal of Business & Economic Statistics 18, 4 (2000), 410–427. http://www.jstor.org/stable/1392223Google Scholar
- Ilya Loshchilov and Frank Hutter. 2017. Decoupled Weight Decay Regularization. In International Conference on Learning Representations.Google Scholar
- Shakir Mohamed, Mihaela Rosca, Michael Figurnov, and Andriy Mnih. 2020. Monte Carlo Gradient Estimation in Machine Learning. The Journal of Machine Learning Research 21, 1 (2020), 5183–5244.Google ScholarDigital Library
- Corrado Monti, Marco Pangallo, Gianmarco De Francisci Morales, and Francesco Bonchi. 2023. On learning agent-based models from data. Scientific Reports 13, 1 (2023), 9268.Google ScholarCross Ref
- Mijung Park, Wittawat Jitkrittum, and Dino Sejdinovic. 2016. K2-ABC: Approximate Bayesian Computation with Kernel Embeddings. In Proceedings of the 19th International Conference on Artificial Intelligence and Statistics(Proceedings of Machine Learning Research, Vol. 51), Arthur Gretton and Christian C. Robert (Eds.). PMLR, Cadiz, Spain, 398–407. https://proceedings.mlr.press/v51/park16.htmlGoogle Scholar
- Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. Curran Associates Inc., Red Hook, NY, USA.Google ScholarDigital Library
- Donovan Platt. 2020. A comparison of economic agent-based model calibration methods. Journal of Economic Dynamics and Control 113 (2020), 103859.Google ScholarCross Ref
- Donovan Platt. 2021. Bayesian Estimation of Economic Simulation Models using Neural Networks. Computational Economics (2021), 1–52.Google Scholar
- Felix Prenzel, Rama Cont, Mihai Cucuringu, and Jonathan Kochems. 2022. Dynamic Calibration of Order Flow Models with Generative Adversarial Networks. In Proceedings of the Third ACM International Conference on AI in Finance (New York, NY, USA) (ICAIF ’22). Association for Computing Machinery, New York, NY, USA, 446–453. https://doi.org/10.1145/3533271.3561777Google ScholarDigital Library
- Arnau Quera-Bofarull, Ayush Chopra, Anisoara Calinescu, Michael Wooldridge, and Joel Dyer. 2023. Bayesian calibration of differentiable agent-based models. ICLR Workshop on AI for Agent-based Modelling (2023).Google Scholar
- Arnau Quera-Bofarull, Joel Dyer, Anisoara Calinescu, J. Doyne Farmer, and Michael Wooldridge. 2023. BlackBIRDS: Black-Box Inference foR Differentiable Simulators. Journal of Open Source Software 8, 89 (2023), 5776. https://doi.org/10.21105/joss.05776Google ScholarCross Ref
- Arnau Quera-Bofarull, Joel Dyer, Anisoara Calinescu, and Michael Wooldridge. 2023. Some challenges of calibrating differentiable agent-based models. ICML Workshop on Differentiable Almost Everything: Differentiable Relaxations, Algorithms, Operators, and Simulators (2023).Google Scholar
- Danilo Rezende and Shakir Mohamed. 2015. Variational Inference with Normalizing Flows. In Proceedings of the 32nd International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 37), Francis Bach and David Blei (Eds.). PMLR, Lille, France, 1530–1538. https://proceedings.mlr.press/v37/rezende15.htmlGoogle Scholar
- Victor Storchan, Svitlana Vyetrenko, and Tucker Balch. 2021. Learning who is in the market from time series: market participant discovery through adversarial calibration of multi-agent simulators. arXiv preprint arXiv:2108.00664 (2021).Google Scholar
- Esteban G Tabak and Cristina V Turner. 2013. A family of nonparametric density estimation algorithms. Communications on Pure and Applied Mathematics 66, 2 (2013), 145–164.Google ScholarCross Ref
- Esteban G Tabak and Eric Vanden-Eijnden. 2010. Density estimation by dual ascent of the log-likelihood. Communications in Mathematical Sciences 8, 1 (2010), 217–233.Google ScholarCross Ref
- Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning 8 (1992), 229–256.Google Scholar
Index Terms
- Gradient-Assisted Calibration for Financial Agent-Based Models
Recommendations
Assisted Parameter and Behavior Calibration in Agent-Based Models with Distributed Optimization
Advances in Practical Applications of Agents, Multi-Agent Systems, and Trustworthiness. The PAAMS CollectionAbstractAgent-based modeling (ABM) has many applications in the social sciences, biology, computer science, and robotics. One of the most important and challenging phases in agent-based model development is the calibration of model parameters and agent ...
Adaptive agents modeling and simulation in artificial financial market
SummerSim '19: Proceedings of the 2019 Summer Simulation ConferenceAgent-based artificial financial market generally models the agents' interactions and their impact on the price dynamics and the market stability. We propose to conceive and simulate a new approach of traders' models that adapt their behaviors to the ...
Human and Artificial Agents in a Crash-Prone Financial Market
We introduce human traders into an agent based financial market simulation prone to bubbles and crashes. We find that human traders earn lower profits overall than do the simulated agents ("robots") but earn higher profits in the most crash-intensive ...
Comments