Learning nonequilibrium control forces to characterize dynamical phase transitions

Jiawei Yan (闫嘉伟), Hugo Touchette, and Grant M. Rotskoff

Phys. Rev. E 105, 024115 – Published 9 February 2022

Abstract

Sampling the collective, dynamical fluctuations that lead to nonequilibrium pattern formation requires probing rare regions of trajectory space. Recent approaches to this problem, based on importance sampling, cloning, and spectral approximations, have yielded significant insight into nonequilibrium systems but tend to scale poorly with the size of the system, especially near dynamical phase transitions. Here we propose a machine learning algorithm that samples rare trajectories and estimates the associated large deviation functions using a many-body control force by leveraging the flexible function representation provided by deep neural networks, importance sampling in trajectory space, and stochastic optimal control theory. We show that this approach scales to hundreds of interacting particles and remains robust at dynamical phase transitions.

Received 12 July 2021
Revised 18 November 2021
Accepted 24 January 2022

DOI:https://doi.org/10.1103/PhysRevE.105.024115

Physics Subject Headings (PhySH)

Dynamical phase transitions Nonequilibrium statistical mechanics

Living matter & active matter

Large deviation & rare event statistics

Statistical Physics & Thermodynamics

Authors & Affiliations

Jiawei Yan (闫嘉伟)¹, Hugo Touchette², and Grant M. Rotskoff^1,*

¹Department of Chemistry, Stanford University, Stanford, California 94305, USA
²Department of Mathematical Sciences, Stellenbosch University, Stellenbosch 7600, South Africa

^*To whom correspondence should be addressed: rotskoff@stanford.edu

Article Text (Subscription Required)

Click to Expand

References (Subscription Required)

Click to Expand

Issue

Vol. 105, Iss. 2 — February 2022

Reuse & Permissions

Access Options

Author publication services for translation and copyediting assistance advertisement

Images

Figure 1
Results of the diffusion in the quartic potential. (a) SCGF for the observable (13) for decreasing temperatures $ε$ . The solid line represents the exact solution (14) in the zero-noise limit. The inserted figure shows the second derivative of the SCGF for $ε = 0.01$ , confirming a second order dynamical phase transition. In this example the hidden layer dimension and number of layers of the neural network are 50 and 2, respectively. A smaller hidden layer dimension such as 10 is able to generate results with similar accuracy but requires longer time for training. We first select 11 $λ$ values uniformly from $- 1$ to 1, where each $λ$ contains 20 replica. At each training step, a total number of 220 trajectories with $T = 5$ are generated with the Euler-Maruyama method ( $d t = 10^{- 3}$ ). The neural network is updated through standard back propagation where the gradient is computed by the adaptive gradient algorithm method (AdaGrad) with a learning rate $5 \times 10^{- 3}$ . The resulting estimation of the SCGF is then refined by changing $λ$ and simulating the resulting driven process. (b) Illustration of the concurrent training with $ε = 0.01$ . Each line corresponds to the evolution of the cost function with a specific $λ$ .
Reuse & Permissions
Figure 2
Comparison of our machine learning approach with the cloning method with feedback. We computed the SCGF for the 1D diffusion system described as in (12) with $ε = 0.01$ . $N$ is the batch size (for the ML approach) or the number of clones (for the cloning method). For the cloning method, we set the trajectory length to be $T = 0.3$ ( $d t = 10^{- 3}$ ), and updated the control potential every 75 steps so the time interval between updating is 22.5. All results were computed in the same personal computer, using CPU only.
Reuse & Permissions
Figure 3
Small entropy production indicates particle clustering. (a) The average entropy production at given $λ$ for different system sizes: $N = 40$ (blue lines, left), 80 (red lines, middle), and 200 (yellow lines, right). (b) The corresponding rate functions ( $N = 40$ bottom, $N = 80$ middle, $N = 200$ top). The inserted figures show snap shots of typical behaviors in the high entropy production phase ( $λ = 0$ ) and low entropy production phase ( $λ = - 0.05$ ), respectively. The arrow represents the direction of motion. In this example the hidden layer dimension and number of layers of the neural network are 1000 and 6 respectively. The batch sizes for results of 40 and 80 particles are 75, and 20 for the 200 particle case. $T = 0.1$ and $d t = 10^{- 4}$ . The density of particles throughout all three cases is $ρ = N / L^{2} = 0.1$ where $L$ is the length of the simulation box. To avoid boundary effect, the input of the neural network is not the absolute position of particles but its relative position to a particular one, i.e., $u ({X^{(i)} - X^{(0)}})$ instead of $u ({X^{i}})$ . This step is essential otherwise the learned control force would force particles to the boundary.
Reuse & Permissions
Figure 4
The estimate of the SCGF and corresponding changes of average entropy production for $N = 40$ with/without replica exchange. The blue line (no kink) corresponds to the results with replica exchange and the red line shows the results without.
Reuse & Permissions
Figure 5
Variance of the estimator. We illustrate the scaling property of the variance of our estimator (B1) using the small noise example. By fixing $λ = 1$ and $ε = 0.01$ , the neural network is trained with (a) a fixed batch size $N = 300$ with various trajectory length $t$ from 1 to 20, or (b) a fixed $t = 5$ and various batch size from 10 to 400. The neural networks in all cases are trained for more than 400 steps, and the variance is estimated by collecting the data from the last 100 steps. The insert figures show the relative absolute error $| \hat{ψ} (λ) - ψ (λ) | / ψ (λ)$ . The gray dashed line represents a $- 1$ slope.
Reuse & Permissions

Physical Review E

covering statistical, nonlinear, biological, and soft matter physics