Abstract
We demonstrate how path integrals often used in problems of theoretical physics can be adapted to provide a machinery for performing Bayesian inference in function spaces. Such inference comes about naturally in the study of inverse problems of recovering continuous (infinite dimensional) coefficient functions from ordinary or partial differential equations, a problem which is typically ill-posed. Regularization of these problems using \(L^2\) function spaces (Tikhonov regularization) is equivalent to Bayesian probabilistic inference, using a Gaussian prior. The Bayesian interpretation of inverse problem regularization is useful since it allows one to quantify and characterize error and degree of precision in the solution of inverse problems, as well as examine assumptions made in solving the problem—namely whether the subjective choice of regularization is compatible with prior knowledge. Using path-integral formalism, Bayesian inference can be explored through various perturbative techniques, such as the semiclassical approximation, which we use in this manuscript. Perturbative path-integral approaches, while offering alternatives to computational approaches like Markov-Chain-Monte-Carlo (MCMC), also provide natural starting points for MCMC methods that can be used to refine approximations. In this manuscript, we illustrate a path-integral formulation for inverse problems and demonstrate it on an inverse problem in membrane biophysics as well as inverse problems in potential theories involving the Poisson equation.
Similar content being viewed by others
Notes
We will use Greek letters to denote fields
References
Alves, C., Colaço, M., Leitão, V., Martins, N., Orlande, H., Roberty, N.: Recovering the source term in a linear diffusion problem by the method of fundamental solutions. Inverse Probl. Sci. Eng. 16(8), 1005–1021 (2008)
Anzengruber, S.W., Ramlau, R.: Morozov’s discrepancy principle for Tikhonov-type functionals with nonlinear operators. Inverse Probl. 26(2), 025001 (2010)
Bertero, M., De Mol, C., Viano, G.: The stability of inverse problems. In: Baltes, H.P. (ed.) Inverse Scattering Problems in Optics, pp. 161–214. Springer, Berlin (1980)
Bui-Thanh, T., Ghattas, O., Martin, J., Stadler, G.: A computational framework for infinite-dimensional bayesian inverse problems part i: the linearized case, with application to global seismic inversion. SIAM J. Sci. Comput. 35(6), A2494–A2523 (2013)
Chow, C.C., Buice, M.A.: Path integral methods for stochastic differential equations. arXiv preprint arXiv:10095966 (2010)
Cotter, S., Dashti, M., Robinson, J., Stuart, A.: Bayesian inverse problems for functions and applications to fluid mechanics. Inverse Probl. 25, 115008 (2009)
Dashti, M., Law, K.J., Stuart, A.M., Voss, J.: Map estimators and their consistency in Bayesian nonparametric inverse problems. Inverse Probl. 29(9), 095017 (2013)
Engl, H.W., Kunisch, K., Neubauer, A.: Convergence rates for tikhonov regularisation of non-linear ill-posed problems. Inverse Probl. 5(4), 523 (1989)
Engl, H., Kunisch, K., Neubauer, A.: Convergence rates for Tikhonov regularisation of non-linear ill-posed problems. Inverse Probl. 5(4), 523 (1999)
Engl, H., Flamm, C., Kügler, P., Lu, J., Müller, S., Schuster, P.: Inverse problems in systems biology. Inverse Probl. 25(12), 123014 (2009)
Enßlin, T.A., Frommert, M., Kitaura, F.S.: Information field theory for cosmological perturbation reconstruction and nonlinear signal analysis. Phys. Rev. D 80(10), 105005 (2009)
Evans, A., Turner, M., Sens, P.: Interactions between proteins bound to biomembranes. Phys. Rev. E 67(4), 041907 (2003)
Farmer, C.: Bayesian field theory applied to scattered data interpolation and inverse problems. In: Iske, A., Levesley, J. (eds.) Algorithms for Approximation, pp. 147–166. Springer, Berlin (2007)
Feynman, R.P., Hibbs, A.R.: Quantum Mechanics and Path Integrals: Emended Edition. DoverPublications. com (2012)
Graham, R.: Path integral formulation of general diffusion processes. Z. für Phys. B 26(3), 281–290 (1977)
Hänggi, P.: Path integral solutions for non-markovian processes. Z. für Phys. B 75(2), 275–281 (1989)
Heller, E.J.: Frozen gaussians: a very simple semiclassical approximation. J. Chem. Phys. 75, 2923 (1981)
Heuett, W.J., Miller III, B.V., Racette, S.B., Holloszy, J.O., Chow, C.C., Periwal, V.: Bayesian functional integral method for inferring continuous data from discrete measurements. Biophys. J. 102(3), 399–406 (2012)
Hoang, V.H., Law, K.J., Stuart, A.M.: Determining white noise forcing from eulerian observations in the navier stokes equation. arXiv preprint arXiv:13034677 (2013)
Hohage, T., Pricop, M.: Nonlinear Tikhonov regularization in Hilbert scales for inverse boundary value problems with random noise. Inverse Probl. Imaging 2, 271–290 (2008)
Hon, Y., Li, M., Melnikov, Y.: Inverse source identification by Green’s function. Eng. Anal. Bound. Elem. 34(4), 352–358 (2010)
Hörmander, L.: The Analysis of Linear Partial Differential Operators III: Pseudo-Differential Operators. Springer, New York (2007)
Itô, K.: Wiener integral and feynman integral. In: Proceedings of the 4th Berkeley Symposium on Mathematical Statistics and Probability, vol. 2, pp. 227–238 (1961)
Kardar, M.: Statistical Physics of Fields. Cambridge University Press, Cambridge (2007)
Lasanen, S.: Measurements and infinite-dimensional statistical inverse theory. PAMM 7(1), 1080101–1080102 (2007)
Lasanen, S.: Non-gaussian statistical inverse problems. Part i: posterior distributions. Inverse Probl. Imaging 6(2), 215–266 (2012a)
Lasanen, S.: Non-gaussian statistical inverse problems. Part ii: posterior convergence for approximated unknowns. Inverse Probl. Imaging 6(2), 267–287 (2012b)
Lemm, J.C.: Bayesian field theory: Nonparametric approaches to density estimation, regression, classification, and inverse quantum problems. arXiv preprint physics/9912005 (1999)
Lieberman, C., Willcox, K., Ghattas, O.: Parameter and state model reduction for large-scale statistical inverse problems. SIAM J. Sci. Comput. 32(5), 2523–2542 (2010)
Liu, J.S.: Monte Carlo Strategies in Scientific Computing. Springer, New York (2008)
Martin, J., Wilcox, L.C., Burstedde, C., Ghattas, O.: A stochastic newton MCMC method for large-scale statistical inverse problems with application to seismic inversion. SIAM J. Sci. Comput. 34(3), A1460–A1487 (2012)
Neubauer, A.: Tikhonov regularisation for non-linear ill-posed problems: optimal convergence rates and finite-dimensional approximation. Inverse Probl. 5(4), 541 (1999)
O’Hagan, A., Forster, J., Kendall, M.G.: Bayesian Inference. Arnold, London (2004)
Oppermann, N., Robbers, G., Enßlin, T.A.: Reconstructing signals from noisy data with unknown signal and noise covariance. Phys. Rev. E 84(4), 041118 (2011)
Peliti, L.: Path integral approach to birth-death processes on a lattice. J. de Phys. 46(9), 1469–1483 (1985)
Pesquera, L., Rodriguez, M., Santos, E.: Path integrals for non-Markovian processes. Phys. Lett. A 94(6), 287–289 (1983)
Petra, N., Martin, J., Stadler, G., Ghattas, O.: A computational framework for infinite-dimensional bayesian inverse problems: part ii. Stochastic newton mcmc with application to ice sheet flow inverse problems. arXiv preprint arXiv:13086221 (2013)
Potsepaev, R., Farmer, C.: Application of stochastic partial differential equations to reservoir property modelling. In: 12th European Conference on the Mathematics of Oil Recovery (2010)
Quinn, J.C., Abarbanel, H.D.: State and parameter estimation using monte carlo evaluation of path integrals. Q. J. R. Meteorol. Soc. 136(652), 1855–1867 (2010)
Quinn, J.C., Abarbanel, H.D.: Data assimilation using a gpu accelerated path integral monte carlo approach. J. Comput. Phys. 230(22), 8168–8178 (2011)
Scherzer, O.: The use of morozov’s discrepancy principle for Tikhonov regularization for solving nonlinear ill-posed problems. Computing 51(1), 45–60 (1993)
Schwab, C., Stuart, A.M.: Sparse deterministic approximation of bayesian inverse problems. Inverse Probl. 28(4), 045003 (2012)
Stuart, A.: Inverse problems: a Bayesian perspective. Acta Numer. 19(1), 451–559 (2010)
Tikhonov, A.N.: On the stability of inverse problems. Dokl. Akad. Nauk. SSSR 39, 195–198 (1943)
Zee, A.: Quantum Field Theory in a Nutshell. Universities Press, New York (2005)
Acknowledgments
This material is based upon work supported by the National Science Foundation under Agreement No. 0635561. JC and TC also acknowledge support from the National Science Foundation through grant DMS-1021818, and from the Army Research Office through grant 58386MA. VS acknowledges support from UCLA startup funds.
Author information
Authors and Affiliations
Corresponding author
Appendix: Functional Taylor Approximations for the Dielectric Field Problem
Appendix: Functional Taylor Approximations for the Dielectric Field Problem
We wish to expand the Hamiltonian
about its extrema \(\epsilon ^*\). We take variations with respect to \(\epsilon (\mathbf {x})\) to calculate its first functional derivative,
Let us define the quantities
Through direct differentiation we find that
Integrating in \(\mathbf {x}\):
We shift \(\phi (\cdot )\rightarrow \phi (\mathbf {x})\), and integrate-by-parts to find
Note that all boundary terms disappear since we can take \(\phi \) to disappear on the boundary. With \(I_1\) computed, we find
Taken to two terms in the series expansion for \(\varphi \), the first variation is
To calculate the second-order term in the Taylor-expansion, we take another variation. Truncated at two terms in the expansion for \(\varphi \):
where after canceling like terms,
It is using this expression that we can construct an approximate probability density for our field \(\epsilon \).
Rights and permissions
About this article
Cite this article
Chang, J.C., Savage, V.M. & Chou, T. A Path-Integral Approach to Bayesian Inference for Inverse Problems Using the Semiclassical Approximation. J Stat Phys 157, 582–602 (2014). https://doi.org/10.1007/s10955-014-1059-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10955-014-1059-y