Marcin Sabok
Skip Abstract Section
Abstract
We make a formal analogy between random sampling and fresh name generation. We show that quasi-Borel spaces, a model for probabilistic programming, can soundly interpret the ν-calculus, a calculus for name generation. Moreover, we prove that this semantics is fully abstract up to first-order types. This is surprising for an ‘off-the-shelf’ model, and requires a novel analysis of probability distributions on function spaces. Our tools are diverse and include descriptive set theory and normal forms for the ν-calculus.
- S. Abramsky, D. R. Ghica, A. S. Murawski, C.-H. L. Ong, and I. D. B. Stark. 2004. Nominal games and full abstraction for the nu-Calculus. In Proc. LICS 2004. 150-159.Google Scholar
Cross Ref
- Robert J. Aumann. 1961. Borel structures for function spaces. Illinois Journal of Mathematics 5 ( 1961 ).Google Scholar
- Giorgio Bacci, Robert Furber, Dexter Kozen, Radu Mardare, Prakash Panangaden, and Dana Scott. 2018. Boolean-valued semantics for stochastic lambda-calculus. In Proc. LICS 2018.Google Scholar
- Nick Benton and Vasileios Koutavas. 2008. A Mechanized Bisimulation for the Nu-Calculus. Technical Report MSR-TR-2008-129. Microsoft Research.Google Scholar
- Chung chieh Shan and Norman Ramsey. 2017. Exact Bayesian inference by symbolic disintegration. In Proc. POPL 2017.Google ScholarDigital Library
- Fredrik Dahlqvist and Dexter Kozen. 2020. Semantics of higher-order probabilistic programs with conditioning. Proc. ACM Program. Lang. 4, POPL, Article 19 ( Dec. 2020 ).Google ScholarDigital Library
- Ugo Dal Lago and Naohiko Hoshino. 2019. The geometry of Bayesian programming. In Proc. LICS 2019.Google ScholarCross Ref
- Thomas Ehrhard, Michele Pagani, and Christine Tasson. 2018. Measurable cones and stable, measurable functions. In Proc. POPL 2018.Google Scholar
- Thomas Ehrhard, Charistine Tasson, and Michele Pagani. 2014. Probabilistic coherence spaces are fully abstract for probabilistic PCF. In Proc. POPL 2014. 309-320.Google Scholar
- M.H. Escardo. 2009. Semi-decidability of may, must and probabilistic testing in a higher-type setting. In Proc. MFPS 2009.Google ScholarDigital Library
- Tobias Fritz. 2020. A synthetic approach to Markov kernels, conditional independence and theorems on suficient statistics. Adv. Math. 370, 107239 (Aug. 2020 ).Google ScholarCross Ref
- T. Gehr, S. Stefen, and M. T. Vechev. 2020. PSI: exact inference for higher-order probabilistic programs. In Proc. PLDI 2020.Google ScholarDigital Library
- Michèle Giry. 1982. A categorical approach to probability theory. In Categorical Aspects of Topology and Analysis. Lecture Notes in Mathematics, Vol. 915. Springer, 68-85.Google Scholar
- Chris Heunen, Ohad Kammar, Sam Staton, and Hongseok Yang. 2017. A Convenient Category for Higher-Order Probability Theory. In Proceedings of the 32nd Annual ACM/IEEE Symposium on Logic in Computer Science (Reykjavík, Iceland) (LICS '17). IEEE Press, Article 77, 12 pages.Google ScholarDigital Library
- Daniel Huang, Greg Morrisett, and Bas Spitters. 2018. An application of computable distributions to the semantics of probabilistic programs. arxiv:1806.07966.Google Scholar
- A. Jefrey and J. Rathke. 1999. Towards a theory of bisimulation for local names. In Proc. LICS 1999.Google Scholar
- Paul Jung, Jiho Lee, Sam Staton, and Hongseok Yang. 2020. A generalization of hierarchical exchangeability on trees to directed acyclic graphs. Annales Henri Lebesgue ( 2020 ). to appear.Google Scholar
- Olav Kallenberg. 2002. Foundations of Modern Probability. Springer, New York.Google Scholar
- Ohad Kammar and Gordon D. Plotkin. 2012. Algebraic foundations for efect-dependent optimisations. In Proc. POPL 2012. 349-360.Google Scholar
- Alexander Kechris. 1987. Classical Descriptive Set Theory. Springer.Google Scholar
- Anders Kock. 2011. Commutative monads as a theory of distributions. Theory and Applications of Categories 26 (Aug. 2011 ).Google Scholar
- Dexter Kozen. 1981. Semantics of probabilistic programs. J. Comput. Syst. Sci. 22, 3 ( 1981 ), 328-350.Google ScholarCross Ref
- Ugo Dal Lago and Francesco Gavazzo. 2019. On bisimilarity in lambda calculi with continuous probabilistic choice. Electron. Notes Theoret. Comput. Sci. 347 ( 2019 ), 121-141. Proc. MFPS 2019.Google Scholar
- James Laird. 2004. A game semantics of local names and good variables. In Proc. FOSSACS 2004. 289-303.Google ScholarCross Ref
- J Lambek and P J Scott. 1988. Introduction to higher order categorical logic. CUP.Google Scholar
- Alexander K. Lew, Marco F. Cusumano-Towner, Benjamin Sherman, Michael Carbin, and Vikash K. Mansinghka. 2019. Trace types and denotational semantics for sound programmable inference in probabilistic languages. Proc. ACM Program. Lang. 4, POPL, Article 19 ( Dec. 2019 ).Google Scholar
- Robin Milner. 1999. Communicating and mobile systems-the Pi-calculus. CUP.Google Scholar
- Eugenio Moggi. 1991. Notions of computation and monads. Inform. Comput. 93, 1 ( 1991 ), 55-92.Google Scholar
- Andrzej S. Murawski and Nikos Tzevelekos. 2016. Nominal game semantics. Found. Trends Program. Lang. ( 2016 ).Google Scholar
- Lawrence M. Murray and Thomas B. Schön. 2018. Automated learning with a probabilistic programming language: Birch. Annual Reviews in Control 46 ( 2018 ), 29-43.Google Scholar
- Aditya Nori, Chung-Kil Hur, Sriram Rajamani, and Selva Samuel. 2014. R2: An eficient MCMC sampler for probabilistic programs. In Proc. AAAI 2014.Google Scholar
- Martin Odersky. 1994. A Functional Theory of Local Names. In Proc. POPL 1994. 48-59.Google Scholar
- Peter Orbanz and Daniel M. Roy. 2015. Bayesian models of graphs, arrays and other exchangeable random structures. IEEE Trans. Pattern Anal. Mach. Intell. 2 ( 2015 ), 437-461.Google Scholar
- Hugo Paquet and Glynn Winskel. 2018. Continuous probability distributions in concurrent games. In Proc. MFPS 2018. 321-344.Google ScholarCross Ref
- Arthur J. Parzygnat. 2020. Inverses, disintegrations, and Bayesian inversion in quantum Markov categories. arXiv: 2001.08375.Google Scholar
- Evan Patterson. 2020. The algebra and machine representation of statistical models. Ph.D. Dissertation. Stanford University Department of Statistics.Google Scholar
- Andrew M. Pitts. 2013. Nominal Sets: Names and Symmetry in Computer Science. Cambridge University Press.Google ScholarCross Ref
- Andrew M. Pitts and Ian Stark. 1993. Observable properties of higher order functions that dynamically create local names, or: What's new?. In Proc. MFCS 1993 (Lecture Notes in Computer Science). 122-141.Google Scholar
- G. D. Plotkin. 1973. Lambda-definability and logical relations. Technical Report SAI-RM-4. School of A.I., Univ.of Edinburgh.Google Scholar
- David Pollard. 2001. A users' guide to measure-theoretic probability. CUP.Google Scholar
- Daniel Roy, Vikash Mansinghka, Noah Goodman, and Josh Tenenbaum. 2008. A stochastic programming perspective on nonparametric Bayes. In Proc. ICML Workshop on Nonparametric Bayes.Google Scholar
- Tetsuya Sato, Alejandro Aguirre, Gilles Barthe, Marco Gaboardi, Deepak Garg, and Justin Hsu. 2019. Formal verification of higher-order probabilistic programs: reasoning about approximation, convergence, bayesian inference, and optimization. Proc. ACM Program. Lang. 3, POPL, Article 38 ( Jan. 2019 ), 30 pages.Google ScholarDigital Library
- Adam Ścibior, Ohad Kammar, Matthijs Vákár, Sam Staton, Hongseok Yang, Yufei Cai, Klaus Ostermann, Sean Moss, Chris Heunen, and Zoubin Ghahramani. 2017. Denotational validation of higher-order Bayesian inference. Proceedings of the ACM on Programming Languages 2 ( Nov. 2017 ).Google Scholar
- Dan Shiebler. 2020. Categorical stochastic processes and likelihood. arXiv: 2005.04735.Google Scholar
- Alex Simpson. 2017. Probability Sheaves and the Giry Monad. In Proc. CALCO 2017.Google Scholar
- Shashi M. Srivastava. 1998. A Course on Borel Sets. Springer, New York.Google Scholar
- Ian Stark. 1994. Names and Higher-Order Functions. Ph.D. Dissertation. University of Cambridge. Also available as Technical Report 363, University of Cambridge Computer Laboratory.Google Scholar
- Ian Stark. 1996. Categorical models for local names. LISP and Symbolic Computation 9, 1 (Feb. 1996 ), 77-107.Google ScholarCross Ref
- Sam Staton. 2010. Completeness for algebraic theories of local state. In Proc. FOSSACS 2010. 48-63.Google ScholarDigital Library
- Sam Staton. 2017. Commutative semantics for probabilistic programming. In Proc. ESOP 2017.Google ScholarDigital Library
- Sam Staton, Dario Stein, Hongseok Yang, Nathanael L. Ackerman, Cameron E. Freer, and Daniel M. Roy. 2018. The Beta-Bernoulli process and algebraic efects. Proc. ICALP 2018.Google Scholar
- S. Staton, H. Yang, N. L.. Ackerman, C. Freer, and D. Roy. 2017. Exchangeable random process and data abstraction. In Proc. PPS 2017.Google Scholar
- Sam Staton, Hongseok Yang, Frank Wood, Chris Heunen, and Ohad Kammar. 2016. Semantics for probabilistic programming: higher-order functions, continuous distributions, and soft constraints. In Proc. LICS 2016. 525-534.Google ScholarDigital Library
- Eijiro Sumii and Benjamin C. Pierce. 2003. Logical relations for encryption. J. Comput. Secur. 11, 4 ( 2003 ), 521-554.Google Scholar
- Nikos Tzevelekos. 2008. Nominal game semantics. Ph.D. Dissertation. Oxford University Computing Laboratory.Google Scholar
- Jan-Willem van de Meent, Brooks Paige, Hongseok Yang, and Frank Wood. 2018. An introduction to probabilistic programming. arxiv:1809.10756.Google Scholar
- Alexander Vandenbroucke and Tom Schrijvers. 2020. P NK: functional probabilistic NetKAT. In Proc. POPL 2020.Google Scholar
- Yu Zhang and David Nowak. 2003. Logical relations for dynamic name creation. In Proc. CSL 2003. 575-588.Google ScholarCross Ref
Index Terms
- Probabilistic programming semantics for name generation
Recommendations
A domain theory for statistical probabilistic programming
We give an adequate denotational semantics for languages with recursive higher-order types, continuous probability distributions, and soft constraints. These are expressive languages for building Bayesian models of the kinds used in computational ...
Program logic for higher-order probabilistic programs in Isabelle/HOL
AbstractThe verification framework PPV (Probabilistic Program Verification) verifies functional probabilistic programs supporting higher-order functions, continuous distributions, and conditional inference. PPV is based on the theory of quasi-...
Highlights- We formalize quasi-Borel spaces in Isabelle/HOL.
- We formalize the probability ...
A Denotational Semantics for Low-Level Probabilistic Programs with Nondeterminism
AbstractProbabilistic programming is an increasingly popular formalism for modeling randomness and uncertainty. Designing semantic models for probabilistic programs has been extensively studied, but is technically challenging. Particular complications ...
Comments