Summary
The paper extends the ergodic theorems of information theory (Shannon-MacMillan-Breiman theorems) to spaces with an infinite invariant measure. An L 1 difference theorem and a pointwisa ratio theorem are proved, for the information of spreading partitions. For the validity of the theorems it is assumed that the supremum f * of the conditional information given the increasing “past” is integrable. Simple necessary and sufficient conditions for the integrability of f * are obtained in special cases: If the initial partition is composed of one state of a null-recurrent Markov chain, then f * is integrable if and only if the partition of this state according to the first return times has finite entropy.
Article PDF
Similar content being viewed by others
References
Billingsley, P.: Ergodic theory and information. New York: Wiley 1965.
Breiman, L.: The individual ergodic theorem of information theory. Ann. math. Statistics 28, 809–811 (1957); correction note, ibid. 31, 809–810 (1960).
Chacon, R. V.: Identification of the limit of operator averages. J. Math. Mech. 11, 957 to 961 (1962).
—, and D. S. Ornstein: A general ergodic theorem. Illinois J. Math. 4, 153–160 (1960).
Chung, K. L.: A note on the ergodic theorem of information theory. Ann. math. Statistics 32, 612–614 (1961).
—: Markov chains with stationary transition probabilities. Berlin-Heidelberg-New York: Springer 1967 (second edition).
Dowker, Y. N.: On measurable transformations in finite measure spaces. Ann. of Math., II. Ser. 62, 504–516 (1955).
Halmos, P. R.: Lectures on ergodic theory. Tokyo: The Mathematical Society of Japan 1956.
- Entropy in ergodic theory. Mimeographed notes. The University of Chicago 1959.
Harris, T. E., and H. Robbins: Ergodic theory of Markov chains admitting an infinite invariant measure. Proc. nat. Acad. Sci. U.S.A. 39, 860–864 (1953).
Hopf, E.: Ergodentheorie. Berlin: J. Springer 1937 (reprinted New York: Chelsea Publishing Co. 1948).
Ionescu Tulcea, A.: Contributions to information theory for abstract alphabets. Ark. Mat. 4, 235–247 (1961).
Karlin, S.: A first course in stochastic processes. New York: Academic Press 1966.
Klimko, E. M., and L. Sucheston: An operator ergodic theorem for sequences of functions. Proc. Amer. math. Soc., to appear.
Krengel, U.: On the global limit behavior of Markov chains and of general nonsingular Markov processes. Z. Wahrscheinlichkeitstheorie verw. Geb. 6, 302–316 (1966).
—: Entropy of conservative transformations. Z. Wahrscheinlichkeitstheorie verw. Geb. 7, 161–181 (1967).
McMillan, B.: The basic theorems of information theory. Ann. math. Statistics 24, 196–219 (1953).
Neveu, J.: Mathematical Foundations of the Calculus of Probabilities. San Francisco: Holden Day 1965.
Parry, W.: On the ergodic theorem of information theory without invariant measure. Proc. London math. Soc., III. Ser. 13, 605–612 (1963).
Sucheston, L.: On the ergodic theorem for positive operators I. Z. Wahrscheinlichkeitstheorie verw. Geb. 8, 1–11 (1967).
Author information
Authors and Affiliations
Additional information
Research of both authors was supported by the National Science Foundation (U. S. A.), under Grant GP 7693.
Rights and permissions
About this article
Cite this article
Klimko, E.M., Sucheston, L. On convergence of information in spaces with infinite invariant measure. Z. Wahrscheinlichkeitstheorie verw Gebiete 10, 226–235 (1968). https://doi.org/10.1007/BF00536276
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF00536276