Information and Entropy in Physical Systems

Abstract

The Landauer Principle connects the information theoretic notion of entropy to the physics of statistical mechanics. When a physical system performs a logical operation that erases or loses information, without a copy being preserved, it must transfer a minimum amount of heat, $k_B T \log (2)$, to the environment. How can there be such a connection between the abstract idea of information and the concrete physical reality of heat? To address this question, we adopt the Jaynes approach of grounding statistical mechanics in the Shannon notion of entropy. Probability is a quantification of incomplete information. Entropy should not be conceived in terms of disorder, but rather as a measure on a probability distribution that characterizes the amount of missing information the distribution represents. The thermodynamic entropy is a special case of the Shannon entropy applied to a physical system in equilibrium with a heat bath so that its average energy is fixed. The thermal probability distribution is obtained by maximizing the Shannon entropy, subject to the physical constraints of the problem. It is then possible to naturally extend this description to include a physical memory device, which must be in a nonequilibrium long-lived metastable state. We can then explicitly demonstrate how the requirement for a fundamental minimum energy dissipation is tied to erasure of an unknown bit. Both classical and quantum cases are considered. We show that the classical thermodynamic entropy is in some situations best matched in quantum mechanics, not by the von Neumann entropy, but by a perhaps less familiar quantity—the quantum entropy of outcomes. The case of free expansion of an ideal quantum gas is examined in this context.