In this second part of our two-part paper we review and analyse attempts since 1950 to use information theoretic notions to exorcise Maxwell’s Demon. We argue through a simple dilemma that these attempted exorcisms are ineffective, whether they follow Szilard in seeking a compensating entropy cost in information acquisition or Landauer in seeking that cost in memory erasure. In so far as the Demon is a thermodynamic system already governed by the Second Law, no further supposition about information and entropy (...) is needed to save the Second Law. In so far as the Demon fails to be such a system, no supposition about the entropy cost of information acquisition and processing can save the Second Law from the Demon. (shrink)

Scientific knowledge grows at a phenomenal pace--but few books have had as lasting an impact or played as important a role in our modern world as The Mathematical Theory of Communication, published originally as a paper on communication theory more than fifty years ago. Republished in book form shortly thereafter, it has since gone through four hardcover and sixteen paperback printings. It is a revolutionary work, astounding in its foresight and contemporaneity. The University of Illinois Press is pleased and honored (...) to issue this commemorative reprinting of a classic. (shrink)

Consider a gas confined to the left half of a container. Then remove the wall separating the two parts. The gas will start spreading and soon be evenly distributed over the entire available space. The gas has approached equilibrium. Why does the gas behave in this way? The canonical answer to this question, originally proffered by Boltzmann, is that the system has to be ergodic for the approach to equilibrium to take place. This answer has been criticised on different grounds (...) and is now widely regarded as flawed. In this paper we argue that these criticisms have dismissed Boltzmann’s answer too quickly and that something almost like Boltzmann’s answer is true: the approach to equilibrium takes place if the system is epsilon-ergodic, i.e. ergodic on the entire accessible phase space except for a small region of measure epsilon. We introduce epsilon-ergodicity and argue that relevant systems in statistical mechanics are indeed espsilon-ergodic. (shrink)

The mathematical theory of communication.

Gases reach equilibrium when left to themselves. Why do they behave in this way? The canonical answer to this question, originally proffered by Boltzmann, is that the systems have to be ergodic. This answer has been criticised on different grounds and is now widely regarded as flawed. In this paper we argue that some of the main arguments against Boltzmann's answer, in particular, arguments based on the KAM-theorem and the Markus-Meyer theorem, are beside the point. We then argue that something (...) close to Boltzmann's original proposal is true for gases: gases behave thermodynamic-like if they are epsilon-ergodic, i.e., ergodic on the entire accessible phase space except for a small region of measure epsilon. This answer is promising because there are good reasons to believe that relevant systems in statistical mechanics are epsilon-ergodic. (shrink)

This article distinguishes two different senses of information-theoretic approaches to statistical mechanics that are often conflated in the literature: those relating to the thermodynamic cost of computational processes and those that offer an interpretation of statistical mechanics where the probabilities are treated as epistemic. This distinction is then investigated through Earman and Norton’s ([1999]) ‘sound’ and ‘profound’ dilemma for information-theoretic exorcisms of Maxwell’s demon. It is argued that Earman and Norton fail to countenance a ‘sound’ information-theoretic interpretation and this paper (...) describes how the latter inferential interpretations can escape the criticisms of Earman and Norton ([1999]) and Norton ([2005]) by adopting this ‘sound’ horn. This article considers a standard model of Maxwell’s demon to illustrate how one might adopt an information-theoretic approach to statistical mechanics without a reliance on Landauer’s principle, where the incompressibility of the probability distribution due to Liouville’s theorem is taken as the central feature of such an interpretation. (shrink)

Entropy is ubiquitous in physics, and it plays important roles in numerous other disciplines ranging from logic and statistics to biology and economics. However, a closer look reveals a complicated picture: entropy is defined differently in different contexts, and even within the same domain different notions of entropy are at work. Some of these are defined in terms of probabilities, others are not. The aim of this chapter is to arrive at an understanding of some of the most important notions (...) of entropy and to clarify the relations between them, After setting the stage by introducing the thermodynamic entropy, we discuss notions of entropy in information theory, statistical mechanics, dynamical systems theory and fractal geometry. (shrink)

A classic source for understanding the connections between information theory and physics, this text was written by one of the giants of 20th-century physics and is appropriate for upper-level undergraduates and graduate students. Topics include the principles of coding, coding problems and solutions, the analysis of signals, a summary of thermodynamics, thermal agitation and Brownian motion, and thermal noise in an electric circuit. A discussion of the negentropy principle of information introduces the author's renowned examination of Maxwell's demon. Concluding chapters (...) explore the associations between information theory, the uncertainty principle, and physical limits of observation, in addition to problems related to computing, organizing information, and inevitable errors. 1962 ed. 81 figures. 14 tables. (shrink)

This book is an attempt to get to the bottom of an acute and perennial tension between our best scientific pictures of the fundamental physical structure of the ...

Roughly speaking, classical statistical physics is the branch of theoretical physics that aims to account for the thermal behaviour of macroscopic bodies in terms of a classical mechanical model of their microscopic constituents, with the help of probabilistic assumptions. In the last century and a half, a fair number of approaches have been developed to meet this aim. This study of their foundations assesses their coherence and analyzes the motivations for their basic assumptions, and the interpretations of their central concepts. (...) The most outstanding foundational problems are the explanation of time-asymmetry in thermal behaviour, the relative autonomy of thermal phenomena from their microscopic underpinning, and the meaning of probability. A more or less historic survey is given of the work of Maxwell, Boltzmann and Gibbs in statistical physics, and the problems and objections to which their work gave rise. Next, we review some modern approaches to (i) equilibrium statistical mechanics, such as ergodic theory and the theory of the thermodynamic limit; and to (ii) non-equilibrium statistical mechanics as provided by Lanford's work on the Boltzmann equation, the so-called Bogolyubov-Born-Green-Kirkwood-Yvon approach, and stochastic approaches such as `coarse-graining' and the `open systems' approach. In all cases, we focus on the subtle interplay between probabilistic assumptions, dynamical assumptions, initial conditions and other ingredients used in these approaches. (shrink)

This is an extensive review of recent work on the foundations of statistical mechanics.

Entropy and information are both emerging as currencies of interdisciplinary dialogue, most recently in evolutionary theory. If this dialogue is to be fruitful, there must be general agreement about the meaning of these terms. That this is not presently the case owes principally to the supposition of many information theorists that information theory has succeeded in generalizing the entropy concept. The present paper will consider the merits of the generalization thesis, and make some suggestions for restricting both entropy and information (...) to specific arenas of discourse. (shrink)

This article argues that most of the approaches to the foundations of statistical mechanics have severed their link with the original foundational project, the project of demonstrating how real mechanical systems can behave thermodynamically.

This book is an attempt to get to the bottom of an acute and perennial tension between our best scientific pictures of the fundamental physical structure of the world and our everyday empirical experience of it. The trouble is about the direction of time. The situation (very briefly) is that it is a consequence of almost every one of those fundamental scientific pictures--and that it is at the same time radically at odds with our common sense--that whatever can happen can (...) just as naturally happen backwards. Albert provides an unprecedentedly clear, lively, and systematic new account--in the context of a Newtonian-Mechanical picture of the world--of the ultimate origins of the statistical regularities we see around us, of the temporal irreversibility of the Second Law of Thermodynamics, of the asymmetries in our epistemic access to the past and the future, and of our conviction that by acting now we can affect the future but not the past. Then, in the final section of the book, he generalizes the Newtonian picture to the quantum-mechanical case and (most interestingly) suggests a very deep potential connection between the problem of the direction of time and the quantum-mechanical measurement problem. The book aims to be both an original contribution to the present scientific and philosophical understanding of these matters at the most advanced level, and something in the nature of an elementary textbook on the subject accessible to interested high-school students. Table of Contents: Preface 1. Time-Reversal Invariance 2. Thermodynamics 3. Statistical Mechanics 4. The Reversibility Objections and the Past-Hypothesis 5. The Scope of Thermodynamics 6. The Asymmetries of Knowledge and Intervention 7. Quantum Mechanics Appendix: Gedankenexperiments with Heat Engines Index Reviews of this book: The foundations of statistical mechanisms are often presented in physics textbooks in a rather obscure and confused way. By challenging common ways of thinking about this subject, Time and Chance can do quite a lot to improve this situation. --Jean Bricmont, Science Albert is perfecting a style of foundational analysis that is uniquely his own...It has a surgical precision...and it is ruthless with pretensions. The foundations of thermodynamics is a topic that has accumulated a good deal of dead wood; this is a fire that will burn and burn. --Simon W. Saunders, Oxford University As usual with Albert's work, the exposition is brisk and to the point, and exceptionally clear...The book will be an extremely valuable contribution to the literature on the subject of philosophical issues in thermodynamics and statistical mechanics, a literature which has been thin on the ground but is now growing as it deserves to. --Lawrence Sklar, University of Michigan. (shrink)

In discussions of the foundations of statistical mechanics, it is widely held that the Gibbsian and Boltzmannian approaches are incompatible but empirically equivalent; the Gibbsian approach may be calculationally preferable but only the Boltzmannian approach is conceptually satisfactory. I argue against both assumptions. Gibbsian statistical mechanics is applicable to a wide variety of problems and systems, such as the calculation of transport coefficients and the statistical mechanics and thermodynamics of mesoscopic systems, in which the Boltzmannian approach is inapplicable. And the (...) supposed conceptual problems with the Gibbsian approach are either misconceived, or apply only to certain versions of the Gibbsian approach, or apply with equal force to both approaches. I conclude that Boltzmannian statistical mechanics is best seen as a special case of, and not an alternatve to, Gibbsian statistical mechanics. (shrink)

Gibbsian statistical mechanics (GSM) is the most widely used version of statistical mechanics among working physicists. Yet a closer look at GSM reveals that it is unclear what the theory actually says and how it bears on experimental practice. The root cause of the difficulties is the status of the Averaging Principle, the proposition that what we observe in an experiment is the ensemble average of a phase function. We review different stances toward this principle, and eventually present a coherent (...) interpretation of GSM that provides an account of the status and scope of the principle. (shrink)

This article highlights the limitations of typicality accounts of thermodynamic behavior so as to promote an alternative line of research: understanding and accounting for the success of the techniques and equations physicists use to model the behavior of systems that begin away from equilibrium. This article also takes steps in this promising direction. It examines a technique commonly used to model the behavior of an important kind of system: a Brownian particle that has been introduced to an isolated fluid at (...) equilibrium. It also accounts for the success of the model, by identifying and grounding the technique’s key assumptions. (shrink)

Luciano Floridi presents a book that will set the agenda for the philosophy of information. PI is the philosophical field concerned with the critical investigation of the conceptual nature and basic principles of information, including its dynamics, utilisation, and sciences, and the elaboration and application of information-theoretic and computational methodologies to philosophical problems. This book lays down, for the first time, the conceptual foundations for this new area of research. It does so systematically, by pursuing three goals. Its metatheoretical goal (...) is to describe what the philosophy of information is, its problems, approaches, and methods. Its introductory goal is to help the reader to gain a better grasp of the complex and multifarious nature of the various concepts and phenomena related to information. Its analytic goal is to answer several key theoretical questions of great philosophical interest, arising from the investigation of semantic information. (shrink)

While the fundamental laws of physics are time-reversal invariant, most macroscopic processes are irreversible. Given that the fundamental laws are taken to underpin all other processes, how can the fundamental time-symmetry be reconciled with the asymmetry manifest elsewhere? In statistical mechanics, progress can be made with this question. What I dub the ‘Zwanzig–Zeh–Wallace framework’ can be used to construct the irreversible equations of SM from the underlying microdynamics. Yet this framework uses coarse-graining, a procedure that has faced much criticism. I (...) focus on two objections in the literature: claims that coarse-graining makes time-asymmetry ‘illusory’ and ‘anthropocentric’. I argue that these objections arise from an unsatisfactory justification of coarse-graining prevalent in the literature, rather than from coarse-graining itself. This justification relies on the idea of measurement imprecision. By considering the role that abstraction and autonomy play, I provide an alternative justification and offer replies to the illusory and anthropocentric objections. Finally, I consider the broader consequences of this alternative justification: the connection to debates about inter-theoretic reduction and the implication that the time-asymmetry in SM is weakly emergent. 1Introduction 1.1Prospectus2The Zwanzig–Zeh–Wallace Framework3Why Does This Method Work? 3.1The special conditions account3.2When is a density forwards-compatible?4Anthropocentrism and Illusion: Two Objections 4.1The two objections in more detail4.2Against the justification by measurement imprecision5An Alternative Justification 5.1Abstraction and autonomy5.2An illustration: the Game of Life6Reply to Illusory7Reply to Anthropocentric8The Wider Landscape: Concluding Remarks 8.1Inter-theoretic relations8.2The nature of irreversibility. (shrink)

Christopher G. Timpson provides the first full-length philosophical treatment of quantum information theory and the questions it raises for our understanding of the quantum world. He argues for an ontologically deflationary account of the nature of quantum information, which is grounded in a revisionary analysis of the concepts of information.

Quantum Information Theory and the Foundations of Quantum Mechanics is a conceptual analysis of one of the most prominent and exciting new areas of physics, providing the first full-length philosophical treatment of quantum information theory and the questions it raises for our understanding of the quantum world. -/- Beginning from a careful, revisionary, analysis of the concepts of information in the everyday and classical information-theory settings, Christopher G. Timpson argues for an ontologically deflationary account of the nature of quantum information. (...) Against what many have supposed, quantum information can be clearly defined (it is not a primitive or vague notion) but it is not part of the material contents of the world. Timpson's account sheds light on the nature of nonlocality and information flow in the presence of entanglement and, in particular, dissolves puzzles surrounding the remarkable process of quantum teleportation. In addition it permits a clear view of what the ontological and methodological lessons provided by quantum information theory are; lessons which bear on the gripping question of what role a concept like information has to play in fundamental physics. Topics discussed include the slogan 'Information is Physical', the prospects for an informational immaterialism (the view that information rather than matter might fundamentally constitute the world), and the status of the Church-Turing hypothesis in light of quantum computation. -/- With a clear grasp of the concept of information in hand, Timpson turns his attention to the pressing question of whether advances in quantum information theory pave the way for the resolution of the traditional conceptual problems of quantum mechanics: the deep problems which loom over measurement, nonlocality and the general nature of quantum ontology. He marks out a number of common pitfalls to be avoided before analysing in detail some concrete proposals, including the radical quantum Bayesian programme of Caves, Fuchs, and Schack. One central moral which is drawn is that, for all the interest that the quantum information-inspired approaches hold, no cheap resolutions to the traditional problems of quantum mechanics are to be had. (shrink)

Despite of its formal precision and its great many applications, Shannon’s theory still offers an active terrain of debate when the interpretation of its main concepts is the task at issue. In this article we try to analyze certain points that still remain obscure or matter of discussion, and whose elucidation contribute to the assessment of the different interpretative proposals about the concept of information. In particular, we argue for a pluralist position, according to which the different views about information (...) are no longer rival, but different interpretations of a single formal concept. (shrink)

On an influential account, chaos is explained in terms of random behaviour; and random behaviour in turn is explained in terms of having positive Kolmogorov-Sinai entropy (KSE). Though intuitively plausible, the association of the KSE with random behaviour needs justification since the definition of the KSE does not make reference to any notion that is connected to randomness. I provide this justification for the case of Hamiltonian systems by proving that the KSE is equivalent to a generalized version of Shannon's (...) communication-theoretic entropy under certain plausible assumptions. I then discuss consequences of this equivalence for randomness in chaotic dynamical systems. Introduction Elements of dynamical systems theory Entropy in communication theory Entropy in dynamical systems theory Comparison with other accounts Product versus process randomness. (shrink)

I argue that quantum mechanics is fundamentally a theory about the representation and manipulation of information, not a theory about the mechanics of nonclassical waves or particles. The notion of quantum information is to be understood as a new physical primitive—just as, following Einstein’s special theory of relativity, a field is no longer regarded as the physical manifestation of vibrations in a mechanical medium, but recognized as a new physical primitive in its own right.

Information theory presupposes the notion of an epistemic agent, such as a scientist or an idealized human. Despite that, information theory is increasingly invoked by physicists concerned with fundamental physics, physics at very high energies, or generally with the physics of situations in which even idealized epistemic agents cannot exist. In this paper, I shall try to determine the extent to which the application of information theory in those contexts is legitimate. I will illustrate my considerations using the case of (...) black hole thermodynamics and Bekenstein's celebrated argument for his formula for the entropy of black holes. This example is particularly pertinent to the theme of the present collection because it is widely accepted as `empirical data' in notoriously empirically deprived quantum gravity, even though the laws of black hole thermodynamics have so far evaded direct empirical confirmation. (shrink)

An important contemporary version of Boltzmannian statistical mechanics explains the approach to equilibrium in terms of typicality. The problem with this approach is that it comes in different versions, which are, however, not recognized as such and not clearly distinguished. This article identifies three different versions of typicality‐based explanations of thermodynamic‐like behavior and evaluates their respective successes. The conclusion is that the first two are unsuccessful because they fail to take the system's dynamics into account. The third, however, is promising. (...) I give a precise formulation of the proposal and present an argument in support of its central contention. †To contact the author, please write to: Department of Philosophy, Logic, and Scientific Method, London School of Economics, Houghton Street, London WC2A 2AE, England; e‐mail: [email protected]. (shrink)

our universe has a radius no smaller than 40 billion light- years—there's no sign, yet, of any edge to the universe. Though this is one piece of evidence for an infinite universe, it is not what really makes physicists think that the universe is ...

Information, entropy, probability: these three terms are closely interconnected in the prevalent understanding of statistical mechanics, both when this field is taught to students at an introductory level and in advanced research into the field’s foundations. This paper examines the interconnection between these three notions in light of recent research in the foundations of statistical mechanics. It disentangles these concepts and highlights their differences, at the same time explaining why they came to be so closely linked in the literature. In (...) the literature the term ‘information’ is often linked to entropy and probability in discussions of Maxwell’s Demon and its attempted exorcism by the Landauer-Bennett thesis, and in analyses of the spin echo experiments. The direction taken in the present paper is a different one. Here we discuss the statistical mechanical underpinning of the notions of probability and entropy, and this constructive approach shows that information plays no fundamental role in these concepts, although it can be conveniently used in a sense that we shall specify. (shrink)

Information theory and statistical mechanics II.

Information theory and statistical mechanics.

Philosophers of physics are very familiar with foundational problems in quantum mechanics and in the theory of relativity. In both fields, the puzzles, if not solved, are at least reasonably well formulated and possess well-characterized solution strategies. Sklar’s book Physics and Chance focuses on a pair of theories, thermodynamics and statistical mechanics, for which puzzles and foundational paradoxes abound, but where there is very little agreement upon the means with which they may best be approached. As he notes in the (...) introduction. (shrink)