First comprehensive introduction to information theory explores the work of Shannon, McMillan, Feinstein, and Khinchin. Topics include the entropy concept in probability theory, fundamental theorems, and other subjects. 1957 edition.

This is a new release of the original 1949 edition.

Noam Chomsky's book on syntactic structures is a serious attempts on the part of a linguist to construct within the tradition of scientific theory-construction ...

A mathematical theory is proposed and exemplified, which covers an extended class of black boxes. Every kind of stimulus and response is pictured by a channel connecting the box with its environment. The input-output relation is given by a postulate schema according to which the response is, in general, a nonlinear functional of the input. Several examples are worked out: the perfectly transmitting box, the damping box, and the amplifying box. The theory is shown to be (a) an extension of (...) the S-matrix theory and the accompanying channel picture as developed in microphysics; (b) abstract and applicable to any problem involving the transactions of a system (physical, biological, social, etc.) with its milieu; (c) superficial, because unconcerned with either the structure of the box or the nature of the stimuli and responses. The motive for building the theory was to show the capabilities and limitations of the phenomenological approach. (shrink)

A classic source for understanding the connections between information theory and physics, this text was written by one of the giants of 20th-century physics and is appropriate for upper-level undergraduates and graduate students. Topics include the principles of coding, coding problems and solutions, the analysis of signals, a summary of thermodynamics, thermal agitation and Brownian motion, and thermal noise in an electric circuit. A discussion of the negentropy principle of information introduces the author's renowned examination of Maxwell's demon. Concluding chapters (...) explore the associations between information theory, the uncertainty principle, and physical limits of observation, in addition to problems related to computing, organizing information, and inevitable errors. 1962 ed. 81 figures. 14 tables. (shrink)