The principle of maximum entropy is a general method to assign values to probability distributions on the basis of partial information. This principle, introduced by Jaynes in 1957, forms an extension of the classical principle of insufficient reason. It has been further generalized, both in mathematical formulation and in intended scope, into the principle of maximum relative entropy or of minimum information. It has been claimed that these principles are singled out as unique methods of statistical inference that agree with (...) certain compelling consistency requirements. This paper reviews these consistency arguments and the surrounding controversy. It is shown that the uniqueness proofs are flawed, or rest on unreasonably strong assumptions. A more general class of inference rules, maximizing the so-called Re[acute ]nyi entropies, is exhibited which also fulfill the reasonable part of the consistency assumptions. (shrink)

Sometimes we receive evidence in a form that standard conditioning (or Jeffrey conditioning) cannot accommodate. The principle of maximum entropy (MAXENT) provides a unique solution for the posterior probability distribution based on the intuition that the information gain consistent with assumptions and evidence should be minimal. Opponents of objective methods to determine these probabilities prominently cite van Fraassen’s Judy Benjamin case to undermine the generality of maxent. This article shows that an intuitive approach to Judy Benjamin’s case supports maxent. This (...) is surprising because based on independence assumptions the anticipated result is that it would support the opponents. It also demonstrates that opponents improperly apply independence assumptions to the problem. (shrink)

Richard Jeffrey espoused an antifoundationalist variant of Bayesian thinking that he termed ‘Radical Probabilism’. Radical Probabilism denies both the existence of an ideal, unbiased starting point for our attempts to learn about the world and the dogma of classical Bayesianism that the only justified change of belief is one based on the learning of certainties. Probabilistic judgment is basic and irreducible. Bayesian conditioning is appropriate when interaction with the environment yields new certainty of belief in some proposition but leaves one’s (...) conditional beliefs untouched (the ‘Rigidity’ condition). Although Richard Jeffrey denied the general applicability of this condition, one of his main contributions to probabilistic thinking is a form of belief updating—now typically called ‘Jeffrey conditioning’ or ‘probability kinematics’—that is appropriate in circumstances in which Rigidity is satisfied, but where the interaction causes one to reevaluate one’s probability judgments over some partition of the possibility space without conferring certainty on any particular element. The most familiar occasion for Jeffrey conditioning is receipt of uncertain evidence: things partially perceived or remembered. But it also serves to illuminate belief updating occasioned by a change in one’s degrees of conditional belief, a kind of belief change largely ignored by classical Bayesianism. I argue that such changes in conditional belief can also be basic (in the sense of not being analyzable as a consequence of conditioning on factual information) and offer a kinematical model for a particular kind change in conditional belief. Both are applied to changes in preference. Finally, I argue that Rigidity can fail when changes of belief give inferential grounds for changes in conditional belief (and vice versa). These failures show that conditioning methods are properly regarded, not as valid rules of inference, but as tools in the ‘art of judgment’. (shrink)

We argue that indeterminate probabilities are not only rationally permissible for a Bayesian agent, but they may even be rationally required . Our first argument begins by assuming a version of interpretivism: your mental state is the set of probability and utility functions that rationalize your behavioral dispositions as well as possible. This set may consist of multiple probability functions. Then according to interpretivism, this makes it the case that your credal state is indeterminate. Our second argument begins with our (...) describing a world that plausibly has indeterminate chances. Rationality requires a certain alignment of your credences with corresponding hypotheses about the chances. Thus, if you hypothesize the chances to be indeterminate, your will inherit their indeterminacy in your corresponding credences. Our third argument is motivated by a dilemma. Epistemic rationality requires you to stay open-minded about contingent matters about which your evidence has not definitively legislated. Practical rationality requires you to be able to act decisively at least sometimes. These requirements can conflict with each other-for thanks to your open-mindedness, some of your options may have undefined expected utility, and if you are choosing among them, decision theory has no advice to give you. Such an option is playing Nover and Hájek’s Pasadena Game , and indeed any option for which there is a positive probability of playing the Pasadena Game. You can serve both masters, epistemic rationality and practical rationality, with an indeterminate credence to the prospect of playing the Pasadena game. You serve epistemic rationality by making your upper probability positive-it ensures that you are open-minded. You serve practical rationality by making your lower probability 0-it provides guidance to your decision-making. No sharp credence could do both. (shrink)

A paper on how to adapt your probabilisitc beliefs when learning a conditional.

This essay is, primarily, a discussion of four results about the principle of maximizing entropy (MAXENT) and its connections with Bayesian theory. Result 1 provides a restricted equivalence between the two: where the Bayesian model for MAXENT inference uses an "a priori" probability that is uniform, and where all MAXENT constraints are limited to 0-1 expectations for simple indicator-variables. The other three results report on an inability to extend the equivalence beyond these specialized constraints. Result 2 established a sensitivity of (...) MAXENT inference to the choice of the algebra of possibilities even though all empirical constraints imposed on the MAXENT solution are satisfied in each measure space considered. The resulting MAXENT distribution is not invariant over the choice of measure space. Thus, old and familiar problems with the Laplacian principle of Insufficient Reason also plague MAXENT theory. Result 3 builds upon the findings of Friedman and Shimony (1971; 1973) and demonstrates the absence of an exchangeable, Bayesian model for predictive MAXENT distributions when the MAXENT constraints are interpreted according to Jaynes's (1978) prescription for his (1963) Brandeis Dice problem. Lastly, Result 4 generalizes the Friedman and Shimony objection to cross-entropy (Kullback-information) shifts subject to a constraint of a new odds-ratio for two disjoint events. (shrink)