and subjective probability theory does not really differ in any fundamental respect other than its interpretation. It’s just that in some cases it seems more convenient, or more appropriate, to use one rather than another, as Jaynes pointed out half a century ago.
In his 1957 paper,7 Jaynes championed the subjectivity side of the probability debate. He noted that both views, subjectivist and objectivist, were needed in physics, but that for some types of problems only the subjective approach would do.
He argued that the subjective approach can be useful even when you know nothing about the system you are interested in to begin with. If you are given a box full of particles but know nothing about them—not their mass, not their composition, not their internal structure—there’s not much you can say about their behavior. You know the laws of physics, but you don’t have any knowledge about the system to apply the laws to. In other words, your ignorance about the behavior of the particles is at a maximum.
Early pioneers of probability theory, such as Jacob Bernoulli and Laplace, said that in such circumstances you must simply assume that all the possibilities are equally likely—until you have some reason to assume otherwise. Well, that helps in doing the calculations, perhaps, but is there any real basis for assuming the probabilities are equal? Except for certain cases where an obvious symmetry is at play (say, a perfectly balanced two-sided coin), Jaynes said, many other assumptions might be equally well justified (or the way he phrased it, any other assumption would be equally arbitrary).8
Jaynes saw a way of coping with this situation, though, with the help of the then fairly new theory of information devised by Claude Shannon of Bell Labs. Shannon was interested in quantifying communication, the sending of messages, in a way that would help engineers find ways to communicate more efficiently (he worked for the telephone company, after all). He found math that