brief excursion into the nature of probability theory and information theory, the essential threads tying game theory and statistical physics together.
For centuries, scientists and mathematicians have argued about the meaning of probability. Even today there exist separate schools of probabilistic thought, generally referred to by the shorthand labels of “objective” and “subjective.” But those labels conceal a tangle of subarguments and technical subtleties that make probability theory one of the most contentious and confusing realms of math and science.
In a way, that’s a bit surprising, since probability theory really lies at the very foundation of science, playing the central role in the process of analyzing experimental data and testing theories. It’s what doing science is all about. You’d think they’d have it all worked out by now. But establishing rules for science is a little like framing a constitution for Iraq. There are different philosophies and approaches to science. The truth is that science (unlike mathematics) is not built on a rock-solid foundation of irreducible rules. Science is like grammar. Grammar arises from regularities that evolve in the way native speakers of a language form their words and string them together. A true grammarian does not tell people how they should speak, but codifies the way that people actually do speak. Science does not emanate from a cookbook that provides recipes for revealing nature’s secrets, but from a mix of methods that somehow succeed in rendering nature comprehensible. That’s why science is not all experiment, and not all theory, but a complex interplay of both.
Ultimately, though, theory and experiment must mesh if the scientist’s picture of nature is to be meaningful and useful. And in most realms of science, you need math to test the mesh. Probability theory is the tool for performing that test. (Different ideas about how to perform the test, then, lead to different conceptions of probability.)