Prime Obsession: Bernhard Riemann and the Greatest Unsolved Problem in Mathematics (2003)

I. The Montgomery-Odlyzko Law tells us that the non-trivial zeros of the Riemann zeta function look like—statistically, that is—the eigenvalues of some random Hermitian matrix. The operators represented by such matrices can be used to model certain dynamical systems in quantum physics. Is there, then, a Riemann operator, an operator whose eigenvalues are precisely the zeta zeros? If there is, what dynamical system does it represent? Could that system be created in a physics lab? And if it could, would that help to prove the Hypothesis?

Research into these questions was under way even before the publication of Odlyzko’s 1987 paper. The previous year, in fact, Michael Berry had published a paper titled “Riemann’s Zeta Function: A Model for Quantum Chaos?” Using results that were being widely circulated and discussed at the time, including some of Odlyzko’s, Berry tackled the following question. Suppose there is a Riemann operator: what kind of dynamical system would it model? His answer was: a chaotic system. To explain this, I must make a brief detour through Chaos Theory.

II. That pure number theory—ideas about the natural numbers and their relations with each other—should have relevance to subatomic physics is not all that surprising. Quantum physics has a much stronger arithmetical component than classical physics, since it depends on the idea that matter and energy are not infinitely divisible. Energy comes in 1, 2, 3, or 4 quanta, but not in , , , or π quanta. That is by no means the whole story, and quantum mechanics could not have been developed without the most powerful tools of modern analysis. Schrödinger’s famous wave equation, for example, is written in the language of traditional calculus. Still, the arithmetical component is there in quantum mechanics, whereas in classical mechanics it is almost entirely absent.

The foundations of classical physics—the physics of Newton and Einstein—are quintessentially analytical, in the mathematical sense. They rest on mathematical analysis, on the notions of infinite divisibility, of smoothness and continuity, of limit and derivative, of real numbers. Newton invented the calculus, too, remember—the ultimate application of the concept “limit”—that eventually took over most of analysis.

Take the classical problem of one body in an elliptical orbit around another, under mutual gravitational attraction. At a certain distance from the parent body (measured by r, a real number) the satellite body has some precise velocity (measured by v, another real number). The relationship between v and r has a precise mathematical expression; v is in fact a function of r, expressed by the so-called vis viva equation familiar to all students of elementary celestial mechanics,

where M and a are some fixed numbers determined by the components and initial conditions of the system under observation—on the masses of the two bodies, and so on.

Now, of course, in practice we cannot attain the infinite precision needed to assign actual real numbers to r and v. We might be able to measure r to 10 decimal places, or even 20; but to pin down a real number you need infinitely many decimal places, and that we cannot get. In the case of any actual orbit, therefore, there will be some modest error in assigning a real number to r, and a corresponding error in the computed value of v. This doesn’t matter much. Kepler’s laws assure us that we will still get a regular ellipse, and the mathematics of the vis viva equation tell us that a 1 percent error in r typically turns up only a 0.5 percent error in v. The situation is manageable, predictable. It is, as mathematicians say, “integrable.”

That, however, is an extremely simple problem. Almost all actual physical problems are more complex than that. Take the case of three bodies under mutual gravitational attraction, for instance—the famous “three-body problem.” Can we solve it with closed-form solutions like the vis viva equation? Is it integrable? By the end of the nineteenth century it was apparent that the answers are: No, we cannot, and it is not. The only way to get solutions is by extensive numerical calculation, leading to approximations.

In 1890, in fact, Henri Poincaré published a definitive paper on the three-body problem, making it clear not only that the problem has no closed-form solutions, but that it has another, even more disturbing quality: Its solutions are sometimes chaotic. That is, if you vary the initial conditions of the problem—the numbers equivalent to M and a in my two-body example—very slightly, the resulting orbits change drastically, beyond all recognition. Poincaré himself commented that one set of conditions produced “orbits so tangled that I cannot even begin to draw them.”

Poincaré’s paper is generally taken to mark the birth of modern chaos theory. Nothing much happened in chaos theory for several decades, mainly because mathematicians had no way to do number-crunching on the scale required to analyze chaotic results. That changed when computers became available, and chaos theory was reborn with the work of meteorologist Ed Lorentz at M.I.T. in the

1960s.¹¹⁶ Chaos theory is now a vast subject embracing many different subdisciplines within physics, mathematics, and computer science.

It is important to grasp that a chaotic system, like a solution to the three-body problem, need not (and, in general, does not) consist of random motions. The beauty of chaos theory is that there are patterns embedded in chaotic systems. While in general a chaotic system never retraces its steps, it does exhibit these recurring patterns; and underlying these patterns are certain regular, but unstable, periodic orbits into which, in theory, if infinite precision were available to the nudger, a chaotic system could be nudged.

III. When modern chaos theory first came up, physicists took it to be entirely a classical matter, with no relevance for quantum theory. Chaos arises from issues like the three-body problem because the numbers defining the initial conditions are real numbers, measuring numbers, infinitely divisible; they can be varied by 1 percent, or by 0.1 percent, or by 0.001 percent…. Since the conditions are infinitely variable, an infinity of outcomes presents itself. In quantum theory, by contrast, you can vary those initial conditions by 1, 2, or 3 units, but not by or 2.749. There should be “no room” for chaos in quantum theory. It is true that there is a degree of uncertainty in quantum mechanics, but the controlling equations are nonetheless linear. Small perturbations lead to small consequences, as with the classical vis viva equation for two-body motion.

Yet in fact, a certain level of chaos can be observed in quantum-scale dynamical systems. The orderly energy-level structure of the electrons in orbit around the nucleus of an atom, for example, can be scrambled into an irregular pattern by the application of a sufficiently strong magnetic field. (This is, in fact, one of the dynamical systems modeled by GUE operators.) The atom’s subsequent behavior is chaotic—wildly different for only slightly different initial conditions.

If such quantum-chaotic systems persist for a period, however, the laws of quantum mechanics eventually impose order on them,

draining away the chaos. The number of permitted states dwindles; the number of forbidden states swells. The bigger and more complex the system, the longer it takes for the quantum rules to assert order, and the larger the number of permitted states…until, on the scale of the everyday world, it would take trillions of years for the quantum order to assert itself, and the number of permitted states is large enough to be taken as infinite. That is why we have chaos in classical physics.

Back in 1971, physicist Martin Gutzwiller found a way to relate chaotic systems on the classical scale with analogous systems down in the quantum world, by allowing the quantum factor, Planck’s constant, in the quantum-mechanical equations to tend to zero, and taking limits. The periodic orbits that underlie a classical-chaotic system correspond to the eigenvalues of the operator defining this “semiclassical” system.

Michael Berry argued that if there is a Riemann operator, it models one of these semiclassical chaotic systems, and its eigenvalues, the imaginary parts of the zeta zeros, are the energy levels of that system. The periodic orbits in the analogous classical-chaotic system would correspond to … the prime numbers! (To their logs, to be precise.) He further argued that this semiclassical system would not have the quality of “time reversal symmetry”—that is, if all the velocities of all the particles in the system were to be instantly and simultaneously reversed, the system would not return to its initial state. (Chaotic systems can be time-reversible or not. The ones that are time-reversible are modeled not by operators of the GUE type, but by another kind belonging to a different ensemble, the GOE—Gaussian Orthogonal Ensemble.)

Berry’s work (much of it in collaboration with a Bristol colleague, Jonathan Keating) is subtle and deep. He has, for example, analyzed the Riemann-Siegel formula in great detail in search of insights into the zeros, and their effects on each other at different ranges. At the time of writing, he has not identified any dynamical system that cor-

responds to the Riemann operator, but thanks to his work, if such an operator exists, we shall know it at once when we see it.

IV. Another researcher, Alain Connes, Professor of Mathematics at the Collège de France in Paris, has taken an alternative approach. Instead of seeking to pin down the kind of operator the zeta zeros might be eigenvalues of, he has actually constructed such an operator.

That was no mean feat. An operator must have something to operate on. The kind of operators I have been speaking about operate on spaces. A flat two-dimensional space will do to illustrate the general principle, with a sheet of graph paper for purposes of visualization, though you must imagine the paper extending to infinity in all directions. Suppose that I rotate that space by 30 degrees counter-clockwise, sending every point of the space to some other point thereby (except the point about which I am rotating—that stays put). This rotation is an instance of an operator. The characteristic polynomial of this particular operator is . The eigenvalues are and ; the trace is .

If you wanted, you could set up a coordinate system to describe all the points of the space, drawing a horizontal x-axis and a vertical y-axis to meet at the rotation point, and marking off distances along them in inches or centimeters, in the usual way. You might then notice that my rotation operator sends the point (x, y) to a new point with different coordinates—actually, to . That is incidental to the nature of the operator, though, which exists, and which moves the points of the space to new points, independent of any coordinate system. A rotation is a rotation, even if you forgot to draw in a pair of axes.

The operators used in mathematical physics operate on much more complicated spaces than that, of course. Their spaces are not merely two-dimensional, nor just three-dimensional like the space we live out our everyday lives in. Nor are they even four-dimensional,

like the one required by Relativity Theory. They are abstract mathematical spaces with infinitely many dimensions. Each point of such a space is a function. An operator transforms one function into another function, that is, in the language of spaces and points, it sends one point to another point.

To get a very elementary idea of how a function might be identified with a point in a space, consider one simple class of functions, the quadratic polynomials p + qx + rx². The family of all such polynomials could be represented by a three-dimensional space, the point with coordinates (p, q, r) standing for the polynomial p + qx + rx². A four-dimensional space could model cubic polynomials; a five-dimensional space could model quartics … and so on. Now, since some functions can be written as series, and a series looks like an infinite polynomial (e^x, for example, as ), you can see how a space of infinitely many dimensions might be useful for modeling functions. Then e^x would be the point in that space located by the infinity of coordinates .

In quantum mechanics, the functions are wave functions, defining the probability that the particles of a system are at certain places, with certain velocities, at a given moment in time. Each point of the space, in other words, represents a state of the system. The operators used in quantum mechanics encode observable features of the system—most famously, the Hamiltonian operator, which encodes the system’s energy. The eigenvalues of the Hamiltonian operator are the fundamental energy levels of the system. Each eigenvalue is particularly associated with a key point—function—of the space, called an eigenfunction, representing the state of the system at that energy level. These eigenfunctions are essential and fundamental states of the system. Every possible state of the system, every physical manifestation, is some linear combination of the eigenfunctions, just as every point in a three-dimensional space can be written as (x, y, z), a linear combination of the points (1,0,0), (0,1,0), and (0,0,1).

Alain Connes has constructed a very peculiar space for his Riemann operator to operate on. The prime numbers are built in to this

space in a way derived from concepts in algebraic number theory. Here is a sketch of Connes’s work.

V. Classical physics is built around real numbers like this, 22.45915771836…, for which—absent a closed form—an infinite number of digits is required to give full theoretical accuracy. Actual physical measurements, though, are approximate, like this: 22.459. That is a rational number, . The entire world of physical experiment can therefore be written down in rational numbers, members of . To pass from the experimental world to the theoretical, we have to complete (see Chapter 11.v). That is, we have to enlarge it, so that if an infinite sequence of numbers in has a limit, that limit is either in itself, or in the enlarged field. The normal and natural way to do this is with , the real numbers, or , the complex numbers.

Algebraic number theory, however, has other ways to complete . In 1897 the Prussian mathematician Kurt Hensel¹¹⁷ devised an entire new family of objects to deal with certain problems in the theory of algebraic fields, like that field that I discussed in Chapter 17.ii. These objects are called “p-adic numbers.” There is one field of these exotic creatures, with infinitely many members in it, for any prime number p. The building blocks of this field are the clock rings of size p, p², p³, p⁴, and so on, that I discussed in Chapter 17.ii. In the symbols I introduced there, they are _p , _p² , _p³ …. The field of 7-adic numbers, for example, is built up from the rings ₇, ₄₉, ₃₄₃, ₂₄₀₁…. Recall my illustration of how a finite field can be used to help build an infinite field? Well, here we are using an infinity of finite rings to build a new infinite field!

The field of p-adic numbers goes by the symbol “_p.” So there is a field ₂, a field ₃, a field ₅, a field ₇, a field ₁₁, and so on. Each is

a complete field: ₂ the field of 2-adic numbers, ₃ the field of 3-adic numbers, and so on.

As the symbol suggests, the p-adic numbers bear a certain resemblance to ordinary rational numbers. However, _p is richer and more complicated than and in some respects is more like , the field of real numbers. In particular, _p can, like , be used to complete .

You might at this point be wondering, “All well and good; but you say there is a field _p of these strange new objects, these p-adic numbers, for any prime number p, and that any old _p can be used to complete . So … which one is best, ₂? ₃? ₁₁? ₄₅₈₂₇? Which prime should Professor Connes use to carry out this stunt, to throw a bridge from the prime numbers to the physics of dynamical systems?”

The answer is, all of them! You see, there is an algebraic concept called an adele that embraces within its broad arms all the _p, for all the prime numbers 2, 3, 5, 7, 11, …. In fact, it embraces real numbers, too! Adeles are built up from ₂, ₃, ₅,

₇, …, and , in much the same way that p-adic numbers are built up from _p, _p², _p³, …. Adeles are, if you like, one further level of abstraction up from p-adic numbers, which are themselves one level of abstraction up from ordinary rational numbers.

If all this has your head spinning, just suffice it to say that we have a class of super-numbers that are simultaneously 2-adic, 3-adic, 5-adic, … and also real. Every one of these super-numbers has all the primes imbedded in it.

The adele is certainly a very abstruse concept. Nothing is so abstruse that it doesn’t find its way into physics eventually, though. In the 1990s mathematical physicists set about constructing adelic quantum mechanics, in which the actual rational-number measurements that show up in experiments were taken to be manifestations of these bizarre creatures hauled up from the lightless depths of the mathematical abyss.

This is the kind of space Alain Connes built for his Riemann operator to play in, an adelic space. Being adelic, it has the prime numbers built in, so to speak. Operators that act on this space are perforce

prime-based. You can now, I hope, see how it is possible to build a Riemann operator whose eigenvalues are precisely the non-trivial zeros of the zeta function, and whose space—the space on which it operates—has the primes built in, in the way I have attempted to describe, while yet being relevant to actual physical systems, actual assemblies of subatomic particles.

The Riemann Hypothesis (RH) is then reduced to the matter of proving a certain trace formula—that is, a formula like Gutzwiller’s, relating the eigenvalues of an operator on Connes’s adelic space to the periodic orbits in some analogous classical system. Having the prime numbers already built in to one side of the formula ought to make everything easy. In a way it does, and Connes’s construction is brilliant, and extremely elegant, with energy levels that are precisely zeta zeros on the critical line. Unfortunately, it has so far offered no clue as to why there might not be zeta zeros off the critical line!

Opinions as to the value of Connes’s work vary widely. Not at all sure that I understood it myself, I canvassed some real mathematicians working in the field. I shall tread carefully here. For all I know, Alain Connes might announce a proof of the RH the day this book comes out, and I don’t want to make anyone look foolish. Here are two quotes from professionals.

Mathematician X: “Tremendously important work! Connes will not only prove the RH, he will give us a Unified Field Theory, too!”

Mathematician Y: “What Connes has done, basically, is to take an intractable problem and replace it with a different problem that is equally intractable.”

I do not feel qualified to tell you which opinion is correct. Given the stature and abilities of X and Y, though, I feel pretty sure that one of them is….¹¹⁸

VI. Other approaches to the RH are still active, of course. The algebraic approach through finite fields that I mentioned in Chapter 17 is very much alive. And, as we glimpsed in Section V above, that ap-

proach has interesting connections with the physical lines of attack. Analytic number theory, too, is still a busy area, and capable of strong results.

There are also indirect approaches. There is, for example, my Theorem 15.2, concerning the M function got by accumulating Möbius μ . That is, as I said, exactly equivalent to the Hypothesis. Analytic number theorist Dennis Hejhal of the University of Minnesota actually uses this as a way to present the RH to nonmathematical audiences, to avoid having to introduce complex numbers. Here, he says (I am paraphrasing his approach, not quoting it), is the RH.

Write down all the natural numbers, starting with 2. Under each number, write its prime factors. Then, ignoring any number with a square factor (or any higher power, which will necessarily include a square), go along the line marking as “heads” any number with an even number of prime factors, “tails” any with an odd number. This gives an infinite string of heads and tails—just like a coin-tossing experiment.

Now, we know very well, from classical probability theory, what to expect from a long run of N coin tosses. On average, we will get heads and tails. But of course, we should hardly ever get exactly these numbers. Suppose we subtract the number of heads from the number of tails. (Or vice versa, depending on which is larger.) What do we expect this excess to be? On average, it is , that is, . This has been known since the time of Jakob Bernoulli, 300 years ago. If you toss a fair coin a million times, on average you have an excess of a thousand heads (or tails). You might have more or you might have less, but on average, as you keep tossing that coin—as N goes off to infinity—the size of the excess grows at a certain rate; at

a rate that is less than , for any number ε , no matter how small. Just like my Theorem 15.2!

In fact, my Theorem 15.2, which is equivalent to the RH, says that the M function grows just like the excess in a coin-tossing exercise. To put it another way, it says that a square-free number is either a head or a tail—has either an even or an odd number of prime factors—with 50–50 probability. This does not seem particularly unlikely and might in fact be true. If you can prove that it is true, you will have proved the RH.¹¹⁹

VII. A less direct probabilistic approach concerns the so-called “Cramér model.” Harald Cramér was, in spite of that accent on his name, Swedish, and yet another insurance company employee—an actuary for Svenska Livförsäkringsbolaget, but also a popular and inspiring lecturer on math and statistics.¹²⁰ In 1934 he published a paper titled “On Prime Numbers and Probability,” in which he put forward the idea that the primes were distributed as randomly as they could be.

One consequence of the Prime Number Theorem (PNT), which I demonstrated in Chapter 3.ix, is that in the neighborhood of some large number N, the proportion of primes is ~ 1 / log N. The log of a trillion, for example, is 27.6310211…, so in the neighborhood of a trillion, around one number in 28 is a prime. Cramér’s model says that aside from this one restraint on their average frequency, the primes are utterly random.

Here is one way to see what this means.¹²¹ Imagine a long line of earthenware jars with the natural numbers painted on them. The numbers go 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, …, to infinity (or some very large number). Into each jar put a number of wooden balls. The number of balls in jar N should be log N (or the nearest whole number). So the number of balls in the first few jars are 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, …. Furthermore, there must be exactly one black ball in each jar; all the rest of the balls in the jar are white. Jars number 2, 3, and 4,

therefore, have only one black ball in them; jars number 5 to 12 have one black and one white; jars number 13 to 33 have one black and two white, and so on.

Now take a clipboard and a large (preferably infinite) sheet of paper, and take a walk along the line. Pull a ball at random from each jar. If it’s black, write down the number of the jar. When you finish, you have a long list of whole numbers starting “2, 3, 4, ….” The chance that 5 is on your list is 50–50, since jar 5 has one white ball and one black. The chance that 1,000,000,000,000 is on your list is 1 in 28.

Now, what can we say about this list? It is not a list of the primes, of course. There are lots of even numbers on it, for example; but only one prime, 2, is even. Well, if the Cramér model is true, the list will be statistically indistinguishable from the primes. Any broad statistical property the primes have—how many you expect to find in intervals of certain lengths, for instance, or the degree of clustering (what Hilbert, in stating his eighth problem, called “condensation”)—this random list will have, too.

For an analogy, consider the decimal digits of π . So far as anyone knows, they are perfectly random.¹²² They never repeat themselves. Digits, and pairs of digits, and triplets and quartets of digits, occur with just the frequency you would expect from pure chance. Nobody has ever been able to detect any pattern in the billions of digits of π now available for inspection. The decimal digits of π are a random sequence of digits … except that they represent π ! So with the primes, on Cramér’s model. They are indistinguishable from any other sequence with frequency 1 / log N, and in that sense they are perfectly random … except, of course that they are the primes!

In 1985 Helmut Maier proved that the Cramér model in the simple form I have sketched above is not a complete picture of the primes. A modified version of the model does give accurate predictions for the distribution of primes, however, and is linked to the RH in ways subtle and indirect. There is a modest hope that further research on this topic will yield insights into the RH.¹²³

VIII. Finally, I cannot resist mentioning the most indirect approach of all, the one through non-deductive logic. This is not, properly speaking, a mathematical topic. Mathematics demands rigorous logical proof before a result can be accepted. Most of the world is not like this, however. In our daily lives we work mainly from probabilities. In courts of law, in medical consultations, in drawing up insurance policies, it is the balance of probabilities that we take into account, not ironclad certainties. Sometimes, of course, we use the actual mathematical theory of probability to quantify the matters under dispute—that is why insurance companies employ actuaries. Much more often we do not, and cannot—think of a law court.

Mathematicians have often cast an interested eye at this side of life. George Pólya actually wrote a two-volume book about it,¹²⁴ in which he made the rather surprising claim that non-deductive logic is better appreciated in mathematics than in the natural sciences. This line of thought has most recently been taken up by Australian mathematician James Franklin. His 1987 paper “Non-deductive Logic in Mathematics,” in The British Journal for the Philosophy of Science, included a section headed “Evidence for the Riemann Hypothesis and other Conjectures.”

Franklin approaches the RH as if it were a courtroom case. He presents the evidence for the RH being true:

• Hardy’s 1914 result that all infinitely many zeros lie on the critical line.

• The RH implies the PNT, which is known to be true.

• “Denjoy’s probabilistic interpretation”—that is, the coin-tossing argument given in this chapter.

• Another 1914 theorem by Landau and Harald Bohr, stating that most zeros—all but an infinitesimal proportion—are very close to the critical line. Note that since the number of zeros is infinite, one trillion counts as an infinitesimal proportion.

• The algebraic results of Artin, Weil, and Deligne, that I mentioned in Chapter 17.iii.

Then the case for the prosecution:

• Riemann himself had no sound reasons to support his statement in the 1859 paper that the RH was “very likely,” and the semi-reasons that might have motivated his statement have been knocked down since.

• The zeta function exhibits some very peculiar behavior high up the critical line, as revealed by the computer-generated results of the 1970s. (Franklin seems not to have known of Odlyzko’s work.)

• Littlewood’s 1914 result on the error term Li(x) – π (x). Says Franklin: “The relevance of Littlewood’s discovery to Riemann’s Hypothesis is far from clear. But it does give some reason to suspect that there may be a very large counterexample to Riemann’s Hypothesis, although there are no small ones.” So far as I can tell, Franklin’s argument here is by analogy. “For some extremely large numbers, the error term misbehaves. It is connected with the zeros of the zeta function.” [See my Chapter 21.] “So perhaps for very large T, the zeta function misbehaves, having zeros off the critical line.”

This is all circumstantial, of course. It should not, however, be dismissed as mere sub-philosophical word-play. The rules of evidence can deliver very persuasive results, sometimes contrary to the strictly argued certainties of mathematics. Consider, for example, the very un-mathematical fact that a hypothesis might be seriously weakened by a confirming instance. Hypothesis: No human can possibly be more than nine feet tall. Confirming instance: A human being who is 8′11¾′′ tall. The discovery of that person confirms the hypothesis … but at the same time casts a long shadow of doubt across it!¹²⁵