Don’t prematurely obsess on a single “big problem” or “big theory”

Millions long for immortality who do not know what to do with themselves on a rainy Sunday afternoon. — Susan Ertz, “Anger in the Sky”

There is a particularly dangerous occupational hazard in this subject: one can become focused, to the exclusion of other mathematical activity (and in extreme cases, on non-mathematical activity also), on a single really difficult problem in a field (or on some grand unifying theory) before one is really ready (both in terms of mathematical preparation, and also in terms of one’s career) to devote so much of one’s research time to such a project. This is doubly true if one has not yet learnt the limitations of one’s tools or acquired a healthy scepticism of one’s own work, as this can lead to the humiliating spectacle of proudly announcing a major breakthrough on a well known problem, only to withdraw the preprint shortly afterwards after serious flaws (often arising from pushing a method well beyond its known limits, and encounting obstructions beyond those limits that were known to the experts) are pointed out in the manuscript.

When one begins to neglect other tasks (such as writing and publishing one’s “lesser” results), hoping to use the eventual “big payoff” of solving a major problem or establishing a revolutionary new theory to compensate for lack of progress in all other areas of one’s career, then this is a strong warning sign that one should rebalance one’s priorities. While it is true that several major problems have been solved, and several important theories introduced, by precisely such an obsessive approach, this has only worked out well when the mathematician involved:

• had a proven track record of reliably producing significant papers in the area already; and
• had a secure career (e.g. a tenured position).

If you do not yet have both (1) and (2), and if your ideas on how to solve a big problem still have a significant speculative component (or if your grand theory does not yet have a definite and striking application), I would strongly advocate a more balanced, patient, and flexible approach instead: one can certainly keep the big problems and theories in mind, and tinker with them occasionally, but spend most of your time on more feasible “low-hanging fruit”, which will build up your experience, mathematical power, and credibility for when you are ready to tackle the more ambitious projects.

See also “Don’t base career decisions on glamour or fame” and “Use the wastebasket”. Henry Cohn also has some related advice for amateur mathematicians. This MathOverflow answer by Minhyong Kim also makes the point that one should accrue some definite mathematical results (preferably as published papers) before one can afford spending this “reputational capital” on philosophising on some “big picture” vision of mathematics.

Addendum: on publishing proofs of famous open problems

If you do believe that you have managed to solve a major problem, I would advise you to be extraordinarily sceptical of your own work, and to exercise the utmost care and caution before releasing it to anyone; there have been too many examples in the past of mathematicians whose reputation has been damaged by claiming a proof of a well-known result to much fanfare, only to find serious errors in the proof shortly thereafter. I recommend asking yourself the following questions regarding the paper:

1. What is the key new idea or insight? How does it differ from what has been tried before? Is this idea emphasised in the introduction to the paper? (As a colleague of mine is fond of saying: “Where’s the beef?”)

2. How does the arguments in this paper relate to earlier partial results or attempts on the problem? Are there clear analogues between the steps here and steps in earlier papers? Does the new work shed some light as to why previous approaches did not fully succeed? Is this discussed in the paper?

3. What is the simplest, shortest, or clearest new application of that idea? A related question: what is the first non-trivial new statement made in the paper, that was not able to have been shown before by earlier methods? Is this proof-of-concept given in the paper, or does it jump straight to the big conjecture with all its additional (and potentially error-prone) complications? In the event that there is a fatal error in the full proof, is there a good chance that a deep and non-trivial new partial result can at least be salvaged?

4. Any major problem comes with known counterexamples, obstructions, or philosophical objections to various classes of attack strategies (e.g. strategy X does not work because it does not distinguish between problem Y, which is the big conjecture, and problem Z, for which counterexamples are known). Do you know why your argument does not encounter these obstructions? Is this stated in the paper? Do you know any specific limitations of the argument? Are these stated in the paper also?

5. What was the high-level strategy you employed to attack the problem? Was it guided by some heuristic, philosophy, or intuition? If so, what is it? Is it stated in the paper? If the strategy was “continue blindly transforming the problem repeatedly until a miracle occurs”, this is a particularly bad sign. Can you state, in high-level terms (i.e. rising above all the technical details and computations), why the argument works?

6. Does the proof come with key milestones – such as a key proposition used in the proof which is already of independent interest, or a major reduction of the unsolved problem to one which looks significantly easier? Are these milestones clearly identified in the paper?

7. How robust is the argument – could a single sign error or illegal use of a lemma or formula destroy the entire argument? Good indicators of robustness include: alternate proofs (or heuristics, or supporting examples) of key steps, or analogies between key parts of the argument in this paper and in other papers in the literature.

8. How critically have you checked the paper and reworked the exposition? Have you tried to deliberately disprove or hunt for errors in the paper? One expects a certain amount of checking to have been done when a major paper is released; if this is not done, and errors are quickly found after the paper is made public, this can potentially be quite embarrassing. Note that there is usually no rush when solving a major problem that has already withstood all attempts at solution for many years; taking the few extra days to go through the paper one last time can save oneself a lot of trouble.

9. How much space in the paper is devoted to routine and standard theory and computations that already appears in previous literature, and how much is devoted to the new and exciting stuff which does not have any ready counterpart in previous literature? How soon in the paper does the new stuff appear? Are both parts of the paper being given appropriate amounts of detail?

Also, to reduce any potential negative reception to such a paper (especially if – as is all too likely – significant errors are detected in it) – any bragging or otherwise self-promoting text with little informative mathematical content should be kept to a minimum in the title, abstract, and introduction of the paper. For instance:

• Example of a bad title: “A proof of the Poincaré conjecture”
• Example of a good title: “The entropy formula for the Ricci flow and its geometric applications”

More generally, given any major open problem, the importance of the problem and its standard history will be a given to any informed reader, and should only be given a perfunctory treatment in the paper, except for those portions of the history of the problem which are of relevance to the proof. Pointing out that countless great mathematicians had tried to solve the problem and failed before you came along is in particularly bad taste and should be avoided completely.

It should also be noted that due to the sheer volume of failed attempts at solving these problems, most professional mathematicians will refuse to read any further attempts unless there is substantial auxiliary evidence that there is a non-zero chance of correctness (e.g. a previous track record of recognised mathematical achievement in the area). See for instance my editorial policy on papers involving a famous problem, or Oded Goldreich’s page on solving famous problems.

See also Scott Aaronson’s “Ten signs a claimed mathematical proof is wrong” and Dick Lipton’s “On Mathematical Diseases”.