There’s a sense in which this is a follow-up to van der Waerden’s theorem, but I’d like to emphasise that it’s very interesting in its own right!
Let’s remind ourselves what van der Waerden‘s theorem says. (If you haven’t come across it before, you might like to read that blog post first.) I’ll phrase it very slightly differently from last time, so that I use the standard terminology, but hopefully you’ll see that it’s the same thing. Imagine that is a large positive integer, and colour each of the numbers from to using one of colours. The theorem tells us that if is large enough (depending on and ) then there are evenly spaced numbers that are the same colour. The fancy name for this is a monochromatic arithmetic progression of length .
One thing mathematicians like to do is to get an understanding of why a theorem is true by testing whether the conditions are all really necessary. It seems reasonable to try to explore what’s really going on with van der Waerden’s theorem. Is there some important feature that occurs because we’re colouring the numbers with colours, or is it really just that if we colour the numbers with colours then there must be quite a lot of numbers with the same colour? It turns out that the latter is true (although one could perhaps argue that the former is also true — but I shan’t dwell on that).
In 1936, Erdős and Turán made a conjecture along these lines. I’ll call it a theorem, since it now is, and once I’ve stated it I’ll carry on telling the story!
Theorem (Szemerédi) Let be an integer greater than or equal to , and let be a positive number less than 1. If is large enough (depending on and ), then any subset of with size at least must contain an arithmetic progression of length .
Let’s try to make sense of this by seeing how this relates to van der Waerden’s theorem. If we colour the numbers from to using colours, then there must be some colour that’s used on at least of them. Let’s say it’s blue, and let’s say the blue numbers form the subset of . Then has size at least where . Szemerédi’s theorem tells us that if is big enough, then must contain an arithmetic progression of length . That is, there must be evenly spaced blue numbers. So Szemerédi’s theorem implies van der Waerden’s theorem: the latter is a special case of the former. This is what I meant by saying that the colouring somehow wasn’t important — what mattered is that if we use only a few colours, then there must be lots of numbers with the same colour.
I hope you’ll agree that it’s an interesting theorem in its own right, but that’s not the whole picture: this theorem is actually pretty important, for reasons to do with the story of its proof. So I’ll go back to that story.
As I said, this was a conjecture in 1936. As far as I know, the first progress was in 1953, when Roth proved the theorem for : he proved that a dense subset of the integers must contain an arithmetic progression of length 3. That might not sound like much, but it was a big step forwards. Roth’s proof used a technique called Fourier analysis. At first sight (to those who have met Fourier analysis) it might seem that this technique has nothing to do with the problem, but Roth was able to deploy it very effectively.
The next big development was in about 1975, when Szemerédi was able to prove the conjecture for all values of , using a combinatorial argument about which I know nothing except that it involved the Szemerédi regularity lemma, which is now a major tool in combinatorics. At this point, it was officially a theorem, and so you might expect mathematicians to leave the subject alone and move on to something else. But the story had hardly got started…
In 1977, Furstenberg gave a totally new proof, using ideas from ergodic theory (another area that seems at first sight to have nothing to do with a combinatorial problem). In particular, he developed the Furstenberg correspondence principle, and this and related ideas have led to many proofs of other results, including some that seemed much harder to prove using combinatorial techniques.
OK, so two different proofs. Surely that’s enough? No! The statement of the theorem talks about being big enough. One might very reasonably ask how big that is. I’m not quite sure whether Szemerédi’s proof gave any information about this, but if it did then it wasn’t very helpful. The nature of Furstenberg’s proof means that it gives no quantitative information: it says there is a large enough , but says nothing about how large it is. Roth’s proof for progressions of length 3 does give a bound (it might be that a smaller number would do, but his proof gives a number that certainly works). So perhaps an approach using Fourier analysis for longer progressions would lead to quantitative information about how large must be?
Unfortunately, it turns out that one cannot use ordinary Fourier analysis for longer progressions. Gowers (who also has a blog) was able to demonstrate that it’s impossible. However, he was able to develop a “higher-order Fourier analysis” that does lead to a proof for longer progressions, and in 2001 he published another proof of Szemerédi’s theorem. This has also led to a great deal of other work: trying to understand this higher-order Fourier analysis, working out the best way to think about it, exploring other applications, and so on. The Green–Tao theorem about arithmetic progressions in the primes is a notable example, but there are many others. Gowers’s proof gives the best known bounds for Szemerédi’s theorem.
But still the story goes on. A few years later, several people (such as Nagle-Rödl-Schacht and Gowers) found another approach, using a “hypergraph regularity lemma”. And one can also deduce the theorem from the density Hales-Jewett theorem, and so on… Tao has a list of approaches on his blog.
Why bother to keep finding new proofs of the same result, especially when most of them don’t lead to improved bounds? I find it remarkable what a range of mathematics is used in these proofs (combinatorics, Fourier analysis, analytic number theory, graph theory, hypergraph theory, ergodic theory, …). Many of them have led on to other major work. It’s also fascinating to explore the connections and differences between the various approaches. Even proofs that use totally different mathematics can have elements in common, and exploring this is a very fruitful way of gaining understanding.
I’ve made some suggestions earlier. That blog post of Tao has some links. The Wikipedia page is a bit thin. A much better place to look would be this Scholarpedia article by Green and Tao. Furstenberg’s book Recurrence in Ergodic Theory and Combinatorial Number Theory describes his approach (as do many other books). Additive Combinatorics by Tao and Vu contains a lot of material about Szemerédi’s theorem.