OpenAI 722 Mathematical Discoveries in One Go

OpenAI 722 Mathematical Discoveries landed in one go. It proved and disproved problems across a sweeping range of mathematics, and the volume alone has senior academics reaching for two words, impressive and baffling, because they're seeing something they can't easily explain. OpenAI didn't name the model. It's described only as an "internal frontier model." And that's it.

A Flood of Proofs

The papers appeared on GitHub. That's a platform far better known for sharing computer code than for hosting mathematical research, and it's a choice that stood out on its own. Days earlier, the preprint server arXiv said it was introducing upload limits to stem the tide of submissions it judged "low-value.

In 2019, AI couldn't pass GCSE maths. Last month, OpenAI solved a Navier-Stokes problem, one of the toughest and most enduring puzzles in fluid dynamics, a result that shows how far these systems have come in just a few years. It's depth. Now it's chasing breadth. OpenAI 722 Mathematical Discoveries follow that sharp upward arc, and they're proof of the climb.

• 722 papers proving and disproving mathematical problems
• No named model, described only as an "internal frontier model"
• Publication on GitHub rather than a traditional journal or preprint server

One Problem, 25 Years

Francis Johnson at University College London spent a quarter century on Wall's D(2) problem, one of the hundreds of puzzles in the release.

"I worked on this problem for 25 years. I produced two books on it. I, personally, gave up," says Johnson. "I'm surprised that [AI] has done it quite so quickly, but I'm not surprised that it's done it."

Johnson has been working with AI models in recent months and says he was "astonished with the sophistication and clarity of the analysis that they gave." The feeling cuts both ways.

"It puts us all in a very strange position. Let's face it, the genie is out of the bottle now. We're going to have to live with it. Human beings are supposed to be adaptable, so we're going to have to adapt. I think for the moment, we just stand back and be astonished," he says.

Trust Takes Time

Celebration may be premature. Kevin Buzzard at Imperial College London examined the 30 papers relevant to his field of number theory. Only seven seemed impressive, he says. Only one was formally verified in Lean, a type of computer analysis that can prove mathematical results beyond reasonable doubt.

"Unfortunately, acceptance of these results by the community will take time, and journalists are going to have to wait while the mathematicians do their job," says Buzzard. "The six unformalised results will have to wait until either an expert is motivated to read and check the text, or a Lean formalisation is produced."

Still, Buzzard says the upward trend in AI mathematics is astonishing. It's astonishing, full stop. And if the new results turn out to be mostly true, he says, the field will get some idea of the new normal, though we don't yet know what that normal looks like or how long it'll take to settle in.

"It has been a long time since there were humans who were experts in all of mathematics, but now we seem to have machines with this property," he says.

Whether OpenAI 722 Mathematical Discoveries will be accepted as genuine additions to human knowledge is now up to the mathematicians.

The Verification Bottleneck

Ben Allanach at the University of Cambridge points out that AI companies have been hiring mathematicians and pouring resources into research. The results have followed. Companies see mathematical discoveries as "an intellectual trophy, an achievable intellectual trophy," he says.

That makes the moment "confusing and disrupting.Allanach adds: "I'm glad I'm not a pure mathematician, because you'd be saying, 'Well, what's the point'."" He also questions whether the scale helps anyone. "Is it coming into the cadre of human knowledge?" he asks. Mathematicians enjoy the process of solving puzzles, he notes, and checking a machine's working is probably not as attractive.

What OpenAI Promises Next

Terence Tao at UCLA said last month that the way discoveries are being made and "dumped" on the mathematical community may harm the field, and that's a serious concern for everyone involved. An independent Advisory Group on Mathematics and Artificial Intelligence was set up to advise AI companies on best practice. OpenAI previously said it would take the advice on board. But OpenAI 722 Mathematical Discoveries, however, appear to go against several of the group's suggestions, and they're not small ones. It's a problem.

OpenAI 722 Mathematical Discoveries arrived without a model name, prompts, or an estimated computation cost attached. The group recommends that AI labs release:

• The model's name
• The prompts used
• The estimated cost of computation
• Formalised proofs wherever possible
• How many comparable problems the models tried and failed to solve
• How the problems were chosen

Lindsay McCallum Rémy at OpenAI wrote in an email announcing the results. She said the company is continuing to explore community-hosted alternatives for this release that meet the committee's guidelines, and for future releases, she wrote, the company is committed to improving the quality of the papers through citations, mathematical exposition, and presentation of the results. We want to give the mathematical community time and space to assess this work.