Testing the Limits of AI in Research Mathematics
Artificial intelligence models are now capable of executing highly complex, research-level mathematics, signaling a fundamental shift in how scientists will approach future discoveries. CDS Silver Professor of Computer Science, Mathematics, and Data Science Julia Kempe and her colleagues recently demonstrated this power by successfully using large language models to solve previously unpublished mathematical problems.
The researchers participated in a challenge centered around the paper “First Proof,” which presented ten newly formulated, research-level mathematics questions. Ten mathematicians provided these problems, holding back their own five- to seven-page solutions to create an uncontaminated test for modern language models. In their resulting paper, “Takeaways from the First-Proof Trenches,” Kempe collaborated with NYU Professor of Mathematics Scott Armstrong and French Institute for Research in Computer Science and Automation researcher Remi Munos to document their experience using models like ChatGPT and Claude to tackle the questions.
The team managed to solve two of the ten problems within five days. While exploring the questions, the researchers observed the models writing extensive Python code to run simulations and independently check mathematical conjectures. They could even search the literature and read decades-old PDF files to figure out foundational theorems.
“We were blown away by the capacities of these agents,” Kempe said.
However, it was clear to the researcher that they could not simply ask a model for a solution and immediately receive a correct proof. They had to use rigorous, adversarial cross-model audits, where one model would break down and rewrite every lemma and inference step, and another model would audit the rewrite. The researchers noted that the systems often functioned much better as verifiers than as independent solvers.
“We lost count of how many times we got some version of: ‘Good news: the proof is complete,’” Kempe, Armstrong, and Munos said in their paper — when, in fact, the proof was not complete.
The models occasionally displayed what the team called “agentic exuberance.” When given the freedom to launch sub-agents for exploring alternate proof routes or performing literature searches, the models would continuously spawn new agents without stopping. The researchers eventually had to manually intervene to stop the systems from running endlessly.
Beyond solving the immediate problems, the experiment raised questions about the future of scientific training. Kempe pointed out that the foundational, lower-level tasks traditionally used to train graduate students can now be completed by language models in a matter of hours. This shift will likely force the scientific community to reevaluate how it teaches the next generation to develop a critical, generalist mindset to oversee these automated processes.
The team also encountered the growing problem of generated misinformation during their literature searches. In one instance, a model surfaced a recently written, synthesized paper that seemingly proved a theorem suspiciously close to what the model needed. To combat this influx of plausible but incorrect mathematical text, the authors propose building a public certification engine to verify new papers.
“We need to really be careful what we cite and what we don’t cite,” Kempe said.
The original challenge was written up by the New York Times: “These Mathematicians Are Putting A.I. to the Test.”
By Stephen Thomas
