AI transitions from mathematical novice to expert

When Michael Brenner delivered a graduate course entitled “Applied Mathematics 201” in the autumn of 2023, the nonlinear partial differential equations proved too challenging for artificial intelligence. AI could only resolve about 30 to 50 percent of the problems during the initial three weeks of the course. 

“It was acceptable, but not particularly impressive,” remarked Brenner, the Catalyst Professor of Applied Mathematics and Applied Physics and of Physics at the John A. Paulson School of Engineering and Applied Sciences.

However, when he conducted the same class this recent spring, everything had evolved. The same AI models that had faltered on simpler problems now excelled at the most challenging ones. Brenner was astonished. “This really brings into question the whole method of instruction in the course,” he noted. 

The course, engaged in by many graduate students, has garnered a reputation for its difficulty. For over twenty years, Brenner has instructed it, traditionally giving students take-home exams; he preferred they have the opportunity to grapple with challenging queries without the pressure of a ticking clock. But if ChatGPT could handle the examination for his students, could he genuinely rely on their submissions? 

He faced two options: Prohibit AI altogether, or accept it and overhaul the course that has been offered at Harvard since before his birth. 

He opted to redesign the course. 

Forms of artificial intelligence have been utilized in mathematics for numerous years. Yet in recent times, advancements in machine learning and significant improvements in publicly accessible large language models have begun to transform the field. While some mathematicians anticipate modest tools that can automate less glamorous tasks, others foresee a comprehensive redefinition of the discipline and a rapid expansion of what’s achievable. 

And the belief that AI struggles with math? Brenner asserts: Simply untrue.

“If you take your favorite large language model from two years ago and you request it to please sum 372 and 476, and it provides an incorrect answer, then you label it as bad at math,” Brenner explained. “However, what I would contend is that’s a misapplication of a large language model. Obviously, if you’re given those two numbers, you should reach for a calculator.”

Evidence of AI’s mathematical prowess is accumulating. In 2024, two AI models from Google DeepMind received a silver medal in the International Mathematical Olympiad, the largest and most prestigious competition for young mathematicians. Additionally, in 2024, Demis Hassabis and John Jumper of DeepMind were awarded the Nobel Prize in Chemistry for their AI model AlphaFold2, which predicted the structure of nearly all 200 million recognized proteins. 

“It is now feasible to create a computational model that leads scientifically to the point where, within years of its publication, it earns a Nobel Prize,” Brenner stated. “That’s unprecedented.”

Knots and murmurations

While some mathematicians remain cautious of AI’s propensity to hallucinate, specialized machine learning systems are expediting mathematical discoveries across various fields. 

“Computers can process data sets that are too expansive for individuals to sift through, and they can identify patterns that humans consider intriguing and significant,” said Michael Douglas, senior research scientist at the Center for Mathematical Sciences and Applications at Harvard. “This has proven very beneficial for mathematicians whose subjects of study can be organized into data sets.”

One instance: knot theory, a field with applications in physics, biology, and chemistry. 

With an infinite array of potential knots, researchers explore databases containing hundreds of millions of mathematical knots in search of connections among them. In 2021, researchers at DeepMind, some of whom were associated with Harvard, employed AI to uncover new connections between knot invariants, the numerical features that characterize each knot’s properties. This breakthrough could have taken human mathematicians years to uncover using conventional methods. 

Another significant advancement occurred in research concerning elliptic curves, seemingly simple structures with profound implications in both pure mathematics and cryptography. When Harvard researchers input curve data into machine learning systems, they observed that the curves’ behavior resembled murmurations — those swirling, coordinated movements of flocks of birds. 

“No mathematician had previously considered looking for that, and they were quite taken aback by the discovery,” Douglas recounted. “They are currently engaged in attempts to prove it.”

However, perhaps the type of AI attracting the most attention in the mathematical community is automated theorem proving. For decades, mathematicians have relied on computer systems to verify that theorems are logically valid. It’s an efficient approach, but translating human-written proofs into formats readable by computers is labor-intensive, and the proofs produced by automated provers tend to be quite extensive. 

Enter generative AI. 

“The expectation, and what people are beginning to pursue now, is that we will utilize large language models to formulate proofs in a manner that the computer can evaluate,” Douglas mentioned. 

It’s resolving one dilemma with another, Douglas noted. Generative AI can almost instantaneously translate
“““html

proofs into formats that automated systems can authenticate, while the validation procedure uncovers any AI-generated inaccuracies or fabrications.

Undoubtedly, just as AI is shaping mathematics, mathematics is also shaping AI. Melanie Weber, assistant professor of applied mathematics and computer science at SEAS, leverages classical geometric tools to construct “geometric” AI models that are more effective and transparent.

“Artificial intelligence is already transforming science. However, the existing models demand extensive data and computational resources, which can be scarce in the sciences and raise sustainability issues. Encoding geometric structures, such as symmetries from physical laws, can enhance the efficiency of models by narrowing their focus to feasible physical conditions,” she indicated. “This implies that the number of cases we need to contemplate during training can be significantly minimized by including such structures. Essentially, this means we require less data and computational power to develop a competent model if we integrate that structure into the model from the outset.”

The future of mathematics

While the pace at which AI will alter mathematics remains uncertain, it is clear that a change is underway. Weber envisions the technology functioning as both a research aide, managing literature assessments and proof validations, and as a platform for brainstorming, aiding mathematicians in addressing challenges more rapidly. Brenner noted that this kind of acceleration could be groundbreaking.

“My aspiration is that we can tackle problems more quickly and accomplish more,” he expressed. “Science is limitless. There are no boundaries.”

Education, Brenner added, is also boundless. In the 2025 iteration of “Applied Mathematics 201,” he eliminated conventional homework assignments. Instead, students were tasked with creating their own problems, having peers validate them, and testing if they could outsmart an AI. (“One of the proficient ones,” Brenner clarified. “If the less capable models can resolve your problem, it doesn’t qualify.”)

By semester’s end, the students devised nearly 700 math challenges of escalating difficulty. The findings on whether AI could resolve them might prove advantageous for researchers.

“It’s astonishing, because I’m teaching math, right? Here we have a scenario where students are inventing math challenges that become progressively more complex and striving to solve them. That is the aspiration.”

Brenner remarked, “Challenges are everywhere. How to enhance climate prediction. How to comprehend what modifications could make Earth more hospitable. Models that would aid in drug discovery. … The possibilities are limitless in this realm, and we are unaware of what could be achieved. But certainly, this is not a mundane time to engage in this work.”

“`