Despite their remarkable abilities, large language models (LLMs) frequently underperform when confronted with difficult new challenges that necessitate intricate reasoning skills.

While an accounting firm’s LLM may thrive in summarizing financial documents, that very model might falter unexpectedly if asked to forecast market fluctuations or detect fraudulent activities.

To enhance LLMs’ adaptability, researchers from MIT examined how a specific training methodology can be effectively utilized to enhance a model’s efficacy on unforeseen, challenging tasks.

They demonstrate that test-time training—a technique that involves momentarily adjusting certain aspects of a model’s internal mechanisms during use—can yield a sixfold increase in precision. The researchers devised a framework for executing a test-time training approach that leverages instances of the new task to amplify these benefits.

This research could increase a model’s versatility, allowing a conventional LLM to adapt to sophisticated tasks requiring planning or abstraction. This may result in LLMs that exhibit greater accuracy in numerous applications demanding logical reasoning, from medical diagnostics to supply chain coordination.

“True learning — what we accomplished here through test-time training — is an ability these models lack post-deployment. They cannot acquire new skills or improve at a given task. However, we have illustrated that with a slight push for actual learning, notable performance enhancements can occur,” says Ekin Akyürek PhD ’25, the primary author of the study.

Akyürek is accompanied on the paper by graduate students Mehul Damani, Linlu Qiu, Han Guo, and Jyothish Pari; undergraduate Adam Zweiger; and senior authors Yoon Kim, an assistant professor in Electrical Engineering and Computer Science (EECS) and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL); and Jacob Andreas, an associate professor in EECS and a CSAIL member. The findings will be presented at the International Conference on Machine Learning.

Addressing difficult domains

Users of LLMs frequently seek to enhance their model’s performance on a novel task through a technique known as in-context learning. They supply the model with several examples of the new task as textual prompts that direct the model’s outputs.

However, in-context learning does not consistently yield results for challenges that necessitate logic and reasoning.

The MIT researchers explored how test-time training could be employed alongside in-context learning to enhance performance on these demanding tasks. Test-time training consists of adjusting certain model parameters—the internal variables utilized to generate predictions—utilizing a small amount of new task-specific data.

The researchers investigated the interplay between test-time training and in-context learning. They examined design choices that optimize the performance enhancements that can be extracted from a general-purpose LLM.

“We discovered that test-time training represents a significantly stronger form of learning. While merely providing examples can marginally elevate accuracy, actually modifying the model with those examples can result in dramatically enhanced performance, especially in challenging areas,” Damani states.

In-context learning necessitates a limited set of task instances, comprising problems along with their solutions. The researchers utilize these instances to construct a task-specific dataset required for test-time training.

To enlarge the dataset, they generate new inputs by making slight modifications to the problems and solutions in the examples, such as by horizontally flipping certain input data. Their findings indicate that training the model with the outputs from this expanded dataset yields the most favorable results.

Moreover, the researchers update only a small fraction of model parameters utilizing a method called low-rank adaptation, which heightens the efficiency of the test-time training procedure.

“This is crucial since our approach must be efficient if it is to be implemented in real-world scenarios. We find that you can achieve significant accuracy improvements with minimal parameter training,” Akyürek remarks.

Acquiring new capabilities

Streamlining the process is vital, as test-time training is applied on an individual instance basis, meaning a user would need to execute this for each task separately. The alterations to the model are fleeting, and the model reverts to its original state after producing a prediction.

A model that typically responds to a query in under a minute might take five to ten minutes to generate an answer with test-time training, Akyürek adds.

“We wouldn’t want to use this approach for every user query, but it is beneficial for particularly challenging tasks where you desire the model to perform exceptionally. There might also be tasks that are too demanding for an LLM to tackle without this method,” he explains.

The researchers evaluated their strategy on two benchmark datasets of exceedingly intricate problems, like IQ puzzles. This led to an accuracy increase as much as sixfold compared to techniques that only leverage in-context learning.

Tasks characterized by structured patterns or those involving entirely unfamiliar data types demonstrated the most significant performance enhancements.

“For simpler tasks, in-context learning may suffice. However, updating the parameters themselves might impart a new skill to the model,” notes Damani.

Looking ahead, the researchers aim to apply these insights towards developing models that are capable of continuous learning.

The ultimate aspiration is to create an LLM that, upon receiving a query, can autonomously determine whether it should employ test-time training to adjust parameters or if it can resolve the task using in-context learning, subsequently implementing the best test-time training approach without human oversight.

This research is partly funded by the MIT-IBM Watson AI Lab and the National Science Foundation.