Can AI comprehend? 

Picture an ant maneuvering through sand, outlining a pattern that coincidentally resembles Winston Churchill. Would you assert that the ant fashioned a likeness of the former British prime minister? As per the late Harvard philosopher Hilary Putnam, most individuals would respond negatively: The ant would have to be aware of Churchill, lines, and sand. 

This thought experiment has taken on fresh significance in our era of generative AI. As firms in artificial intelligence unveil increasingly sophisticated models that reason, research, create, and analyze, the interpretations behind those actions become elusive swiftly. What does it truly mean to consider, to comprehend, to possess knowledge? The implications are substantial for AI’s utilization, yet scholars of intelligence are still grappling with these questions.

“When we observe entities that communicate like humans, capable of executing numerous tasks akin to humans, composing proofs and poetry, it’s instinctive for us to conclude that the only explanation for their abilities is that they harbor a mental representation of the world, analogous to humans,” stated Keyon Vafa, a postdoctoral fellow at the Harvard Data Science Initiative. “Our field is making strides to understand what it would even signify for something to comprehend. There’s undeniably no agreement.” 

In human thought, expressing an idea suggests comprehension of it, remarked senior lecturer on philosophy Cheryl Chen. We presume that someone who states “It’s raining” is knowledgeable about weather, has felt rain on their skin, and possibly the annoyance of forgetting an umbrella. “For true comprehension,” Chen mentioned, “you need to be somewhat embedded in the world in a manner that ChatGPT is not.”

Nonetheless, modern AI systems can appear astonishingly persuasive. Both extensive language models and other forms of machine learning consist of neural networks — computational frameworks that transmit information through layers of neurons loosely inspired by the human brain. 

“Neural networks contain numerical values; we refer to them as weights,” explained Stratos Idreos, Gordon McKay Professor of Computer Science at SEAS. “These values start off randomly. We input data into the system, perform mathematical operations based on those weights, and obtain outcomes.” 

He provided the example of an AI designed to spot tumors in medical images. You feed the model numerous images confirmed to have tumors and numerous others confirmed not to. Can the model accurately ascertain if a new image contains a tumor? If the prediction is incorrect, you supply the system with additional data and adjust the weights, gradually guiding the system toward the correct output. It may even identify tumors overlooked by doctors. 

Vafa dedicates much of his research to rigorously assessing AI, striving to determine both what the models genuinely comprehend and how we would definitively ascertain it. His criteria hinge on whether the model can consistently demonstrate a world model, a steady yet adaptable framework that enables it to generalize and reason even in uncharted situations.

At times, Vafa observed, it does appear to be a yes. 

“If you examine large language models and pose questions they presumably haven’t encountered before — like, ‘If I intended to balance a marble atop an inflatable beach ball placed on a pot atop grass, what sequence should I arrange them in?’ — the LLM would provide the correct response, despite that specific inquiry not being among its training data,” he noted. This implies the model possesses an effective world model — in this scenario, the principles of physics.

However, Vafa contends that the world models frequently disintegrate under closer scrutiny. In a paper, he and a group of colleagues trained an AI model on street directions throughout Manhattan, subsequently requesting routes between various locations. Ninety-nine percent of the time, the model produced precise directions. Yet, when they attempted to construct a cohesive map of Manhattan from its data, they discovered the model had fabricated streets, crossed Central Park, and traversed diagonally across the city’s renowned right-angled grid. 

Rather than functioning from a consistent understanding of reality, he argues, AI memorizes numerous rules and applies them as effectively as possible, which appears deliberate most of the time but occasionally exposes its inherent inconsistencies.

Sam Altman, the CEO of OpenAI, has stated that we will attain AGI — artificial general intelligence, capable of performing any cognitive task a human can — “relatively soon.” Vafa is vigilant for more elusive indicators: that AIs consistently exhibit coherent world models — in other words, that they comprehend. 

“I believe one of the significant obstacles to achieving AGI is that it’s ambiguous how to define it,” remarked Vafa. “This is why identifying methods to assess how effectively AI systems can ‘comprehend’ or if they possess strong world models is crucial — it’s difficult to envision any concept of AGI that doesn’t entail having a sound world model. The world models of current LLMs are deficient, but once we figure out how to evaluate their quality, we can advance towards enhancing them.”

Idreos’ team at the Data Systems Laboratory is creating more efficient strategies so AI can process larger datasets and deduce more rigorously. He envisions a future where tailored, intrinsically designed models address critical issues, such as pinpointing treatments for rare ailments — even if the models lack awareness of what the illness is. Regardless of whether that qualifies as understanding, Idreos noted, it certainly qualifies as beneficial.