Envision a sphere rebounding down a staircase. Now consider a stream of water cascading down those very same stairs. The sphere and the water exhibit markedly different behaviors, and it appears that various regions of your brain are designated for interpreting visual input concerning each type of material.

In a recent investigation, MIT neuroscientists have pinpointed areas within the brain’s visual cortex that react preferentially when observing “objects” — that is, rigid or malleable items like a bouncing sphere. Other neural areas show heightened activation when looking at “substances” — liquids or granular matter such as sand.

This differentiation, previously unobserved in the brain, may assist in guiding the brain’s approach to interacting with various forms of physical materials, according to the researchers.

“When you observe some liquid or viscous material, you engage with it in a different manner than with a solid object. For a solid, you might lift or grasp it, while for liquid or viscous materials, you’re likely to require a tool to manage it,” explains Nancy Kanwisher, the Walter A. Rosenblith Professor of Cognitive Neuroscience; a member of the McGovern Institute for Brain Research and MIT’s Center for Brains, Minds, and Machines; and the lead author of the study.

MIT postdoctoral researcher Vivian Paulun, who will join the faculty at the University of Wisconsin at Madison this fall, is the primary author of the paper, which is published today in the journal Current Biology. RT Pramod, an MIT postdoctoral researcher, and Josh Tenenbaum, an MIT professor of brain and cognitive sciences, are also contributors to the study.

Substances vs. objects

Many years of brain imaging research, including initial studies by Kanwisher, have unveiled areas within the brain’s ventral visual pathway that are crucial for recognizing the contours of 3D objects, specifically a region known as the lateral occipital complex (LOC). An area in the brain’s dorsal visual pathway, termed the frontoparietal physics network (FPN), examines the physical characteristics of materials, such as mass or stability.

Despite significant insights into how these pathways respond to various features of objects, the majority of past studies have been centered on solid items, or “objects.”

“No one has examined how we interpret what we refer to as ‘substances’ — that is, liquids, sand, honey, water, and various gooey materials. Thus, we resolved to investigate that,” states Paulun.

These viscous materials operate quite differently than solids. They flow instead of rebound, and dealing with them typically necessitates containers and tools like spoons. The researchers speculated that these physical properties might call for the brain to allocate specialized regions to interpret them.

To investigate how the brain processes these materials, Paulun utilized a software application tailored for visual effects professionals to produce over 100 video clips displaying various types of objects and substances interacting with their environment. In the videos, these materials could be observed sloshing or tumbling within a transparent container, being dropped onto another object, or rebounding or flowing down a staircase.

The researchers employed functional magnetic resonance imaging (fMRI) to examine the visual cortex of participants as they viewed the videos. They discovered that both the LOC and the FPN respond to “objects” and “substances,” but each pathway exhibits distinct subregions that react more vigorously to one category or the other.

“Both the ventral and dorsal visual pathways appear to have this subdivision, with one component reacting more intensely to ‘objects,’ and the other to ‘substances,’” Paulun notes. “We haven’t observed this previously because no one has inquired about it before.”

Roland Fleming, a professor of experimental psychology at Justus Liebig University of Giessen, described the outcomes as a “significant advancement in the scientific understanding of how our brains represent the material properties of our environment.”

“We have long recognized the psychological distinction, but this is the first occasion it has been effectively mapped to distinct cortical structures in the brain. Now we can explore the different computations unique brain regions employ to process and represent objects and materials,” states Fleming, who was not a part of the study.

Physical engagements

The results imply that the brain may utilize different methods to represent these two categories of materials, akin to the artificial physics engines utilized in video game graphics. These engines typically depict a 3D object as a mesh while fluids are represented as collections of particles that can be rearranged.

“The intriguing hypothesis we can derive from this is that perhaps the brain, similar to artificial game engines, possesses separate computations for simulating and representing ‘substances’ and ‘objects.’ This is something worth investigating further,” Paulun notes.

The researchers also propose that these regions may have evolved to aid the brain in comprehending essential distinctions, enabling it to strategize interactions with the physical world. To explore this possibility further, the team intends to examine whether the areas involved in processing solid objects are also activated when a neural circuit associated with planning to grasp items is in use.

Additionally, they hope to determine if any regions within the FPN correlate with the processing of more specific attributes of materials, like liquid viscosity or an object’s bounciness. In the LOC, they aim to investigate how the brain conveys alterations in the shape of fluids and malleable substances.

This research was supported by the German Research Foundation, the U.S. National Institutes of Health, and a U.S. National Science Foundation grant awarded to the Center for Brains, Minds, and Machines.