“`html
A novel imaging method created by MIT researchers may allow quality-control robots in a warehouse to look through a cardboard shipping container and observe that the handle of a mug concealed beneath packing peanuts is damaged.
Their method utilizes millimeter wave (mmWave) signals, which are the same kind of signals employed in Wi-Fi, to generate precise 3D models of objects that are obstructed from sight.
The waves can penetrate typical barriers like plastic containers or interior walls and bounce off concealed items. This system, termed mmNorm, gathers those reflections and inputs them into an algorithm that estimates the shape of the object’s surface.
This innovative technique achieved 96 percent reconstruction accuracy across a variety of common items with intricate, curvy forms, such as silverware and a power drill. Cutting-edge baseline methods reached only 78 percent accuracy.
Moreover, mmNorm does not necessitate extra bandwidth to attain such high precision. This efficiency could facilitate the method’s application in diverse environments, from factories to assisted living facilities.
For example, mmNorm could empower robots in a factory or home to differentiate between tools hidden in a drawer and recognize their handles, allowing for more effective gripping and manipulation of the items without causing harm.
“We’ve been intrigued by this issue for some time, but we’ve run into obstacles because previous methods, though mathematically refined, weren’t advancing us toward our goals. We needed to devise a fundamentally different approach to utilizing these signals compared to what has been implemented for over fifty years to unlock new applications,” states Fadel Adib, associate professor in the Department of Electrical Engineering and Computer Science, director of the Signal Kinetics group at the MIT Media Lab, and lead author of a paper on mmNorm.
Adib’s co-authors on the paper include research assistants Laura Dodds, the primary author, Tara Boroushaki, and former postdoctoral researcher Kaichen Zhou. This research was recently showcased at the Annual International Conference on Mobile Systems, Applications and Services.
Reflecting on reflections
Conventional radar methods send mmWave signals and receive reflections from the environment to identify hidden or distant items—a technique called back projection.
This approach is effective for large objects, like an airplane concealed by clouds, but the image clarity is insufficient for small items such as kitchen gadgets that a robot may need to recognize.
In examining this challenge, the MIT researchers discovered that current back projection methods overlook a significant property known as specularity. When a radar system emits mmWaves, nearly every surface the waves encounter behaves like a mirror, producing specular reflections.
If a surface faces the antenna, the signal will bounce off the object toward the antenna, but if the surface is oriented differently, the reflection will move away from the radar and will not be captured.
“By leveraging specularity, our concept is to try to estimate not only the position of a reflection in the environment but also the orientation of the surface at that point,” says Dodds.
They developed mmNorm to estimate what is termed a surface normal, which refers to the direction of a surface at a specific point in space, and apply these estimates to recreate the curvature of the surface at that location.
By combining surface normal estimates at each spatial point, mmNorm employs a specialized mathematical formulation to reconstruct the 3D object.
The researchers built an mmNorm prototype by attaching radar to a robotic arm that continuously gathers measurements as it moves around a concealed item. The system compares the strength of the signals it receives at various locations to gauge the curvature of the object’s surface.
For instance, the antenna will capture the most robust reflections from a surface directed straight at it and weaker signals from surfaces that are not directly facing the antenna.
Since multiple antennas on the radar capture a portion of the reflections, each antenna “votes” on the direction of the surface normal based on the signal strength it recorded.
“Some antennas may have a very strong vote, while others may have a much weaker vote, and we can aggregate all votes to produce a single surface normal that is unanimously supported by all antenna positions,” says Dodds.
Furthermore, as mmNorm estimates the surface normal from all points in space, it generates multiple potential surfaces. To pinpoint the correct one, the researchers applied techniques from computer graphics, formulating a 3D function that selects the surface most representative of the received signals. This is used to create a final 3D reconstruction.
Finer details
The team assessed mmNorm’s ability to reconstruct over 60 objects with intricate shapes, such as the handle and curve of a mug. It produced reconstructions with approximately 40 percent less error than leading-edge techniques, while also accurately estimating the position of an object.
Their new method can also distinguish between several objects, such as a fork, knife, and spoon concealed within the same box. It also performed well for items composed of a variety of materials, including wood, metal, plastic, rubber, and glass, as well as combinations of materials, though it is ineffective for items hidden behind metal or very thick walls.
“Our qualitative outcomes truly speak for themselves. The noticeable improvement simplifies the development of applications that utilize these high-resolution 3D reconstructions for new tasks,” says Boroushaki.
For instance, a robot can differentiate between various tools in a box, determine the exact shape and location of a hammer’s handle, and subsequently plan to grasp it for use in a task. One could also utilize mmNorm with an augmented reality headset, allowing a factory worker to view lifelike representations of completely concealed objects.
It could also be integrated into existing security and defense applications, yielding more accurate reconstructions of concealed items in airport security scanners or during military reconnaissance.
The researchers aim to investigate these and additional potential applications in their future work. They also aspire to enhance the resolution of their technique, increase its efficiency for less reflective objects, and enable the mmWaves to effectively image through thicker obstacles.
“This work truly signifies a paradigm shift in the way we are conceptualizing these signals and the 3D reconstruction process. We’re eager to see how the insights we’ve gained here can have a widespread impact,” Dodds says.
This work is funded, in part, by the National Science Foundation, the MIT Media Lab, and Microsoft.
“`