“`html

In the faint illumination of the lab, companions, relatives, and unknowns observed the depiction of a pianist performing for them, the pianist’s fingers cast upon the shifting keys of an actual grand piano that filled the atmosphere with melody.

Gazing at the ethereal musicians, faces and forms blurred at their boundaries, several listeners expressed a strong yet odd belief: “experiencing someone’s presence” while “also realizing that I am the sole individual in the room.”

“It’s challenging to put into words,” remarked another listener. “It felt as though they were present with me, yet simultaneously, not.”

This essence of absence is central to TeleAbsence, a venture from the MIT Media Lab’s Tangible Media group that focuses on innovations designed for creating illusory communication with those who have passed and with former selves.

However, instead of a “Black Mirror”-style situation that recreates literal loved ones, the initiative led by Hiroshi Ishii, the Jerome B. Wiesner Professor of Media Arts and Sciences, aims for what it describes as “poetic encounters” that transcend time and memory.

The project recently released a positioning paper in PRESENCE: Virtual and Augmented Reality that outlines the design concepts behind TeleAbsence, and how it could assist individuals in coping with grief and contemplating how they wish to be remembered.

The spectral pianists of the MirrorFugue project, developed by Tangible Media alumna Xiao Xiao ’09, SM ’11, PhD ’16, stand as one of the most recognized instances of the endeavor. On April 30, Xiao, now the director and principal investigator at the Institute for Future Technologies of Da Vinci Higher Education in Paris, presented findings from the initial experimental study of TeleAbsence through MirrorFugue at the 2025 CHI conference on Human Factors in Computing Systems in Yokohama, Japan.

When Ishii discussed TeleAbsence at the XPANSE 2024 conference in Abu Dhabi, “about 20 individuals approached me afterward, and all expressed that they had tears in their eyes … the presentation reminded them of a spouse or a father who had passed away,” he shares. “One thing is evident: They yearn to see them again and converse with them again, metaphorically.”

Messages in bottles

As the leader of the Tangible Media group, Ishii has been a global pioneer in telepresence, utilizing technology to forge connections across physical distances. However, when his mother passed away in 1998, Ishii notes that the anguish of the loss drove him to contemplate how deeply we desire to connect across the span of time.

His mother was a poet, and one of his initial experiments in TeleAbsence involved creating a Twitterbot that shared snippets of her poetry. Others who followed the account online were so touched that they began sharing images of flowers to the feed to pay tribute to the mother and son.

“That marked a pivotal moment for TeleAbsence, and I wanted to broaden this idea,” Ishii states.

Illusory communication, like the shared poems, is one essential design tenet of TeleAbsence. Although users are aware that the “dialogue” is one-sided, the researchers indicate that it can be soothing and therapeutic to possess a concrete method to reach across time.

Discovering methods to transform memories into tangible forms is another crucial design principle. One of the projects developed by Ishii and his team consists of a collection of glass bottles, reminiscent of the soy sauce containers Ishii’s mother utilized while cooking. When one opens a bottle, the sounds of chopping, sizzling onions, a softly playing radio, and a maternal voice bring a son closer to his mother.

Ishii mentions that sight and sound dominate the current modalities of TeleAbsence technologies because, though the senses of touch, smell, and taste are acknowledged as potent memory triggers, “capturing that kind of multimodal moment is quite a significant challenge.”

Simultaneously, one of the core tenets of TeleAbsence is the essence of absence. These are the physical reminders, or traces, of an individual that serve to evoke both the person and the awareness that they are no longer present. One of the most poignant examples, the researchers note, is the enduring “shadow” of Hiroshima resident Mitsuno Ochi, her silhouette embedded in stone steps 260 meters from where the atomic bomb exploded in 1945.

“Abstraction holds great significance,” Ishii asserts. “We desire something to evoke a moment, not to recreate it physically.”

With the bottles, for example, individuals have inquired of Ishii and his colleagues if it might be more evocative to fill them with a fragrance or beverage. “But our philosophy is to create a bottle that is entirely empty,” he elaborates. “The essential aspect is to allow individuals to envision based on the memory.”

Other significant design principles in TeleAbsence encompass traces of reflection — the ephemeral remnants of faint pen marks and blotched ink on a preserved letter, for example — and the notion of remote time. TeleAbsence should extend beyond merely prompting a memory of a cherished one, the researchers argue, and instead facilitate a sensation of being transported to spend a moment in the past with them.

Time travelers

For Xiao, who has played the piano throughout her life, MirrorFugue is a “deeply personal undertaking” that enabled her to revisit a part of her childhood that was almost forgotten.

Her parents relocated from China to the United States when she was an infant — but it took eight years for Xiao to follow. “The piano, in a sense, was almost like my primary language,” she reminisces. “And when I eventually moved to America, my brain replaced fragments of my childhood where my operating system was once in Chinese, and now it’s predominantly in English. Yet throughout this entire journey, music and the piano remained constant.”

MirrorFugue’s “sense of being nearly there yet not quite, and the desire to connect with one’s past self, stems from my own yearning to reconnect with my historical self,” she adds.

The new MirrorFugue study provides some empirical evidence supporting the concept of TeleAbsence, she notes. Its 28 participants were outfitted with sensors to monitor variations in their heart rate and hand movements during the experience. They were comprehensively interviewed about their perceptions and feelings afterward. The images captured came from pianists ranging from children in the early stages of their lessons to professional pianists like the late Ryuichi Sakamoto.

The researchers found that the emotional experiences reported by the listeners were significantly affected by whether they knew the pianist, as well as whether the pianist was known to the listeners as alive or deceased.

Some participants positioned their hands alongside the apparitions to play spontaneous duets. One daughter, who mentioned she hadn’t paid much attention to her father’s playing while he was alive, was newly impressed by his skill. One individual felt empathy watching his past self grapple with a new piece of music. A young girl, mouth slightly agape in concentration and her small fingers dancing over the keys, presented her mother with a past daughter that couldn’t be captured in old photographs.

The yearning for individuals from the past and former selves can be “a profound sadness that will remain forever,” remarks Xiao. “You will always carry it with you, but it also renders you sensitive to certain aesthetic experiences that are simultaneously beautiful.”

“Once you’ve had that experience, it truly resonates,” she adds, “And I believe that’s why TeleAbsence resonates with so many individuals.”

Uncanny valleys and curated memory

Highly conscious of the potential ethical implications of their research, the TeleAbsence scientists have collaborated with grief researchers and psychologists to enhance their understanding of the ramifications of constructing these temporal bridges.

For instance, “one insight we gained is that it depends on how long ago a person died,” Ishii explains.

“`

“Immediately following death, when it becomes exceedingly challenging for numerous individuals, this depiction is significant. However, crucial informed choices must be made regarding whether this prolongs the mourning too much.”

TeleAbsence has the potential to provide solace to the dying, he asserts, by “realizing that there exists a way for them to persist for their heirs.” He urges individuals to contemplate assembling “high-quality, concise content,” like their social media updates, that could serve this aim.

“Naturally, many families do not share ideal connections, so I can easily envision scenarios where a descendant may lack interest” in engaging with their forebears via TeleAbsence, Ishii remarks.

TeleAbsence must never completely replicate or create new material for a cherished one, he emphasizes, referencing the emergence of “ghost bot” startups, businesses that gather information on individuals to fabricate an “artificial, generative AI-based avatar that communicates what they never articulated, or performs gestures or facial expressions.”

A recently trending video featuring a mother in Korea “reunited” in virtual reality with an avatar of her deceased daughter, Ishii expresses, left him “very disheartened, as they are treating grief as entertainment, a spectacle for an audience.”

Xiao believes there still might be a place for generative AI within the TeleAbsence domain. She is drafting a research proposal for MirrorFugue that would encompass representations of historical pianists. “I think we are at the stage with generative AI where we can produce hand movements, and we can transcribe the MIDI from audio to evoke figures like Franz Liszt or Mozart, significant historical icons.”

“Now, of course, it becomes somewhat complex, and we have deliberated this, the role of AI and how to circumvent the uncanny valley, how to prevent misleading individuals,” she notes. “However, from a researcher’s viewpoint, the potential to empirically investigate these matters genuinely excites me.”

The significance of emptiness

Together with Ishii’s mother, the PRESENCE document was also dedicated “in heartfelt memory” to Elise O’Hara, a cherished administrative assistant of the Media Lab who collaborated with Tangible Media until her untimely passing in 2023. Her existence — and her absence — are profoundly felt each day, according to Ishii.

He contemplates whether TeleAbsence could one day emerge as a common term “to signify something that was present, but is no longer.”

“When a spot on a bookshelf is designated for a book that should be there,” he remarks, “my students say, ‘oh, that’s a teleabsence.’”

Much like an abrupt silence within a composition, or the vacant white space in a painting, emptiness can carry significant meaning. It’s a notion that we should cultivate more in our lives, suggests Ishii.

“As we are now so occupied, inundated with notification alerts from our smartphones, and perpetually distracted,” he proposes. “Thus, if concepts like emptiness and impermanence, the presence of absence, can be embraced, individuals may start to think a bit more poetically.”