NASA sent a doctor to the International Space Station in the world’s first “Holoportation” achievement

But this is not science fiction. When NASA flight surgeon Joseph Schmid was flown to the International Space Station in October last year, that illusion was made possible by Microsoft’s “universal printing” technology, which allows users to interact with 3D representations of remote participants in real time.

“This is [a] An entirely new way of human communication across vast distances,” Schmid says. “It’s an entirely new way of human exploration, where our human being is able to travel far from the planet.”

Schmid and other team members during a holoportation session. (ESA/Thomas Pesquet)

Unlike traditional holographic projections that appear to hover in the air for anyone to see, Holoportation technology requires the use of an augmented reality headset, such as Microsoft’s HoloLens, so that the wearer can perceive (and interact with) the remotely captured person(s), Filmed using a multi-camera setup in their actual location.

In this case, European Space Agency (ESA) astronaut Thomas Pesquet, who was aboard the International Space Station and wearing this headset, had a two-way conversation with Schmid and members of his medical team, along with Fernando de la Pena Laca, CEO of ESA. AEXA Aerospace, which is developing custom Holoportation software (the kind that made this ISS session possible).

Although Microsoft’s Holoportation technology has been around – in various stages of development – for several years, it has never been used for something as ambitious as this before: connecting Earth-centric medical researchers with astronauts on a mission, orbiting the planet hundreds of miles in altitude. the sky .

However, this exact type of ability – bridging physical gaps to connect people across vast distances in space – could be important for future space exploration missions. In this way, scientists can virtually interact with 3D representations in real time of remote participants on Earth, space stations, or other spacecraft, enabling collaboration that can be more engaging and immersive than standard 2D video calls.

“Our physical body is not there, but our human being certainly is,” Schmid says.

“Imagine that you can bring the best trainer or actual designer of a particularly complex technology next to you wherever you are working on it.”

NASA flight surgeon Joseph Schmid flew to the International Space Station. (ESA/Thomas Pesquet)

The next step in the evolution of the technology is to enable fully bidirectional Holoportation interactions.

During this experiment, Pesquet was the only participant to wear an augmented reality headset that enabled him to perceive the other participants as digital holograms, since Schmid and the other participants did not wear such devices themselves.

Once all participants are similarly equipped, the prospects of jumping into someone else’s reality can become even more informative and transformative for astronauts outside the world—whether you’re consulting Earthbound doctors about a medical problem, or sharing important insights about mission goals with NASA researchers.

“What it really plays out is the opportunities for long-duration spaceflight and more deep space flight,” said Christian Mander, research director for space infrastructure company AxiomSpace, An explanation for the Verge in 2021.

“Where you really talk about wanting to create a human connection between your crew – no matter where they travel – and back to someone on this planet.”