A new system makes 'teleportation' possible’ in remote meetings

The combination of robotics, video and numerous sensors and visualization technologies of the 'Beaming' project’ allows a series of people physically separated by thousands of kilometers to have the sensation of sharing the same virtual space in which they can interact with each other.

A team of European scientists has developed, within the framework of the 'Beaming' project, a teleconferencing system that exponentially increases the realism of these meetings. Using technologies belonging to immersive virtual reality, Researchers have managed to simulate the feeling of being in another place. Thus, Noisy sound connections or blurry images typical of teleconferences are a thing of the past, according to the Trends website 21.

The project is being developed by a consortium of eleven R&D and neuroscience technology companies and groups from 7 European countries and coordinated by the Spanish entity Starlab: University of Barcelona (Spain), Consortium Institute of Biomedic Research August Pi and Sunyer (Spain), University College London (Great Britain), Swiss Federal Institute of Technology Zurich (Swiss), Sant'Anna Higher School of University Studies and Specialization (Italia), Technion – Israel Institute of Technology (Israel), Interdisciplinary Center Herzliya (Israel), IBM Haifa Research Lab (Israel), Aalborg University (Denmark) y Technical University of Munich (Germany).

“When we meet people in the flesh it is possible to extract subtle information from facial expressions, quirks or who watches who” explained Stephen Dunne of Starlab. “There is a huge amount of non-verbal communication that is lost even with the highest quality video conferencing technology. It is not possible to shake hands or decide to explore the room, For example", continue.

In reality it is not a teleportation in the strict sense., but the project Beaming ('Teleportation with augmented media applied to natural network meetings') uses techniques and technologies belonging to immersive virtual reality to simulate the feeling of being in another place. On the other side of the line, the user is represented by an avatar - a life-size robot, For example- that does the work of the eyes, ears and mouth. The teleporting “traveler” has numerous sensors and its head is covered by an immersive screen. This projector allows you to see and hear everything that the robotic avatar sees through its video camera, listen through their microphones and also respond as if you were in their place.

A movement of the head also moves the robot's head and the voice is reproduced identically. There are motion sensors located on the arms that detect movements so that the robot can repeat them.. If someone touches the robot's hands, pressure is transmitted to a pressure pad placed on the lecturer's hand.

The technology has been tested in practice in a pioneering interview between a project scientist and a BBC journalist based respectively in Spain and at University College London. (United Kingdom). The journalist managed to interview the scientist and even “high-five” him at the end of the exhibition.

Today's technology creates the market of the future

“What we have achieved is a real demonstration of what is possible with the technologies available today”, added Dunne. “We use commercially available products and combine them in innovative ways. This is not a pioneering technological development, "But it is innovative in that it demonstrates the great capacity offered by the coordination of different communication channels to achieve exciting results.".

One of the largest tasks for the Beaming partners was the development of a data architecture framework for the system. It defines the compression and transmission of visual data, sound, of movement and pressure between the “traveler” and his remote environment. It also establishes the way in which the remote three-dimensional model must be generated so that the “traveler” can transmit a sensation of intense presence.. “The goal of the framework is to make Beaming completely independent of the hardware or software used”, explained Mr.. Dunne. “Any type of robot or sensor can be used, For example. We have also set out to define the minimum amount of data necessary so that the Beaming experience remains completely immersive.”.

emotional information

Although head devices, pressure pads and three-dimensional graphical interfaces are capable of generating an intense feeling of being in another location, It is also necessary to know what they feel on the other side of the line., For example, those sitting around a table who have to interact with the avatar. Talking to a robot with a human voice can be disconcerting.

“We gave importance to making the interaction as natural as possible. Regardless of whether you are interacting with a robot or another type of virtual avatar, we wanted to provide a natural experience that did not get in the way of normal communication.”, Dunne explained.

A simple improvement was to give the robot more expressive facial features. The team replaced the robot's mechanical eyes with an LCD screen. “We use graphics for the eyes and mouth because they look much more natural.”. The feeling of tranquility that a robot transmits is greater if you look into its eyes and they are normal. It's easier to ignore that you're dealing with a machine.", Dunne clarified.

Research is still underway dedicated to studying ways to record and transmit data about the physiological and emotional state between communication participants.. For this purpose, experiments were carried out to find out to what extent the heart rate, the facial expressions and even the brain waves of the communicator are capable of transmitting information about their emotional state. Current technology can be programmed to take advantage of this information and recognize basic emotional states such as the difference between stress and calm.. The challenge now is to communicate the signals to all those involved through the avatar.

Will travel be relegated to the past?

“We are not saying that Beaming will radically change meetings or that it will save millions of tons of carbon dioxide because travel will no longer be necessary,” Dunne said., but we believe that the idea of “teleport” someone to another location where they can move and interact using a robot could be useful initially in some very specific applications”.

“It is ideal for transferring a person with specific skills or knowledge to another place”, Dunne continued. “Allows total physical immersion in the destination, feel the environment and interact with it. We consider that it could be effective for front-line surgeons to “the teletransport” to operating rooms around the world and share their skills and knowledge and even carry out operations”. “As a project, our goal is to demonstrate that this type of virtual travel and interaction is possible. Now it is up to each partner to convert the results into real applications and products. “We move away from the field of scientific curiosity to study possible services”, concludes the researcher.