VRTogether showcases world’s first live 3D video conference, using point clouds, over a commercial 5G network

Last week, attendees at the VRDays Europe conference witnessed VRTogether’s demonstration of the worlds’ first live 3D video call over a commercial 5G network, which leveraged point cloud technology using a standard smartphone. This demonstration corroborates the benefits of 3D video for many sectors and applications, proving how a doctor can better assess trauma through a live 3D view while using commercially available equipment and services.

The Horizon 2020 project VRTogether, led by the i2CAT Foundation, participated in the 6^th edition of VRDays Europe to showcase the potential of VRTogether’s social Virtual Reality platform. On 4^th November, project partners CWI and Sound held a live demo at De Kromhouthal in Amsterdam, consisting of a volumetric video conferencing (based on point clouds) between a doctor, in a medical examination room, and an acting injured patient, in the street. At the VRDays stage, the doctor was captured using Azure Kinect cameras. The patient, who was outside near Science Park in Amsterdam, was captured with a Samsung S20 Ultra 5G smartphone, and streamed in real-time over the standard 5G network of Dutch telecommunications provider KPN. The whole experience was captured by a virtual camera, projected on a big screen at the VRDays stage, and broadcasted live to an international audience.

The VRTogether platform was used for orchestrating the experience, with an end-to-end pipeline for the delivery of volumetric video as point clouds. This platform provides a real-time experience, allows for optimization mechanisms based on the context of interaction and human behavior and it is extensible, so the technical components can be customized depending on the needs of the interaction. The platform uses an award-winning compression algorithm for point clouds developed by the Distributed and Interactive Systems group at CWI.

During VRDays Europe, which took place from 4th to 6th November, the VRTogether project partners hosted a booth within the Virtual Trade Show, an exhibition held in an immersive virtual environment. Moreover, the project was selected to participate in the Immersive Funding Market, an online matchmaking event bridging the gap between academia and industry. Finally, VRTogether’s Innovation Manager, Patrick De Lange, presented the project’s innovative features and applications at ‘The Future of Work’ conference.