Fraunhofer HHI presents latest VR and AR developments

16 Aug 2017

Research center showcasing photonics systems at broadcasting and electronics fairs IBC and IFA, in September.

New photonics-led technologies from the Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute indicate that there are almost no limits in VR and AR worlds, as it prepares to show them at leading broadcasting and electronics fairs in Europe, next month.

The research organization says that both subjects “are still in the hype phase”, but require further development in relation to production, image capture and streaming. Therefore the HHI has further developed its established technologies before it presents them to interested expert visitors and curious tech fans alike at IBC in Amsterdam, and IFA in Berlin.

OmniCam-360 with closed sphere and live stitching

In the field of high-resolution recording and rendering of 360-degree scenes around a viewpoint suitable for presenting highly realistic impressions, HHI has over recent years further developed and refined its OmniCam-360. It enables, for example, sport or musical events to be rendered in an Ultra High Definition panorama up to 360 degrees around a viewer.

The latest HHI development now makes it possible to provide UHD panoramic content for VR glasses. “Thanks to the real-time solution, the ten single camera segments are smoothly merged to an UHD video signal that can be transmitted to VR glasses such that a spectator experiences a truly immersive experience,” say the developers. “For the use of VR glasses an UHD receiver decodes the HEVC panorama video and the full panorama is down-sampled to UHD and re-encoded.”

At IBC 2017, a new very light version of the OmniCam-360 will be presented. With an 11th camera aimed at the sky, this new camera system is closing the sphere. With the use of the latest Indie Cam generation, the image quality of the whole system is significantly improved. HHI adds that the new generation of OmniCam-360 "is now at a cost level that is comparable to other 360 degree camera systems on the market."

Tile-based DASH streaming

HHI also believes that the resolution of virtual worlds is improvable: “image quality is often still small. But a new technology known as Tile-Based Streaming should change this now,” say the researchers.

“Only high-resolution content can give a complete sense of immersion in VR video applications, but the large amount of data poses a challenge to common transport systems and decoders."

Tile-based DASH streaming for virtual reality with lightweight HEVC Tile Aggregation allows easy adaptation on-the-fly of the 360-degree-video data stream. This technology allows significant reduction of throughput and decoder requirements without requiring complex transcoding.

3D human body reconstruction

Another highlight in Virtual Reality is the 3D Human Body Reconstruction, developed by the Fraunhofer HHI. In this approach, the technology captures a real person’s 3D image with multiple cameras at the same time and creates naturally-moving dynamic 3D models.

Afterwards, these constructions can be integrated in computer graphics, virtual worlds or even real scenes for a variety of purposes. A novel capture and lighting system is under construction that allows 360 degrees volumetric video acquisition and creation of full dynamic 3D Human Body Reconstruction.