Chiba researchers simplify generation of 3D holographic displays...

19 Oct 2023

...while a team at Tohoku manipulates the behavior of light “as if it were under the influence of gravity”.

Holograms have long held the promise of offering immersive 3D experiences, but the challenges involved in generating them have limited their widespread use. Leveraging recent developments in deep learning, researchers from Chiba University, Japan, have now developed what they describe as a “game-changing” approach that utilizes neural networks to transform 2D color images into 3D holograms.

This approach can simplify 3D hologram generation and can find applications in numerous fields, including healthcare and entertainment. Holograms that offer a 3D view of objects provide a level of detail that is unattainable by regular 2D images.

Holograms, which offer enormous potential for medical imaging, manufacturing, and virtual reality, are traditionally constructed by recording the 3Ddata of an object and the interactions of light with the object. However, this technique is computationally highly intensive as it requires the use of a special camera to capture the 3D images. This makes the generation of holograms challenging and limits their widespread use.

Deep-learning methods have also been proposed for generating holograms. These can create holograms directly from the 3D data captured using RGB-D cameras that capture both color and depth information of an object. This approach circumvents many computational challenges associated with the conventional method and represents an easier approach for generating holograms.

The Chiba researchers led by Professor Tomoyoshi Shimobaba of the Graduate School of Engineering, propose a novel approach based on deep learning that further streamlines hologram generation by producing 3D images directly from regular 2D color images captured using ordinary cameras. Yoshiyuki Ishii and Tomoyoshi Ito of the Graduate School of Engineering, Chiba University were also a part of this study, published in Optics and Lasers in Engineering.

Prof. Shimobaba commented, “There are several problems in realizing holographic displays, including the acquisition of 3D data, the computational cost of holograms, and the transformation of hologram images to match the characteristics of a holographic display device. We undertook this study because we believe that deep learning has developed rapidly in recent years and has the potential to solve these problems.”

Three neural networks

The Chiba approach employs three deep neural networks (DNNs) to transform a regular 2D color image into data that can be used to display a 3D scene or object as a hologram. The first DNN makes use of a color image captured using a regular camera as the input and then predicts the associated depth map, providing information about the 3D structure of the image.

Both the original RGB image and the depth map created by the first DNN are then utilized by the second DNN to generate a hologram. Finally, the third DNN refines the hologram generated by the second DNN, making it suitable for display on different devices. The researchers found that the time taken by the proposed approach to process data and generate a hologram was superior to that of a state-of-the-art graphics processing unit.

Prof. Shimobaba added, “Another noteworthy benefit of our approach is that the reproduced image of the final hologram can represent a natural 3D reproduced image. Moreover, since depth information is not used during hologram generation, this approach is inexpensive and does not require 3D imaging devices such as RGB-D cameras after training.”

In the near future, this approach can find potential applications in heads-up and head- mounted displays for generating high-fidelity 3D displays. Likewise, it can revolutionize the generation of an in-vehicle holographic head-up display, which may be able to present the necessary information on people, roads, and signs to passengers in 3D.

Photonic crystals bend light ‘like gravity’

A collaborative group of researchers at Tohoku University, Japan, has manipulated the behavior of light as if it were under the influence of gravity. The findings, which were published in Physical Review A, have significant implications for the world of optics and materials science, for example in the development of “6G” communications.

Einstein’s theory of relativity has long established that the trajectory of electromagnetic waves – including light and terahertz electromagnetic waves – can be deflected by gravitational fields. Scientists have recently theoretically predicted that replicating the effects of gravity - i.e., pseudogravity - is possible by deforming crystals in the lower normalized energy (or frequency) region.

“We set out to explore whether lattice distortion in photonic crystals can produce pseudogravity effects,” said Professor Kyoko Kitamura from Tohoku University’s Graduate School of Engineering.

Photonic crystals possess certain properties that enable scientists to manipulate and control the behavior of light, serving as traffic controllers for light within crystals. They are constructed by periodically arranging two or more different materials with varying abilities to interact with and slow down light in a regular, repeating pattern. Furthermore, pseudogravity effects due to adiabatic changes have been observed in photonic crystals.

Kitamura and her colleagues modified photonic crystals by introducing lattice distortion: gradual deformation of the regular spacing of elements, which disrupted the grid-like pattern of protonic crystals. This manipulated the photonic band structure of the crystals, resulting in a curved beam trajectory in-medium. Specifically, they employed a silicon distorted photonic crystal with a primal lattice constant of 200 micrometers and terahertz waves. Experiments successfully demonstrated the deflection of these waves.

“Much like gravity bends the trajectory of objects, we came up with a means to bend light within certain materials,” said Kitamura. Associate Professor Masayuki Fujita from Osaka University added, “Such in-plane beam steering within the terahertz range could be harnessed in 6G communication. Academically, the findings show that photonic crystals could harness gravitational effects, opening new pathways within the field of graviton physics.”