Penn State metasurface allows real-time multi-dimensional images

10 Sep 2024

Combination of metasurface design and machine learning reveals polarization data more easily.

A project at Pennsylvania State University (Penn State) has developed a metasurface intended to extract polarimetric imaging data more rapidly and cost effectively than current methods.

Described in Science Advances, the new component should allow mutli-dimensional imaging to be carried out easily within a single camera system.

The Penn State team took inspiration from the natural world, where naturally evolved vision system have provided animals with sensitivity to several characteristic parameters of light that human eyes detect poorly or not at all, including polarization.

"As the animal kingdom shows us, the aspects of light beyond what we can see with our eyes holds information that we can use in a variety of applications," said Xingjie Ni from Penn State's NanoLight Nanophotonics and Optoelectronics Laboratory.

"To do this, we effectively transformed a conventional camera into a compact, lightweight hyperspectro-polarimetric camera by integrating our metasurface within it."

The NanoLight lab has already achieved a number of metasurface breakthroughs, including the 2023 development of a metalens 80 millimeters in size and suitable for use in large optical systems and telescopes, leveraging advances in deep UV photolithography.

Its new development, by contrast, was intended to be small enough to be attached to conventional camera systems, enhancing their performance and making it possible to acquire hyperspectro-polarimetric images from a single snapshot.

Imaging for medical and consumer applications

"Our metasurface features thousands of spectro-polarimetric superpixels, each comprising arrays of judiciously designed chiral meta-atoms," noted the team in its paper. "The metasurface was placed in conjunction to the detector chip of a standard monochromatic camera, enabling it to image objects with both wavelength and polarization data."

The intensity information captured by the camera is subsequently decoded by a machine-learning backend previously trained on 1.8 million images, to produce four-dimensional (4D) image data of the input.

Penn State designed its surface so that each superpixel has both distinct spectral and polarization responses, by incorporating meta-atoms exhibiting strong chirality and enabling different polarization responses for all polarization states. This compares with the use of narrowband filters or gratings in a conventional spectrometer, where the wavelength information is mingled by those filters and then must be resolved in the post-processing.

"At 28 frames per second, primarily limited by the speed of the camera we used, we are able to rapidly recover both spectral and polarization information using our neural network," said Penn State's Bofeng Liu. "This allows us to capture and view the image data in real-time."

Researchers tested their metasurface and neural network by video-recording transparent "PSU" letters under different laser beam illuminations. They also captured images of the scarab beetle, known for reflecting circularly polarized light visible to others of its species.

Penn State envisages its technology being employed across a range of applications. Hyperspectro-polarimetric data could be used by clinicians to differentiate the material and structural properties of tissues within the body, potentially aiding in the diagnosis of cancerous cells. Alternatively the same ability could be of value in consumer sectors.

"We could bring our camera along to the grocery store, take snapshots and assess the freshness of the fruit and vegetables on the shelves before buying," commented Xingjie Ni. "This augmented camera opens a window into the unseen world."