Camera inspired by avian eye improves multispectral imaging

04 Jun 2024

IBS South Korea develops new type of device that specializes in object detection.

A project at South Korea's Institute for Basic Science (IBS) has developed a perovskite-based camera with a design mimicking the eye of a bird.

Published in Science Robotics, the project's findings point towards new camera units capable of detecting and tracking remote objects in dynamic environments.

Bioinspired vision systems have been heavily researched in recent years, motivated by their novel optics architectures and imaging capabilities. Insect eyes have been a particularly fruitful source, as in the 3D camera mimicing a fly's eye developed by UCLA.

The IBS project looked instead at the eyes of birds, where evolution has developed central foveae or depressions in the avian retina, helping to refract incident light and create a magnified image of a target. In birds requiring sharp vision for hunting prey and detecting predators, these foveae are especially deep and pointed.

Bird eyes also have four cone photoreceptors that respond to ultraviolet (UV) as well as visible RGB light. This multispectral tetrachromatic vision enables birds to acquire abundant visual information and effectively detect target objects in a dynamic environment.

"Despite notable advances in artificial vision systems that mimic animal vision, the exceptional object detection and targeting capabilities of avian eyes via foveated and multispectral imaging remain under-explored," commented the IBS team in its paper.

The project modeled avian eyes by first designing an artificial fovea with a Gaussian profile, and fabricated it from a clear elastomer. This was then mounted on a set of four vertically stacked perovskite layers, each responsive to different wavelengths from UV to RGB through careful bandgap control.

Distant objects spotted quickly and accurately

"We developed a new transfer process to vertically stack the photodetectors," commented Jinhing Park from IBS. "By using a perovskite patterning method developed in our previous research, we were able to fabricate the multispectral image sensor that can detect UV and RGB without additional color filters."

The new design is intended to overcome one key limitation of imaging systems, whereby a central target object may be in focus but its surroundings are not. The IBS camera perceives both a distant object through magnification in the foveal region and nearby objects in the peripheral region, distinguishing UV and RGB light in those areas.

By comparing these two fields of vision, the bird-eye-inspired camera can achieve greater motion detection capabilities than a conventional camera, according to the IBS project. In addition, the new camera is more cost-effective and lightweight, distinguish UV and RGB light inherently without added filters or components.

In simulations the new camera demonstrated a confidence score of 0.76 in object recognition, said to be about twice as high as an existing camera system's confidence score of 0.39. The motion detection rate also increased by 3.6 times compared to the existing camera system, indicating significantly enhanced sensitivity to motion.

"Birds' eyes have evolved to quickly and accurately detect distant objects while in flight," commented Dae-Hyeong Kim of IBS. "Our camera can be used in areas that need to detect objects clearly, such as robots and autonomous vehicles. The camera has great potential for use on drones operating in environments similar to those in which birds live."