Stanford brings depth perception to standard digital cameras

30 Mar 2022

Acoustic modulator coverts image sensor into time-of-flight imaging system.

Potential applications for time-of-flight (ToF) optical sensors continue to proliferate, with the market for 3D imaging and sensing expected to continue its rapid growth.

A project at Stanford University has now developed a form of intensity modulator that could bring ToF capability to conventional CMOS image sensors, potentially allowing many existing imaging platforms to be converted for use in ToF distance-sensing applications.

As reported in Nature Communications, the Stanford team is particularly interested in applying the new technology to lidar platforms, as a route towards more cost-effective and convenient lidar systems in automotive vehicles.

"Existing lidar systems are big and bulky," said Stanford's Okan Atalar. "But someday, if you want lidar capabilities in millions of autonomous drones or in lightweight robotic vehicles, you are going to want them to be very small, very energy efficient, and offering high performance."

According to the Stanford team, the breakthrough could enable megapixel-resolution lidar, a threshold not possible today. Higher resolution would allow lidar to identify targets at greater range, so that autonomous cars might be able to distinguish a cyclist from a pedestrian from farther away. The same technology could also allow any image sensor available today, including the billions in smartphones, to capture 3D images with minimal hardware additions.

Adding 3D imaging to standard sensors can in theory need just the addition of a light source and a modulator that turns the light on and off very quickly, but in practice the second modification is much more difficult than the first.

"Existing modulators can do it, but they require relatively large amounts of power; so large, in fact, that it makes them entirely impractical for everyday use," commented the Stanford team.

The project's solution exploits acoustic resonance, using a thin wafer of lithium niobate coated with two transparent electrodes as a simple acoustic modulator. As a piezoelectric material, lithium niobate can be induced to vibrate at high and controllable frequencies, creating an acoustic wave through the crystal that can rotate the polarization of light.

New type of low cost energy efficient lidar

A polarizing filter placed after the modulator converts this rotation into intensity modulation, effectively turning the light on and off millions of times a second. The project believes that this acoustic approach can be more energy efficient than other conventional modulation approaches.

This work parallels other recent advances in the use of lithium niobate as a modulator material, such as research at the University of Rochester into use of a thin film of lithium niobate bonded on a silicon dioxide layer to create "the smallest lithium niobate modulator yet."

In proof-of-concept trials using a standard camera and a 635 nanometer source, the Stanford device proved able to generate ToF data for metallic targets located around 1 and 2 meters away, with "target distances reconstructed to centimeter-level accuracy" according to the team.

The modulator shows broadband behavior, potentially allowing the modulation of multiple different wavelengths limited only by the transparency window of lithium niobate. A high spatial resolution for depth reconstruction is enabled by the megapixel resolution of the camera.

Since the work reported in its paper, the project has further reduced the energy consumption of the device; a key factor if small-scale lidar using standard image sensors and 3D smartphone cameras is to become a reality.

"This work could become the basis for a new type of compact, low-cost, energy-efficient lidar – 'standard CMOS lidar' – that could find its way into drones, extraterrestrial rovers and other applications," commented the Stanford team.