SPIE Optics + Photonics Digital Forum: Omnidirectional stereo camera for autonomous driving

26 Aug 2020

Hitachi group develops more economical, compact lidar system for the growing ADAS technology market.

Compact and low-cost devices are needed for autonomous driving to image and measure distances to objects 360-degree around. Researchers at Japanese technology giant Hitachi have developed an omnidirectional stereo camera based on two hyperbolic mirrors and a single set of a lens and sensor, which makes the camera more cost efficient than many conventional ADAS optical systems.

At the ongoing SPIE Optics + Photonics Digital Forum, Ryota Kawamata described his group’s development of a compact omnidirectional stereo camera for autonomous driving applications. In particular, the Hitachi group has devised a new calibration method for its economical camera.

Kawamata introduced his presentation by saying, “Based on the original calibration method by Mei and Rives, we have improved calibration accuracy by considering higher order radial distortion, detailed tangential distortion, an image sensor tilt, and a lens-mirror offset.”

This method is applied to Hitachi’s prototype omnidirectional stereo camera. Its beneficial features are that it reduces the root mean square of the calibration accuracy by up to 2.2 times. “For ADAS applications and more, distance error is less than 8% up to objects 14 meters apart, which is improved more than seven times compared to the original method, although the error is still larger than the target value of 5%,” he said.

The group is still working on improving the performance of the device and method: “We consider that the remaining calibration error is due to distortion of the glass cylinder and a degraded optical resolution. As future work, we plan to make further improvements in the calibration and optical resolution of the prototype. In addition, a rectification method for cylindrically expanded images needs to be developed,” he added.

Kawamata also explained the likely market need for cheaper lidar-based sensing systems that offer a wider field of view – indeed a 360-degree view around the vehicle. “We expect that demand for ADAS and AD systems will increase by about 400 per cent between 2018 to 2030. Within that growth, the ratio of high-level systems featuring an omnidirectional FOV will also grow significantly.”

“In order to realize an omnidirectional view at a low cost, you would usually need to use a combination of lidars to measure the distances to objects by calculating the time to receive reflected light. For example, Waymo uses five of them to cover 360 degrees around its system. In that arrangement, the distance error of 0.1 per cent is very low but the cost is high, at around $2000 per system.”

Parallax view

“We are using stereo cameras that measure the distance using the parallax [angle difference] between the two cameras. Our solution is to use four wide-FOV stereo cameras to create the omnidirectional view. This time, the distance error of 5% is low enough but the system is more affordable at around $500.”

The Hitachi group’s improved results include the RMS being improved from 1.4 pixel to 0.65 pixels, Kawamata explained. “We have also conducted a demonstration, in which we installed a camera in a golf cart to collect images and data. We also evaluated our experimental error, which we found to be 8% for objects 14m apart, against the theoretical predicted error of 5%.

“To adjust our model, we need to remove the remaining distortion due to variable thickness of the camera’s glass cylinder, improve the image resolution and develop and accurate rectification method.”

The online Optics + Photonics Digital Forum, which runs through Friday, 28 August, after which presentations will continue to be accessible via the website, includes over 1,200 on-demand technical presentations, 300 posters, and 900 manuscripts across three tracks: Nanoscience and Engineering Applications, Organic Photonics and Electronics, and Optical Engineering and Applications.