Low-cost sensor puts 3D cameras in the picture

23 Nov 2004

CMOS sensor technology is making 3D imaging more affordable and is stimulating a range of new applications such as intelligent car airbags. James Tyrrell talks to two European firms planning to launch products in 2005.

From Opto & Laser Europe December 2004

Three-dimensional cameras that capture quantitative depth data at video rates for around the price of a web-cam could become a familiar sight, thanks to low-cost CMOS image-sensor technology. German-based PMD Technologies (PMDTec) and Centre Suisse d'Electronique et de Microtechnique (CSEM) are both planning to put CMOS-based 3D sensors on the market in 2005.

PMDTec was set up in 2002 on the back of pioneering research led by Rudolf Schwarte at the University of Siegen's Zentrum für Sensorsysteme. It is a subsidiary of Audi Electronics Venture (the electronics innovations arm of car-maker Audi). This gives the company a firm foothold in the automotive sector - an industry keen to exploit the latest innovations in sensor technology. Swiss technology incubator CSEM, which has 14 spin-off and start-up companies to its credit, has been developing its three-dimensional sensor during the past 10 years.

Technology Cameras from PMDTec and CSEM rely on the time-of-flight principle and use a CMOS sensor and pulsed active illumination to generate a 3D image. The systems function by measuring the phase shift between a reference signal sent directly to the detector's smart pixels and light reflected by the scene.

Crucially, each pixel within the array is able to compute the phase difference directly on-board the sensor chip. This built-in functionality means that the sensor can pre-process the signal, removing the need for expensive high-speed electronics. Relating the phase difference measured by each pixel to the speed of light gives the distance travelled by light falling onto the detector. Information from all the pixels is then brought together to create a 3D image. The cameras typically have a range of 7.5 m and use intensity-modulated near-infrared LEDs as an illumination source. "We modulate at around 20 MHz, and this corresponds to a modulation wavelength of 15 m, or 2 x 7.5 m," Nicolas Blanc, head of CSEM's photonics division, told OLE.

Although sufficient over short ranges, the measurement scheme breaks down at distances greater than half of the modulation wavelength because it becomes ambiguous. "This means that if you have objects at 8 m, they will be measured as if they are 0.5 m away from the camera," explained Blanc.

To overcome this problem, the two groups borrowed concepts from techniques such as two-wavelength interferometry. "When we use two modulation frequencies, the ambiguity problem disappears," said Blanc.

"You can also use other modulation schemes such as chirping or pseudo-noise," PMDTec CEO Bernd Buxbaum told OLE. "We have cameras working with pseudo-noise in applications where we have to cover a distance-range of up to 20 m." Although limited in terms of range, the simpler coding techniques have advantages when it comes to illumination. This is because more sophisticated schemes can cause the illumination source to dim if modulation frequency conflicts with the rise-time of the LEDs. Restoring illumination power means using either a larger number of LEDs or faster diodes, such as lasers. "We wanted to build a cheap camera and so we decided to use LEDs," said Buxbaum. Both units use infrared rather than visible light to illuminate the scene. "For most purposes, this is more appropriate because you don't want to disturb the people in a room," said Blanc. "For machine-vision applications you may want to use visible [light]."

Illumination also plays a role in defining camera resolution. Both CSEM and PMD have devices with 120 x 160 pixels. "It's an active measurement system, so if you realize a megapixel camera based on this principle, you need a huge amount of light," explained Buxbaum. "This is because, as the pixels go down in size, so does the signal [photocurrent] amplitude." There is a natural trade-off between the benefits of enhanced resolution and the cost of additional lighting.

PMDTec is thinking about developing a VGA (480 x 640) sensor next year, although, as Buxbaum revealed, it is not an automatic step for the company to take. "What we have learned from many applications and discussions with customers is that they don't need a megapixel 3D [camera]," he explained. "The amount of data coming from a megapixel 3D camera is so high, it is too much." As a low-cost approach for 3D applications requiring only moderate performance and a depth resolution of a few millimetres, CMOS technology appears to have few rivals. Unlike laser scanners, which take time to assemble a full 3D image, CMOS 3D cameras acquire all of their data simultaneously in one shot. It turns out that the CMOS sensors can run at speeds sufficient for video.

"We have to integrate for a certain time to generate enough photons in the device," explained Buxbaum. "[But] if we do that, we can go up to frame rates of 50 Hz, or in some applications up to 200 Hz, without any moving parts in the device."

The CMOS time-of-flight approach has advantages over stereo techniques, which use a pair of cameras and correlation algorithms to extract depth information from a scene. "It only works if you have sufficient structure in the scene. If the walls are white, you won't be able to use passive triangulation at all," said Blanc. "The other point is that a lot of processing power is required to compute the distance. We make a direct phase measurement, which, in that sense, is really metrology."

Applications Both CSEM and PMDTec feel that over the next 2-3 years, new safety legislation will stimulate a strong market for CMOS 3D sensors in the automotive sector. "We [PMDTec] have developed a 64 x 16 chip for automotive applications like pedestrian safety or stop-and-go," revealed Buxbaum. "For the car interior, we have a 64 x 64 chip which will be used to steer airbags more intelligently. "If the driver or passenger is very close to the dashboard, you do not want to inflate the airbag at speed," cautioned Blanc. "Another case is if you have a child seat," added Buxbaum. CSEM is developing automotive products in partnership with Luxembourg-based occupant-sensing expert IEE. "The pressure is coming from car manufacturers," said Blanc. "They really want to have these features in their cars."

People who use lifts may also benefit from the CMOS technology. CSEM is working with Swiss firm CEDES to improve elevator systems. Blanc pointed out that, although today's sensors work well for applications such as door safety, 3D cameras could give much more information. For example, you could make multiple elevator systems more efficient by including data such as the number of people waiting for the lift and its current occupancy.

Marketplace Outside of the automotive sector, PMD is planning to launch its first product, a range sensor, for around €400-500 at the Hannover fair in April 2005. Currently at the prototype phase, Buxbaum is confident that with mass-production technology, its 3D cameras could be priced at around €100.

CSEM's Nicolas Blanc shares a similar vision. "Today, we have [3D camera] demonstrators that we sell for more than €5000, but these are [almost] single pieces, so it's not comparable," said Blanc. "In terms of volume production, it has to be below €100."

CSEM's camera is slightly different to the PMDTec device as it features a combined CCD/CMOS sensor. "Most of the sensor is CMOS for the read-out, the control and most of the data-processing, but within the pixel we use a CCD option. Typically, some of the very early data-processing, especially the demodulation of the incoming signal, is done in the charge domain," said Blanc. "We only start to convert this electrical charge into a voltage later on, when we already have a strong signal. This means that we are significantly less sensitive to noise."

Both cameras interface to a PC and have possible web-cam applications. "If somebody is moving in the background and you only want to focus on the person sitting in front of the monitor, you can do it easily using a 3D," explained Buxbaum. "The solutions that are on the market so far are really expensive. If cheap cameras are available, we believe the market will open up."

The two European groups, which know each other well, think that there is plenty of room in the marketplace. "There are other companies, for example in the US, working on similar technologies," said Buxbaum. "We need to open the market. It doesn't matter if we open it or if the Swiss guys open it - there is enough for all of us."

For more information PMDTec www.pmdtec.com
CSEM www.csem.ch