 |
29 Jun 2011
The start-up has grabbed the attention of world's media, but faces competition in the race to commercialize a plenoptic camera.
Rarely does a purely optical innovation grab the attention of the world’s media in the way that Lytro’s did last week. The Stanford start-up’s combination of plenoptic technology and slick image processing software will allow refocusing of images after they have been taken – however poorly focused the original image - is a prospect that has got many amateur photographers salivating, even if some of the professionals remain skeptical.
Lytro is the brainchild of Ren Ng, a Stanford University PhD who, along with investment partner Ben Horowitz - co-founder of “TiVo” and now of the venture capital firm Andreesen Horowitz - sparked the media frenzy with a pair of blog posts unveiling the company and its plans on June 21. Within hours, hundreds of major news outlets had covered Lytro and its magic camera.
But neither the technology nor the company is new. Originally, Ng had called his start-up “Refocus Imaging” – naturally enough, given that this is what the technology actually does. According to Refocus’ only SEC filing, in August 2008 Greylock Partners provided $250,000 of seed funding, but the company has now raised close to $50 million, with Andreesen Horowitz leading Lytro’s latest $37.6 million series C round of funding.
The company’s technology has also been demonstrated in a high-profile manner before. The clunkily stage-managed Youtube video below shows Ng appearing at the Intel Developer Forum back in 2008, while still under the “Refocus” guise, for example.
So how does what Horowitz calls the “magic camera” work? Via the Lytro web site, Ng points us to his award-winning 2006 thesis, which over the past week may well have become the most downloaded PhD thesis of all time.
Lens/microlens combination
The optical premise, which revolves around capturing the entire light field of a scene (rather than just the conventional two-dimensional image) through a combination of conventional optics and a microlens array, is actually nothing new. In a 2005 Stanford Tech Report, Ng refers to plenoptic methods pioneered more than a century ago (the Nobel laureate Gabriel Lippmann was the first patent the idea, in 1908), and particularly to the optical design of the plenoptic camera by Adelson and Wang in the 1990s.
Ng also describes the similarity between the Lytro design and the “insect eye” sensors that rely on microlens arrays, and which have been tipped to find widespread use in future mass-produced cameras. But Ng and his Stanford colleagues actually went one step further, combining fundamental aspects of both human and insect eyes. “Our optical design can be thought of as taking a human eye (camera) and replacing its retina with an insect eye (microlens/photosensor array),” he wrote in the Stanford Tech Report, before adding: “No animal has been discovered that possesses such a hybrid eye.”
At the time of that report, Ng was still working on his PhD, but clearly saw the commercial potential in the work, writing: “Playing with the focus may provide digital photo albums with new entertainment value in that different subjects of interest in each photo can be discovered as they “pop” when the correct focal depth is found.”
Resolution trade-off
Impressive as it is, the technology is not without its drawbacks. What the plenoptic design does is to use the pixels of a photosensor array in a different way to a normal camera – to capture an image at all focal lengths, effectively to infinity. The price of that is paid in the two-dimensional resolution, or “sharpness” of the image, and the photographs provided by Lytro to show off its refocusing “magic” certainly appear to lack the sharpness of conventionally focused high-resolution digital image (to try out refocusing an image from Lytro, click any area of the photo below).
According to Lennart Wietzke from the Germany company Raytrix, which has already commercialized a number of plenoptic cameras capable of the same “trick” that Lytro is suggesting (although not yet at a consumer-friendly price), this trade-off is unavoidable: “The decrease of resolution will be always a disadvantage of light field cameras, since pixels are stored on the image sensor from different points of view,” he explained. In a Raytrix camera, it means that an 8 megapixel sensor actually produces 2 megapixel refocusable images.
The march of Moore’s law will come to Lytro’s assistance here – if you had been wondering why anybody might reasonably want a 100 megapixel camera, this application could provide the answer. Indeed, the software company Adobe has been working on a 100 megapixel plenoptic camera that takes 19 images of a scene to produce a three-dimensional digital photo with the kind of sharpness that amateur photographers have come to expect.
Intellectual property
Aside from battling the resolution problem, Lytro will surely find its intellectual property subject to scrutiny. Although the acres of media coverage might suggest that it will be the first company to commercialize the image refocusing technology, Raytrix has already got there – albeit for primarily industrial and scientific imaging applications. “Raytrix intellectual property consists of a microlens technology with different types of lenses to significantly increase the depth of field when refocusing the image after the picture has been taken. We own this property and have not licensed it yet,” the Kiel-based company told optics.org.
Meanwhile, Ng and Ben Horowitz are cited as co-inventors on two granted US patents describing the Lytro technology and assigned to Stanford University. One of those patents (7,723, 662) describes how the approach can be beneficial for microscopy applications, while a second – awarded just six weeks before Lytro’s coming-out party – describes a more generic set of applications including video. Mitsubishi Electric researchers have also been awarded a patent protecting a 4D light-field camera design.
But the main challenge facing Lytro will be to produce a light-field camera at a price that consumers will find acceptable. Raytrix quotes a basic camera price of just under €3000 for its “R5” plentopic camera, although various hardware and software accessories, including the refocusing software, push that figure to just over €10,000. The advanced “R11” version of the camera, which incorporates a CCD rather than CMOS image sensor, costs nearly three times as much.
Optical secret
The “secret” of the technology used by Raytrix is a microlens array featuring different types of microlenses. “The R5 camera uses four different microlens types with individual focal lengths,” said the company. That yields not only software-enabled refocusing with a six-fold extension of the depth of field possible with a regular camera, but also a 3D depth estimation of a scene and 3D multi-views. And by simply changing the main lens, users can switch between macroscopic and microscopic applications.
The German company told optics.org that its light-field technology could be used in many different settings, including the consumer market – the key question will be whether or not Raytrix or Lytro can produce the technology cheaply enough in a hand-held format. If they can – and Lytro says that it will launch just such a camera by the end of this year – then the hype may well be justified. It could be the start of the latest revolution in photography, thanks largely to optical innovation and the mass production of microlens arrays and sensors.
About the Author
Mike Hatcher is the Editor in Chief of optics.org
 |  |  |  |  |  |  |
| © 2024 SPIE Europe |
|
