Arizona sharpens imaging for eye-tracking

01 Apr 2025

Deflectometry technique promises a wide range of applications.

From Ford Burkhart in Tucson

An optics lab at the University of Arizona (UA) is using new 3D imaging techniques based on deflectometry to create innovative eye-tracking technology, with apps ranging from entertainment headsets to industrial engineering.

Deflectometry is a 3D-imaging technique that allows for the measurement of reflective surfaces with very high accuracy. Current applications include scanning large telescope mirrors or other high-performance optics for the slightest imperfections or deviations from prescribed shapes.

Florian Willomitzer, associate professor at UA’s Computational 3D Imaging and Measurement Lab, said, “With our deflectometry-based method, we can use the information from more than 40,000 surface points, theoretically even millions, all extracted from only one single, instantaneous camera image.”

“I’m very excited about this stuff and how it will work,” said Willomitzer, who received his Ph.D. at Friedrich-Alexander University Erlangen-Nuremberg, Germany, at the Institute of Optics, Information and Photonics. The technology is moving toward commercialization at Tech Launch Arizona, a UA division that guides research products into market.

How it works

The 3D imaging technology uses advanced computation to follow gaze direction information from tens of thousands of surface points on a person’s eye. That’s a big step up from the dozen or so points seen in current eye-tracking methods, UA stated.

Using deflectometry for applications outside the 3D industrial inspection of specular surfaces, such as lenses or astronomical mirrors, is a research focus of Willomitzer’s group. The team pairs deflectometry with advanced algorithms typically used in computer vision research to achieve an accurate estimate of the eye's gaze direction

Their research, published today (1 April, 2025) in Nature Communications, has found that integrating the powerful 3D imaging technique known as deflectometry with advanced computation has the potential to significantly improve state-of-the-art eye-tracking technology

The approach can improve eye-movement tracking accuracy in virtual and augmented reality headsets, automotive driving assistance, and industrial engineering, said the team, based at Arizona’s Wyant College of Optical Sciences

It could also be used in analysis of paintings and other artwork, or in tablet-based 3D imaging to measure the shape of skin lesions. Current eye-tracking methods, the team said, can only capture directional information of the eyeball from a few sparse surface points

“More data points provide more information that can be potentially used to significantly increase the accuracy of the gaze direction estimation,” said Jiazhang Wang, a postdoc researcher in the lab. “This is critical, for instance, to enable next-generation applications in virtual reality. We have shown that our method can easily increase the number of acquired data points by a factor of more than 3,000, compared to conventional approaches.”

‘Seeing the unseen’

“The unique combination of precise measurement techniques and advanced computation allows machines to see the unseen, giving them superhuman vision beyond the limits of what humans can perceive,” Willomitzer said.

Instead of depending on a few infrared point light sources to acquire information from eye surface reflections, the new method uses a screen displaying known structured light patterns as the illumination source. Each of the more than 1 million pixels on the screen can act as an individual point light source.

By analyzing the deformation of the displayed patterns as they reflect off the eye surface, the researchers can obtain accurate and dense 3D surface data from both the cornea, which overlays the pupil, and the white area around the pupil, known as the sclera.

In a previous study, the team explored how the technology could seamlessly integrate with virtual reality and augmented reality systems by potentially using a fixed embedded pattern in the headset frame or the visual content of the headset itself -– still images or video –- as the pattern that is reflected from the eye surface.

That could significantly reduce system complexity, the researchers said. “To obtain as much direction information as possible from the eye cornea and sclera without any ambiguities, we use stereo-deflectometry paired with novel surface optimization algorithms,” Wang said.

The team said they had demonstrated accuracy comparable to commercial eye-tracking systems in real human eye experiments. “Our goal is to close in on the 0.1-degree accuracy levels obtained with the model eye experiments,” Willomitzer said. He said the team hopes their eye tracking technology will find apps in neuroscience research and psychology.

He added: “This work is a great example of how domain-specific expertise — in this case, precise 3D measurement of specular surfaces — can be leveraged to solve challenges in a completely different application area. This kind of 'out-of-the-box' thinking highlights the versatility of optical technologies, which we hope will benefit the optics community as a whole.”