Fraunhofer smartphone app looks inside objects

14 Feb 2017

Prism-free HawkSpex system can verify constituents of fruit, consumer products and more for quality control.

A new smartphone-based inspection app developed at the Fraunhofer Institute for Factory Operation and Automation (IFF) in Magdeburg, Germany, can look inside objects and display particular constituents. It has numerous uses, say the developers, for instance, fruit can be scanned for pesticide residues.

The so-called HawkSpex will also enable consumers to quickly discover whether a car has been in an accident and resprayed. The IFF says that the mobile app could significantly enhance user’s knowledge of previously unverifiable information.

Although phone-based systems that perform such scans already exist, users usually have to clamp additional parts such as a prism onto the front of the integrated camera. This is costly and impractical and can interfere with a smartphone’s design. Prof. Udo Seiffert, Expert Group Manager at the Fraunhofer IFF commented, “What makes our app special is that users don’t need anything for a scan other than the camera already integrated in their smartphones.”

No hyperspectral camera needed

So how did the development engineers manage to get by without a prism? Project manager Dr. Andreas Herzog explained, “Such scanning techniques usually require a special hyperspectral camera: it adjusts to different colored light with each scan and ascertains how much of a particular wavelength is reflected by an object, thus generating a complete spectral fingerprint of the object.”

The IFF development engineers instead are using a mathematical model to extract a variety of data points from an object, such as its constituent elements / molecules, from the spectral fingerprint. “Since hyperspectral cameras aren’t integrated in smartphones, we simply reversed this analytical principle,” said Prof. Seiffert.

“The smartphone camera provides a broadband three-channel sensor, which illuminates an object with different wavelength and scans every wavelength and.” This means that, instead of the camera measuring luminous intensity in different colors, the display successively illuminates the object with a series of different colors for fractions of a second.

Thus, if the display casts only red light on the object, the object can only reflect red light – and the camera can only measure red light. Intelligent analysis algorithms enable the app to compensate for a smartphone’s limited computing performance as well as the typically limited performance of the camera and display.

The first laboratory version of the app is now finished, IFF reported earlier this month. The engineers are now developing a variety of initial applications before it can be released to private users. The system first has to be installed with reference scans to be able to analyze, for example, whether apples contain pesticides. Seiffert said he hopes that the HawkSpex app will be launched on the market around the end of 2017.

Reference scans are not always needed, though. Some problems only require measuring different distributions of substances or materials rather than specifying individual constituents. Purchasing a car is one example: In this case, the app compares paint to determine whether it is exactly the same color everywhere or has been touched up.

Seiffert added, “There are so many conceivable uses that the market will surely overrun us. That is why we will be relying on a user-generated approach modeled on the approach employed by Wikipedia.”

He believes that the app has “extremely interesting commercial potential” and could be used to develop sectors that wouldn’t really benefit from high-precision scanners: quality control of foods, effectiveness of cosmetic products or even agriculture, for example. “Farmers, for instance, could easily obtain information on whether their crops are sufficiently supplied with nutrients or fertilizer is needed,” Seiffert concluded.