Evaluating the state of play for AI and optical design at SPIE Optics + Photonics

An exploration of some of the questions surrounding artificial intelligence (AI) and its impact on optical design was on tap at SPIE Optics + Photonics 23-27 August in San Diego, California. For example, University of Laval Full Professor Simone Thibault looked at the utility and limitations of using ray tracing for generating synthetic data to train AI models across various applications. He presented several use cases where ray tracing serves as a critical tool for training datasets when experimental data is scarce.

Whether designing an optical system for snow density detection or using scattering profiles to simulate radiation dose distributions in radiotherapy, Thibault said, “what all these have in common [is that] they need data, and most of the time it's difficult to get those data.”

Ray tracing, he said, can simulate light scattering in snow-like environments to train AI for density estimation. In biomedical imaging, ray tracing could be used to generate synthetic patterns to train AI for lens-less imaging for red blood cell identification, or for digital holographic microscopy to assist, for example, in research on diseases like Parkinson’s Disease. Ray tracing might also be used to train AI for testing of optical components by creating diverse datasets simulating manufacturing tolerances and misalignments.

But there is a reality gap when transitioning from synthetic to real-world data, Thibault warned. Neural networks trained on perfect synthetic images, he said, often fail when applied to real-world data due to a lack of diversity. To improve performance, he suggested introducing random imperfections like detector noise, sensor responsiveness, and manufacturing errors into simulations. Future considerations, he said, might include shifting the paradigm from using optics to generate images (to train AI) to using images to define optical specifications.

In a related talk, Jacob A. Sacks, a PhD student at the University of Rochester, posed two questions: “Is AI ready for lens design? And equally as important, is lens design ready for AI?” His talk, he said, would set forth “a framework for thinking about whether a particular task or part of lens design can be automated easily with AI, or whether it's going to be a little bit more difficult.”

More specifically, Sacks said, the framework he proposes would evaluate the integration of agentic large language models (LLMs), which use tools and memory to operate in iterative loops for certain tasks, effectively removing humans from the loop. He suggested these could be deployed for certain optical lens design tasks based on task verifiability. The verifier's rule, he explained, is an idea that was popularized by OpenAI researcher Jason Wei. Its basic thesis is that the feasibility of training AI to solve a task is proportional to how verifiable the task is. The verifier’s rule rates optical design tasks based on five criteria: objective truth, speed of verification, scalability, reliability (low noise), and continuity.

In Sacks’ evaluation, the two highest-scoring optical design tasks (out of four) were writing macros/programming (transforming an input into an output using code) and local optimization. That is, maximizing the performance of a current design. Sacks rated them 4.5 out of 5 and 4 out of 5, respectively. Automating local optimization, he said, is limited by software capability.

Sacks rated the task of global optimization — finding potential new designs via agentic LLMs — as 2 out of 5 mainly because finding potential new designs is difficult to verify quickly or at scale. Optical engineering as a task garnered a 1 out of 5 rating mainly because solving general problems with light is not easily verifiable or objective.

For his own research on zoom lens design, Sacks used a Python-based Monte Carlo search tool to find paraxial layouts for zoom lenses. He used Claude Code to prompt the system for a full-frame visible zoom lens for mirrorless cameras. Using the verifier’s rule, he determined that AI successfully identified diverse design forms, including positive and negative front lens groups. He found that while AI can optimize specific metrics like vignetting and size, it cannot yet determine relevant metrics or ensure a search is truly exhaustive without human intervention.

“Is AI ready for lens design?” Sacks asked. “I would say that it is, and we have a lot of very powerful AI tools at our disposal. But is lens design ready for AI? I would say not quite yet, and things that we need to work on to make this happen are making more lens design tasks highly verifiable.”