LensDesigner: A Self-Improving Agent for Optical Lens Design

1 Introduction

• The authors argue that LLM agents are promising for experience-dependent optical design because they can use memory and skill modules to learn and self-evolve, while their use in this domain remains unexplored.The introduction describes lens design as non-convex and high-dimensional, with results sensitive to an expert-proposed initial structure.Source: §1 · pp. 1–2
• LensDesigner uses LensLib100K and Optics-Aware Retrieval to supply each new design requirement with a physically valid initial seed.The library is based on an enhanced implementation of QGSO and spans common lens specifications.Source: §1 · p. 2
• In an interactive optical-design environment, the agent makes macroscopic changes such as inserting elements or altering glass materials and receives immediate simulator feedback.Feedback concerns the optical merit function and structural validity.Source: §1 · p. 2
• A curriculum agent generates increasingly difficult valid design tasks, while LensDesigner reflects on trajectories and stores extracted heuristics for reuse.Traditional local optimization then performs fine-grained parameter tuning toward a local optimum.Source: §1 · p. 2
• On LensArena’s 120 diverse tasks, LensDesigner generates valid structures consistently, achieves the highest success rate, and yields the smallest RMS spot radius among existing methods.The benchmark includes common configurations and extreme systems, including ultra-wide-angle lenses with FoV up to 180° and large-aperture lenses with F-number as small as 0.7.Source: §1 · p. 2
• The paper proposes an autonomous agent framework that mirrors expert intuition and introduces a continuous evolution mechanism to accumulate and reuse optical design heuristics.Its contributions also include LensLib100K and LensArena.Source: §1 · p. 3

2 Related Work

• Existing automation includes global optimization algorithms [11, 19], neural networks, and retrieval methods such as LensNet, but may require manual tuning, long convergence, or fixed database distributions.Traditional design relies on expert intuition and tools such as Zemax, which the passage characterizes as numerical calculators rather than autonomous decisionmakers.Source: §2 · p. 3
• Although LLM agents have been applied in chemistry [4], robotics, and software engineering, autonomous agents remain largely unexplored for optical design.The paper identifies optical design as an empirical and numerically sensitive domain.Source: §2 · p. 3

3.1 Lens Design Preliminaries

• The paper focuses on classical imaging with spherical lenses, arranging refractive components to meet FoV, F-number, and EFFL specifications.A standard system comprises an object plane, optical elements, stops, and an image plane; RMS spot radius indicates image sharpness and aberrations.Source: §3 · p. 3, §3.1 · p. 3

3.2 Overview

• The framework retrieves physically compatible seeds, coordinates an optical tool suite for structural changes and local optimization, and accumulates design expertise through a closed loop.The system reflects on its design process while operating under physical guardrails.Source: §3.2 · p. 4

3.3 LensLib100K

• LensLib100K is built from 180,000 QGSO designs, and Optics-Aware Retrieval ranks seeds using precomputed optical metrics plus asymmetric FoV and aperture penalties.The library uses real glass catalogs and ray aiming for manufacturability and high-FoV ray tracing.Source: §3.3 · p. 4

3.4 Self-Evolving for Optical Lens Design

• Guarded optical edits, two-tier evaluation, and a curriculum agent that distills strategies from successful and failed attempts form the refinement loop.DLS ray tracing verifies the final design, while validated skills are stored for future prompts.Source: §3.4 · pp. 5–6

3.5 LensArena Benchmark

• LensArena addresses evaluation on isolated cases with 120 queries spanning FoV 5°–180° and #F 0.7–8.0; 70% are out-of-distribution relative to LensLib100K.It groups queries into three difficulty levels and defines success by task-specific RMS spot-radius thresholds.Source: §3.5 · p. 6

4.1 Experimental Setup

• Final RMS, EFFL, and F-number values come solely from Zemax; Avg RMS is averaged only over successful designs.Source: §4.1 · pp. 6–7

4.2 Results

• 80.0% success rate on Hard (versus LensNet’s 30.0% best and DeepLens’s 0%); Hard-query Avg RMS is 12.7 µm, reached in tens of seconds.The main results are evaluated on LensArena; the reported Hard-query RMS is an example of a high-quality solution.Source: Table 1 · p. 7, §4.2 · pp. 7–8
• Figure 4 reports superior structural rationality, higher success, and significantly lower RMS spot radius than the baselines.Source: Figure 4 · p. 7

4.1 Experimental Setup

• Baselines include LensNet retrieval with geometric scaling, genetic-algorithm QGSO, curriculum-based DeepLens, and same-foundation-model in-context learning.Source: §4.1 · p. 7

4.2 Results

• Figure 5 examples pair rational optical-power distribution across glass elements with tightly concentrated focal-plane spots.The layouts manage internal ray angles to minimize spherical aberrations and chromatic dispersion.Source: Figure 5 · p. 8, §4.2 · p. 8

4.3 Ablation Study

• Removing LensLib100K forces scratch design; replacing Optics-Aware Retrieval with Vanilla RAG raises Hard-query RMS to 189.6 µm versus 48.8 µm.The authors attribute the difference to physically grounded candidate matching rather than semantic proximity.Source: §4.3 · p. 9
• Ablating the Optical Tool Suite or Macro Orchestration reduces success and increases aberration errors.The authors conclude that open-ended LLM reasoning alone is insufficient for optical design.Source: §4.3 · p. 9
• Removing reflection or the skill library lowers performance on medium and hard tasks; the full pipeline has the highest success rates and lowest Avg RMS across tiers.Source: §4.3 · p. 9

5 Conclusion

• LensDesigner significantly outperforms optimization-based and deep-learning methods in success rate and design efficiency, particularly on complex tasks with extreme optical parameters.The paper also introduces LensArena, a benchmark of 120 diverse optical design tasks.Source: §5 · p. 9

A.1 The Physical Imaging Process and Essential Components

• A spherical lens system traces rays from an object plane through optical elements and stops to an image plane, where they form the focused image.Lens elements are characterized by surface curvatures, thickness, and material properties; the aperture stop restricts the traversing light bundle.Source: §A.1 · p. 13

A.2 Detailed Explanation of Optical Parameters

• FoV, F-number, sensor format, EFFL, total track, and RMS spot radius describe viewing breadth, aperture, coverage, focal geometry, length, and image sharpness.A smaller F-number means a larger aperture, while a smaller RMS spot radius corresponds to greater imaging sharpness.Source: §A.2 · pp. 13–14

B Detailed Related Work

• Existing optimization, neural-network, and retrieval methods can require manual intervention and lengthy tuning, while retrieval is constrained by its existing library.The paper contrasts these limitations with its aim of generating compliant configurations through text interaction.Source: B Detailed Related Work · p. 14

C Detailed Optics-Aware Retrieval Mechanism

• LensLib100K is constructed with enhanced QGSO, which initializes diverse lens structures and iteratively optimizes, selects, and mutates promising candidates.The described generation process outputs diverse structures after the prescribed number of generations.Source: C Detailed Optics-Aware Retrieval… · p. 15
• LensLib100K candidates are physically characterized with an offline paraxial ray tracer before database inclusion, using metrics such as EFFL, total track, and RMS spot radius.This grounds selection in precomputed physical states rather than language-based estimates.Source: C Detailed Optics-Aware Retrieval… · p. 15
• The retrieval distance prioritizes physical constraints, assigning FoV a symmetric weight of 10.0 and treating aperture mismatch asymmetrically.RMS spot radius has a minimal weight of 0.01 as a tiebreaker to limit early overfitting to paraxial approximations.Source: C Detailed Optics-Aware Retrieval… · p. 15
• Retrieval first selects the top 20 candidates by physical distance, then uses a lightweight language model to choose five seeds and routes out-of-distribution targets to a recovery procedure.Out-of-distribution targets are defined by FoV deviation above 10° or F-number deviation above 0.5.Source: C Detailed Optics-Aware Retrieval… · p. 15

• The methodology is organized around physical equations, algorithmic guardrails, and prompt engineering strategies.The section frames these as a technical breakdown of the autonomous optical design framework.Source: D Detailed Methodology · p. 16
• The framework's described engineering measures aim to ensure physical validity in optical design.Source: D Detailed Methodology · p. 16
• The section identifies deterministic convergence as an intended outcome of its equations, guardrails, and prompt strategies.Source: D Detailed Methodology · p. 16

D.1 Optical Tool Suite Implementation

• EFFL alignment deterministically scales radii, thicknesses, and semi-diameters by EFFLtarget/EFFLcurrent, with a 300% upper limit, and writes the update to internal state.After DLS optimization, finalized surface geometries are also synchronized to the agent's internal state.Source: §D.1 · pp. 16–17
• The split-lens tool divides a positive lens into two elements while conserving its original optical power, with the power ratio bounded from 0.2 to 0.8 and defaulting to 0.5.Source: §D.1 · p. 16
• Local optimization adds an EFFL soft penalty to the RMS objective, while Zemax DLS keeps glass central thickness fixed to avoid near-zero, unmanufacturable elements.The local optimizer also rejects curvature sign flips and stops early when RMS reaches a 3 µm noise floor.Source: §D.1 · pp. 16–17

D.2 Macro Orchestration and Workflow

• The agent follows a sequential workflow of retrieval, macroscopic alignment, microscopic optimization, and final verification, using a rapid paraxial filter for initial checks.The filter rejects EFFL deviations greater than 2% and mathematically infeasible combinations; a repeated-action circuit breaker can force a strategy shift.Source: §D.2 · p. 17

D.3 Knowledge-Grounded Self-Evolving Mechanics

• The self-evolving mechanism records geometric actions alongside resulting RMS and EFFL changes, and its filters prioritize successful out-of-distribution trajectories while retaining useful negative lessons.A failed glass substitution, for example, can yield a reusable material-boundary rule.Source: §D.3 · p. 17
• Additional LensArena layouts and spot diagrams show robust performance across diverse design specifications and superior imaging convergence compared with baseline methods.Source: Figure 9 · p. 18
• Distillation rejects common optical heuristics, deduplicates skills, and persists validated skills in a JSON library injected into later prompts.Skills may still qualify as novel when extreme parameters yield an EFFL scaling factor above 2.0 or a single-step RMS improvement above 80%.Source: §D.3 · pp. 17–18

D.4 General Implementation Details

• The framework uses LangChain, a ten-step sliding-window scratchpad, and a Flask bridge to connect the Python agent with Zemax OpticStudio.The reported hardware includes an NVIDIA 4090 GPU, Intel Core i9-14900K CPU, and 64GB of RAM.Source: §D.4 · p. 18

• The section presents additional qualitative results from the LensArena benchmark.Source: E More Qualitative Results · p. 18
• The referenced results are shown in Figure 9.Source: E More Qualitative Results · p. 18
• The section directs readers to further benchmark examples rather than reporting a new quantitative result.Source: E More Qualitative Results · p. 18

F.1 More Implementation Details

• Qwen3-32b runs the tool-calling agent, while Gemini-3-flash-preview handles retrieval reranking and self-evolve trajectory distillation.Lens-spec embeddings use text2vecbase on a locally persisted FAISS index; all LLM temperatures are set to 0.Source: §F.1 · p. 18
• The simulator couples an in-house Python paraxial ray-tracer with Ansys Zemax OpticStudio through ZOS-API and a Flask HTTP bridge.The ray-tracer supports offline RAG labeling, online check_spec verification, and local_optimize finite-difference gradient descent.Source: §F.1 · pp. 18–19
• Zemax is the source of truth for final RMS, EFFL, F-number, and merit-function values, with CDGM glass and Damped-Least-Squares optimization.The ReAct controller uses a 10-step sliding-window scratchpad and allows at most 25 reasoning iterations per query.Source: §F.1 · p. 19

G Data Examples from LensArena

• An easy example specifies FOV 30°, F/4.5, half image height 10.8 mm, target RMS < 0.048 mm, and distortion ≤ 5%.The query lists effective focal length 40.31.Source: G Data Examples from LensArena · p. 19
• A second easy example specifies FOV 25°, F/5.9, half image height 2.0 mm, target RMS < 0.056 mm, and distortion ≤ 5%.The query lists effective focal length 9.02.Source: G Data Examples from LensArena · p. 19
• A medium example specifies FOV 70°, F/2.6, half image height 5.3 mm, target RMS < 0.057 mm, and distortion ≤ 5%.The query lists effective focal length 7.57.Source: G Data Examples from LensArena · p. 19
• LensArena examples span three difficulty levels and include ultra-wide-angle and large-aperture challenges, with each query assigned an RMS success target.The figure caption identifies the most challenging queries as including ultra-wide-angle and large-aperture lenses.Source: Figure 10 · p. 20

H.1 Detailed Limitations

• The framework is primarily optimized for coaxial spherical systems, uses a subset of common glasses, and faces ray-tracing API latency.Off-axis, tilted, and non-imaging systems are not natively supported; broader glass catalogs or temperature-dependent refractive-index data would increase search complexity, while a lightweight proxy could accelerate screening.Source: §H.1 · pp. 20–21

H.2.1 Intended Uses

• The authors envision reducing design cycles for consumer, medical, and automotive applications, supporting lens-trade-off education, and enabling cross-disciplinary customization.These are stated intended uses, including enabling robotics or biology researchers to customize imaging systems without years of domain-specific training.Source: §H.2.1 · p. 21

H.2.2 Potential Risks and Mitigations

• The authors flag possible dual-use in surveillance or specialized weaponry and advocate safety-alignment protocols to flag requests that violate security norms.The concern is framed as theoretical.Source: §H.2.2 · p. 21
• Generated designs may miss manufacturing tolerances, so the paper calls for expert optomechanical tolerance analysis before fabrication; it also notes computational power use.The authors describe the system as augmentative and say distilling experience into a compact library mitigates environmental impact by reducing future brute-force optimization.Source: §H.2.2 · p. 21