PaperlayerView exact revision on arXiv

Source-linked AI summary

KATOsuper: Surrogate-accelerated neural topology optimization with sensitivity-consistent Fourier neural operators

Shengyu Yan, Jasmin Jelovica

arXiv:2609.27216v1cs.CEcs.AIcs.LGmath.NA

TL;DR

Topology optimization relies on repeated finite-element solves, and surrogate methods can struggle with inconsistent objective and sensitivity predictions. KATOsuper couples neural-reparameterized optimization with an SC-FNO that derives sensitivity through the predicted objective path. It reports 15–110× deployment-time speedup over MATLAB OC/MMA baselines.

  • Problem

    Topology optimization requires repeated finite-element analysis, while independently predicted objectives and sensitivities can be problematic for nonlinear stress optimization.

    Source: §1 · p. 1, §4.1 · p. 26

  • Method

    KATOsuper couples neural-reparameterized topology optimization with an SC-FNO that derives density sensitivity by automatic differentiation through the predicted objective field.

    Source: §5 · p. 29

  • Results

    15–110× deployment-time speedup over MATLAB OC/MMA baselines, with 110× acceleration at 512×256 resolution.

    Source: §5 · p. 30

  • Takeaways & Limitations

    Gradient-direction alignment matters more than exact magnitude accuracy for successful optimization with periodic FEA recalibration.

    Source: §5 · p. 29

    At 64× resolution extrapolation, compliance increases 29% compared with ground-truth OC results.

    Source: §4.3 · p. 28

Abstract

from arXiv · show

Topology optimization (TO) remains computationally intensive due to repeated finite element analysis (FEA) evaluations required at each iteration. While neural network-based surrogates offer potential acceleration, existing approaches often suffer from gradient inconsistency between predicted objectives and sensitivities, leading to optimization instability. This work presents KATOsuper, an objective-agnostic framework that couples neural-reparameterized topology optimization with a Sensitivity-Consistent Fourier Neural Operator (SC-FNO). The framework employs the forward_split architecture, which derives deployed sensitivities via automatic differentiation through the predicted objective field and thereby preserves consistency between the predicted objective and the gradient used for optimization. The case studies include three 2D benchmark problems and three 3D structures considering compliance or stress minimization. A physics-informed multi-channel input encoding with Fourier position embedding enables resolution-invariant learning, supporting zero-shot extrapolation beyond the training resolution, with useful performance at moderate scaling factors and topology-preserving exploration at up to 64x without retraining. The framework extends to 3D through KATO3D, featuring novel KANConv3D blocks with learnable B-spline activations. KATOsuper demonstrates 15--110x deployment-time speedup over MATLAB baselines while maintaining competitive optimality, with the clearest gains observed in complex 3D and stress-optimization cases. The insight that sensitivity direction matters more than magnitude enables robust optimization even with approximate physics evaluation, extensible to other differentiable physics-driven design objectives.

1. Introduction

Topology optimization is slowed by repeated finite-element solves, while surrogate approaches face gradient-consistency and generalization challenges. KATOsuper combines neural-reparameterized optimization with an SC-FNO evaluator designed to provide consistent sensitivities and support resolution extrapolation.

1. Introduction

  • Fine 3D topology optimization can require hours or days because each iteration needs costly finite-element analysis, with stress objectives adding further computational burdens.A typical 128×128 2D optimization also requires hundreds of finite-element solves.Source: §1 · p. 1
  • Prior neural-reparameterized methods still require FEA at every step, leaving the principal computational bottleneck unresolved.KATOsuper adds surrogate evaluation to the KATO progression, which includes neural reparameterization [35], KATO, and sensitivity-constrained training [52].Source: §1 · pp. 2–3
  • Existing surrogates may predict objectives and sensitivities separately, while convolutional models can require retraining at new resolutions and face domain shift.The paper also notes deployment barriers from Linux-centric toolchains and MPI workflows.Source: §1 · p. 2
  • KATOsuper derives the deployed sensitivity by automatic differentiation through the predicted objective field, preserving consistency between the objective and optimizer gradient.The forward_split design separates gradient-enabled density inputs from detached context channels.Source: §1 · pp. 2–3
  • Fourier position embedding and multi-channel physics inputs support zero-shot resolution extrapolation, with topology-preserving exploration reported up to 64× without retraining.Optional online learning recalibrates the surrogate through periodic FEA calls during optimization.Source: §1 · p. 3
  • KATOsuper reports 15–110× deployment-time speedup over MATLAB OC/MMA baselines while maintaining competitive optimization quality in the evaluated cases.The contribution summary also describes KATO3D benchmarks for compliance and structural stress.Source: §1 · p. 3

2.1. Problem formulation

The formulation optimizes element-wise material densities subject to a volume constraint. It uses SIMP interpolation and considers compliance or p-norm stress objectives.

2.1. Problem formulation

  • The design assigns each finite element a continuous density variable and minimizes an objective subject to a volume-fraction constraint.The density vector is ρ, the displacement field is u, and the volume limit is v̄.Source: §2.1 · p. 4
  • SIMP relates element stiffness to density through a penalized power law, with E_min ≈ 10^-9 E_0 preventing singularity and p_pen > 1 setting the penalization power.Source: §2.1 · p. 4
  • The study formulates both compliance minimization and stress minimization using a p-norm von Mises stress measure.The relaxation factor q and exponent p control the stress formulation, with p controlling its approximation to maximum stress.Source: §2.1 · p. 4

2.2. KATOsuper architecture overview

KATOsuper separates density generation from physics evaluation, allowing either FEA or a surrogate evaluator to serve the optimization loop. Its latent-space generator produces constrained density fields, while the evaluator returns objectives and sensitivities.

2.2. KATOsuper architecture overview

  • The Generator-Evaluator design decouples density generation from physics evaluation, so FEA and surrogate evaluators can be swapped without changing the generator.Source: §2.2 · p. 4
  • The KATO generator maps a latent vector to a density field and applies a differentiable projection to satisfy the volume-fraction constraint.The architecture can use standard convolutional or KAN convolutional layers.Source: §2.2 · p. 4
  • FEA computes ground-truth objectives and sensitivities, whereas SC-FNO predicts the objective field and derives sensitivity by automatic differentiation.SC-FNO inference scales as O(N log N), compared with O(N^1.5) for sparse direct FEA solvers.Source: §2.2 · p. 4
  • Each optimization step generates a density, evaluates its objective and density sensitivity, then updates the generator parameters and latent vector by gradient descent.KATOsuper uses AdamW, whose decoupled weight decay regularizes the latent vector and generator weights.Source: §2.2 · pp. 4–5

2.3. Physics embedding: unified input design

SC-FNO receives a physics-informed multi-channel representation and derives density sensitivities through the same predicted-objective path used at deployment. Fourier position features support resolution transfer, while KATO3D uses spline-activated KANConv3D blocks for volumetric optimization.

2.3. Physics embedding: unified input design

  • The SC-FNO input encodes density and the complete problem specification, using 24 channels in 3D and 18 in 2D after removing z-axis channels.Table 1 identifies channel 0 as gradient-tracked and describes the encoding as objective-agnostic.Source: §2.3 · p. 5, Table 1 · p. 6
  • Fourier position embedding maps normalized coordinates to sinusoidal features, supporting resolution-invariant learning across 2D and 3D problems.The shared normalized coordinate range enables zero-shot resolution transfer.Source: §2.3 · p. 5
  • Gaussian-blurred load and boundary-condition channels convey mechanical-feature proximity to distant regions, using a blur scale of σ_b = 2.0 voxels.Source: §2.3 · pp. 5–6
  • Forward_split tracks gradients through density while detaching context channels, so automatic differentiation obtains sensitivity from the predicted objective field.This makes the deployed objective and sensitivity follow the same computational path.Source: §2.3 · p. 6
  • KATO3D uses a latent-space KANConv3D generator with learnable B-spline activations, which provide a smoother density-field prior than ReLU-based architectures.Spline transformations capture finer topological details without increasing network depth.Source: §2.4 · p. 7, Figure 3 · p. 7
  • The 3D FEA formulation assembles stiffness from 8-node hexahedral elements, computes strains and stresses, and aggregates von Mises stress with a p-norm.The operations are vectorized using Einstein summation rather than explicit element loops.Source: §2.4 · p. 7

2.5. SC-FNO architecture

SC-FNO predicts objective fields and derives deployed sensitivities through the same prediction path. Its architecture combines input and output projections with Fourier-domain operator blocks and resolution-dependent mode clamping.

2.5. SC-FNO architecture

  • SC-FNO predicts the objective field and derives deployed sensitivities by automatic differentiation through that same prediction path.The network outputs the objective field exclusively, allowing differentiable objectives represented through this path without architecture modification.Source: §2.5 · pp. 8–9
  • The input projection lifts the multi-channel input, with 18 channels for 2D and 24 for 3D.Source: §2.5 · p. 8
  • Each FNO block combines Fourier-domain spectral convolution with a 1×1 convolution and GELU activation.The spectral weights operate on the first retained Fourier modes.Source: §2.5 · p. 8
  • The number of retained Fourier modes is clamped according to input-grid size at inference to support zero-shot extrapolation to unseen resolutions.At higher resolutions, clamping retains modes that fit the trained operator's spectral capacity while preserving learned physics relationships.Source: §2.5 · pp. 8–9
  • The output projection maps the representation to a single-channel objective field using the same output-head structure across objective-specific surrogate instances.Source: §2.5 · p. 9

2.6. Data normalization

The framework uses objective-agnostic normalization to address the extreme range of objective and sensitivity values in topology optimization. It applies different transforms to non-negative versus signed or heavy-tailed objectives, then corrects sensitivities when converting back to physical quantities.

2.6. Data normalization

  • Objective and sensitivity values span 10^-60 to 10^2, motivating specialized normalization transforms to stabilize gradient-based training.Source: §2.6 · p. 9
  • Non-negative objectives use Log1p scaling, while signed or heavy-tailed objectives use LogModulus scaling.LogModulus treats positive and negative values symmetrically.Source: §2.6 · p. 9
  • Median and interquartile range provide robust location and scale estimates, reducing sensitivity to outliers common in early optimization.The robust statistics are computed from the training distribution.Source: §2.6 · p. 9
  • At inference, a Jacobian correction converts normalized sensitivities to physical quantities to keep them physically sound across objectives.Source: §2.6 · p. 9

2.7. Offline training

Offline SC-FNO training uses optimization-trajectory samples labeled with FEA objectives and sensitivities, combining objective, sensitivity, and edge losses. Training gives extra attention to extreme sensitivity regions relevant to stress-driven updates.

2.7. Offline training

  • Each trajectory-frame training sample pairs density with an objective and FEA-computed sensitivity, and the offline loss combines three objectives balanced by AWL.Source: §2.7 · p. 9
  • The sensitivity loss emphasizes the top 5% most-negative sensitivity values with weight 5.0, while Sobel operators preserve sharp sensitivity boundaries.AWL balances objective, sensitivity, and edge losses despite their heterogeneous scales.Source: §2.7 · pp. 9–10
  • For stress objectives, forward_split differentiates a smoother predicted objective field, while extreme-sensitivity weighting and Sobel edge loss target critical regions and geometric discontinuities.These mechanisms address high-magnitude sensitivities near loads, supports, and re-entrant corners.Source: §2.7 · p. 10

2.8. Online learning

Periodic FEA calibration adapts the offline-trained surrogate during optimization, including when the generator explores out-of-distribution configurations. The online loss combines objective and sensitivity errors with gradient-direction alignment.

2.8. Online learning

  • Offline-trained surrogates may fail on out-of-distribution structural configurations, motivating online adaptation through periodic FEA calibration.Source: §2.8 · p. 10
  • FEA calibration runs for the first 5 optimization steps and then every 4 iterations, with the surrogate fine-tuned for 3 gradient steps at each calibration.The interval is an empirical compromise between calibration frequency and computational overhead; systematic ablation is left for future work.Source: §2.8 · p. 10
  • The online loss combines objective and sensitivity MSE with cosine alignment because topology optimization depends on sensitivity direction, not magnitude.Each periodic FEA call both validates the surrogate prediction and supplies fine-tuning data.Source: §2.8 · p. 10

2.9. High-performance implementation

The high-performance implementation combines faster FEA solving, cached mesh operations, and GPU execution optimizations. Together, these changes speed FEA solves, and the complete implementation is publicly available.

2.9. High-performance implementation

  • 14.6× speedup over SciPy’s splu for sparse Cholesky factorization comes from Intel MKL PARDISO, with factorization reused for adjoint sensitivity computation.Source: §2.9 · p. 11
  • 4–8× speedup for repeated FEA solves comes from caching DOF mappings and free-DOF subsampling indices across iterations.Source: §2.9 · p. 11
  • TF32 provides 3× throughput over FP32 for lifting and projection matrix multiplications, while Dynamo fuses element-wise operations to reduce kernel-launch overhead.TF32 is described as sufficiently precise for physics regression tasks.Source: §2.9 · p. 11
  • 4–6× average FEA-solve speedup over the default SciPy sparse solver approaches optimized MATLAB solver performance.The combined optimizations yield this average speedup.Source: §2.9 · p. 11
  • The complete KATOsuper implementation, including the generator and SC-FNO surrogate, is publicly available.Source: §2.9 · p. 11

3. Experiments

The experiments evaluate KATOsuper across 2D benchmarks and 3D settings, including resolution extrapolation and online FEA correction for new geometries. They report accelerated optimization, close-to-direct-FEA results in 2D compliance cases, useful but declining performance as extrapolation grows, and generator-side memory traffic as the main high-resolution bottleneck.

3.1. Benchmark cases and experimental setup

  • Training uses optimization-trajectory frames across the benchmark geometries; 3D split-load and ship-seat cases test transfer beyond the trained cantilever using online FEA calibration.For the 2D compliance surrogate, masked sensitivity loss falls from about 1.7 to 0.035, while Sobel edge loss reaches 0.021 and objective-field loss reaches 0.122.Source: §3.1 · pp. 11–12, Figure 7 · p. 13

3.2. 2D compliance optimization

  • KATOsuper differs from KATO by less than 1% in compliance across the 2D benchmarks, while KATO has the lowest values on MBB and L-shaped cases.KATO’s compliance is 98.32 on MBB and 492.79 on the L-shaped bracket; KATOsuper’s is 99.05 and 496.40, respectively.Source: §3.2 · pp. 14–15, Figure 8 · p. 14, Figure 9 · p. 15

3.3. 2D stress optimization

  • On the L-shaped bracket, KATO reduces p-norm stress 16.4% versus MMA; KATOsuper’s cantilever p-norm and maximum stresses are 383.99 MPa and 115.58 MPa.KATO records 502.81 MPa p-norm stress on the bracket, while KATOsuper records 511.96 MPa p-norm and 293.90 MPa maximum stress there.Source: §3.3 · pp. 16–17, Figure 14 · p. 18
  • KATOsuper achieves 91× speedup over MMA and 19× over KATO in stress optimization by replacing FEA and stress post-processing with surrogate inference.For cantilever compliance optimization, it completes each iteration in 0.025s and achieves 19× speedup over MATLAB OC at 128 × 64 resolution.Source: §3.2 · p. 16, Figure 11 · p. 16, §3.3 · p. 18

3.4. Resolution extrapolation

  • At 4× resolution KATOsuper’s compliance is 121.79 versus OC’s 112.04; at 16× it is 113.84, within 8.5% of KATO’s 104.90.At 16× resolution, KATO beats OC by 6.5%, and KATOsuper is within 1.5% of OC; the results are separate nonconvex trajectories, not a monotonic capability ranking.Source: §3.4 · pp. 18–20, Figure 16 · p. 19, Figure 17 · p. 20
  • At 64× resolution, KATOsuper’s compliance rises to 133.79 versus OC’s 103.72, yet its structure retains clear load paths and enables rapid exploration.It completes 50 iterations in 39s versus 2745s for MATLAB OC, a 70× speedup; the proposed workflow refines promising candidates with KATO iterations.Source: Figure 18 · p. 20, §3.4 · p. 20

3.5. 3D compliance optimization

  • The 3D cantilever proof of concept yields compliance 674.40 with online KATOsuper, close to OC’s 673.04, while KATO reaches 637.63.KATO3D uses KANConv3D blocks with learnable B-spline activations; the tested meshes remain modest because 3D training data are substantially more costly to generate.Source: Figure 19 · p. 21, §3.5 · p. 21
  • The 3D split-load cantilever tests out-of-plane redistribution under two downward load patches on the free end.At 32 × 32 × 16 resolution, the two 5 × 5 patches sit at the front and back of the domain; fixed boundaries are on the right face.Source: §3.5 · p. 21
  • For the 3D split-load cantilever, compliance is 637.48 for KATO, 643.47 for OC, and 640.33 for KATOsuper, whose result remains in the same performance range.The comparison reports similar compliance values while KATOsuper provides substantial acceleration.Source: §3.5 · p. 21, Figure 20 · p. 22
  • On the 3D ship seat-base, KATOsuper uses 17 FEA calls in 30 steps and reaches compliance 14.66 versus KATO’s 14.05 after binarization and FEA recomputation.The same cantilever-trained surrogate is adapted online to the new geometry, yielding a connected, physically plausible structure with a modest compliance penalty.Source: §3.5 · p. 22, Figure 21 · p. 23
  • Online fine-tuning raises 3D sensitivity-direction cosine similarity from below 0.1 initially to approximately 0.95 by step 24, while loss stabilizes below 0.2.Calibration runs for the first five steps and every four steps thereafter; step 24 is the 10th online learning intervention.Source: §3.5 · p. 22, Figure 22 · p. 23
  • Online FEA correction halves 3D cantilever optimization time from 51.9s to 29.2s and reduces split-load cantilever time from 129.6s to 57.4s, a 2.2× speedup.These are online-hybrid timings with periodic FEA correction, separate from deployment timings.Source: §3.5 · p. 22, Figure 23 · p. 24

3.6. Efficiency summary

  • 15–110× deployment-time speedup over MATLAB OC/MMA baselines, including 110× at 512 × 256 and 15× over MATLAB OC in the 3D case, is reported for KATOsuper.KATO only matches MATLAB speed in the 3D deployment comparison, and KATOsuper is faster than both MATLAB and the open-source Python baseline in all completed cases.Source: Table 5 · p. 24, §3.6 · p. 24
  • At 1024 × 512, MATLAB uses 24.27 GB of CPU RAM, Python runs out of memory, and KATOsuper uses 10.72 GB of GPU memory while keeping CPU memory below 8 GB.The reported 10.72 GB measurement is substantially below the 80 GB A100 requirement of a previous KATO implementation at 512 × 256.Source: §3.6 · pp. 24–25, Table 6 · p. 25
  • At 1024 × 512, generator-side KAN B-spline activation expansion makes KATOsuper memory-bound, yet it remains more than two orders of magnitude faster than FEA-based KATO.At this resolution, temporary tensors reach hundreds of megabytes; the source identifies generator memory traffic as the next bottleneck after removing the sparse-solver bottleneck.Source: §3.6 · pp. 25–26

4. Discussion

The discussion argues that sensitivity consistency is particularly important for complex stress objectives, while surrogate evaluation and GPU execution reduce the computational burden of topology optimization. It also describes online adaptation, resolution and training-data limitations, and potential extensions.

4.1. Sensitivity consistency as the key enabler

  • Independent objective and sensitivity heads can sometimes yield acceptable compliance trajectories, but they fail catastrophically on localized, nonlinear stress sensitivities; forward_split differentiates the predicted objective to provide a coherent descent direction.The discussion attributes stress-head failures to sign errors and local noise decoupled from predicted stress values.Source: §4.1 · p. 26

4.2. Computational efficiency and scalability

  • 15–110× deployment-time speedup over MATLAB OC/MMA baselines, with speedup increasing at higher resolutions as SC-FNO replaces repeated linear-system solves with learned-operator evaluation.SC-FNO is described as scaling with O(N log N) computation and O(N) memory, compared with super-linear factorization costs for sparse direct solvers.Source: §4.2 · p. 26

4.2.1. The physical memory wall

  • At 1024 × 512, PARDISO iteration time rises from about 3 seconds to over 87 seconds when element count increases 4×, while SC-FNO avoids the sparse-factorization storage bottleneck.The surrogate stores approximately 2M network weights independently of mesh resolution.Source: §4.2.1 · pp. 26–27

4.2.2. End-to-end GPU acceleration

  • Keeping both generator and evaluator on GPU removes KATO’s CPU/GPU execution split; PyTorch changes also improve iteration time and maximum stable resolution.Against KATO(legacy), per-iteration time falls from 0.36s to 0.19s at 128×64, and maximum stable resolution rises from about 40,000 to over 520,000 elements.Source: §4.2.2 · p. 27, §4.2.3 · p. 27

4.2.4. Automatic differentiation for sensitivity analysis

  • For p-norm stress, SC-FNO obtains density sensitivities by backpropagation, avoiding case-specific manual adjoint derivations for differentiable objectives.The text identifies thermal compliance and coupled thermo-mechanical problems as possible extensions.Source: §4.2.4 · p. 27

4.2.5. Online learning for practical 3D applications

  • Periodic FEA calibration every 4 iterations cuts 3D optimization time by about 50% versus pure FEA while maintaining high gradient cosine similarity; adaptation supports related boundary-condition runs.The reported 3D split-load benchmark values are KATO 637.48, OC 643.47, and KATOsuper 640.33; online calibration is framed as adaptation, not zero-shot boundary-condition extrapolation. Offline preparation also requires data-generation and training time, and may not pay off for only a few optimizations.Source: §4.2.5 · pp. 27–28

4.3. Limitations

  • Zero-shot resolution extrapolation trades numerical accuracy for scale: at 64×, compliance increases 29% versus ground-truth OC, although structures remain topologically valid.The surrogate’s training resolution limits captured physical mode frequencies; SC-FNO also assumes test problems are within the training distribution, and online FEA corrections reduce speedup.Source: §4.3 · pp. 28–29

4.4. Future directions

  • Future work includes extending inputs for multiphysics, adding uncertainty-aware FEA fallback, and exploiting the decoupled architecture for transfer learning across related design domains.The proposed multiphysics extension requires objectives computable through a forward pass of the physics solver.Source: §4.4 · p. 29

5. Conclusion

KATOsuper couples neural-reparameterized topology optimization with a sensitivity-consistent Fourier neural operator to reduce repeated FEA costs. It reports substantial deployment speedups alongside competitive tested optimization quality, with resolution extrapolation and online learning broadening its use in the evaluated settings.

5. Conclusion

  • KATO3D extends neural-reparameterized optimization to volumetric structures using a latent-space KANConv3D generator, while the decoupled generator-evaluator design supports modular adaptation.The 3D experiments are described as a proof-of-concept for volumetric surrogate-accelerated optimization.Source: §5 · pp. 29–30
  • The SC-FNO forward_split design derives deployed sensitivities by automatic differentiation through the predicted objective field rather than independently regressing objective and sensitivity.The design combines the sensitivity-constrained training principle with a TO-specific inference architecture.Source: §5 · p. 29
  • Zero-shot inference reaches 64× the training resolution without retraining, with a 29% compliance increase at that scale and topology-valid structures for exploration.The reported comparison is against ground-truth OC results.Source: §5 · p. 29
  • The framework supports compliance and stress minimization across 2D benchmarks and 3D compliance, with KATO reducing L-shape p-norm stress by 16.4% versus MMA.The L-shape values are 502 versus 602 MPa p-norm stress.Source: §5 · p. 29
  • 15–110× deployment-time speedup over MATLAB OC/MMA baselines accompanies competitive optimality in the tested settings.The reported gains are greater at higher resolutions where FEA memory scaling dominates.Source: §5 · p. 30
  • The tested training sets comprise about 6,200 frames for 2D and 3,200 for 3D, with online learning able to compensate for limited offline data in those settings.Source: §5 · p. 30
Loading 2609.27216v1…