Source-linked AI summary
Neural reparameterization improves structural optimization
Stephan Hoyer, Jascha Sohl-Dickstein, Sam Greydanus
TL;DR
Structural-optimization solution quality depends strongly on how the problem is parameterized. The paper optimizes a neural network that outputs structural densities; across 116 tasks, this approach produced qualitatively and quantitatively better designs than baselines.
Problem
Structural-optimization solution quality depends strongly on parameterization, and the effect of parameterization is consequential for final designs.
Source: §1 · p. 1Method
The approach reparameterizes structural optimization in terms of a CNN's output and optimizes the network parameters for each task.
Source: §5 · pp. 4–5Results
Takeaways & Limitations
The authors conclude that parameterization has a powerful effect on solution quality in numerically optimized structural-design tasks.
Source: §5 · p. 4
Abstract
from arXiv · showhide
Structural optimization is a popular method for designing objects such as bridge trusses, airplane wings, and optical devices. Unfortunately, the quality of solutions depends heavily on how the problem is parameterized. In this paper, we propose using the implicit bias over functions induced by neural networks to improve the parameterization of structural optimization. Rather than directly optimizing densities on a grid, we instead optimize the parameters of a neural network which outputs those densities. This reparameterization leads to different and often better solutions. On a selection of 116 structural optimization tasks, our approach produces the best design 50% more often than the best baseline method.
1 Introduction
Structural optimization outcomes depend strongly on parameterization, yet the deep image prior is underexplored in computational science. The paper proposes reparameterizing grid-based optimization with a neural network and evaluates it on 116 structural optimization tasks.
1 Introduction
- In a multi-story building example, changing the optimization method or using pixels versus a neural network leads to different final designs.The design task distributes material over a two-dimensional grid to maximize structural resilience.Source: §1 · p. 1, Figure 1 · p. 1
- The proposed reparameterization moves optimization from a grid basis to a neural-network basis.The CNN parameterization, constraint step, and physics simulation are differentiable, with gradients computed by automatic differentiation.Source: §1 · pp. 1–2, Figure 2 · p. 2
- On 116 structural optimization tasks, the neural-network approach produced solutions that were quantitatively and qualitatively better than baselines.The approach reparameterizes the optimization problem from a grid basis to a neural-network basis.Source: §1 · pp. 1–2
2 Methods
The method optimizes a neural network that outputs structural densities, while retaining a differentiable constraint and physics pipeline. The experiments compare this approach with pixel-based optimization and stronger topology-optimization baselines across a diverse task suite.
2 Methods
- The computational graph runs a neural-network forward pass, enforces constraints, simulates physics, and computes a scalar compliance loss; optimization can target either design inputs or network parameters.The implementation makes the CNN parameterization, constraint step, and physics simulation differentiable, and uses automatic differentiation for gradients.Source: §2 · p. 2
- Designs use a regular grid of linear finite elements, with filtered densities determining stiffness and compliance subject to equilibrium, volume, and density constraints.The limiting linear solve U=K^-1F uses sparse Cholesky factorization.Source: §2 · p. 2
- A constrained sigmoid maps unconstrained logits to densities satisfying the volume constraint; its offset is found by binary search and differentiated through by implicit differentiation.The mapping enforces valid densities during the forward pass.Source: §2 · p. 2
- Pixel and CNN parameterizations are both optimized with L-BFGS, while MMA and OC are included because they proved stronger than Pixel-LBFGS.The authors note that standard topology-optimization algorithms are ill-suited to training neural networks.Source: §2 · p. 3
- The 116-task suite spans 28 categories, volume fractions from 0.05 to 0.5, and between 2^11 and 2^16 elements.It combines selected classic problems with hand-designed bridges, trees, ramps, walls, and buildings.Source: §2 · p. 3
- The CNN uses a U-net-inspired architecture with only the upsampling half retained and its initial activation made trainable; the authors frame differentiable physics models as a general strategy for computational science.The five-stage network uses tanh nonlinearities, resizing, normalization, convolutions, and learned biases.Source: §2 · pp. 2–3
3 Analysis
Neural reparameterization matched MMA on small problems and performed better on large ones, where CNN-LBFGS solutions were more likely to have low error. The CNN designs were often simpler and structurally different from baseline designs.
3 Analysis
- CNN reparameterization performed as well as MMA on small problems and better on large problems, while outperforming OC and Pixel-LBFGS on both sizes.For large problems, defined as ≥215 grid points, CNN-LBFGS solutions were more likely to have low error.Source: §3 · p. 3
- Performance was reported as typical median performance over 101 CNN random seeds and as best-of-ensemble performance using the same initializations across models.Other models used constant initialization, which was better than their median performance.Source: §3 · p. 3
- The authors hypothesized that optimizing CNN weights enables structures at several spatial scales to be optimized at once, addressing grid mesh-dependency and spiderweb structures.They examined qualitative examples from all 116 tasks and highlighted five in Figure 5.Source: §3 · p. 4
- CNN-LBFGS designs had fewer spiderweb artifacts; on the cantilever beam, they used eight supports versus MMA’s eighteen.The authors interpret simpler structures as evidence of a bias toward large-scale structure, especially on large problems.Source: §3 · p. 4
- CNN-LBFGS often converged to structures unlike the baselines, including a single branching support rather than double columns on the thin support bridge, and pillars rather than branching patterns on the roof task.The baseline structures more closely resembled one another than they did CNN-LBFGS designs.Source: §3 · p. 4
4 Related work
The paper contrasts its multi-scale reparameterization with mesh-based topology optimization and with CNN methods trained on structures produced by conventional optimization. Its qualitative examples show CNN-parameterized solutions often differ from baselines and are often simpler.
4 Related work
- Best-of-ensemble CNN solutions were best or near-best in 99 of 116 tasks, compared with 66 of 116 for MMA.Figure 5 shows five qualitative examples; the scores report relative difference from the best overall design in each row.Source: Figure 5 · p. 4
- The CNN examples were qualitatively different from the baselines and often involved simpler, more effective structures.The paper illustrates these differences with five examples in Figure 5.Source: Figure 5 · p. 4
- Unlike locally refined meshes, the proposed method permits optimization at multiple spatial scales at once.Grid meshes are the most common topology-optimization parameterization; polyhedral meshes and locally refined meshes are also used.Source: §4 · p. 4
- Most prior CNN approaches train on structures generated by regular topology optimization, which can reduce computation but relaxes physics and design constraints and limits outputs to the training data.The paper contrasts these approaches with its method, which it says produces better designs while obeying exact physics constraints.Source: §4 · p. 4
5 Conclusions
The paper reparameterizes structural optimization in terms of a convolutional neural network’s output, optimizing the CNN parameters for each task. Across 116 tasks, the framework produced qualitatively and quantitatively better designs.
5 Conclusions
- The method reparameterizes structural optimization tasks using a convolutional neural network’s output, motivated by the inductive biases of untrained deep image models.The CNN parameters are optimized separately for each task.Source: §5 · pp. 4–5
- Across 116 tasks, the framework produced qualitatively and quantitatively better designs.Source: §5 · p. 5