Source-linked AI summary
VisFlow: Interpretable Flow Matching for Visibility Reconstruction, Characterization, and Recognition
Rahul Moorthy, Volkan Isler
TL;DR
Small geometric perturbations can abruptly change polygon visibility graphs, challenging graph-to-geometry learning. VisFlow combines continuous SDF representations with explicit polygon evolution, improving reconstruction and characterization results over state-of-the-art methods.
Problem
Small geometric changes can alter multiple visibility relations, while existing SDF-based generative methods do not tie intermediate states to polygon formation.
Source: §1 · pp. 1–2Method
VisFlow learns approximately linear SDF latent trajectories from triangle-to-polygon evolution and uses visibility-conditioned flow matching with differentiable visibility supervision.
Source: §1 · p. 2, §4.1 · p. 5Results
Takeaways & Limitations
The trajectory-supervision formulation also applies to protein structure reconstruction from contact graphs.
Source: §1 · p. 3, §6 · p. 9The formulation focuses on simple polygons without boundary crossings or holes.
Source: §3 · p. 3
Abstract
from arXiv · showhide
Methods for learning continuous representations, such as latent manifolds, are fundamental and ubiquitous in modern machine learning. Yet many geometric objects combine continuous shape information with discrete combinatorial structure. This combination presents a fundamental challenge for representation learning because small geometric perturbations can induce abrupt changes in the underlying discrete representation. As a result, the smoothness assumptions that make continuous representations effective no longer hold. Polygons provide a canonical example. Their vertex locations vary continuously, but infinitesimal changes in geometry can dramatically alter the associated visibility graph. Existing methods address the inverse mapping from visibility graphs to polygon geometry using vertex-, triangulation-, or signed-distance-function (SDF)-based representations. Vertex- and triangulation-based approaches construct polygons through explicit geometric operations. However, they are less effective at modeling the relationship between geometry and visibility. SDF-based approaches capture this relationship more naturally. Their intermediate states, however, do not correspond to explicit stages of polygon construction. We present VisFlow, a novel, interpretable flow formulation that combines continuous SDF representations with explicit polygon evolution. We evaluate VisFlow on Visibility Reconstruction and Visibility Characterization. Visibility Reconstruction generates one polygon consistent with a visibility graph (G), while Visibility Characterization characterizes all such polygons. VisFlow maintains 20% more visibility relations on Visibility Reconstruction and improves coverage by 30% on Visibility Characterization compared to state-of-the-art methods.
1 INTRODUCTION
Small geometric changes can abruptly alter a polygon’s discrete visibility graph, making graph-to-geometry learning difficult. VisFlow combines continuous SDFs with explicit polygon evolution for visibility-conditioned generation.
1 INTRODUCTION
- Small geometric perturbations can change several visibility relations at once, so nearby geometries may be far apart in the discrete graph representation.This discontinuity complicates learning the inverse mapping from a visibility graph to its geometric realization.Source: §1 · pp. 1–2
- Vertex- and triangulation-based representations model construction but struggle to preserve target visibility, while SDF methods capture geometry–visibility relationships but use unstructured intermediate states.VisFlow asks whether explicit construction can be incorporated into SDF-based generation.Source: §1 · p. 2
- VisFlow learns SDF latent trajectories from an initial triangle to a target polygon and conditions rectified flow on the visibility graph.A differentiable Visibility Prediction Network supplies visibility supervision during training and guides inference optimization.Source: §1 · p. 2
- VisFlow improves F1-score by 20% on Visibility Reconstruction and coverage by 30% on Visibility Characterization compared with state-of-the-art methods.The paper also introduces a Visibility Reconstruction dataset with 22% greater shape diversity than the state-of-the-art dataset.Source: §1 · p. 3
- The paper extends trajectory supervision to protein structure generation from contact graphs.It also presents a Visibility Reconstruction dataset with 22% greater shape diversity than the state-of-the-art dataset.Source: §1 · p. 3
2 RELATED WORK
Related work spans visibility-graph problems, interpretable flow matching, and polygon propagation. VisFlow distinguishes itself by combining continuous SDF representations with geometry-driven polygon-evolution supervision.
Visibility Graph Reconstruction, Recognition, and Characterization
- Learning-based visibility methods use SDFs to preserve visibility better than GNN- and mesh-based alternatives, but coarse grids limit geometry and diffusion states do not model polygon formation.VisFlow addresses these limitations with continuous SDFs and explicit polygon-evolution supervision.Source: §2 · p. 3
Flow Matching
- Standard flow matching uses unstructured source distributions, whereas prior physically meaningful variants change the dynamics; VisFlow uses geometric evolution to supervise latent space while retaining source-to-target flow transport.The cited variants include transitions between Ising-system equilibrium states and partially known ODE/PDE dynamics.Source: §2 · p. 3
Polygon Propagation Approaches
- Prior polygon-propagation methods evolve shapes for boundary fitting, while VisFlow uses polygon evolution to supervise generation from a visibility graph.Examples include grassfire propagation, neighbor-based vertex updates, and learned contour-fitting methods.Source: §2 · p. 3
3 PROBLEM FORMULATION
The paper studies visibility graphs of simple, simply connected polygons and distinguishes reconstruction, characterization, and recognition as separate tasks.
3 PROBLEM FORMULATION
- The formulation focuses on simple polygons without boundary crossings or holes, and a visibility graph is valid when at least one such polygon realizes it.For polygon P, V(P) denotes its visibility graph.Source: §3 · p. 3
- Visibility Reconstruction generates one polygon P whose visibility graph equals the given valid graph G.That is, V(P) = G.Source: §3 · p. 3
- Visibility Characterization generates all simple polygons P whose visibility graph equals the given valid graph G.The task concerns the set of realizations sharing the same graph.Source: §3 · p. 4, Figure 2 · p. 4
- Visibility Recognition determines whether an arbitrary graph G is realized by any polygon P.The figure contrasts this validity test with reconstruction and characterization.Source: §3 · p. 4, Figure 2 · p. 4
4 METHOD
VisFlow encodes continuous polygon SDFs into latents whose intermediate states correspond to polygon formation. A visibility predictor and graph-conditioned flow network support training and generation from triangle initializations.
4 METHOD
- VisFlow encodes polygon signed distance functions (SDFs) as spatially localized latents, learns approximately linear polygon-formation trajectories from reversed grassfire evolution, and sequentially trains visibility prediction, the SDF latent generator, then the flow network.Source: §4 · p. 4
4.1 SDF LATENT GENERATOR
- The SDF Latent Generator reverses grassfire collapse-to-triangle trajectories so interpolated latent states decode to corresponding polygon-formation stages conditioned on their visibility graphs.Its encoder–decoder uses spatially localized patch latents and visibility-conditioned vertex decoding.Source: §4.1 · p. 5
4.2 VISIBILITY PREDICTION NETWORK
- The Visibility Prediction Network approximates non-differentiable polygon visibility, enabling visibility-loss gradients during generator training and gradient-based vertex refinement at inference.The pretrained predictor is frozen during both uses.Source: §4.2 · pp. 5–6
4.3 FLOW NETWORK
- The Flow Network learns graph-conditioned rectified flow between encoded triangle and polygon latents; sampling decodes the evolved latent and refines vertices against target visibility and self-intersection losses.It predicts velocity from the interpolated latent, time, and target visibility graph.Source: §4.3 · p. 6
5 RESULTS
The results section evaluates VisFlow on polygon visibility reconstruction, characterization, and recognition, as well as protein contact-graph reconstruction. It uses graph-pair classification metrics for reconstruction and neighborhood exploration as a proxy for characterization coverage.
5 RESULTS
- The experiments compare VisFlow with task-specific baselines trained on the dataset's training split and evaluated on its test split.For Visibility Reconstruction, each generative approach produces 50 polygon samples, and the highest predicted visibility F1 sample is selected; the GNN baseline produces one polygon.Source: §5 · p. 6, §5.1 · p. 7
- Visibility Reconstruction is evaluated as vertex-pair graph classification, while Visibility Characterization estimates coverage by neighborhood exploration.For characterization, breadth-first search retains nodes only when visibility F1 exceeds T and polygons meet the Chamfer-distance threshold Td; diversity is assessed by varying Td.Source: §5 · p. 7
5.1 VISIBILITY RECONSTRUCTION
- VisFlow outperforms all baselines on Visibility Reconstruction.The flow model already exceeds the baselines without inference-time optimization, which adds further improvement; continuous SDF representation is the primary source of the reconstruction gain.Source: §5.1 · p. 7, Table 1 · p. 8
5.2 VISIBILITY CHARACTERIZATION
- VisFlow achieves substantially higher Visibility Characterization coverage than VisDiff, particularly at larger F1 and distance thresholds.Inference-time optimization adds gains, but polygon propagation is the primary source of the coverage improvement.Source: Table 2 · p. 8, §5.2 · p. 8
5.3 VISIBILITY RECOGNITION
- VisFlow achieves 80% Visibility Recognition accuracy near its mean reconstruction F1, while VisDiff classifies all inputs as invalid.The test set uses visibility graphs from polygons with holes added to half of the test polygons.Source: Figure 5 · p. 8, §5.3 · p. 8
5.4 CONTACT-GRAPH RECONSTRUCTION
- On FragFold150, VisFlow substantially outperforms the baselines for protein contact-graph reconstruction.The evaluation compares reconstructed and target contact graphs using accuracy, precision, recall, and F1; VisDiff and MeshGPT are omitted as their architectures are specialized for other representations.Source: Table 3 · p. 9, §5.4 · p. 9
6 CONCLUSION
VisFlow combines continuous SDF representations with explicit polygon evolution for geometric realization from visibility graphs. The paper reports gains on visibility reconstruction and characterization, while noting a fixed-patch assumption in its SDF representation.
6 CONCLUSION
- VisFlow combines continuous SDF representations with explicit polygon evolution for learning geometric realizations from visibility graphs.The conclusion attributes reconstruction-accuracy improvements primarily to the continuous SDF representation and coverage and diversity gains to trajectory-supervised polygon evolution.Source: §6 · p. 9
- On the new visibility-graph reconstruction dataset, VisFlow improves F1-score by 20% on Visibility Reconstruction and coverage by 30% on Visibility Characterization.The dataset has 22% greater shape diversity than the state-of-the-art dataset.Source: §6 · p. 9
- The patch-based SDF representation assumes shape information is distributed uniformly across fixed spatial patches.Future work will add patch radius as a parameter to relax this assumption.Source: §6 · p. 9
7 APPENDIX
The appendix describes polygon evolution and visibility predicates, then details a spatially structured SDF representation for generating polygons from visibility graphs.
7 APPENDIX
- The appendix outlines preliminaries, SDF latent-generator and flow-network architectures, visibility prediction, and further evaluations and ablations.It also covers contact-graph reconstruction and generated-dataset statistics.Source: §7 · p. 12
7.1 PRELIMINARIES / 7.2 SDF LATENT GENERATOR ARCHITECTURE
- Visibility requires the segment between two vertices to satisfy in-cone conditions at both endpoints and avoid intersections with non-incident polygon edges.The SDF generator encodes polygons with 36 spatially localized patch latents and uses visibility-conditioned vertex queries to decode polygon vertices.Source: §7.1 · p. 12, §7.2 · pp. 12–13, Figure 8 · p. 13
7.1 PRELIMINARIES
- Grassfire flow moves polygon vertices inward along local SDF-normal directions until an edge collapse reduces the vertex count; training reverses this trajectory to form polygons from a triangle.Intermediate grassfire states supervise corresponding points along the latent trajectory.Source: Figure 7 · p. 12, §7.1 · p. 12
7.3 FLOW NETWORK ARCHITECTURE
The flow network conditions a spatial SDF latent representation on a target visibility graph. The visibility predictor reconstructs test-set graphs perfectly, and VisFlow also outperforms baselines on the original VisDiff dataset.
7.3 FLOW NETWORK ARCHITECTURE
- A time-conditioned U-Net predicts velocities over a 6 × 6 map of spatial SDF patch latents, conditioned on per-vertex embeddings of the target visibility graph.The graph embeddings are produced by an MLP and supplied through cross-attention.Source: §7.3 · p. 13
7.4 VISIBILITY PREDICTION NETWORK
- The Visibility Prediction Network perfectly reconstructs test-set visibility graphs and substantially outperforms a conventional transformer baseline.It multiplies endpoint in-cone probabilities and combines them with an aggregated intersection probability to predict visibility.Source: §7.4 · p. 13, Table 4 · p. 13, Figure 9 · p. 14
7.5 Visibility Reconstruction
- VisFlow outperforms all baselines on the original VisDiff dataset.Table 5 reports accuracy, precision, recall, and F1-score for this comparison.Source: §7.5 · p. 14, Table 5 · p. 14
7.6 CONTACT-GRAPH RECONSTRUCTION
The contact-graph extension retains trajectory-supervised latent flow while replacing polygon-specific components with protein representations and a residue contact-graph encoder.
7.6 CONTACT-GRAPH RECONSTRUCTION
- The protein contact graph is a binary residue adjacency matrix, with an edge when two residues’ Cα atoms are within 8 Å.The backbone decoder predicts coordinates for proteins up to 270 residues and is conditioned on sequence and the target graph.Source: §7.6 · pp. 14–15
- The contact reconstruction loss combines Kabsch-aligned coordinate error with weighted binary cross-entropy for residue pairs separated by at least six sequence positions.The loss weights are 0.05 for contacts and 0.95 for non-contacts.Source: §7.6 · p. 15
- After training the SDF encoder and backbone decoder, the method trains a contact-graph-conditioned Rectified Flow model over the latent representation.Inference uses 60 Euler steps.Source: §7.6 · p. 15
7.7 ABLATION STUDY
The ablations report benefits from visibility loss and optimization, while comparison with continuous SDF latent diffusion finds polygon propagation especially beneficial for characterization rather than reconstruction.
7.7 ABLATION STUDY
- Adding optimization to VisFlow improves performance on both Visibility Reconstruction and Visibility Characterization.Tables 7 and 8 report reconstruction metrics and characterization coverage, respectively.Source: §7.7 · p. 15, Table 7 · p. 16, Table 8 · p. 16
- Incorporating visibility loss substantially improves Visibility Reconstruction performance.Table 6 reports accuracy, precision, recall, and F1-score for the loss ablation.Source: §7.7 · p. 15, Table 6 · p. 16
7.8 HOW DOES VISFLOW COMPARE WITH LATENT DIFFUSION?
- Compared with Cont-VisDiff, polygon propagation provides no meaningful reconstruction improvement but yields substantial gains in characterization.The authors attribute the characterization gains to more effective exploration of distinct polygons consistent with a target graph.Source: §7.8 · p. 16, Table 9 · p. 16, Table 10 · p. 16
7.9 VERTEX PREDICTION ARCHITECTURE
With ground-truth SDFs and visibility graphs, VisFlow recovers polygon vertices more reliably than VisDiff. The accompanying dataset broadens geometric and visibility-graph diversity, including more highly concave polygons.
7.9 VERTEX PREDICTION ARCHITECTURE
- With ground-truth SDFs and visibility graphs, VisFlow recovers polygons more reliably than VisDiff, whose coarse SDF resolution limits geometric detail needed for accurate vertex recovery.The comparison isolates vertex-prediction architectures while supplying both methods with ground-truth SDFs.Source: §7.9 · p. 17, Table 11 · p. 17
7.10 DATASET GENERATION
- The dataset contains 10,000 polygons with unique visibility graphs, plus 20 visibility-preserving vertex perturbations per polygon; test graphs are excluded from training.Polygons are generated with 2-opt moves, and sampling across link-diameter values broadens concavity coverage.Source: §7.10 · p. 17
- The dataset has a larger average link diameter and wider link-diameter range than compared datasets, with maximum diameter rising from 9.0 in VisDiff to 11.0.Link diameter and visibility-graph edge density characterize variation; Figure 11 shows their training distributions after resampling across link-diameter values.Source: §7.10 · p. 17, Table 12 · p. 17, Figure 11 · p. 18