GeoGAE: Scalable Graph-Level Autoencoding via Hyperball Cloud Representations

1 INTRODUCTION

• Fixed-size vectors are especially useful when they can reconstruct objects, making generation, transformation, and optimization equivalent operations in R^m.Examples include token, image, and time-series embeddings.Source: §1 · p. 1
• Many graph-generation methods target diverse graphs with desired properties rather than reconstruction of a specific graph, while padded node embeddings impose a graph-size limit.Some generation methods use node or edge embeddings as auxiliary representations that can be padded into a graph-level vector.Source: §1 · p. 1
• Prior reversible graph-vector methods either include node order in the embedding or require complex matching between input and output nodes.The cited order-independent approaches use matching mechanisms.Source: §1 · pp. 1–2
• GeoGAE maps graphs to Euclidean hyperball clouds with a unique node order, then uses Transformer encoding and decoding plus a static rule to recover the discrete graph.The embedding-space dimension can be any natural number, generalizing beyond 2D or 3D molecular graph representations.Source: §1 · p. 2
• GeoGAE achieves state-of-the-art topology reconstruction across diverse benchmarks and scales to large graphs where baselines suffer out-of-memory errors or severe degradation.Its hyperball representation provides canonical deterministic node ordering, and a static geometric rule reconstructs graph topology.Source: §1 · p. 2

2 RELATED WORK

• Standard pooling and virtual-node methods can irreversibly lose precise topological detail, making them poorly suited to exact reconstruction from latent representations.These methods aggregate node features for graph-level representations used in predictive tasks.Source: §2 · p. 2
• GeoGAE replaces lossy aggregation with a reversible hyperball-cloud representation designed to preserve exact topology within a fixed-size vector.Its decoder maps the representation continuously back to a discrete graph.Source: §2 · p. 3
• Unlike graph-generation approaches that map prior noise distributions to graph distributions, GeoGAE targets deterministic, reversible mapping between a unique graph and its latent embedding.The cited approaches include sequential adjacency generation, diffusion, coordinate generation, and continuous-time diffusion over discrete states.Source: §2 · p. 3
• Graph autoencoders with node embeddings encode a matrix of node vectors and decode it back, typically using message-passing GNNs.These architectures follow a VAE-style encoder-decoder structure.Source: §2 · p. 3
• Fixed-size graph autoencoders include padding-based, recursive, and permutation-equivariant designs, but face size, ordering, or node-assignment challenges.GeoGAE uses a typically unique node order and iteratively applies its Transformer decoder to generate arbitrary-size outputs.Source: §2 · p. 3

3.1 FORMAL PROBLEM DESCRIPTION

• The task is to encode variable-size undirected, unweighted graphs without self-loops as fixed-size vectors and reconstruct them end-to-end.The encoder maps graphs to R^m and the decoder maps those vectors back to graphs.Source: §3.1 · p. 3

3.3 HYPERBALL CLOUD

• A graph is represented by hyperballs whose intersections encode adjacency, with variable radii allowing exact representations in low-dimensional spaces.Equal-sized hyperballs guarantee lossless representation in n^2 dimensions, while variable radii can bypass the packing constraint described for a hub topology.Source: §3.3 · p. 4
• Centering, principal-axis alignment, and skewness-based reflection resolution make the geometric representation deterministic and typically unique, except for highly symmetrical topologies.The experiments use the smallest geometric embedding size that achieves 100% F1 on all dataset graphs.Source: §3.3 · pp. 4–5

3.4 NODE ORDERING AND BUNDLES

• Nodes are ordered greedily using spatial proximity, radii, and distance from the cloud center, then grouped into contiguous bundles to reduce Transformer attention cost.When node count is not divisible by bundle size, the final bundle is padded by repeating the last node embedding; smaller datasets use b = 1.Source: §3.4 · p. 5

3.5 ARCHITECTURE

• The encoder maps ordered bundles to tokens and concatenates C learned [CLS] outputs into z; the decoder uses z to regenerate hyperballs and predicts node-level stopping logits.The reconstructed hyperball cloud is converted back to a graph using the static spherical intersection rule.Source: §3.2 · p. 4, §3.5 · pp. 5–6, Figure 2 · p. 6

3.6 MULTI-LAYER TEACHER FORCING

• Multi-layer teacher forcing stacks K weight-sharing decoder instances, supervising later instances on preceding outputs to narrow the training–inference gap.The base instance receives the right-shifted target sequence, while each subsequent instance receives the preceding layer’s output sequence.Source: §3.6 · p. 6

3.7 NODE AND EDGE ATTRIBUTES

• Node attributes widen the encoder input and are reconstructed by a separate MLP, while edge attributes modify attention keys and values and are decoded from connected-node pairs.During training, the edge-attribute decoder uses ground-truth graph edges; the total training loss is specified in Appendix B.Source: §3.7 · pp. 6–7

4 EXPERIMENTAL STUDY

• The study uses seven datasets split into training, validation, and test subsets in a 70:15:15 ratio.The evaluation covers topological and feature reconstruction, large-graph scalability, and architectural ablations.Source: §4 · pp. 7–8

4.1 TRAINING

• Models are evaluated over five independent runs with distinct fixed seeds and pre-computed splits, reporting means and standard deviations.GeoGAE latent sizes are aligned with ReGAE for comparison, and baseline hyperparameters are tuned by dataset-specific random search.Source: §4.1 · p. 8

4.2 RESULTS

• 56.5% and 90.9% F1 on REDDIT-BINARY and COLLAB, respectively, while PIGVAE, ReGAE, and GRALE destabilize or produce trivial predictions.Table 3 reports test-set edge reconstruction F1 weighted by vertex count.Source: §4.2 · p. 8, Table 3 · p. 9
• On AIDS, GeoGAE achieves 80.4% topology F1, 85.0% edge F1, and 0.8 node RMSE, maintaining feature fidelity without sacrificing topology.Table 4 reports topology and feature reconstruction, using F1 for discrete attributes and RMSE for continuous properties.Source: §4.2 · p. 8, Table 4 · p. 9
• 0.00 on QM9 and 0.23 on AIDS are the reported node-count errors, with GeoGAE predicting graph cardinality dynamically.Table 5 reports mean size error; adjacency matrices with incorrect sizes are zero-padded before F1 scoring.Source: Table 5 · p. 9, §4.2 · p. 9

5 CONCLUSIONS

• GeoGAE combines a hyperball-cloud representation and deterministic node ordering with a Transformer to compress variable-sized graphs into fixed-dimensional vectors and reconstruct topology.The conclusion reports that it matches or exceeds state-of-the-art performance on most datasets and scales to large graphs where prior methods fail.Source: §5 · p. 9
• Mapping each discrete graph to a hyperball cloud requires continuous optimization per graph, adding preprocessing overhead that is performed once per dataset.Highly symmetrical graphs can remain sensitive to minor spatial perturbations; scaling also challenges graph-size prediction and topology preservation under node bundling.Source: §5 · p. 10
• The paper states that Proposition 1 has a constructive proof and that the deterministic node-ordering mechanism is detailed in Algorithm 1.Hyperparameters, hardware and software environments, execution-time estimates, and code are documented in the appendices or supplementary materials.Source: REPRODUCIBILITY STATEMENT · p. 10

A PROOF OF PROPOSITION 1

• Each ball has radius 0.5√(2n − 1), and its center is a vector of 0s with exactly n entries equal to 1.The construction uses at most n^2 coordinates.Source: A PROOF OF PROPOSITION 1 · p. 14
• Adjacent nodes' centers share a 1 at a single coordinate, while non-adjacent nodes' centers share none.The construction coordinates the 1s of each new center with those of earlier adjacent balls.Source: A PROOF OF PROPOSITION 1 · p. 14
• Adjacent ball centers are separated by √(2n − 2) and their balls intersect; non-adjacent centers are separated by √(2n) and their balls do not intersect.Source: A PROOF OF PROPOSITION 1 · p. 14

B.1 ORDERING NODES

• The ordering algorithm begins by selecting the node whose ball center is nearest the mean of the centers.The embeddings are normalized so that their mean, x̄, is zero.Source: §B.1 · p. 14
• The ordering algorithm returns the geometrically embedded nodes in the selected sequence.The algorithm's input consists of geometric embeddings with each node's ball center and radius.Source: §B.1 · p. 14

B.2 FORMAL MECHANICS OF NODE BUNDLING

• When node count is not divisible by bundle size b, copies of the final node pad the sequence to a multiple of b.The padded sequence is reshaped into bundle blocks and projected by an MLP into encoder tokens.Source: §B.2 · p. 14
• During decoding, each token is mapped to b geometric embeddings and b stopping logits, which truncate padded nodes in the final bundle.Source: §B.2 · p. 14

B.3 MASKING ENCODER TOKENS

• Distinct [CLS] tokens cannot attend to one another, while each attends to every vertex and vertices attend to one another.The mask retains self-attention on the diagonal; separate class tokens are thereby encouraged to capture different graph information.Source: §B.3 · pp. 14–15, Figure 4 · p. 15

B.4 OPTIMIZATION OBJECTIVE

• The total objective combines geometric and hyperball reconstruction, end-of-sequence prediction, and optional KL-divergence and node- and edge-feature reconstruction losses.KL loss is omitted when VAE is disabled, and feature losses are omitted when feature decoding is disabled.Source: §B.4 · pp. 15–16
• Losses are accumulated over all K decoder layers to mitigate exposure bias.The geometric node embedding is predicted at each decoder layer, and its reconstruction is trained alongside the other losses.Source: §B.4 · p. 15
• The decoder predicts graph size through stop-token logits, trained as binary targets that switch to 1 at or beyond the graph’s actual size.The stopping loss uses focal loss to address the imbalance between mostly non-stop tokens and stop tokens.Source: §B.4 · pp. 15–16

• The study uses MUTAG, AIDS, IMDB-BINARY, QM9, SYNTHETIC NEW, COLLAB, and REDDIT-BINARY.These are the datasets identified for the experimental study.Source: C BENCHMARK DATASETS · p. 16

D HARDWARE AND COMPUTATIONAL RUNTIME

• Experiments primarily used an HPC supercomputer equipped with NVIDIA A100 80GB GPUs, with secondary environments used to distribute work under cluster scheduling and queue constraints.The study reports approximately 32,000 GPU-hours across preliminary exploration, tuning, baseline evaluation, and ablation work.Source: D HARDWARE AND COMPUTATIONAL RUNTIME · pp. 16, 18
• The benchmark set covers molecular graphs, movie collaboration networks, and Reddit discussion graphs, alongside datasets with multiple continuous regression targets.QM9 supplies optimized 3-D coordinates and quantum-chemical properties, while MUTAG and AIDS include atom and bond information.Source: D HARDWARE AND COMPUTATIONAL RUNTIME · p. 17

E SETUP OF GRALE, PIGVAE, REGAE

• Baseline methods shared dataset-reading, splitting, training-loop, metric, and reporting code, with hyperparameters tuned separately by algorithm and dataset except for COLLAB and REDDIT-BIN.For those two datasets, parameters came from original papers or were tuned manually because of limited computational capacity.Source: E SETUP OF GRALE, PIGVAE, REGAE · p. 18

Multi-layer Teacher Forcing and Exposure Bias

• On AIDS, reducing decoder iterations from K = 3 to K = 1 lowers topological F1 from 84.12% to 72.59% and raises average size difference from 0.23 to 4.67.Performance and size prediction stabilize at K = 2; Figure 5 shows the effects across decoder counts.Source: F ABLATION STUDY · p. 19, Figure 5 · p. 21

Impact of Variational Regularization (VAE)

• Enabling VAE regularization lowers QM9 topological F1 from 99.38% to 95.88%, while the deterministic model outperforms the VAE variant across all three tested datasets.The paper frames this as a trade-off between exact deterministic reconstruction and a smooth, sampleable latent space.Source: F ABLATION STUDY · pp. 19–20

Influence of Node and Edge Features on Topology

• Providing node and edge attributes without feature decoding improves topological F1, including MUTAG from 61.29% to 63.61% and AIDS from 84.12% to 86.00%.The authors interpret attributes as auxiliary signals that implicitly regularize the topological representation.Source: F ABLATION STUDY · p. 20

Impact of Bundle Size

• Increasing bundle size from b = 1 to b = 8 lowers topological F1 from 84.1% to 79.5%, while size error is lowest at b = 2 (0.09 versus 0.23 at b = 1).The study retains b = 1 as default for topology fidelity and reserves larger bundles as a scaling trade-off; Figure 6 plots both outcomes.Source: F ABLATION STUDY · p. 20, Figure 6 · p. 22