PaperlayerView exact revision on arXiv

Source-linked AI summary

FlatLand: Personalized Graph Federated Learning via Tailored Lorentz Space

Jiahong Liu, Ram Samarth B B, Xinyu Fu, Menglin Yang, Weixi Zhang, Rex Ying, Irwin King

arXiv:2608.21096v1cs.LG

TL;DR

Graph federated learning must handle structurally heterogeneous clients, while existing personalized approaches overlook their differing geometries. FlatLand assigns clients tailored Lorentz spaces and decouples time-like personalized parameters from space-like shared parameters. Experiments report improved personalization, especially with compact representations, while the authors note that hyperbolic geometry is not universally optimal.

  • Problem

    Graph federated learning faces structural heterogeneity, and existing personalized methods overlook the intrinsic geometric properties of diverse graph structures.

  • Method

    FlatLand embeds clients in tailored Lorentz spaces and separates personalized time-like parameters from shared space-like parameters for aggregation.

  • Results

    FlatLand achieves superior performance to its Euclidean counterpart, particularly in low-dimensional settings, and improves personalization when compact representations are required.

  • Takeaways & Limitations

    Non-Euclidean geometry is presented as a promising direction for personalized federated learning on heterogeneous graph data.

  • Takeaways & Limitations

    Hyperbolic geometry is not universally optimal because some clients may be closer to Euclidean or positively curved geometries.

Abstract

from arXiv · show

Federated learning enables privacy-preserving collaborative training, but highly heterogeneous client data remain challenging, especially in graph federated learning where clients possess structurally diverse graphs. Existing personalized federated learning (PFL) methods ignore the intrinsic geometric properties of diverse graph structures. We propose FlatLand, a novel personalized federated learning method that embeds different clients' data in tailored Lorentz space of hyperbolic geometry. Our key insight is that hyperbolic geometry naturally accommodates the intrinsic negative curvature prevalent in real-world graphs, while the time-like dimension in Lorentz space provides a principled way to encode client-specific heterogeneity. We develop a parameter decoupling strategy that separates heterogeneous information (captured in time-like parameters) from common knowledge (preserved in space-like parameters), enabling direct aggregation without requiring client similarity estimation and extra calculation modules. Empirical results on diverse federated graph learning tasks demonstrate that FlatLand achieves superior performance, particularly in low-dimensional settings.

1. Introduction

Graph federated learning faces structural heterogeneity that Euclidean personalized methods model imperfectly. FlatLand uses tailored Lorentz spaces and decouples heterogeneous from shared parameters for direct aggregation.

  • Graph federated learning amplifies data heterogeneity because clients have structurally diverse graph topologies.In severe cases, federated learning may underperform local training.
  • Existing graph PFL methods often rely on Euclidean geometry and additional mechanisms such as similarity estimation or auxiliary modules.Simple parameter disentanglement can fail, while more advanced mechanisms incur computational overhead.
  • Observations: Client graphs predominantly exhibit negative Ricci curvature, while curvature varies substantially across clients.These observations indicate hyperbolic structure and intrinsic geometric heterogeneity beyond simple statistical differences.
  • Lorentz Geometry: Lorentz geometry offers low-distortion representation and a time-like dimension for encoding client-specific geometric heterogeneity.Tailored curvature can better accommodate clients with different graph properties.
  • FlatLand: FlatLand embeds client data in tailored Lorentz spaces and designates heterogeneity-related parameters as personalized while aggregating shared parameters.The strategy avoids auxiliary modules and client-similarity estimation.
  • Results: FlatLand outperforms its Euclidean counterpart, particularly in low-dimensional settings relevant to communication-efficient federated learning.

2. Related Work

Prior personalized graph federated learning methods address heterogeneity through clustering, personalized modules, or related mechanisms. FlatLand instead personalizes client geometry and separates geometric information from shared knowledge during aggregation.

  • Personalized graph federated learning commonly clusters clients by gradients or introduces additional personalized modules.
  • Existing hyperbolic federated methods use hyperbolic distances, prototypes, or hyperbolic GNNs within federated pipelines.
  • FlatLand assigns personalized client geometries and separates client-specific geometric information from shared knowledge during aggregation.This enables direct aggregation without client clustering or extra similarity estimation.

3. Preliminaries

The preliminaries define the Lorentz hyperbolic model, its scale-dependent geometry, and the personalized federated learning setting. They also motivate fully Lorentz networks and identify the optimization goals for heterogeneous clients.

  • Lorentz Model of Hyperbolic Geometry: The Lorentz model represents d-dimensional hyperbolic space as a manifold embedded in flat Minkowski space of dimension d+1.
  • Lorentz Model of Hyperbolic Geometry: A Lorentz space with scale parameter K has constant sectional curvature −1/K, so different K values induce different hyperbolic geometries.
  • Lorentz Model of Hyperbolic Geometry: Lorentz vectors contain one time-like coordinate and d space-like coordinates subject to the Lorentzian constraint and positive-time condition.
  • Lorentz Model of Hyperbolic Geometry: Euclidean inputs are mapped into Lorentz space, and different K values map them to different Lorentz surfaces.
  • Fully Lorentz Neural Networks: Fully Lorentz networks reduce the need for space projections and use Lorentz transformations to improve computational efficiency and parameter interpretability.
  • Problem Statement: PFL optimizes client models with specific and shared parameters under heterogeneous client distributions.The stated goals include representing local properties, reflecting heterogeneity, and communicating shared information without additional computations.

4. Motivation and Insights

Client graphs show predominantly negative and client-varying curvature, motivating tailored hyperbolic geometries. Lorentz space additionally separates heterogeneity-bearing time-like information from aggregatable space-like information.

  • Geometric Evidence: Ricci curvature distinguishes hyperbolic, flat, and spherical geometries while quantifying curvature strength.
  • Geometric Evidence: Client graphs mostly have negative curvature, and their curvature varies across clients, revealing non-Euclidean geometric heterogeneity.
  • Why Lorentz Space: Lorentz geometry is theoretically motivated as a way to faithfully capture non-Euclidean client data properties.
  • Why Lorentz Space: The Lorentz model's time–space decomposition supports personalized time-like parameters and aggregation of space-like parameters.
  • Tailored Curvature: Different client curvatures require tailored Lorentz scale parameters because one K cannot simultaneously provide small distortion for all clients.
  • Heterogeneity Encoding: The Lorentz time-like component retains mutual information with client identity, whereas the space-like component carries no additional client-specific information.
  • FlatLand Intuition: FlatLand's additional time-like dimension metaphorically represents client-specific curved worlds while the flat space supports exchanging common information.

5. The FlatLand Framework

FlatLand assigns each client a tailored, learnable Lorentz space and decouples client-specific geometric information from transferable parameters. The server aggregates only space-like shared parameters while retaining time-like and curvature-related personalization locally.

  • Framework overview: FlatLand assigns each client a tailored Lorentz space and uses parameter decoupling to mitigate heterogeneity without clustering or auxiliary modules.The method is built directly upon FedAvg.
  • Learnable curvature initialization: Each client learns a positive Lorentz scale K_c, while Forman-Ricci curvature provides a lightweight structure-aware initialization prior.The scale is updated during local training rather than fixed from pre-training statistics.
  • Parameter decoupling strategy: The transformation parameters split into personalized time-like parameters and shared space-like parameters.The time-like contribution is controlled by m^(l), whereas M^(l) operates on transferable space-like dimensions.
  • Decoupled aggregation: Only space-like parameters are uploaded and averaged; personalized parameters and each client’s Lorentz scale remain local.The aggregated space-like parameters are combined locally with m^(l) and K_c.
  • Decoupled aggregation: Keeping time-like parameters and K_c local avoids mixing scale-sensitive quantities from different client-specific Lorentz spaces.The resulting transformation still maps outputs into the same client-specific Lorentz space L_Kc.

6. Analysis

FlatLand’s analysis examines correctness, convergence, and efficiency, showing that server aggregation preserves client-specific Lorentz geometry while retaining FedAvg’s convergence rate and near-FedAvg overhead.

  • Analysis overview: FlatLand’s analysis evaluates correctness, convergence relative to FedAvg, and computational efficiency.The section also reports node-classification comparisons in Tables 1 and 2 and dimensionality experiments in Figure 4.
  • Experimental evaluation: The node-classification evaluation covers real-world and heterophilic datasets with varying client counts, reporting mean and standard deviation over five trials.Statistically significant results are highlighted when p < 0.05.
  • Correctness: After shared-parameter aggregation, updated transformations still map client vectors into the same client-specific Lorentz space with the same sectional curvature.This property is stated by Proposition 6.1.
  • Convergence: O(1/T) remains the convergence rate under the standard FedAvg scheme because FlatLand changes only the geometric parameterization of local updates.Server aggregation continues to average shared space-like parameters without client-similarity estimation.
  • Efficiency: FlatLand adds minimal operations, including an O(1) exponential map and curvature estimation that can be mitigated through pre-computation.The analysis characterizes the overhead as comparable to FedAvg while noting reduced communication overhead.

7. Experiments

Experiments evaluate FlatLand across node and graph classification, low-dimensional embeddings, partial participation, and component ablations. FlatLand generally performs competitively or best, with gains linked to tailored curvature and time-like parameter decoupling.

  • Main Experimental Results: FlatLand achieves the best or competitive performance across heterogeneous node-classification datasets, with especially clear gains on CiteSeer, ogbn-arxiv, and Minesweeper.Euclidean FL baselines can underperform local training under strong heterogeneity, while FedHGCN also struggles without explicit client-specific geometry and parameter decoupling.
  • Main Experimental Results: FlatLand significantly enhances Lorentzian graph classification, outperforming Euclidean baselines across single- and multiple-dataset settings.Local Lorentz training helps in CHEM but not necessarily in BIO-CHEM-SN, indicating that client datasets do not uniformly exhibit prominent hyperbolic features.
  • Varying Embedding Dimensions: FlatLand consistently outperforms competing methods as embedding dimensions decrease from 64 to 4, while its advantage over baselines becomes increasingly significant.Reducing dimensionality has relatively small impact on hyperbolic methods compared with Euclidean counterparts, supporting compact representations for communication-efficient federated learning.
  • Partial Client Participation: At 10% client participation, FlatLand reaches 81.82% accuracy versus 18.14% for FedAvg, and it consistently outperforms all baselines across participation rates.The authors attribute this robustness to separating personalized time-like parameters from shared space-like parameters.
  • Ablation Study: Removing tailored curvature lowers performance, whereas learning client-specific Lorentz scale parameters lets FlatLand surpass the Local (L) baseline.The ablation sets K = 1 for all clients without tailored curvature, showing that adaptive geometry matters for federated transfer.
  • Ablation Study: Removing time-like parameter decoupling causes fluctuations and degradation, while removing Lorentz geometry makes the Euclidean variant underperform.Client-level results show FlatLand improves most clients, especially weaker clients such as c_8 in CiteSeer; without decoupling, performance deteriorates for clients such as c_7.

8. Conclusions

FlatLand personalizes federated graph learning by modeling heterogeneous client distributions in tailored Lorentz spaces and separating personalized from shared parameters. The paper reports improved personalization, especially with compact representations, while identifying mixed-curvature modeling as future work.

  • FlatLand assigns clients tailored Lorentz spaces and learns client-specific Lorentz scale parameters to preserve local geometric structure.
  • Its parameter decoupling strategy separates personalized time-like parameters from shared space-like parameters for common-information aggregation.
  • Experiments on node- and graph-level benchmarks show improved personalization, particularly when compact representations are required.
  • Hyperbolic geometry is not universally optimal because some clients may be closer to Euclidean or positively curved geometries.Future work proposes mixed-curvature spaces, richer Lorentz backbones, and broader data modalities beyond graph benchmarks.

Impact Statement

The paper frames personalized federated graph learning as a response to heterogeneous client requirements and structurally diverse graphs. It connects this challenge to non-Euclidean geometry, privacy-preserving aggregation, and Lorentz-space representations.

  • Personalized federated learning learns client-adapted models because conventional federated learning may not generalize well to every client.
  • Graph personalization methods use gradient clustering, additional personalized modules, or client-similarity estimation to address client-specific graph structure.
  • FlatLand differs from prior hyperbolic federated methods by adaptively modeling client geometries with different curvatures and separating geometric information from shared knowledge.
  • Hyperbolic space has constant negative curvature, and the Lorentz manifold is a numerically stable hyperboloid representation of that geometry.
  • The Lorentz manifold is defined as a hyperboloid with constant curvature −1/K, where K is a positive Lorentz scale parameter.
  • Forman-Ricci curvature is computed before training, making curvature estimation a preprocessing cost rather than part of the per-round federated loop.

C.3. Derivation of Parameters Disentanglement

FlatLand’s disentanglement strategy treats the time-like dimension as the carrier of client heterogeneity and the space-like dimensions as the platform for shared information. Its aggregation complexity matches FedAvg at equal hidden dimension, while storage increases by O(d + 1) per client.

  • “Flatland” denotes the space-like dimensions where common information is exchanged and integrated, while the time-like component carries heterogeneous information.
  • The key component capturing data heterogeneity at layer l is the time-like dimension of the hyperbolic embedding, not the original input.
  • Time-like parameters remain aligned with heterogeneity information across layers, supporting the proposed decoupling strategy.
  • FlatLand and FedAvg have the same aggregation time complexity when their hidden embedding dimensions are equal.FlatLand aggregates only shared parameters while maintaining personalized parameters.
  • FlatLand requires extra O(d + 1) storage per client because of the additional time-like dimension and Lorentz scale parameter.The passage states that the increase is small because d is typically small.
  • A single Lorentz scale parameter cannot make all clients’ distortions simultaneously small when their average Forman-Ricci curvatures differ.

D.2. Proof for Theorem 4.3

Under isotropy and scale-conditioned assumptions, the proof identifies the time-like coordinate as the carrier of client-specific geometric variation. It shows that differing client-conditioned time-like distributions imply positive mutual information with client identity, while angular information is client-independent.

  • Under the isotropy assumption, the angular component u is independent of client identity and satisfies I(u; C) = 0.
  • If clients induce different pushforward laws for the time-like coordinate, that coordinate is not independent of client identity and satisfies I(x_t; C) > 0.
  • The conditional mutual information satisfies I((x_t, x_s); C | K_c) = I(x_t; C | K_c) under the stated assumptions.
  • The theorem combines these results to characterize angular independence, time-like client information, and conditional information equivalence.
  • The theoretical claims apply under isotropy and scale-conditioned assumptions and do not assert strict client invariance of space-like coordinates for every trained network or graph distribution.

D.4. Convergence Analysis

FlatLand retains FedAvg-level convergence behavior while accommodating heterogeneous data through geometric parameter decoupling. Without regularization, its rate is consistent with FedAvg; with regularization, the bound reflects data heterogeneity similarly to FedProx.

  • FlatLand achieves a convergence rate of O(1/T) without regularization, consistent with FedAvg.
  • With regularization, FlatLand’s convergence rate is bounded by a constant reflecting data heterogeneity, analogous to FedProx’s guarantees.
  • Regularization is applied only to the shared block, constraining locally updated shared parameters toward server-shared parameters.
  • The method averages local updates of shared parameters while keeping client-specific parameters local.
  • The analysis reports that additional local operations are offset by reduced communication overhead and enhanced Lorentz-space representation capability.
  • Lorentz transformations preserve the Lorentzian norm constraint, so aggregation keeps representations on each client’s specific hyperboloid despite differing scale parameters.

E.1. Datasets

FlatLand is evaluated across federated node- and graph-classification benchmarks spanning citation, product, molecular, bioinformatics, and social-network graphs. The evaluation compares geometric personalization against Euclidean, hyperbolic, local-training, and ablated alternatives, with results emphasizing learnable client-specific curvature and parameter decoupling.

  • Datasets: The node-classification evaluation uses Cora, CiteSeer, ogbn-arxiv, and Photo, partitioning each graph into client subgraphs with METIS.
  • Datasets: The graph-classification evaluation covers 13 non-IID datasets from molecular, bioinformatics, and social-network domains.
  • Baselines: The comparison includes FedAvg, FedProx, FedPer, GCFL, FedHGCN, and Euclidean and Lorentz local-training baselines.
  • Ablations: FlatLand’s ablations attribute gains to Lorentz representations, separating personalized time-like from shared space-like parameters, and client-specific curvature.
  • Convergence and design: FlatLand reaches higher final performance with competitive convergence speed while retaining simple shared-parameter averaging without client-similarity estimation or clustering optimization.
  • Ablations: Learnable curvature outperforms fixed curvature by 1–3% across datasets, while curvature initialization has only a marginal effect.
Loading 2608.21096v1…