Source-linked AI summary
Federated Learning on the American Science Cloud using APPFL
Zilinghan Li, Abhijit Chunduru, Harinarayan Krishnan, Eric Chagnon, Peter Nugent, Kibaek Kim, Ravi Madduri
TL;DR
AmSC’s existing services do not support model training across organizational boundaries when data cannot be centralized. The paper maps APPFL’s federated orchestration onto AmSC primitives, showing how identity, execution, tracking, and model hosting can support federated computing as a platform service for privacy-constrained collaborations.
Problem
AmSC provides hosting, inference, tracking, and cross-facility execution, but lacks a means to train models across institutions when protected or proprietary data cannot be centralized.
Method
The paper maps APPFL’s federated-learning orchestration onto AmSC primitives for authentication, distributed execution, experiment tracking, and model hosting.
Results
The mapping shows that federated computing can be realized largely by composing existing AmSC services with APPFL rather than building new infrastructure.
Takeaways & Limitations
Offering federated computing as an AmSC service would support privacy-constrained scientific collaborations, including public-private model-building partnerships.
Abstract
from arXiv · showhide
The American Science Cloud (AmSC), established under the Genesis Mission of the U.S. Department of Energy (DOE), aims to integrate DOE high-performance computing systems, experimental facilities, and data resources into a single, coordinated, AI-driven discovery platform. AmSC's early services focus on curated artifacts, such as gated inference access to hosted models, experiment tracking, and function execution across computing facilities. However, what these services lack is a means to train a model across organizational boundaries where data cannot be centralized due to policy, privacy, or scale. This is, by definition, a use case for federated learning (FL) and a growing class of scientific AI. In this paper, we show that this gap can be bridged by deploying the orchestration logic of the Advanced Privacy-Preserving Federated Learning (APPFL) framework as a scalable cloud service on top of the primitives AmSC already provides: project-scoped authentication that supports secure and reliable federation membership, function execution that drives distributed training at each site, experiment tracking that records round-level performance, and finally, the model-hosting and inference infrastructure that can be leveraged to distribute the federated trained models to authorized participants. We argue that offering federated computing as an important AmSC service would unlock privacy-constrained scientific collaborations, enabling public-private partnerships in model building while exercising and enhancing the platform's own federated infrastructure.
I. INTRODUCTION
AmSC provides services for hosting, serving, tracking, and executing AI artifacts, but lacks support for privacy-preserving training across institutions. The paper proposes composing APPFL with existing AmSC primitives to provide federated computing with limited new infrastructure.
- AmSC currently offers model access, cataloging, experiment tracking, cross-facility execution, and shared data catalogs.
- Cross-institutional scientific AI often cannot centralize protected, export-controlled, or proprietary data for model training.
- Federated learning sends models to data, computes local updates in place, and exchanges aggregated parameters or gradients instead of pooling data.
- APPFL supplies the orchestration layer needed to compose AmSC identity, distributed execution, tracking, and model services into an iterative federated training loop.
- The paper maps APPFL onto existing AmSC primitives and identifies further capabilities needed to support federated training.
A. APPFL: Cross-Silo Federated Learning for Science
APPFL is designed for cross-silo scientific federated learning across HPC, institutional, and cloud environments. The paper situates this approach within distributed workflow research and positions its AmSC mapping as a novel platform-level composition.
- A. APPFL: Cross-Silo Federated Learning for Science: APPFL is an open-source framework for cross-silo scientific deployments with separate server, client, communicator, and related training responsibilities.
- A. APPFL: Cross-Silo Federated Learning for Science: Its server supports synchronous, asynchronous, heterogeneity-aware, and queue-aware aggregation strategies for differing facility capabilities and queue times.
- A. APPFL: Cross-Silo Federated Learning for Science: APPFL differs from other cross-silo frameworks in this paper’s design role: AmSC would provide shared identity, transport, orchestration, and monitoring while APPFL supplies federated learning logic.
- B. Related Work: The approach connects federated training to established research on orchestration, fault tolerance, data management, and provenance across administrative domains.
- B. Related Work: The paper claims to be the first work proposing federated learning as a composition of AmSC platform services.
III. FEDERATED COMPUTING AS A COMPOSITION OF AMSC PRIMITIVES
The proposed design separates federated learning logic from generic distributed-systems infrastructure. APPFL components are mapped to AmSC services that can host identity, execution, transport, and monitoring functions.
- The design keeps federation logic such as aggregation, scheduling, and privacy mechanisms in APPFL while integrating surrounding infrastructure through AmSC services.
A. From Inference-as-a-Service to Aggregation-as-a-Service
The paper extends AmSC’s inference-service model toward on-demand federated aggregation. A prototype already lets authorized users launch an aggregation server through a Jupyter notebook for federated experiments.
- AmSC can invoke federated aggregation on demand, analogous to its authenticated model-hosting and inference services.
- A prototype on NERSC’s Spin allows authorized users to launch an aggregation server through a Jupyter notebook for FL clients.
B. Client Execution and Scheduling
AmSC can execute federated client training across DOE and external sites, but heterogeneous facilities and short recurring jobs create scheduling challenges. The paper proposes event-triggered or recurring execution tied to new global model versions.
- Client execution: FL clients run as batch jobs or Globus Compute endpoints that must remain available across rounds.
- Scheduling: AmSC provides reliable scheduling on some DOE leadership computing facilities, but support is not uniform across federation sites.
- Scheduling: Many short, recurring FL jobs make queue delays and straggling clients dominate round-completion time.
- Scheduling: A recurring-reservation or event-triggered primitive could launch local training whenever a new global model version appears.
C. Federated Identity
AmSC’s project-scoped identity tokens can authorize participants in cross-institutional federated experiments. A bridge to Globus federated identity would support incremental adoption by existing APPFL deployments.
- Project-scoped AmSC tokens can gate federation membership and access to deposited or retrieved models.
- Existing APPFL and APPFLx deployments use Globus federated identity, so bridging the two regimes enables incremental platform-identity adoption.
- A writable transport for model updates could avoid separate firewall-traversal and authentication negotiations for each deployment.
D. Privacy Mechanisms and Secure Enclaves
APPFL provides algorithmic privacy mechanisms, while AmSC secure enclaves can add a hardware trust anchor for aggregation. Cross-facility observability remains necessary to reconstruct and diagnose federation-wide execution state.
- Privacy mechanisms: APPFL supplies differential privacy for model updates and secure aggregation that prevents inspection of individual contributions.
- Secure enclaves: Executing aggregation inside an AmSC secure enclave adds a hardware trust anchor when federation members do not fully trust the hosting facility.
- Cross-facility observability: No single party can currently reconstruct round progression, client staleness, communication timings, and failure attribution across facilities.
- Cross-facility observability: A standardized telemetry format should combine round events, client participation, update provenance, communication timings, and facility monitoring.
F. Model Hosting as the Federation’s Shared State
Federated training requires the server’s authoritative global model to persist between rounds as a versioned artifact. Each checkpoint should retain provenance linking it to participating clients and aggregation metadata.
- The server maintains the authoritative global model between FL rounds, with a checkpoint created at each round.
- Provenance records link each model version to participating clients and aggregation metadata.
- AmSC’s Model Access Gateway authenticates inference access but does not yet provide a user-writable artifact store.
IV. CONCLUSION
The paper argues that federated computing should become a first-class AmSC service by composing existing platform primitives with APPFL. This approach identifies the remaining capabilities needed to support privacy-constrained, cross-facility collaborations.
- AmSC can add federated computing without building the entire capability from scratch, because APPFL supplies orchestration logic that composes existing platform primitives.The proposed composition uses AmSC identity, function execution, experiment tracking, and model hosting alongside APPFL’s iterative training loop.
- The remaining platform extensions are aggregation-as-a-service, recurring event-triggered client execution, cross-facility observability, and writable federation-scoped model exchange.
- Closing these gaps would let AmSC serve privacy-constrained collaborations that a host-and-serve catalog cannot support.
- The paper presents its APPFL-to-AmSC mapping and implementation agenda as a starting point for making federated learning a first-class platform service.