Source-linked AI summary
Design and Operation of a Federated GPU Cluster for Digital Humanities within DHinfra.at
Florian Atzenhofer-Baumgartner, David Fleischhacker, Max Resch, Lukas Waldhofer, Michael Otto
TL;DR
The paper addresses how non-HPC Digital Humanities researchers can use GPU infrastructure through accessible, quota-aware interfaces rather than conventional batch workflows. It documents a federated cluster control plane, inference services, operations, and a limited-budget implementation playbook. The resulting platform provides always-on model endpoints and tested self-service workflows, while leaving quota reconciliation across institutions and heterogeneous-GPU caching as open questions.
Problem
GPU-dependent researchers need model endpoints, GPU-attached notebooks, and visible quotas, but batch-oriented HPC serves these expectations only partly.
Method
The paper documents a federated GPU cluster whose control plane links identity federation, project quotas, approval workflows, model access, and HTTPS inference services.
Results
The platform provides always-on OpenAI-compatible model endpoints without requiring partner researchers to hold cluster accounts, and its control plane is testable through a local Docker Compose testbed.
Takeaways & Limitations
A domain-specific GPU cluster can prioritise inference endpoints, quotas, and self-service interfaces for non-HPC users while retaining interactive and SSH access.
Takeaways & Limitations
Quota reconciliation across institutions and caching on heterogeneous GPUs remain unresolved platform-specific questions.
Abstract
from arXiv · showhide
We describe the design, implementation, and operation of a small federated GPU cluster built for Digital Humanities (DH) research within the Austrian DHinfra.at project. The system spans two university sites, brokers logins from a national identity federation, and exposes compute through three interfaces: interactive notebooks, SSH, and an OpenAI-compatible inference API. The core of the platform is a control plane that maps federated identity, per-project quotas, and a model catalogue onto self-service interfaces. We document this control plane in detail, including the request-and-approval queue that mediates every privileged cluster effect and the two-tier model-serving setup, which combines always-on services with on-demand model swapping and lets researchers consume large language models over HTTPS without holding a cluster account. We report the constraints that shaped the build: a fixed power budget, EU-wide procurement, a team of about 1.5 full-time equivalents, and the relationship between prioritized access and utilization. We also record the main build decisions against the alternatives we researched and did not adopt. The platform is released under the Apache License 2.0. We close with a playbook for teams building a domain-specific GPU cluster on a limited budget. This is a living technical report, versioned and updated as the platform evolves.
1 Introduction
DHinfra.at addresses the mismatch between GPU-dependent DH workflows and batch-oriented HPC by documenting a reproducible federated cluster and its control plane. The report also covers operations, alternatives, licensing, and a practical build playbook.
- GPU-dependent research increasingly requires model endpoints, GPU-attached notebooks, and visible quotas, while batch-oriented HPC serves these expectations only partly.
- The report documents a reproducible federated GPU-cluster architecture that brokers national-federation logins for Digital Humanities research.
- Its control plane combines a request-and-approval queue with an inference gateway for administering cluster access and model use.
- The report covers procurement, staffing, prioritized access, utilization, researched alternatives, and an Apache-licensed platform release.
- It concludes with a playbook for building a domain-specific GPU cluster for LLM workloads on a limited budget.
2 Background and Related Work
The paper situates its platform between accessible HPC, multi-tenant AI infrastructure, and LLM-serving systems. Its specific contribution is a policy layer tailored to small, federated DH and GLAM-oriented deployments.
- Interactive and accessible HPC: The platform extends interactive HPC by binding notebook and compute access to federated identity and per-project quotas.
- AI/ML on shared and multi-tenant platforms: It adapts scoped credentials, per-tenant accounting, and heterogeneous hardware from shared AI/ML platforms to a smaller operational scale.
- LLM serving and MLOps: Its distinctive contribution around mature LLM-serving components is a policy layer with token budgets, model access-control lists, and an OpenAI-compatible gateway.
- HPC in the humanities: The design targets humanities workloads whose organisational, cultural, and exploratory characteristics fit batch allocation poorly.
3 DHinfra.at: Context and Requirements
DHinfra.at is a multi-institution Austrian infrastructure project whose GPU/HPC component required governance, training, and hardware/software preparation before service integration. Its requirements prioritise federated access, self-service controls, and HTTPS model delivery.
- The project progressed through consortium formation, requirements and training, procurement preparation, FOSS and test-environment work, and operational integration toward a mid-2026 launch.
- The setting differs from conventional HPC through three recurring requirements: federated identity, self-service configuration, and models served over HTTPS.
- Users authenticate with existing ACOnet institutional credentials without a separate account application.
- A console exposes quotas, keys, and project setup, with prioritized access and monitoring as the operational default.
- The platform serves inference endpoints to clients that never authenticate directly to the cluster.
4 Hardware and Network
The deployment combines tendered, heterogeneous accelerators across two sites with high-speed inter-site connectivity and power-aware constraints. Hardware diversity supports workload specialization but complicates placement and accounting.
- EU-wide public tendering delivered current-generation accelerators at reasonable cost, with upgrade pathways.
- H200 NVL nodes support large-context and concurrent inference, while L40S and RTX PRO 6000 cards handle cheaper exploratory and serving workloads.
- Table 1 records the current two-site GPU inventory, including seven additional RTX PRO 6000 Blackwell cards ordered for Graz and differing MIG support.
- Heterogeneous memory means a model fitting one card may not fit another, making scheduling, placement, and cost accounting more complex.
- Deployment was also shaped by a fixed power budget, 400 Gbit fabric, an ACOnet high-speed inter-site link, and emergency-power signalling.
5 Software Stack
The software stack combines open-source identity, scheduling, container, and service components through a FastAPI control plane that exposes self-service research access. Its control plane separates authority, approval, budgeting, and model-serving concerns while offering both interactive compute and HTTPS-based inference.
- 5.3 Interactive access: Researchers access the platform through JupyterHub-backed Slurm notebooks, SSH over VPN, or an inference API for users who do not work from a terminal.JupyterHub sessions use Slurm allocations, while the inference API provides a separate access path to model services.
- 5.4 The control plane: The FastAPI console maps identity, quotas, and a model catalogue onto self-service interfaces, with Postgres storing budgets, models, usage, keys, queues, audits, and identity cache data.Redis provides authentication caching, budget hot-counters, and login rate limiting; scheduled work is limited to identity reconciliation and project expiry.
- 5.1 Identity and access: Authentik owns federated identity, Slurm owns job resources and accounting, and the control plane maintains derived caches rather than duplicating those authorities.Identity is synchronized through webhooks and periodic reconciliation, while SlurmDBD remains authoritative for job history.
- 5.4 The control plane: Privileged changes are recorded as queued requests and applied by a separate worker, keeping cluster credentials and privileged side effects out of the public-facing service.The split also records approvers and reasons and makes the cluster-side worker independently testable and rate-limited.
- 5.4 The control plane: The inference gateway authenticates bearer keys, checks model ACLs and group token budgets, limits per-model concurrency, streams requests, and records usage on completion.Budgets apply to project groups in daily and weekly windows, and model-specific input/output cost factors support accounting across models with different serving costs.
- 5.5 Model serving: Model serving uses always-on resident models for common workloads and an on-demand tier that loads infrequently used models when requested and unloads them when idle.The router holds no GPUs; Slurm jobs launch vLLM or llama.cpp engines, so GPUs are held only while a model is loaded.
6 Operations
Operations balance accessibility and maintainability against constrained staffing, procurement overhead, and lower peak utilization. Early operation showed that prioritized access exposed inefficient resource use, while training and detailed specifications reduced downtime and external dependence.
- A 1.5-FTE core team operates dedicated, independently networked clusters with simplicity, security, and maintainability as priorities.
- Prioritized access and utilization: Prioritized access exposes optimization gaps and inefficient resource use, so operators monitor usage and contact affected users.
- Procurement and tendering: EU-wide tendering delivered state-of-the-art hardware at reasonable cost but imposed substantial overhead and sometimes required a second tender round.
- Procurement and tendering: Teams should vet bidders, maintain direct contact, involve legal support early, and expect to renegotiate specifications and governance after acceptance.
- Concurrent development: Concurrent hardware, software, technical, policy, and partner development made staff training a major cost, but investment built expertise, reduced downtime, and lowered reliance on external experts.
7 Use Cases
The platform supports an iterative DH workflow from interactive model exploration through evidence-based selection, deployment, remote HTTPS use, and downstream research outputs. Its scheduling and serving policies must accommodate workloads ranging from text and image processing to VR and adaptive tutoring.
- Researchers explore model architectures, sizes, quantizations, and prompts interactively in JupyterHub before selecting a model based on domain-task evidence.
- Selected models can be deployed as APIs or batch jobs, including HTTPS inference calls from projects running on another institution’s virtual machine.
- Secondary storage and custom virtual machines support projects, whose results then feed downstream research outputs.
- Pilots span automatic text recognition, image and video classification, VR applications, adaptive AI tutoring, and varied humanities datasets.
8 A Playbook for a Domain-Specific GPU Cluster
The playbook recommends a simple, self-service, policy-driven GPU platform whose complexity grows with workload needs. Its core choices are endpoint-oriented delivery, centralized identity, mediated privileged effects, project metering, tiered serving, explicit utilization targets, and testable control-plane changes.
- For non-HPC-native DH and GLAM users, make a quota-managed inference endpoint the primary deliverable and place the cluster behind it.
- Use one identity authority with reconciled caches so federated ownership remains clear and the system stays debuggable.
- Separate recorded intent from cluster effect: approve requests, then let a credential-holding worker apply changes for auditability and least privilege.
- Meter project consumption with token budgets, per-model cost factors, frozen window caps, and logged overrides.
- Combine always-on batched serving for steady demand with on-demand model swapping for the long tail, while keeping routing and metering separate from GPU execution.
- Mixed GPUs can reduce cost, but placement and accounting must track each card’s capabilities and the model it serves.
- Accept lower utilization when accessibility and fast iteration are priorities, state the target explicitly, and support it with monitoring and user contact.
- Budget staff and specifications because procurement needs time and legal support, while detailed tender work also builds operational knowledge and may require later renegotiation.
9 Open-source release
The platform is released under Apache License 2.0 as a public, self-hostable integration for small and mid-size domain-specific deployments. Its distinctive contribution is the open control plane binding identity federation, quota metering, model access control, and inference services without per-seat or per-GPU licensing.
- Apache License 2.0 release includes cluster-side configuration and operator runbooks for public use.
- The stack fills a gap for self-hostable domain-specific deployments by enabling identity-federated, quota-metered inference without per-seat or per-GPU licensing.
- The platform’s authored contribution is the control plane and integration layer connecting open-source components including Slurm, Authentik, Enroot/Pyxis, vLLM, and Traefik.
10 Open Questions and Outlook
The platform leaves open questions about reconciling federated identity with institutional quotas, caching across heterogeneous GPUs, and scheduling interactive inference alongside batch workloads. Its outlook expands consortium access and integration while positioning small, domain-specific HPC as complementary to general HPC.
- 10 Open Questions and Outlook: Federated identity and institutional quota accounting remain unresolved as the consortium widens across institutions.The platform derives per-project budgets from one identity authority, but institutions may also want to account for their own members.
- 10 Open Questions and Outlook: Model caching remains an open problem because heterogeneous GPU memory profiles change the effective capacity of KV and weight caches.The report calls for caching and cache-admission strategies that account for placement-dependent capacity.
- 10 Open Questions and Outlook: Mixed scheduling must reconcile latency-sensitive inference with throughput-oriented batch work competing for the same accelerators.The interaction between gateway admission control and Slurm scheduling remains unresolved despite fair-share accounting.
- 10 Open Questions and Outlook: Current work targets full CLARIAH-AT access, deeper Krems integration, and a public launch in the second half of 2026.Planned integration includes distributed storage, coordinated tracked GPU use, and continued CLI- and API-first interfaces.
11 Conclusion
The conclusion identifies the control plane as the platform’s main engineering contribution rather than the standard accelerator and network hardware. It emphasizes reproducibility alongside explicit trade-offs in accessibility, utilization, and staffing, while treating the report as a living document.
- 11 Conclusion: The control plane makes standard cluster hardware usable to non-HPC researchers by federating identity, metering projects, and serving models over HTTPS.It separates recording a request from applying its privileged cluster effect.
- 11 Conclusion: The report documents the control-plane layer in enough detail to reproduce and records the operational trade-offs a small team must decide explicitly.The stated trade-offs concern accessibility, utilization, and staffing.
- 11 Conclusion: The platform is maintained as a living document that will be updated as it moves toward public operation.The report’s operational account is therefore intended to evolve with deployment.