Source-linked AI summary
Materials Cloud, a platform for open computational science
Leopold Talirz, Snehal Kumbhar, Elsa Passaro, Aliaksandr V. Yakutovich, Valeria Granata, Fernando Gargiulo, Marco Borelli, Martin Uhrin, Sebastiaan P. Huber, Spyros Zoupanos, Carl S. Adorf, Casper W. Andersen, Ole Schütt, Carlo A. Pignedoli, Daniele Passerone, Joost VandeVondele, Thomas C. Schulthess, Berend Smit, Giovanni Pizzi, Nicola Marzari
TL;DR
Computational research needs practical ways to share data, software, workflows, and infrastructure so results can be assessed, verified, and reproduced. Materials Cloud addresses this need through an open platform built around FAIR resource sharing and AiiDA provenance graphs, combining archival, dissemination, simulation, analytics, and educational services. The platform supports browsing and reuse of computational results while remaining accessible through web-based and turnkey tools.
Problem
Open computational science needs sustainable platforms that make data, software, infrastructure, and complex simulation workflows findable, accessible, interoperable, reusable, and reproducible.
Method
Materials Cloud combines archival, curated-data, simulation, analytics, and educational services with AiiDA-based recording of computational workflow provenance.
Results
Materials Cloud provides interconnected, FAIR and reproducible workflow outcomes alongside turnkey workflows accessible through web-based services.
Takeaways & Limitations
Researchers can browse, download, reproduce, and extend shared computational results, while non-specialists can use selected workflows through accessible interfaces.
Takeaways & Limitations
Submitting new tools and interactive visualisations still requires technical expertise, and a common information-exchange language between open-science platforms remains unresolved.
Abstract
from arXiv · showhide
Materials Cloud is a platform designed to enable open and seamless sharing of resources for computational science, driven by applications in materials modelling. It hosts 1) archival and dissemination services for raw and curated data, together with their provenance graph, 2) modelling services and virtual machines, 3) tools for data analytics, and pre-/post-processing, and 4) educational materials. Data is citable and archived persistently, providing a comprehensive embodiment of the FAIR principles that extends to computational workflows. Materials Cloud leverages the AiiDA framework to record the provenance of entire simulation pipelines (calculations performed, codes used, data generated) in the form of graphs that allow to retrace and reproduce any computed result. When an AiiDA database is shared on Materials Cloud, peers can browse the interconnected record of simulations, download individual files or the full database, and start their research from the results of the original authors. The infrastructure is agnostic to the specific simulation codes used and can support diverse applications in computational science that transcend its initial materials domain.
Introduction
Materials Cloud is designed to translate open-science and FAIR principles into a practical platform for sharing, managing, and reusing computational research. It addresses the complexity of computational materials workflows by combining open infrastructure, workflow management, and accessible services.
- Open access to data, software, and infrastructure supports scientific assessment, verification, and reproduction.
- Materials Cloud is designed as an open-science platform for using existing data, submitting content, and launching simulations with limited technical requirements.
- Computational materials simulations require flexible workflow design and recording because they combine multiple scales, iterative processing, and large numbers of candidate calculations.
- The platform aims to support open simulation codes, workflow management, turnkey solutions, and FAIR sharing of data and workflows.
- Materials Cloud implements this vision as an ecosystem spanning the research lifecycle, from learning and simulation to publishing curated results for subsequent research.
Results
Materials Cloud organizes research resources across five sections supporting the research life cycle, from learning and simulation to publishing curated, FAIR and reproducible results. Its ARCHIVE, DISCOVER, and EXPLORE services connect persistent data records, interactive visualizations, and AiiDA provenance graphs, while WORK, LEARN, and Quantum Mobile support simulation access, turnkey applications, and education.
- The five sections—LEARN, WORK, DISCOVER, EXPLORE, and ARCHIVE—support researchers across the research life cycle and aim to make outputs FAIR and reproducible.The sections mirror the cycle from learning and simulating to publishing curated results that can start new research.
- Reproducibility beyond FAIR: EXPLORE: EXPLORE lets peers inspect complete AiiDA graphs, navigate parent and child nodes, download data, and continue research from the original authors’ database.AiiDA records calculations, codes, inputs, outputs, and how outputs are reused in subsequent calculations.
- FAIR data: ARCHIVE and DISCOVER: ARCHIVE provides moderated, free, open-access publication of computational materials data with persistent identifiers, public metadata, machine-readable harvesting, and long-term preservation.Records are guaranteed preservation for at least 10 years; general records have a 5 GB limit and AiiDA databases a 50 GB limit.
- FAIR data: ARCHIVE and DISCOVER: DISCOVER adds curated interactive visualizations and links each displayed data item to an AiiDA provenance-graph node for reproducibility.Examples include material properties, crystal structures, electronic band structures, and phonon visualizations.
- Simulation services: WORK and AiiDA lab: WORK provides browser-accessible simulation tools and AiiDA lab applications designed to make defined calculations and workflows accessible to students, experimental scientists, and computational scientists.The graphical interface supports turnkey solutions for non-specialists, while Docker supports diverse software frameworks.
- Education and outreach: LEARN and Quantum Mobile: LEARN hosts lectures, tutorials, seminars, and synchronized slides, while Quantum Mobile supplies a uniform, preconfigured environment for quantum-mechanical materials simulations across major operating systems.Quantum Mobile has been used in university courses and tutorials, helping reduce software installation and configuration time.
Discussion and Outlook
Materials Cloud shares interconnected computational outcomes and turnkey workflows through a web-accessible open-science platform. Its continued development faces technical, interoperability, governance, and long-term support challenges.
- Discussion and Outlook: Materials Cloud makes interconnected calculations and workflows findable, accessible, interoperable, reusable, and reproducible, while WORK launches turnkey workflows through a browser.These capabilities span DISCOVER, EXPLORE, ARCHIVE, and WORK.
- Discussion and Outlook: Since its 2018 launch, Materials Cloud has grown into a central repository for research data, workflows, and tools across MARVEL, MaX, and partner projects.The ARCHIVE is open to worldwide submissions and provides moderated long-term storage.
- Discussion and Outlook: Submitting new tools and interactive visualisations still requires technical expertise, motivating work toward lower barriers and a platform-as-a-service architecture.The ARCHIVE submission procedure is also planned to move to the Invenio framework.
- Discussion and Outlook: A common language for information exchange between open-science platforms remains unresolved, with proposed solutions including semantic assets, ontologies, interoperable formats, and APIs.The paper notes that convergence could connect these efforts to schema.org.
- Discussion and Outlook: Long-term support must cover continued development and maintenance of diverse services, not only storage of files.The paper frames sustained research infrastructure as a comparatively low-cost, forward-looking investment.
Methods
Materials Cloud uses modular web applications backed by AiiDA APIs, containers, and isolated JupyterHub environments. Virtualised deployment, backups, automation, and open-source distribution support operation and redeployment.
- Architecture: Independent AngularJS frontends for LEARN, WORK, DISCOVER, and EXPLORE connect to specialised backends including AiiDA REST APIs, Docker tools, and JupyterHub.JupyterHub runs one Docker container per AiiDA user.
- Architecture: The modular architecture enables individual sections to be updated without affecting the rest of the service.Section content is served directly by applications or through containerised user interfaces.
- Services: WORK and DISCOVER contributions run in containers, while each AiiDA lab account receives a private container with persistent storage and compute resources.Accounts may connect to high-performance computing resources owned by the account holder.
- Services: The EXPLORE frontend accesses calculations, workflows, codes, and data through the standardised AiiDA REST API in JSON format.The same JavaScript application can browse local AiiDA databases as well as static databases on Materials Cloud.
- ARCHIVE: ARCHIVE stores record files in OpenStack Swift with daily tape backups, while its server manages metadata and delegates file requests through short-lived unique URLs.The ARCHIVE section is only loosely coupled to the rest of the platform.
- Deployment: Materials Cloud runs on duplicated virtual machines in an OpenStack cloud, with periodic backups and automated deployment through Ansible playbooks.The modular architecture and OpenStack infrastructure simplify redeployment for load balancing, federation, or in-house use.
- Data availability: The manuscript’s datasets and those underlying Materials Cloud DISCOVER and EXPLORE sections are available in the Materials Cloud Archive under Creative Commons licenses.
A Supporting Information
The supporting information documents Materials Cloud’s deployment, provenance, and frontend–backend data flows. It shows a multi-VM architecture and the AiiDA graph and API pathways underlying the platform.
- Data flow: The frontend–backend flow validates, parses, and translates browser requests into AiiDA queries, then returns database responses as JSON.The reverse path carries response data back to the browser.
- Deployment: Materials Cloud services are distributed across multiple virtual machines, with at least one clone of each for development and testing.
- Provenance: AiiDA can generate a complete provenance graph for the electronic band-structure calculation of Bi2Se2Te.The graph is generated from an AiiDA database using the verdi node graph generate command.