Computing Environments for Reproducibility: Capturing the "Whole Tale"

1. Introduction

• Computationally intensive research has outgrown publication practices that remain largely unchanged since scientific journals began in the 1660s.The passages note that publications are often disconnected from the data and code that produced findings, while shared data can be difficult to discover or reuse.Source: §1 · p. 1
• Whole Tale captures persistent versions of research data and code, workflow provenance, and data lineage, including parameters, intermediate data, and outputs.It is intended to support the research pipeline from prepublication collaboration through publication and post-publication access and reuse.Source: §1 · p. 2
• Whole Tale combines linked data and code with a computational narrative, presented with an on-demand virtual computer containing relevant data, methods, software, and analysis frontends.The paper describes this as strengthening the publication’s data layer and building a third layer that communicates the process of inquiry.Source: §1 · p. 2
• Whole Tale builds on science-gateway infrastructure and extensible APIs for authentication, data and compute management, process recording, identifiers, and publishing reproducible tales.Its architecture is designed to let other gateways leverage those APIs.Source: §1 · p. 2

2.1. Materials Science

• Materials-science workflows combine curated data, transformations, machine-learning models, and large-scale composition scans; Whole Tale could publish these methods with data and analyses under a persistent identifier.Readers could access the resulting tale to reproduce the steps in the Whole Tale environment.Source: §2.1 · p. 3

2.2. Astronomy

• Astronomical simulation analyses rely on large-scale computing, community packages, scripts, and interactive sessions, while observational studies may combine data slices from multiple registries.Whole Tale aims to support collaborative, reusable analysis scripts and datasets in a shared environment.Source: §2.2 · p. 3

2.3. Archaeology

• Archaeology faces capacity challenges in acquiring, managing, analyzing, and synthesizing datasets for major research questions, prompting calls for infrastructure and tools.The SKOPE collaboration would use YesWorkflow to represent prospective workflows and combine them with retrospective records of executions, arguments, and, where possible, program variables [MSK+15].Source: §2.3 · pp. 3–4
• Community efforts to share archaeological methods through open code repositories and research compendia provide real-world use cases Whole Tale aims to support and enhance [Mar17, MBM17, BBJ+17].Source: §2.3 · p. 4

3. Toward Reproducibility

• Traditional publications often cannot capture the complexity of computational steps behind scientific conclusions, and reproducibility depends on aligned support from researchers, funders, publishers, and institutions.The section points to open questions despite steps being taken across stakeholder groups.Source: §3 · p. 4
• Recommendations call for sharing data, software, workflows, and environment details in trusted repositories, with persistent identifiers, citation, documentation, open licensing, and reproducibility checks.Other recommendations include funding research and pilot studies to support reproducibility.Source: §3 · p. 4
• Research communities and institutions have advanced reproducibility through digital-artifact badges, policy efforts, open-data initiatives, and infrastructure programs [ACM16, B+16, Nat17, Ber03, EUD17, Ope17, Eur17].The section also cites calls for long-term access to datasets and code needed to replicate published findings.Source: §3 · p. 5
• Recommendations emphasize integrating transparency and reproducibility into research practice, including detailed methods, separate review of scientific and technical contributions, and reuse of open resources.These practices are described as supporting confidence, verifiability, falsifiability, cooperation, and subsequent research.Source: §3 · p. 6
• Whole Tale represents one initiative to address gaps in research and dissemination infrastructure for reproducible computational research.Source: §3 · p. 6

4. Design Requirements

• Whole Tale is intended to manage scholarly objects across the data lifecycle, from ingest or creation through publication, to support reproducibility and downstream reuse.Its requirements include data ingestion, identity management, data publication, and deployment of user-facing frontends.Source: §4 · p. 6

4.1. Data Ingestion

• Researchers must discover and obtain data from distributed sources; Whole Tale aims to reduce these barriers by ingesting data from repositories, storage systems, web sources, and local devices.Repository interfaces include OAI-PMH and REST APIs.Source: §4.1 · pp. 6–7

4.2. Analysis Frontends

• Whole Tale must support extensible frontends that users can find by data type, interface, or provider, deploy on demand, and use directly with data.It must manage frontend execution and capture user steps so the environment can be packaged, published, and shared.Source: §4.2 · p. 7

4.3. Persistent Identification

• Scholarly objects include publications, data, and computational processes, so Whole Tale requires persistent identifiers and flexible ways to organize objects for collaboration and discovery.Source: §4.3 · p. 7

4.4. Authentication and Authorization

• Whole Tale aims to interoperate with existing providers’ identity systems, support preferred identities and fine-grained permissions, and use standard protocols for integration.Sensitive data, including restricted medical or government data, is outside this work’s scope.Source: §4.4 · p. 7

4.5. Reproducibility: Defining a Tale

• A tale captures a research activity’s data, methods, metadata, and provenance, with unobtrusive recording, sharing, repository publication, persistent identifiers, and links to publications.Other researchers should be able to instantiate and execute a published tale in the same state; tales also carry licensing metadata for their components.Source: §4.5 · p. 8

5. Architecture and Implementation

• Flexible APIs connect services for data, frontends, and tales to containerized execution across different compute resources.The architecture uses microservices to manage data, users, frontends, and tales.Source: Figure 1 · p. 8, §5 · p. 8

5.1. General Architecture

• Girder-backed metadata abstracts physical data locations, while Whole Tale plugins add project objects and tasks for building images, creating instances, and transferring remote data.External data can be represented by reference, leaving data management to external systems and copying data only when needed.Source: §5.1 · pp. 8–9

5.2. The Whole Tale Workflow

• External datasets are registered by identifier as Girder folders and items, while a tale combines a data folder with a Docker image built from a versioned Dockerfile recipe.Remote registration begins as a shallow copy; raw data is copied and cached as users interact with it.Source: §5.2 · pp. 9–10

5.3. Web Interface

• The web application lets users manage data, create and share analysis frontends, and publish or reproduce tales; data can also be registered from repositories or uploaded from a desktop.Users can browse public datasets and use standard file and folder operations.Source: §5.3 · p. 10

5.4. Authentication and Authorization

• Globus Auth provides identity options and delegated tokens for accessing external services, allowing Whole Tale to outsource core authentication functionality.Users can link identities so permissions granted to one identity apply to the linked set.Source: §5.4 · pp. 10–11

5.5. Data Management

• The Data Management System presents distributed primary data through a POSIX filesystem, using provider-specific transfers and cache eviction to balance storage constraints with access latency.Tale-specific sessions select subsets of accessible data and are exposed through a FUSE filesystem.Source: §5.5 · pp. 12–13

5.6. Tale Execution and Management

• Tale execution uses a Docker image and mounted data folder, with the management daemon starting the container and recording its instance for the web interface.Execution requires a Docker Engine and lightweight proxy and management daemons; supported preconfigured frontends include Jupyter and RStudio.Source: §5.6 · pp. 13–14

5.7. Tale Representation

• A tale is represented by its execution environment, data, and metadata, including a versioned Git commit, data source and access details, and publication information.The environment records a repository URL and commit ID; data descriptors include identifiers and filesystem paths.Source: §5.7 · p. 15

6. Related Work

• Science gateways capture domain-focused analyses, while Whole Tale offers a general model for sharing computational processes on arbitrary datasets and linking them with publications.Prior gateway work records user steps as workflows or other standard formats [JWS+14].Source: §6 · p. 15
• Workflow systems such as Galaxy and Kepler support flexible routines but prescribe analysis models that exclude Whole Tale's interactive, user-specific analyses.Source: §6 · p. 15
• Repository-based analysis environments support data stored in their own repositories but lack a standard model for sharing analyses and capturing complete provenance.Examples include Wolfram Data Repository and Cloud Kotta [BCGD16a] [BCGD16b].Source: §6 · p. 15
• Interactive notebook services provide similar analysis capabilities but lack standard models for data discovery, computational-process capture, and publication linkage.Examples include JupyterHub, Tmpnb, and Binder.Source: §6 · p. 16
• Scholarly-object platforms and publisher services collect data and code alongside publications, with some assigning persistent identifiers such as DOIs to hosted objects.Examples include Zenodo, RunMyCode, ResearchCompendia, and SparseLab.Source: §6 · p. 16
• W3C PROV and Research Objects model provenance or research units; the paper says Whole Tale will add similar provenance support.PROV and ProvONE have been incorporated into DataONE.Source: §6 · p. 16

• Whole Tale combines data discovery and ingestion, flexible-front-end analysis, isolated-container computation, and publication of verifiable, reproducible processes.Source: §7 · p. 16
• Its architecture uses microservices and interoperability software that leverage existing cyberinfrastructure, with a web interface for users and REST APIs for application developers.Source: §7 · p. 16
• Whole Tale working groups are engaging science communities to pilot the capabilities and evaluate their use for enabling reproducible science.Source: §7 · p. 16