PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond FAIR: A Fitness Function Framework for Sustainable Research Software

Irdin Pekaric, Philipp Zech

arXiv:2610.09580v1cs.SEcs.CR

TL;DR

FAIR-aligned assessment does not cover all sustainability concerns, including environmental responsibility and long-term security. The paper extends an existing fitness-function framework with environmental and security checks. It concludes that eight atomic checks broaden assessment beyond FAIR and that FAIR-compliant artifacts may still lack support for efficient repeated computation or systematic security maintenance.

  • Problem

    FAIR-aligned frameworks leave environmental and security sustainability unassessed, although both affect research software’s long-term viability.

    Source: §2.3 · p. 3

  • Method

    The paper extends the FAIR-oriented fitness-function framework with environmental and security functions, including formal, implementation-independent environmental checks.

    Source: §1 · p. 2, §2.1 · p. 2, §3.2 · p. 3

  • Results

    The framework adds eight atomic checks, and FAIR-compliant artifacts may still lack efficient repeated-computation support or systematic security maintenance.

    Source: §6 · p. 10

  • Takeaways & Limitations

    Sustainable research software should also be environmentally responsible, transparent in execution, deployable with limited infrastructure burden, and maintainable from a security perspective.

    Source: §6 · p. 10

    The implementation targets Python-based GitHub repositories, and heuristic checks make the framework a lightweight first layer of evidence rather than a replacement for manual review.

    Source: §5.2 · p. 9

Abstract

from arXiv · show

Research software sustainability is often assessed through the FAIR principles: findability, accessibility, interoperability and reusability. While important, FAIR does not cover all relevant sustainability concerns. Research software should also be environmentally responsible and secure over time. Excessive resource consumption increases environmental cost, while insecure software creates maintenance overhead, technical debt and barriers to reuse. To this end, in this paper, we extend a previously proposed fitness function framework for sustainable research software beyond FAIR. We introduce two additional sets of fitness functions: environmental functions targeting resource efficiency and execution footprint, and security functions targeting dependency health, vulnerability exposure and secure configuration. Together, these functions broaden continuous software assessment from FAIR compliance to a more complete view of sustainability. The resulting framework treats research software sustainability as a multidimensional property that includes FAIR, environmental responsibility and long-term security.

1. Introduction

Research software is a critical scientific output, but sustaining it remains difficult. The section situates the work in FAIR-oriented assessment and motivates extending that assessment to environmental responsibility and security.

1. Introduction

  • Research software encodes methods, enables reproducibility, and lets others build on published results, yet sustaining it remains a persistent challenge.The text also notes that research software is not required to meet standards often expected of other scientific methods, including peer review and reproducibility.Source: §1 · p. 1
  • FAIR offers a framework for research-software assessment, and prior fitness functions operationalize its four dimensions as continuously evaluated metrics.FAIR covers findability, accessibility, interoperability, and reusability; the functions are described as a coherent, low-overhead mechanism for evaluating compliance.Source: §1 · pp. 1–2
  • FAIR alone leaves environmental cost and security outside the sustainability picture, although both affect research software’s long-term viability.The text connects insecure software with maintenance overhead, technical debt, and barriers to reuse.Source: §1 · p. 2
  • The paper extends the Zech and Pekaric (2026) framework with environmental functions for resource efficiency and execution footprint, and security functions for dependency health, vulnerability exposure, and secure configuration.The resulting framework treats sustainability as multidimensional, encompassing FAIR, environmental responsibility, and long-term security.Source: §1 · p. 2

2. Background and Related Work

Related work develops fitness functions and measures FAIR for research software, but existing FAIR-aligned approaches leave environmental and security sustainability unaddressed.

2.1. Fitness Functions

  • Fitness functions are repeated automated checks of non-functional properties; Zech and Pekaric (2026) applied them to research software’s four FAIR dimensions.Ford et al. (2022) describe their use in evolutionary architectures; each function targets one well-scoped property and can flag regressions during development.Source: §2.1 · p. 2

2.2. Related Work

  • Research software sustainability is a documented challenge, and FAIR principles introduced for data have been adapted to software, including FAIR4RS and an earlier life-sciences formulation.The passage attributes the sustainability challenge to Carver et al. (2022), FAIR4RS to Barker et al. (2022), and the earlier formulation to Lamprecht et al. (2020).Source: §2.2 · p. 2
  • FAIRsoft covers more than 43,000 life-sciences tools and finds software generally findable but barely interoperable, with licensing and dependency-documentation gaps.Hasselbring et al. (2020) also argue that open-source availability is a prerequisite for meaningful FAIR compliance.Source: §2.2 · p. 2
  • FAIR work also spans machine-learning workflows, while only 3% of HICSS papers from 2017 to 2024 had a functional, publicly available artifact repository.The passage notes that metadata and tools alone are insufficient for ML without rich provenance and standardized documentation.Source: §2.2 · p. 2

2.3. Research Gap

  • Existing FAIR-aligned frameworks leave environmental and security properties unassessed, even though vulnerable dependencies, insecure configurations, and high computational resource use can affect long-term viability.The paper addresses this gap by adding environmental and security functions that make those properties measurable and automatable within the development workflow.Source: §2.3 · p. 3

3. Methodology

The methodology follows design science research to extend an existing fitness-function artifact beyond FAIR. It elicits requirements and derives implementation-independent functions from them.

3. Methodology

  • The study follows the Design Science Research paradigm and builds on the fitness-function framework and artifact of Zech and Pekaric (2026) and Pekaric and Zech (2026).Its stated design-science problem is to improve research software sustainability by designing fitness functions that address environmental and security requirements beyond FAIR.Source: §3 · p. 3
  • The design goal is to deliver sustainable research software beyond FAIR through fitness functions addressing environmental and security sustainability requirements.This is the stated context, artifact, requirement, and goal of the design-science problem.Source: §3 · p. 3
  • The method first elicits extended sustainability requirements beyond FAIR, then derives formal, implementation-independent fitness functions from those requirements.The paper identifies these as two sequential steps.Source: §3 · p. 3

3.1. Extended Sustainability Requirements

The paper adds eight requirements across environmental responsibility and long-term security, selected as an initial operational set beyond FAIR.

3.1. Extended Sustainability Requirements

  • The authors elicit eight additional requirements covering environmental responsibility and long-term security, following the earlier approach of deriving requirements from FAIR.The requirements address computational costs of running, deploying, and maintaining software, and security concerns around exposure, integrity, and maintainability.Source: §3.1 · p. 3
  • The eight requirements were selected because they address concerns beyond FAIR, can be observed through repository or execution evidence, and are actionable by developers.The authors characterize them as an initial operational set rather than an exhaustive sustainability taxonomy.Source: §3.1 · p. 3
  • Broader infrastructure and hardware-level concerns remain outside the current scope of the requirements.The requirements are an initial operational set, not an exhaustive taxonomy.Source: §3.1 · p. 3

3.2. Fitness Functions for Sustainability

The section defines four atomic environmental fitness functions as Boolean checks over artifacts, benchmarks, or environments. They assess resource efficiency, footprint transparency, infrastructure-aware deployment, and recomputation avoidance.

3.2.1. Resource Efficiency

  • Resource efficiency passes when measured consumption stays within defined thresholds for a benchmark scenario.Thresholds are configurable implementation defaults, not universal sustainability limits; maintainers or domain communities should set them for representative workloads and environments.Source: §3.2.1 · p. 3

3.2.2. Footprint

  • Footprint transparency passes when required execution-footprint information is documented and fails when it is missing, incomplete, or unreadable.Examples include runtime requirements, dependency specifications, and platform constraints.Source: §3.2.2 · p. 4

3.2.3. Infrastructure-Aware Deployment

  • Infrastructure awareness checks whether deployment avoids unnecessary platform constraints, excessive dependency overhead, and avoidable hardware specialization in a given environment.Because these properties depend on the target environment, implementations may vary by context.Source: §3.2.3 · p. 4

3.2.4. Recomputation Avoidance

  • Recomputation avoidance passes when an artifact supports mechanisms such as caching, checkpointing, modular execution, or parameterized reruns.The function fails when it detects no support for reducing unnecessary recomputation.Source: §3.2.4 · pp. 4–5

3.3. Fitness Functions for Security

The section introduces four atomic, implementation-independent security fitness functions designed for development or continuous-integration workflows. They assess dependency health, secure configuration, integrity and provenance, and ongoing security maintenance.

3.3.1. Dependency Health

  • Dependency health checks whether declared external components are maintained and free of vulnerabilities beyond accepted constraints.Outdated or abandoned dependencies, or those with known vulnerabilities beyond the accepted threshold, cause failure.Source: §3.3.1 · p. 5

3.3.2. Secure Configuration

  • Secure configuration fails when an artifact exposes insecure defaults or unsafe settings, such as secrets, unsafe permissions, or plaintext communication.The check also targets excessive privileges and debug-oriented defaults that increase attack surface.Source: §3.3.2 · p. 5

3.3.3. Integrity and Provenance

  • Integrity and provenance passes when artifacts support traceability through mechanisms such as version pinning, trusted sources, or traceable build and deployment processes.Missing, incomplete, or unverifiable integrity or provenance guarantees cause failure.Source: §3.3.3 · pp. 5–6

3.3.4. Maintainable Security Posture

  • Maintainable security assesses observable evidence that security issues can be monitored and updated with reasonable effort over time.Examples include vulnerability detection, dependency-update workflows, security policies, and remediation support; the function fails when sufficient evidence is absent.Source: §3.3.4 · p. 6
  • The functions use repository- and execution-level indicators, not complete measurements of environmental impact or security.Their outcomes should be interpreted as lightweight automated evidence for the corresponding sustainability properties.Source: §3.3.4 · p. 6

4. Results

The extended framework implements independent checkers for sustainability properties beyond FAIR and evaluates them on the HICSS artifact and several widely used Python repositories. Manual inspection found the checked outcomes consistent with repository evidence, while the authors caution that these indicators do not fully capture broader sustainability constructs.

4. Results

  • Each new function runs as an independent checker over artifact metadata and repository evidence, returning a Boolean result with diagnostic evidence; evaluation covered the HICSS artifact and seven widely used Python repositories.The HICSS artifact served as a running example, and Tab. 2 summarizes results across the evaluated artifacts.Source: §4 · p. 6
  • Manual inspection found one passing and, where available, one failing outcome per fitness function consistent with repository evidence, but did not establish full construct coverage.The authors describe this as initial evidence that the implementations detect the repository-level properties they operationalize.Source: §4 · pp. 6–7

footprint transparency.

  • The HICSS artifact passed infrastructure awareness and basic footprint-transparency checks, but lacked stronger deployment and execution-profile documentation.The checker found no obvious infrastructure constraints, while container or deployment specifications, CI workflows, and explicit deployment documentation were missing.Source: footprint transparency · p. 7, §4.1 · p. 7, Figure 1 · p. 7
  • The HICSS artifact failed recomputation avoidance because evidence for caching, checkpointing, parameterized reruns, and related mechanisms did not meet the configured threshold.The authors suggest experiment parameters, modular execution scripts, intermediate-result caching, or checkpointing as improvements.Source: footprint transparency · pp. 7–8

4.2. Security Fitness Functions

  • The HICSS artifact passed dependency health: its three declared dependencies were bounded, and the checker detected no risky patterns.The authors note that dependencies can later become outdated or vulnerable, making repeated checks useful.Source: §4.2 · p. 8, Figure 2 · p. 8
  • The HICSS artifact passed secure configuration: static pattern matching found no obvious insecure patterns and detected .gitignore protection for .env files.The checker is diagnostic, not a complete security audit; findings in examples, documentation, or tests may be false positives.Source: §4.2 · p. 8
  • The HICSS artifact passed integrity and provenance checks using basic traceability evidence, but lacked stronger machine-actionable citation and environment-verification evidence.Missing items included CITATION.cff, lock and container files, checksums, and explicit release metadata.Source: §4.2 · p. 8
  • The HICSS artifact failed maintainable security because it lacked evidence such as a security policy, dependency-update automation, security or CI workflows, and static-analysis tooling.The authors identify these gaps as evidence that systematic long-term security maintenance is not established.Source: §4.2 · p. 8

5. Discussion

The discussion presents environmental and security checks as practical, measurable extensions to FAIR, while emphasizing that the current implementation is lightweight and bounded by heuristic checks and its Python/GitHub scope.

5. Discussion

  • Regular checks can flag omissions before they become barriers to reuse, replication, or maintenance, and can run manually or in CI.The implementation targets Python-based GitHub repositories, although its underlying principles are language-agnostic and adaptable to other ecosystems.Source: §5 · p. 9

5.1. Sustainability Implications

  • Environmental checks make resource consumption, execution footprint, infrastructure requirements, and recomputation avoidance visible to support more responsible execution.They can also encourage developers to document requirements, support modular execution, and reduce unnecessary computation.Source: §5.1 · p. 9
  • Security checks address dependency health, secure configuration, integrity and provenance, and maintainable security to support long-term reuse.Researchers can run and interpret the checks locally, without centralized quality-assurance infrastructure.Source: §5.1 · p. 9

5.2. Limitations and Future Work

  • The implementation is limited to Python-based GitHub repositories, and heuristic security findings may reflect documentation, tests, or examples rather than actual vulnerabilities.A clean check does not prove a repository is secure; the framework is a lightweight first layer of evidence, not a substitute for manual review or security auditing.Source: §5.2 · p. 9
  • Resource-efficiency results depend on the workload and threshold, and lightweight benchmarks may not represent realistic scientific use.Binary outcomes can give minor and extensive non-compliance the same result; benchmark execution may also increase build time, so expensive checks can run periodically.Source: §5.2 · p. 9
  • Planned extensions include configurable benchmarks, domain-specific thresholds, maturity levels, broader ecosystem support, and integration into evaluation and institutional workflows.The authors also plan to investigate domain-specific languages for modeling checks and generating executable checks from those models.Source: §5.2 · p. 9

6. Conclusion

The conclusion frames sustainability as extending beyond FAIR, with eight atomic checks for environmental and security properties. It reports that FAIR-compliant artifacts may still lack support for efficient repeated computation or systematic security maintenance.

6. Conclusion

  • FAIR-compliant artifacts may still lack support for efficient repeated computation or systematic security maintenance.The framework adds eight atomic environmental and security checks, covering resource efficiency, footprint transparency, infrastructure-aware deployment, recomputation avoidance, dependency health, secure configuration, integrity and provenance, and maintainable security posture.Source: §6 · p. 10
Loading 2610.09580v1…