Source-linked AI summary
Developing a Unified Verification and Validation Activity Standard at JPL
Maximilian Vierlboeck, Ellen Van Wyk, Natalia Sanchez, Bogdan Oaida, Torrance James Eberhart, Marie Piette Gomez, Christopher Delp, Marijke G. Jorritsma, Alex Desharnais
TL;DR
JPL’s V&V practice diverged across projects, leaving a fragmented information architecture and limiting consistency. The paper develops a unified V&V Activity schema through practitioner-centered design and implements it in Jama. The authors report that initial deployments and user tests validated the approach, while implementation experience exposed platform constraints requiring mitigation.
Problem
JPL’s V&V practice lacked consistency across projects, producing a fragmented information architecture and increasing inefficiency and infrastructure overhead.
Source: §2.2 · p. 4Method
The team developed a unified V&V Activity schema through human-centered design, using a relationship-based architecture and controlled customization pathways.
Source: §1 · p. 2, §3.1 · p. 5, §6.4 · p. 13Results
Initial deployments and user tests validated the approach, and implementation in Jama was reported to improve traceability, cross-project synergies, and efficiency despite platform constraints.
Source: §6.6 · p. 15, §8.1 · p. 17Takeaways & Limitations
The implementation preserves institutional consistency while allowing projects to tailor workflows through controlled customization.
Source: §6.4 · p. 13Jama limits calculations to data within the platform, and scheduled external synchronization introduces latency despite mitigation measures.
Source: §6.6 · p. 14
Abstract
from arXiv · showhide
Verification and validation practices (V&V) at NASA's Jet Propulsion Laboratory (JPL) have diverged over the past decade, creating fragmentation that increases overhead, reduces cross-project efficiencies, and inhibits institutional knowledge transfer. We present a unified V&V activity schema developed through human-centered design workshops involving 29 practitioners across multiple mission types and disciplines. The schema builds on a relationship-based architecture that allows for separating methods (Test, Analysis, Inspection, Demonstration, and Review of Design) while maintaining a common attribute set. Formalized as a platform-agnostic SysML model, the schema defines bidirectional relationships between requirements, V&V activities, venues, and evidence. Implementation in JPL's Jama platform demonstrates controlled customization through templates and modular item types, balancing rigor with agility while enabling automations, pattern reuse, and digital thread integration.
1. INTRODUCTION
JPL’s mission-specific V&V practices have become fragmented, while mission-critical work still demands rigor alongside efficient development. The paper presents a unified V&V Activity standard intended to standardize practice without losing flexibility for different missions.
1. INTRODUCTION
- JPL’s mission novelty and operational constraints make V&V resource-intensive, while diverse project-specific approaches have fragmented practice and limited reuse and cross-project learning.Systems inaccessible after launch cannot be recalled, repaired, or patched; the paper frames maintaining rigor while increasing development agility as a central tension.Source: §1 · p. 2
- A human-centered design process involving practitioners across disciplines and projects produced a common V&V Activity schema for flagship and low-cost, high-risk missions.The stated contributions include relationship-based traceability and reuse, integration strategies for requirements management, and initial flight-project deployments.Source: §1 · p. 2
- The schema aims to improve development efficiency through standardization while retaining flexibility for mission-specific environments.The paper describes the standard as a response to increasing divergence in how V&V was implemented.Source: §1 · p. 2
2. STATE OF PRACTICE OVERVIEW
JPL’s V&V practice retained formal requirements verification, but its information architecture and project-level implementation varied substantially. That divergence became the focus of an institutional effort to unify practice during the migration to Jama.
2.1 Fundamental Definitions & Distinctions
- JPL’s V-Model approach requires formal verification of each requirement, but the institutional plan leaves project information architecture largely unspecified.V&V includes complementary verification and validation processes, with methods selected according to confidence, cost, schedule, and technical risk.Source: §2.2 · p. 3, §2.1 · p. 3
2.2 Verification and Validation at JPL
- Across 24 instrument and full-mission projects, DNG contained 511 unique fields across 20 V&V artifact types, indicating highly fragmented information management.The count excludes information stored with requirements for closeout.Source: §2.2 · p. 3
- Fragmentation reflected both extensive project customization and divergent activity-centric versus requirement-centric approaches, with little uniformity within either paradigm.The paper links the divergence to the DNG transition, which expanded customization while its user-training curriculum was not updated; it describes the resulting loss of consistency as a shifting baseline that increased infrastructure overhead.Source: §2.2 · p. 4
- The move from DNG to Jama created an opportunity for JPL practitioners to redirect practice toward a unified approach.A practitioner-led effort begun in 2024 developed into the formal standard described in the paper.Source: §2.2 · p. 4
3. DEVELOPMENT & HUMAN-CENTERED DESIGN
The team used human-centered design to develop a unified V&V Activity standard around practitioner needs. Workshops gathered and prioritized use cases, shaped the schema, and provided feedback on the resulting prototype and its applicability.
3. DEVELOPMENT & HUMAN-CENTERED DESIGN
- The team chose a V&V Activity-centric standard developed through human-centered design to address diverse stakeholder needs while retaining mission-critical rigor.The design team selected participatory workshops using the Double Diamond model to support a schema for JPL’s broad range of projects, roles, and perspectives.Source: §3 · p. 4, §3.1 · p. 5
3.1 Background and Methodology
- Four 3-hour workshops with 29 participants over two months used Discover, Define, Develop, and Deliver stages to gather needs, prioritize use cases and fields, prototype, and test the schema.The team documented about 500 unique inputs; the approach used divergence to collect ideas and convergence to narrow them.Source: §3.1 · p. 5
3.2 Outcomes
- Practitioners identified traceability, results, and evidence as needs across V&V phases, with scope, content, priorities, schedules, resources, and reporting especially important in planning.The workshops also produced an implementation-agnostic workflow for V&V responsibilities across the project lifecycle.Source: §3.2 · p. 6
3.3 Feedback and Reflection
- In the post-workshop survey, 85% of respondents agreed the MVP could be used by all JPL missions, and 100% agreed it emerged from shared understanding.A participant noted that starting from scratch could overwhelm participants with inputs; the authors suggest preparing representative use cases before future workshops.Source: §3.3 · p. 6
4. SCHEMA BUILDING BLOCKS
The schema extends existing requirements management structures with explicit relationships among requirements, V&V activities, venues, and evidence. Separate method-specific activity types share core attributes, while additional features support traceability and controlled project customization.
4. SCHEMA BUILDING BLOCKS
- The relationship-based schema extends existing requirements structures so V&V Activities can link directly to requirements and support multi-directional traceability.Projects can select the activity methods they need while retaining a common core structure.Source: §4 · p. 6, §4.1 · p. 6
4.2 Expansion & Additional Schema Aspects
- Separate V&V Activity types share core attributes while carrying method-specific information; venue items record where test and demonstration activities occur.Explicit relationships connect requirements, activities, venues, and evidence, supporting change-impact identification and analyses across requirements and system elements.Source: §4.2 · pp. 6–7
- Outbound references, templates, and project-specific pick lists provide controlled ways to connect external information and adapt content while preserving a consistent schema.Core pick lists standardize shared vocabulary, while extensible lists accommodate project terminology; together with item types and relationships, these elements form a tool-agnostic foundation.Source: §4.2 · pp. 7–8
5. DATA MODEL & INFORMATION ARCHITECTURE
The schema separates a platform-independent V&V information model from its implementation while defining shared and method-specific activity attributes and explicit relationships among system elements. Its SysML model also supports implementation, evolution, and interoperability across platforms.
5. DATA MODEL & INFORMATION ARCHITECTURE
- The platform-agnostic architecture separates the logical data model from its implementation, allowing it to evolve with technology while retaining clear interfaces and compatibility.The schema is intended to transfer across platforms without losing its conceptual architecture.Source: §5 · pp. 8–9
5.1 SysML Information Model
- The SysML model defines a shared base activity with method-specific subclasses and controlled vocabularies, while a separate relationship schema specifies directional links among requirements, activities, and evidence.The base captures common identifiers, ownership, status, scheduling, and evidence fields; subclasses add method-specific attributes. “Verified By” and its inverse “Verifies” define bidirectional flow between requirements and activities.Source: Figure 3 · p. 8, §5.1 · pp. 9–10, Figure 5 · p. 10
5.2 SysML Information Model Incorporation
- As an executable specification, the SysML model can guide API-driven updates, interoperability contracts, implementation validation, and extensions that add activity types without modifying the core schema.It also serves as a visual bridge between implementation teams and V&V domain experts.Source: §5.2 · pp. 10–11
6. JAMA IMPLEMENTATION & PRINCIPLES
Jama implementation translates the technology-agnostic V&V schema into an operational system, balancing institutional consistency with project-specific flexibility while working around platform constraints.
6.1 Platform & Project Configuration
- The base project template supplies V&V item types, relationship rules, pick lists, permissions, integrations, and a folder structure, then projects tailor it through controlled customization.The standard frames what projects record without prescribing all project-specific content.Source: §6.1 · p. 11
6.2 Item Type Implementation
- Jama uses separate item types for five V&V methods with shared core attributes; projects can enable needed types and customize picklists through Entity items.Because Jama lacks native inheritance, attribute replication and naming conventions preserve the model’s logical class structure.Source: §6.2 · p. 12, Figure 6 · p. 12
6.3 Relationships & Traceability
- “Verified By” links requirements to downstream activities and “Executed In” links tests to venues; relationship rules and trace views expose gaps and support impact analysis.Suspect links flag affected items after changes, while traceability views navigate by requirement, venue, method, or system element.Source: §6.3 · pp. 12–13
6.4 Templates & Content Customization
- Optional field templates, extensible pick lists, custom views, and report templates let projects tailor workflows and communications while preserving schema integrity and institutional standardization.For example, analysis templates can guide assumptions and limitations while allowing project-specific content.Source: §6.4 · p. 13
6.5 Interfaces with External Systems
- Jama hyperlinks reference external evidence without duplicating it, while APIs support bidirectional integration; future Syndeia links could connect Jama activities with Jira and other repositories.The paper describes Syndeia integrations as future capabilities, not as already implemented integrations.Source: §6.5 · pp. 13–14, Figure 8 · p. 14
6.6 Current Limitations & Mitigations
- Jama’s lack of webhook initiation makes real-time synchronization challenging, and platform-limited rollups exclude external metrics; scheduled polling and supplemental reporting mitigate these constraints.Document-based approvals remain necessary, and replicated attributes create maintenance overhead when the schema changes; governance and controlled document generation address these issues.Source: §6.6 · pp. 14–15
- The Jama implementation operationalizes the technology-agnostic information model as a unified framework that supports traceability, cross-project synergies, and efficiency despite platform constraints.The paper reports these outcomes for the implemented system and describes migration and broader adoption as a path toward increasing institutional benefits.Source: §6.6 · p. 15
7. PROJECT ROLLOUT & IMPLEMENTATION
The rollout balances standardization with project lifecycle constraints: early projects can adopt fully, while established projects may migrate partially or retain legacy schemas. New projects use the standard from setup, and existing-project pilots validate a controlled migration approach.
7.1 Approach and Goals
- Projects at different lifecycle phases have different capacity for change, so early projects can adopt fully while development- or operations-stage projects may migrate partially or bridge from retained legacy schemas.The strategy also uses structured migration assessment and early pilot engagement for user acceptance testing.Source: §7.1 · p. 15
- Automatic integrations, automation capabilities, and solution accelerators offer adoption incentives, while making the standard the foundation for new projects gradually increases its use with less disruption.Over time, legacy schemas are expected to converge toward the standard through lifecycle-aligned adoption.Source: §7.1 · p. 16, Figure 9 · p. 16
7.2 Roll-Out, Status, and Planned Steps
- New projects begin with the standard in their project setup, while existing-project migration captures a baseline, maps legacy attributes, and imports activities while maintaining relationships.Pilot migrations validated the approach; complexity assessments consider volume, custom attributes, and integration requirements.Source: §7.2 · p. 16
- The rollout remains phased, with plans to expand adoption, refine migration steps, and develop integrations; implementation experience has validated core design principles while identifying refinement opportunities.The approach balances standardization with flexibility for project-specific tailoring.Source: §7.2 · p. 17
8. SUMMARY AND FUTURE WORK
The paper presents a flexible, relationship-based V&V standard and reports benefits for coordination, traceability, planning, and reuse. It also outlines future work toward reusable patterns, analytics, model synchronization, deeper integrations, and data-driven guidance.
8.1 Summary
- The standard connects requirements, activities, and evidence through explicit relationships, using a platform-independent SysML model and an implementation with controlled customization pathways.It separates methods into related item types while maintaining a common core structure.Source: §8.1 · p. 17
8.2 Benefits & Insights
- Bidirectional traceability can flag affected activities when requirements change, supporting impact assessment while reducing coverage gaps and redundant work.A common framework also improves cross-discipline coordination by clarifying dependencies and shared resources.Source: §8.2 · p. 17
- Distinct but related method types support targeted planning and resource optimization, while standardized templates and information structures are expected to save documentation time.Shared architecture also supports reusable automations and integrations across adopting projects, and early adopters report easier cross-project knowledge transfer.Source: §8.2 · p. 18
8.3 Future Work & Pending Efforts
- Future work proposes reusable V&V patterns, analytics across project relationships, and direct mapping of SysML models to implementation environments.Planned directions also include deeper external-system integrations, model-based planning, and machine-learning or AI support for effort estimates, issue prediction, and strategy recommendations.Source: §8.3 · pp. 18–19
BIOGRAPHIES
The biographies describe contributors in systems engineering, human-centered design, flight systems, and digital engineering, with experience spanning mission projects, requirements platforms, and workflow modernization.
BIOGRAPHIES
- Maximilian Vierlboeck works on enterprise requirements platforms and digital engineering, while Marijke Jorritsma focuses on human-centered design for complex aerospace systems.Ellen Van Wyk brings workflow and user-experience design across flight projects and technical teams.Source: BIOGRAPHIES · pp. 20–21
- Natalia Sanchez leads systems-engineering practice initiatives and has over 15 years of V&V and cross-cutting engineering experience across planetary-science and Earth-observing missions.Bogdan Oaida's work spans instrument systems engineering, requirements and V&V architecture, and mission lifecycle roles.Source: BIOGRAPHIES · pp. 20–21
- Torrance Eberhart develops digital-engineering frameworks and requirements-maturity dashboards, while Marie Piette Gomez adapts integrated systems infrastructure to flight-project needs.Christopher Delp has led digital-engineering and model-based systems-engineering efforts, including OpenMBEE and the Europa Clipper digital environment.Source: BIOGRAPHIES · p. 21