PaperlayerView exact revision on arXiv

Source-linked AI summary

Testing Agentic Service Systems: An Agentic Living Lab Methodology Across the Physical-Digital Continuum

Zhongjie Wang, Xiang He, Xiao Wang, Mingyi Liu, Shuang Yu, Hanchuan Xu

arXiv:2610.35978v1cs.SE

TL;DR

Task success alone does not establish that an agent acted with authority, fulfilled obligations, or behaved consistently across execution substrates. The paper proposes an episode-based testing methodology that makes participant roles and reality bindings explicit; a constructed witness demonstrates machine-checkable differences in fulfillment, while empirical validation remains open.

  • Problem

    Agents may complete a task through unauthorized action, leave obligations unresolved, or behave differently when enterprise timing, physical dynamics, or human intervention enter the episode.

    Source: §1 · pp. 1–2

  • Method

    The methodology treats the service episode as the test unit, separates tested agents from human-role participants and surrogates, and makes substrate capabilities and realization changes explicit.

    Source: TQ3 — Under what reality is the test… · pp. 2–3, §4.4 · p. 12

  • Results

    A constructed travel witness shows that output equivalence is insufficient for fulfillment equivalence and that the case can be represented and machine-checked under the stated contract semantics.

    Source: §7.5.2 · p. 29

  • Takeaways & Limitations

    When the same business hypothesis is tested across substrates, reported changes must be disclosed rather than attributed to the agent under test.

    Source: §4.6 · p. 14

    The constructed designs and limited runner do not validate practical superiority, while proxy validity and cross-substrate comparability remain empirical questions.

    Source: §9.5 · p. 39

Abstract

from arXiv · show

Task success is insufficient for testing agents that act for principals, change persistent business state, and affect physical processes: a useful result may be unauthorized, leave obligations unresolved, or depend on the execution substrate. We propose an Agentic Living Lab methodology organized around three questions: what is the test unit, who and what participate, and under what reality is the test executed? The unit is a service episode containing delegated roles, obligations, world state, and evidence. The method separates agents under test from environment actors and distinguishes real humans, human traces, rule surrogates, and agent-based proxies. Realization Profiles, capability-declared Reality Substrate bindings, Grounding Profiles, and Semantic Delta disclosures make changes across software, enterprise systems, simulation, and physical settings explicit. A four-plane reference architecture separates experiment control, participants, world realization, and evidence evaluation, with cross-cutting authority and governance controls. Runtime semantics distinguish intent, normative admission, and factual reconciliation. Real stakeholder grounding, participant calibration or validation, and governed feedback are required for the Living Lab claim, but not synchronous human participation in every run. Released JSON Schemas, deterministic fixtures, and a limited reference runner make selected rules machine-checkable; they are not a complete platform or empirical validation. Constructed travel and industrial designs illustrate authorization and fulfillment failures, proxy sensitivity, and substrate changes. Agent quality, proxy validity, and cross-substrate effect stability remain empirical questions.

1 Introduction

Testing agentic services requires more than checking task completion: actions may lack approval, leave obligations unresolved, or behave differently across execution settings. The paper proposes an episode-centered method that specifies who participates and how the test reality is realized, with an Agentic Living Lab as reference infrastructure.

1 Introduction

  • Agents can reach a goal without approval or leave obligations unresolved, and performance may weaken when enterprise timing, physical dynamics, or human intervention enter the episode.The paper frames testing around authorized action, fulfilled obligations, and realized value—not task completion alone.Source: §1 · pp. 1–2

TQ1 — What is tested?

  • The service episode is the test unit, retaining delegated roles, obligations, world state, and evidence across the interaction.The paper asks how to test systems that act for principals, interact with people and institutions, and may affect physical reality.Source: TQ1 — What is tested? · p. 2, §1 · p. 2

TQ2 — Who and what participate?

TQ3 — Under what reality is the test executed?

  • A complete Agentic Living Lab requires interfaces for real-stakeholder grounding, proxy calibration or challenge, consequential decision review, and governed feedback, but not synchronous human participation in every run.Without these stakeholder-grounding and feedback interfaces, an environment is Living-Lab-ready rather than a complete Living Lab.Source: TQ3 — Under what reality is the test… · pp. 2–3
  • Realization Profiles, capability-declared Reality Substrate bindings, Grounding Profiles, and Semantic Delta disclosures make changes across execution settings explicit, without claiming sim-to-real equivalence.The architecture also specifies four planes: Experiment Control, Participant & Agent, World & Reality, and Evidence & Oracle.Source: TQ3 — Under what reality is the test… · p. 3
  • The travel and industrial cases are constructed designs, not empirical validation; comparative agent quality, proxy validity, and cross-substrate effect stability remain open empirical questions.The paper also leaves the SUT’s internal cognitive architecture out of scope and does not treat private chain-of-thought as authoritative evidence.Source: TQ3 — Under what reality is the test… · p. 3

2 Testing Model for Agentic Service Systems

The testing model treats a service episode as a continuing, evidence-bearing interaction among principals and roles, rather than reducing evaluation to an agent–task pair. It defines actor boundaries, multiple outcome judgments, four observable protocol boundaries, and a minimal runtime contract for admission, execution, reconciliation, and evidence.

2.1 Service episode as the first-class test unit

  • A service episode is a bounded enactment of one or more service relationships, recorded with roles, world state, governing agreements, events, and retained evidence.The record preserves incomplete, delayed, or contradictory observations rather than replacing uncertainty with an agent-generated narrative.Source: §2.1 · pp. 3–4

2.2 System under test, environment actors, and human-role realizations

  • The SUT boundary determines whether agent interactions are internal behavior or externally provisioned environment inputs, and the Run Manifest freezes actor membership.Human-role software remains a harness actor unless declared part of the tested system; replacing a person with a proxy does not remove the need to represent and evidence role decisions.Source: §2.2 · pp. 4–5

2.3 Multi-judgment test oracle: capability is not fulfillment

  • Run completion, task success, service fulfillment, authority, continuity, evidence, human work, and stakeholder value are distinct judgments that must not be collapsed into one score.The oracle reports a non-aggregated vector, while proxy choice can affect interaction patterns and conclusions about system quality.Source: §2.3 · p. 5

2.4 R1–R4 as observable testing boundaries

  • R1 requires current authority, prerequisite, policy, resource, and evidence checks before consequential dispatch, with the admission decision recorded.R2 makes commitments effective only through an agreed authorization protocol, not merely because a model generated a promise.Source: §2.4 · p. 6
  • R3 blocks future use of revoked authority without erasing incurred obligations; R4 requires accepted outcome evidence and discharged or transferred obligations for fulfilled closure.Failed, cancelled, breached, disputed, or unresolved episodes can terminate without being labeled fulfilled.Source: §2.4 · p. 6

2.5 Minimal semantic kernel

  • The minimal runtime contract observes versioned state, admits actions using fail-closed guards, executes only admitted actions, reconciles uncertain effects, and commits canonical evidence.ALLOW permits proceeding to reservation, not immediate dispatch; fixtures exercise selected decision-level consequences rather than end-to-end platform enforcement.Source: §2.5 · p. 7

3 Positioning Against Existing Evaluation and Experimental Paradigms

Existing benchmarks, Living Labs, testbeds, and multi-agent deployments provide important capabilities, but the paper’s contribution is a portable, version-pinned service-episode test contract. The broader Agentic Living Lab is a nonexclusive setting for combining compatible test modes with stakeholder grounding and feedback.

3.1 Stateful agent benchmarks and simulated users

  • The proposed test contract extends stateful benchmarks with portable, conformance-testable semantics for delegation, commitments, human roles, factual status, closure, evidence, and reality declarations.Adding database tables alone is insufficient unless transition, evidence, and reporting semantics are enforced.Source: §3.1 · p. 8
  • Human proxies require explicit realization, calibration, and sensitivity because simulator design and grounding can affect alignment with real-human judgments.The paper treats proxies as an option to govern, not a category to reject.Source: §3.1 · p. 8

3.2 Living Labs, testbeds/TEFs, and real-world MAS

  • Living Labs, physical–virtual testbeds, and real-world multi-agent deployments are established; the increment is keeping actors, authority, obligations, effects, and evidence aligned as realization changes.Living Lab usage still requires stakeholder grounding and governed feedback; a synthetic harness without these is only Living-Lab-ready.Source: §3.2 · p. 8

3.3 Digital twins, AgentOps, normative systems, and reproducibility objects

  • The method synthesizes existing normative and usage-control models by binding delegation, commitments, participant realizations, factual reconciliation, reality disclosure, and evidence into one version-pinned episode.Digital twins, AgentOps, provenance systems, and runtime contract monitoring are identified as adjacent antecedents.Source: §3.3 · p. 8

3.4 Compact positioning

  • The Agentic Living Lab is a methodological rather than exclusivist claim: existing platforms can host the method if they preserve episode semantics and pass conformance conditions.It names broader socio-technical infrastructure that combines test modes with stakeholder grounding and feedback.Source: §3.4 · p. 9

4 Agentic Living Lab Testing Methodology: Participants, Grounding, and Reality

The methodology defines an Agentic Living Lab through stakeholder-grounded service episodes and explicit participant and world realizations. It makes substrate capabilities and cross-substrate changes visible so comparisons can be interpreted against declared conditions.

4.1 Definition and three forms

  • A complete Agentic Living Lab requires real-stakeholder involvement in scenario or acceptance definition, human-role calibration or validation when relevant, and governed feedback into future experimental assets.Without these socio-technical bindings, an environment remains Living-Lab-ready regardless of its technical sophistication.Source: §4.1 · p. 9

4.2 Human participation as a configurable test dimension

  • Human roles can be enacted by real people, historical traces, rule surrogates, or agent proxies, without requiring synchronous human participation in every Run.Proxy-sensitive conclusions call for matched proxy reruns or justification against human traces or participants.Source: §4.2 · pp. 10–11

4.3 Realization and grounding

  • Realization Profiles record where participants and world objects are instantiated, while Grounding Profiles record the real-world evidence supporting their interpretation.Physicality is descriptive rather than a maturity score, and the method defines no universal fidelity axis.Source: §4.3 · pp. 11–12

4.4 The physical–digital continuum and Reality Substrate

  • Reality Substrates declare capabilities such as observation, action, synchronization, snapshotting, reset, and rollback-related operations, which vary across software, shadow, HIL, and physical settings.Physical reality generally cannot be forked or rolled back.Source: §4.4 · p. 12, Table 6 · p. 12

4.5 A minimal Reality Substrate contract

  • Versioned, object-level Reality Bindings expose each adapter’s capabilities and constraints; missing required capabilities cause planning or launch to fail closed.Bindings cannot be silently replaced during a Run; rebinding creates evidence and triggers Semantic Delta and comparability checks.Source: §4.5 · p. 13, Table 7 · p. 13

4.6 Semantic Delta and comparability

  • Semantic Delta disclosures classify substrate changes as comparison-preserving, comparison-with-caveat, or comparison-breaking relative to the target hypothesis.In the BusinessRules-to-FMU example, the comparison is retained with caveats because temporal, observational, and failure semantics change; outcomes should not be attributed to the AUT without reporting those deltas.Source: §4.6 · p. 14, Table 8 · p. 14

5 Agentic Living Lab Reference Test Harness Architecture

The reference harness organizes controlled service-episode tests across experiment control, participants, world realization, and evidence evaluation. Authority and grounding/governance span these planes, while Experiment Cells bind an isolated run's components.

5 Agentic Living Lab Reference Test Harness Architecture

  • The harness is deliberately thinner than a production-agent platform but configures actors and worlds, executes episodes, admits consequential actions, collects evidence, and computes multi-judgment verdicts.Its four core planes are joined by two cross-cutting rails.Source: §5 · p. 15

5.1 Experiment Control Plane

  • The Control Plane versions studies and expands declared experimental factors into concrete runs with pinned artifacts and versions.It supports declared randomization, matched comparisons, proxy-sensitivity designs, and cross-spectrum migrations.Source: §5.1 · p. 15

5.2 Participant & Agent Plane

  • The Participant & Agent Plane separates the SUT from environment actors and permits real humans and version-pinned proxies to share a study.Multiple AUTs inside one SUT are internal behavior, while the frozen participant boundary makes that classification auditable across reruns.Source: §5.2 · pp. 16–17

5.3 World & Reality Plane

  • The World & Reality Plane coordinates recorded business state, event ordering, invariants, admission inputs, and factual reconciliation across object-level world bindings.A run can bind different objects to different substrates, such as an ERP shadow, MES twin, FMU machine, physical sensor, and software counterparty.Source: §5.3 · p. 17

5.4 Evidence & Oracle Plane

  • The Evidence & Oracle Plane records consequential boundaries and derives judgments, with replay, counterfactual forks, failure attribution, and conformance checks.Automatic checks can be combined with authorized human review, whose decisions are themselves versioned evidence.Source: §5.4 · p. 18

5.5 Cross-cutting rails: authority/contract and grounding/governance

  • The authority/contract rail defines role-bound rights and collaboration contracts, while grounding/governance records consent, calibration, realization, incidents, and feedback assetization.A proxy can exercise only the rights attached to the role it realizes.Source: §5.5 · p. 18

5.6 Experiment Cell and external interfaces

  • An isolated Experiment Cell binds one run's SUT, actors, world and substrates, authority state, scenario, evidence sidecar, and oracle.The architecture is implementation-independent; compatibility is judged at normative episode, interchange, and conformance boundaries rather than by product names.Source: §5.6 · p. 18

6 Runtime Test Semantics and Conformance Protocol

The runtime keeps participant decisions, agent intents, admission, effects, reconciliation, and fulfillment distinct. Versioned authority, contracts, world state, and evidence govern consequential actions across digital and physical substrates.

6.1 From participant events and ActionIntent to evidence-coupled verdicts

  • Participant decisions are canonical evidence-bearing events that can change delegation, review state, or observations, but are not scored as AUT actions unless the study includes that actor in the SUT.An ActionIntent and a participant decision follow distinct event paths.Source: §6.1 · p. 19
  • Admission uses a fail-closed priority: DENY dominates ESCALATE, which dominates HOLD, and only an all-PASS applicable guard set yields ALLOW.Guard evaluation uses version-pinned state and evidence; stale views or failed resource reservations require re-observation and fresh admission.Source: §6.1 · pp. 19–20

6.2 Action lifecycle, factual authority, and reconciliation

  • Normative authority determines what an actor may do, while factual authority determines the best evidence-backed record currently accepted under a declared reconciliation policy.Acknowledgment does not prove a physical effect; accepted facts remain subject to the evidence and policy available at that time.Source: §6.2 · pp. 20–21
  • Reconciliation policies must specify admissible sources, conflict and timeout handling, review authority, and acceptance conditions; policy changes can alter comparison validity.A reconciled fact is not absolute truth, and new evidence is appended rather than rewriting the historical record.Source: §6.2 · p. 21

6.3 Delegation and collaboration guards

  • Delegation bounds a role's discretion, whereas collaboration contracts specify what independently accountable parties owe one another and how fulfillment or disputes are handled.Revoking delegation changes prospective authority but does not erase an existing collaboration obligation.Source: §6.3 · pp. 21–22

6.4 HOLD, ESCALATE, contested closure, and reference algorithm

  • HOLD and ESCALATE keep an action blocked pending a fresh admission, and an episode with a mandatory contested or unknown fact cannot be labeled Fulfilled.A later approval triggers fresh admission rather than mutating the prior decision.Source: §6.4 · p. 22

6.5 Reality substrates, rollback, and physical compensation

  • Physical events cannot be rolled back; counterfactuals after field events use a digital reconstruction and remain model-based rather than reruns of physical history.The protocol freezes model identity, variable bindings, solver settings, state, authority context, and evidence for industrial bindings.Source: §6.5 · p. 25

6.6 Experiment Cell runtime, scenario semantics, and evidence boundary

  • Canonical Evidence Records capture boundary decisions and effects, while derived evaluation interprets those records; transport success alone establishes neither authorized transitions nor fulfilled obligations.The credential boundary assumes complete mediation and isolated credentials; security assurance still requires separate implementation and adversarial validation.Source: §6.6 · pp. 25–26

7 Test Workflow and Constructed Experiment Designs

The workflow configures and freezes studies, then evaluates episodes and feeds reviewed incidents into versioned research assets. Constructed travel designs illustrate how identical outputs can differ in authorization and fulfillment.

7.1 Study lifecycle; 7.2 Replay and counterfactual fork

  • A Study declares its question, comparator, domain, scenario, actor realizations, controls, reality and grounding profiles, frozen run manifest, and evaluation plan.Replay reconstructs authoritative history from a deterministic recorded world; a counterfactual fork changes one declared factor but does not itself prove causal identification.Source: §7.1 · pp. 26–27, §7.2 · p. 27

7.3 Failure attribution

  • Failure attribution distinguishes agent choices from authority, contract, world-model, environment, substrate, evaluator, governance, or human-proxy causes.Categories can overlap, allowing primary and contributing causes rather than forcing every unresolved observation into agent success or failure.Source: §7.3 · p. 27

7.4 Feedback assetization

  • Real incidents become reusable assets only after source and permission checks, minimization, expert and causal review where possible, and ethics/safety review.The Lab evolves through reviewed, versioned releases rather than silently changing the world during ongoing experiments.Source: §7.4 · p. 28

7.5.1 Study setup

  • The travel study models a delegated agent, multiple providers, bookings and payment, and rules for disruption, rebooking, refunds, evidence, and closure.Automatic rebooking is limited by a declared extra-cost budget, while nonrefundable purchases above the threshold require renewed approval.Source: §7.5.1 · p. 28

7.5.2 A constructed witness: same output, different fulfillment

  • In the unresolved travel trace, the agent buys without required approval, retains conflicting carrier authority, relies on unverified accessibility evidence, and closes with an unresolved obligation.The comparison trace obtains approval, reserves budget, reconciles the old hold, retains provider confirmation, and discharges residual obligations before closure.Source: §7.5.2 · p. 29
  • Identical final travel itineraries and purchase prices can mask different fulfillment: output equivalence is insufficient for fulfillment equivalence.The constructed trace demonstrates representability and machine checking, not practical superiority.Source: §7.5.2 · p. 29

7.5.3 Failure diagnosis and agent revision

  • A travel agent proposing a replacement above its delegated cost limit is denied before dispatch, diagnosing authority constraints missing from proposal planning rather than flight-search inability.The same world, disturbance, and evaluator can be reused across revised agent versions.Source: §7.5.3 · pp. 29–30

7.5.4 A strict single-variable ablation

  • A strict mechanism ablation holds agent, prompt, tools, world, seed, delegation, and hidden events fixed while changing only the specified control.The practical claim must be narrowed if the guard fails to reduce unauthorized dispatch or unresolved obligations, or its review burden causes greater total service loss.Source: §7.5.4 · p. 30

7.6 Human-role proxy sensitivity test

Proxy sensitivity is tested by changing one human-role realization under matched conditions and reporting whether agent conclusions depend on that choice.

7.6 Human-role proxy sensitivity test

  • The matched proxy-sensitivity test freezes the AUT/SUT, scenario, world, contracts, environment actors, disturbances, and primary measures while changing one human-role realization.It uses at least two behaviorally distinct proxies and, where feasible, a human trace or smaller real-human condition as a grounding anchor.Source: §7.6 · p. 30
  • A direction change under proxy realizations triggers reporting, but is not an inferential significance test or proof of faithful human modeling.Empirical claims should add repetitions, uncertainty intervals or effect magnitudes, and appropriate inference when study scale permits.Source: §7.6 · pp. 30–31
  • A triggered conclusion is bounded to tested participant realizations until further human grounding, and the test does not establish proxy fidelity.It identifies whether the agent-quality conclusion may depend on how the human role was instantiated.Source: §7.6 · p. 31

7.7 Industrial agent: separating substrate sensitivity from agent improvement

The industrial design separates changes in the environment substrate from changes to the agent, then extends testing through shadow feedback and targeted HIL. These are constructed designs for staged investigation, not measured field results.

7.7.1 Study setup

  • A production-recovery episode coordinates purchasing, scheduling, inventory, and logistics across multiple principals and enterprise systems under physical constraints.The software-native setup uses twins for ERP, MES, and WMS, with a business-rule machine model.Source: §7.7.1 · p. 31

7.7.2 Substrate-sensitivity experiment

  • Changing the machine realization while holding the agent and study conditions fixed tests substrate sensitivity separately from agent improvement.The FMU binding is version-pinned; the migration also discloses changes to time, observable state, constraints, rollback, and evidence.Source: §7.7.2 · p. 31, Table 14 · p. 32
  • A schedule that passes business rules may violate a thermal constraint in the mechanistic model, illustrating how substrate changes can alter conclusions without changing the agent.The example is constructed, not a measured field result.Source: §7.7.2 · p. 31

7.7.3 Agent-improvement experiment

  • Only after fixing the FMU realization does the design compare a mechanism-aware agent revision, isolating agent design under a common substrate.The revised agent evaluates candidate schedules against the mechanistic model before issuing a scheduling intent.Source: §7.7.3 · p. 32

7.7.4 Shadow feedback and Lab self-correction

  • Read-only shadow operation exposes live assumptions while consequential intents remain simulation-only, and evidence must distinguish agent failure from model omissions or ambiguous real-world evidence.Reviewed incidents can become versioned regression scenarios, parameter sets, validators, or calibration assets.Source: §7.7.4 · p. 32

7.7.5 HIL and controlled physical validation

  • HIL is warranted when the question depends on controller timing, communication delays, sensor noise, or hardware execution, while the rest of the study can remain digital.Increasing physicality is tied to the claim under test, not a universal maturity sequence.Source: §7.7.5 · p. 32

8 Evaluation, Conformance, and Implementation Status

The section specifies reporting, falsifiable research questions, conformance artifacts, and a minimum path toward empirical validation. The released runner checks only a selected subset; the paper reports neither agent-quality results nor a running Living Lab prototype.

8.1 Minimum reporting vector and measurement discipline

  • The reporting vector calls for explicit measures of action attempts, human review, evidence coverage, commitment recovery, and stakeholder value rather than collapsing all quality into one score.Studies should report harness overhead when performance matters, variation and tail loss, and whether conventional automation suffices; hypotheses H1–H6 remain future questions.Source: §8.1 · p. 33

8.2 H1–H6: falsifiable questions, not established results

  • H1–H6 are falsifiable questions, not established findings, covering relational state, dual contracts, persistence, evidence, substrate stability, and participant/world grounding.The proposed comparisons include false rejections, review cost, human attention, and whether effects persist across declared substrate migrations.Source: §8.1 · p. 33, §8.2 · p. 34

8.3 Normative reference-interchange schemas and conformance artifacts

  • The reference package makes selected interchange semantics machine-readable through normative schemas and deterministic fixtures, while temporal rules require the schemas, runtime transitions, and expected-output fixtures together.C3/C7 remain prose oracles rather than released machine-readable fixtures.Source: §8.3 · pp. 35–36, Table 16 · p. 36
  • The small runner executes only the released fixture subset and cannot dispatch actions or verify actual side effects, so it is not a World Kernel, security proof, or agent evaluation.Unsupported admission and closure cases raise errors rather than defaulting to ALLOW or FULFILLED.Source: §8.3 · p. 36

8.4 Minimum empirical validation design required for stronger claims

  • The proposed minimum empirical study compares runnable agents under matched conditions and reports task success alongside fulfillment, authority, evidence, human-work, value, and failure-attribution measures.A second design holds the SUT fixed while varying human-role realization or Reality Substrate; these are proposed tests, not completed experiments.Source: §8.4 · pp. 36–37

8.5 Implementation status

  • The paper validates selected object-level semantics but supplies no running Living Lab prototype, adapter SDK, or working FMU/HIL/field integrations.Persistent world execution, remote reconciliation, a production Human Proxy Factory/Real Human Bridge, and stakeholder governance are also not implemented.Source: §8.5 · p. 37, Table 17 · p. 37

9 Discussion and Conclusion

The discussion defines the Living Lab as a stakeholder-connected testing infrastructure for service systems, while emphasizing that proxy and substrate choices bound what results support. The paper offers a method and reference architecture, not empirical proof of superiority or platform completeness.

9.1 Why a Living Lab if the paper is about testing?

  • A Living Lab connects testing to stakeholders who co-define goals, exercise authority, calibrate proxies, review disputes, and govern real incidents into future test assets.Repeated runs need not include real humans if the program maintains a path to stakeholder grounding, proxy challenge, and feedback.Source: §9.1 · p. 38

9.2 Why not simply call this Agent testing?

  • The method targets service systems where identifiable principals, service relationships, persistent state, obligations, or physical consequences make authorization and fulfillment part of the test oracle.Generic tool-use benchmarks remain useful without this protocol when their claims do not depend on those properties.Source: §9.2 · p. 38

9.3 Reproducibility, human realism, and external validity

  • Digital tests favor reset and repeatability, while human and physical conditions contribute authentic timing, preferences, workarounds, noise, institutions, and consequences.Proxies can distort cooperation, language, risk perception, timing, and escalation, so a result against one proxy is scoped to that declared realization unless broader evidence supports generalization.Source: §9.3 · p. 38

9.4 Boundaries to adjacent systems

  • The method is portable to existing benchmarks and platforms that preserve its episode, participant, authority, and evidence semantics; its claim is not exclusive capability.Digital twins, FMUs, and HIL are treated as Reality Substrates within the method.Source: §9.4 · pp. 38–39

9.5 Limitations and next empirical step

  • The constructed designs and limited schemas, fixtures, and runner do not establish practical superiority or platform completeness, and four central limitations remain empirical or unresolved.These include independent implementation conformance, role-dependent proxy validity, factual adjudication, and whether Semantic Delta supports causal comparability.Source: §9.5 · p. 39

9.6 Conclusion

  • The method organizes tests around the episode, the actors and roles, and the reality and grounding of execution; its distinctions aim to make evaluation more falsifiable, diagnosable, and portable.A useful outcome is not necessarily authorized, a completed run is not necessarily fulfillment, and a proxy is not necessarily faithful.Source: §9.6 · p. 39

A Normative Run Manifest and Evidence Record Profile Summary

The profile specifies canonical records for runs, participants, world realizations, and evidence. These records preserve the versions, bindings, and references needed to interpret a run without prescribing internal implementation choices.

Run Manifest

  • A Run Manifest identifies the run, study, release, world snapshot, scenario and seed, SUT artifact, participant realizations, relationship contracts, policies, time model, and evaluator configuration.It also records realization and grounding profiles and may include a Semantic Delta.Source: Run Manifest · p. 40

Participant Realization Profile

  • The Participant Realization Profile records SUT/AUT membership, environment actors, human-role realizations, decision rights, proxy calibration references, and an optional Real Human Bridge.Proxy calibration records can include asset identity, population and sampling, procedures, permitted use, access, and privacy constraints.Source: Run Manifest · p. 40, Participant Realization Profile · pp. 40–41

Reality Substrate and object-level Reality Binding

  • Object-level Reality Bindings record substrate and adapter versions, capabilities, endpoints or artifacts, I/O and time bindings, policies, and any Semantic Delta reference.A capability declaration states what an adapter claims to support; operational conformance remains an implementation property.Source: Run Manifest · p. 40, Reality Substrate and object-level… · p. 40

Evidence Record

  • An Evidence Record captures episode sequence, principal-role-carrier bindings, relevant versions, intent and admission, action or fact state, world changes, reconciliation, provenance, and timestamps.Correlation and causation identifiers support reconstruction of the episode boundary.Source: Evidence Record · p. 41
  • Implementations may use richer records, but must export the canonical meaning without semantic loss for fields required by the relevant conformance case.The profile preserves observation, effect, and reconciliation references as applicable.Source: Evidence Record · p. 41

B Conformance Cases and Fixture Index

The conformance materials make a selected normative subset executable through deterministic fixtures and a small reference runner, while retaining other cases as prose oracles. They test authority, obligations, and uncertain effects rather than treating task completion alone as sufficient.

B Conformance Cases and Fixture Index

  • Seven deterministic fixtures and a small reference runner make the released normative subset executable and inspectable, but do not constitute a full Lab kernel.Additional cases remain prose oracles.Source: B Conformance Cases and Fixture Index · p. 41

Released fixtures

  • Released fixtures cover denial under revoked authority, waitable evidence holds, authorized escalation, human revocation, and preserving active commitments when replacing a carrier.A successor action remains blocked until fresh authority exists; task-state copying must not erase a surviving commitment.Source: Released fixtures · pp. 41–42
  • Missing closure evidence prevents Fulfilled status, while an unverified remote or physical effect remains Unknown rather than being fabricated as success or blindly redispatched.Residual obligations and evidence gaps remain visible.Source: Released fixtures · p. 42

Prose conformance oracles retained for future machine fixtures

  • Prose oracles cover shared-resource contention and contested mandatory facts; a complete future suite would add machine inputs and expected outputs for both, plus independent implementations in two runtimes.Only one reservation may succeed under the declared concurrency policy, and contested facts block Fulfilled closure pending review or explicit non-fulfilled/disputed termination.Source: Prose conformance oracles retained for… · p. 42

C Protocol and Standard Mapping

Table 18 maps representative standards and technologies to integration questions rather than claiming novelty or precedence. The protocol binds selected capabilities to a versioned service episode and its authority, obligations, substrate, and evidence boundaries.

C Protocol and Standard Mapping

  • Table 18 is a reuse and integration map: existing standards and research families supply mechanisms, while the protocol specifies how selected capabilities bind to service episodes and their state and evidence boundaries.The bindings include versioned episodes, authority and obligation state, substrate declarations, and evidence boundaries.Source: C Protocol and Standard Mapping · p. 42, Table 18 · p. 43
Loading 2610.35978v1…