PaperlayerView exact revision on arXiv

Source-linked AI summary

TUPLE-Bench: Evaluating and Mitigating Privacy-Utility Trade-offs in Typed Decision Models

Yiqu Yang, Qiaoyuan Zheng, Yang Tian, Yu Fan, Elliott Ash

arXiv:2610.23668v1cs.CRcs.AI

TL;DR

Typed decision readouts support downstream decisions but can reveal information beyond a recipient’s authorization. TUPLE-Bench audits leakage and utility across release views and introduces WCPQ; full-native outputs increased secret recovery over choice-only releases while reducing cost, and WCPQ reduced leakage while preserving specified workflow behavior and cost.

  • Problem

    Decision interfaces expose probabilities and sometimes confidence as well as a selected category, raising the question of what information recipients are authorized to learn from these fields.

    Source: §1 · p. 1

  • Method

    TUPLE-Bench evaluates paired synthetic cases through refusal and coarse-to-fine audits of release views, then tests WCPQ for leakage reduction under specified workflow constraints.

    Source: §1 · p. 1, §8 · p. 8

  • Results

    Full-native outputs increased secret recovery over choice-only releases by 11.46–23.61 percentage points across nine models while reducing mean downstream cost in all 27 model–task comparisons.

    Source: §9 · p. 8

  • Takeaways & Limitations

    Confidentiality should be assessed at the level of released fields and specified workflows, since correct withholding and preserved workflow behavior do not guarantee that protected information remains hidden.

    Source: §8 · p. 8, §9 · p. 8

    The synthetic cases, specified contracts, and constructed costs characterize disclosure under evaluated assumptions, not real-world leakage prevalence or deployment costs.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Typed decision models often return more than a selected category. They also expose probabilities for each candidate and, sometimes, a separate confidence score. These fields support downstream decisions but may reveal information beyond a recipient's authorization. We introduce TUPLE-Bench (Typed decision models' Utility and Privacy Leakage Evaluation Benchmark), a benchmark of paired synthetic cases in which recipients may learn a coarse operational category but not its fine-grained cause. Across nine models, instructions to withhold the cause fail to protect it: 91.88%-99.93% of responses reveal the cause through direct selection or residual scores. Even when causes are excluded from the candidate set, full outputs enable 62.85%-75.35% recovery on held-out cases, compared with 51.04%-57.64% using only the selected category (50% chance). Full outputs also reduce mean downstream cost in all 27 model-task comparisons. However, confidence fields increase leakage without consistently reducing cost. We further introduce workflow-constrained privacy quantization (WCPQ), which reduces secret recovery while preserving the behavior and cost of specified downstream workflows. Our findings highlight the need to audit decision interfaces field by field and workflow by workflow.

1 Introduction

Typed decision readouts can help downstream decisions while exposing information beyond a recipient’s authorization. TUPLE-Bench audits that leakage and introduces workflow-constrained privacy quantization (WCPQ) as a mitigation.

1 Introduction

  • The selected answer alone leaves open what information a decision interface should release, because readouts may reveal causes beyond recipients’ permissions.The motivating example distinguishes a permitted coarse outcome—parcel delay—from its fine-grained cause.Source: §1 · p. 1
  • Instructions to withhold a protected mechanism still left it recoverable through direct selection or residual scores after withholding.Across nine models, the probe recovered the mechanism from nearly every response.Source: §1 · p. 1
  • Across nine models, full-native releases increased protected-mechanism recovery by 11.46–23.61 percentage points over choice-only outputs while reducing downstream decision cost.The audit examines cases where protected mechanisms are excluded from the candidate set.Source: §1 · p. 1
  • TUPLE-Bench comprises 720 counterfactual pairs across three domains, with recipient-specific disclosure contracts and paired privacy–utility evaluation.The benchmark’s audits and workflow-constrained mitigation are summarized in Figure 1.Source: §1 · p. 1, Figure 1 · p. 2
  • The paper introduces WCPQ to canonicalize native responses while preserving specified downstream workflow behavior and cost, and empirically evaluates its leakage reduction.Source: §1 · p. 2

2 Related Work

Related work spans privacy inference and output defenses, contextual disclosure norms, confidence and decision utility, and typed decision readouts. This paper evaluates protected-mechanism recovery alongside downstream decision cost.

2 Related Work

  • Prior work studies inversion, sensitive-attribute and membership inference, label-only attacks, and output defenses, including privacy-preserving quantization.This work audits a runtime mechanism conditioned on an authorized category and constrains replacements by frozen workflow behavior rather than score distortion.Source: §2 · p. 2
  • Contextual and agent privacy benchmarks examine disclosure norms, while POLAR-Bench studies privacy–utility trade-offs under adversarial elicitation (Mireshghallah et al., 2024; Li et al., 2024, 2025; Shao et al., 2024; Zheng et al., 2026).Role-conditioned refusal probes also examine access-control reasoning (Klisura et al., 2026).Source: §2 · p. 2
  • Calibration and selective-classification research address confidence and prediction risk, whereas this paper measures downstream decision cost and protected-mechanism recovery.The related work cautions against assuming score suppression is always beneficial.Source: §2 · p. 2
  • Typed-decision probabilities can disagree with generated answers or violate coherence; this paper audits recipient-visible release views derived from the same native response (Li et al., 2026).Source: §2 · p. 2

3 Problem Setting

The problem setting separates the authorized coarse category from a protected mechanism and evaluates what recipients can infer from released decision fields. Privacy recovery and downstream cost are measured separately for each release.

3 Problem Setting

  • The model receives recipient-visible context X and private evidence E, while the protected mechanism S maps to an authorized coarse category T = ϕ(S).The decision question presents coarse T categories rather than fine-grained S values.Source: §3 · p. 2
  • Each paired world shares public context and authorized outcome T but differs in protected mechanism S and private evidence E; release views derive from one saved native response.Each release is assessed for downstream utility and conditional protected-mechanism recovery.Source: Figure 2 · p. 3
  • The native response contains a selected category, scores over coarse candidates, and possibly a confidence field; a release mechanism maps it to recipient-visible information.The authors do not assume scores or confidence are calibrated.Source: §3 · p. 3
  • The legitimate consumer observes context, costs, and the released object, while the privacy decoder additionally receives authorized T and the two registered mechanism hypotheses.Conditioning on T and the closed hypothesis set isolates leakage beyond the authorized category.Source: §3 · p. 3
  • For each release, the evaluation reports mean downstream cost and secret-recovery accuracy, plus recovery gain over no output, keeping privacy and utility separate.Figure 2 summarizes this coarse-to-fine evaluation.Source: §3 · p. 3
  • The refusal probe separately measures direct fine-mechanism selections and residual recovery after the model selects withhold.The model is instructed to withhold because disclosing the mechanism exceeds recipient permission.Source: §3 · p. 3

4 Benchmark Construction

TUPLE-Bench uses synthetic operational cases with recipient-specific permissions and paired worlds that share authorized information but differ in protected causes. Its construction includes consistency checks and multi-stage review.

4.1 Domains and recipient contracts

  • TUPLE-Bench contains 720 counterfactual pairs (1,440 worlds) across delivery, equipment-service, and IT operations; each domain has four authorized classes and sixteen protected mechanisms.Table 1 presents coarse categories and example protected mechanisms; the coarse-to-fine audit offers only coarse categories as candidates.Source: §4.1 · p. 3, Table 1 · p. 4

4.2 Paired cases and consistency checks

  • Pairs are registered before their two worlds are generated, with distinct mechanisms mapped to the same authorized category and corresponding private evidence.The paired worlds are represented as Wj = (X, B, Ej, S = sj, T), j ∈ {0, 1}.Source: §4.2 · p. 4
  • Each pair shares public context, costs, authorized category, recipient permissions, candidate text and order, and permitted-check laws, while differing in mechanism and private evidence.Both worlds and their derived renderings remain within the same train, validation, or test split.Source: §4.2 · p. 4, §4.1 · p. 4

4.3 Quality Control

  • Cases are AI-generated hypothetical events, reviewed by three independent judges through blind inference, contract verification, and diversity screening.Acceptance requires agreement on intended labels and satisfaction of pair-level criteria; the dataset underwent repeated review and targeted repair.Source: §4.3 · p. 4

5 Evaluation Protocol

The evaluation audits both refusal-related residual scores and the information exposed by different release views. It measures privacy through secret recovery and utility through frozen downstream policies across three action families.

5.1 Refusal and residual-score audit

  • The refusal probe measures withholding and recovery from direct selections or residual scores, including recovery conditional on withholding.Residual recovery counts only a unique, correct top remaining mechanism score with positive fine-score mass; ties and zero mass count as failures.Source: §5.1 · p. 4

5.2 Readouts and release policies

  • Five release views range from no prediction fields to choice, confidence, probabilities, or all native fields, derived from the same saved response.Using the same response avoids repeated-call variation.Source: §5.2 · pp. 4–5

5.3 Legitimate downstream utility

  • Three downstream action families evaluate utility: selective routing, one-step clarification, and asymmetric-cost routing.The asymmetric-cost task tests the value of probabilities beyond the selected label; wrong routes cost 10, trusted review 2, and clarification checks 0.4 authored benchmark units.Source: §5.3 · p. 5

5.4 Secret inference decoding

  • Secret inference decoders identify the protected mechanism from each release, given the authorized category and two registered mechanism hypotheses.Decoders are trained separately by system, domain, and view; test worlds are decoded independently without private evidence, gold mechanism, case or pair identifiers, or paired responses.Source: §5.4 · p. 5

6 Workflow-Constrained Privacy Quantization (WCPQ)

WCPQ transforms native responses into canonical outputs while preserving the behavior and cost of specified frozen workflows. It chooses a replacement from an ordered codebook that matches the original workflow signature.

6 Workflow-Constrained Privacy Quantization (WCPQ)

  • WCPQ is a trusted-backend post-processing method that maps native responses to releases without accessing the protected mechanism or private evidence at release time.It canonicalizes outputs while preserving specified downstream workflow behavior and cost.Source: §6 · pp. 5–6
  • The workflow signature records selected category, field availability, and routing, review, and clarification actions for every possible check outcome.WCPQ constructs an ordered codebook from a fixed probability grid and training responses, including valid confidence replacements; each codeword retains the input's selected category.Source: §6 · p. 6
  • WCPQ selects the first codeword that preserves the complete workflow signature, validates the replacement, and gives covered responses with shared public context and signature the same canonical output.The matching set contains codewords whose workflow signature equals the native response's signature.Source: §6 · p. 6
  • Preserving all workflow branches guarantees equal cost for each specified workflow and hypothetical coarse truth, including correct and incorrect decisions.The guarantee applies only to the specified frozen workflows.Source: §6 · p. 6
  • The defense-aware evaluation retrains secret decoders on defended releases and tests all worlds, including native-response fallbacks, while workflow preservation alone does not guarantee confidentiality.Attackers know the codebook, mapping, and frozen workflows; required actions and fallback responses may still reveal the protected mechanism.Source: §6 · p. 6

7 Results

Across nine models, refusal, ordinary outputs, and downstream workflows expose distinct privacy–utility patterns. WCPQ reduces recovery while preserving specified workflow behavior and cost.

7.1 Refusal does not prevent score leakage

  • 91.88%–99.93%: overall recovery across nine models combines correct direct selections with correct residual recovery after withholding.Among the eight models that select withhold, conditional residual recovery is 87.53%–99.91%; Sage and Mercury Decide withhold on 98.96% and 97.92% of responses yet overall recovery remains 98.26% and 99.51%.Source: §7.1 · p. 6

7.2 Scores reveal mechanisms absent from the candidate set

  • 62.85%–75.35%: full-native recovery exceeds choice-only recovery of 51.04%–57.64%, by 11.46–23.61 percentage points across all nine models.No output recovers exactly 50% because paired worlds share public inputs; full-native recovery varies by domain, including 53.13% in delivery and 78.13% in equipment for d1.Source: §7.2 · p. 6, Figure 4 · p. 7

7.3 Probabilities improve utility; confidence is less consistent

  • 9.52%–75.68%: full native lowers pooled mean business cost relative to choice in all 27 model–task comparisons.Confidence alone is not uniformly useful: TypeSafe's choice+confidence raises selective-routing cost from 0.278 to 0.285, while Sage's raises clarification cost from 0.111 to 0.147.Source: §7.3 · p. 7

7.4 Privacy–utility trade-offs depend on model and task

  • Privacy–utility frontiers vary by task: OpenJev has the lowest recovery at 62.85% but not the lowest cost in any task, while the non-dominated set changes with task.Choice has lower recovery but higher pooled cost than full native in all three tasks; choice-only clarification is non-inferior for six of nine models, whereas no choice-only selective or asymmetric policy is.Source: §7.4 · p. 7, Figure 5 · p. 8

7.5 WCPQ reduces recovery while preserving workflow costs

  • 10.07–24.65 percentage points: WCPQ reduces recovery relative to native responses while preserving every action branch and cost across all 18 frozen workflow/cost combinations.Codebook coverage is 99.31% for TypeSafe and 100% for OpenJev and Clef; complete workflow plans agree in every test case.Source: §7.5 · p. 7, Table 3 · p. 8

8 Discussion

Confidentiality depends on the full released object: withholding can coexist with residual-score leakage, and probabilities can reveal protected mechanisms even when they are not candidate labels. Release choices must also account for downstream cost, confidence-field leakage, and the limits of workflow-preserving mitigation.

8 Discussion

  • Protected mechanisms can be recovered from residual scores despite correct withholding, and probability readouts can reveal them even when they are absent from the candidate set.Confidentiality therefore needs to be assessed at the level of the released object, not just the selected label.Source: §8 · p. 8
  • Choice-only releases meet the prespecified cost tolerance for six of nine models in one-step clarification, but not in selective or asymmetric-cost routing.Confidence fields increase recovery without consistently reducing cost.Source: §8 · p. 8
  • Model- and workflow-specific release policies are motivated over uniformly suppressing or exposing all scores.The findings connect leakage and utility to the release decision.Source: §8 · p. 8
  • WCPQ can remove some privacy-relevant distinctions while preserving specified workflow actions and costs, but required actions and fallback responses may still reveal protected information.Workflow preservation does not guarantee confidentiality.Source: §8 · p. 8

9 Conclusion

TUPLE-Bench audits typed decision readouts through refusal and coarse-to-fine evaluations, then assesses workflow-constrained mitigation. Across nine models, full-native releases raise secret recovery relative to choice-only outputs while lowering mean downstream cost in every model–task comparison.

9 Conclusion

  • Full-native outputs increase secret recovery over choice-only releases by 11.46–23.61 percentage points across nine models, while reducing mean downstream cost in all 27 model–task comparisons.Confidence fields increase leakage without consistently improving utility.Source: §9 · p. 8
  • Correct withholding can still expose protected mechanisms through residual scores, and full-native outputs can reveal secrets even when protected mechanisms are not candidate labels.The evaluation uses refusal and coarse-to-fine audits with release views derived from the same native response.Source: §9 · p. 8
  • WCPQ empirically reduces leakage while preserving specified workflow behavior and cost, motivating field-by-field and workflow-by-workflow audits of decision interfaces.Source: §9 · p. 8

Limitations

The evaluation has boundaries in its synthetic setup, response sampling, and attack scope. Its results characterize specified assumptions and evaluated attacks rather than establishing real-world validity or a formal confidentiality guarantee.

Limitations

  • AI-generated synthetic cases, role-specific disclosure contracts, and constructed costs bound the findings to specified assumptions, not real-world leakage prevalence or deployment costs.The evaluation covers only a few domains and does not establish real-world validity or statistical independence across pairs.Source: Limitations · p. 9
  • Each world has one saved response, so repeated-call variability is unmeasured, and bootstrap intervals exclude refitting and selection uncertainty.Source: Limitations · p. 9
  • Passive, closed-set attacks exclude open-set extraction, adaptive querying, timing, and provider logging; recovery reflects the evaluated attacker library, not a formal confidentiality guarantee.Utility also depends on specified costs and idealized checks, while confidence semantics vary across providers.Source: Limitations · p. 9

Ethical Considerations

The evaluated attacks use fictional, researcher-controlled cases rather than other customers’ data. The setup distinguishes information available to the evaluated model from information available to the recipient-side attacker.

Ethical Considerations

  • The attacks use fictional, researcher-controlled cases, not other customers’ data.Source: Ethical Considerations · p. 9
  • In the coarse-to-fine audit, fine-grained labels are registered attacker hypotheses, not production candidates; the refusal probe exposes them as candidates only to test withholding.Source: Ethical Considerations · p. 9
  • Private evidence is supplied to the evaluated model, not the recipient-side attacker, and published artifacts should exclude credentials and unrelated raw logs.Artifacts should disclose model versions and call accounting and distinguish recipient-facing leakage from provider-side data handling.Source: Ethical Considerations · p. 9

AI Assistance Statement

The appendix documents AI assistance and specifies refusal and coarse-to-fine audits, recipient contracts, counterfactual pair construction, and quality controls for the benchmark.

AI Assistance Statement

  • The authors used generative AI for implementation, literature searches, synthetic case generation, quality control, formatting, and numerical checks, and reviewed the final materials.They take responsibility for the final content and artifacts.Source: AI Assistance Statement · p. 9

A Refusal Probe Specification

  • The refusal probe reuses registered cases and mechanisms, tests withholding when fine mechanisms remain explicit candidates, and checks whether residual scores still reveal them.It distinguishes selected-label compliance from response-level confidentiality.Source: A Refusal Probe Specification · p. 11, §A.1 · p. 11

A.1 Shared cases and refusal rendering

  • Each refusal case preserves its context and protected mechanism but offers sixteen fine mechanisms plus WITHHOLD, rather than the four coarse production candidates.The recipient remains unauthorized to learn the protected mechanism.Source: §A.1 · p. 11

A.2 Selected-label compliance

  • Selected-label compliance is assessed separately from policy violations and correct direct mechanism selections, with valid-response rates reported for all nine models.An incorrect fine-grained selection violates the withholding policy but does not count as correct secret recovery.Source: §A.2 · p. 12

A.3 Residual-score recovery

  • Residual recovery measures correct mechanism identification among WITHHOLD responses; it is undefined when that subgroup is empty, and ties or zero fine-score mass receive no credit.Recovery requires a unique maximum matching the protected mechanism.Source: §A.3 · p. 12

B.1 Compiled views and schema; B.2 Recipient contracts

  • The compiled views separate model inputs, consumer-visible context, and evaluator-only labels, while recipient contracts define confidentiality relative to an authorized coarse task.The three-domain workflow routes among four coarse classes and may include trusted review or an approved check; the frozen ontology has four coarse classes and sixteen mechanisms per domain.Source: §B.1 · p. 12, §B.2 · p. 12, Table 5 · p. 13

B.3 Counterfactual equivalence

  • Counterfactual certificates hold public context, coarse category, and downstream costs and check laws constant while protected mechanisms differ, preventing the utility channel from depending on the secret.The certificates verify equivalence across permitted actions and checks.Source: §B.3 · p. 13

B.4 LLM-as-a-judge quality-control protocol

  • Quality control uses blind inference, separate contract verification, and diversity screening; a pair passes only when all eleven criteria and both blind inferences agree without unresolved alternatives.Semantic review complements structural certificates but does not establish statistical independence.Source: §B.4 · p. 14

C Utility-Policy Specification

This section specifies how released fields support downstream workflows and how secret-recovery attacks and evaluation setups are defined. It also describes workflow-preserving privacy quantization and alternatives, together with audit and implementation checks, noting that preserving declared utility does not by itself guarantee confidentiality.

C Utility-Policy Specification

  • Policies are selected on validation data and frozen before test scoring; selective routing chooses a coarse route or trusted review under a shared task and cost structure.Candidate posterior estimators include an empirical training prior and regularized logistic regression; full-vector views may also use the native probability vector.Source: C Utility-Policy Specification · p. 15, §C.1 · p. 16

C.2 One-step clarification

  • One-step clarification lets a policy act immediately or purchase a permitted class-targeted check before routing; the selected check, answer, and final action form a visible transcript.Each domain permits four binary checks, which return yes exactly when the corresponding coarse class is true; an incorrect final route costs 10.Source: §C.2 · p. 16, Figure 7 · p. 16

C.3 Asymmetric-cost routing

  • Asymmetric-cost routing selects actions using a predeclared nonuniform action-by-class cost matrix that is fixed before testing and shared across release views within each task.The matrix entries are authored benchmark units, not empirical estimates of operational loss; an evaluator-only oracle-T policy is a reference bound, not a deployable release.Source: §C.3 · pp. 16–17

C.4 Utility tolerance

  • The utility tolerance is an absolute non-inferiority margin of δ = 0.1 cost units against a richer reference release, assessed using the upper endpoint of the paired 95% cost-difference interval.The margin is frozen before test evaluation; a small point estimate alone does not satisfy the descriptive interval check.Source: §C.4 · p. 17

D.1 Secret inference decoding

  • Secret-inference attacks identify one of two registered mechanisms on held-out cases using permitted release fields, with five views ranging from no output to full native fields.For probability-bearing views, attack features may use raw, log-ratio, and residual representations; these transformations do not alter saved API responses.Source: §D.1 · p. 17, §D.2 · p. 18

D.5 Grouped out-of-fold fitting and stacking

  • Attacks are trained separately by model, domain, and release view, with grouped cross-fitting and validation selection before test scoring; uncertainty intervals use 2,000 pair-clustered bootstrap resamples.Reported recovery is empirical for the specified decoder, not an estimate of optimal attack power or a formal privacy guarantee.Source: §D.5 · p. 19, §D.7 · p. 19

D.6 No output baseline

  • 50% is the expected accuracy of uniform guessing, but the no-output baseline is the primary reference because paired worlds have identical inputs and opposite secrets.The baseline uses public information, the authorized coarse label, the registered hypothesis set, and the common status signal, but no prediction fields.Source: §D.6 · p. 19

E Models, APIs, and Native Response Fields

  • The evaluation covers nine systems, each with 1,440 valid coarse-decision responses containing probability and confidence fields; requests retain the recipient contract and four coarse candidates.The compiled instruction permits the recipient to learn the coarse business category, not the fine mechanism, and uses typed JSON requests.Source: E Models, APIs, and Native Response… · p. 20, §E.1 · p. 20

E.1 Call configuration / E.2 Native-response provenance

  • Gateway-mediated calls retain native model endpoints and disable caching; the final coarse-decision export retains 1,437 original responses and replaces three invalid records.Re-queries target schema validity rather than answer correctness, and each repaired case contributes only its replacement response.Source: §E.1 · p. 20, §E.2 · pp. 20–21

E.3 Field semantics

  • Missing fields are never reconstructed, and Sage’s confidence equals max_i p_i, so finite-sample attack differences from that redundant view reflect fitting or selection rather than new information.Other confidence fields follow their recorded API representation without assumptions of calibration or independence.Source: §E.3 · p. 21

F.1 Attacker input and hypothesis-order checks

  • The attacker uses only declared consumer-view inputs and excludes identifiers, private reports, gold secrets, and paired-world responses; privacy attacks remain distinct from utility-cost analyses.Validation selection is saved before test scoring, and the primary privacy results use validation-selected secret inference; refusal-probe scoring is separate from coarse-to-fine analysis.Source: §F.1 · pp. 21–22, §F.3 · p. 22

F.2 Fixed cost-profile sensitivity

  • Six fixed cost profiles replay the same native responses, with policies fitted per profile; profiles are not selected by test performance and do not validate operational costs.The replay varies public cost context, not prompts, and is distinct from varying the utility-tolerance margin δ.Source: §F.2 · p. 22

G Benchmark Comparisons

  • Table 12 compares benchmark designs across controlled secret pairs, conditional secret recovery, same-response field audits, and workflow costs.Its comparison defines privacy–utility tradeoff jointly, excluding privacy-compliance accuracy alone.Source: Table 12 · p. 23

H.1 Strict mapper and utility guarantee

  • Equal complete workflow plans let strict WCPQ preserve declared routes, review and check costs, and continuations, but required behavior or native-response fallback can still reveal secrets.The mapper releases the first code preserving all 18 frozen policies, or retains the native response if none matches; this guarantees behavior only for the declared policies and simulator.Source: Figure 9 · p. 22, §H.1 · p. 23

H.2 Learned bounded-cost extension

  • The learned extension enumerates 18 policies under three per-output cost budgets (α = 0, 0.04, 0.20); zero budget blocks cost increases but may change actions.Positive budgets allow utility loss; these are per-output bounds, not percentages of mean cost or the main paper’s non-inferiority margin.Source: §H.2 · p. 24

H.3 Independent probability-field baselines

  • Independent probability-field baselines quantize, smooth, or add Gaussian noise to probabilities while retaining choice and native confidence, with fresh defense-aware decoders for each release.Each operation starts from the same native response, is not sequential, and uses the same 18 frozen consumers; Gaussian outcomes average three predeclared seeds across 288 test worlds.Source: §H.3 · p. 24

H.4 Evaluation and additional results

Across the appendix’s audits, protected mechanisms are recovered through direct selections and residual scores, and full-native recovery exceeds choice-only recovery for all nine models. Strict WCPQ preserves specified workflow plans and realized costs in the tested consumers.

I.1 Overall recovery and complete nine-model counts

  • JevK5 recovery ranged from 86.67% in equipment service to 95.42% in delivery operations, while d1 ranged from 90.83% in delivery to 97.50% in equipment.The statistics use selected final responses; TypeSafe’s final 1,440 valid responses must not be described as 1,440 valid initial attempts.Source: §I.2 · p. 25, §I.1 · p. 25

H.4 Evaluation and additional results

  • 286/288 TypeSafe responses and 288/288 OpenJev and Clef responses were covered; strict WCPQ matched 5,184/5,184 response-policy comparisons per model and preserved costs across six profiles.For TypeSafe, 283/288 payloads changed; three covered responses already equaled their selected codeword.Source: §H.4 · pp. 24–25, Table 13 · p. 26

I.2 Refusal-residual results

  • 99.10% of JevK5 withheld responses had a unique correct residual winner, compared with 100% for Clef, Clef-flash, Decider, and Mercury Decide; remaining failures mostly had incorrect unique winners, not ties.All eight models with a nonempty withholding subgroup had positive fine-score mass on every withheld response, but nonzero mass alone did not establish successful recovery.Source: §I.2 · p. 26
  • 1,400/1,415 Sage recoveries and 1,403/1,433 Mercury Decide recoveries followed withholding, whereas all 1,396 verified OpenJev recoveries were direct selections.TypeSafe had 548 correct direct selections and 874 correct residual recoveries.Source: §I.2 · p. 26

I.3 Full-native cross-model privacy–utility results

  • OpenJev and Clef-flash are non-dominated for asymmetric and selective routing; clarification also includes TypeSafe, Sage, and Decider.Membership is descriptive, based on unrounded means, and does not compare release views.Source: §I.3 · p. 27

I.4 Primary releases and domain results

  • Paired full-native-minus-choice recovery intervals are entirely positive for all nine models; choice-plus-confidence intervals are positive for eight, with Mercury Decide’s interval including zero.These are fixed-fit, unadjusted comparisons and do not establish simultaneous significance across models.Source: §I.4 · p. 27

I.5 Complete-response and both-worlds-correct subsets

  • 60.85%–74.23% recovery persisted among pairs with correct coarse predictions in both worlds, suggesting coarse errors alone do not explain the signal.This post hoc diagnostic is descriptive, not the primary endpoint or an independent test.Source: §I.5 · pp. 27–28

I.6 Attack-selection robustness

  • All saved individual-candidate, equal-weight, and learned-OOF tracks exceeded 50% aggregate full-native recovery; the primary strategy was selected on validation, not by the largest test value.The track comparison concerns selection and ensembling, not equal effectiveness of every attack family.Source: §I.6 · p. 28

I.7 Independent probability-field ablations

This appendix compares adapted decoders and recipient policies across transformed releases and documents case generation and validation. It also specifies staged evidence and contract checks for paired cases, event-family comparisons, quotation rules, model-query references, artifact licensing, and a rubric for distinguishing supported defects from uncertainty.

I.7 Independent probability-field ablations

  • The ablations separately quantize probabilities, release only the top two scores, apply invertible temperature smoothing, or add normalized Gaussian noise while retaining choice and confidence.Top-two scores are not renormalized and omitted scores are not treated as zeros; the temperature decoder applies the public inverse. The comparisons use adapted recipients and decoders, not WCPQ’s frozen-workflow feasibility guard or per-output cost bounds.Source: §I.7 · pp. 28–31

I.8 Paired utility comparisons

  • At δ = 0.1, six clarification comparisons meet the prespecified upper-interval criterion, while none of the other task comparisons do.This descriptive non-inferiority check does not establish that other releases are significantly inferior and provides no simultaneous familywise guarantee.Source: §I.8 · p. 32, Table 33 · p. 40

I.9 Cost-profile sensitivity results

  • All 162 model–task–profile mean-cost differences favor full native, ranging from 0.0014 to 0.2778 benchmark cost units.The smallest advantage is Mercury Decide clarification under quarter-price checks; the largest is Clefflash selective routing under doubled review cost. These are repeated measurements on the same test cases, not independent replications or a per-example guarantee.Source: §I.9 · p. 33, Table 34 · p. 41, Figure 14 · p. 43

I.7 Independent probability-field ablations

  • Recovery was 201/288 for native, 183/288 for quantization, 200/288 for top-two, 201/288 for temperature, and 60.88% on average for noise.Quantization reduced recovery by 6.25 percentage points versus native (paired 95% interval: [−10.43, −2.43]); noise reduced it by 8.91 points ([−13.19, −4.97]). Quantization left authored asymmetric and selective costs and actual traces unchanged, but mean cost fell from 0.0611 to 0.0556 despite one world with increased cost, so this is not a per-world guarantee.Source: §I.7 · pp. 31–32, Table 31 · p. 39

J.1 Case generation

  • Case-generation instructions require paired worlds to share public context and differ only in the protected mechanism and its necessary diagnostic consequences.They call for concrete diagnostic evidence that distinguishes plausible mechanisms, varied event-level causal discriminants, and hypothetical facts not selected using evaluated-model responses or attack outcomes.Source: §J.1 · p. 44

J.2 Common validation instructions

  • Validation instructions require independent, evidence-based review within the stated synthetic task, with later-stage labels and model or attack outcomes withheld.Reviewers are told not to infer unstated facts or assume intended labels are correct, and not to demand field prevalence or real-world cost studies unless needed to resolve a supplied contradiction.Source: §J.2 · pp. 44–45
  • The common validation threshold requires sufficient, coherent, discriminating evidence rather than proof against every imaginable cause, and applies the same materiality standard to PASS, FAIL, and UNCERTAIN.A contradicted supplied claim is FAIL; comparably supported explanations or an absent necessary fact require UNCERTAIN, without double-counting wording preferences.Source: §J.2 · p. 45

J.3 Stage A: blind inference

  • Stage A withholds intended labels and the registered hypothesis pair, requiring separate blind T/S inferences for each world from its observations.The catalog is only a candidate ontology; use UNCERTAIN or OTHER when warranted, and do not assume the worlds share T.Source: §J.3 · p. 45

J.4 Stage B: contract verification

  • Stage B requires all eleven contract checks, both frozen Stage A label matches, and no unresolved alternative for PASS; any failed condition means FAIL.The checks cover business result, permitted treatment, secret disclosure, naturalness, causal evidence, alternatives, shared public information, labels, changed mechanism, approved checks, and diagnostic-family characterization.Source: §J.4 · pp. 46–47

J.5 Stage C: event-family comparison

  • Stage C compares every unordered pair in the complete same-T pool as related, independent, or insufficiently evidenced, and conservatively groups related and uncertain edges under a five-event cap.For twenty events, the matrix contains 190 pairwise relations; semantic comparison does not establish statistical independence or real-world validity.Source: §J.5 · pp. 47–48

J.6 Literal-excerpt format guidance

  • Quotation guidance requires literal excerpts from the supplied data and preserves warranted negative or uncertain judgments without changing acceptance conditions.Structured values must not be combined or paraphrased, and numeric or structural comparisons may be explained without a quote.Source: §J.6 · pp. 48–49

J.7 Model-query instructions

  • The model-query instructions refer readers to existing Appendix A refusal-probe and Appendix E coarse-decision instruction blocks rather than reproducing them.Source: §J.7 · p. 49

K Artifacts License

Loading 2610.23668v1…