PaperlayerView exact revision on arXiv

Source-linked AI summary

ReCast: Contract-Preserving Protection for Fixed-Interface Multimodal Reasoning

Bingchen Pei, Lichong Chen, Bingxi Zhao, Ziang Wu, Sirui Wang, Min Zhang, Yanhao Chen, Qingxu Liu, Qiang Gao, Chang-Tien Lu, Bo Gao

arXiv:2610.01184v1cs.CRcs.AIcs.CL

TL;DR

Private chart and speech inputs can expose sensitive content when sent to remote models, while text-only protections cannot satisfy fixed media interfaces and identity anonymization may leave task content intact. ReCast rewrites source-specific content while preserving task relations and modality, using local numerical remapping and operand recovery. On 4,000 held-out ChartQA and NMSQA examples, it retains 92.43% of unprotected accuracy with 7.95% solver-bound source-content leakage.

  • Problem

    Remote numerical reasoning over private charts and recordings requires media inputs that fixed service interfaces accept, while existing text-only and identity-focused protections leave this need unmet.

    Source: §1 · p. 1

  • Method

    ReCast uses a shared evidence-query record to jointly rewrite source content and remap numbers, reconstructs the required modality, then restores protected operands locally before executing the remote solver’s program.

    Source: §1 · p. 2

  • Results

    92.43% of unprotected accuracy was retained with 7.95% solver-bound source-content leakage on 4,000 held-out ChartQA and NMSQA examples.

    Source: §1 · p. 2

  • Takeaways & Limitations

    ReCast separates content protection from modality through a shared record while keeping inverse numerical recovery local for fixed-interface reasoning.

    Source: §5 · p. 10

    Correct recovery requires faithful evidence and correct operand and operation selection; decisions based on unpreserved differences or ratios need not transfer unless deferred until after restoration.

    Source: §3.3 · p. 6

Abstract

from arXiv · show

Remote multimodal models offer strong numerical reasoning capabilities over charts and speech, but sending private inputs risks exposing sensitive content. Text-only sanitization cannot directly satisfy fixed media interfaces, while identity anonymization leaves the underlying task content exposed. We introduce ReCast, an agentic plug-in framework that replaces source-specific content while preserving task-relevant relations and the required input modality. ReCast locally converts inputs into a shared textual evidence-query record, jointly rewrites entities and topics with a distilled 4B model, and substitutes values through a locally invertible, role-aware numerical map. A reconstruction agent generates and validates the required media from the protected record. The remote solver returns a program whose protected operands are restored locally before execution. On 4,000 held-out ChartQA and NMSQA examples, ReCast achieves 75.10% accuracy, retaining 92.43% of unprotected remote accuracy, while a model-based audit flags source-content leakage in 7.95% of solver-bound requests. It outperforms all evaluated local baselines, preserving the benefit of remote reasoning while reducing source-content exposure under existing media interfaces.

1 INTRODUCTION

Private numerical evidence often arrives as charts or recordings, but text-only protections do not satisfy fixed media interfaces and identity anonymization can leave task content exposed. ReCast jointly protects source-specific content while preserving task relations and modality, and retains 92.43% of unprotected accuracy with 7.95% solver-bound source-content leakage on 4,000 held-out examples.

1 INTRODUCTION

  • Remote multimodal models offer stronger numerical reasoning than on-device models, but fixed service schemas require chart images or speech waveforms.Protection must therefore extend beyond text to the media consumed by these services.Source: §1 · p. 1
  • Text-only protections cannot supply the required media, while media-level protections may preserve the underlying task content.Existing methods include visual instruction rewriting and symbolic scene representations, as well as speaker anonymization, speech perturbation, and learned image transformations.Source: §1 · p. 1
  • ReCast jointly replaces topics, entities, and values across evidence and query while preserving their relations and required modality.A reconstruction agent generates and validates the surrogate media; the remote solver returns a program whose protected operands are restored locally before execution.Source: §1 · p. 2, Figure 1 · p. 2
  • A model-independent evidence-query record links modular evidence capture, rewriting, and modality reconstruction without retraining.The distilled local 4B Rewriter jointly changes names and domain while preserving relations, and a role-aware numerical map supports exact answer recovery through local inversion.Source: §1 · p. 2
  • 92.43% of unprotected accuracy was retained, with 7.95% solver-bound source-content leakage across 4,000 held-out ChartQA and NMSQA examples.Distillation added 6.40 accuracy points and reduced leakage from 39.75% to 7.95%.Source: §1 · p. 2

2 RELATED WORK

Prior work addresses text privacy, local-remote collaboration, and media privacy, but ReCast’s related-work discussion emphasizes the need to preserve evidence-query relations while delivering protected content in the modality a fixed interface accepts.

2.1 TEXT PRIVACY

  • Text privacy methods range from word-level differential-privacy substitutions to coherent rewriting and adversarial anonymization.Task utility and semantic structure are also considered, but text protection alone does not provide a chart or waveform required by a fixed media interface.Source: §2.1 · p. 3

2.2 LOCAL-REMOTE COLLABORATION

  • Local-remote collaboration limits what a stronger remote model sees, while Zhang et al. (2026) recover answers from programs over substituted numbers in textual numerical-reasoning instances.ReCast extends that recovery scheme to media inputs and outputs using an evidence-query record that makes cross-field references explicit and supports reconstruction checks.Source: §2.2 · p. 3

2.3 MULTIMODAL AND MEDIA PRIVACY

  • Textual or symbolic intermediaries motivate explicit representations but cannot directly replace media required by fixed interfaces.Media-level methods preserve modality while targeting voice identity, sensitive speech segments, or visual privacy attributes; SPTI releases synthetic datasets rather than protected media-query pairs tied to a computation.Source: §2.3 · p. 3
  • ReCast transforms source-specific entities, topics, and values per request while retaining evidence-query relations for numerical reasoning.Reconstruction provides interface compatibility, while the inverse numerical map remains local for answer recovery.Source: §2.3 · p. 4

3 METHOD

ReCast separates source-specific content from task-relevant relations in a shared textual record, then reconstructs protected evidence in the modality required by the interface. A remote solver returns a program whose operands are restored and executed locally.

3 METHOD

  • ReCast captures evidence, jointly protects its semantic and numerical content, reconstructs the protected record in the required modality, and restores solver-program operands locally before execution.The framework uses a shared evidence-query record to connect these stages.Source: §3 · p. 4

3.1 TASK AND FRAMEWORK OVERVIEW

  • Requests aim to reduce exposure of source-specific entities, topics, wording, and values while retaining evidence-query associations needed for local answer recovery.The source and accepted release modalities can be text, image, or audio; another release modality is used only when the interface permits it.Source: §3.1 · p. 4

3.2 MULTIMODAL EVIDENCE CAPTURE

  • Local evidence capture extracts chart associations and queried attributes, transcribes speech, or normalizes text while preserving field boundaries and numerical spans.The shared record has id, modality, context, question, and choices fields, allowing capture plug-ins to be replaced if they remain schema-compatible.Source: §3.2 · p. 5

3.3 JOINT REWRITING AND NUMERICAL PROTECTION

  • Joint rewriting changes entities and topics across evidence and query while retaining assignments, cross-references, requested operations, and answer type; numerical literals remain unchanged at this stage.Chart and speech rewrites preserve modality-specific correspondences and links.Source: §3.3 · p. 5, Figure 3 · p. 5
  • A request-specific injective numerical map is locally invertible, preserves within-band order and grouped-year intervals, but not general differences, ratios, or cross-class comparisons.Repeated values receive the same replacement, and non-year integers from 0 to 15 use identity assignments.Source: §3.3 · pp. 5–6
  • The solver returns a program with protected operands, which are restored locally before execution; correct recovery requires faithful evidence and correct operand and operation selection.Decisions relying on unpreserved differences or ratios need not transfer unless deferred to program execution after restoration.Source: §3.3 · p. 6

3.4 MODALITY RECONSTRUCTION AND RELEASE VALIDATION

  • Reconstruction generates interface-compatible media from the protected record, targeting evidence fidelity rather than the original chart appearance or speaker identity.Charts use specification generation, local rendering, and review-guided repair; speech uses synthesis and transcription checks, while text is serialized directly.Source: §3.4 · p. 6, Figure 4 · p. 6
  • Release requires both protected-record validation and a local source-consistency check, but these operational checks are not a proof of semantic correctness or a formal privacy guarantee.Protected-record checks cover rendered or transcribed content; source checks compare restored assignments and dependencies with the original input.Source: §3.4 · pp. 6–7

4 EXPERIMENTS

Experiments evaluate ReCast’s reasoning accuracy and source-content leakage, then test cross-dataset generalization and pipeline components. The results show high retained remote accuracy with low detected leakage, alongside modality-dependent effects.

4.1 EXPERIMENTAL SETUP

  • Evaluation uses 2,000 held-out examples each from ChartQA and NMSQA, with ChartX and Spoken-MQA reserved for out-of-distribution testing.Source: §4.1 · p. 7

4.2 MAIN RESULTS

  • 6.40 accuracy points over untuned Qwen3-4B accompanies leakage dropping from 39.75% to 7.95%; RLAA and Eternis outputs retain 80.25% and 94.95% source content.Source: Table 1 · p. 8, §4.2 · p. 9
  • RECAST beats Codex CLI by 10.10 points overall and DeepSeek Harness by 4.40 overall, but trails DeepSeek by 0.50 on NMSQA.Source: Table 2 · p. 8, §4.2 · p. 9

4.3 CROSS-DATASET EVALUATION

  • 62.40% on ChartX and 79.40% on Spoken-MQA correspond to drops of 7.80 and 0.60 points; leakage is reported descriptively across differing datasets.Source: Table 3 · p. 8, §4.3 · p. 9

4.2 MAIN RESULTS

  • 75.10% overall accuracy (NAcc. 92.43%) with 7.95% overall solver-bound leakage shows RECAST retains most remote utility while reducing source exposure.Source: §4.2 · pp. 8–9

4.4 ABLATIONS

  • In the Stage 1 input-path ablation, accuracy rose from 44.60% to 65.85% (+21.25 points), with gains on image (+19.90) and audio (+22.60), supporting the decoupled Stage 1 design.Source: Table 4 · p. 9, §4.4 · p. 10
  • 76.85% versus 75.10% overall favors protected text, but reconstructed images lead by 0.40 points on ChartQA; synthesized speech trails protected text by 3.90.Source: Table 5 · p. 9, §4.4 · p. 10

4.5 VALIDITY OF THE LLM JUDGE

  • 91.50% agreement with the human majority (κ = 0.83) on 400 annotated pairs supports the LLM judge as a proxy for human assessment.Source: §4.5 · p. 10

5 CONCLUSION

RECAST protects numerical reasoning through fixed multimodal interfaces by replacing source-specific content while preserving task relations and media format. On ChartQA and NMSQA, it retains much of unprotected remote accuracy while reducing solver-bound source-content exposure.

5 CONCLUSION

  • On ChartQA and NMSQA, RECAST retains 92.43% of unprotected remote accuracy, while a model-based audit flags source-content leakage in 7.95% of solver-bound contexts.Source: §5 · p. 10
  • End-to-end accuracy with the distilled local 4B Rewriter falls 0.55 percentage points below the offline teacher configuration.Source: §5 · p. 10
  • The reconstruction agent improves chart accuracy over the evaluated alternatives, and results on ChartX and Spoken-MQA support transfer beyond the distillation datasets.Source: §5 · p. 10
  • RECAST replaces source-specific content while preserving task relations and the media format required by fixed interfaces, with local operand recovery separating remote reasoning from computation on original values.Source: §5 · p. 10

AI USE STATEMENT

The authors report using generative AI tools for manuscript language and presentation assistance and code editing. They state that authors reviewed and verified the research and take responsibility for the final work.

AI USE STATEMENT

  • Generative AI tools assisted with language editing, manuscript organization and presentation feedback, and code editing.Source: AI USE STATEMENT · p. 10
  • The authors reviewed and verified all research ideas, methodological decisions, experimental results, and scientific claims.Source: AI USE STATEMENT · p. 10
  • The authors state that AI-generated suggestions were not treated as authoritative sources and that they take full responsibility for the final content.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

The ethics statement identifies the evaluation benchmarks and leakage-calibration annotators, and specifies what stays local in deployment. It also notes that sensitive deployments can add contractual and access controls.

ETHICS STATEMENT

  • ChartQA and NMSQA are public benchmarks, and the three annotators in the human calibration are paper authors.Source: ETHICS STATEMENT · p. 11
  • In deployment, original inputs and inverse maps stay local; the leakage judge receives original inputs only within the evaluation workflow.Source: ETHICS STATEMENT · p. 11
  • Sensitive deployments can combine RECAST’s reduction of source content sent to remote services with contractual and access controls on those services.Source: ETHICS STATEMENT · p. 11

A.1 EVALUATION DATA

The appendix specifies the evaluation subsets, component implementations, and numerical mapping and recovery procedure used in the experiments.

A.1 EVALUATION DATA

  • The in-distribution test uses 2,000 numeric-answer examples each from ChartQA and NMSQA, while the out-of-distribution evaluation samples 2,000 each from ChartX and Spoken-MQA.OOD datasets do not contribute to Rewriter distillation or checkpoint selection, and sample IDs are fixed before evaluation.Source: §A.1 · p. 15, Table 6 · p. 15

A.2 COMPONENTS

  • The implementation uses Qwen3-VL and Qwen3-ASR for capture, Qwen3-4B with QLoRA for rewriting, DeepSeek-V4 for chart specifications, and Qwen3-TTS for synthesis.The remote solver is Qwen3.5-Omni-Plus; alternative Stage 3 implementations receive the same protected records and use the same solver.Source: §A.2 · p. 15

A.3 NUMERICAL MAPPING AND RECOVERY

  • The numerical map preserves nonnegative integers from 0 to 15 unless they are years, and uses shared replacements across evidence and query; locally restored protected operands let the program compute on original values.Other values are replaced within magnitude bands, with ordering preserved; deployment draws a fresh map per request.Source: §A.3 · pp. 15–16

A.4 DISTILLATION DATA

The appendix documents the distillation corpus and the evaluation and release procedures used to assess ReCast and related methods. It specifies data acceptance, matching, leakage aggregation, local references, release checks, and text-anonymizer diagnostics.

A.4 DISTILLATION DATA

  • Accepted rewrites must preserve the five-field schema, id, modality, choice order, nonempty context and question, and each field’s numerical literals.Accepted pairs are deterministically split into training and development sets.Source: §A.4 · p. 16
  • The 15,795 accepted pairs span chart types, question operations, and topics; teacher rewriting replaces source vocabulary with substitute entities and topics.Figures 5–8 characterize the corpus, including operation shares and word frequencies before and after rewriting.Source: §A.4 · p. 17, Figure 5 · p. 17, Figure 6 · p. 18

A.5 REQUEST LOGS

  • Solver requests are logged before transmission, and reconstruction requests are logged separately; original inputs and inverse maps remain local.The leakage judge receives original inputs only within the evaluation workflow.Source: §A.5 · p. 18

B.1 ANSWER MATCHING

  • Numerical answers use an acceptable-reference comparison with rd = 10^-6 for ChartQA and NMSQA, while nonnumeric aliases use normalized exact matching.References derive from the original answer, question, and trusted local evidence, and compared methods must use identical sample IDs.Source: §B.1 · p. 18

B.2 LEAKAGE AGGREGATION

  • A sample is counted as leaked if any of four binary judge labels is positive across its solver-bound requests.All examples produced a request after validation, so the reported leakage rate over examples equals the rate over delivered requests.Source: §B.2 · p. 19

B.3 LOCAL REFERENCES

  • Local models provide zero-leakage references because their original inputs never leave the device; Remote-native provides the full-exposure reference by design.Remote-native sends original media and the query directly to the solver, defining the unprotected NAcc reference.Source: §B.3 · p. 19

B.4 RELEASE CHECKS

  • Release requires both a protected-record check and a source check to pass.Checks cover chart labels, values, associations, and queried attributes, or speech quantities, entities, coreference, event order, and dependencies.Source: §B.4 · p. 19

B.5 TEXT ANONYMIZER DIAGNOSTICS

  • Text-anonymizer diagnostics measure adapter coverage and judge-positive leakage among accepted records using a shared five-field interface.RLAA uses Qwen3-4B with up to three revisions and numerical-literal checks; Eternis uses its official prefill and removes quantity replacements.Source: §B.5 · p. 20, Table 11 · p. 20

B.6 HUMAN CALIBRATION OF THE LEAKAGE JUDGE

The leakage judge is calibrated against human annotations, and the appendix reports native exposure diagnostics for PSIC, SPTI, and AudioShield. These diagnostics have different scopes and do not establish comparable end-to-end answering performance.

B.6 HUMAN CALIBRATION OF THE LEAKAGE JUDGE

  • The judge’s precision and recall against human labels are 90.6% and 92.4%, respectively; reweighted human-labeled leakage is 8.70%, versus 7.95% from the judge.The calibration used 400 solver-bound pairs annotated by three annotators.Source: §B.6 · pp. 20–21, Table 12 · p. 21

B.7 PSIC DIAGNOSTIC

  • PSIC receives 564 positive image-only judgments among 2,000 ChartQA images (28.20%), and its protected outputs cannot support end-to-end answering under the stated protocol.Its diagnostic omits the question and downstream solver request; PSIC also lacks operand recovery for the original numerical answer.Source: §B.7 · p. 21

B.8 SPTI DIAGNOSTIC

  • SPTI’s caption-based diagnostic detects no source-content leakage, but shuffled voting leaves images unpaired with original questions, preventing sample-level end-to-end answering.The no-leakage finding is specific to that diagnostic.Source: §B.8 · p. 21

B.9 AUDIOSHIELD DIAGNOSTIC

  • AudioShield’s audit labels 12 of 2,000 protected-audio transcripts positive (0.60%), but suppression of recognized speech also contributes to the low leakage rate.Its protected audio cannot support end-to-end answering under the stated protocol because the method is designed to suppress recognizable speech rather than preserve task content.Source: §B.9 · p. 21

C HYPERPARAMETERS

The appendix identifies the rewriter and teacher models and specifies prompts for evidence capture, joint rewriting, remote reasoning, and source-content auditing. The prompts define the information each stage should preserve or return.

C HYPERPARAMETERS

D PROMPT SPECIFICATIONS

  • Chart evidence capture extracts query-relevant chart evidence—labels, values, units, associations, ordering, and visual attributes—rather than answering the query.The resulting description becomes task evidence in the shared textual record.Source: D PROMPT SPECIFICATIONS · p. 22
  • Joint rewriting changes entities, topics, and descriptions while preserving evidence-query correspondence, task relations, the requested operation, and answer type.Numerical literals remain unchanged during rewriting and are handled separately by numerical mapping.Source: D PROMPT SPECIFICATIONS · p. 22
  • The remote solver receives only the protected artifact and query, then returns an executable Python program with protected operands marked for local restoration.It is instructed to report unavailable evidence rather than infer or fabricate values.Source: D PROMPT SPECIFICATIONS · p. 22
  • The audit compares original and complete solver-bound contexts across semantic, lexical, correspondence, and linkability views, excluding generic reasoning structure alone.The audit includes retries and format repairs and compares each protected context only with its own original.Source: D PROMPT SPECIFICATIONS · p. 22

E VISUALIZATION RESULTS

The figures show selected output comparisons for ChartQA images and NMSQA audio across ReCast and other protection methods.

E VISUALIZATION RESULTS

  • Figure 9 presents four ChartQA comparisons, including input images and outputs from SPTI, PSIC, RLAA, Eternis, and ReCast.SPTI samples are unpaired; other outputs follow the source sample ID.Source: Figure 9 · p. 23
  • Figure 10 presents five NMSQA audio comparisons across input transcripts and outputs from AudioShield, RLAA, Eternis, and ReCast.The comparisons include protected-audio ASR for AudioShield and protected text for RLAA and Eternis.Source: Figure 10 · p. 24
Loading 2610.01184v1…