Source-linked AI summary
DynaContext: Self-Improving Dynamic Contextualization of Optimized Prompts for Heterogeneous Parameter Extraction
Joe Yu, Shibin Thomas Stanley Paul, Sven Mayer
TL;DR
Static optimized prompts cannot adapt to changing schemas, constraints, and evidence across heterogeneous extraction instances. DynaContext combines an offline-optimized extraction core with inference-time contextualization and validation-gated self-improvement. Across heterogeneous gold facts, the full configuration reaches 71.0 F1 and outperforms the deployed static-prompting pipeline by 17.3 F1 points on average.
Problem
Static optimized prompts are reused across instances even when component categories require different schemas, unit constraints, demonstrations, and evidence.
Method
DynaContext composes an offline-optimized extraction core with each item’s schema, evidence, unresolved fields, and validated demonstrations, while deterministic validation, judging, and human review gate outputs and memory updates.
Results
71.0 F1 across 850 heterogeneous gold parameter facts is achieved by the full configuration, compared with 51.8 F1 for an unoptimized, demonstration-free control.
Takeaways & Limitations
The optimized core and dynamic demonstrations together improve heterogeneous parameter extraction while retaining validation and human oversight for self-improvement.
Takeaways & Limitations
The benchmarks are small, drawn from one industrial corpus, and unbalanced across categories; public-benchmark evaluation remains future work.
Abstract
from arXiv · showhide
Automated prompt and skill optimization typically produces a single static instruction that is reused across inference instances until the next optimization cycle. However, this approach cannot adapt when the required context, constraints, and evidence vary from one instance to another. For instance, parameter extraction from electronic component descriptions breaks this assumption: resistors, capacitors, transistors, and connectors require different fields, unit constraints, and demonstrations, and each input provides a different evidence state. We introduce DynaContext, a framework that combines an offline-optimized extraction core, learned with GEPA or SkillOpt, with inference-time contextual adaptation and validation-gated self-improvement. DynaContext routes each item through internal, external, or fallback evidence paths and composes an item-specific prompt from the core, schema, evidence, unresolved fields, and validated demonstrations. Deterministic validation and an LLM judge gate every output, uncertain cases go to human review, and only human-verified corrections enter the demonstration memory. On a single-category benchmark, average accuracy increases from 86.6% for the base prompt to 96.9% for standalone SkillOpt and 98.6% for the best DynaContext configuration. Across 850 heterogeneous gold parameter facts, average field-level F1 increases from 51.8% for an unoptimized, demonstration-free control to 59.2% with dynamic demonstrations alone, 66.9% with the optimized core alone, and 71.0% with both. Holding the model fixed, the full configuration outperforms the deployed static-prompting pipeline by 17.3 F1 points on average.
1 Introduction
Static optimized prompts cannot adapt when schemas, unit constraints, demonstrations, and evidence differ across component-extraction instances. DynaContext addresses this by composing per-instance context around an offline-optimized extraction core and validation-gated improvement.
- Heterogeneous extraction instances differ in target schema, unit constraints, relevant demonstrations, and available evidence.
- Electronic-component parameter extraction requires category-conditional schemas, separated values and units, provenance for populated fields, and abstention for unsupported fields.Inputs also arrive with internal, external, or description-only evidence states.
- DynaContext separates offline optimization from inference-time composition and adapts each prompt using the optimized core, schema, evidence, unresolved fields, and validated demonstrations.GEPA and SkillOpt can provide the offline extraction core.
- The framework routes items deterministically through internal, external, or fallback evidence paths, with route-specific demonstration budgets and fallback actions.These constraints keep per-instance adaptation within explicit safety boundaries.
- Dynamic demonstrations and the optimized core jointly raise heterogeneous benchmark performance to 70.98 F1, gaining 17.32 F1 over the deployed pipeline.The ablation attributes gains of +7.47 F1 to dynamic demonstrations and +15.14 to the optimized core.
2 Related Work
Prior work covers product attribute extraction, automated prompt and skill optimization, tool-using agents, and validation systems. DynaContext is complementary because it conditions optimized artifacts on each instance’s schema, evidence state, and unresolved fields while validating updates.
- Attribute and parameter extraction from product text: Product-text extraction spans sequence tagging, large multi-source benchmarks, and LLM-based extraction, with few-shot demonstrations a strong baseline for unit-normalized extraction.
- Attribute and parameter extraction from product text: UPE differs from flat attribute extraction through category-conditional schemas, value-unit normalization, provenance requirements, and abstention when evidence is missing.
- Automatic prompt and skill optimization: Automatic prompt optimizers evolve instructions or jointly tune instructions and demonstrations, but their artifacts remain fixed across inference instances until reoptimization.
- Automatic prompt and skill optimization: DynaContext complements prompt optimization by conditioning optimized artifacts on instance schema, evidence state, and unresolved fields at inference time.It also gates every artifact update behind validation.
- Tool-using and self-improving agents: Tool-using agents and schema-grounded memory systems provide related foundations for evidence acquisition, retrieval, and validated durable state.
- LLM-based judging, validation, and reliability: LLM judges and guardrail frameworks motivate DynaContext’s emphasis on evaluator reliability and policy enforcement around agentic systems.
3 Problem Formulation
The paper formulates heterogeneous component extraction as closed-world, schema-grounded structured prediction over current-item evidence. Outputs normalize values and units, preserve provenance, and abstain on unsupported fields.
- 3.1 Task Definition: Each input component record is modeled as x_i = (d_i, m_i, u_i, c_i, E_i), combining description, optional identifiers and category hint, and available evidence.
- 3.1 Task Definition: The output is a closed-world structured record containing the resolved category, selected schema, normalized MPN, field values, unresolved fields, validation status, and route metadata.
- 3.1 Task Definition: The system must select the applicable schema, identify supporting current-item evidence, normalize values and units, and leave unsupported schema fields null.
- 3.2 Evidence Sources: Evidence sources include trusted internal exact-MPN records, structured external API specifications, official datasheets, and the raw description.
- 3.2 Evidence Sources: Retrieved demonstrations guide interpretation, formatting, and null behavior but cannot justify a non-null value for the current component.
- 3.3 Output Representation: Category schemas define allowed fields, value types, unit classes, and null behavior, while each emitted non-null value carries provenance.
- 3.3 Output Representation: Fields outside the selected schema are rejected, and schema fields without support remain null.
- 3.3 Output Representation: Heterogeneous evaluation scores non-null gold field-value pairs with field-level precision and recall aggregated as F1, while the fixed-schema study isolates extraction performance.
4 Method
DynaContext uses route-aware evidence acquisition and category-specific contextualization rather than a static extraction prompt. Validation, review, and promotion rules create a gated self-improvement loop.
- Route selection: DynaContext preprocesses each component, normalizes units, extracts a canonical MPN, and selects an internal, external, or fallback route.Internal matches reuse stored records; verified MPN evidence selects category-specific extraction; otherwise the system resolves category and schema through fallback processing.
- Route selection: Internal matches reuse complete trusted records, while external evidence restricts LLM completion to fields missing from verified parameters.Retrieved values cannot be regenerated or overwritten on the external route, and the LLM is not invoked for complete internal records.
- Fallback extraction: Fallback extraction resolves a three-level category, selects its specification schema, retrieves category-matched demonstrations, and extracts the full schema.Demonstrations are retrieved only on the fallback route using embedding similarity, BM25 overlap, or hybrid ranking.
- Contextual prompt composition: At inference time, DynaContext composes prompts from the optimized extraction core, resolved schema, evidence, unresolved fields, and validated demonstrations.The optimized core remains stable while schema, evidence summary, missing-field set, and demonstrations vary by item; internal routes instantiate no extraction prompt.
- Output assembly: Route-specific source fusion retains the highest-priority usable candidate only when it is schema-valid, unit-compatible, and attributable.Conflicting incompatible non-null values are logged and sent to review; outputs are postprocessed, unit-standardized, and serialized to JSON.
- Validation and self-improvement: Only accepted human decisions update persistent memory, while rejected cases and judge-only approvals are excluded from promotion.GEPA revises the judge prompt offline from accepted and corrected traces, and updates are promoted only when held-out quality improves without increasing false approvals.
5 Experiments
The experiments test whether optimized extraction cores and dynamically retrieved demonstrations improve parameter extraction across fixed-schema and heterogeneous benchmarks. Results show complementary gains, with dynamic context improving accuracy and F1 while increasing cost relative to Static-10.
- Experimental setup: The evaluation uses a 142-record ceramic-capacitor benchmark and a 133-record heterogeneous benchmark spanning 13 categories and 850 gold parameter facts.The studies isolate fixed-schema context selection and heterogeneous schema-grounded extraction, respectively.
- Experimental setup: The experiments compare Base Prompt, GEPA, and SkillOpt extraction cores with no retrieval or Vector, BM25, and Hybrid demonstrations.The heterogeneous ablation crosses Base Prompt and SkillOpt with demonstrations on or off, while the controlled single-category study evaluates all three cores.
- Single-component experiments: 97.09% average accuracy with retrieved context exceeds 93.05% without retrieval across the single-component sweep.The strongest result reaches 99.50% with GEPA, DynaContext, DeepSeek-V4-Pro, and Vector retrieval.
- Multi-component experiments: 19.22 F1 points is the joint gain from SkillOpt and dynamic demonstrations over the 51.76-F1 demonstration-free Base Prompt on heterogeneous extraction.Dynamic demonstrations alone add 7.47 points, while the optimized core alone adds 15.14 points; all paired effects are positive across eight deployments.
- Multi-component experiments: 77.93 F1 is the strongest heterogeneous result, achieved by DeepSeek-V4-Pro with SkillOpt and three Hybrid-retrieved demonstrations.Moderate context sizes usually win: k3 or k5 is optimal for six of eight models.
- Cost and analysis: DynaContext costs 1.48 times as much as Static-10 on average while gaining 17.32 mean F1 points, and larger models are not the best budget alternative.GPT-5.5 costs roughly fifteen times as much as GPT-5.4 mini while scoring 1.72 F1 points lower.
6 Limitations
The evaluation is limited by small, imbalanced, proprietary datasets from one industrial corpus, changing external dependencies, and heuristic components whose sensitivity remains untested.
- The benchmarks contain 142 single-category and 133 heterogeneous records over 850 gold facts from one industrial corpus, so corpus-specific effects cannot be ruled out.
- The heterogeneous benchmark mirrors production mix rather than a designed sample, with 74% of gold facts from ceramic capacitors and chip resistors and sparse representation for five categories.Per-category estimates for rare families are noisy, and aggregate results are dominated by two families.
- Public-benchmark evaluation, including WDC-PAVE, is left for future work.
- Fixed heuristics, proprietary interfaces, external parts APIs, and time-varying datasheet retrieval constrain reproducibility and leave threshold and retrieval-weighting analyses for future work.Experiments use cached responses with recorded query dates, while datasheet extraction remains limited.
- The underlying offer records cannot be released because of confidentiality constraints.
7 Conclusion
DynaContext frames electronic-component parameter extraction as a schema-grounded, source-aware, self-improving agent problem. Its resulting configuration gains 17.32 F1 over the deployed pipeline, while the remaining gap is attributed to schema resolution.
- DynaContext combines an offline-optimized core with schemas, evidence, unresolved fields, and validated demonstrations at inference time.Deterministic validation, a GEPA-optimized judge, and human review gate outputs and memory updates.
- 17.32 F1 points is the resulting configuration’s gain over the deployed pipeline across eight deployments.
- The heterogeneous gain is attributed by a two-factor ablation to demonstrations and the optimized core in roughly a one-to-two ratio.
- The remaining gap to the fixed-schema ceiling is identified as a schema-resolution gap.