Source-linked AI summary
KUPAS MASTER: Distilling the Tacit Expertise of Master Practitioners into Agent-Ready Experience Corpora
Changmian Wang, Yuchao Ma, Xuchao Lu, Chen Zhang, Ping Sun, Jiazheng Wang, Shan Wang, Xuanwen Chen, Yihe Sun, Ziyu Lu, Jianqiang Huang, Hongzhi Li, Ziqing Xia, Kaihua Tang, Xian-Sheng Hua, Qinghua Zheng
arXiv:2609.37673v1cs.AIcs.CL
TL;DR
Professional experience often leaves cues, reasoning, conditions, and conflict unresolved in routine records, limiting how LLM agents can use it. KUPAS MASTER structures and reviews that experience as traceable assets and callable skills; its agent scored 9.83 points above raw-corpus RAG and improved in all seven dimensions.
Problem
Existing methods retrieve and use records, but questions remain about filling experience gaps, expressing conditions of use, handling conflicts, and checking whether procedures can be followed.
Source: §1 · p. 2Method
KUPAS MASTER converts practitioner experience into traceable assets and callable skills through semantic alignment, individual distillation, organizational consolidation, and cross-review.
Source: §12 · p. 27Results
9.83 points: the KUPAS MASTER agent improved on raw-corpus RAG by this amount and gained in all seven dimensions.
Source: §12 · p. 27Takeaways & Limitations
Run records show assets and skills supplied judgment conditions, identified risk boundaries, and organized task procedures.
Source: §12 · p. 27Graph associations require careful interpretation: a missing edge does not establish a cognitive leap, and association does not establish causation.
Source: §3.3 · p. 9
Abstract
from arXiv · showhide
Experienced professionals know more than just facts and conclusions. They know which cues matter, why a judgment is reasonable, and which action to take. Routine work records often leave out this tacit knowledge, making it difficult for Large Language Model (LLM) agents to use professional experience effectively. We introduce KUPAS MASTER, an experience engineering platform built around nine-layer cognitive corpus construction. It turns heterogeneous work records and practitioner interviews into traceable, reusable experience corpora for agents. Six case elements preserve the task process: context, cues, judgment, action, boundaries, and outcomes. Nine-layer cognitive corpus construction organizes tacit experience along nine extraction dimensions and stores the resulting assets in six libraries: rules, constraints, best practices, negative examples, corner cases, and skills. Semantic alignment, individual experience distillation, organizational consolidation, and cross-review preserve source evidence, conditions of use, and unresolved disagreements. The platform packages these assets into callable skills with explicit inputs, steps, dependencies, and stopping conditions, connecting experience collection to task execution and evaluation feedback. Using authorized samples from 20 randomly selected practitioners, the platform processed 1,576 source files into 23,024 individual experience records and 13,113 organizational assets. The evaluation spans multiple professional domains. Under common task inputs and scoring criteria, the base model, raw corpus retrieval-augmented generation (RAG), and KUPAS MASTER agent scored 70.63, 79.75, and 89.58, respectively. The KUPAS MASTER agent improved on raw-corpus RAG in all seven scoring dimensions. The platform provides a practical path from individual tacit experience to organizational knowledge and agent capabilities.
1 Introduction · 2 System Overview and Task Scope
KUPAS MASTER turns dispersed professional experience into traceable, reviewable knowledge linked to task conditions and agent execution. Its workflow scopes tasks, collects and reviews evidence, consolidates experience, and uses evaluation to inform revision.
1 Introduction
- Professional experience is distributed across practitioners and work records, which can omit decision cues, intermediate reasoning, conditions for action, and rare failures.Without the conditions behind a useful rule, another practitioner or agent may apply it to the wrong case; existing LLM retrieval and agent methods do not by themselves resolve gaps, conflicts, or procedural followability.Source: §1 · p. 2
- KUPAS MASTER links experience to its task, conditions, and evidence, and combines six case elements, nine extraction dimensions, and six asset types in a reviewable workflow.It retains context and review status, consolidates experience while preserving context-dependent alternatives and flagging conflicts, and turns assets into callable procedures for evaluation.Source: §1 · pp. 2–3
- The authorized sample of 20 randomly selected practitioners contained 1,576 source files, 30,762 corpus chunks, 23,024 individual experience records, and 13,113 organizational assets.The counts correspond to source collection, parsing, experience extraction, and organizational consolidation, respectively.Source: §1 · p. 2
- 70.63 for base-model agent A, 79.75 for raw-corpus RAG agent B, and 89.58 for KUPAS MASTER agent C across 177 questions and 531 responses under shared tasks and a scoring rubric.Scores average practitioner ratings equally across 20 practitioners, with each score combining seven weighted dimensions.Source: §1 · p. 2
2.1 Using the system
- A task specification defines the practitioner’s role, work object, environment, tools, triggers, case boundaries, completion criteria, and authorized decisions before evidence collection and targeted follow-up.The seven-step workflow aligns terms, entities, and events; extracts individual candidate assets; consolidates and cross-reviews them; and uses agent evaluation to feed corpus revision.Source: §2.1 · pp. 3–4, Figure 2 · p. 4
- KUPAS MASTER keeps case records of particular events separate from reusable asset records, allowing interpretations to change while preserving original events and many-to-many links between cases and assets.A case may support several assets, and an asset may draw on several cases.Source: §2.1 · p. 4
2.2 Platform responsibilities
- The platform manages experience acquisition, structuring, review, selection, and maintenance, while models handle task understanding and reasoning and domain tools provide observations or authorized actions.Traceable runs record the task specification, model and version, asset versions, retrieval configuration, and tool permissions; model changes prompt evaluation on fixed tasks with other conditions held constant.Source: §2.2 · p. 4
3 Structured Representation of Expert Experience · L1 Knowledge
KUPAS MASTER structures expert experience as linked task cases, extraction dimensions, and reusable assets, retaining the evidence, conditions, and uncertainty needed to trace and review experience. Its nine-layer framework connects source records to reviewed knowledge and agent-usable assets.
3.1 Task cases
- Task cases record context, cues, judgment, action, boundaries, and outcomes while preserving event order, unresolved outcomes, conflicting accounts, and failed actions.Repeated actions can be timestamped or partially ordered; missing reasons remain explicit, and separate practitioner accounts are not collapsed into certainty.Source: §3.1 · p. 6
3.2 The nine-layer framework and extraction dimensions
- The nine overlapping extraction dimensions provide a shared vocabulary for evidence and context, judgment and action, and practice and monitoring.A single statement may span multiple dimensions or functional groups.Source: Figure 3 · p. 5, §3.2 · p. 6
3.3 Nine-layer cognitive corpus construction
- Nine linked pipelines extract and review experience from narratives, reasoning, logs, and feedback, with cross-layer links and reusable assets retaining source evidence, conditions, and review status.Methods depend on available data; outputs begin as candidates, and thresholds require calibration. Prediction verification requires the relevant action and trigger to occur, while hard constraints cannot be overridden by risk scores.Source: §3.3 · pp. 7, 9, 10
L1 Knowledge
- L1 aligns entities and relations to graph anchors with source locations and event times, providing L2–L9 a shared basis for object alignment and source tracing.Uncertain links remain candidates, changing attributes retain temporal versions, and rare knowledge is not automatically discarded.Source: §3.3 · p. 7
3.4 Functional correspondences with brain systems
- The framework uses brain-system functions as organizing analogies for experience dimensions, including semantic representation for L1 and L5 and prediction for L5 and L6.Other correspondences include goal-directed selection for L2, cognitive control and value comparison for judgment and decision, monitoring, and habits.Source: §3.4 · p. 10
3.5 Six experience asset libraries
- The six asset libraries contained 13,113 organizational assets, including 3,550 rules, 2,892 best practices, 1,955 skills, 1,948 corner cases, 1,623 constraints, and 1,145 negative examples.Negative examples preserve failures and corrections; corner cases flag situations needing adjustment, while skills combine asset types into procedures with inputs, steps, state transitions, and outputs.Source: §3.5 · pp. 10–12
3.6 Evidence, review, and uncertainty
- Review separates extraction confidence, evidence sufficiency, and applicability, while asset states and version links preserve corrections and the original cases behind changed experience.Activation also requires deployment and runtime checks.Source: §3.6 · p. 12
3.7 Graph structures and requirements for agent use
- Agent-use interfaces require machine-readable scope, exclusions, sources, review status, versions, and skill execution conditions, while graph edge types distinguish temporal, procedural, associative, and proposed causal links.Interface validation checks required fields and constraints; task evaluation checks whether the agent selects and uses the right asset in context.Source: §3.7 · p. 12
4 Experience Acquisition and Corpus Construction
Experience acquisition and corpus construction turn heterogeneous work evidence into source-traceable, structured records through semantic alignment, staged extraction, and review.
4.1 Collecting heterogeneous evidence
- The authorized sample spans 1,576 files in 13 formats, while collection combines existing records with task-appropriate observation, interviews, and logs to capture decisions, outcomes, and exceptions.Process records preserve event order and distinguish information available at the time from later knowledge; interviews can recover omitted observations, rejected alternatives, and failure conditions.Source: §4.1 · pp. 12–13
4.2 Semantic alignment
- Semantic alignment normalizes terms and units without erasing original wording, context, alternatives, uncertainty, or source locations across text, multimodal observations, and derived claims.A numerical interpretation of an ambiguous phrase is supported only by measurements or a reviewed domain definition.Source: §4.2 · p. 13
4.3 Narrative reconstruction and structured extraction
- The S0–S3 pipeline preserves source-linked case narratives, extracts decision elements, then structures supported records using nine layers spanning knowledge, attention, judgment, decision, association, anticipation, monitoring, habit, and constraints.Candidate rules must retain prerequisites and exceptions; each stage records its source span, proposed output, revisions, and review decision.Source: §4.3 · p. 14
4.4 Quality control and corpus versions
- Quality checks combine structural validation, evidence-and-scope review, and task review, with practitioner cross-review to expose assumptions and distinguish personal habits from broadly usable procedures.Experiments and deployments fix a data version and record source files, processing programs, active assets, and review decisions; earlier versions remain available for checking reported results.Source: §4.4 · p. 15
- 79.75 in B versus 70.63 in A on composite mean, while evidence sufficiency and accuracy rose from 62.18 to 77.27 with raw-corpus retrieval.The composite mean increased from 70.63 in A to 79.75 in B; the evidence sufficiency and accuracy score increased from 62.18 to 77.27.Source: §4.4 · p. 15
5 Organizational Experience Consolidation
KUPAS MASTER consolidates practitioner experience by comparing applicability, evidence, and compatibility while preserving conditions and unresolved disagreements. Cross-review and versioned decision records keep organizational assets traceable from their sources through use.
5.1 From individual accounts to organizational assets
- Task identifiers, entities, terms, and asset types are aligned before semantic similarity and scope overlap identify candidates for comparison; similarity alone does not justify merging.Comparison checks whether assets address the same decision, can hold under the same conditions, and have compatible judgments or actions.Source: §5.1 · p. 15
5.2 Agreement, different conditions, and conflicts
- Agreement under the same conditions retains independent sources, while context-dependent recommendations remain separate and alternatives are compared using cases matched on difficulty, environment, resources, and outcome definition.An unresolved contradiction remains linked for review rather than becoming an active rule; a nonempty conflict region requires adjudication, while unknown overlap or compatibility remains unresolved.Source: §5.2 · pp. 15–16, Figure 7 · p. 16
- For workplace injury mediation, experience is retrieved only when insurance type, liable party, eligible compensation items, and mediation authority are known; otherwise the agent verifies missing applicability conditions.Ordinary compensation, third-party commercial insurance, and statutory work injury insurance require separate procedures.Source: §5.2 · p. 16
5.3 Cross-review and version management
- Cross-review checks sources, conditions, exceptions, consistency, and possible misuse; updates create versions, record consolidation decisions, and identify skills needing revalidation after rule changes.Version records preserve original contributions and the exact assets used in a run.Source: §5.3 · p. 16
- Across six libraries, 37 completed cross-review jobs for 8 practitioners covered 818 candidates; human verification was complete for 9 candidates and awaiting for 331.Automated processing completion and human review status are recorded separately.Source: §5.3 · p. 17
- The three-configuration evaluation demonstrates the value of organizational assets and skills in professional task execution, with source-to-review traceability retained.For the workplace injury mediator, two rule-library review jobs used the same source version; each involved 37 source assets and 21 candidates, with processing status retained.Source: §5.3 · p. 17
6 Skill Construction and Agent Integration
KUPAS MASTER packages reviewed experience assets into callable procedures with explicit inputs, steps, dependencies, constraints, and stopping conditions. Its execution model filters assets by applicability and authorization, checks prerequisites and actions, and records traceable task execution.
6.1 Skill invocation and execution requirements
- Skills combine rules, best practices, constraints, negative examples, and corner cases into checkable procedures linked to their supporting assets.The specification also defines applicability, inputs, output structure, ordered steps, tool dependencies, stopping conditions, evidence references, and versions; asset links enable targeted revalidation after corrections.Source: §6.1 · p. 17
6.2 Selection and execution
- Retrieval retains active, authorized assets with true or unknown applicability, while invocation requires satisfied prerequisites and dependency and action checks.Unknown applicability prompts a targeted question or observation rather than being treated as false or satisfied; false applicability excludes the asset.Source: Figure 8 · p. 17, §6.2 · p. 18
- Before consequential actions, the agent checks constraints and tool permissions independently of relevance ranking, and an unknown mandatory condition blocks action.Clarification targets decision-relevant missing fields and triggers another state check; output validation checks schema, evidence, and stopping conditions, with failed checks allowing bounded retry, more information, replanning, or human referral.Source: Figure 8 · p. 17, §6.2 · p. 18
- Skills may branch, so prerequisites must be rechecked before actions as new observations can invalidate them; task records preserve actual sequences, failed attempts, and human intervention.Runtime logs record asset and skill calls for tracing and branch testing.Source: §6.2 · p. 18
7 Case Study and Task Runs
In workplace injury mediation, KUPAS MASTER scored higher than the base model and raw-corpus RAG on the practitioner’s evaluation questions. The case runs show the agent using mediation skills and structured experience to organize evidence, judgments, and procedures.
7.1 Comparing the three configurations in workplace injury mediation
- The workplace injury mediator supplied 2 interview files, which produced 167 experience assets, and completed rule-library cross-review jobs.Source: §7.1 · p. 18
- For a traffic-accident detour scenario, the KUPAS MASTER agent used a mediation skill, rules, and corner cases, and was rated best on all 8 comparison items, including 1 tie with raw-corpus RAG.It distinguished injury-determination boundaries, mediation authority, and high-risk issues, and generated an output file.Source: §7.1 · p. 18
- For a grade-ten disability case, the agent used a mediation skill, rules, and best practices to organize wage evidence, compensation calculations, termination reasons, and judicial confirmation into a complete process.It generated a “Preliminary Analysis of a Workplace Injury Mediation Case” file.Source: §7.1 · pp. 18–19
- On 8 evaluation questions, A, B, and C scored 73.34, 80.80, and 90.58, respectively; C exceeded B by 9.78 points.Mean end-to-end times for A, B, and C were 13, 31, and 53 seconds, respectively.Source: §7.1 · p. 19
8 Evaluation Design
The evaluation compares base-model inference, raw-corpus retrieval, and KUPAS MASTER assets and skills under common task and scoring conditions. It links task outcomes to the platform workflow through corpus, run, and asset-use records.
8.1 Evaluation objectives
- The evaluation aims to measure gains over the base model and raw-corpus RAG while connecting task results to case representation, experience collection, consolidation, skill construction, and boundary controls.Tasks are framed as having clear inputs, bounded goals, and assessable outputs; mediation tasks can test missing-fact identification, information gathering, response checking, and escalation.Source: §8.1 · p. 20
8.2 Design principles
- All configurations follow a common protocol, while B retrieves raw domain records and C uses experience assets and skills derived from the same authorized materials.Retrieval, skill calls, and end-to-end time are retained to analyze quality and resource use.Source: Figure 9 · p. 19, §8.2 · p. 20
8.3 Experimental setup and data
- The sample comprises 1,576 source files from 20 practitioners, yielding 23,024 individual experience records and 13,113 organizational assets.The 177 questions span seven domains and produce 531 responses; an LLM scores each configuration on seven weighted dimensions, with traces available for five practitioners and 53 questions.Source: §8.3 · pp. 20–21
8 Evaluation Design
- 177 questions receive independent answers from all three configurations, and the KUPAS MASTER agent is evaluated against the base model and raw-corpus RAG.The same tasks, input requirements, and scoring criteria apply to direct inference, raw-corpus RAG, and the experience-guided agent.Source: §8 · p. 19, Figure 9 · p. 19, §8.3 · p. 20
8.4 Metrics and judgment
- Performance is measured by the weighted composite score and the fraction of questions scoring at least 60; workflow success additionally requires meeting a domain rubric and all mandatory boundary conditions.Practitioner-cluster resampling estimates 95% intervals for score gains while retaining dependence among each practitioner’s questions.Source: §8.4 · p. 21
- The framework also assesses evidence support, procedural completeness, applicability errors, prohibited actions, refusals, escalation quality, latency, and token use.It distinguishes citation coverage from factual support and specifies randomized, masked human review as an extension.Source: §8.4 · pp. 21–22
9 Experimental Results and Analysis
Across 20 practitioners, KUPAS MASTER outscored raw-corpus RAG and the base-model configuration; gains spanned seven dimensions and accompanied asset retrieval and skill use, with higher latency.
9.1 Composite scores
- 89.58 for C exceeded 79.75 for B and 70.63 for A by 9.83 and 18.96 points; C led both baselines for all 20 practitioners and on 176 of 177 questions.The 95% practitioner-cluster bootstrap interval for C minus B was [9.20, 10.45].Source: §9.1 · p. 22, §9 · p. 22, Figure 10 · p. 23
9.4 Sources of gains and runtime behavior
- Consolidation turned 23,024 individual entries into 13,113 organizational assets, while detailed traces showed knowledge-base retrieval in 52 of 53 runs and skill calls in 46.The 53-question trace subset used rules in 46 runs, corner cases in 35, best practices in 33, constraints in 28, and negative examples in 18; calls can overlap.Source: §9.4 · pp. 23, 25
9.2 Performance by dimension
- C’s mean scores ranged from 87.7 to 91.2 across seven dimensions, and all 20 practitioners scored higher with C than B on every dimension.The dimensions cover result correctness, actionability, professional judgment, evidence, tool and skill use, requirement understanding, and boundaries and compliance.Source: §9.2 · p. 22, Figure 11 · p. 24
9.3 Gains at each stage
- B−A gained 9.13 points through raw-corpus RAG, while C−B gained 9.83 through nine-layer assets and skills; the two gains were similar in size.Figure 12 separates the total change from A to C into these two stages.Source: §9.3 · pp. 22–23, Figure 12 · p. 24
9.4 Sources of gains and runtime behavior
- Assets and skills contributed conditions, risk boundaries, and procedures, supporting flood-response judgments, safety hazard checks, and social-insurance plans, checklists, review forms, and ledgers.The traces connect experience extraction to asset retrieval and skills that organize assets into steps, checks, and deliverables.Source: §9.4 · p. 25
- C’s average advantage over B remained about 9.3 points without the tool-and-skill-use dimension, while gains held across corpus sizes and formats without a stable size–gain relationship.Spreadsheets, JSON, and documents differ in structure, but the platform organizes them into a common representation.Source: §9.4 · p. 25
9.5 Efficiency
- 57.1 s was C’s mean end-to-end latency, versus 34.0 s for B and 16.2 s for A; C’s mean practitioner-level P90 latency was 68.4 s, versus 41.1 s and 21.1 s.The authors describe the roughly one-minute response time as accommodating experience retrieval, asset checks, and skill execution in business analysis, planning, and professional review.Source: §9.5 · p. 25
10 Commercial Deployment
KUPAS MASTER is a commercial platform with cloud and local deployment options, separating corpus production, asset management, agent execution, and evaluation records. Customers can configure local deployments for industry workflows and confidentiality requirements.
10 Commercial Deployment
- The commercial platform separates corpus production, asset management, agent execution, and evaluation records, managing access through permissions, versions, and run records.The experiments used the release dated September 16, 2026.Source: §10 · p. 26
10.1 Customized local deployment
- Local deployments on customer-owned servers or private clouds can configure domain terms, library structures, review rules, and skill workflows for industry and business processes.Source: §10.1 · p. 26
- For strict confidentiality, local inference and internal services can keep source materials, assets, task data, outputs, and logs within the customer environment, with access governed by authentication, permissions, isolation, and audit logs.Business data need not pass through external model APIs or third-party services, and customers can control data storage, use, and access while retaining operation records.Source: §10.1 · p. 26
11 Related Work
KUPAS MASTER situates its contribution across work on eliciting professional expertise, retrieval, agent memory, executable skills, and professional AI. It focuses on reviewed practitioner experience with supporting evidence and conditions of use for agent execution.
Acquiring and organizing professional experience
- KUPAS MASTER adapts incident-based cognitive-task methods and organizational knowledge creation to organize expert judgments, actions, and feedback for extraction, review, and reuse.Its nine dimensions structure experience for agents.Source: §11 · p. 26
Retrieval and structured corpora
Memory and experience reuse
- Agent-memory research spans memory architectures, experience reflection, reusable feedback, and structured or persistent memory [44] [45] [46] [47] [48] [49] [50]; KUPAS adds externally collected accounts, claim-level evidence, conditional disagreements, and review before use.It brings practitioner expertise and supporting evidence into agent memory and reuse.Source: §11 · p. 26
Skills and execution frameworks
AI in professional and scientific work
- Professional and scientific AI studies include conversational diagnosis, disease management, scientific-instrument workflows, and scientist-guided discovery; KUPAS improves agents through external experience corpora at inference without base-model retraining.These studies inform evidence organization, expert assessment, and task validation in professional settings.Source: §11 · p. 27
12 Conclusion
KUPAS MASTER converts practitioners’ tacit experience into traceable, reusable assets and callable skills through a nine-layer cognitive corpus workflow. Its agent scored higher than both the base model and raw-corpus RAG in the reported evaluation.
12 Conclusion
- Six case elements, nine extraction dimensions, and six asset types connect case records to traceable experience assets and callable skills.Semantic alignment, individual distillation, organizational consolidation, and cross-review retain sources, conditions, and disagreements; callable skills specify inputs, steps, dependencies, and stopping conditions.Source: §12 · p. 27
- 70.63, 79.75, and 89.58 were scored by the base model, raw-corpus RAG, and KUPAS MASTER agent, respectively; the agent gained 9.83 points over RAG and improved in all seven dimensions.The evaluation used authorized samples from 20 randomly selected practitioners; the platform processed 1,576 source files into 23,024 individual records and 13,113 organizational assets. Across 177 questions and 531 responses, practitioner means were weighted equally under a common task protocol and shared rubric.Source: §12 · p. 27
- Future development will expand the asset base and applications, streamline experience capture and reuse, and strengthen sharing, review, updates, and application feedback.The platform is presented as a foundation for staff development, business collaboration, and professional services, with traceable assets intended to make organizational know-how reusable across tasks.Source: §12 · p. 27
A Team Members … C Detailed Experimental Analysis
The appendix describes a practitioner-assistant workflow and evaluates how structured experience supports task performance, execution, boundary handling, and updates. Under the common evaluation protocol, configuration C scores higher than A and B across the reported comparisons.
A Team Members
- The team-members section identifies project and research teams.The supplied passage does not provide names or further team details.Source: A Team Members · p. 31
B Case Study: System Workflow
- The hypertension assistant workflow aligns imported guidelines and field definitions, extracts evidence-linked criteria and exceptions, and organizes reviewed experience into skills with inputs, steps, and stopping conditions.The agent produces cited outputs with verification steps, and evaluation errors feed revisions to the corpus, assets, or skills.Source: B Case Study: System Workflow · p. 32, Figure 13 · p. 32
C Detailed Experimental Analysis
- Scores rise from 70.63 for A to 79.75 for B and 89.58 for C across 177 questions and 531 responses.C gains 9.83 points over B and wins 176/177 paired question comparisons against B; all 20 practitioner means and all 7 scoring dimensions improve from B to C.Source: Table 9 · p. 33, C Detailed Experimental Analysis · p. 33
- With the same uploaded materials, C adds 9.83 points (12.3%) over raw-corpus RAG configuration B across all 20 practitioners and all 7 dimensions.Raw retrieval contributes 9.13 points over A; the passage attributes the further gain to structuring experience and executable skills.Source: C Detailed Experimental Analysis · p. 33
- The platform consolidates 23,024 published individual entries into 13,113 organizational assets, whose evidence-linked fields and processing histories support traceability.Across six library types, 486 of 818 candidates passed cross-review (59.4%); the libraries enter a shared index of 43,882 Elasticsearch documents.Source: C Detailed Experimental Analysis · p. 33
- C calls skills in 166/217 comparison reports (76.5%) and generates files in 116/217 (53.5%); boundary assets accompany 95/99 C-only boundary-judgment wins.C gains 14.38 points in appropriate tool and skill use and 9.89 in output actionability over B. The boundaries and compliance score gains 9.03 points from B to C; without boundary-asset calls, C-only wins are 94/127 (74.0%).Source: C Detailed Experimental Analysis · pp. 33–34
- Across 19 repeated equity-method comparisons, C is among the winners each time; its practitioner-weighted mean latency is 57.1 s, versus 16.2 s for A and 34.0 s for B.Libraries retain versions for batch rebuilding, and retrieval can vary across runs. C is 1.68× B in mean latency; its mean practitioner-level P95 latency is 71.6 s, with latency associated with library retrieval, skill calls, and file generation.Source: C Detailed Experimental Analysis · p. 34
- All 20 practitioners improve across 19 industry labels and 7 dimensions, while focused collections also score highly, including 91.36 from 36 assets and 90.58 from 167 assets.The asset-count correlation with C-minus-B gain is 0.07; raw chunk count correlates with B-minus-A gain at 0.61 (p ≈0.004).Source: C Detailed Experimental Analysis · p. 34
D Additional Results and Plots
The appendix visualizes question-weighted score distributions and difficulty groups, alongside practitioner-weighted composite scores, dimension means, and latency. It also reports report-level comparisons, corpus composition, and relationships between corpus scale and score gains.
Additional Results and Plots
- A/B/C means were 70.9/79.9/89.6; pass rates at 60 were 87.57%/98.87%/100.00%, and fractions scoring at least 80 were 7.3%/58.8%/99.4%.C reduces low-scoring answers and brings nearly all answers above 80.Source: D Additional Results and Plots · p. 34, Figure 14 · p. 35
- C achieved a score of 89.6 at a mean time of 57.1 seconds, with mean-time ratios of C/A = 3.53 and C/B = 1.68.The equal-practitioner-weighted A/B/C scores were 70.6/79.8/89.6 and mean times were 16.2/34.0/57.1 seconds.Source: D Additional Results and Plots · pp. 34–35, Figure 15 · p. 35
- Of 13,113 organizational assets, rules and best practices account for 49.1%, and 17/20 practitioners have all six library types.The six libraries contain 3,550 rules, 2,892 best practices, 1,955 skills, 1,948 corner cases, 1,623 constraints, and 1,145 negative examples.Source: D Additional Results and Plots · pp. 35–36, Figure 16 · p. 36
- The platform organized 1,576 source files across 13 formats, with 2 to 378 files per practitioner, preserving source relationships across documents, tables, structured data, and multimedia.Source: D Additional Results and Plots · p. 36
- C won every row, ties included, across four categories in 62 reports; across 217 reports, it won alone on 1,109/1,345 rows (82.5%) and tied on 214 (15.9%).C called skills in 166 reports and generated files in 116; these report-level comparisons supplement the 177-question scored evaluation.Source: D Additional Results and Plots · p. 36, Figure 18 · p. 37
- Organizational asset count had a 0.07 Spearman correlation with the C-minus-B gain, while raw chunk count had a 0.61 correlation with the B-minus-A gain.The passage reports that experience organization and use add value beyond raw retrieval.Source: D Additional Results and Plots · p. 36, Figure 19 · p. 38
- C scored 90.8, 89.3, and 89.8 on easy, medium, and hard questions, exceeding both baselines in every group.Its gains over B were about 8.7, 10.0, and 9.5 points, respectively.Source: D Additional Results and Plots · p. 36, Figure 20 · p. 38