Source-linked AI summary
Credo: Reusable Declarative Primitives for Agentic Workflows
Duo Lu, Andrew Crotty, Uğur Çetintemel
TL;DR
Searched harnesses can encode effective but task-specific execution knowledge in opaque imperative programs, limiting reuse. Credo recovers that knowledge as typed declarative primitives, catalogs it with scope and provenance, and compiles harnesses for new targets. Preliminary evaluation reports close round-trip preservation of accuracy and cost plus effective cross-domain transfer, while identifying cost-based compilation and catalog maintenance as open challenges.
Problem
Searched harnesses mix reusable logical steps, runtime signals, physical choices, and prompt strategies with task-specific code, leaving the knowledge opaque and requiring new tasks to restart search.
Method
Credo extracts skeletons, beliefs, policies, and prompt templates from searched harnesses, tags their transfer scope, records provenance in a catalog, and compiles compatible primitives for target tasks.
Results
Round-trip compilation closely preserves accuracy and cost, while cross-domain compilation demonstrates effective transferability across retrieval domains.
Takeaways & Limitations
Credo treats searched harnesses as reusable declarative artifacts and frames persistent agentic-execution knowledge as a database-systems research opportunity.
Takeaways & Limitations
The current compiler is rule-based rather than cost-based, and catalog maintenance under stale, redundant, erroneous, or dominated primitives remains an open challenge.
Abstract
from arXiv · showhide
An LLM application depends on both a model and a harness: the program that determines what each call sees, how many calls to make, and which answers to trust. Coding agents can now discover strong harnesses by searching over candidate programs, but the resulting artifact is an opaque block of imperative code whose logical steps, runtime signals, physical execution decisions, and prompt strategies remain implicit and task-specific, forcing subsequent tasks to start the harness search process from scratch. The potential for reuse, however, is substantial. A searched harness encodes significant knowledge, such as the logical steps that work, the signals that matter, the physical operator decisions that adapt execution, and the prompt strategies that are effective, yet this knowledge is buried in imperative code with no inspectable or reusable structure, nor does it carry any provenance or metadata. Credo addresses this problem by recovering a structured declarative description of a searched harness, tagging each extracted primitive with relevant metadata, and cataloguing all of it with provenance. A compiler can then bind stored primitives to generate harnesses for new tasks without having to start the search over from scratch. This paper provides preliminary results demonstrating the potential of our approach and lays out a related research agenda that the database community is well-positioned to tackle, including cost-based compilation over declarative catalogs and catalog maintenance under model and workload drift.
1 Introduction
Agentic harness search can improve LLM applications, but it produces monolithic imperative programs that obscure reusable knowledge and force later tasks to search again. Credo proposes recovering that knowledge as typed, catalogued declarative primitives for reuse across tasks.
- Harnesses determine model and tool calls, supplied information, answer-acceptance conditions, and therefore program-level quality and cost.A harness may invoke a larger model only when a confidence estimate from a smaller model falls below a threshold.
- Agentic harness search evaluates candidate programs using observed accuracy and cost to guide subsequent iterations.The search agent proposes and executes candidates against a benchmark, following an evolutionary coding pattern.
- The resulting harness is monolithic, mixing logical steps, intermediate signals, conditional implementation choices, prompt strategies, and benchmark-specific constants.This structure leaves potentially generalizable knowledge embedded in task-specific imperative code.
- Credo recovers four typed primitive kinds: skeleton steps, beliefs, policies, and prompt templates, then stores them with provenance and metadata.The compiler can synthesize harnesses for new tasks by reusing extracted primitives from the catalog.
- Credo’s preliminary results report more than an order of magnitude in cost savings over agentic harness search.The paper presents this result as evidence of the approach’s promise and identifies a broader database research agenda.
2 The Credo Framework
Credo transforms a searched harness into a typed declarative description, tags and catalogs its reusable primitives, and compiles those primitives into harnesses for new targets. The framework separates logical plans from physical choices while preserving provenance and supporting fidelity and reuse evaluation.
- Credo converts a searched harness.py into a declarative description that can reconstruct or adapt the harness while separating reusable methods from source-dataset decisions.The finance-to-legal example illustrates recovery into typed primitives, catalog storage, and recompilation for a new workload.
- Structured Declarative Description: The skeleton is a DAG of named operator steps and dependencies that records what the harness computes and in what order.The finance example includes parse_query, retrieve, rerank, generate, and self_check.
- Structured Declarative Description: Beliefs are typed derived attributes available at plan points and consumed by policies to make physical operator decisions.They may be input-derived or computed from intermediate operator outputs.
- Structured Declarative Description: Policies evaluate beliefs and select physical implementations for logical steps, such as dense retrieval with k=12 or keyword search with k=8.This separates a step’s logical purpose from how it runs.
- Structured Declarative Description: Prompt templates encode discovered reasoning strategies, instruction patterns, output formats, or personas as natural-language templates with bindable slots.They are the physical operator implementations associated with skeleton steps.
- Building and Serving the Catalog: Credo performs offline catalog construction followed by online compilation, with abstraction, scope tagging, and cataloguing before target-specific reuse.Later targets use a cheaper compilation step instead of repeating source-harness search.
- Building and Serving the Catalog: Only functional L1 patterns enter the shared catalog; source-specific L2 patterns remain with their source, even after constants are lifted into slots.The two-tier catalog preserves both reusable general primitives and complete source-specific descriptions with provenance.
- Building and Serving the Catalog: The compiler binds open slots, connects primitives from target input to answer, and assembles a runnable harness.py, falling back to search if no combination meets the threshold.Evaluation checks both round-trip fidelity and compilation for new targets without another search.
3 Preliminary Results
Credo’s preliminary evaluation tests whether searched harnesses can be recovered into faithful declarative descriptions and reused across unseen domains. The results show near-round-trip fidelity, useful retrieval transfer, and reasoning transfer whose cost-accuracy value depends on source policies, while searched harnesses can overfit domain-specific data.
- Experimental design: Credo evaluates catalog construction, declarative fidelity, and primitive reuse across retrieval and reasoning datasets with search, in-distribution, and cross-distribution splits.The experiments use four datasets spanning finance, legal, math, and polyreason, with 100 search queries, 100 in-distribution queries, and 300 cross-distribution queries per dataset.
- 3.1 Catalog Construction: Table 3 finds retrieval programs are roughly twice as large as reasoning programs, at approximately 26 versus 14 primitives, while source-specific step counts are 0.0 across all domains.The result indicates that recovered logical plans transfer cleanly and source-specific details remain in decision logic rather than functional skeleton steps.
- 3.2 Declarative Fidelity: Round-trip compilation preserves harness behavior across all four domains, with accuracy differences no larger than ±0.011 and mean cost changes within ±0.013 ¢/q.A recover-and-compile call costs about $2.3–2.8 and takes a few minutes, compared with $43–72 and hours for the original search.
- 3.3 Primitive Reuse: Credo’s primitives improve both retrieval transfers, raising legal→finance accuracy from 15.7 to 28.3 and finance→legal accuracy from 43.4 to 66.3.The finance transfer recovers 76% of the distance to the searched result, while the compiler selects retrieval and model-routing primitives and specializes target-specific slots.
- 3.3 Primitive Reuse: Reasoning transfer depends on source-policy operating points: math→polyreason lowers cost to $0.03–0.04 per question but underperforms brief-only, whereas polyreason→math improves accuracy.The two math→polyreason variants reach 38.7 and 34.9 versus 43.3 brief-only; polyreason→math reaches 44.6 and 45.4 versus 39.6 brief-only.
- 3.3 Primitive Reuse: Searched harnesses show domain overfitting on cross-distribution data, while Credo’s compiled harnesses avoid the same drops and can outperform searched programs on transfer.Searched polyreason falls from 72% to 47% and math from 67% to 37%; Credo also costs about $2.5 and takes two minutes per recover-and-compile pass.
4 Open Challenges
Credo’s preliminary results reveal open challenges in turning recovered harness primitives into a full system for managing agentic execution. These challenges include cost-based compilation, catalog maintenance, execution-context recovery, and pluggable backends.
- Research Agenda: The paper presents Credo as a first step toward persistent and reusable optimization knowledge for agentic workload execution.It identifies cost-based optimization and declarative primitive specification as database systems problems.
- Cost-Based Compilation: Credo’s current compiler uses role matching and natural-language metadata rather than cost-based optimization.A cost-based compiler would need to estimate conditional operator quality, account for primitive interactions, and remain robust to sparse observations.
- Declarative Catalog Maintenance: Catalogs may degrade as they accumulate redundant, stale, erroneous, or dominated primitives.Proposed maintenance mechanisms include eviction, consolidation, and drift detection using belief distributions or selectivity.
- Execution Context Recovery: Credo currently recovers harness programs but not other searched execution context, including system prompts, skill definitions, and tool configurations.The remaining finance→legal accuracy gap may reflect context that the current abstractor does not capture.
- Pluggable Backends: Pluggable backends could target execution systems beyond Python harness code, but probabilistic operators require a precise behavioral contract.The paper contrasts behavioral fidelity tests with stronger semantic-equivalence guarantees available in traditional UDF settings.
5 Related Work
Credo connects LLM workflow construction and database techniques for recovering declarative structure. Its distinction is the post-construction artifact: reusable intermediate plan representations that expose computation, runtime information, and decisions independently of the original program.
- Positioning: Credo compares systems by whether their post-construction artifacts independently expose computation, runtime information, and resulting decisions.This representation-level comparison places Credo between LLM workflow systems and database techniques for declarative recovery.
- LLM Workflow Systems: Orchestration frameworks typically retain developer-authored program structure, prompts, and conversations, while declarative LLM systems specify computations and may optimize execution.ReAct delegates more control flow to model reasoning than frameworks such as LangChain and AutoGen.
- Search and Optimization: Coding-agent and prompt-optimization methods search programs or prompts but generally retain optimized artifacts tied to the searched setting.Context optimizers may additionally accumulate structured playbooks alongside the program.
- Database Precedent: Database UDF recovery exposes expressions and dependencies from imperative functions, enabling optimization across the procedural boundary.Credo adopts this representation-level objective for harnesses whose operators may invoke stochastic models.
- Reusable Artifacts: Prior reuse methods span typed-operator priors, capability bases, code abstractions, workflow repositories, and skill libraries.Credo focuses on the intermediate plan representation needed to manage transfer as a systems decision rather than on individual capabilities or complete programs.
6 Conclusion
The paper argues that searched harnesses should become managed declarative artifacts rather than remain only imperative implementations. Credo catalogs recovered primitives and provenance, compiles compatible structure for new domains, and preliminary results show preserved accuracy and cost alongside effective transfer.
- Conclusion: Credo recovers typed harness descriptions, records primitives and provenance in a catalog, and compiles compatible structure for new domains without another search.The system separates reusable harness structure from decisions tied to the source dataset.
- Conclusion: Preliminary evaluation shows that round-trip compilation closely preserves measured accuracy and cost while demonstrating effective cross-domain transfer.The conclusion characterizes these findings as preliminary results supporting the potential of managing searched harnesses declaratively.
- Research Agenda: Agentic workload execution produces optimization knowledge that is currently discarded, motivating persistent and reusable representations.The paper frames cost-based optimization and declarative primitive specification as research problems suited to database systems.