PaperlayerView exact revision on arXiv

Source-linked AI summary

Hybrid AI Patterns: Thirteen Ways to Couple Language Models with Rules, and How to Choose Among Them

Siegfried Handschuh, Götz-Henrik Wiegand, Lorena Raichle

arXiv:2610.40036v1cs.AI

TL;DR

Practitioners often reinvent ways to combine language models with executable rules for rule-governed documents and transactions. This paper defines a thirteen-pattern catalogue, shared invariants, and a decision guide grounded in measurements. Results vary by task: the hybrid excels on some rule-heavy tests, while a model alone performs better in a triage comparison.

  • Problem

    Practitioners applying language models to rule-governed documents, conversations, and transactions tend to reinvent coupling designs project by project.

    Source: §1 · p. 3

  • Method

    The paper presents thirteen coupling patterns in catalogue form, with shared invariants and a decision guide grounded in pre-registered measurements.

    Source: §1 · p. 3, §2.4 · p. 7

  • Results

    In synthetic intake protocols, the hybrid decided all 770 runs correctly versus the model’s 39 of 60 at 64 questions and 31 of 60 at 96; in triage, the model alone scored 274 of 280 versus the hybrid’s 243 of 280.

    Source: §4.2 · p. 34

  • Takeaways & Limitations

    The patterns make coupling trade-offs explicit, including when model-only performance is as good as or better than a hybrid.

    Source: §4.1 · p. 27, §4.2 · p. 34

    Catalogue completeness is unproven, and most measurements use synthetic protocols and one model family.

    Source: §11 · p. 74

Abstract

from arXiv · show

Language models handle text well, and rule engines give conclusions with proofs. Systems that combine the two are usually described by example, and the same coupling turns up under different names in agent frameworks, guardrail toolkits and neuro-symbolic research. This document defines thirteen patterns for coupling a language model with an executable rule language, in the catalogue form introduced for object-oriented design: intent, motivation, applicability, participants, collaborations, consequences, implementation, known uses and related patterns. Four patterns concern what crosses the boundary between model and rules, five who controls whom, three how the model takes part in the reasoning, and one how knowledge is kept over time. Five invariants hold for every combination, and five failure patterns name the couplings that break them. We relate the catalogue to published pattern catalogues and taxonomies of hybrid and neuro-symbolic systems and show where they overlap with it. A decision guide leads from a problem to a pattern. It rests on our measurements, including the cases in which a model alone did as well as the hybrid or better, and it reports what each design cost.

1 Introduction

Language models and executable rules can be coupled in many ways, but practitioners often reinvent those couplings. The paper presents a reusable pattern catalogue, maps it to published work, and offers a measurement-grounded decision guide.

1 Introduction

  • Organisations apply language models to documents, conversations, and transactions governed by explicit rules, yet practitioners tend to reinvent ways to combine them.One option puts rules in prompts; another keeps them executable and assigns the model tasks rules cannot perform, such as reading text or proposing candidates.Source: §1 · p. 3
  • Patterns provide a shared vocabulary for recurring coupling problems and make trade-offs explicit, including cases where a model alone is better.The catalogue follows the structure used for object-oriented and system-architecture patterns.Source: §1 · p. 3
  • The patterns were derived from FrameX applications but can run on any rule engine that can request model calls, store answers as data, and derive conclusions from them.FrameX notation is used because the measurements were made with that engine.Source: §1 · p. 3
  • The catalogue defines thirteen coupling patterns, five shared invariants, and five failure patterns that break those invariants.It also maps the patterns to published catalogues and taxonomies and provides a decision guide grounded in pre-registered measurements.Source: §1 · p. 3

2 Foundations

The foundations specify how FrameX connects model calls to rule execution, define five invariants for those couplings, and explain the elementary moves and catalogue conventions used to describe the patterns.

2.1 What FrameX adds for coupling

  • FrameX listings assume basic F-Logic syntax and an explicit world declaration; under an open world, missing facts are unknown, and negation is true only over declared-closed elements.A condition without a class can also match a request object, so rule subjects should be class-qualified and request fields kept distinct from derived slots.Source: §2.1 · pp. 4, 6
  • A FrameX external request becomes an object for the host to execute; its answer returns as a receipt, and failed calls add an error rather than result rows.Receipts store flat values as facts, while the model call can be replayed from a stored answer on an identical request.Source: §2.1 · pp. 4–5
  • FrameX adds read-only operations through which a model can query or validate rules, alongside host measurements of quotes and rule-selected context.The host checks whether quoted text and values occur in source text; rules determine whether those measurements count and which statements enter context.Source: §2.1 · p. 6

2.2 Five invariants

  • Every combination treats model text as data, records dated answers for replay and audit, lets rules decide, accepts statements using checkable evidence, and keeps failures distinct.The division of labour assigns perception, language, and ambiguity to models, and guarantees, audit, and explanation to rules; an explicit validated record passes between them.Source: §2.2 · p. 6

2.3 Four elementary moves

  • The four moves are Request, Receipt, Accept, and Ask: answers enter as receipts, while Accept decides whether a proposal becomes a fact, is refused, or remains open.Ask derives a question for a person or evidence needed to settle an open goal; its answer also returns as a receipt.Source: Figure 1 · p. 7, §2.3 · p. 7
  • The thirteen patterns are compositions of the four elementary moves, differing in what triggers a request and what Accept decides.Critique and Repair is the only pattern that repeats a move, and does so under a bound.Source: §2.3 · p. 7, Table 3 · p. 8

2.4 The catalogue form

  • The catalogue uses pattern headings adapted from Gamma et al. and boxology symbols to show participants and collaborations; rule graphs show one program’s derivation chain.The catalogue adds an Evidence heading for measured consequences, and patterns 12 and 13 were added from hackathon submissions.Source: §2.4 · p. 7

3 Patterns at the boundary: what crosses it

This section describes boundary patterns that determine what a language model contributes between unstructured inputs or human language and rule-based reasoning. The patterns include proposing facts, drafting checkable formal artefacts, formalizing questions, and selecting passages or phrasing around rule-supplied values, with attention to repair feedback and safeguards.

3.1 Pattern 1: Model as Sensor

  • Model as Sensor has the model read unstructured inputs and propose facts; rules decide acceptance, for example by checking a verbatim quote and whether the document type permits the statement.Accepted facts retain provenance and a reason; ungrounded answers remain rejected with their reason.Source: §3.1 · pp. 10–11
  • Use Model as Sensor when rules can express the decision and readings can be measured; a plausible wrong value that passes every check can still slip through, and an unknown leaves the criterion open.It costs one call per input and question; the paper recommends a second reader or cross-check for values that pass measurable checks.Source: §3.1 · pp. 10–11
  • Rule-derived lists were correct on 10 of 10 questions versus model-alone exact lists on 0 of 6; under strict quote checking, extraction scored 45% against 60% for the model alone.Extraction cost 43.6 cents once, against 1.2 cents per question for the model alone. The strict quote rule refused correct values, so acceptance rules must fit the reader’s ability to quote.Source: §3.1 · pp. 13–14

3.2 Pattern 2: Model as Author

  • Model as Author has the model draft a formal artefact, which a checker tests for syntax, allowed vocabulary, regressions, and changed decisions before an expert adopts, edits, or refuses it.Writing becomes cheap while checking becomes the bottleneck; without complete checks, a well-formed draft that passes every case may still be wrong where no case covers it.Source: §3.2 · pp. 14–15
  • A change request fixed six of six failing cases, broke none, and passed review; vocabulary-mapping suggestions remained recorded proposals.The passage also situates the pattern alongside domain-model acquisition with experts, action-schema generation, and an engineered model of an actor, with the model in the engineer’s role.Source: §3.2 · p. 18
  • In five live runs, every model-drafted rule passed all six hidden cases; three passed on the first attempt and two after one repair.Across 36 drafts for 12 policies, messages naming the fix raised repairs within two attempts from 8 of 15 to 14 of 15. Static checks found every seeded structural fault at every size at no model cost; model reviews found an unreachable rule in only 2 or 3 of 10 artefacts at 64 rules.Source: §3.2 · p. 18
  • Repair succeeded in 5 of 10 runs with “expected ‘;’ ” feedback and 10 of 10 when the message named the fix.The passage reports that, in a checking study, model reviews of code and prompts found an unreachable rule in only 2 or 3 of 10 artefacts at 64 rules, while static checks found every seeded structural fault at every size at no model cost.Source: §3.2 · p. 18

3.3 Pattern 3: Model as Translator

  • The translator lets the model turn a person’s question into a checked request and phrase the rules’ verdict and proof, while the rules retain decision-making.Requests can use closed, read-only operations or validated free queries; the host checks the reply against the decision.Source: §3.3 · pp. 18–19
  • A back-translation helps catch a correctly answered but wrongly translated question, and explanations must bind claims to proof; free queries can still ask the wrong question.Closed operations preserve invariant 1 without qualification, while free queries can be checked for form, vocabulary, and read-only status but inherit the consequences of Model as Author.Source: §3.3 · p. 19
  • On a clean five-document case, model-only answered correctly in 12 of 12 runs at lower cost than the hybrid, 0.28 against 0.72 cents per case.On easy cases, the hybrid adds a proof, a statement of missing evidence, and a fixed decision word.Source: §3.3 · p. 21

3.4 Pattern 12: Model Selects, Rules Write

  • The model selects passage identifiers and phrases around hidden values, while rules verify selections and insert source values so model-written content never reaches the person.The model sees numbered passages but not the values; rules refuse identifiers outside the source and phrasing that contains a number of its own.Source: §3.4 · pp. 23–24
  • The pattern suits verbatim source-backed answers, but cannot synthesize across passages; remaining errors are irrelevant or missing selections, and it costs one call per selection and phrasing.Source: §3.4 · pp. 23–24
  • Official terms and all facts appeared verbatim in 99 % of 120 alerts with Apertus v1.5 8B versus 45 % with free translation (70B: 98 % versus 71 %).A beneficial-ownership tool got 14 of 14 cases right with 8B on a CPU, with no held-out set; figures came from committed result files and were not rerun.Source: §3.4 · p. 25
  • Recall was 0.764 for 8B versus 0.894 for 70B on 537 court decisions, while a no-model rule baseline matched 70B on the statistics QA main set.Source: §3.4 · p. 26

4 Patterns of control: who decides

This section surveys control patterns that determine how rules, models, and sometimes people share decisions, including rule-directed dialogue, bounded model repair, delegation, competence-based routing, and combined verdicts. It describes the reported measurements and demonstrations, including cases that were not measured, to show the value and limits of these couplings.

4.1 Pattern 4: Model Proposes, Rules Dispose

  • Pattern 4 lets the model propose actions or choices, but rules admit, refuse with reasons, or narrow proposals before the host executes them.Rules can require evidence for completion; effects become auditable and bounded, but refused proposals can waste turns, constraint conflicts go to a person, and admission does not establish that a proposal is best.Source: §4.1 · p. 27
  • 16.4 % of simulated paths violated constraints when the model chose the path, versus 10.5 % when a rule engine did, over three seeds.This Simulator Disposes result is reported from Hack Apertus submissions; Pattern 4 itself has not been isolated in a measurement.Source: §4.1 · pp. 29–30

4.2 Pattern 5: Rules Steer the Model

  • Pattern 5 makes rules choose what the model is asked, when, with which context and tools, and the next step; the model phrases the current question.A judge assesses whether an answer meets the step, and rules branch, re-ask, hand over, or end the process; unanswered questions do not decide branches.Source: §4.2 · p. 32
  • Pattern 5 can execute a protocol completely regardless of length while keeping the model’s work per call small, but costs one or more calls per turn.Rules must catch a judge verdict that contradicts its own reason.Source: §4.2 · p. 32
  • After one judge correction, the hybrid reached 278 of 280 triage runs and 199 of 200 unseen callers, level with the model alone within noise, at lower cost.In the initial head-to-head, the model alone scored 274 of 280 against the hybrid’s 243 of 280; the hybrid’s errors shared one cause, corrected in the judge. Across synthetic protocols, the hybrid decided all 770 runs correctly, while a model applying the same rules reached 39 of 60 at 64 questions and 31 of 60 at 96, with 45 of 50 errors unsafe.Source: §4.2 · p. 34

4.3 Pattern 6: Critique and Repair

  • Rules classify check failures and provide the problem for a bounded retry; repeated failures stop, while undetectable plausible errors remain outside the check’s protection.Naming the fix raised first-request repairs from 5 of 10 to 10 of 10; in a recorded run, a draft missing its final full stop passed all cases after a repair request specified “expected ‘.’ ”.Source: §4.3 · pp. 36–38

4.4 Pattern 7: Division by Competence

  • Rules decide covered cases; unknown, unclear, or conflicting cases go to a model for a recorded assessment and then to a person, while recurring model verdicts can become candidate rules.The model assessment travels with the case but does not decide it; the rules’ own result, not model confidence, is the gate.Source: §4.4 · p. 38
  • Division by Competence keeps cases covered by program rules with the rules and has the library assess other cases, handing them on with the model’s reason; its demonstration has not yet been measured.The proposed measurement would report the share handed on, the accuracy of rule-decided cases, and cost per decided case.Source: §4.4 · pp. 39–40

4.5 Pattern 13: Fail-Safe Union

  • Fail-Safe Union declares an order of outcomes and selects the strictest verdict; silence, failure, or an answer outside the order counts as cautious, and the sources behind the result are named.Adding a source can only tighten the outcome, but false alarms accumulate and the hold rate must be measured; a failed call must not count as agreement.Source: §4.5 · pp. 42–43
  • Use Fail-Safe Union when outcomes can be risk-ordered, stricter errors cost less, sources see different things, and a cautious outcome can be followed up; avoid it when errors cost alike or strict outcomes cause harm.In those cases, report disagreement as unclear or hand it on instead.Source: §4.5 · p. 42
  • 178 of 300, 89 of 300, and none of 300 invoices were paid by the model, code checks, and strict union; the union held 7 of 200 legitimate invoices versus 2 for the model.On a blind run before later checks, the respective figures were 337, 174, and 29 of 600.Source: §4.5 · p. 44
  • Redaction recall was 38.7% for rules, 57.7% for the model, and 78.5% for the union, yet 81% of documents still leaked at least one entity.The union is stricter than either source, but the results show that it is not safe by itself.Source: §4.5 · pp. 44–45

5 Patterns in the reasoning itself

Patterns 8–10 bring models into rule-based reasoning by having them propose hypotheses, critique outcomes, or learn from rule-generated supervision, while rules retain authority over facts, conclusions, and production checks. They cover evidence-gated review and testing, as well as using rule-labeled cases to produce examples or evaluation data, with training kept outside the rule engine.

5.1 Pattern 8: Model as Hypothesis Generator

  • The model proposes hypothetical explanations; rules test each in a session copy and seek deciding evidence, while hypotheses never become facts and incomplete search proves nothing.The pattern targets diagnosis, reconciliation, investigation, and unknown goals where candidate generation needs world knowledge but testing needs exactness.Source: §5.1 · p. 47
  • Four of five candidates passed the amount test, including an “unrecorded deposit” that contradicted the ledger’s direction.The fuller pattern previews each candidate in a session copy, retaining it only if it derives the observation without violating constraints, then identifies evidence questions that would decide among survivors.Source: §5.1 · p. 48

5.2 Pattern 9: Model as Critic of the Rules

  • The model reviews rule decisions against their rationale, but its remarks only open human review and never change rules or conclusions.A remark must identify a changed case and quote the rationale; a reviewer disposes of findings, approval waits for their disposition, and the critic supplements rather than weakens complete checks.Source: §5.2 · p. 51
  • In the pilot, the critic found 29 of 30 seeded defects, stayed silent on 29 of 30 clean changes, and all 34 remarks passed verbatim-quote checks.In a separate checking study, reviews almost always found contradictory rules and unknown facts, but found missing rules in only 3 of 10 rule reviews at 64 rules and none of the code or prompt reviews; harder defects and longer reviews remain unmeasured.Source: §5.2 · p. 53

5.3 Pattern 10: Rules Teach the Model

  • Rules teach the model through labels, selected examples, and grades with proofs, while remaining in production as an independent check.Source: §5.3 · p. 55
  • Use this pattern when a model must work where rules are not called, learn domain vocabulary, or needs trustworthy labels for training or prompt examples.Source: §5.3 · p. 55
  • Rules label generated cases with decisions and missing requirements, select relevant examples, and grade model answers against the labels, recording missed gaps.The resulting labels and grades, with proofs, can leave the engine as training or evaluation data; training itself happens outside the engine.Source: §5.3 · pp. 55–56
  • The labels are only as good as the rules, and the model’s learned output is never the authority; the same rules can grade the taught model.Source: §5.3 · p. 55

6 Patterns over time

Rule-Governed Memory keeps remembered statements with their utterances and quotes, while rules govern admission, supersession, forgetting, and what the model may recall. It offers traceable, recomputed memory, at the cost of a Sensor call per utterance and a growing store.

6.1 Pattern 11: Rule-Governed Memory

  • Rules select the statements for each prompt, and the reply must cite only statements that were sent and remain current.A rule checks the cited identifiers against the selected context and current memory.Source: §6.1 · pp. 59–60
  • Rules decide which remembered statements are admitted, current, and available to the model, rather than leaving memory selection to the model.Each statement retains its utterance and quote; source-specific slot policies determine what a source may establish.Source: §6.1 · p. 59
  • Newer admitted statements supersede older ones without deleting them, so the system can retain a value's history.Supersession applies to statements about the same slot; the current value is derived by rules.Source: §6.1 · p. 59
  • Forgetting an utterance withdraws statements based on it and recomputes acceptance, allowing an older statement to become current again.Recorded model answers that rest on the forgotten utterance are also removed, preventing replay from restoring them.Source: §6.1 · pp. 59–61
  • Every remembered value has a source, quote, date, and history, but the design costs one Sensor call per utterance and a growing store.Only a request to forget removes content; a wrongly admitted statement persists until superseded or forgotten, and source conflicts need a person.Source: §6.1 · p. 60
  • The pattern is illustrated by one demonstration run and has not yet been measured.The library memory.fx derives admission, supersession, context selection, answers, refused citations, and removal of forgotten utterances from the answer store.Source: §6.1 · p. 62

7 Compositions

Applications combine patterns rather than using each coupling in isolation. Three combinations recur across the catalogue's source applications.

7 Compositions

  • Three combinations recur: a steered flow with a judge, repair, and hand-over; an agent with rule-admitted proposals and rule-steered context; and an authoring loop driven by checker feedback.The combinations are 5 + 1 + 6 + 7, 4 + 5 + 6, and 2 + 6, respectively.Source: §7 · p. 64, Table 5 · p. 64
  • Rule-Governed Memory combines Model as Sensor for storing with Rules Steer the Model for recall.Table 5 lists pattern compositions in the applications from which the catalogue was drawn.Source: §7 · p. 64, Table 5 · p. 64

8 Failure patterns

Five failure patterns name recurring couplings that break the catalogue's invariants. They resemble valid patterns, so the catalogue identifies what each breaks and the pattern that can replace it.

8 Failure patterns

  • Silent Rule Override changes a verdict outside the rules, leaving the deciding step out of the proof and breaking invariant 3.Write the correction as a rule that reads the receipt; a correction that only tightens outcomes can instead be a source in a Fail-Safe Union.Source: §8 · p. 65
  • Circular Evaluation uses cases, labels, or criteria derived from the system being measured, so the numbers measure agreement with oneself.Fix cases, truths, and criteria before the first model call, and derive labels by rules or construction.Source: §8 · p. 65
  • Self-Judge lets the answer's author, or a same-family model with the same prompt, judge correctness, risking shared errors and failing independence.Use rules for checkable claims and a different model or prompt for the rest.Source: §8 · p. 65
  • Prose Policy keeps the decision policy only in prompt sentences, making it untestable against cases and leaving no trace of which sentence governed an answer.Turn the policy into rules and let Rules Steer the Model put only the current step's needs in the prompt.Source: §8 · p. 65
  • Decorative Check records a check that no rule uses, so the check changes no outcome and breaks invariant 4.The proposed remedy is to have an acceptance rule read every check and inspect for receipt fields without outgoing rule-graph links.Source: §8 · p. 65

9 Related catalogues

The catalogue relates its patterns to hybrid-system taxonomies and software-engineering pattern collections, then identifies what existing classifications miss. Its additions include control direction, symbol status and time, while its comparisons also yield practical cautions about checks, feedback and pattern choice.

9.1 Mapping

  • Published taxonomies classify component types, data flow and enhancement direction, while agent and software-engineering catalogues name mechanisms and workflows.The mapping compares the catalogue with neuro-symbolic classifications, guardrail toolkits and systems that translate for solvers or generate candidates for critics.Source: §9 · p. 66, §9.1 · p. 66

9.2 The patterns in boxology notation

  • The boxology notation represents instances, models, processes and actors, but does not by itself encode control direction, symbol status with provenance, or iteration over time.The catalogue uses model:gen and model:sem labels and adds dashed request arrows; requests, receipts, accept and ask encode control, provenance and status.Source: §9.2 · pp. 66–67

9.3 What the catalogue adds

  • All thirteen patterns are Symbolic[Neuro]: rules issue requests and model answers return as data, a direction chosen for accountability through loose coupling and explicit provenance.The catalogue also treats dated receipts as identities and records for iterations, while Rule-Governed Memory handles supersession, withdrawal and current-value queries over time.Source: §9.3 · pp. 67–68
  • The catalogue distinguishes failed calls, malformed replies, unclear judgments and logical unknowns, making model answers replayable observations that can be routed by result.The authors also report that no catalogues they read made every model answer replayable or separated those outcomes.Source: §9.3 · p. 68

9.4 What the catalogue lacks

  • The catalogue identifies uncatalogued or unnamed cases, including rules deriving conversational steps, recurring answers becoming rules, and models reviewing rules themselves.It places several other gaps or candidates—such as voting across answers, routing before a model, and pooled critics—alongside these cases.Source: §9.3 · p. 68, §9.4 · p. 68

9.5 What the catalogue learns from them

  • A sound check supplies correctness: back-prompting from a sound validator raised executable plans from about 12% to 82% in one planning domain.The comparison is reported for LLM-Modulo; the authors state that the same correctness principle applies to patterns 4 and 8.Source: §9.5 · p. 69
  • Self-correction without external feedback lowered accuracy, while rules can serve as the external signal; judges’ answer-order bias can be addressed by asking in both orders and requiring agreement.The reported judge gave the same verdict after answer swaps only about two thirds of the time; references and examples reduced failures.Source: §9.5 · p. 69
  • The catalogue composes thirteen patterns from four elementary moves and pairs its decision guide with pre-registered measurements, unlike the cited agent-pattern decision model, which is not empirically validated.Practitioners advise starting with the simplest option and using evaluator loops only when criteria are clear.Source: §9.5 · p. 69

10 Choosing a pattern

The guide selects a coupling by the problem and its measurable forces, first checking whether a hybrid is needed. Its pre-registered measurements show where hybrids help and where a model alone is as good or better, and the guide favors the simplest option that meets the requirement.

10.1 First question: is a hybrid needed?

  • Through 48-question conversations, prompted reasoning was not measurably less accurate than the hybrid; the accuracy break-even lay at 64 questions.The model scored 58 or more of 60 against 60 of 60 for the hybrid in the shorter conversations.Source: §10.1 · p. 70
  • On a clean five-document review, the model alone decided 12/12 runs correctly at 0.28 rather than 0.72 cents and in a third of the time.The guide also reports that pricing cached-input discounts removes the hybrid’s conversation-cost advantage at 64 and 96 questions.Source: §10.1 · p. 70
  • A hybrid is needed when proof or auditability, prohibited actions, many rules or long processes, aggregation, weaker available models, or checked rule changes are requirements.The guide says its questions are not exclusive and may lead to composed patterns.Source: §10.1 · p. 70, §10.2 · p. 70

10.2 From problem to pattern

  • On the measured ladder, a 9B model sufficed for the hybrid, whereas no model up to 70B sufficed as a pure model; some cases remained ambiguous.Lists and counts are exact in rules.Source: §10.2 · pp. 70–71

10.3 What the guide rests on

  • 770/770 hybrid decisions were correct across synthetic protocols of 4–96 questions, while the model applying the rules itself fell to 39/60 at 64 questions and 31/60 at 96, with 45 of 50 errors unsafe.The designs and criteria were fixed before the runs.Source: §10.3 · p. 71
  • Static checks found every seeded structural fault in rules at every size with no model cost, while model reviews lost unreachable and missing rules as size grew.Five of five drafting runs ended with a rule passing all hidden cases, and error messages naming the fix affected the number of repairs needed.Source: §10.3 · p. 72
  • At 100 agreements, a model alone produced no exact exhaustive list at low effort, while the hybrid answered 10/10 correctly.A completeness check left no silently wrong answer; the extraction cost was paid once.Source: §10.3 · p. 72
  • For classifying 77 intents with an 8-billion-parameter model, rules directing a group-then-label choice gained 6 points over one prompt with all labels; putting groups into one prompt instead collapsed.Source: §10.3 · p. 72

11 Limits

The catalogue is drawn from one engine and its applications, checked against literature, and is not proven complete. Its measurements are uneven across patterns and rely largely on synthetic protocols and one model family.

11 Limits

  • The catalogue comes from one engine and its applications; coverage of couplings seen so far around a symbolic, offline core does not prove completeness.Source: §11 · p. 74
  • Patterns 12 and 13 draw on hackathon results that were not rerun, and patterns 4, 7, 8, 10, 11, 12 and 13 lack their own FrameX measurements.Pattern 9 is supported only by a pilot on easy cases; patterns 12 and 13 are checked against recorded receipts only.Source: §11 · p. 74
  • Most measurements use synthetic protocols and one model family, while the capability ladder mixes model families and reasoning modes.Couplings requiring constraint solving, plan search, probabilistic inference over model scores, differentiable logic inside the model or training itself are outside the catalogue.Source: §11 · p. 74

A Boxology legend

The legend uses established boxology node kinds and labels, while identifying the request arrow and notes as additions by the authors.

A Boxology legend

B Pattern libraries

These pattern libraries use executable rules to organize model-derived information and calls: libraries are loaded separately, while rules constrain, check, route, and record outputs, outcomes, evidence, and conditions for completion or approval. They also govern which model-derived statements remain accepted, current, recallable, and citable, including routing forgetting requests.

Library structure

  • Libraries for patterns 1–11 include each pattern’s program in library form, tests, and a recorded run; patterns 12 and 13 have no libraries yet.The library is loaded before the program, which retains domain-specific decisions, actions, cases, tests, and labels; library-defined names carry prefixes to avoid collisions.Source: B Pattern libraries · p. 76

Sensor library

  • Pattern 1’s sensor library turns model answer rows into proposed source statements, accepting only quoted statements allowed for that source kind and rejecting unsupported rows.The host checks every quote against the source text; a failed model call proposes nothing.Source: sensor.fx: pattern 1, Model as Sensor · p. 76

Author and repair libraries

  • The pattern 2 author library checks every draft while keeping cases hidden from the model, and pattern 6 repair retries failed drafts with the rule and checker feedback.It derives outcomes including adoptable, stalled, and gave-up states, with attempts bounded by the policy’s maximum.Source: author.fx: pattern 2, Model as Author,… · pp. 77–78

Translator library

  • Pattern 3’s translator library validates model-translated fields against each field’s closed allowed list, refuses invalid requests, and asks the model to phrase the rule-derived decision.Decision rules read only valid fields; the display-only all-fields-valid check is kept out of those rules to avoid negating over receipts.Source: translator.fx: pattern 3, Model as… · pp. 78–79

Pattern 4: Model Proposes, Rules Dispose

  • In Model Proposes, Rules Dispose, the model proposes task actions, but rules admit only granted actions, track outcomes, and bound steps.Completion requires an allowed finish action and recorded evidence; otherwise the task can stop at its step bound.Source: agent.fx: pattern 4, Model Proposes,… · pp. 79–80

Pattern 7: Division by Competence

  • In Division by Competence, rules decide cases they cover and route other cases to a model assessment and a person.The assessment is recorded as a note; a failed model call produces a failure note.Source: competence.fx: pattern 7, Division by… · p. 80

Pattern 8: Model as Hypothesis Generator

  • In Model as Hypothesis Generator, rules test model-proposed candidates, retain survivors, and identify what a person should ask about.Candidate fields are kept by row and field, and a candidate never becomes a fact.Source: hypothesis.fx: pattern 8, Model as… · pp. 80–81

Pattern 9: Model as Critic of the Rules

  • In Model as Critic of the Rules, remarks count only when grounded in a changed case and a found quote, and approval requires every remark to be disposed of.The library also derives a blocked state for a failed critique call.Source: critic.fx: pattern 9, Model as Critic… · p. 81

Pattern 10: Rules Teach the Model

Rule-Governed Memory

  • Rules accept statements only when their source kind may establish the slot, reject ineligible statements, and mark older accepted statements noncurrent when a newer one exists.A statement is current when accepted and not marked as having a newer replacement.Source: memory.fx: pattern 11, Rule-Governed… · p. 83
  • Forgetting marks an utterance as forgotten and invokes a separate operation to remove its text from the answer store.The rules also prevent forgotten utterances from establishing accepted statements.Source: memory.fx: pattern 11, Rule-Governed… · p. 83
  • Recall selects only current statements matching a step’s required slots, and rules refuse answers that cite statements not sent to the model or no longer current.The example also tracks whether the model’s cited identifiers were among the identifiers supplied in context.Source: memory.fx: pattern 11, Rule-Governed… · p. 83

C Rule graph legend

Table 9 lists the elements used in the rule graphs and flow diagram; a companion script draws the graphs from a program using Graphviz.

C Rule graph legend

Related papers

Loading 2610.40036v1…