Source-linked AI summary
Walking on the DARKSIDE
Aldo Gangemi, Emanuele Bottazzi
TL;DR
LLMs can engage with sophisticated-sounding nonsense as if it were grounded because they do not natively track discourse exclusions, and POLANYI++ can reify those assumptions into an Extended Knowledge Graph. DARKPOLANYI adds explicit exclusion tracking and warrant classification to steer an LLM, achieving higher BSBench coverage than an unscaffolded baseline while partially scaffolding the pattern-vs-path gap.
Problem
LLMs tend to answer sophisticated-sounding questions built on fabricated authorities or misapplied mechanisms rather than questioning their grounding, and POLANYI++ inherits this vulnerability in its generated knowledge graph.
Method
DARKPOLANYI extends POLANYI++ with DARKSIDE’s NegativeTrail, contradiction tests, LabyrinthSignature, DelegationRiskAssessment, and Warrant Axis for classifying named commitments.
Results
1.96/2 mean judge coverage and 97.0% perfect identifications for DARKPOLANYI exceeded the unscaffolded baseline’s 1.84/2 and 91.5% perfect identifications on BSBench.
Takeaways & Limitations
The results support using an explicit XKG as persistent state for exclusions and the warrant axis as an epistemic firewall over fabricated commitments.
Takeaways & Limitations
BSBench is heavily weighted toward software/DevOps and finance, with sparse STEM nonsense outside physics; a domain-balanced extension remains a follow-up.
Abstract
from arXiv · showhide
Large Language Models (LLMs) recognise patterns but do not natively track the path of exclusions that a coherent discourse demands. When an input rests on a fabricated authority, a misapplied mechanism, or a surreptitious analogy, an unsteered LLM tends to engage with it as if it were grounded, and to reify the misstep into any structured output it generates. Logic-Augmented Generation (LAG) with POLANYI++, an LLM-steering method that uses heuristics, ontologies and problem solving methods for tacit knowledge extraction, produces an Extended Knowledge Graph (XKG) in OWL2, but inherits the same vulnerability: a sophisticated nonsensical input is reified into the graph alongside the legitimate triples, and is hardly detectable by automated reasoners since the XKG is generated jointly with the wrong assumptions. We introduce DARKSIDE, a coherence auditing method on top of POLANYI++. It formalises the trail as an explicit data structure of accumulated exclusions over discourse time, complemented by a warrant axis that classifies each named referent as Warranted, Unattested, Misattributed or Fabricated, with an escalation rule that pushes the DelegationRiskAssessment to UNSAFE when the fabricated rate is positive or the unsupported rate exceeds a threshold. We evaluate DARKSIDE as a steering layer over a Gemini 3 on BSBench, a 100-item adversarial corpus of sophisticated-sounding nonsense across software engineering, finance, healthcare, physics and law, with Claude Sonnet 4.6 as an independent judge. The empirical evidence supports an architectural claim: when an LLM forward pass is wrapped in an ontology-mediated negative-trail apparatus, the structural pattern-vs-path gap can be partially scaffolded. The XKG functions as the missing memory, and the warrant axis as an epistemic firewall.
1 Introduction
The paper identifies a structural gap between LLM pattern recognition and tracking the cumulative exclusions required for coherent discourse, then proposes DARKSIDE as a coherence-auditing extension to POLANYI++. Evaluation on BSBench reports stronger nonsense identification with the steered system than with an unscaffolded baseline.
- Motivation: BSBench contains 100 authoritative-sounding questions whose premises can be causally empty despite using real variables and grammatically correct framing.The example attributes quarterly EBITDA variance to invoice typography and dashboard colors, which have no stated causal channel into earnings.
- Motivation: LLMs tend to answer sophisticated-sounding questions on their presented terms rather than questioning whether their assumptions are grounded.The paper frames this as a structural difficulty with tracking commitments and exclusions over discourse time.
- Contributions: The introduction presents the XKG as an externalized memory for discourse exclusions and the warrant axis as an epistemic firewall.The listed contributions include the ontology vocabulary, warrant escalation, and BSBench evaluation.
- Method: DARKSIDE adds an explicit NegativeTrail, contradiction tests, LabyrinthSignature, DelegationRiskAssessment, and warrant classifications to POLANYI++.Its warrant axis distinguishes Warranted, Unattested, Misattributed, and Fabricated commitments, while escalation rules identify unsafe delegation.
- Evaluation: DARKPOLANYI combines POLANYI++ and DARKSIDE to steer a single Gemini 3 forward pass and evaluate audits with an independent Claude Sonnet 4.6 judge.The evaluation uses 100 BSBench items and a 0–2 coverage axis against gold nonsense rationales.
- Results: 1.96/2 mean coverage was achieved by DARKPOLANYI, compared with 1.84/2 for unscaffolded Gemini 3 Pro on the same benchmark.The reported perfect-identification counts were 96/99 for the steered system and 86/94 for the baseline.
2 Background and Related Work
The background argues that coherent understanding requires maintaining exclusions and distinguishing functional from discourse-breaking contradictions. It positions POLANYI++ as ontology-driven graph construction and DARKSIDE’s NegativeTrail and warrant axis as distinct from fixed-graph, training-distribution, decoding, and factuality approaches.
- Via negativa: DARKSIDE treats tracking what discourse excludes over time as essential for labyrinthine tasks whose prior decisions constrain later choices.The paper connects this negative work to delegated natural-language tasks and notes that tests externalize it in some programming domains.
- Related work: LLMs struggle with negation, contradiction, and long-range coherence, with generated summaries and stories containing substantially more incoherences than human originals.The cited studies also report failures to detect character behavior that contradicts established traits.
- Via negativa: The via negativa is defined as maintaining successive exclusions so that understanding preserves what is at issue rather than merely identifying positive patterns.The paper adapts the theological term to communication and discourse coherence.
- Contradiction: Contradictions can be encapsulated within a narrative or compromise the discourse, so an auditing tool must distinguish functional from vain contradictions.Examples include fictional nonexistence, unreliable narration, and Catch-22; the distinction requires tracking compatibility with prior commitments.
- Ontology-grounded extraction: POLANYI++ makes heuristics, methods, and the instruction set first-class components while constructing an ontology-grounded knowledge graph on the fly.Its approach is compared with ontology-driven retrieval-augmented generation, but it constructs rather than retrieves the graph.
- Distinction from adjacent methods: NegativeTrail accumulates discourse-time exclusions online, whereas adjacent methods re-weight, forbid, or prune a fixed hypothesis space.The warrant axis likewise classifies input referents against producer grounding before the audit verdict, rather than classifying outputs against a reference.
3 The POLANYI++ Framework
POLANYI++ is a function-based, deterministic framework that compiles registered heuristics, methods, ontologies, and source inputs into an Operational Instruction Set and an OWL2 Extended Knowledge Graph. Its three phases separate task relevance scoring, instruction assembly, and a single steered LLM generation pass.
- Function specification: POLANYI++ defines E = f(I, O, S, B, H, T, M, U), producing an OWL2 Turtle Extended Knowledge Graph from instruction, ontology, source, graph, heuristic, task, method, and prior inputs.The framework supports optional defaults and can generate a missing base graph with BASEONTOGRAPH.
- Outputs: The output is a Turtle file with provenance-annotated extension triples and a mandatory human-readable synthesis embedded as an ontology comment.The synthesis must include Executive Semantics, Hidden Dynamics, Strategic Anomalies, and Methodological Outputs.
- Registries: The framework uses 28 registered heuristics spanning interpretive layers and 29 methods, including TACIT, discovery-loop, AGENTIC, ACTIVE-INFERENCE, and ISITPROFILE methods.Heuristics are organized into foundational, mandatory, optional interpretive, and meta groups.
- Three-phase orchestration: Phase A scores heuristic and method relevance, Phase B assembles a pruned Operational Instruction Set, and Phase C performs one steered LLM forward pass.Phase B is a pure-Python assembly step, while Phase C receives the OIS together with the source and base graph.
4 DARKSIDE: Coherence Auditing via the Negative Trail
DARKSIDE audits coherence by maintaining discourse-time exclusions and distinguishing functional contradictions from idle hallucination drift. Its warrant axis classifies referents and applies a conservative delegation-risk escalation rule.
- Negative trail: DARKSIDE formalises the via negativa as a NegativeTrail that accumulates typed, provenance-bearing exclusions over discourse time.The method also includes LabyrinthSignature scoring for path-tracking demand.
- Mode A: Mode A extracts commitments, derives exclusions, checks new commitments against the trail, and audits constancy across the completed XKG.Exclusions arise through negation, incompatibility, causal-path closure, and selectional restriction.
- Mode B: Mode B audits the pipeline’s synthesis and provenance consistency, then assigns TRUSTLESS, SUPERVISED, or UNSAFE delegation-risk verdicts.The verdict reflects verifiability, mixed-task status, and path-dependency or constancy demands.
- Mode C: Mode C performs pre-commit checks against a live trail, blocking or repairing candidates and triggering periodic path-integrity warnings.Repair strategies include REPHRASE, RETRACT, PERSPECTIVALIZE, and ACKNOWLEDGE.
- Warrant axis: The warrant axis labels referents Warranted, Unattested, Misattributed, or Fabricated and aggregates fabricatedRate and unsupportedRate.Each commitment also carries source, note, and confidence metadata.
- Escalation rule: The OIS escalates DelegationRiskAssessment to UNSAFE when fabricatedRate > 0 or unsupportedRate > 0.40.The rule is intentionally conservative and deterministic for downstream consumers.
5 Experimental Setup
The experiment evaluates DARKPOLANYI on a 100-item corpus of sophisticated-sounding nonsense using Gemini 3 Pro with an independent Claude Sonnet 4.6 judge. The setup compares the full steered pipeline with an unscaffolded Gemini baseline using natural-language audit outputs.
- Task: The audit directive requests coherence, constancy, contradiction, warrant-axis, warrant-profile, and delegation-risk assessments for each case.It classifies named referents as Warranted, Misattributed, Fabricated, or Unattested and escalates on the specified thresholds.
- Pipeline: The pipeline injects a reification mandate requiring every named referent to receive a typed warrant classification and verbalised risk verdict.This mandate is inserted into the Operational Instruction Set when DARKSIDE and AUDIT are detected.
- Dataset: BSBench contains 100 well-formed nonsense questions annotated with 13 techniques and gold rationales identifying each semantic flaw.Its cases span software engineering, finance, physics, regulatory/legal, healthcare, and other technical domains.
- Arms and judging: The comparison uses DARKPOLANYI with Gemini 3 Pro against Gemini 3 Pro alone, while Claude Sonnet 4.6 judges natural-language audit text rather than graph triples.Both arms process the same questions; the steered arm supplies synthesis comments from each per-case XKG.
6 Evaluation
DARKPOLANYI outperformed an unscaffolded Gemini 3 Pro baseline on BSBench, while illustrative audits show that the warrant axis detects fabricated commitments and escalates delegation risk. The evaluation also identifies residual category-mistake errors and limitations involving judge dependence, domain imbalance, style contamination, model-family overlap, and incomplete ontology coverage.
- Headline results: 1.96/2 mean coverage was achieved by DARKPOLANYI on 99 valid cases, with 96 perfect identifications and a 97.0% top-score rate.One case failed extraction because the prescribed HUMAN-READABLE SYNTHESIS comment was absent.
- Headline results: The unscaffolded Gemini 3 Pro baseline scored 1.84/2 on 94 valid cases, with 86 perfect identifications and a 91.5% top-score rate.Six baseline cases failed parsing in the evaluation pipeline.
- Headline results: DARKPOLANYI eliminated partial credits and reduced top-score misses relative to the strong baseline, while outperforming the earlier 0/100 result reported for prior models.The comparison uses different evaluation conditions for the earlier result, so the direct baseline comparison is the Gemini 3 Pro arm on the same items.
- Error analysis: The warrant axis accounts for cases involving fabricated frameworks, authorities, and nested fabrications, whereas both systems share errors involving wrong unit of analysis, false granularity, or reified metaphor.The paper supplies the precise per-case technique and domain breakdown as supplementary material.
- Illustrative audits: In case bsb-73, two fabricated commitments produced fabricatedRate = 0.66 and unsupportedRate > 0.40, escalating DelegationRiskAssessment to UNSAFE.The audit still marked automated API fingerprinting tools as Warranted and received judge score 2.
- Ablation and threshold sensitivity: Under the two-case threshold ablation, both cases remained UNSAFE across the full τ sweep because the fabricated-rate disjunct fired, while disabling either disjunct exposed complementary threshold behavior.Case 73 flipped only at τ=0.80 when the fabricated disjunct was disabled.
7 Discussion
The discussion interprets the empirical result as support for ontology-mediated steering that externalises discourse exclusions and uses warrant classifications to govern delegation. It also identifies generalisation, verification, repair, and replication directions, while noting the evaluation’s domain imbalance.
- 7 Discussion: The empirical result is consistent with an architecture where POLANYI++ supplies explicit state and DARKSIDE models exclusions and fabricated commitments.The XKG acts as an external memory, while the warrant axis functions as an epistemic firewall; the system scaffolds around rather than refutes the pattern-vs-path gap.
- 7 Discussion: DARKSIDE’s warrant axis generalises across text-audit tasks through four-class classification and supports TRUSTLESS, SUPERVISED, and UNSAFE delegation verdicts.The proposed mapping treats TRUSTLESS as a tool-call greenlight, SUPERVISED as a human-in-the-loop signal, and UNSAFE as a provenance-backed refusal.
- 7 Discussion: Three follow-up directions are proposed: SHACL verification, multi-pass repair after UNSAFE verdicts, and closed-book replication with an unseen model.The repair direction rewrites the Operational Instruction Set for a second call, while closed-book replication packages the schema and instructions for independent evaluation.
- 7 Discussion: The research bundle will release instructions, runners, the BSBench snapshot, the DARKSIDE OWL2 vocabulary, SHACL shapes, per-case XKGs, and judgments under a permanent w3id IRI.The release is intended to support replication and the proposed closed-book evaluation.
8 Conclusion
The conclusion presents DARKPOLANYI as POLANYI++ extended with DARKSIDE’s negative trail and delegation-risk assessment. On BSBench, the steered system achieved higher judge coverage and perfect-identification rates than an unscaffolded Gemini 3 Pro baseline, supporting partial scaffolding of the pattern-vs-path gap.
- 8 Conclusion: DARKPOLANYI extends POLANYI++ with a Negative Trail and Delegation Risk Assessment for coherence auditing and trustless delegation.Its Warrant Axis classifies entities as Warranted, Unattested, Misattributed, or Fabricated, escalating the verdict to UNSAFE under fabrication.
- 8 Conclusion: 1.96/2 mean judge coverage and 97.0% perfect identifications exceeded the unscaffolded Gemini 3 Pro baseline’s 1.84/2 and 91.5%.The evaluation used the 100-item BSBench adversarial corpus.
- 8 Conclusion: Ontology-mediated negative-trail steering partially scaffolds the structural pattern-vs-path gap and supports refusing sophisticated-sounding nonsense on its own terms.The conclusion characterises the XKG as missing memory and the warrant axis as an epistemic firewall.