Source-linked AI summary
A Literate Programming Environment for Human and Machine Agents
Adam T. Burke
TL;DR
Language-aware coding agents can neglect declarative artifacts, while existing approaches may require translation layers and human inspection. This paper presents a programming language and toolset with a parse-tree-based name-graph intended to improve agents’ use of context windows.
Problem
Coding agents may neglect declarative artifacts, and existing approaches can require translation layers and human inspection.
Method
The paper presents a programming language and toolset centered on a parse-tree-based name-graph for natural-language-literate coding.
Results
The paper reports a working programming language and toolset designed to help coding agents make more effective use of the context window.
Takeaways & Limitations
Co-locating prose and executable elements provides the basis for machine-assisted literate programming within the context window.
Takeaways & Limitations
The paper focuses on the language and tooling environment’s design and example programs rather than experimental evaluation.
Abstract
from arXiv · showhide
This paper introduces an environment for constructing literate programs in concert with language-aware machine agents. This environment includes a grammar for executable program essays, a parser that treats names as first-class objects, an internal name-graph which relates prose, names and executable artifacts, and a binding mechanism for existing languages and testing toolsets. This supports co-location of code with its most relevant natural language and structured data context, making better use of Large Language Model (LLM) context windows. It also provides LLM coding agents with a toolset more analogous to the symbol-aware search and usage information available in human programmer-facing Integrated Development Environments (IDEs). We describe a working implementation with bindings to three established programming languages, and several example programs.
1 Introduction
The paper proposes notlob, a literate programming environment in which human and machine agents work with essays combining natural language, executable code, and machine-checkable artifacts. Its name-graph and tooling are intended to improve agents’ use of context and provide IDE-like support.
- Natural-language text dominates specification and design organization in programs developed with language-aware AI tooling.
- Notlob combines prose and executable elements in a machine-interpretable language, with embedded code fragments using existing languages and libraries.
- Its name-graph relates concept names and executable symbols, while associated tools expose those relationships as IDE-like support for agents.
- Co-locating prose, code, and validation keeps relevant context together and supports navigation through the name-graph within the LLM context window.
- The paper focuses on language and tooling design and example programs; experimental evaluation is left to future work.
2 Design
Notlob structures programs as technical-report-like prose with embedded executable blocks, keeping explanations, execution, and supporting checks in one source artifact. It delegates execution to existing languages, tools, and libraries through bindings.
- 2 Design: Notlob consists of structured prose with embedded executable blocks, and its file macro-structure is modeled on a technical report.
- 2 Design: The main body contains runtime functions, data structures, and underlying concepts, while supplemental text contains tests, appendices, references, and imports.
- 2 Design: Unlike upstream specification pipelines, notlob co-locates prose, code, and checks in a single source artifact.
Preprint
Notlob combines prose, executable code, tests, and structured semantic relationships in a single literate-programming source format. Its grammar, bindings, and name-graph support both execution and navigation by human and machine agents.
- Overview: A notlob file co-locates explanation, executable code, and supporting checks so relevant context remains adjacent for human and machine agents.The format supports putting explanation, execution, and checks together in one file, with context often immediately adjacent.
- Grammar: The grammar uses Markdown-style headings, indented executable blocks, and sigils to distinguish language bindings, tests, and execution entry points.Reserved headings include #Binding, #Tests, #Appendix, and #References; sigils include ~language, ~property, ~example, ~test, and ~run.
- Language bindings: Bindings define the programming language, unit-test library, property-test library, and linter used to interpret executable blocks.Examples distinguish exposition-oriented ~example blocks, supplementary ~test blocks, and formal invariants declared with ~property.
- Names and semantics: Names, titles, and headings are first-class syntax objects used to connect prose, concepts, and executable artifacts.The name-graph treats names as load-bearing structures and relates them to prose and code through the parse tree.
- Name-graph: The name-graph provides file-adjacency and concept-adjacency, enabling navigation and consistency checks across prose and code.It can be explored by human and machine programmers and used for consistency checks across the codebase, including prose.
Preprint
Table 2 presents an overview of commands in the Notlob toolchain.
- Table 2 provides a command overview for the Notlob toolchain.
3 Implementation
The implementation provides a Python-based Notlob project with language bindings, deterministic file organization, command-line tooling, and graph-based support for machine agents.
- Implementation: The design ideas are implemented in an open-source Python project using a Lark grammar as its basis.
- Language integration: Bindings support Haskell, Python, and TypeScript, while sigils provide shortcuts for external tooling and generated-source handling.The implementation includes ~external, ~on-build, and ~keep-generated-src for executable-layer integration.
- Project layout: Notlob files use the .lob suffix and a deterministic heading-to-directory layout enforced by the compiler.A heading such as #Roman Numerals maps to roman/numerals.lob.
- Toolchain: Command-line tools compile projects, resolve dependencies, run tests, render Markdown, export or query the name-graph, and check semantic consistency.Missing imports and unused references are errors, while possible typos and style violations produce information-only warnings.
- Machine-agent support: The name-graph is intended as structured input for LLMs to identify semantic inconsistencies and provide graph-based navigation analogous to IDE shortcuts.Machine agents can use graph queries for navigation similar to jumping to type declarations or function callers.
Preprint
The paper illustrates notlob through Roman Numerals, Petri Net Chomper, and Pleiades, showing how prose, code, tests, and name-graph structure support agent-assisted development. These examples also expose context-sharing limitations and demonstrate both useful agent feedback and unflagged assumption propagation.
- Roman Numerals: Roman Numerals demonstrates notlob headings, subheadings, examples, properties, and a name-graph linking modules, symbols, tests, definitions, uses, and tests.Nodes reference source-file locations and receive unique addresses for navigation among related code elements.
- Petri Net Chomper: Petri Net Chomper is a web game using Petri nets as maps, developed as an extended TypeScript example and teaching tool.Its ten source files cover Petri-net structures, gameplay, rendering, and I/O, while overview.lob co-locates a 73-line prose overview with the project.
- Petri Net Chomper: Chomper prompted improved integration with external build and execution points, while platform-agent optimisations undermined notlob’s context-sharing advantages.The latter observation is presented as a tooling issue rather than a fundamental architectural limitation of coding agents.
- Pleiades: Pleiades began from a human-authored prose specification, function-signature sketch, and failing notlob tests before an agent implemented most code interactively.Human editing helped maintain code and prose organisation.
- Pleiades: The Pleiades agent closely followed the design sketch, detected a type-signature inconsistency, added BCE-date handling, and used unfamiliar astronomical libraries idiomatically.The added historical-date module addressed the lack of BCE support in Python standard libraries and included test coverage.
- Pleiades: Agent review produced increasingly fine-grained criticism and a horizon-calculation change that affected the calculated dates and may improve the paper’s result.The agent also propagated spelling mistakes and under-examined geographical assumptions from the original specification without flagging them.
Preprint
The paper reports exploratory implementation and critic-session results, including bug discovery and recurring difficulties preserving declarative artifacts alongside executable code.
- Experiments: An experimental Rust binding port exposed instability in a clamping calculation that other testing had not caught.
- Design guidance: The design guidance recommends prose that adds motivation, constraints, complexity, or conceptual dependencies rather than redescribing code.
- Experiments: Two Claude Code critic sessions reviewed notlob projects and the tool independently of particular codebases.
- Critic sessions: The critic sessions found errors of function and drift between prose and code, and suggested promoting implicit prose principles into checks.
- Observed pattern: Across four artifact types, agents repeatedly neglected declarative scaffolding, including grammars, USES nodes, property tests, and schemas.
- Observed pattern: Human inspection and intervention were necessary to retain these artifacts as load-bearing consistency points.
5 Related Work
Related work spans languages and tools that structure prompts, documentation, tests, and executable artifacts; notlob combines these concerns around durable code-and-prose organization.
- LLM-aware languages: LMQL, DSPy, and related tools structure LLM queries or data pipelines using program syntax and natural-language interfaces.
- LLM-aware languages: Unlike prompt-structuring tools focused on program syntax, notlob emphasizes durable code and documentation artifacts for humans and coding agents.
- Literate programming: Notlob is language-independent across bindings, with a literate layer orthogonal to the code language, and is positioned nearer noweb than WEB.
- Literate programming: Under the paper’s taxonomy, notlob is an Interoperable Literate Programming project because source can be modularly organized and made into reusable executable artifacts.
- Literate programming: Goldfish Scheme experiments report that smaller models using literate programming can outperform larger models using conventional programming languages alone.
- BDD and DSLs: Notlob differs from BDD through the absence of a translation layer, allowing natural language, asserted facts, and executable quality mechanisms to reinforce one another.
- BDD and DSLs: The interleaved prose and executable fragments make notlob a kind of Embedded DSL, although it is not primarily a DSL toolkit.
Preprint
The paper contrasts specification-first workflows with notlob’s co-located, iterative workbench, while framing natural-language programming as requiring explicit structure and consistency.
- Positioning: The paper describes notlob as evolving from literate programming and Behaviour Driven Development.
- Positioning: Specification Driven Development treats structured natural-language specifications as upstream artifacts from which LLMs generate code.
- Positioning: Criticism of SDD identifies LLM non-determinism affecting generated-code quality and difficulty articulating intent at useful abstraction levels.
- Notlob’s contrast: Notlob co-locates specifications, code, and tests at one workbench for frequent human-agent iteration rather than placing specifications upstream of code.
- Natural-language programming: The paper also presents a concern that implicit AI prompt languages lack specification, syntax, semantics, and consistency across models and releases.
6 Conclusion and Future Work
The conclusion presents notlob as a language and toolset for co-locating prose, code, and checks through a parse-tree-based name graph, with implementation and future expansion described.
- Conclusion: Notlob addresses natural-language-literate coding-agent problems using literate-programming techniques.
- Conclusion: Its design provides conceptual co-location within files and a parse-tree-based name graph for more effective context-window and external-memory use.
- Implementation: The language and toolset were implemented in Python with several small- to medium-sized example programs.
- Future work: Future work includes bindings for multiple execution languages, experiments with zero- or one-shot prompts, notlob-aware agents, and larger systems.