PaperlayerView exact revision on arXiv

Source-linked AI summary

Sharing State Between Prompts and Programs

Ellie Y. Cheng, Logan Weber, Tian Jin, Michael Carbin

arXiv:2512.14805v2cs.PLcs.AI

TL;DR

Prompts evaluated separately from programs require developers to transfer program state manually. This paper introduces shared program state and a schema for prompt–program interaction, then implements the abstraction in NIGHTJAR; it achieves comparable or higher pass rates while reducing code, with runtime overhead as a tradeoff.

  • Problem

    When prompts run in a separate environment from the program, developers must manually transfer data and program state across the boundary.

    Source: §1 · p. 1

  • Method

    Shared program state lets prompts read and write program variables, compute with program objects, and implement control flow; a natural function interface schema specifies prompt–program interaction.

    Source: §1 · p. 2

  • Results

    NIGHTJAR achieves comparable or higher average pass rates than manual implementations, while shared program state reduces average lines of code.

    Source: §6 · p. 9

  • Takeaways & Limitations

    Offloading interoperability work to an LLM agent supports live computations adapted to program inputs and intermediate values, which the paper says is not scalable to implement manually.

    Source: §8 · p. 10

    Shared program state blurs the isolation boundary by allowing prompts to read and write memory, and NIGHTJAR provides partial safety mechanisms.

    Source: §7 · p. 10

Abstract

from arXiv · show

The rise of large language models (LLMs) has introduced a new type of programming: natural language programming. Users write prompts, which are instructions in natural language, to direct LLMs to perform tasks such as natural language processing, code generation, reasoning, etc. An emerging area of research enables interoperability between prompts and programs. We present a novel programming abstraction, shared program state, that removes the manual work required to enable interoperability between prompts and program states. With shared program state, programmers can write prompts that directly access program variables, compute with program objects, and implement control flow in the program. We present a schema for specifying natural function interfaces that extend programming systems to support programs with prompts and leverage this schema to specify shared program state as a natural function interface. We implement shared program state in the Nightjar programming system. Nightjar enables programmers to write Python programs containing prompts that share the Python program state. We show that Nightjar programs achieve comparable or higher task accuracy than manually written implementations (+4-19%), while decreasing the lines of code by 39.6% on average. The tradeoff is that Nightjar may incur runtime overhead (0.4-4.3x manual implementations).

1 INTRODUCTION

Existing systems isolate prompt evaluation from program execution, leaving programmers to transfer program state manually; shared program state lets prompts access and operate on that state directly.

1 INTRODUCTION

  • Natural language programming uses prompts as instructions to direct LLMs, offering a higher level of abstraction with fewer operational details than traditional programs.The paper situates this practice amid growing LLM capabilities and prompting use across tasks.Source: §1 · p. 1
  • In isolated program state, prompts cannot see program state, so programmers manually transfer data and broader program state between prompt evaluation and program execution.Figure 1 contrasts this manual work with the NIGHTJAR shared-state implementation.Source: §1 · p. 1, Figure 1 · p. 2
  • Shared program state lets prompts read and write program variables, compute with program objects, and implement control flow in the program.The abstraction is intended to remove manual work enabling prompt–program interoperability.Source: §1 · pp. 1–2
  • A natural function interface specifies how prompts interact with host-program execution, using effects for requested actions and handlers for their implementations.The authors formalize and design a schema for these interfaces based on effect and handler constructs.Source: §1 · p. 2
  • NIGHTJAR programs achieve comparable or higher task accuracy than manually written implementations (+4-19%), reduce lines of code by 39.6% on average, and may incur 0.4-4.3x runtime overhead.The reported runtime is relative to manual implementations.Source: §1 · p. 3

2 RELATED WORK

Related work frames prompts as a form of programming and prompt–program interoperability as related to language interoperability. Existing approaches range from isolated prompt state to partially shared state.

Natural Language as Programming

  • Earlier natural-language programming work treats natural language as documentation or macros, while later prompting work frames prompting as programming; this paper unifies state between prompts and programs.The paper situates language interoperability as a longstanding effort to compute across programming languages using common data.Source: §2 · p. 3

Isolated Program State

  • Most prior interoperability systems isolate prompt evaluation from program execution, serializing data into prompts and optionally reifying returned strings; code-interpreter systems also isolate execution.The paper distinguishes its model as sharing the host program state with prompt evaluation.Source: §2 · p. 3

Programmer-Defined Tools

  • Some isolated-state systems use programmer-defined functions for tool calls, whereas shared program state does not require custom functions for prompts to operate on program state.The described systems include Beurer-Kellner et al. (2023), Chase (2022), Dong et al. (2025), and Firebase Team.Source: §2 · p. 4

Partially Shared Program State

  • Partially shared systems support limited access such as function arguments or serialization, but the surveyed methods do not enable variable writes or control flow; AWL shares webpages, not program state.Marvin and MTP do not manipulate mutable objects in-place, while AskIt and ANPL use LLM-generated functions.Source: §2 · p. 4

3 SHARED PROGRAM STATE BY EXAMPLE

NIGHTJAR makes prompts first-class Python code that can work directly with program variables, objects, and control flow. Its example shows this shared state in a citation-graph program and reports higher pass rate than a manual implementation, with runtime overhead.

3.2 SHARED PROGRAM STATE

  • Prompts use shared scopes, heap, and control state to read and write variables, mutate Python objects, and alter the surrounding program’s control flow.For example, a prompt can read the query, update the graph in place, and break out of the loop when the user exits.Source: §3 · p. 4, §3.2 · pp. 4–5

3.1 EXAMPLE PROGRAM

  • The example program represents a citation graph with paper identifiers as nodes and a dictionary of citing-paper sets as edges.Its loop reads queries, updates the graph, prints a response, or exits when the user is finished.Source: §3.1 · p. 4

3.2 SHARED PROGRAM STATE

  • Without shared program state, programmers must manually serialize and reify data structures, update heap objects, and invoke control-flow primitives.Shared program state abstracts over this intermediary connecting code.Source: §3.2 · p. 5

3.3 EXECUTING PROGRAMS WITH SHARED PROGRAM STATE

  • NIGHTJAR achieves higher pass rate than the manual implementation on the graph task, while incurring runtime overhead.The comparison uses a 25-node graph and six natural-language queries.Source: Table 1 · p. 5, §3.3 · p. 5

4 A NATURAL FUNCTION INTERFACE FOR SHARED PROGRAM STATE

The paper specifies shared program state through a natural function interface: a contract describing how prompts interact with a host program. Its schema uses effects and handlers to define those interactions, including control-flow changes that can cancel prompt evaluation.

4.1 NATURAL FUNCTION INTERFACES

  • A natural function interface specifies values crossing between prompt and host, prompt effects, and handlers that implement those requests.The interface generalizes tool use through effects and handlers.Source: §4.1 · p. 6
  • Unlike typical function calling, a handler can cancel prompt evaluation instead of resuming it, allowing the prompt and host program to coordinate control flow.The interface assumes support for serialization or reification, checkpointing, restoration, and cleanup when evaluation is cancelled.Source: §4.1 · p. 6

4.2 SHARED PROGRAM STATE AS A NATURAL FUNCTION INTERFACE

  • The shared-state interface defines Lookup and Assign for allowed variables, Ref, Deref, and Set for heap references, and Goto for allowed program labels.Goto jumps to a label without resuming prompt evaluation; the specification assumes a host language with variable bindings, mutable state, and control-flow constructs.Source: §4.2 · pp. 6–7

5 NIGHTJAR PROGRAMMING SYSTEM

NIGHTJAR implements shared program state for Python by translating prompts into effects handled against Python’s program state. It adds Python-specific support and runtime optimizations to the general interface.

5 NIGHTJAR PROGRAMMING SYSTEM

  • NIGHTJAR implements the shared-state interface for Python, using an LLM agent to translate prompts into effects.Source: §5 · p. 7

5.1 IMPLEMENTATION

  • NIGHTJAR uses variable-reference and assignment syntax, heap references for mutable objects, and transformed source labels for valid Python control-flow operations.Handlers restrict reads and writes to permitted variables, reject invalid references, and expose labels only when valid in the prompt’s evaluation context.Source: §5.1 · pp. 7–8

5.2 SYSTEM ENGINEERING

  • Python specialization adds host-language features and Python computation, while eager variable loading and response caching reduce agent effects or repeat API calls.Eager loading can add token cost by placing otherwise-unselected variable values and types in context; caching helps until the agent reads unseen inputs.Source: §5.2 · p. 8

6 EVALUATION

On SPSBench, NIGHTJAR’s average pass rate is comparable to or higher than manual implementations, while shared program state reduces implementation length. The evaluation also identifies runtime overhead and limitations of the unspecialized baseline.

Evaluation setup

  • SPSBench comprises 25 programs, and the comparison includes manual implementations, code-interpreter-enabled manual implementations, and unspecialized NIGHTJAR.The evaluation averages pass rates across five runs and sets limits of 300 tool calls or effects and 1,000 seconds.Source: §6 · pp. 8–9

RQ2: Results

  • NIGHTJAR achieves comparable or higher average pass rate than Manual Implementation and Manual Implementation (Code Interpreter), but has runtime overhead versus chat-completion-only Manual Implementation.Manual Implementation (Code Interpreter) also incurs runtime overhead.Source: Table 2 · p. 9, §6 · p. 9

RQ1: Results

  • 17.0 fewer lines of code on average: shared program state uses 25.1 lines versus 42.0 for Manual Implementation.The reported decreases range from 0 to 60 lines.Source: §6 · p. 9

RQ2: Results

  • NIGHTJAR’s unspecialized baseline has lower pass rate than the manual baselines because it lacks Python-specific features such as first-class functions and abstract classes.Additional engineering reduces runtime overhead by 2.0–2.1x compared with NIGHTJAR (Baseline).Source: §6 · p. 9

7 LIMITATIONS & FUTURE WORK

The paper identifies open boundaries in extending shared program state, including incomplete safety mechanisms and future opportunities in performance, security, and development tools.

Safety & Security

  • NIGHTJAR's partial safety mechanisms are incomplete even though shared program state lets prompts directly read and write memory objects.This blurs the boundary maintained by systems that isolate prompts from programs.Source: §7 · p. 10
  • Future work could develop richer safety mechanisms, including designs inspired by ownership abstractions that enable zero-copy communication on a shared heap.Heap isolation can protect against inconsistent states but requires marshaling and copying data, which introduces high overheads.Source: §7 · p. 10
  • The discussion highlights defenses against malicious inputs and unreliable LLM-computed data as relevant safety directions for shared program state.It points to preprocessing inputs and postprocessing generated data or code as inference-time defenses.Source: §7 · p. 10

Performance

  • Future work could optimize execution time and correctness through effect optimizations, parallel generation, caching, and prompt optimization.The paper frames NIGHTJAR as opening work on execution time and correctness for systems with shared program state.Source: §7 · p. 10

Program Development Tools

  • The paper calls for development tools that combine traditional program needs, such as finding bug-triggering inputs, with prompt robustness testing for a given LLM.Examples include debuggers, testing tools, and program analyses.Source: §7 · p. 10

8 CONCLUSION

Shared program state lets programmers focus on core program logic by offloading prompt–program interoperability work to an LLM. NIGHTJAR uses an agent to plan, implement, and execute natural-language instructions in response to live inputs and intermediate values.

8 CONCLUSION

  • NIGHTJAR's adaptive computations use an LLM agent to plan, implement, and execute instructions based on program inputs and intermediate values.The conclusion characterizes this as a promising class of systems that can remain robust under changing contexts without large-scale manual engineering.Source: §8 · p. 10

A.1 FULL MANUAL IMPLEMENTATION PROGRAM

The appendix presents a manual Python graph-program example and formal natural function interface fragments. Together, they illustrate graph serialization, prompt-driven operations, and interface variants for tool use and isolated state.

A.1 FULL MANUAL IMPLEMENTATION PROGRAM

  • The manual program represents a graph with node and edge collections, serializes it, and provides a reification path back to a Graph object.The serializer places nodes and edges in a dictionary; reification rebuilds a node set and edge dictionary.Source: §A.1 · p. 17
  • The prompt receives the query and serialized graph, then returns a response, a termination flag, or an updated graph through a structured output schema.The prompt instruction specifies that graph edges run from cited papers to papers that cite them.Source: §A.1 · p. 17

A.2 EXAMPLE PROGRAM INPUTS

  • Example inputs include citation-count and reachability questions, graph updates and removals, and an exit request.One query asks to update the graph so paper 5 cites paper 14.Source: §A.2 · p. 17

A.3 EXAMPLE TRACE

  • The example trace handles a graph update by adding paper 5 to the citing-paper set for paper 14, then exits when given an exit query.The update succeeds and the resulting edge set includes 5.Source: §A.3 · pp. 18–19

B.1 NATURAL FUNCTION INTERFACE

  • The basic natural function interface distinguishes host values from NFI values and specifies serialization, reification, and prompt evaluation.The formal fragment labels these operations α, γ, and nat, respectively.Source: §B.1 · p. 20

B.2 NATURAL FUNCTION INTERFACE FOR TOOL USE

  • The tool-use interface adds an available-tool set F, allowing a prompt to call a named tool with reified arguments and resume with its serialized result.The formal rule represents this as a call to f followed by resumption.Source: §B.2 · p. 20

B.3 NATURAL FUNCTION INTERFACE WITH ISOLATED PROGRAM STATE

  • The isolated-state interface uses constant prompt inputs and outputs, with no program-state effects and a no-op NFI handler.Its formal fragment is presented separately from the tool-use interface.Source: Figure 7 · p. 21, §B.3 · p. 21

C BENCHMARKS

This section identifies the programs covered by SPSBench and describes how system prompts guide natural-language execution and generated Python implementations, including benchmark pass-rate figures.

C BENCHMARKS

  • Table 3 lists SPSBench programs and their descriptions.The supplied passage gives the table’s scope but no individual program names or benchmark results.Source: Table 3 · p. 22

D SYSTEM PROMPTS

  • Nightjar’s system prompts specify how natural-language instructions interact with Python variables and how execution should proceed.The shared-Python prompt defines <variable> for use inside a natural block and <:variable> for variables accessible afterward; it also prescribes discovery, planning, execution, reflection, and completion steps.Source: D SYSTEM PROMPTS · p. 25
  • For BaseModel objects, the shared-Python prompt directs the assistant to inspect the JSON schema, construct JSON, and validate it with model_validate_json.It also instructs the assistant to address errors rather than repeat the same tool call.Source: D SYSTEM PROMPTS · p. 26
  • The code-generation prompt instructs the model to replace a natural-language comment with valid Python, honoring variable-reference syntax and using `nj_llm` for LLM calls with optional JSON-schema output.The prompt also says to implement only the specified comment and not leave code unimplemented.Source: D SYSTEM PROMPTS · p. 29

E.1 AVERAGE PASS RATE BY BENCHMARK

  • Figures 8 and 9 show average pass rate by SPSBench benchmark with standard deviations for each method, using Claude-Sonnet-4-20250514 and GPT-4.1-2025-04-14, respectively.The supplied captions do not provide the plotted values or establish a method comparison.Source: Figure 8 · p. 31, Figure 9 · p. 31

E.2 SHARED PROGRAM STATE IMPLEMENTATION ABLATIONS

The ablations show that Python-specialized NIGHTJAR improves pass rate, while runtime remains an important tradeoff. Results also vary with the base model, benchmark features, and optimization choices.

E.2 SHARED PROGRAM STATE IMPLEMENTATION ABLATIONS

  • LLM code generation is faster than NIGHTJAR’s interpreter but has poor accuracy because it cannot adapt computation to program inputs or intermediate values.The approach can inspect the program only statically.Source: §E.2 · p. 32
  • NIGHTJAR (Baseline) lacks language-specific features such as first-class functions and abstract methods, and its pass rate is near or slightly below manual implementations on the feature-free subset.It is within standard deviation of manual implementation for GPT-4.1 and slightly below it for Sonnet 4.Source: §E.2 · p. 32
  • On the feature-free subset, Sonnet 4 reaches parity with Shared Eval and Isolated Eval, while GPT-4.1 fares worse with Shared Eval but better with Isolated Eval.The authors interpret this disparity as evidence that Sonnet 4 uses Python expressions with shared state more effectively than GPT-4.1.Source: §E.2 · pp. 32–33
  • NIGHTJAR (Shared Python) outperforms the programmer implementation in pass rate on complete SPSBench for both Sonnet 4 and GPT-4.1, while retaining runtime overhead.Its use of language-specific features and freeform Python statements supports the pass-rate improvement.Source: §E.2 · p. 33
  • Additional optimizations reduce NIGHTJAR runtime, but caching saves only the effects emitted before program inputs are read.SPSBench test cases use different program inputs, limiting the caching benefit.Source: §E.2 · p. 33

E.3 LOCALLY HOSTED LLM

  • With GPT-OSS 20B, NIGHTJAR has lower pass rate than manual implementation but outperforms static LLM code generation.The authors attribute the lower pass rate to NIGHTJAR’s reliance on the base model’s agentic capabilities.Source: §E.3 · p. 35, Table 6 · p. 35

E.4 TEMPERATURE 0

  • With GPT-4.1 at temperature 0, NIGHTJAR reaches parity with manual implementations, but executions remain non-deterministic with a closed model.The authors say a locally hosted open model with non-determinism controls, or caching responses to repeated effect traces, would enable deterministic executions.Source: §E.4 · p. 36, Table 7 · p. 36

E.5 LARGE DATA STRUCTURE

Pass-by-reference lets NIGHTJAR query parts of large objects instead of encoding them in full. In the graph case study, it supports queries beyond the pass-by-copy context limit, while runtime tracks the number of emitted effects.

E.5 LARGE DATA STRUCTURE

  • Pass-by-copy encodes objects as strings whose length grows with object size, whereas pass-by-reference sends a constant-size reference and inspects only relevant object parts.The latter is more efficient when a prompt needs only part of a large object.Source: §E.5 · p. 36
  • Pass-by-copy fails to answer queries after the encoded graph exceeds the context window beyond 1,000 nodes; NIGHTJAR’s pass-by-reference implementations answer large-graph queries with higher pass rates.NIGHTJAR’s Python specialization further improves pass rate over the baseline, while pass-by-copy generally has lower accuracy.Source: §E.5 · pp. 36–37, Figure 10 · p. 37

E.6 NUMBER OF EFFECTS TO RUNTIME

  • Runtime is proportional to the number of emitted effects, with different LLMs showing different runtime-to-effect ratios.Each emitted effect corresponds to one inference pass through the base LLM.Source: §E.6 · p. 37, Figure 11 · p. 38

E.7 ADDITIONAL RESULTS ON LLM MATH REASONING BENCHMARK

On GSM8K, NIGHTJAR’s accuracy is higher than or equal to the manual implementations, while its runtime overhead is similar to using the OpenAI Code Interpreter. This benchmark does not meaningfully require formal program state, and NIGHTJAR’s overhead can be reduced through system optimization.

E.7 ADDITIONAL RESULTS ON LLM MATH REASONING BENCHMARK

  • The evaluation uses GPT-4.1 on all 1,319 problems in GSM8K, a benchmark that does not meaningfully use formal program state.The authors frame this setting as equivalent to an LLM agent using code-evaluation tools with isolated program states.Source: §E.7 · p. 38
  • NIGHTJAR incurs slightly higher runtime overhead than the basic manual implementation, but overhead is similar to OpenAI Code Interpreter tool use.The authors report that an optimized NIGHTJAR version can further reduce runtime overhead.Source: §E.7 · p. 38
  • NIGHTJAR achieves higher or equal accuracy to the manual implementations on GSM8K.Table 8 reports pass rate, average runtime, and average tool calls or effects over 3 runs.Source: Table 8 · p. 38, §E.7 · p. 38

F FAILURE ANALYSIS

Most NIGHTJAR execution-program runs pass all test cases, and reasoning or math errors account for most failures. Error patterns differ by model, while one illustrated graph failure reverses the requested citation direction.

F FAILURE ANALYSIS

  • Most NIGHTJAR execution-program runs pass all test cases; reasoning or math errors account for most failures among the rest.The analysis labels 125 runs across 25 programs and 5 runs per program; only a few failures involve giving up, hallucinated data, or incorrect state operations.Source: Table 9 · p. 39, F FAILURE ANALYSIS · p. 39
  • GPT-4.1 encountered more effect errors than Sonnet 4, often confusing effects or hallucinating variable names; Sonnet 4 errors were usually syntax or data-type assumptions.GPT-4.1 generally corrected errors, though it sometimes ignored them or used a workaround.Source: F FAILURE ANALYSIS · p. 39, Table 10 · p. 39
  • The error-recovery analysis summarizes how the LLM responds when effect handling produces an error.Table 11 reports the methods of recovery.Source: Table 11 · p. 39, F FAILURE ANALYSIS · p. 39
  • In the graph example, the requested update that paper 5 cites paper 14 was reversed so paper 14 cited paper 5.The example is presented as an execution that did not pass its test case.Source: F FAILURE ANALYSIS · pp. 39–40
Loading 2512.14805v2…