Source-linked AI summary
PDL: A Declarative Prompt Programming Language
Mandana Vaziri, Louis Mandel, Claudio Spiess, Martin Hirzel
TL;DR
LLM applications can be brittle, while existing prompting frameworks may complicate use or reduce developer control. PDL offers a typed, data-oriented YAML language for composing prompts and program behavior; using generated PDL helped identify inconsistent GSMHard ground truth in 10% of the dataset.
Problem
LLM applications can be brittle, and existing prompting frameworks may require complex features or imperative programming or remove developer control over exact prompts.
Source: §1 · p. 1Method
PDL is an orthogonal, typed, YAML-based data-oriented language whose blocks append data to prompt context and support templating, control flow, and LLM calls.
Source: §1 · pp. 1–2Results
10% of the GSMHard dataset had ground truth inconsistent with its question, identified through LLM screening followed by manual filtering.
Source: §5.3 · p. 9Takeaways & Limitations
Human-readable generated PDL code helped researchers inspect apparent mismatches and identify incorrect ground truth in GSMHard.
Source: §5.3 · p. 9
Abstract
from arXiv · showhide
Large language models (LLMs) have taken the world by storm by making many previously difficult uses of AI feasible. LLMs are controlled via highly expressive textual prompts and return textual answers. Unfortunately, this unstructured text as input and output makes LLM-based applications brittle. This motivates the rise of prompting frameworks, which mediate between LLMs and the external world. However, existing prompting frameworks either have a high learning curve or take away control over the exact prompts from the developer. To overcome this dilemma, this paper introduces the Prompt Declaration Language (PDL). PDL is a simple declarative data-oriented language that puts prompts at the forefront, based on YAML. PDL works well with many LLM platforms and LLMs. It supports writing interactive applications that call LLMs and tools, and makes it easy to implement common use-cases such as chatbots, RAG, or agents. We hope PDL will make prompt programming simpler, less brittle, and more enjoyable.
1 Introduction
LLMs enable many useful tasks, but their brittleness and the tradeoffs in existing prompting frameworks motivate a simpler approach that preserves developer control. PDL is a typed, orthogonal, YAML-based language that keeps prompts central while supporting interactive applications and common prompting patterns.
1 Introduction
- LLMs can perform many useful tasks, yet sometimes hallucinate or fail to follow expected syntax and types.Prompting patterns include in-context learning, chaining, RAG, tool use, program-aided language models, and agents.Source: §1 · p. 1
- Existing prompting frameworks trade off bespoke convenience, imperative control, or prompt automation against simplicity and developer control.The paper frames the goal as making LLM programming less brittle while keeping it simple and leaving developers in control.Source: §1 · p. 1
- PDL applies orthogonal, typed, data-oriented language design to keep prompts at the forefront while balancing prompt control with declarative simplicity.Its design intentionally blurs the line between programs for chaining and tools and data for prompts.Source: §1 · p. 1
- PDL is an orthogonal, typed language based on YAML, with JSON-valued variables, optional JSON Schema types, and an interpreter that performs dynamic type checking.Representing programs as data supports transformations and can let LLMs generate PDL programs.Source: §1 · p. 2
- PDL blocks append data to prompt context, which becomes input to subsequent LLM calls, and the language supports looping, conditionals, functions, file includes, and Jinja2 templating.It supports local and hosted LLMs, and templates can cover entire programs as well as prompts.Source: §1 · p. 2
- The paper presents an overview, language tour, tooling, case studies, related work, and conclusion, and describes PDL as open-source and available for use.The introduction characterizes PDL as a simple yet powerful language for LLM prompt programming.Source: §1 · p. 2
2 Overview
The chatbot example illustrates PDL as a sequence of blocks that contribute data to background context. Its loop reads user input, calls an LLM with accumulated context, and exits when the user enters “quit.”
2 Overview
- A PDL program executes blocks that generate data for background context, using model calls, input, JSON data, code, and control blocks to express pipelines.The described control blocks include if-then-else, for, and repeat.Source: §2 · p. 2
- In the chatbot, a read block obtains the user's query, and a contribute clause adds that response to context without printing it as the block's result.Figure 1(a) shows the code and Figure 1(b) its execution trace.Source: §2 · p. 2, Figure 1
- The model block uses accumulated context as its prompt, while the text block concatenates nested-block results and the stop parameter halts generation after two consecutive newlines.On the first loop iteration, the context contains the query prompt and the user's question.Source: §2 · p. 2
- The until condition uses a Jinja2 expression to stop the loop when question equals “quit”; PDL uses ${...} syntax because {{...}} interacts poorly with YAML.On the second iteration, the model call sees the prior output, and the user's subsequent “quit” input ends the loop.Source: §2 · p. 3
3 Language
PDL embeds a declarative language in YAML, organizing programs as blocks that build prompt context. Its block kinds and reusable keywords support control flow, data construction, typing, templating, and modularity.
3 Language
- Every PDL program is a valid YAML document following the PDL schema, and a program consists of one block or a list of blocks.Blocks may be expressions or structured blocks.Source: §3 · p. 3
- PDL offers 15 block-body kinds, including model calls, reads, data construction, control flow, functions, and code execution.Model blocks call an LLM; reads take input from a file or stdin.Source: §3 · pp. 3–4
- PDL supports modularity through include: blocks and functions, and its code: block executes code, with Python supported at the time of writing.The interpreter provides sandboxing to reduce risks from executing arbitrary code.Source: §3 · pp. 5–6
- Control constructs include if:, for:, and repeat:; lists nested within them implicitly return the last block's result unless wrapped or joined differently.A text: block or join: can combine loop results instead.Source: §3 · p. 5
- Optional keywords compose across block kinds to assign variables, decorate results with roles, control result and context contributions, and parse or type-check structured outputs.The spec: keyword uses a subset of JSON Schema; parsers include JSON, YAML, regex, and JSONL.Source: §3 · p. 5
- PDL uses Jinja2 expressions to template prompts and entire chains of calls, while its own if: and for: constructs replace Jinja2 statements.Expressions can be basic values, Jinja2 expressions, or strings containing Jinja expressions.Source: §3 · p. 6
4 Tooling
PDL tooling supports writing, running, and inspecting programs through an interpreter, editor and notebook integrations, a live execution visualizer, and a Python SDK.
4 Tooling
- The interpreter runs PDL from a command line, can stream LLM output incrementally, and supports container sandboxing.Sandboxing is recommended when executing LLM-generated actions or code.Source: §4 · p. 6
- VSCode support provides syntax highlighting, autocomplete, keyword tooltips, and error checking, partly driven by the JSON schema defining valid PDL.Source: §4 · p. 6
- The Jupyter %%pdl cell magic enables PDL code cells to share variables and background context across cells, with an option to reset context.Source: §4 · p. 6
- The live document visualizer displays nested execution-trace boxes and lets users select a box to inspect the PDL code that produced it.Source: §4 · p. 6
- The Python SDK lets larger Python applications call PDL, while a recommended practice is to pass data between PDL and Python as JSON objects.Optional type checking uses PDL's spec: alongside TypedDict or Pydantic on the Python side.Source: §4 · p. 6
5 Case Studies
The case studies apply PDL to retrieval-augmented generation, a ReAct agent, and generating executable PDL with an LLM. They also report a code-editing agent result and identify inconsistencies in GSMHard.
5.1 Retrieval-Augmented Generation
- The RAG example retrieves five similar MBPP training samples, adds them with an instruction and test query to the context, then calls a Granite 3 model to generate code.The authors also report using PDL with Codellm-Devkit to retrieve context for coding tasks.Source: §5.1 · pp. 7–8
5.2 ReAct Agent
- The ReAct example represents the think-act-observe loop with PDL variable definitions for thought, action, and observation.The thought is model-generated natural language; the figure is identified as a ReAct agent in PDL.Source: §5.2 · p. 8, Figure 4 · p. 8
- 23.7% of SWE-bench Lite instances were resolved by a submission using a PDL code-editing agent, the first to do so with only open-source models and in the same ball-park as frontier models.The authors report this as higher than any previous results with open-source models.Source: §5.2 · p. 9
5.3 Generating PDL from PDL with LLMs
- LLM-generated PDL can make plans executable as compositions of model and code calls, rather than plans expressed only as text, JSON, or Python code.The section explores this meta-generation approach on GSMHard.Source: §5.3 · p. 9
- For GSMHard, the authors adapt PAL by asking an LLM to generate PDL: textual chain-of-thought uses PDL text blocks, while arithmetic uses PDL code blocks.The generated program is extracted and executed in Python, with few-shot samples and a question as input.Source: §5.3 · p. 9
- 10% of GSMHard data points had ground truths inconsistent with their questions, identified through LLM screening followed by manual filtering.The authors say readable generated PDL made it easier to inspect mismatches and remove false positives.Source: §5.3 · p. 9
6 Related Work
Related work spans prompting frameworks that trade simplicity, control, and automation, as well as frameworks focused on agents. PDL is positioned among declarative and lower-level approaches without being limited to a single use case.
6 Related Work
- LangChain and MiniChain aim to address framework complexity, but both are Python frameworks; PDL shares MiniChain’s motivation and uses YAML rather than Python.The paper describes LangChain as powerful but complex and MiniChain as offering fewer, composable features.Source: §6 · p. 10
- Guidance and LMQL provide more structure than LangChain, while PDL takes inspiration from LMQL but relies less on imperative Python code.Crouse et al. specify agentic-loop internals with finite state machines but do not introduce a full-fledged prompting language.Source: §6 · p. 11
- DSPy and Vieira automate prompt derivation but take away control over exact prompts; Lale adjusts automation versus control, and SGLang targets computational performance.Future work will explore whether PDL’s declarative nature can enable similar computational-performance optimizations.Source: §6 · p. 11
- Agent-focused frameworks such as AutoGen prioritize agent support, whereas PDL aims for a more balanced stance in which agents are one of several prompting techniques.The cited survey identifies increasingly steep learning curves as a major pitfall of prompting frameworks.Source: §6 · pp. 9, 11
7 Conclusion
PDL is a declarative, data-oriented language built from YAML blocks that contribute data to prompt context. The conclusion presents it as ready to use and open-source, while describing automatic optimizations as future work.
7 Conclusion
- PDL programs comprise YAML blocks that are either literal data or produce data, with each block contributing to the background context used by later LLM calls.The conclusion describes this as PDL’s declarative, data-oriented model.Source: §7 · p. 11
- PDL’s declarative nature may support future automatic optimizations for speed, accuracy, and security.The paper says these optimizations are forthcoming future work.Source: §7 · p. 11
- The authors describe PDL as ready to use and open-source.The conclusion provides the project’s GitHub repository.Source: §7 · p. 11