"We Need Structured Output": Towards User-centered Constraints on Large Language Model Output

• Current LLMs cannot guarantee compliance with user-defined constraints, while existing controllability methods tend to address only a narrow range of constraints.The paper identifies a gap in understanding real-world usage scenarios and rationales.Source: §1 · p. 1
• The authors surveyed experienced industry professionals to investigate real-world use cases, benefits, and preferred ways to specify output constraints.The research questions cover what constraints users need, their impacts, and how users want to articulate them.Source: §1 · pp. 1–2
• The survey identifies low-level constraints on structure and length alongside high-level semantic and stylistic guidelines that should prevent hallucination.Respondents also reported benefits for development, integration, product requirements, trust, and user experience, and did not always prefer natural-language prompting for defining constraints.Source: §1 · p. 2
• The paper presents ConstraintMaker, an early tool for experimenting with, testing, and applying constraints on LLM output formats.Its contributions also include a taxonomy based on 134 use cases, an overview of benefits, and an exploration of GUI versus natural-language preferences.Source: §1 · p. 2

• The online survey asked respondents to describe three use cases, including the desired constraint, required precision, and workflow importance, and to compare GUI and natural-language control.It was distributed for two weeks in Fall 2023 to users of an internal prompt-prototyping platform at a large technology company.Source: §2 · p. 2
• 51 respondents contributed 134 unique use cases, which the researchers analyzed inductively to develop themes for the research questions.All respondents had prompt-design experience; most reported extensive experience.Source: §2 · p. 2
• The single-corporation industry sample may not represent the wider industry or casual users who use LLMs conversationally.The authors also note that open-ended questions might have reduced the response rate, while novel findings appeared to saturate toward the survey’s end.Source: §2 · p. 2
• Table 1 summarizes six categories of constraint use cases from respondents’ submissions, with some cases assigned to multiple categories.Its final column indicates whether the majority expected exact or approximate adherence.Source: Table 1 · p. 3

• Low-level constraints specify structure, multiple-choice output, or length, while high-level constraints govern semantic or stylistic guidelines and prevent hallucination.Examples include JSON or Markdown formats, including or avoiding terms or actions, and adhering to a tone.Source: §3 · p. 3
• Respondents wanted control over JSON schemas, such as required keys and answer counts, and formal guarantees even when few-shot examples worked fairly well.They viewed generating valid JSON alone as insufficient.Source: §3 · pp. 3–4
• For classification and labeling, respondents wanted only the result, because extra conversational explanation could confuse downstream parsing.The authors relate this need for succinct outputs to specialized downstream uses that differ from conversational response expectations.Source: §3 · p. 4
• High-level constraints ask outputs to stay within the input’s subject matter and avoid improvisation, including changes beyond the requested edits.One reported example expected code changes to add debug print statements only; another described unwarranted syntax changes.Source: §3 · p. 4

4.1 Increasing Prompt-based Development Efficiency

• Deterministic output formatting could reduce trial-and-error prompt testing, repeated requests, inference costs, and latency, with one respondent estimating savings of dozens of work hours weekly.Respondents also described post-generation matching, filtering, and verification code that constraints could reduce.Source: §4.1 · p. 4

4.2 Integrating with Downstream Processes and Workflows

• Constraints can help ensure outputs are compatible with downstream modules, executable code, and synthetic datasets that meet their intended specifications.Respondents also wanted constraints to carry across model migrations to avoid additional prompt-engineering.Source: §4.2 · p. 5

4.3 Satisfying UI and Product Requirements

• Length and format constraints can help generated content fit UI bounds and platform character limits, while reducing variability that may clutter the interface.Respondents noted that content exceeding UI bounds may be discarded.Source: §4.3 · p. 5

4.4 Improving User Experience, Trust, and Adoption

• Respondents linked constraint mechanisms to mitigating hallucinations, addressing safety and privacy concerns such as PII, and fostering trust and user adoption.They noted that users may not invest more time in tools they consider inaccurate.Source: §4.4 · p. 5

5 HOW TO ARTICULATE OUTPUT CONSTRAINTS TO LLMS

• Respondents preferred GUI for low-level constraints and natural language for high-level constraints.The GUI was favored for objective, quantifiable requirements; natural language was favored for complex, vague, or open-ended ones.Source: §5 · p. 5, §5.1 · p. 5, §5.2 · p. 6

5.1 The case for GUI: A Quick, Reliable, and Flexible Way of Prototyping Constraints

• GUI controls were valued for making hard requirements more reliable, reducing ambiguity, and supporting rapid experimentation.Respondents also said visible choices could clarify model capabilities and limitations.Source: §5.1 · pp. 5–6

5.2 The Case for NL: More Intuitive and Expressive for Complex Constraints

• Natural language was preferred for extended context, numerous choices, and vague or nuanced constraints that are cumbersome to express through a GUI.Some respondents also wanted an API field, potentially using a formal notation, to express constraints and reference external resources.Source: §5.2 · p. 6

6 THE CONSTRAINTMAKER TOOL

• ConstraintMaker lets users combine and edit constraint primitives, while a finite-state decoding method uses regex to enforce the GUI-defined output format.Its initial primitives included JSON objects, multiple choice, lists, ordered lists, and free-form text.Source: §6 · p. 6

6.1 Iterative Design and User Feedback

• Five expert prompters tested the tool in informal sessions, and their feedback informed iterative design refinements.The sessions lasted around 30 minutes each, alongside author self-experimentation.Source: §6.1 · p. 6

6.1.1 ConstraintMaker enables an intuitive separation of concerns.

• Participants found separating the task instruction from the expected output format more intuitive and effective for steering consistent results.They also envisioned reusing constraints across prompts to reduce repeated constraint crafting and post-migration prompt engineering.Source: §6.1.1 · p. 6

6.1.2 Constraint-prototyping GUI needs to cater to both developers and non-developers.

• The interface hides regex for non-technical users but lets advanced users edit it directly for fine-grained adjustments.Participants found visible regex distracting, while developers often wanted to modify it after drafting a constraint in the GUI.Source: §6.1.2 · p. 7

6.1.3 “Inserting” words among constraints.

• Adding an Exact text primitive followed feedback that inserting labels between constrained sections yielded better results than using Some text and List alone.One example placed “Short story:” before the story and “Suggestions:” before the list.Source: §6.1.3 · p. 7

6.1.4 Automatically inferring constraints based on prompts.

• Participants proposed inferring constraints from user prompts to streamline setup, help novices learn available options, and encourage considering output format before deployment.They also compared the proposed feature to code auto-completion.Source: §6.1.4 · p. 7

7 CONCLUSION

• The paper introduces a user-centered taxonomy of real-world output-constraint scenarios, benefits, and preferred methods.It presents the taxonomy as a framework and a source of practical insights into user requirements and preferences.Source: §7 · p. 7
• The paper also presents ConstraintMaker, an early GUI-based tool for iteratively prototyping and testing output constraints.The conclusion points to more controllable, customizable, and user-friendly human–LLM interfaces as a future direction.Source: §7 · p. 7

A THE SURVEY INSTRUMENT

• The survey collected job roles, prompt-design experience, and intended prompt audiences, screening out casual LLM users.Only participants reporting experience designing at least three prompts or extensive prompt-engineering experience could continue.Source: A THE SURVEY INSTRUMENT · p. 9
• Participants reported three real-world use cases and described each desired constraint, an example, required precision, and workflow importance.The instrument also asked when respondents preferred GUI or natural language and invited alternative preferences.Source: A THE SURVEY INSTRUMENT · p. 9
• Preference questions covered length, format, allowed or excluded words, topics, libraries, and style, using a seven-point scale from natural language to GUI.Examples included exact word or paragraph limits, JSON, and persona imitation.Source: A THE SURVEY INSTRUMENT · p. 9