PaperlayerView exact revision on arXiv

Source-linked AI summary

INDICT: Code Generation with Internal Dialogues of Critiques for Both Security and Helpfulness

Hung Le, Yingbo Zhou, Caiming Xiong, Silvio Savarese, Doyen Sahoo

arXiv:2407.02518v2cs.SEcs.AIcs.CLcs.CRcs.MAcs.PL

TL;DR

Code-generation LLMs can produce harmful or insecure code when instructions are malicious or ambiguous, despite the dual-use nature of useful code. INDICT uses interacting safety- and helpfulness-driven critics, external knowledge, and feedback before and after execution; evaluations report improved safety and helpfulness across diverse tasks.

  • Problem

    Code-generation LLMs may produce harmful or insecure code under malicious or ambiguous instructions, while useful programs can also be repurposed for harm.

    Source: §1 · p. 1

  • Method

    INDICT pairs autonomous safety- and helpfulness-driven critics, grounds their critiques with external tools, and applies feedback during generation and after execution.

    Source: §1 · p. 2, §3.4 · p. 5

  • Results

    INDICT improved safety and helpfulness across 8 diverse tasks, 8 programming languages, 5 benchmarks, and models ranging from 7B to 70B parameters.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results highlight the potential of autonomous, tool-enhanced multi-critic systems for code-related tasks and beyond.

    Source: §5 · p. 10

    INDICT relies on LLMs following role instructions, requiring carefully written prompts to specify critic requirements and assess safety and helpfulness criteria.

    Source: A Limitations · p. 17

Abstract

from arXiv · show

Large language models (LLMs) for code are typically trained to align with natural language instructions to closely follow their intentions and requirements. However, in many practical scenarios, it becomes increasingly challenging for these models to navigate the intricate boundary between helpfulness and safety, especially against highly complex yet potentially malicious instructions. In this work, we introduce INDICT: a new framework that empowers LLMs with Internal Dialogues of Critiques for both safety and helpfulness guidance. The internal dialogue is a dual cooperative system between a safety-driven critic and a helpfulness-driven critic. Each critic provides analysis against the given task and corresponding generated response, equipped with external knowledge queried through relevant code snippets and tools like web search and code interpreter. We engage the dual critic system in both code generation stage as well as code execution stage, providing preemptive and post-hoc guidance respectively to LLMs. We evaluated INDICT on 8 diverse tasks across 8 programming languages from 5 benchmarks, using LLMs from 7B to 70B parameters. We observed that our approach can provide an advanced level of critiques of both safety and helpfulness analysis, significantly improving the quality of output codes ($+10\%$ absolute improvements in all models).

1 Introduction

Code-generation models can produce harmful or insecure code when instructions are malicious or ambiguous, while useful code can also be misused. INDICT combines safety- and helpfulness-driven critics, external knowledge, and feedback at generation and execution stages; evaluations report gains on both dimensions.

1 Introduction

  • Safety- and helpfulness-driven critics interact autonomously to collaboratively optimize both security and correctness of generated responses.The critics have orthogonal goals rather than operating independently.Source: §1 · p. 2
  • Critics use relevant code snippets and natural-language queries with external tools to ground their critiques for the code generator.The paper names web search and code interpreters as examples of tools.Source: §1 · p. 2
  • Feedback is applied both during initial code generation and after code is observed in an execution environment.The authors argue post-hoc feedback alone is not proactive enough for security-sensitive tasks.Source: §1 · p. 2
  • Across 8 tasks, 8 programming languages, and 5 benchmarks, INDICT improved safety and helpfulness across models from 7B to 70B parameters.The paper reports new state-of-the-art performance on security measures while maintaining or improving code helpfulness.Source: §1 · p. 2

2 Related Work

Prior work studies code-generation security risks and methods for improving LLM safety or helpfulness. INDICT is presented as combining code-execution feedback, external knowledge, dual quality goals, and autonomous multi-agent collaboration.

2 Related Work

  • Table 1 compares INDICT with self-refinement or self-critique, multi-agent, and finetuning approaches across three stated contribution directions.These directions are execution-based feedback enhanced with external knowledge, both safety and helpfulness, and interactive supervision-free collaboration.Source: Table 1 · p. 3
  • Studies report vulnerabilities in generated code and demonstrate that complex prompts can jailbreak advanced LLMs into producing malicious code.Cybersecurity benchmarks evaluate common coding scenarios and security-aware tasks.Source: §2 · p. 3
  • Prior safety and helpfulness work includes preference-data finetuning, code-specific finetuning strategies, and human-provided natural-language feedback.INDICT is framed as a more efficient method for generating code targeting both safety and helpfulness.Source: §2 · p. 3

3 INDICT Framework

INDICT treats code generation as an actor producing a response to an instruction and optional code context, then uses specialized critics, dialogue, tools, and execution feedback to revise outputs.

3.1 Problem Definition

  • Code generation is modeled as an LLM generating a token sequence from a natural-language instruction and optional related code context.Responses may contain natural-language explanations or refusals as well as code programs.Source: §3.1 · p. 4

3.2 Safety-driven and Helpfulness-driven Critics

  • Separate safety- and helpfulness-driven critics assess generated code against security risks and alignment with task intentions, and the actor revises its solution based on their critiques.The critics are initialized as LLMs with system prompts establishing their distinct roles.Source: §3.2 · p. 4

3.3 Autonomous Collaboration between Critics

  • The two critics interact autonomously, with dialogue summaries reused across turns to support their collaborative critique process.Summarizing interactions can limit computation overhead and reduce computation cost on the actor LLM.Source: §3.3 · p. 5

3.4 Knowledge-grounded Critics with External Tools

  • Critics use generated textual keywords, optionally with code snippets, to query external tools and obtain observations for more grounded critiques.The framework distinguishes code search and code review actions.Source: §3.4 · p. 5, Figure 3 · p. 5

3.5 Preemptive and Post-hoc Critic Feedback

  • INDICT applies critic feedback after initial generation and again after execution, conditioning post-hoc feedback on results such as errors or unit-test outcomes.Safety and helpfulness critics retain a persistent dialogue context across preemptive and post-hoc iterations.Source: §3.5 · p. 6

4 Experiments

INDICT was evaluated across insecure-code generation, security attacks, open-ended generation, baseline comparisons, and ablations. Across these settings, the reported results show gains in safety and helpfulness, including stronger performance than the tested baselines.

4 Experiments

  • Experiments used CommandR and Llama/Codellama models from 7B to 70B parameters, with actor and critics initialized from the same checkpoint and up to five INDICT rounds per sample.The insecure-coding evaluations covered CyberSecEval-1 Autocomplete and Instruct tasks and CVS; safety used rule-based detection, while helpfulness used GPT-based comparisons against prior models or ground truth.Source: §4 · p. 6, §4.1 · pp. 6–7, Figure 4 · p. 7

4.1 Insecure coding practice tasks

  • INDICT with CommandR and Llama3 achieved more than 80% safe outputs on CyberSecEval-1 and more than 90% on CVS, with helpfulness winning rates up to 70%.The results were consistent across programming languages, with Javascript on CyberSecEval and C++ on CVS among the few helpfulness exceptions.Source: §4.1 · p. 7

4.3 Open-ended generation tasks

  • On HarmBench, CommandR+INDICT improved average safety from 23% to 51%; on CAMEL, at least 70% of outputs improved in both safety and helpfulness over direct generation.CAMEL results showed minor performance drops after four INDICT rounds.Source: Table 2 · p. 8, Figure 6 · p. 8, §4.3 · p. 9

4.2 Security attack tasks

  • INDICT achieved 76% safety on Cyber Attack and more than 90% on Interpreter Abuse and Prompt Injection using CodeLlama and Llama3 models.The evaluation covered models from 8B to 70B parameters; CommandR also gained security despite being a weaker model.Source: §4.2 · p. 8

4.4 Comparison to baselines

  • On CyberSecEval-1 validation, INDICT achieved more than 90% security and 81% helpfulness, outperforming seven representative baselines, including instruction-enhanced versions.The comparison used GPT4o-mini as the base model and allowed up to three revision rounds.Source: §4.4 · p. 9

4.5 Ablation analysis

  • Ablations found the best overall average safety and helpfulness when both critics were used in preemptive and post-hoc stages, with external tools improving results and single critics often optimizing only one quality.Across multiple rounds, INDICT improved consistently, while direct generation gains were marginal and no-tool performance curves converged faster.Source: Table 3 · p. 10, Table 4 · p. 10, §4.5 · p. 10

5 Conclusion

INDICT uses autonomous interaction between safety- and helpfulness-focused critics to improve code generation. Experiments across coding tasks and languages show benefits on code-related tasks and beyond.

5 Conclusion

  • INDICT uses two critic models, focused respectively on safety and helpfulness, to generate grounded critiques that collaboratively improve the actor LLM’s code outputs.The critics interact autonomously and can use external tools.Source: §5 · p. 10
  • Experiments span diverse downstream coding tasks, programming languages, and attack tactics.The conclusion also reports benefits on code-related tasks and beyond.Source: §5 · p. 10
  • The authors present an autonomous, tool-enhanced multi-critic system as a promising direction for improving code generation.Source: §5 · p. 10

A Limitations

INDICT depends on LLMs following role instructions and on difficult-to-specify safety and helpfulness criteria, so critics may miss relevant concerns. Its tool-assisted revisions are conditional, its operation costs more than direct generation, and the paper excludes some code-related safety concerns from evaluation.

A Limitations

  • Critics may fail to detect the right safety or helpfulness concerns because role-following depends on LLM capabilities and the criteria are difficult to define for code.The framework therefore requires carefully written role prompts and criteria.Source: A Limitations · p. 17
  • Relevant external knowledge may help a critic revise an initially incorrect critique, but that improvement is conditional on the retrieved knowledge being relevant.The revised critique is passed to the actor LLM.Source: A Limitations · p. 17
  • INDICT costs more to run than direct generation because it activates multiple LLMs and uses external tools, though the authors consider it more affordable than relevant finetuning methods.Those finetuning methods can require substantial computation and annotation effort.Source: A Limitations · p. 17

B Ethical Statement

  • Practical adaptations should control agent autonomy and check tool actions and outcomes for unethical consequences.The paper specifically suggests limiting web-search queries and sandboxing code execution.Source: B Ethical Statement · p. 17
  • The evaluation does not consider conventional safety concerns such as social bias or offensive content in code comments and variable names.The authors call for more study of these scenarios.Source: B Ethical Statement · p. 17

C.1 Societal Impacts

The authors describe INDICT as a potential mitigation layer for harmful or insecure code, including in high-risk domains, while also warning that autonomous critics could be misused. They recommend limiting agent autonomy and tool access and adding safeguards.

C.1 Societal Impacts

  • INDICT could serve as an extra mitigation layer in high-risk domains while preserving helpfulness, and the authors also report benefits on open-ended tasks beyond code.The examples of high-risk domains are military, medicine, and banking.Source: §C.1 · p. 17
  • Table 5 compares INDICT with self-refinement, multi-agent, and finetuning methods across execution feedback, tool use, multi-critic collaboration, and supervision requirements.Its caption describes INDICT as integrating execution feedback and external knowledge while targeting both helpfulness and safety.Source: Table 5 · p. 18
  • Reversing critic goals could steer an autonomous system toward generating more insecure or useless code.The paper warns that malicious users could use such a system to create more sophisticated attacks against LLMs.Source: §C.1 · p. 18

C.2 Safeguards

  • The authors recommend limiting critic autonomy and tool access, filtering web searches, and using stronger external analyzers or evaluators as safeguards.They note these tools were not used in the experiments for fair baseline comparisons.Source: §C.2 · p. 18

D Comparison to Related Work

INDICT differs from prior actor-critic approaches by targeting both safety and helpfulness, grounding feedback with external tools, and coordinating multiple critics. The paper also contrasts it with multi-agent methods that use a single criterion or simply add safety instructions.

D Comparison to Related Work

  • INDICT targets both helpfulness and safety, whereas most existing actor-critic approaches focus on a single criterion such as functional correctness.The authors report that simply adding safety instructions to these methods is suboptimal.Source: D Comparison to Related Work · p. 18
  • INDICT grounds critic feedback with external tools, while many existing methods rely only on code-test results as external feedback.Source: D Comparison to Related Work · p. 19
  • INDICT uses multi-critic collaboration to produce feedback for both safety and helpfulness without supervision.The comparison table identifies these features alongside execution-based feedback and tool enhancement.Source: D Comparison to Related Work · pp. 18–19
  • The paper reports that current multi-agent methods, even with added safety instructions, perform poorly relative to the desired outcome and remain far from optimal.It also notes that prior agent approaches focused on code analysis do not always improve generated code effectively.Source: D Comparison to Related Work · p. 19

E Details of Experimental Setups

The experimental setup spans three task families and five benchmarks, with fixed critic-dialogue budgets, staged tool access, and benchmark-specific sampling. The authors also describe evaluation procedures, model licensing, comparison constraints, and compute costs.

E Details of Experimental Setups

  • The experiments used five outer action rounds and one critic interaction per round, retaining interaction context across rounds.The authors selected this budget to balance outer and inner loops against compute cost.Source: E Details of Experimental Setups · p. 19
  • Critics used Wikipedia, code interpreters, web search, and GPT-3.5; preemptive feedback used CodeSearch, while post-hoc feedback could use code interpreters.Original task instructions were not sent directly to the OpenAI API; critic-generated text or code snippets served as queries.Source: E Details of Experimental Setups · p. 19
  • INDICT was evaluated on insecure coding practice, security attacks, and open-ended generation across five benchmarks.The benchmarks were CyberSecEval-1, CyberSecEval-2, CVS, CAMEL, and HarmBench.Source: E Details of Experimental Setups · p. 19, Table 6 · p. 20
  • CVS and CAMEL samples were randomly drawn to match original data distributions, while HarmBench gained five 320-prompt red-teaming splits.The HarmBench augmentations used ZS, PAP, JB, TAP, and PAIR methods.Source: E Details of Experimental Setups · p. 20
  • Evaluation anonymized model names and randomized response order, and code-generation scoring used extracted code snippets rather than accompanying text.The framework itself did not have access to evaluators such as static analyzers or AI classifiers.Source: E Details of Experimental Setups · p. 20
  • The study used open-source CommandR, Llama-2, CodeLlama, and Llama-3 models under their stated academic-use licenses.The authors report complying with Meta license conditions for Llama and CodeLlama.Source: E Details of Experimental Setups · pp. 20–21
  • The authors could not compare against several newer finetuning methods because of limited compute and unreleased checkpoints at submission.They said they would attempt those comparisons in future work.Source: E Details of Experimental Setups · p. 21
  • INDICT incurred about 3–4× the time cost of one LLM, and some Llama3-70B runs exceeded the available hardware memory.The reported setup used 2–8 Nvidia A100 40GB GPUs and up to 600GB of CPU memory.Source: E Details of Experimental Setups · p. 21

F INDICT vs. Finetuning methods

On a CodeLlama-7b-instruct validation split, the authors compared INDICT with CodeUltraFeedback finetuning. Adding INDICT to the best finetuned model improved both reported helpfulness and safety scores.

F INDICT vs. Finetuning methods

G Additional Ablation Analysis

The ablations compare critic-dialogue design choices, including summarization, retrieval and tool-based critiques, and separate versus shared critics. The reported variants show lower safety or helpfulness scores than the proposed framework.

G Additional Ablation Analysis

  • RAG and tool-based variants underperformed INDICT, with the authors reporting that their queries were often too vague to retrieve meaningful snippets.The tested RAG variant retrieved from the original task description; the tool-based variant used web search or Wikipedia.Source: G Additional Ablation Analysis · p. 22, Table 8 · p. 22
  • 87% safety and 76% helpfulness resulted when one critic handled both quality criteria.The authors propose that language-model training objectives may not optimize security and helpfulness equally; their design instead delegates these goals to separate critics that collaborate.Source: G Additional Ablation Analysis · p. 22, Table 8 · p. 22
  • 87% safety and 72% helpfulness followed removal of the critic summarizer and exposure of the full dialogue history to the actor.The authors suggest the longer context may make it harder for the actor to use all critic feedback and note its added computation.Source: Table 8 · p. 22, G Additional Ablation Analysis · p. 22

H Details of Experimental Results

This section presents INDICT’s reported safety, benign-output, and helpfulness results across coding and attack benchmarks, alongside the prompts and knowledge-query setup used by its critics. It describes a helpfulness critic focused on performance and accuracy and a summarizer that balances those goals with security and safety.

I Instruction Prompts

  • The templates use {question} and {answer} for the task and latest actor output, append execution results to {answer} during posthoc feedback, and use {scratchpad} for prior critic interactions.The appendix provides example prompts in Listings 1–10.Source: I Instruction Prompts · p. 22
  • Critics use zero-shot prompts to generate knowledge queries, while the prompt set includes separate safety and helpfulness roles, a discussion summarizer, and search and code-review actions.Query instructions use the Search[keyword] format; code snippets may support searches or interpreter-based checks.Source: I Instruction Prompts · pp. 22, 25, 26, 27

H Details of Experimental Results

  • CommandR+INDICT sets new instruction-split safety results and leads on C#, Java, and Python autocomplete; all three tested CVS models improve safety, exceeding ground-truth secure solutions.On autocomplete, CommandR+INDICT is comparable to the prior state of the art; on CVS, the reported measure is the percentage of outputs classified as secure.Source: Table 10 · p. 23, Table 9 · p. 23, Table 12 · p. 24
  • On CVS, Llama3-70b-instruct+INDICT makes more than 65% of outputs more helpful than ground-truth solutions; CyberSecEval-1 autocomplete outputs also surpass GPT models on helpfulness.The CyberSecEval-1 helpfulness results report significant improvements on both autocomplete and instruction splits.Source: Table 11 · p. 23, Table 13 · p. 24
  • Llama3-70b-instruct+INDICT achieves more than 92% benign outputs on prompt injection; INDICT also raises benign outputs above 76% on Cyber Attack and 80% and 90% on Interpreter Abuse.The Interpreter Abuse results correspond respectively to CommandR and Codellama-13b-instruct; the tables also report results by attack type.Source: H Details of Experimental Results · p. 22, Table 14 · p. 24, Table 16 · p. 25

I Instruction Prompts

  • The helpfulness critic checks solution correctness and task alignment, prioritizes performance and accuracy over security, and debates a security-focused critic.The prompt asks the critic to diagnose execution steps and explain any mismatch with the task’s requirements or intentions.Source: I Instruction Prompts · pp. 25, 28
  • The summarizer weighs performance and security equally, then judges whether the solution is correct, safe, and aligned with task requirements and intentions.The prompt also directs it to reject harmful, unethical, racist, sexist, toxic, dangerous, or illegal content.Source: I Instruction Prompts · p. 29
  • The summarizer is instructed to incorporate supporting facts from the critics’ discussion to make its analysis informative and useful.The helpfulness critic and summarizer prompts both limit their responses to a few sentences.Source: I Instruction Prompts · p. 29

J Qualitative Analysis

The qualitative examples show INDICT improving both code safety and helpfulness relative to strong baselines, while also illustrating failures caused by nontrivial errors.

J Qualitative Analysis

Related papers

Loading 2407.02518v2…