PaperlayerView exact revision on arXiv

Source-linked AI summary

On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex

Terry Yue Zhuo, Zhuang Li, Yujin Huang, Fatemeh Shiri, Weiqing Wang, Gholamreza Haffari, Yuan-Fang Li

arXiv:2301.12868v3cs.CL

TL;DR

Whether few-shot prompt-based semantic parsers are vulnerable to adversarial attacks remains an open question, against a backdrop of costly in-domain supervision. This study evaluates CODEX using meaning-preserving perturbations and tests prompt-example strategies; it finds vulnerability alongside several robustness improvements from in-context learning.

  • Problem

    Whether prompt-based semantic parsers share adversarial vulnerability remains an open question, while supervised semantic parsing requires costly in-domain data.

    Source: §1 · p. 1

  • Method

    The study evaluates CODEX on meaning-preserving perturbations and tests demonstration quantity, few-shot adversarial examples, and example-selection strategies.

    Source: §3.1 · p. 3

  • Results

    CODEX is vulnerable, particularly to sentence-level perturbations; more demonstrations and higher-complexity example sampling can improve robustness, while adversarial prompt examples have limited effect.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors frame the findings as a basis for future research on robustness in prompt-based semantic parsing.

    Source: §6 · p. 9

    The study focuses on text-to-SQL tasks and calls for research across different datasets and logical forms, as well as alternative adversarial-training strategies.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Semantic parsing is a technique aimed at constructing a structured representation of the meaning of a natural-language question. Recent advancements in few-shot language models trained on code have demonstrated superior performance in generating these representations compared to traditional unimodal language models, which are trained on downstream tasks. Despite these advancements, existing fine-tuned neural semantic parsers are susceptible to adversarial attacks on natural-language inputs. While it has been established that the robustness of smaller semantic parsers can be enhanced through adversarial training, this approach is not feasible for large language models in real-world scenarios, as it requires both substantial computational resources and expensive human annotation on in-domain semantic parsing data. This paper presents the first empirical study on the adversarial robustness of a large prompt-based language model of code, \codex. Our results demonstrate that the state-of-the-art (SOTA) code-language models are vulnerable to carefully crafted adversarial examples. To address this challenge, we propose methods for improving robustness without the need for significant amounts of labeled data or heavy computational resources.

1 Introduction

The introduction asks whether few-shot prompt-based semantic parsers share the adversarial vulnerability of supervised parsers and how their robustness might be improved. Experiments on CODEX show vulnerability and report mixed effects from prompt-based interventions.

1 Introduction

  • The experiments probe CODEX on GeoQuery and Scholar and address prompt-example quantity, adversarial examples, and example selection.Prompt-based parsers use a few demonstration examples for in-context learning; this can reduce annotation costs and achieve results comparable to fully supervised parsers.Source: §1 · pp. 1–2
  • Prompt-based parsers are vulnerable to adversarial attacks, especially sentence-level perturbations.Evaluation across several perturbation approaches also found vulnerability to various attack types.Source: §1 · pp. 1–2
  • More demonstration examples in the prompt can improve prompt-based parsers’ in-domain robustness.The study also tests adding adversarial examples and selecting examples through non-random strategies.Source: §1 · p. 2
  • Adding adversarial examples to prompts has limited effect on improving parser robustness.Source: §1 · p. 2
  • Few-shot examples with higher linguistic complexity can yield stronger robustness for prompt-based parsers.Source: §1 · p. 2

2 Related Work

Related work describes prompt-based learning as an alternative to large annotated training sets and surveys adversarial attacks and defenses. This study extends prompt-based semantic parsing research by examining robustness.

2 Related Work

  • Prompt-based learning uses prompt templates and supports zero- and few-shot tasks, including semantic parsing.The paper extends prior semantic-parsing work by studying the robustness of prompts with limited examples.Source: §2 · p. 2
  • NLP adversarial attacks include character-level manipulations, sentence-level rewriting, and adversarial word substitutions.Source: §2 · p. 2
  • Adversarial training incorporates adversarial examples into training data, but can sometimes harm neural models’ generalization.Source: §2 · p. 2

3 Robustness Evaluation for Prompt-based Semantic Parsing

The evaluation framework constructs meaning-preserving adversarial test sets and assesses CODEX using execution-based accuracy measures. It includes multiple perturbation types, human filtering, and metrics that distinguish original, perturbed, and robust performance.

3 Robustness Evaluation for Prompt-based Semantic Parsing

  • Table 1 shows three GeoQuery perturbation examples, each of which changes CODEX’s prediction.Source: Table 1 · p. 3

3.1 Construction of the Evaluation Corpus

  • The evaluation framework creates perturbed utterances paired with ground-truth logical forms, with perturbations intended to preserve meaning.An adversary is a meaning-preserving utterance on which the parser fails to produce the correct logical form.Source: §3 · p. 2, §3.1 · p. 3

3.1.1 Adversarial Perturbations

  • The evaluation includes five word-level perturbations: typos, random deletion, random swaps, context-aware substitutions, and context-aware insertions.Together with two sentence-level perturbations, these form seven robustness evaluation sets per benchmark.Source: §3.1.1 · p. 3
  • Sentence-level perturbations include appending interrogation statements and rewriting complete utterances.The study uses distraction-based statements and Quillbot paraphrases.Source: §3.1.1 · p. 3

3.1.2 Data Filtering

  • The two-stage filtering process ranks 20 generated examples by Sentence-BERT similarity, then has human experts choose the best of the top 10.Source: §3.1.2 · pp. 3–4

3.2 Evaluation Metrics

  • Execution accuracy is used because string-based BLEU and Exact Match are unsuitable when equivalent logical forms use different naming conventions.Source: §3.2 · p. 4
  • The framework reports standard accuracy on original tests, perturbation accuracy on perturbed tests, and robust accuracy on perturbed cases whose original counterparts were parsed correctly.Robust accuracy is the fraction of that subset parsed correctly after perturbation.Source: §3.2 · p. 4

4 Improving Robustness of Prompt-based Semantic Parsers

The section investigates whether in-context learning can improve parser robustness, given that it differs from conventional supervised training. It examines standard and adversarial prompting, as well as strategies for selecting demonstrations.

4 Improving Robustness of Prompt-based Semantic Parsers

  • Large language models such as Codex predict logical forms through in-context learning from prompts containing utterance–logical-form examples and, optionally, a table schema.Source: §4 · p. 4
  • Conventional adversarial training is unsuitable for in-context learning because prompt tokens limit demonstration count and the model receives no gradient updates.Source: §4 · p. 4
  • The paper investigates whether in-context learning can improve parser robustness like conventional supervised training, and evaluates variants of in-context learning and example sampling.Source: §4 · p. 4

4.1 Standard In-context Few-shot Learning

  • In standard few-shot learning, the model predicts a logical form from the input utterance conditioned on demonstration pairs and a table schema, with demonstration count limited by a budget.Source: §4.1 · p. 4

4.2 Adversarial In-context Few-shot Learning

  • Adversarial in-context learning adds perturbed adversarial examples to the prompt’s demonstration examples.Source: §4.2 · p. 4

4.3 In-context Few-shot Selection

  • Because example pools supply prompting candidates, the study examines whether selection strategy affects parser robustness.Source: §4.3 · pp. 4–5
  • Random sampling selects utterances randomly, while confidence-based sampling chooses those with the parser’s lowest confidence in its predicted logical form.Source: §4.3 · p. 5
  • Diversity-based sampling selects cluster-center examples using edit-distance or utterance-feature distances; perplexity-based sampling selects utterances with highest or lowest GPT-2 perplexity.Source: §4.3 · p. 5

5 Experiments

Experiments on GeoQuery and Scholar evaluate CODEX’s adversarial vulnerability and test whether in-context examples and sampling strategies change robustness. More standard examples improve robustness, while adversarial examples in prompts provide only marginal gains; sampling strategy and lexical diversity also matter.

5.1 Setup

  • The evaluation uses CODEX on GeoQuery and Scholar query splits, with no logical-form template overlap across train, test, and development sets.The experiments use code-davinci-002, identified as the most powerful CODEX variant, with 175B parameters.Source: §5.1 · p. 5

RQ1: How vulnerable is the prompt-based semantic parser to adversarial examples?

  • Zero-shot CODEX is more vulnerable to sentence-level than word-level perturbations; typo-based attacks reduce its accuracy by only 3%.The study evaluates zero-shot performance on GeoQuery and Scholar, with perturbation accuracy reported by attack method.Source: Table 2 · p. 6, RQ1: How vulnerable is the… · p. 6

RQ2: Does standard in-context few-shot learning improve the robustness?

  • For few-shot CODEX, adding standard in-context examples improves robustness and standard execution performance, while the gap between perturbation and standard accuracy widens and robust accuracy grows slowly as shots increase.Source: RQ2: Does standard in-context few-shot… · p. 7

RQ3: Does adversarial in-context learning improve robustness?

  • Adversarial in-context examples yield only marginal robustness gains, while doubled standard examples outperform adversarial variants on both standard and robust accuracy.On Scholar, adversarial in-context learning raises standard accuracy from 23% to more than 33%, while robust accuracy improves only marginally.Source: Table 5 · p. 7, RQ3: Does adversarial in-context… · pp. 7–8

RQ4: What is the impact of sampling techniques on the robustness of parsers that utilize standard in-context few-shot learning?

  • Cluster sampling performs better overall, whereas PPL. Desc performs consistently poorly; CODEX’s performance varies with the few-shot sampling strategy.The comparison covers robust, perturbation, and standard accuracy on GeoQuery and Scholar.Source: Table 6 · p. 8, RQ4: What is the impact of sampling… · p. 8

RQ4-Ablation: Why does CODEX react differently to various sampling strategies?

6 Conclusion

The study finds that Codex is susceptible to adversarial attacks, while forms of in-context learning can improve robustness. It calls for further work on prompt-based semantic parsing robustness.

6 Conclusion

  • The prompt-based semantic parser remains susceptible to adversarial attacks, even when based on a large pre-trained language model.The experiments also indicate that various forms of in-context learning can improve its robustness.Source: §6 · p. 9
  • The study examines semantic-parser robustness in prompt-based few-shot learning and the effects of in-context learning.It focuses on CODEX.Source: §6 · p. 9
  • The authors present the findings as a catalyst for future studies of robustness in prompt-based semantic parsing with large language models.Source: §6 · p. 9

Limitations

The study focuses on how prompt design affects CODEX and limits its scope to text-to-SQL tasks. It calls for research on alternative adversarial-training strategies and robustness across datasets and logical forms.

Limitations

  • The study examines how prompt design affects CODEX’s execution performance, with its focus limited to text-to-SQL tasks.Source: Limitations · p. 9
  • The authors identify alternative adversarial-training strategies for prompt-based semantic parsers as a subject for future research.Source: Limitations · p. 9
  • They encourage further investigation of semantic-parser robustness across different datasets and logical forms.Source: Limitations · p. 9

A Experiments

The experiments use the query-split of GeoQuery and Scholar, sample CODEX generations with specified decoding settings, and use a mid-2022 model version.

A Experiments

  • GeoQuery has 877 natural-language/logical-form pairs; the query-split test sets contain 182 GeoQuery pairs and 315 Scholar pairs.The query-split is described as a more challenging evaluation scenario.Source: A Experiments · p. 13
  • CODEX generations use temperature 0, a maximum of 200 sampled tokens, and stop strings “--”, “\n\n”, “;”, and “#”.Source: A Experiments · p. 13
  • The paper’s code-davinci-002 version is the one available around mid-2022.Source: A Experiments · p. 13

B Adversarial Examples

Table 8 presents adversarial utterances generated from one GeoQuery question using several perturbation strategies; all alter Codex’s output.

B Adversarial Examples

Loading 2301.12868v3…