Source-linked AI summary
Goal Driven Discovery of Distributional Differences via Language Descriptions
Ruiqi Zhong, Peter Zhang, Steve Li, Jinwoo Ahn, Dan Klein, Jacob Steinhardt
TL;DR
The paper addresses the difficulty of manually discovering meaningful differences between large corpora. It formalizes D5, builds evaluation resources and systems for goal-driven natural-language discoveries, and finds that goal conditioning improves correctness and relevance while supporting diverse exploratory applications. The evaluation remains limited for diversity, causality, and expert-knowledge-heavy discoveries.
Problem
Exploratory analysis of large corpora is useful but can be ad hoc and laborious, motivating machine assistance for discovering distributional differences.
Method
The paper formulates D5 as goal-conditioned natural-language generation and evaluates systems with diagnostic SYND5 and open-ended OPEND5 strategies.
Results
The goal-conditioned variant is correct 12% more often on SYND5 and produces relevant candidate discoveries 31% more often on OPEND5.
Takeaways & Limitations
D5 systems can support exploratory analyses across domains by producing useful discoveries from open-ended corpus pairs.
Takeaways & Limitations
The evaluation does not reward diverse discoveries, does not reliably assess causal conclusions, and cannot evaluate discoveries requiring heavy expert knowledge.
Abstract
from arXiv · showhide
Mining large corpora can generate useful discoveries but is time-consuming for humans. We formulate a new task, D5, that automatically discovers differences between two large corpora in a goal-driven way. The task input is a problem comprising a research goal "$\textit{comparing the side effects of drug A and drug B}$" and a corpus pair (two large collections of patients' self-reported reactions after taking each drug). The output is a language description (discovery) of how these corpora differ (patients taking drug A "$\textit{mention feelings of paranoia}$" more often). We build a D5 system, and to quantitatively measure its performance, we 1) contribute a meta-dataset, OpenD5, aggregating 675 open-ended problems ranging across business, social sciences, humanities, machine learning, and health, and 2) propose a set of unified evaluation metrics: validity, relevance, novelty, and significance. With the dataset and the unified metrics, we confirm that language models can use the goals to propose more relevant, novel, and significant candidate discoveries. Finally, our system produces discoveries previously unknown to the authors on a wide range of applications in OpenD5, including temporal and demographic differences in discussion topics, political stances and stereotypes in speech, insights in commercial reviews, and error patterns in NLP models.
1 Introduction
D5 formalizes goal-driven discovery of differences between text distributions as a natural-language generation task, with evaluation strategies for both known and open-ended discoveries. Goal conditioning improves correctness and relevance, while OPEND5 supports exploratory analyses across diverse applications.
- Task formulation: D5 takes a user-specified exploration goal and a corpus pair as input, producing a natural-language discovery predicate that is valid and relevant to the goal.The task formalizes exploratory analysis so it can be automated, benchmarked, learned, and analyzed.
- Evaluation: Because open-ended discoveries lack fixed human-written references, the paper combines diagnostic SYND5 evaluation with subjective open-ended evaluation on OPEND5.SYND5 tests recovery of known differences in synthetic corpora, whereas OPEND5 contains 675 realistic problems across multiple domains and uses manual evaluation.
- Results: 12% more often on SYND5 and 31% more often on OPEND5, the goal-conditioned variant is respectively more correct and more relevant than the unguided system.The comparison evaluates a system designed to describe corpus-level differences without goals against a goal-conditioned variant.
- Applications: OPEND5 is envisioned as a growing repository that lets improved systems facilitate exploratory analysis through discoveries about reviews, discussions, speeches, lyrics, and NLP errors.The authors demonstrate this paradigm by automatically producing useful discoveries on open-ended problems.
- Limitations: The evaluation does not encourage diverse findings, does not always establish causal conclusions, and cannot assess discoveries requiring heavy expert knowledge.The authors identify these limitations through concrete OPEND5 examples and note that more problems can reveal further weaknesses.
- Learning: The system can be trained with a self-supervised algorithm using problem repositories without reference solutions, improving descriptions of differences between small text groups.This proof of concept trains language models to propose more valid hypotheses.
3 Evaluation Metrics for Open-Ended D5 problems
D5 discoveries are evaluated by whether they express true corpus differences and address the user’s exploration goal, with novelty and significance also explored as subjective criteria. The system proposes goal-conditioned hypotheses and uses language-model validation to filter candidates before output.
- Evaluation metrics: Validity measures whether a discovery is a truth predicate that occurs more often in Corpus A than Corpus B, estimated from human judgments over sampled texts.The empirical validity estimate also includes a one-sided t-test for the null hypothesis V ≤ 0.
- Evaluation metrics: Relevance measures whether a valid discovery addresses the specified exploration goal, using blinded human or language-model ratings from 0 to 2.A discovery can have perfect validity yet be irrelevant when it merely restates the corpus distinction rather than the requested property.
- Evaluation metrics: The evaluation also explores novelty, defined by generation difficulty, and significance, defined by the benefit of learning a discovery.These metrics are presented as additional subjective criteria beyond validity and relevance.
- Hypothesis proposer: The system prompts a language model with corpus samples and the exploration goal to generate 60 initial hypotheses across multiple prompts.Multiple sample sets let the proposer inspect more of the corpus pair despite prompt-length limits.
- Hypothesis validator: A validator language model approximates each hypothesis’s validity, applies a t-test with p′ ≤ 0.001, and outputs the surviving hypotheses as discoveries.The validator is fine-tuned with additional Turker annotations to better simulate human judgments.
- Self-supervised learning: Self-supervised training raises automated validity V′ from 0.22 to 0.37 and human-evaluated validity V from 0.07 to 0.10, with p-value 0.02.This proof-of-concept uses Flan-T5-xxl on small groups of text samples and provides preliminary evidence for improving hypothesis validity.
5 Quantitative Evaluation on SYND5 and OPEND5
SYND5 and OPEND5 enable quantitative and human evaluation of D5 systems across controlled and open-ended problems. Goal conditioning improves accuracy and relevance, while validator use and stronger proposers further improve SYND5 performance.
- SYND5 evaluation: Validator use, goal conditioning, and gpt-4 each improve SYND5 accuracy, with gpt-4 outperforming text-davinci-003 when both goals and validation are used.The comparison evaluates 8 combinations of proposer, validator, and goal settings; the gpt-4 comparison is significant at p < 1%.
- SYND5 evaluation: The conclusions remain unchanged when predicate-equivalence judgments use text-davinci-003 or when semantically similar discoveries are counted as correct.These are the reported robustness checks for the SYND5 comparison.
- Robustness and extensions: The experiments extend beyond synthetic text through additional proposer models and a human-written New York Times version of SYND5, where goals and validators still improve performance.The extensions use gpt-3.5-turbo, Flan-T5-xxl, and labeled NYT articles varying by topic and location.
- OPEND5 relevance evaluation: Using the exploration goal increases the relevance of text-davinci-003 hypotheses on OPEND5, with the conclusion independently reproduced by each evaluator at p < 10^-8.Agreement was moderate at Kappa=0.56, and additional evaluators—including crowdworkers and language models—supported the same conclusion.
- OPEND5 evaluation: Goal conditioning also benefits novelty and significance in the corresponding OPEND5 experiments.The paper reports these findings in Appendix 14 rather than the main quantitative table.
6 Qualitatively Analyzing Discoveries and Limitations with OPEND5
The OPEND5 analysis produces concrete discoveries across applications while exposing important limits in how D5 findings are evaluated. The evaluation may provide informative but unreliable validity signals, and it does not yet capture diversity, expert knowledge, or causality.
- 6.1 Producing Discoveries on OPEND5 and Analyzing Them: The analysis selected 21 discoveries for manual validation and retained 13 with significantly non-zero estimated validity under a false-discovery-rate-controlled threshold.The retained discoveries had p-values below 7%, with the threshold determined using Benjamini–Hochberg at a 10% false discovery rate.
- 6.1 Producing Discoveries on OPEND5 and Analyzing Them: The system produced discoveries across political speeches, NLP errors, airline reviews, and gender differences in self-reported happy moments.Examples include differences in immigrant framing, informal tone, legroom concerns, and mentions of children or family.
- 6.1 Producing Discoveries on OPEND5 and Analyzing Them: The approximated validity score V′ correlated 71% by Spearman and 66% by Pearson with human validity ratings V, but remained an unreliable signal.The authors characterize V′ as informative yet unreliable for practitioners.
- 6.2 Concrete Examples in OPEND5 Inform Limitations of D5 Evaluation: The evaluation does not reward diverse discoveries, so systems may repeat one finding through similar paraphrases instead of covering multiple valid differences.The paper illustrates this with alternative findings about airline fees and customer service, plus paraphrases about rude staff.
- 6.2 Concrete Examples in OPEND5 Inform Limitations of D5 Evaluation: Validity judgments can require domain experts, and observed corpus differences do not establish causal relationships or rule out evaluator bias.The paper gives medical interpretation and gendered family-mention examples, and notes additional scope limitations including single-predicate discoveries and incomplete OPEND5 coverage.
7 Related Work and Discussion
The paper situates D5 among inductive reasoning, exploratory analysis, and automated discovery, then tests robustness across judges, models, and synthetic versus human-written text. It also details crowdsourced validity scoring and computational costs.
- Related Work: D5 extends inductive reasoning with language models from discovering patterns in text to goal-conditioned descriptions of distributional differences.Related work covers rule induction, zero/few-shot instruction inference, and analogous inductive reasoning in vision.
- Related Work: The paper connects D5 to classical exploratory methods and to the broader challenge of automatically generating and evaluating meaningful discoveries.Examples include regression, n-gram models, and topic models, while prior epistemological work on discovery generation remains sparse.
- Robustness Checks: The evaluation combines diagnostic SYND5 experiments with robustness checks using alternative similarity judgments and semantically similar outputs.Using text-davinci-003 as the judge and counting semantically similar discoveries preserves the paper’s conclusions.
- Robustness Checks: The same goal-and-validator advantage appears with gpt-3.5-turbo and flan-t5-xxl and on an NYT-based extension containing topic and location differences.The NYT extension uses 9 topic values and 10 location values; open-sourced models lag significantly behind in the reported comparison.
- Evaluation and Cost: Crowdsourced validity scoring averages three Turkers’ calibrated ratings, while the study reports approximately $2.4 per OpenAI problem and around $2,500 total API cost.Turkers rate predicate truth on a six-option scale, and the paper reports 3,000 A100 GPU hours overall.
12 Prompt to Judge Predicate Similarity
The predicate-similarity judge asks whether an output discovery and reference are equivalent, related, or unrelated, using few-shot examples to guide Claude-v1.3.
- Similarity Prompt: Claude-v1.3 judges whether an output predicate matches the reference as equivalent, similar, or irrelevant through six few-shot predicate pairs.The evaluation prompt asks for yes, related, or no, with yes used for equivalence and yes or related for semantic similarity.
- Similarity Prompt: The examples distinguish exact semantic matches, related concepts, and unrelated attributes such as Korean language versus Japanese language.Examples include environmental protection versus sustainability, political figures versus diplomacy, and sports versus recruiting teammates.
13 Relevance Rating with External Non-Authors
External evaluators reproduce the finding that adding exploration goals improves the relevance of generated hypotheses, supporting robustness beyond the authors’ judgments.
- External Evaluation: Relevance improvements from goal conditioning persist across Turkers, gpt-3.5-turbo, Claude-v1.3, and GPT-4 evaluators.The authors report that the conclusion remains robust across five evaluator types when including the authors’ evaluations.
14 Meaningfulness: Relevance, Novelty, and Significance
The paper separates discovery validity from meaningfulness and evaluates meaningfulness through relevance, novelty, and significance. Goal-conditioned generation improves all three dimensions, despite only moderate inter-annotator agreement.
- Evaluation Criteria: Meaningfulness is evaluated independently of validity through relevance, novelty, and significance, each rated from 0 to 2.This separates whether a discovery is true from whether it is useful for the stated exploration goal.
- Relevance: Relevance measures alignment with the exploration goal, distinguishing direct information about the target property from unrelated or merely adjacent content.For writing-style analysis, “write in first person” receives the highest relevance rating, while “argue for abortion” receives zero.
- Novelty: Novelty measures how difficult a discovery is to generate before inspecting the corpora, with specific non-tautological findings rated more highly.The evaluation is blinded to samples, so a pattern can remain novel even if it becomes obvious after viewing the data.
- Significance: Significance measures the benefit of learning a valid discovery for the first time, favoring findings that can guide concrete action.The rubric distinguishes observations that merely describe a problem from discoveries that suggest specific product improvements.
- Goal-Conditioned Results: With the exploration goal, GPT-3 produces more relevant, novel, and significant hypotheses, with higher-than-zero ratings 31%, 21%, and 28% more often, respectively.Inter-annotator agreement is moderate, but the differences are statistically significant and robust across evaluators.
15 Full Pipeline of the Proposer
The proposer extracts representative corpus differences, prompts GPT-3 to generate hypotheses, rewrites them into sample-level predicates, and optionally supplies examples. A validator evaluates the resulting discoveries, with cross-problem splits and statistical filtering used for reported results.
- Obtaining representative samples: Representative samples are selected by fine-tuning RoBERTa-Large to distinguish the corpora, then retaining high-confidence samples for GPT-3 prompting.The method samples from the top-p percentile identified by the classifier.
- Selecting samples to prompt GPT-3: GPT-3 proposes hypotheses from sampled examples, with prompt length constraints and optional problem-specific examples intended to improve relevance.The pipeline selects samples from both corpora and can add manually selected example hypotheses.
- Rewriting hypotheses with GPT-3: Because generated hypotheses often describe whole corpora rather than individual samples, GPT-3 rewrites comparative statements into predicates suitable for validation.The system detects comparative forms using part-of-speech tags and prompts GPT-3 to remove them.
- Validating discoveries: Cross-problem validation uses folds split by problem, preventing the validator from being fine-tuned on pairs from the problem being evaluated.This design tests generalization to unseen D5 problems.
- Validating discoveries: Reported discoveries are screened for validity using sampled validation data, retaining 13 findings with significant nonzero validity under a 10% false discovery rate.The retained findings use a p-value below 7% determined by the Benjamini–Hochberg procedure; estimating validity for selected discoveries cost approximately $1500.
18 More Example Discoveries on OPEND5
The system generates interpretable discoveries across temporal, topical, linguistic, and NLP-error comparisons, while the paper identifies important limits in predicate form, evaluation, causality, prompting, and domain expertise.
- Analyzing errors in NLP systems: NLP error analysis found that cases missed by Tk-Instruct-11B but solved by Curie more often use positive or uplifting language, with V ≈0.12 and AUCROC ≈0.67.The authors suggest Curie may confuse undermining with negative sentiment rather than a logical relation.
- More example discoveries: Across OPEND5, discoveries identify differences in lyrics, SNLI versus MNLI premises, bot versus human tweets, and news headlines across years.Examples include violence references, physical activity, finance-related keywords, disasters and crimes, and coronavirus topics.
- Beyond truth predicates: The system’s truth-predicate formulation cannot represent arbitrary natural-language discoveries, real-valued properties, or combinations of predicates without substantial redesign.The paper suggests real-valued expressions and shallow decision trees as possible extensions.
- Prompt sensitivity: The proposer’s performance may depend on hand-crafted prompt formatting and manually annotated goals, which were not tested systematically.The paper leaves sensitivity to paraphrasing and improved prompts for future work.
- Evaluation and domain limitations: The evaluation cannot establish causal relations, and meaningfulness judgments depend on domain expertise and implicit community norms.Some predicates require experts to interpret, while the authors’ OPEND5 problems may not reflect what domain experts actually care about.
20 Self-Supervised Learning with Open-Ended Problems: A Proof of Concept
The paper uses self-supervised validation signals from open-ended problems to train a proposer toward more valid hypotheses. In a proof-of-concept on clustered text samples, both automated and human validity improved after training.
- Algorithm: The algorithm samples multiple hypotheses, estimates their validity, and uses selected prompt-completion pairs to fine-tune an initial language model.It is designed for settings where validating discoveries is easier than generating them.
- Data and setting: The proof of concept uses Flan-T5 on mini-problems that describe two groups of four samples drawn from text clusters.The data contains 33 merged domain corpora and 4503 automatically generated clusters, with 28 corpora reserved for evaluation.
- Results: Automated validity V′ increased from 0.22 to 0.37, while Turker-evaluated validity V increased from 0.07 to 0.10 with p-value 0.02.The result provides preliminary evidence that self-supervised training can improve hypothesis validity, although the automated validator may imperfectly match human judgments.
21 Comparing D5 to Naïve Bayes
Compared with Naive Bayes unigram features, D5 produces semantically coherent, goal-related descriptions that can capture differences not salient at the word level.
- Comparison with Naive Bayes: D5 directly generates semantically coherent descriptions, reducing the need to infer an underlying pattern from top unigram features.Naive Bayes remains effective when the target difference is saliently reflected by individual words.
- Comparison with Naive Bayes: D5 can capture differences expressed through sentence structure rather than individual words and restrict descriptions to differences related to the user’s goal.Naive Bayes instead selects any discriminative unigram, whether or not it is goal-related.
22 Annotation Interface to Collect Human-Generated Hypotheses
The annotation interface collected human-written predicates distinguishing two groups of text samples and recorded which samples satisfied each predicate. Preliminary comparisons found GPT-3 generated more valid hypotheses than humans and often proposed novel ones, leading the authors to abandon fine-tuning with these annotations.
- Annotation Interface: Annotators viewed five samples from each corpus, wrote natural-language predicates, and marked which samples satisfied each predicate.The interface was designed to support fine-tuning and more accurate validation.
- Preliminary Results: GPT-3 generated 4-6 valid hypotheses per text-group pair, compared with 2-4 from human annotators.The comparison used initial experiments on clusters formed from the wikitext-2 dataset.
- Preliminary Results: GPT-3 produced a novel hypothesis that humans had not considered in nearly every text corpus, while human-only hypotheses occurred in about a third of samples.Approximately one third of valid generated hypotheses were variations of another valid hypothesis.
- Preliminary Results: Because GPT-3 performed close to the authors’ hypothesis-writing ability, the authors decided not to fine-tune T5 using human annotations.The annotation effort was therefore discontinued as a research direction.
23 Datasets
The dataset collection assembles text corpora from varied sources, including news, reviews, job postings, arguments, dialogues, policy statements, and language-understanding datasets. Each corpus is transformed or categorized using source-specific rules and metadata.
- Source Coverage: The collection covers news headlines and articles, commercial reviews, job postings, advertisements, policy statements, and social-media discussions.Examples include ABC and Examiner headlines, airline and Amazon reviews, technology jobs, advertising transcripts, presidential statements, and Reddit posts.
- Source Coverage: Other sources target language understanding and generation, including Boolean questions, student essays, fake news, and natural-instruction tasks.The collection also includes Armenian job postings and datasets categorized by essay quality or news authenticity.
- Dataset Processing: Corpus construction applies source-specific processing such as extracting years, standardizing names, splitting text, selecting subsets, and assigning samples to categories.The cited datasets use these operations to balance availability, shorten descriptions, retain selected ratings, or define labels.
- Source Coverage: Several datasets represent argumentative, conversational, or socially meaningful language, including convincingness, negotiation outcomes, debate evidence, deception, and legal violations.These corpora use labels such as convincingness, success or failure, topic, deception-related categories, and violation status.