PaperlayerView exact revision on arXiv

Source-linked AI summary

A Property Induction Framework for Neural Language Models

Kanishka Misra, Julia Taylor Rayz, Allyson Ettinger

arXiv:2205.06910v1cs.CL

TL;DR

This paper asks how language-trained LMs represent conceptual knowledge and generalize novel properties. It develops a two-stage property-induction framework, finding that models in most cases favor concepts within the same taxonomic category, consistent with taxonomic bias in their representations.

Abstract

from arXiv · show

To what extent can experience from language contribute to our conceptual knowledge? Computational explorations of this question have shed light on the ability of powerful neural language models (LMs) -- informed solely through text input -- to encode and elicit information about concepts and properties. To extend this line of research, we present a framework that uses neural-network language models (LMs) to perform property induction -- a task in which humans generalize novel property knowledge (has sesamoid bones) from one or more concepts (robins) to others (sparrows, canaries). Patterns of property induction observed in humans have shed considerable light on the nature and organization of human conceptual knowledge. Inspired by this insight, we use our framework to explore the property inductions of LMs, and find that they show an inductive preference to generalize novel properties on the basis of category membership, suggesting the presence of a taxonomic bias in their representations.

Introduction

The introduction asks how language-trained LMs represent conceptual knowledge and whether they generalize novel properties in ways that reflect taxonomic structure. It presents a property-induction framework and reports a preference for generalizing within taxonomic categories.

Introduction

  • The framework trains LMs to judge concept–property statements, then tests how they generalize entirely novel properties.The second stage adapts the property-judgment models with new property information before testing generalization.Source: Introduction · p. 1
  • Models prefer projecting novel properties on the basis of taxonomic category membership, suggesting represented taxonomic features guide generalization.The authors test generalization from concepts such as robins to birds such as sparrows, rather than non-birds such as zebras.Source: Introduction · p. 1
  • The framework studies non-deductive inference, unlike natural language inference, bringing human-like generalization that is seldom studied in LMs into focus.It complements work on LMs’ inductive biases concerning syntactic structure and universal linguistic constraints.Source: Introduction · p. 1
  • The work connects diagnosis of LMs’ conceptual knowledge with analysis of how their representations guide generalization, and with connectionist approaches to property induction.It situates this work alongside studies of conceptual knowledge in LMs and prior connectionist models of property induction.Source: Introduction · p. 2

Testing Property Induction with Arguments

The section presents property induction as an argument that projects a property from premise concepts to a conclusion concept. It uses synthetically created novel properties to probe generalization across conceptual relations.

Testing Property Induction with Arguments

  • Property-induction stimuli take the form of arguments, with a premise stating a property of one concept and a conclusion extending it to another.The example moves from “Robins have sesamoid bones” to “All birds have sesamoid bones.”Source: Testing Property Induction with… · p. 2
  • Argument strength measures how much belief in premise statements strengthens belief in the conclusion.The premise and conclusion subjects are termed premise and conclusion concepts, respectively.Source: Testing Property Induction with… · p. 2
  • Nonce-word properties such as “can dax” simulate blank properties, helping probe conceptual relations without relying on familiar property knowledge.Blank properties are synthetically created and unknown to participants, encouraging generalization based on relations between the concepts.Source: Testing Property Induction with… · p. 2

The Framework

The framework first trains LMs to judge concept–property statements, then adapts them with novel premise information and measures projection to conclusion concepts. It treats the resulting judgments as evidence about the models’ generalization.

The Framework

  • Property induction is modeled as estimating the probability of a conclusion given a premise, with two requirements: judge property–concept associations and accept new property information for generalization.The framework estimates truth probabilities for concept–property sentences rather than sequence probabilities, to avoid surface-level confounds.Source: The Framework · p. 2

Stage 1: Eliciting Property Judgments using LMs

  • The first stage fine-tunes LMs to classify concept–property sentences as true or false, with evaluation properties disjoint from those used in training.The model outputs p(True | s, φ), intended to reflect the truth of the sentence as internalized by the LM.Source: Stage 1: Eliciting Property Judgments… · p. 2

Stage 2: Induction as Adaptation to New Knowledge

  • The second stage adapts the model on premise sentences by backpropagation, freezes the updated parameters, and scores projection of the novel property to conclusion concepts.The generalization score measures the strength of projecting the property to one or more concepts in the generalization set; parameters are reset before subsequent trials.Source: Stage 2: Induction as Adaptation to… · p. 3

Investigating LMs on Property Judgments

The property-judgment experiment evaluates LMs on concept–property associations using disjoint property sets. The authors report strong test performance across three model families, while noting that property-norm data can yield misleading negative examples when elicitation is incomplete.

Tested LMs

  • The experiment fine-tunes BERT, RoBERTa, and ALBERT on property-knowledge data and evaluates truth judgments on properties unseen during training.The models are the largest variants in their respective families, and test performance is measured with F1 scores.Source: Investigating LMs on Property Judgments · pp. 3–4

Ground-truth Property Knowledge Data

  • Property-norm datasets may omit obvious properties, making unelicited concept–property pairs unreliable as negative examples.The authors found that “has a mouth” was elicited for only 6 of 152 animal concepts, and report producing a revised dataset of 521 concepts and 3,735 properties.Source: Investigating LMs on Property Judgments · pp. 3–4
  • The authors generate negative samples from concepts maximally similar to those associated with each property, making judgments finer-grained than comparisons with distant concepts.They split data into 80/10/10 training, validation, and testing sets, with properties disjoint across these sets.Source: Investigating LMs on Property Judgments · p. 4

Results

LMs using Property Induction

The section tests whether three property-judgment LMs generalize novel properties by taxonomic membership and examines whether property overlap in training data explains that preference.

Motivation

  • The experiment tests whether LMs share humans’ tendency to generalize novel properties according to category membership.Prior human studies found a preference to generalize unfamiliar properties based on biological taxonomies and category membership.Source: LMs using Property Induction · p. 4

Data

  • Across 2,400 trials per model, adaptation sets used 1–5 concepts and eight novel properties, with Within, Outsidesimilar, and Outsiderandom test sets.Outsidesimilar was selected using model-specific embedding cosine similarity; Outsiderandom was sampled independently of the model.Source: LMs using Property Induction · p. 5

Method

  • In each trial, models adapted until perfect accuracy, then received generalization scores for each of the three test sets.Figure 2 plots average scores across properties as adaptation-set size varies.Source: LMs using Property Induction · p. 5, Figure 2 · p. 5

Results and Analysis

  • All three models had significantly higher average generalization scores for Within than for either Outside set (p < .001).The pattern suggests a preference to generalize newly learned properties to members of the same superordinate category.Source: Figure 2 · p. 5, LMs using Property Induction · p. 5

Property overlap analysis

  • Property overlap positively predicted generalization scores for all three LMs, alongside adaptation-set size and cosine similarity (p < .001).The analysis modeled overlap, cosine similarity, and their interaction, with random intercepts for novel property and trial.Source: LMs using Property Induction · p. 5

Taxonomic generalization with conflicting property overlap

  • p < .001: Each model favored Within over Outside even when the Outside category had greater property overlap with the adaptation concepts.The comparison used six concepts with greater property overlap outside their own superordinate category, separating overlap from taxonomic membership.Source: Figure 3 · p. 6, LMs using Property Induction · p. 6

General Discussion and Conclusion

The paper introduces property induction as a way to study how LMs generalize novel concept properties. The models generally favor taxonomic category membership, and that preference persists when property overlap is separated from category membership.

General Discussion and Conclusion

  • The framework adapts property-judgment LMs to novel property inputs, then uses their projections to probe inductive preferences and conceptual knowledge.Property induction simulates a process cognitive scientists use to study how people project novel information about concepts.Source: General Discussion and Conclusion · p. 6
  • In most cases, the LMs preferred projecting novel properties to concepts in the same taxonomic category, even when property-overlap effects were teased apart.Property overlap in training was statistically predictive of projection, but did not account for the persistent taxonomic preference.Source: General Discussion and Conclusion · p. 6
  • The authors cannot identify the source of the taxonomic bias; they suggest it may reflect latent taxonomic knowledge acquired during pretraining.They propose that knowledge learned while predicting words in context may be implicitly activated when models generalize novel properties.Source: General Discussion and Conclusion · p. 6
  • Future tests could compare how biological properties generalize across taxonomies with how behavioral properties generalize through shared features.The authors present this as one route to studying context-specific flexibility in LM representations.Source: General Discussion and Conclusion · p. 6
Loading 2205.06910v1…