PaperlayerView exact revision on arXiv

Source-linked AI summary

Cognitive Thermometers: Machine Learning and Logical Complexity

Shane Steinert-Threlkeld, Jakub Szymanik

arXiv:2610.10724v1cs.CL

TL;DR

The paper addresses how to explain semantic patterns in natural languages when logical complexity depends on choices of representation. It uses machine learning to measure ease of learning across meanings, finding that learning and logical complexity can agree but that learning is the stronger predictor of monotonicity; the paper argues that ease of learning is conceptually prior to complexity.

  • Problem

    The paper asks how constrained variation distinguishes humanly possible linguistic systems, while noting that logical complexity depends on prior choices of representational language.

    Source: §1 · p. 1, §3 · p. 5

  • Method

    The paper uses machine learning as a cognitive thermometer, measuring learnability across semantic spaces and comparing it with logical complexity.

    Source: §1 · p. 2, §2 · p. 3

  • Results

    Learning effort and logical complexity can be positively correlated, but ease of learning explains more variance in monotonicity than complexity does.

    Source: §3 · p. 5

  • Takeaways & Limitations

    The authors argue that ease of learning is conceptually prior to complexity because it has more explanatory power in their account.

    Source: §3 · p. 4

    The ease-of-learning invariance thesis is restricted to sufficiently natural spaces, and delimiting the learners and domains where convergence holds remains an open central problem.

    Source: §4 · p. 7

Abstract

from arXiv · show

How does the human mind represent semantic categories? Why do natural languages favor certain meanings over others? Prior explanations have relied on logical definability and complexity, but these are highly sensitive to the choice of logical language, rendering some design choices unmotivated. In this article, we propose that machine learning provides a somewhat more agnostic approach to measuring semantic complexity. We review emerging evidence that logic and machine learning often yield converging results on relative complexity and its resulting effects in semantic typology. Where they diverge, learning appears to be a better explanation than logical complexity. We argue that treating machine learning models as ``cognitive thermometers'' enables a unified approach to complexity that bridges symbolic logic and connectionist AI.

1 Ease of Learning and Semantic Universals

The section frames constrained variation in language as a central question for theoretical linguistics and presents ease of learning as an explanation for semantic universals. It proposes machine learning as a way to measure learnability across possible meanings at scale.

1 Ease of Learning and Semantic Universals

  • Theoretical linguistics asks what distinguishes humanly possible linguistic systems from the much larger space of mathematically possible systems.This question concerns constrained variation across language, including meaning.Source: §1 · p. 1
  • The ease-of-learning hypothesis says meanings with distinguished properties are easier to learn, and languages may lexicalize them while using grammar for more complex meanings.For example, if the hypothesis holds, monotone quantifiers should be easier to learn than non-monotone ones.Source: §1 · p. 2
  • Machine learning models measure the learnability of mathematically possible meanings to build a heatmap, with warmer meanings representing easier-to-learn meanings.The approach can measure learnability at a scale infeasible with human learners and avoids confounds from people already knowing languages with the relevant universals.Source: §1 · p. 2

2 Example: Monotonicity of Quantifiers

Quantifier monotonicity provides an example of how learning models can test whether a semantic property tracks learnability. Monotone quantifiers were learned more quickly than non-monotone ones, and broader tests found that greater monotonicity correlated with easier learning.

2 Example: Monotonicity of Quantifiers

  • In predicate replacement, upward monotonicity preserves entailment when a predicate is replaced by a more general one, while downward monotonicity involves a more specific replacement.For example, “Most Seattleites like to go hiking” entails “Most Seattleites like to go outdoors.”Source: §2 · p. 2
  • The proposed lexical universal is that determiners denote monotone quantifiers, with monotonicity expressed as a property of a generalized quantifier’s characteristic function.Determiners include words such as “some,” “all,” and “most.”Source: §2 · p. 2
  • The authors report that adults’ acquisition of a novel quantifier tracks the same universals, including monotonicity, found for neural learners.The cited artificial-language-learning study provides complementary human-learning evidence.Source: §2 · p. 2
  • Monotone quantifiers were reliably learned more quickly than non-monotone quantifiers, although all tested meanings were learnable.Figure 1 compares accuracy on unseen instances across training steps for monotone and non-monotone quantifiers.Source: Figure 1 · p. 3, §2 · p. 3
  • A broader study sampled quantifiers from a context-free grammar and found that degree of monotonicity correlated with ease of learning.Degree of monotonicity was generalized as mutual information between a function and its minimal monotone extension.Source: §2 · p. 3
  • The authors report similar learning patterns for other quantifier universals, color-term convexity, and certain attitude predicates, treating models as measurement devices across large semantic spaces.They call these models cognitive thermometers because they build a heatmap over possible meanings.Source: §2 · p. 3

3 (Machine) Learning and (Logical) Complexity

The section compares machine-learning ease of learning with logical and algorithmic complexity as accounts of semantic regularities. It argues that learning is less tied to representational choices and can be more explanatory, while complexity sometimes converges with learning.

  • The authors argue that ease of learning has more explanatory power than complexity in many cases and is conceptually prior to complexity measures.Source: §3 · pp. 3–4
  • Ease of learning explains significantly more variance in degree of monotonicity than logical complexity, while adding complexity beyond learning explains very little.Source: §3 · p. 5
  • Monotone quantifiers tend to have shorter minimum description lengths than non-monotone ones, and neural-network learning effort and logical complexity are positively rank-correlated across Boolean categories.Source: §3 · p. 5
  • For 65,536 Boolean categories, meanings expressible in at most eight symbols with nine operators require up to sixteen under NAND and up to thirty-five under {OR, NOT}.Source: Figure 3 · p. 5
  • Complexity depends on representation language or encoding: category-learning data recover the Boolean language of thought in well under one percent of simulations, while machine learning fixes neither choice.Source: §3 · pp. 3, 5, 6
  • The measured learnability ordering is reproduced across LSTMs and Transformers, supporting its stability beyond any one learner’s inductive bias.Source: §3 · p. 6

4 Moving Forward

Machine-learning measures place semantic universals in warmer regions of meaning spaces, motivating an ease-of-learning ordering that sufficiently general learners share. The authors argue this ordering can unify symbolic complexity measures with learning, while stressing that its scope in natural meaning spaces remains to be delimited.

  • Machine-learning measures place semantic universals in warm regions of meaning spaces.The paper presents this as a finding from its cognitive-thermometer approach.Source: §4 · p. 6
  • The ease-of-learning invariance thesis proposes that sufficiently general learners share an ordering of meanings that semantic universals track.Learners may differ in absolute learning effort while preserving the ordering up to an additive constant.Source: §4 · p. 6
  • Ease of learning is proposed as the invariant that symbolic simplicity measures approximate, agreeing when their primitives fit and diverging otherwise.The analogy is with Kolmogorov complexity, where changing the universal machine shifts complexity by a constant independent of the item.Source: §4 · pp. 6–7
  • No-free-lunch results mean the thesis fails over wholly unnatural spaces; identifying natural spaces and domains of convergence remains the central problem.The proposed convergence is limited to sufficiently natural spaces and cognitively plausible, domain-general learners.Source: §4 · p. 7
  • If the thesis holds, a shared learnability scale could organize universals beyond quantifiers and help infer human learners’ priors from typological patterns.The paper names color, kinship, and modality as possible domains, and communicative efficiency as a pressure the scale might coincide with.Source: §4 · p. 7
Loading 2610.10724v1…