Source-linked AI summary
Are Language Models Models?
Philip Resnik
TL;DR
The paper asks whether language models qualify as “model systems” for language processing and examines that question across Marr’s three levels. It concludes that LMs are poor cognitive model systems but can still serve productively as tools for developing explanatory models.
Problem
The paper examines whether language models are “model systems” of language processing, meaning models with structured correspondences between model entities and relationships and those in the real-world system.
Method
The commentary evaluates language models as models of language processing at Marr’s implementation, algorithmic/representational, and computational theory levels.
Results
The paper concludes that LMs are poor candidates as “model systems” at all three levels, citing weak implementation-level correspondence, problems with convergence-based arguments, and unresolved links to biologically plausible algorithms.
Takeaways & Limitations
LMs can be productive tools for developing explanatory models, including generating plausible text, providing probability proxies, and supplying world knowledge to neurobiologically plausible models.
Takeaways & Limitations
At the computational theory level, LM architectures still require mapping to biologically plausible real-time algorithms distributed across diverse specialized components.
Abstract
from arXiv · showhide
Futrell and Mahowald claim LMs "serve as model systems", but an assessment at each of Marr's three levels suggests the claim is clearly not true at the implementation level, poorly motivated at the algorithmic-representational level, and problematic at the computational theory level. LMs are good candidates as tools; calling them cognitive models overstates the case and unnecessarily feeds LLM hype.