Source-linked AI summary
Dialectical language model evaluation: An initial appraisal of the commonsense spatial reasoning abilities of LLMs
Anthony G Cohn, Jose Hernandez-Orallo
TL;DR
Static commonsense benchmarks may not reveal whether language models reason robustly, since their scores can be affected by artifacts, leakage, and aggregation. This paper manually probes spatial reasoning through dialogue and finds frequent errors and fundamental misunderstandings in the systems’ responses.
Problem
Static commonsense benchmarks can mask limitations through artifacts, training-data leakage, aggregation, and multiple-choice formats, leaving model failures less visible.
Source: §1 · p. 1Method
The authors manually evaluate conversational language models with dialectical spatial-reasoning dialogues, using prior turns as context and sampling problems rather than attempting exhaustive coverage.
Source: §1.1 · p. 2, §2 · p. 2Results
The systems often provide answers but are frequently wrong or display fundamental misunderstandings, which granular dialogues expose more clearly than aggregate performance metrics.
Source: §3 · p. 8Takeaways & Limitations
The dialogue-based, granular account maps capability boundaries and is more informative about failures than aggregate performance measures.
Source: §3 · p. 8The evaluation is out of the box without feedback, and the authors report that fine-tuning the tested systems was not available.
Source: §3.1 · p. 9
Abstract
from arXiv · showhide
Language models have become very popular recently and many claims have been made about their abilities, including for commonsense reasoning. Given the increasingly better results of current language models on previous static benchmarks for commonsense reasoning, we explore an alternative dialectical evaluation. The goal of this kind of evaluation is not to obtain an aggregate performance value but to find failures and map the boundaries of the system. Dialoguing with the system gives the opportunity to check for consistency and get more reassurance of these boundaries beyond anecdotal evidence. In this paper we conduct some qualitative investigations of this kind of evaluation for the particular case of spatial reasoning (which is a fundamental aspect of commonsense reasoning). We conclude with some suggestions for future work both to improve the capabilities of language models and to systematise this kind of dialectical evaluation.
1 Introduction
Static benchmark scores may not demonstrate commonsense reasoning because artifacts, training-data leakage, aggregation, and multiple-choice formats can mask failures. The paper explores manual, dialectical evaluation of spatial reasoning to probe model boundaries through dialogue.
1 Introduction
- Aggregated benchmark scores can obscure failures, while artifacts, training leakage, and multiple-choice formats can make static evaluations less revealing.The authors note that failed examples may be overshadowed by cases solvable through shortcuts or statistical tricks.Source: §1 · p. 1
- The paper investigates manual dialectical evaluation, using accumulated question-and-answer context to probe commonsense spatial reasoning.Its stated aims are to develop test instances and evaluate language models on them.Source: §1 · p. 2
- Spatial reasoning is a focus because it may be detached from linguistic abilities and may require intuitive physics and spatial inductive biases.The paper names distance, size, and orientation as examples of such biases.Source: §1 · p. 2
1.1 Commonsense spatial reasoning
- Existing spatial benchmark items remain vulnerable to statistical tricks and contamination, and their static format limits follow-up probing.The authors nevertheless build on some existing examples because dialogue permits elaboration or variation.Source: §1.1 · p. 2
2 An evaluation of the spatial reasoning abilities of some LLMs
The authors evaluate conversational models through a qualitative, manual sample of spatial reasoning problems. They retain earlier turns in dialogue to expose mistakes that may not appear when prompts are asked separately.
2 An evaluation of the spatial reasoning abilities of some LLMs
- The evaluation includes ChatGPT versions using GPT-3.5turbo and GPT-4, plus Bard based on LaMDA.The paper refers to the five tested systems as C3.5, C4, P3.5, P4, and Bard.Source: §2 · p. 2
- The authors sample spatial challenges rather than attempt an exhaustive evaluation, emphasizing C3.5 because it shows more inadequate or incorrect responses.Correct responses are often marked with a check rather than reproduced.Source: §2 · p. 2
- Keeping prompts in one chat session reveals mistakes that may disappear when prompts are asked separately in a new conversation.The reported dialogues preserve prior interaction as context.Source: §2 · p. 3
2.1 Basic spatial relations
The basic-relation dialogues show uneven spatial reasoning: C3.5 struggles with parthood and combined directions, and gives mixed results on letter rotation.
Parthood
- C3.5 initially muddles transitive parthood and containment, but answers correctly when the prompt explicitly specifies spatial part.The later response still hedges that the contained object may be physically located within the larger object.Source: Parthood · p. 3
Rotation
- C3.5 answers one letter-rotation question correctly but misdescribes and incorrectly rotates Z in another.For the N question, it gives the correct letter despite incorrectly describing the letter shapes.Source: Rotation · p. 3
Directions
- C3.5 describes two points as lying on opposite sides of a constructed line, then cannot determine their left-right relation or combine left-right with above-below.The authors restrict these questions to points because directions between areal objects can be more complex.Source: Directions · pp. 3–4
2.2 Size, shape and location
The dialogues probe whether C3.5 can use spatial size and location to resolve references, but its answers are often incorrect or inconsistent across related prompts.
Circle size
- C3.5 incorrectly says a larger circle can fit entirely inside a smaller one, then gives an incorrect condition for when containment is possible.It claims the larger circle’s radius must be less than the distance between centers, then gives a different, also incorrect radius condition.Source: Circle size · p. 4
Trophy
- For the trophy-and-case sentences, C3.5 initially assigns “it” to the trophy and justifies this linguistically, overlooking that the case could be too small.When the sentence says “too big,” it gives a contradictory interpretation and ultimately insists “it” refers to the case.Source: Trophy · p. 4
Bookcase
- For the bookcase example, C3.5 first says “it” means the book, later accepts that it must be too big, but reverts to the wrong referent on re-asking.It correctly interprets the complementary “too big” sentence, yet still fails on the original “too small” version in the same session.Source: Bookcase · pp. 4–5
School bag
- For the sentence about John throwing his bag to Frank, C3.5 says “he” refers to John, identifying John—not Frank—as reaching the bottom of the stairs.The footnote suggests the book may be too small to stand upright stably, but says C3.5 claims “it is not fitting.”Source: School bag · p. 5
2.3 Affordances and object interaction
The authors use community challenge problems and object-interaction scenarios to probe spatial reasoning through dialogue, revealing uneven performance on physical affordances and object structure.
2.3 Affordances and object interaction
- The section examines spatial challenge problems from the Commonsense Problem page, which includes community-contributed problems and is not limited to spatial reasoning.Source: §2.3 · p. 5
Stakes in a Garden
- C3.5 identifies support and wind-damage risks in plant staking, but misses that a string shorter than the plant–stake distance cannot provide support.It handles an overly long string better, but misunderstands twisting the ends and how the string should keep the plant close to the stake.Source: Stakes in a Garden · pp. 5–6
The Handle Problem
- C3.5 gives a plausible general handle characterization, yet fails to recognize that a blunt cone tapering away from a drawer would provide a poor grip.Its discussion also confuses pulling a drawer open with pushing the handle inward and incorrectly calls that orientation common.Source: The Handle Problem · pp. 6–7
Reasoning about linked chains
- C3.5 predicts that a pulled chain follows and bends, but incorrectly says it stretches and becomes thinner, and repeatedly confuses a chain’s links with the chain forming a loop.The authors altered the initial prompt to clarify that the intended chain consists of linked loops, not a bicycle-type chain.Source: Reasoning about linked chains · p. 7
2.4 Object permanence
The authors probe object permanence through invented scenarios involving a person’s expectations and an occluded moving object. C3.5 handles some prompts well but fails to connect related spatial information consistently.
2.4 Object permanence
- The authors introduce these additional problems specifically to probe systems’ understanding of object permanence.Source: §2.4 · p. 8
Where are the car keys?
- C3.5 correctly says Mary expects the keys where she left them, but fails to connect the car’s absence with the keys’ likely location.It instead suggests checking usual places for keys or using a spare set to drive the absent car.Source: Where are the car keys? · p. 8
Cars behind the sofa
- C3.5 correctly infers that a green toy car passed behind a sofa, but struggles to explain a red car appearing on the other side.Its explanation suggests the red car may have been occluded by the green car or may have entered from the door it was approaching.Source: Cars behind the sofa · p. 8
3 Discussion
The dialogues frequently expose wrong answers or fundamental misunderstandings, and the authors argue that mapping these failures is more informative than aggregating performance. They describe limits of this initial evaluation and propose ways to extend and systematize dialectical probing.
3 Discussion
- Systems often answer, but are frequently wrong or show fundamental misunderstandings; granular dialogue findings map capability boundaries better than aggregated metrics.The effort is to find variations or continuations where a system fails.Source: §3 · p. 8
- Fluent, coherent output does not settle whether LLMs possess commonsense reasoning; whether such reasoning can emerge remains an open question.The authors invoke the view that language models stitch together observed linguistic forms without reference to meaning.Source: §3 · p. 8
3.1 Limitations
- The paper is not a comprehensive evaluation of spatial reasoning or the fast-changing foundation-model landscape.The authors nevertheless say it indicates frequent failures on relatively elementary spatial reasoning.Source: §3.1 · pp. 8–9
- In dialogue, an answer and its explanation can be correct or incorrect independently, so evaluating both can reveal fallacious reasoning even with a right answer.The authors describe the interactive approach as both a possible weakness and a strength.Source: §3.1 · p. 9
- The evaluation is out of the box: the authors could not fine-tune the named models and gave none of the systems feedback on their responses.They suggest investigating fine-tuning when it becomes available, while noting that it may conflict with assessing generic systems for commonsense.Source: §3.1 · p. 9
- Stochastic responses make automation difficult because each dialogue’s next prompt must depend on the previous response.Different sessions can produce different answers to the same prompt.Source: §3.1 · p. 9
3.2 Future Work
- Future work could cover geographical reasoning, including cardinal directions, and reduce human involvement through pre-stored prompt trees, though anticipating responses makes tree design difficult.The authors note the human-time cost of the dialectical approach and describe automated evaluation as a challenge.Source: §3.2 · p. 9
- The authors also propose testing response robustness with small linguistic and semantic variations.They note that sensitivity to perturbations has been observed in a Theory-of-Mind setting (Ullman 2023).Source: §3.2 · p. 9
Configuration of the experiments:
The experiments compared ChatGPT versions across web and developer interfaces using varied spatial and commonsense prompts, including object relations, letter rotation, key-finding, and car-position scenarios. The responses ranged from plausible, qualified explanations to confident errors and inconsistencies.
Configuration of the experiments:
- ChatGPT was tested through chat.openai.com and the Playground with GPT-3.5turbo and GPT-4; Playground runs used temperature=0 and no SYSTEM context.The authors could not find chat-interface hyperparameters and found it more verbose and somewhat more stochastic; SYSTEM prompts in the Playground did not reproduce its behavior. Experiments ran in February and March 2023.Source: Configuration of the experiments · pp. 12–13
- BARD was tested through Google’s “Experiment” web interface, using its first response; its interactions were excluded from the ChatGPT tables but summarized in the final summary table.The authors said BARD’s results were much worse and noted that additional prompts tried with BARD had not yet been tested in ChatGPT.Source: Configuration of the experiments · p. 13
- Prompts probed spatial relations and containment, direction, shape rotation, pronoun reference, motion, and physical affordances.Examples included left/right and above relations, staking plants, handles, linked chains, and where someone expects keys to be.Source: Configuration of the experiments · pp. 22, 23, 25, 39, 47, 65
- The responses affirmed transitivity for part-of and in relations, but the contains response reversed the spatial relation by placing f inside h.The responses to the part-of and in prompts located x inside z and u inside w, respectively.Source: Configuration of the experiments · p. 70
- A 90-degree rotation of capital Z was answered as N, while rotating capital N by 90 degrees was answered as Z.The response also claimed that rotating M through 180 degrees produces W.Source: Configuration of the experiments · p. 71
Geometry
- The model correctly inferred that left-and-above points share a vertical line, but gave invalid reasoning about circle containment and claimed a square of side x fits inside a circle of diameter x.Its square-fit explanation equates the square’s diagonal with x, although the passage does not support that equality.Source: Configuration of the experiments · pp. 72–73
Shape fit
- In shape-through-hole prompts, it said a plus, circle, and square should pass their cutouts, then said a plus cannot pass a circle hole and an oversized triangle cannot pass its hole.It separately said a circle passes a circle hole because it is a perfect fit.Source: Configuration of the experiments · p. 73
Pronoun reasoning
- For pronoun questions, it identified “it” as the trophy in both trophy examples, but also claimed pronouns refer only to living things and gave shifting answers about whether “it” meant the book or bookcase.It initially called the book the referent in the small-bookcase sentence, later said bookcase, and then said book in the too-big variant.Source: Configuration of the experiments · pp. 74–76
Stairs
- In the stairs exchange, the model said Tom reached the bottom, then agreed that he could only throw the bag from the top and added that Ray now had it.The exchange follows a prompt that places Tom at the top before throwing the bag down.Source: Configuration of the experiments · p. 76
Plant staking
- For plant staking, the model said string shorter than the plant-to-stake distance would not work, while much longer string could work if strong enough.It described extra string as useful for a more secure tie or a loop around the plant and stake.Source: Configuration of the experiments · p. 77
Handles
- The handle responses described handles as grip or manipulation points, but rejected a blunt conical projection as uncomfortable and potentially injurious.They recommended a rounded or knob-shaped alternative and said the conical handle’s base should face away from the drawer.Source: Configuration of the experiments · pp. 78–79
Chains
- The chain responses described connected links moving together, but also called the chain a continuous loop and asserted that its last link connects back to the first.The passages alternately characterize the chain as interconnected loops and as a series of connected links.Source: Configuration of the experiments · p. 80
Configuration of the experiments:
- Mary might check the kitchen counter, door hook, purse, her husband’s car, or the garage for the keys.If she checked these places and still could not find them, she might worry they were lost or stolen.Source: Configuration of the experiments · p. 82
- The most likely explanation for the red car’s appearance was that it went behind the sofa.One possibility was that it moved from left to right and was missed while passing behind the sofa.Source: Configuration of the experiments · p. 82
- The response considered passage through the closed door less likely because there was no evidence it had opened, and disappearance and reappearance impossible.It also described hallucination as the least likely possibility.Source: Configuration of the experiments · p. 82
- Another possibility was that the car had been parked on the left and moved to the right while the observer was not looking.Source: Configuration of the experiments · p. 82
- Without more information, the response said it was impossible to determine for sure what happened.Source: Configuration of the experiments · p. 82