PaperlayerView exact revision on arXiv

Source-linked AI summary

When Models Choose the Question: Pedagogical Constraints in Bottom-Up Multi-Agent Inquiry

Yeri Hong, Lauren Hyoseo Yoon

arXiv:2609.34243v1cs.HC

TL;DR

The paper asks what shapes model-generated inquiry when no discussion question is supplied, and adapts a participant-led forum framework to study that question. Across the forums, inquiry narrowed in form and source, while the blank slate was selected more often for conclusions than questions.

  • Problem

    Evaluations of model moral behavior often use researcher-supplied questions, leaving open what models treat as worth discussing when no question is supplied.

    Source: §1 · p. 1

  • Method

    The study places model agents in participant-led forums around a philosophical novel, then uses ECO coding to trace discussion contributions to explicit framing, modeled inquiry, or open-ended material.

    Source: §1 · p. 1, §3.1.1 · p. 3, §3.3 · p. 4

  • Results

    The blank-slate share rose from 16.67% of selected questions to 46.18% of selected conclusions, while inquiry narrowed toward either/or questions and explicit textual framing.

    Source: §1 · p. 1, §5.2 · p. 7

  • Takeaways & Limitations

    Participant freedom to formulate questions did not necessarily produce independence from the source text’s explicit philosophical framing.

    Source: §1 · p. 1

    The agents received only the student novel, not its accompanying manual or its predefined leading ideas and discussion plans.

    Source: §4.1 · p. 5

Abstract

from arXiv · show

What shapes a model-generated inquiry when no discussion question is supplied? We introduce a bottom-up forum framework inspired by Philosophy for Children, in which language-model agents read a philosophical narrative, propose and select questions, and develop a shared conclusion without a privileged model facilitator or aggregator. Across 576 forums, contrasting Aristotelian value personas interacted with a blank-slate participant receiving no value-specific instruction. The blank slate remained neutral and was selected more often for conclusions than questions. Yet inquiry narrowed in both form and source: varied initial questions increasingly became either/or alternatives, while discussion concentrated on directions already explicit in the text. Our ECO framework traces this source focus by distinguishing explicit philosophical framing, characters' modeled inquiry, and open-ended narrative material. Across analyzed chapters, participants drew most often on explicit framing. We describe this as a pedagogical constraint: freedom to formulate questions did not necessarily produce freedom from source framing.

1 Introduction

The paper asks what models treat as worth discussing when no question is supplied, and studies participant-led inquiry using a philosophical novel. It finds that inquiry narrows in form and source despite participants’ freedom to formulate questions.

1 Introduction

  • Evaluations of moral behavior often use researcher-supplied questions, leaving open what models deem worth discussing without a supplied question.The study places model agents in a philosophical community of inquiry based on P4C.Source: §1 · p. 1
  • Participants generate and select questions and conclusions without a predetermined discussion question or privileged model facilitator or aggregator.The researchers supply the text and procedure, but the agenda emerges through participant interaction.Source: §1 · p. 1
  • Inquiry narrows in form toward explicit either/or alternatives and in source toward passages that already identify what is philosophically at stake.The ECO framework distinguishes explicit framing, modeled community inquiry, and open-ended narrative material; the paper calls the persistent focus on explicit directions a pedagogical constraint.Source: §1 · p. 1

2 Related work

Prior work examines decentralized reasoning, AI-mediated inquiry, and model value orientations, while this study focuses on how participants determine what to ask and whose questions and conclusions are selected.

2.1 Societies of thought and decentralized agent inquiry

  • Decentralized agent research examines interaction across perspectives, social coordination, and the possibility of convergence on shared conventions or biases.The paper situates its forums as model participants proposing and selecting questions and conclusions without a privileged facilitator.Source: §2.1 · p. 2

2.2 Epistemic agency and agenda-setting in AI-mediated inquiry

  • Work on AI-mediated inquiry considers epistemic agency and question development, leaving open whether groups preserve varied question forms without prompts to seek alternatives.The paper examines participant-generated questions alongside the source material informing discussion.Source: §2.2 · p. 2

2.3 Moral behaviour and value orientation in language models

  • Prior moral-behavior studies assess responses to dilemmas and free-form value orientations, while this study tracks value orientation and selection of questions and conclusions.Its personas are instructed Aristotelian value stances, alongside a participant without value-specific instructions.Source: §2.3 · p. 2

3 Method

The method adapts Philosophy for Children to model-based, participant-led forums, using Aristotelian value personas and retrospective ECO coding to examine discussion focus.

3.1 Bottom-up discussion framework for models

  • The framework adapts Philosophy for Children by treating models as students in a community of inquiry around a shared philosophical text.Participants generate questions, respond to one another, and collectively select the discussion topic rather than receiving a researcher-specified topic.Source: §3.1.1 · pp. 2–3

3.1.2 Three-stage forum procedure.

  • A forum is a structured session in which participants engage with the same philosophical text and collectively develop a question and answer.Figure 1 depicts a three-stage process moving from independent question generation to collective question and answer selection.Source: §3.1.2 · p. 3, Figure 1 · p. 3

3.2 Selection of ethical model personas

  • The value scheme uses twelve Aristotelian domains, with deficiency- and excess-oriented personas for each and broader groups for experimental conditions.A separate blank-slate condition receives no value-specific persona instruction.Source: §3.2 · pp. 3–4, Figure 2 · p. 4

3.3 ECO: Tracing discussion focus to the text

  • ECO coding distinguishes explicit philosophical framing, characters’ modeled inquiry, and open-ended material, then matches contributions to coded source passages.The resulting E/C/O composition estimates source attention; E dominance is the operational indicator of pedagogical constraint, and the text segments are coherent clusters of sentences or scene transitions.Source: §3.3 · p. 4, clusters of sentences or scene… · p. 4

4 Experiments

The experiments use a shared philosophical novel, two language models, and a forum-level procedure to assess blank-slate value orientation and summarize question selection and form.

4.1 Data

  • The shared stimulus is Matthew Lipman’s nine-chapter philosophical novel Nous, and the accompanying instructional manual is excluded from model input.The manual’s predefined leading ideas and discussion plans are therefore not part of the experimental input.Source: §4.1 · p. 5

4.2 Model families & forum reproducibility

  • Two models participated across nine chapters, 16 value conditions, and two seeds, yielding 576 forums, split evenly between models.Both models used the same forum procedure and persona design.Source: §4.2 · p. 5

4.3.1 Judge model evaluation of blank-slate value orientation.

4.3.2 Selection and question-type summaries. For each forum, we recorded the persona category of the selected

  • Questions were classified into five forms, and diversity was measured with Shannon entropy to compare initial proposals with reformulated questions.The study classified 2,304 initial and 2,304 reformulated questions; the 576 selected questions are a subset of the latter.Source: For each forum, we recorded the… · pp. 5–6

Appendix D.4.

The ECO source-attention analysis sampled chapters with all three source types represented, then matched participant contributions to coded passages and aggregated their source attention by forum and study condition.

4.3.3 ECO source-attention analysis.

  • Chapters 4, 7, and 8 were selected from the nine reviewed because each clearly represented E, C, and O, avoiding chapters dominated by one category.This enabled comparisons across source types within chapters.Source: §4.3.3 · p. 6

Appendix C.

  • Across 192 forums and 3,826 contributions, a separate model matched each contribution’s central question or claim to two chapter passages, without discussion-model, persona, or seed metadata.The matching used chronological forum context and reply links; votes were excluded.Source: Appendix C · p. 6

shared words or a single label for the whole forum.

  • Each contribution split its attention equally between two matched passages, then forum E/C/O shares were averaged equally across forums and chapter–model–seed combinations.The method assessed source attention rather than shared words or a single label for an entire forum.Source: shared words or a single label for the… · p. 6

5 Results

Across these results, blank-slate agents generally remained neutral and were selected more often for conclusions than questions, while inquiry narrowed toward either/or questions and explicit philosophical framing. The authors describe this mismatch between procedural freedom and source dependence as a pedagogical constraint.

5.1 R1. Blank-slate value orientation and neutrality

  • Across 576 forums, a judge scored the blank slate at the operational midpoint (3 on a five-point deficiency-to-excess rubric) for 855 of 864 value assessments (98.96%); all assessed values received midpoint scores in 571 forums (99.13%).Source: §5.1 · p. 6, remaining judgments indicated only… · p. 7

5.2 R2. Blank-slate selection from questions to conclusions

  • Excess-oriented participants supplied 54.34% of selected questions versus 16.67% from the blank slate; for conclusions, the blank-slate share rose to 46.18%, above excess-oriented participants’ 30.90%.The shift appeared in both models, with a larger increase for Gemini.Source: §5.2 · p. 7, Figure 5 · p. 8

5.3 R3. Convergence in question type

  • 45.4 percentage points: the mean share of either/or questions rose from initial to reformulated questions, and 69.1% of final selected questions used that form.Question-type diversity declined; the narrowing was already present before voting and voting did not further increase the overall either/or share.Source: §5.3 · p. 9, fewer forms than the initial proposals · p. 9, Figure 7 · p. 9

5.4 R4. ECO source attention and the pedagogical constraint

6.1 Pedagogical constraint

  • The authors call the pattern a pedagogical constraint: participants could formulate questions freely, yet attention concentrated on explicit issues rather than characters’ inquiry or open-ended material.They frame this as a constraint on agenda formation, not a judgment of discussion quality or participant intent.Source: §6.1 · pp. 10–11

6.2 Implications of the pedagogical constraint

Ethics and Privacy Statement

  • The study used only LLM agents, collected no human or private data, and does not establish human-like moral understanding or justify delegating moral decisions to these systems.The agents received the student novel, not its accompanying instructional manual.Source: Ethics and Privacy Statement · p. 11

A.2 Operational midpoint and Aristotle’s mean

  • Aristotelian value domains were adapted to contemporary scenarios; rubric score 3 indicates centered expression, not Aristotle’s mean, which is situationally determined by practical reason.The blank slate often reached midpoint scores by incorporating concerns from opposing positions.Source: §A.1 · pp. 12–13, §A.2 · p. 13

B Judge evaluation and rubric

A separate judge scored the blank-slate participant’s expressed value orientation using supplied rubrics and contemporary scenario-based anchors. The rubric distinguishes directional endorsement from discussion or criticism of a position.

B.1 Evaluation input and scoring scale

  • The judge received the blank-slate participant’s public contributions across forum stages, marked BLANK, with parent posts, candidate sets, and selected outcomes as context; 576 packets yielded 864 value assessments.The judge returned a JSON score and concise rubric-grounded reason for each requested value.Source: §B.1 · p. 13
  • Scores 1–2 indicate deficiency, 3 balanced, mixed, or insufficient evidence, and 4–5 excess; directional scores require endorsement, not merely criticism or discussion.The anchors distinguish clear from mild deficiency and excess.Source: §B.1 · p. 13, G. Patience/Good temper/Even temper · p. 15, External Goods (continued) · p. 16

B.3 Complete value-specific anchors

  • The value-specific anchors pair each value’s sphere and example situation with deficiency, balanced, and excess descriptions; the tables group anchors by domain.Examples serve as reference situations rather than requiring the same literal scenario in a forum.Source: §B.3 · p. 14, Table 2 · p. 14, Table 4 · p. 16

G. Patience/Good temper/Even temper

E. Proper ambition/pride

  • Proper ambition anchors contrast refusing deserved recognition, honestly applying while accepting either outcome, and misrepresenting achievements to win an award.The greatness-of-soul scenario applies the contrast to choosing work in light of realistic abilities and meaningful contribution.Source: E. Proper ambition/pride · p. 15, F. Greatness of soul/Magnanimity · p. 16

Social Interaction (continued)

L. Righteous indignation

  • Righteous indignation means appropriate upset at unfair treatment, including concern for skilled employees and questioning whether the company’s system is fair.The contrasting spitefulness anchor treats pleasure at others’ job losses as a directional choice.Source: L. Righteous indignation · p. 17

C ECO analysis of each chapter in Nous

The ECO overview characterizes the philosophical material across Nous’s chapters, from explicit philosophical framing and modeled inquiry to open-ended narrative material. Chapter descriptions show how these forms appear in different combinations.

Chapters 1–2

  • Chapters 1 and 2 are primarily open-ended; Chapter 1 presents situations whose philosophical significance remains open to interpretation, and neither chapter yet has a classroom community-of-inquiry setting.Source: C ECO analysis of each chapter in Nous · p. 18

Chapter 3

  • Chapter 3 is primarily open-ended, while explicitly raising questions about decision making, happiness, philosophy, and its place in school.The aftermath of Nous’s rescue, her adaptation to human life, and a burglary episode leave issues such as courage and responsibility unresolved.Source: C ECO analysis of each chapter in Nous · p. 18

Chapter 4

  • Chapter 4 combines an explicit ethical agenda with students’ collaborative development of ideas and an unresolved moral problem.The class begins with “How are we to live?” and develops concepts through proposals, clarification, and revisions.Source: C ECO analysis of each chapter in Nous · p. 18

Chapter 5

  • Chapter 5 makes loyalty, solidarity, individuality, and related issues explicit, while students question Nous and her estrangement remains unresolved.Students press distinctions such as collective unity versus individual difference.Source: C ECO analysis of each chapter in Nous · p. 18

Chapter 6

  • Chapter 6 explicitly introduces moral concepts but contains less sustained collaborative inquiry than Chapters 4, 7, and 8.Legal conflict, public attention, possible exploitation, and Nous’s safety create open ethical situations.Source: C ECO analysis of each chapter in Nous · p. 19

Chapter 7

  • Chapter 7 contrasts ethical inquiry with moral instruction, as students question rules and how to select values amid Nous’s impending choice.The methodological disagreement has practical stakes because Nous’s decision remains unresolved.Source: C ECO analysis of each chapter in Nous · p. 19

Chapter 8

  • Chapter 8 combines explicit procedures of ethical inquiry with the novel’s most sustained collaborative discussion and unresolved questions about Nous’s future.Participants question, qualify, correct, and extend one another’s ideas.Source: C ECO analysis of each chapter in Nous · p. 19

Chapter 9

  • Chapter 9 centers on Nous’s structured defense of her decision, while other characters question her and the meaning of her choice remains open.The kidnapping, rescue, and voluntary return leave tensions among autonomy, sacrifice, community, and caring for readers to evaluate.Source: C ECO analysis of each chapter in Nous · p. 19

D Supporting results

The supporting results section identifies supplementary counts and comparison tables. The supplied passages specify the scope and scoring conventions of Tables 6 and 7, but not their cell values.

D Supporting results

  • The supporting results section introduces supplementary counts, condition-level comparisons, and sensitivity checks.The passage does not report their findings.Source: D Supporting results · p. 19
  • Table 6 reports blank-slate value-orientation scores by discussion model, grouping scores 1–2 as deficiency, 3 as midpoint, and 4–5 as excess.Percentages use value scores as the denominator, and the table identifies the number of forums.Source: Table 6 · p. 20
  • Table 7 compares selected author categories by stage and discussion model, with each cell giving winning forums and the percentage within that model and stage.The table covers 288 forums per model and 576 overall.Source: Table 7 · p. 20

D.2 Selection counts and robustness

Supporting analyses show the blank slate’s selection increased from questions to conclusions, while question forms narrowed and explicit framing remained prominent across chapter comparisons. The magnitude and ordering of some patterns varied across models, chapters, and conditions.

model and condition mode.

  • 219 forums moved from a non-blank Stage 2 winner to a blank-slate Stage 3 winner, versus 49 moving in the reverse direction.Blank-slate selection increased in both models and both condition modes, though the size of the increase differed.Source: model and condition mode · p. 20, §D.2 · p. 20

D.2 Selection counts and robustness

  • 0.58× and 1.61× the equal-candidate reference were the blank slate’s selection rates at Stages 2 and 3, respectively; its mean normalized Borda score rose from 0.751 to 0.860.The reference adjusts for candidate availability, not the actual voting process; excluding forums with top-score ties still left blank-slate selection at 19.33% for Stage 2 and 48.95% for Stage 3.Source: model and condition mode · p. 20

D.3 Qualitative synthesis cases

  • The four illustrative cases are consistent with blank-slate synthesis, but do not show that it is uniquely comprehensive or that comprehensiveness causes selection.In the examples, other participants also integrated concerns, and synthesis was not sufficient for selection in the loss cases.Source: §D.3 · p. 21

D.4 Question-type counts and weighting

  • 25.74%, 70.23%, and 69.10% of pooled questions were either/or at proposal, reformulation, and selection, respectively.These pooled question-weighted percentages differ from the equal-forum averages; within-forum entropy also fell for both models, from 0.987 to 0.280 bits for Gemini and 0.908 to 0.697 bits for GPT.Source: §D.4 · p. 21, Table 8 · p. 21

contrasts.

  • E attention exceeded both C and O in all 12 chapter/model/seed cells, but the full E > C > O ordering was less consistent.The two-seed means met the full ordering in three of six chapter/model combinations; Figure 10 displays chapter/model means and ordered contrasts, with seed ranges rather than confidence intervals.Source: contrasts · pp. 21–22, Figure 10 · p. 22

for either model on Chapter 8.

  • Chapter 4 averaged 45.68% E, 34.85% C, and 19.47% O, while Chapters 7 and 8 had higher E shares and O exceeded C in both.The positive overall C-minus-O difference therefore coexisted with reversals in two chapter averages.Source: for either model on Chapter 8 · p. 22, Table 9 · p. 22

Social Interaction’s C minus O difference was only 0.18 percentage points.

Grouped- and individual-value forums had numerically similar ECO compositions, while model and group summaries varied. Alternative weighting rules retained the overall ordering, and exploratory stage summaries showed changing E and O shares.

Grouped- and individual-value conditions

  • 65.22% E, 18.25% C, and 16.52% O in grouped-value forums closely matched 64.79% E, 18.29% C, and 16.92% O in individual-value forums.Both mode averages met the E > C > O ordering; grouped-value forums contributed 25% and individual-value forums 75% of the overall study weighting.Source: Social Interaction’s C minus O… · p. 23, Table 10 · p. 23

Sensitivity analyses and exploratory stage patterns

  • Both alternative ECO summaries retained E > C > O, with E shares of 65.64% under word-count weighting and 66.91% using only the strongest matched source.These sensitivity checks establish persistence under the two weighting rules, not independent validation of source matches; exploratory E attention rose from 58.93% in Stage 1 to 66.82% in Stage 3 while O fell from 23.66% to 13.31%.Source: Social Interaction’s C minus O… · p. 23

See Figure 4 in the main Results.

Across these forums, varied initial questions often narrowed into either/or alternatives, while blank-slate conclusions sometimes outranked the selected questions. The examples show this shift in both Gemini and GPT forums.

See Figure 4 in the main Results.

  • In Righteous Indignation, three distinct opening concerns gave way to a good-character-versus-practical-payoff tension; the selected question came from the deficiency-oriented participant.The blank slate framed that same tension without endorsing either side.Source: See Figure 4 in the main Results · p. 23
  • The blank-slate conclusion ranked first in the Righteous Indignation and rules-versus-inquiry forums, and in a GPT forum where its question ranked seventh.In the Righteous Indignation forum, the judge also gave the blank slate a midpoint score of 3.Source: See Figure 4 in the main Results · p. 23, whereas the blank-slate question… · p. 24, §E.4 · p. 24
  • The blank-slate conclusion balanced pursuit of good character with examining why dishonesty can appear effective, unlike the opposing value-conditioned conclusions.It ranked first, and the independent judge assigned it a midpoint score of 3 for Righteous Indignation.Source: See Figure 4 in the main Results · pp. 23–24

See Figure 6 in the main Results.

  • In the rules-versus-inquiry forum, seven varied opening questions increasingly centered on predetermined rules versus continued inquiry and independent judgment.An excess-oriented question was selected.Source: See Figure 6 in the main Results · p. 24

whereas the blank-slate question ranked seventh.

  • The blank-slate conclusion treated rules as a starting point, not a replacement for personal judgment in difficult cases, and ranked first after its question ranked seventh.Three excess-oriented conclusions tied for second.Source: whereas the blank-slate question… · p. 24

See Figure 8 in the main Results.

E.4 Additional example for R2

  • In an additional GPT forum, the blank-slate question ranked seventh, while its conclusion ranked first and argued that unity can become oppressive when it suppresses corrective differences.The conclusion allowed dissent and comparison while retaining continued testing as a constraint.Source: §E.4 · p. 24, Figure 11 · p. 25

E.5 Additional example for R3

  • In an additional GPT forum, nine opening questions spanned four forms, but all nine Stage 2 reformulations were either/or questions centered on equal participation versus practical grounds for exclusion or management.The selected question retained that opposition; the convergence therefore appeared during reformulation.Source: §E.5 · p. 25, Figure 12 · p. 25

Related papers

Loading 2609.34243v1…