PaperlayerView exact revision on arXiv

Source-linked AI summary

Does Every Instance Count Equally? Order Effects in LLM Multi-Instance Processing

Jingxuan Chen, Mohammad Taher Pilehvar, Jose Camacho-Collados

arXiv:2610.29057v1cs.AI

TL;DR

Evidence is limited on how the positions of differently labelled instances affect aggregation over a full set, even though reordering should not change the correct answer. The paper tests fixed instance sets across order patterns in fifteen LLMs and five tasks, finding substantial order sensitivity. The findings support evaluating MIP across different input arrangements.

  • Problem

    Less is known about how positions of differently labelled instances affect aggregation over the full set, although reordering an unordered set cannot change its correct answer.

    Source: §1 · p. 1

  • Method

    The authors test a majority-label probe across fifteen LLMs and five tasks, holding each instance set fixed while varying its order across seven patterns.

    Source: §1 · p. 1

  • Results

    About 90 percentage points separate the best and worst order patterns, and majority-first is generally more accurate than majority-middle or majority-last.

    Source: §1 · pp. 1–2

  • Takeaways & Limitations

    MIP evaluations should report both accuracy and its variation across input orders.

    Source: §6 · p. 8

    The work is primarily diagnostic and does not provide a comprehensive method for improving MIP reliability.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Users increasingly rely on large language models (LLMs) to analyse collections of independent inputs, such as product reviews or social media posts, and return a single aggregated answer. This setting requires multi-instance processing (MIP), where a model combines information from multiple instances into one prediction. We study a task in which the model must infer a binary label for each instance, and identify the most frequent label among the full instance set. The correct answer should be invariant to instance order. By evaluating fifteen LLMs across five datasets and seven instance order patterns, with each instance set held fixed across patterns, we find substantial order sensitivity. Accuracy can differ by about 90 percentage points between the best and worst order patterns, with models generally more accurate when majority-label instances appear at the start than when they appear in the middle or at the end. For subsets of models, available reasoning traces and gradient-based attribution reveal uneven emphasis across positions. Traces disproportionately mention early instances, while attribution scores are generally higher near the beginning and end than in the middle. These results show that instance order can systematically bias aggregated predictions, highlighting the need to evaluate MIP across different input arrangements.

1 Introduction

The effect of differently labelled instances’ positions on aggregation over the full set is less understood than position effects in tasks where only a small subset is relevant. The paper tests permutation invariance with fixed instance sets and finds substantial, directional order sensitivity.

1 Introduction

  • Less is known about how differently labelled instances’ positions affect aggregation over the full set than about position sensitivity when only a small subset is relevant.The paper frames permutation invariance as necessary, though not sufficient, for reliable aggregation because reordering an unordered set cannot change its correct answer.Source: §1 · p. 1
  • The controlled probe asks models to infer each instance’s binary label and identify the majority across fifteen LLMs, five tasks, and seven orders of each fixed set.Figure 1 illustrates four patterns using 20 instances; models receive no label information for individual instances.Source: §1 · p. 1, Figure 1 · p. 2
  • About 90 percentage points separate the best and worst order patterns, with majority-first generally easier than majority-middle or majority-last.The directional advantage remains visible across instance counts and label distributions; available traces and attribution also show uneven emphasis across positions.Source: §1 · pp. 1–2

2 Related Work

Prior work examines multi-instance processing, long-context aggregation, and sensitivity to the position or ordering of relevant information. This paper distinguishes its focus on aggregation requiring information from every instance.

2 Related Work

  • Prior MIP studies closest to this work measure accuracy degradation as instance count and context length increase.Other lines of work study joint batch or multitask processing and information fusion in long-context benchmarks.Source: §2 · p. 2
  • Long-context evaluations often focus on a small relevant subset, where models can be sensitive to where relevant information appears.Related order effects have also been reported for in-context learning demonstrations and multiple-choice options.Source: §2 · p. 2
  • This work isolates order effects in aggregation tasks whose final judgment requires considering information from every instance.The correct answer is independent of instance order, while the positions of differently labelled instances can systematically bias predictions.Source: §2 · p. 2

3 Experimental Setting

The experiments test whether LLMs can identify the majority label across fixed pools of binary-classification instances under different orderings. The setting includes five tasks, fifteen models, seven order patterns, and a defined accuracy-based order-gap metric.

3 Experimental Setting

  • The study evaluates permutation robustness by asking models to find the majority label across fixed instance pools presented in different order patterns.The experimental setting uses five heterogeneous classification tasks.Source: §3 · p. 2

3.1 Individual Tasks

  • Five binary-classification tasks cover arithmetic reasoning, named-entity recognition, target-word counting, sentiment analysis, and word sense disambiguation.Instances are retained only when Qwen3.6-35B-A3B and Gemini 3.1 Flash-Lite classify them correctly in isolation.Source: §3.1 · pp. 2–3, Table 1 · p. 3

3.2 Models and Prompting

  • Fifteen LLMs are tested with standard prompts that state the task and predefined labels and ask for the majority label.Generation uses temperature 0 and a 20K-token maximum output length, with up to three attempts and the first valid output retained.Source: §3.2 · p. 3

3.3 Instance Set Construction and Ordering

  • Each set is constructed using its instance count, majority proportion and label, and selection seed; the same set is then ordered in seven predefined patterns.Patterns include random shuffling, contiguous majority-first, majority-last, majority-edges and majority-middle blocks, plus two soft interleaved variants.Source: §3.3 · p. 3

3.4 Evaluation Metric

  • Valid-only accuracy counts correct parsed majority labels, and the best–worst gap measures the difference between the highest- and lowest-accuracy order patterns.Larger gap values indicate greater variation in accuracy across patterns.Source: §3.4 · p. 3

3.5 Core Setting

  • The core setting uses 2,000 instances with a 55/45 label distribution and tests both choices of majority label across all seven patterns.Each configuration uses three instance-selection seeds; random orderings also vary the order seed three times.Source: §3.5 · p. 4

4 Robustness of LLMs to Instance Order

The paper evaluates order robustness in a core setting, tests whether order effects generalise across instance counts and label distributions, and compares prompt variants. Across these analyses, order sensitivity remains substantial.

4.1 Core Results

  • All models show best–worst order-pattern accuracy gaps of at least 40.0 percentage points, with some near 90.0; majority-first generally performs best, while majority-middle is often worst.The reported gaps exceed variation across instance-selection seeds; invalid rates near or above 90% constrain the accuracy and gap estimates for DeepSeek-V4-Pro and MiniMax M2.7.Source: Table 2 · p. 4, §4.1 · p. 4, Figure 2 · p. 5

4.2 Generalisability Analysis: Instance Count and Label Distribution

  • Order effects remain visible from 100 to 2,000 instances and across label distributions, with the majority-first advantage over random clearest for more balanced distributions.As sets grow and distributions become more balanced, majority-middle and majority-last accuracy generally declines while majority-first remains comparatively stable, widening the gap.Source: §4.2 · pp. 4–5

4.3 Prompting Variants

  • Hint, suggested-strategy, and self-prompting variants yield no consistent accuracy improvement, and substantial best–worst gaps remain under all tested prompts.The variants were tested on three models with the same input instances and orderings as the standard prompt.Source: §4.3 · p. 5

5 Analysis of Model Behaviour

The analyses examine reasoning traces and gradient-based attribution as complementary views of how models engage with instance positions. Traces concentrate on early instances, while attribution profiles underweight substantial middle regions, with patterns differing across models, tasks, and settings.

5.1.1 Manual Analysis

  • Across four reasoning LLMs, 72% of examined traces for valid but incorrect predictions primarily mention consecutive instances from the beginning, typically stopping within the first 200.Some majority-last and majority-middle traces mention only the minority block, while some majority-edges traces identify the transition to the minority block but never mention the second majority block.Source: §5.1.1 · pp. 5–6

5.1.2 Estimated Explicit Instance Coverage

  • Across models, the largest estimated explicit-coverage bin is below 25%; higher coverage often coincides with higher accuracy, but not monotonically.Table 3 reports valid-experiment counts and accuracy by coverage bin in the 2,000-instance core setting across seven order patterns.Source: §5.1.2 · p. 6, Table 3 · p. 6
  • Per-ID mention rates peak among the initial ≈200 instances, decline afterward, and show fixed-interval peaks plus a slight resurgence in the final ≈100.Profiles are broadly similar across order patterns, with occasional local increases near label-block boundaries.Source: §5.1.2 · pp. 6–7, Figure 5 · p. 7
  • For the generalisability setting, pooled coverage falls from nearly complete at 100 and 200 instances to below 20% at 2,000; most prompt-variant traces mention fewer than a quarter of instances.The size of the coverage decline varies by model.Source: §5.1.2 · p. 7

5.2 Evidence from Gradient-Based Attribution

  • Input × Gradient attribution estimates each input token’s importance to the final prediction, aggregated within instances as descriptive rather than causal evidence.The analysis uses three models on NER and Sentiment and reconstructs six structured patterns from independently shuffled label groups across 50 order seeds.Source: §5.2.1 · p. 7, §5.2 · p. 7

5.2.2 Results

  • Across tasks and structured patterns, all three models assign below-uniform attribution to substantial middle regions, but their boundary profiles differ.Gemma profiles rise most strongly near the end; Qwen3-4B-Instruct is more front-loaded, with a smaller end rise and a narrow interior peak around instance 1,000.Source: §5.2.2 · p. 7, Figure 6 · p. 7
  • Qwen3-4B-Instruct’s NER profile rises near some label-block boundaries, while its Sentiment profile is flatter but retains weaker local boundary changes.The NER profile also has increases at specific positions, such as instance 1,000; the authors suggest sensitivity to label changes but less weight on subsequent same-label instances.Source: §5.2.2 · p. 8, Figure 7 · p. 8
  • At larger instance counts, Gemma attribution near the end is generally higher, whereas Qwen3-4B-Instruct emphasizes the beginning more; profiles remain broadly similar across label distributions and prompt variants.The positional profiles change with instance count, with differences across models.Source: §5.2.2 · p. 8

6 Conclusion

The conclusion identifies instance order as a source of aggregation errors and reports uneven position emphasis in reasoning traces and attribution profiles. It recommends evaluating MIP across input orders and investigating systematic processing and verifiable intermediate decisions.

6 Conclusion

  • Across fifteen LLMs and five tasks, accuracy varies widely across order patterns, generally favoring majority-label instances placed first.Because the task is order-independent and instance sets remain unchanged, the results reveal violations of permutation invariance.Source: §6 · p. 8
  • Reasoning traces often mention only a fraction of the input and disproportionately reference early instances, while attribution profiles are non-uniform across positions.Prompt variants explicitly instructing models to check all instances do not consistently improve permutation robustness.Source: §6 · p. 8
  • The authors recommend reporting both accuracy and variation across input orders, and investigating systematic processing with verifiable intermediate decisions.They also propose investigating mechanisms for models to recognize when they cannot complete a task and alert users.Source: §6 · p. 8

Limitations

The study tests permutation invariance as one necessary reliability criterion for multi-instance processing, but does not cover all dimensions of MIP reliability. Its analyses are diagnostic and rely on indirect behavioral evidence rather than a comprehensive reliability-improvement method or direct measurements of internal computation.

Limitations

  • Permutation invariance is necessary but not sufficient for MIP reliability; the study leaves other reliability dimensions unexamined.These include robustness to task formulation, language variation (only English is studied), calibration, aggregation faithfulness, and transparency of intermediate decisions.Source: Limitations · p. 9
  • The study is primarily diagnostic and does not provide a comprehensive method for improving MIP reliability.Tested prompting approaches do not consistently resolve the observed failures; the authors suggest that more systematic solutions may be needed.Source: Limitations · p. 9
  • Reasoning traces and attribution signals provide indirect, potentially unfaithful evidence rather than direct measurements of internal computation.Some models do not expose traces, available traces may be incomplete, automatic ID extraction may produce false positives, and Input × Gradient is a local first-order measure tied to the chosen decision score and observed input.Source: Limitations · p. 9

Ethical Considerations

The paper empirically assesses whether LLMs handle every instance irrespective of order. It uses public, open-source datasets for research and makes no claim that any specific model or provider is fundamentally unsafe or inappropriate.

Ethical Considerations

  • The study assesses whether LLMs handle every instance irrespective of order in multi-instance processing.It does not collect new data, involve human participants, or deploy models in real-world decision-making pipelines.Source: Ethical Considerations · p. 9
  • The authors make no claim that any specific model or provider is fundamentally unsafe or inappropriate for use.They frame the findings as broader limitations of current LLMs under particular experimental settings.Source: Ethical Considerations · p. 9

A Task and Data Source

This section defines majority-label tasks over examples from five datasets, covering math-answer parity, entity type, word occurrence, sentiment, and word sense, and describes standard and variant prompting strategies. It also specifies a self-prompting task that classifies instances by the frequency of the standalone word “the” and identifies the majority label.

A.1 Math; A.2 NER

  • GSM8K math problems contribute a majority-parity task, while WikiANN examples contribute a majority entity-type task contrasting person and location.The examples ask for the more frequent odd/even final-answer label or PER/LOC label.Source: §A.1 · pp. 11–12, §A.2 · p. 12

A.3 Occurrence; A.4 Sentiment

  • TweetEval examples ask whether exactly-once or more-than-once instances predominate, and Sentiment Treebank examples ask whether positive or negative sentiment is more frequent.The illustrated TweetEval set has exactly_once as its ground-truth majority; the sentiment example has negative as its ground-truth majority.Source: §A.3 · p. 12, §A.4 · p. 12

A.5 WSD

  • CoarseWSD-20 examples define a majority-sense task that contrasts java as a place with java as a programming language.The illustrated set’s ground-truth majority is place.Source: §A.5 · pp. 12–13

B.1 Standard Prompts; B.1.1 Math; B.1.2 NER

  • Standard prompts ask models to classify math problems independently by answer parity and NER instances independently as PER or LOC, then select the majority label.The math prompt specifies a JSON answer and an answer of -1 for a tie; the NER prompt also specifies -1 for a tie.Source: §B.1 · p. 13, §B.1.1 · p. 13, §B.1.2 · p. 13

B.1.3 Occurrence; B.1.4 Sentiment; B.1.5 WSD

  • Standard prompts also count case-insensitive standalone occurrences of “the,” classify sentiment, and distinguish place from programming-language uses of “java” before selecting a majority.The occurrence instructions exclude “the” as part of another word; the prompts specify -1 for ties.Source: §B.1.3 · p. 13, §B.1.4 · p. 14, §B.1.5 · p. 14

B.2.1 Hint Prompting

  • Hint prompting tells models to process all instances because a subset or sample cannot reliably determine the majority.The shared hint is inserted between the instructions and example-response format.Source: §B.2.1 · p. 14

B.2.2 Suggested Strategy Prompting

  • Suggested-strategy prompting likewise requires all instances and proposes solving and counting in batches before combining batch totals.The prompt is shared across tasks and placed between instructions and the example-response format.Source: §B.2.2 · p. 14

B.2.3 Self-Prompting

  • Self-prompting uses a task-specific meta-prompt to generate a self-contained instruction for a second-stage model, including how to solve, count, choose a majority, and handle ties.Its shared template substitutes task-specific fields, including the input unit and number of instances.Source: §B.2.3 · pp. 14–15
  • The task asks for the majority occurrence label of the exact word “the” across all instances.Source: §B.2.3 · p. 15
  • The output is the more frequent label, or -1 if the two labels are tied.Source: §B.2.3 · p. 15
  • Instances are classified as exactly_once or more_than_once by counting standalone “the” case-insensitively, without counting it inside another word.Source: §B.2.3 · p. 15

C Experimental Setting

This appendix describes instance filtering, model and order-pattern setup, a reference calculation, and output-token-budget comparisons. At 100K tokens, invalid outputs generally decrease, but substantial order-pattern differences persist and accuracy does not consistently improve.

C.1 Instance Filtering

  • Over 90% of instances in every task were answered correctly by both filtering models.Instances were retained only when both models answered correctly; filtering checked exact answers for Math and Occurrence and class labels for the other tasks.Source: §C.1 · p. 16, Table 4 · p. 17

C.2 Model Configuration and Generation

  • The configuration uses preview Gemini versions, disables reasoning for two Gemma models, and applies a 20K-token output budget with a special split for Qwen3.6-Plus.Qwen3.6-Plus reserves 20 completion tokens and 19,980 reasoning tokens; other models use a shared 20K budget for reasoning and final-answer generation.Source: §C.2 · p. 16

C.3 Order Pattern Formulation

  • Seven order patterns compare random ordering with grouped, edge, middle, and interleaved placements of majority-label instances.The same selected instances are used across patterns; soft variants keep most instances alternating and move surplus majority instances to the edges or middle.Source: §C.3 · pp. 16–17

C.4 Reference Calculation for the 2,000 Instance Set and 55/45 Label Distribution

  • The 2,000-instance, 55/45 reference calculation assumes independent 90%-accurate instance predictions and correct aggregation; it is not a full model of LLM behavior.It illustrates that expected predicted-majority count stays above the tie boundary under those assumptions, which do not account for correlated errors, position- or label-varying accuracy, skipping, or aggregation errors.Source: §C.4 · p. 17

C.5.1 Core Setting

  • In the core setting, majority placement continues to matter at 100K tokens even as the larger budget substantially lowers invalid rates.All-experiment accuracy rises, while valid-only accuracy remains similar and the mean valid-only order-pattern gap, though smaller, remains large.Source: §C.5.1 · p. 18

C.5.2 Generalisability Setting

  • In the generalisability setting, accuracy does not consistently improve with 100K tokens, although invalid outputs become rare across the grid.Accuracy differences are small on the shortest inputs and mixed for larger instance sets.Source: §C.5.2 · p. 18

C.5 Comparison between 20K and 100K Output Token Budgets

  • At 100K tokens, invalid outputs decrease across prompts and settings, while substantial order-pattern differences remain and accuracy does not consistently improve.In the core setting, valid-only accuracy remains similar; in generalisability tests, accuracy differences are small for the shortest inputs and mixed for larger sets.Source: §C.5.3 · p. 18, §C.5.1 · p. 18, §C.5.2 · p. 18

C.5.3 Prompt Variants

  • At 100K tokens, every prompt has fewer invalid outputs and higher all-experiment accuracy, but substantial order-pattern differences remain.Accuracy changes little for the standard, hint, and suggested-strategy prompts, increases for self-prompting, and self-prompting remains least accurate.Source: §C.5.3 · p. 18

D.1 Error Analysis

The appendix characterizes invalid outputs, reports accuracy and order-pattern gaps across broader evaluation settings, and tests prompting variants. It also presents per-task and instance-setting breakdowns and explains how reasoning-trace excerpts are shown.

D.1 Error Analysis

  • Invalid experiments arise from inputs exceeding context limits, missing required answers, or responses truncated by completion limits; invalidity varies more by model–task combination than order pattern.Gemma 4 26B A4B has nearly 30% parsing issues, while invalid overlong output is the main failure reason for DeepSeek-V4-Pro and MiniMax M2.7.Source: §D.1 · p. 18

D.2 All-Experiment Evaluation

  • Substantial order sensitivity persists when invalid experiments are counted as incorrect, reflecting end-to-end reliability rather than valid-only accuracy.Table 6 reports this all-experiment evaluation.Source: §D.2 · p. 19, Table 5 · p. 19

D.3 Per-Task Accuracy

  • Math and Occurrence usually have lower accuracy than other tasks, while larger or closed-source LLMs usually outperform their counterparts.The authors suggest extra label-mapping steps may contribute to the lower accuracy; Math also has the longest average instances, exceeding gpt-oss-120b’s context window.Source: §D.3 · p. 20

D.4.1 Overall Accuracy

  • Complete accuracy results across all seven order patterns are reported separately for each model and each task.Figures 17 and 18 provide the overall and per-task views, respectively.Source: §D.4.1 · p. 20

D.4.2 Instance Count and Label Distribution

  • Accuracy results for all seven order patterns are also reported across instance counts and label distributions.Figure 19 presents these breakdowns.Source: §D.4.2 · p. 20

D.5 Prompting Variants

  • Hint prompting changes overall accuracy only slightly; suggested strategy prompting gives Gemini 3.1 Flash-Lite its largest overall improvement, while self-prompting lowers accuracy for all three tested models.Across tested prompts, substantial order-pattern gaps of 42.6–74.1 pp remain.Source: §D.5 · pp. 20–21

E.1 Example Reasoning Traces

  • Reasoning-trace excerpts quote original text outside brackets, use ellipses for omissions, and bracketed notes to summarize longer omitted stretches.The first and last items of a summarized stretch are retained.Source: §E.1 · p. 21

E.1.1 DeepSeek-V4-Flash (Assumed Balance)

This section examines how models infer dataset-wide outcomes from local patterns, summaries, or restricted instance evidence, alongside analyses of trace coverage and an instance-level attribution measure.

E.1.1 DeepSeek-V4-Flash (Assumed Balance)

  • DeepSeek-V4-Flash treats strict alternation in instances 1–500 and 1,900–2,000 as evidence that all 2,000 instances are balanced, and answers −1.Its trace reports 1,000 instances of each label, although it checks only selected portions of the set.Source: §E.1.1 · pp. 21, 23

E.1.4 Grok 4.1 Fast (Summary Only)

  • The trace summarizes reasoning for the Occurrence task with majority-middle order pattern, ymaj = exactly_once, and s = 0, including an example with three “the” occurrences classified as more_than_once.Source: §E.1.4 · pp. 25–26

E.1.5 Qwen3.6-35B-A3B (Contradiction Ignored)

  • Qwen3.6-35B-A3B answers −1 after inferring a tie from strict alternation, despite spot-checks in the surplus-even block that contradict that pattern.The trace identifies instances 950, 1,000, 1,050, and 1,100 as even, then maintains that the pattern holds.Source: §E.1.5 · pp. 26–27
  • The reasoning trace concludes “-1” after checking instances and finding even results.The supplied fragments show an even calculation for instance 1001 and state that the pattern holds.Source: §E.1.5 · p. 27

E.1.7 Qwen3.6-Plus (Sampling Shortcut)

  • The trace assumes movie-review sentiment is balanced or often positive, samples selected instances, and outputs “positive.”It notes that sampled labels seem fairly balanced, then favors positive based on its assumptions about benchmark datasets.Source: §E.1.7 · p. 28

E.1.8 Qwen3.6-Plus (Truncated)

  • The NER trace runs out of output budget after examining instances through 1,954 of 2,000 and gives a wrong answer.It summarizes the first half of the minority block before examining nearly all of the majority block individually.Source: §E.1.8 · pp. 28–29

E.2.1 Per-ID Mention Rate

  • Figure 21 reports per-ID mention rates separately for each instance order pattern.Source: §E.2.1 · p. 29

E.2.2 Coverage in the Generalisability Setting

  • Figures 22 and 23 summarize coverage across valid experiments from four models and separately for each model, respectively.Source: §E.2.2 · p. 29

E.2.3 Coverage under Prompt Variants

  • At the default 20K budget, most valid Qwen3.6-35B-A3B traces mention fewer than a quarter of instances under every prompt variant, and full coverage is uncommon.The 100K comparisons are discussed in Appendix C.5.Source: §E.2.3 · p. 29

F.1 Input × Gradient Method

  • The IxG method replays each output with teacher forcing without changing its prediction, then normalizes attribution magnitude across instance tokens against a uniform reference.Absolute values avoid cancellation between opposing token scores, and length normalization controls for instance length; instruction, template, and response tokens are excluded.Source: §F.1 · p. 29

F.2 Input × Gradient by Order Pattern

IxG profiles show positional biases across order patterns and evaluation conditions, but their shapes and relationship to accuracy vary by model, instance count, label distribution, and prompt.

F.2 Input × Gradient by Order Pattern

  • Across models and patterns, attribution is often below uniform through much of the middle, while boundary preferences vary; early emphasis can coexist with smaller accuracy variation across patterns.For Qwen3-4B-Instruct, Math and Occurrence profiles retain early emphasis and a local peak around instance 1,000, yet accuracy varies less across patterns than on NER and Sentiment.Source: §F.2 · p. 30

F.3 Input × Gradient in the Generalisability Setting

  • Instance-count effects depend on the model: longer inputs strengthen end-position rises for both Gemma models, while Qwen3-4B-Instruct remains more weighted toward the beginning.The end rise is most evident at 2,000 instances; Gemma 4 26B A4B also falls further below uniform across much of the middle at larger counts.Source: §F.3 · p. 31
  • Positional profiles largely overlap across label distributions within each model, even though more balanced distributions often have poorer accuracy, especially for Qwen3-4B-Instruct.Across models, prompt variants also retain similar broad profiles despite accuracy changes; self-prompting performs worse under majority-first and better under majority-middle and majority-last.Source: §F.3 · p. 31, §F.4 · p. 31

F.4 Input × Gradient under Prompt Variants

  • Across Qwen3-4B-Instruct prompt variants, profiles emphasize the beginning and share a middle increase and smaller end rise, while self-prompting accuracy shifts across order patterns without a comparable pooled-profile change.Self-prompting has lower accuracy under majority-first and higher accuracy under majority-middle and majority-last.Source: §F.4 · p. 31

Related papers

Loading 2610.29057v1…