PaperlayerView exact revision on arXiv

Source-linked AI summary

The Missing Coefficients: Bayesian Pairwise Merging for Model Personalization

Yaling Shen, Tongtong Wu, Siyuan Yan, Gholamreza Haffari

arXiv:2609.39055v1cs.LG

TL;DR

Users may find pairwise output comparisons easier than specifying the numerical reward weights needed to merge shared experts. BPM infers a posterior over those weights and uses its mean as merge coefficients; across three tasks, it improves on uniform merging while characterizing uncertainty and preference following.

  • Problem

    Existing expert-merging methods assume preference weights are known, although users may find it easier to compare outputs than specify numerical trade-offs.

    Source: §1 · p. 1

  • Method

    BPM infers a posterior over reward weights from pairwise choices and reward-score differences, then uses its mean to merge shared reward-specialized experts without per-user policy training.

    Source: §1 · p. 1, §3.1 · p. 3

  • Results

    Across three generation tasks, BPM improves on uniform merging, simulated personas favor models fitted to their own feedback, and BPM coefficients vary less than MLE across limited-feedback subsets.

    Source: §1 · p. 2, §7 · p. 9

  • Takeaways & Limitations

    BPM provides personalized models from pairwise feedback while also quantifying plausible reward weights and coefficient ambiguity.

    Source: §7 · p. 9

    BPM’s coefficient-stability advantage does not imply higher output preference: MLE and adapted SS often yield more preferred outputs, while RM can be more stable but less preferred at both main-table budgets.

    Source: §6.1 · p. 8

Abstract

from arXiv · show

How can we personalize a shared expert library from a user's pairwise choices? Prior work can realize different reward trade-offs by merging reward-specialized experts, given a vector of trade-off weights. In practice, users can more naturally choose between outputs than specify numerical weights. The challenge is therefore to turn these choices into the coefficients required for merging, while accounting for ambiguity when feedback is limited. Our key idea is to treat the unknown reward weights as latent variables: infer a posterior over them from pairwise choices and reward-score differences, and use its mean directly as the merge coefficients. We instantiate this idea as Bayesian Pairwise Merging (BPM), whose posterior also characterizes which reward trade-offs remain plausible given the feedback. We evaluate BPM on radiology summarization, image captioning, and story generation, spanning text-to-text and image-to-text generation. With 100 feedback per simulated persona, BPM achieves macro decided win rates of 91.7%, 77.1%, and 64.3% against uniform merge. For six pairs of simulated personas, each prefers the model fitted to its own feedback, a pattern also observed in a human proof-of-concept. In simulations under BPM's model and prior, its nominal 90% intervals for temperature-scaled reward weights achieve task-averaged marginal coverage of 88.9% and 89.2% with only 10 and 25 comparisons, respectively. BPM thus enables personalization from pairwise feedback without per-user policy training, while characterizing the coefficient ambiguity left by limited feedback.

1 INTRODUCTION

BPM addresses the gap between users’ pairwise choices and the numerical coefficients needed to personalize shared expert merges. It infers reward-weight uncertainty and uses the posterior mean to construct personalized models without per-user policy training.

1 INTRODUCTION

  • Existing merging methods assume preference weights are known, although users may find comparing outputs easier than specifying numerical trade-offs.Uncertain reward-weight estimates do not necessarily produce the intended trade-offs in a merged model.Source: §1 · p. 1
  • BPM infers a posterior over user-specific reward weights and uses its mean to merge shared reward-specialized experts.The posterior represents support for different trade-offs and uncertainty remaining after limited feedback.Source: §1 · pp. 1–2
  • Across three generation tasks, BPM’s inferred coefficients improve on uniform merging, while simulated personas favor models fitted to their own feedback.Controlled feedback-preference changes also produce corresponding behavioral shifts.Source: §1 · p. 2
  • The contribution links preference differences to model behavior, beyond aggregate gains in model performance.The paper also reports less variation in BPM coefficients than MLE across limited-feedback subsets and near-nominal interval coverage under BPM’s model and prior.Source: §1 · p. 2

2 RELATED WORK

Prior work can implement specified trade-offs through optimization, policies, decoding, or merging, and can infer preferences from comparisons. BPM focuses on obtaining preference weights from an individual’s pairwise choices and using them to merge shared experts.

2 RELATED WORK

  • Preference-conditioned policies, weighted-reward optimization, decoding-time composition, and model merging implement trade-offs once a preference vector is given [Wu et al., 2023; Zhou et al., 2024; Yang et al., 2024; Zhong et al., 2024; Wang et al., 2024b;a; Shi et al., 2024; Rame et al., 2023; Xie et al., 2025; Chen & Kwok, 2025; Li et al., 2025].Multi-teacher on-policy distillation instead integrates specialized teachers through additional training.Source: §2 · p. 2
  • BPM infers reward weights from a fixed offline set of pairwise judgments, then uses them to merge shared experts without per-user policy training.This addresses how to obtain preference weights themselves, rather than how to realize trade-offs after weights are specified.Source: §2 · pp. 2–3
  • Pairwise-feedback methods estimate reward functions, reward weights, or preference distributions, while BPM uses predefined reward dimensions without other users’ preference data.Other approaches select from a fixed response pool or estimate policy-combination coefficients from online feedback [P˘adurean et al., 2025; Chow et al., 2026].Source: §2 · p. 2

3 BAYESIAN PAIRWISE MERGING

BPM infers reward weights from pairwise comparisons and uses their posterior mean as coefficients for merging a shared library of reward-specialized experts. The posterior also represents uncertainty, while optimality of the resulting merge depends on an additional correspondence assumption.

3.1 PROBLEM FORMULATION

  • Each task’s shared expert library comprises models fine-tuned on separate reward dimensions, and BPM combines their parameters using merge coefficients λ.The feedback contains pairwise choices and compared-output reward scores, not persona descriptions or explicit weights.Source: §3.1 · p. 3

3.2 BAYESIAN REWARD-WEIGHT INFERENCE

  • BPM standardizes reward scores, forms normalized reward contrasts, and models choices with a temperature-scaled Bradley–Terry likelihood over simplex-constrained reward weights.The shared Dirichlet prior gives equal expected weight to each reward dimension; posterior sampling uses NUTS and marginalizes over temperature.Source: §3.1 · p. 3, §3.2 · p. 4

3.3 POSTERIOR-GUIDED MODEL MERGING

  • BPM uses the posterior mean of reward weights as merge coefficients; without feedback, this returns the uniform merge, requiring only inference and one merge.The posterior mean lies in the simplex and minimizes posterior expected squared error in estimating the weights.Source: §3.3 · p. 4
  • Under the stated preference-model assumption and Proposition 2’s conditions, the posterior mean converges to the true weights; merge optimality additionally requires preference–merge correspondence.That correspondence assumption is not required to construct a personalized model.Source: §3.3 · p. 4
  • Feedback provides no direct likelihood information in directions orthogonal to the observed reward-contrast span, so comparison count alone does not determine posterior uncertainty.BPM summarizes uncertainty using the ratio of average posterior to prior standard deviations, with 1 as the no-feedback baseline.Source: §3.3 · p. 5

4 EXPERIMENT SETUP

The experiments evaluate shared reward-specialized expert libraries on three text and vision-language generation tasks. Simulated personas provide pairwise feedback, and the evaluation compares BPM with alternative personalization methods under a shared protocol.

4.1 TASKS AND MODEL LIBRARIES

  • The evaluation covers radiology summarization, image captioning, and story generation, using 29, 21, and 16 reward-specialized experts, respectively.Additional summarization libraries vary training seeds or use RLOO or DPO to assess robustness to library training choices.Source: §4.1 · p. 5

4.2 PERSONALIZATION AND EVALUATION PROTOCOL

  • Simulated personas provide preferences through descriptions or adjustable rubric weights, with BPM not given the rubric weights.Prompt-based judges provide pairwise feedback on disjoint test inputs; only valid, non-tied choices agreeing across presentation orders are retained.Source: §4.2 · p. 5
  • BPM is compared with expert weighting, response selection, prompting, policy training, the base policy, and uniform merging, using shared sampled feedback subsets where applicable.Evaluation uses 451 radiology reports, 500 images, and 500 story prompts; macro rates equally weight personas and average feedback draws.Source: §4.2 · p. 6

5 RESULTS

BPM improves on uniform merging across the three generation tasks and supports preference-specific model behavior. It also responds to controlled shifts in feedback preferences.

5.1 PERSONALIZATION PERFORMANCE

  • BPM improves on uniform merging across all three tasks at both feedback budgets.On story generation, the gains also persist with gpt-oss-20 as an alternative judge.Source: §5.1 · p. 6, Table 1 · p. 7
  • BPM outperforms ridge merge and preference-based reranking, while MLE and adapted SS often yield more preferred outputs.Comparisons with MAP vary by task and budget.Source: §5.1 · p. 6
  • In all three tasks, personalization yields gains over uniform merge with as few as 10 comparisons, and those gains generally increase as the feedback budget grows.Source: §5.1 · p. 7

5.2 PREFERENCE FOLLOWING

  • Each persona favored its own feedback-fitted model in all six simulated persona pairs; one author-annotator also showed preference reversals under two descriptions.The human proof-of-concept used Picture Book and Magazine Editor descriptions.Source: §5.2 · p. 7
  • As feedback shifts from Literary toward Genre, win rates against uniform decrease under the fixed Literary judge and increase overall under the fixed Genre judge.Rubric-based personas retain BPM’s macro advantage over compared methods on story generation.Source: §5.2 · p. 7

6 ANALYSIS

BPM’s posterior means yield less coefficient variation than MLE under limited feedback, but stability does not guarantee higher output preference. Its posterior also represents plausible trade-offs and coefficient uncertainty, with interval coverage evaluated under its assumed model and prior.

6.1 COEFFICIENT STABILITY AND OUTPUT PREFERENCE

  • At n ≤100, BPM has lower coefficient variation than MLE across all three tasks; its advantage over adapted SS holds across tasks at n = 10 but varies at larger budgets.The comparison measures mean pairwise ℓ1 distance across three feedback subsets.Source: §6.1 · pp. 7–8
  • Greater coefficient stability does not ensure higher output preference: MLE and adapted SS often yield more preferred outputs, while RM can be more stable but less preferred than BPM.BPM improves on uniform merging while reducing coefficient variation relative to MLE under limited feedback.Source: §6.1 · p. 8

6.2 UNCERTAINTY UNDER LIMITED FEEDBACK

  • Under BPM’s model and prior, nominal 90% intervals for β = w/τ achieve 88.9% coverage after 10 comparisons and 89.2% after 25.Prior perturbations reduce coverage in some settings, and evidence of interval contraction at these budgets is limited.Source: §6.2 · p. 8, Figure 3 · p. 8
  • Lower posterior uncertainty is associated with higher win rates in some task–budget settings, but the association is weak on story generation.The comparison uses persona-level averages across three feedback subsets.Source: §6.2 · p. 8, Figure 4 · p. 9

6.3 FROM INFERRED WEIGHTS TO MODEL BEHAVIOR

  • The Literary and Librarian posteriors favor expressive writing and clear, coherent storytelling, respectively; their means provide persona-specific coefficients while the full posteriors represent plausible alternatives.These interpretations concern reward trade-offs under the model and prior.Source: §6.3 · p. 9
  • On summarization, BPM outperforms each library’s own uniform merge for both RLOO- and DPO-trained experts, for every persona at both feedback budgets.Each library is compared with its own uniform merge.Source: §6.3 · p. 9
  • Summarization policies trained with inferred Ward and Emergency Doctor weights are preferred over uniform-weight training and training with the other persona’s weights.This extends the reported usefulness of inferred weights beyond model merging.Source: §6.3 · p. 9

7 CONCLUSION

The conclusion presents BPM as a way to infer merge coefficients from pairwise feedback while retaining uncertainty about plausible reward weights. The appendix passages describe its inference pipeline, LoRA merging construction, and conditions governing weight and temperature identification.

7 CONCLUSION

  • BPM infers a posterior over reward weights from pairwise choices and uses its mean to merge shared experts without per-user policy training.The conclusion reports improvements over uniform merging across three generation tasks and preference-specific behavior under controlled feedback shifts.Source: §7 · p. 9

A.1 NOTATION

  • Algorithm 1 standardizes rewards, samples the posterior, computes the posterior mean and uncertainty, then uses the mean as merge coefficients.The algorithm names NUTS for posterior sampling and includes a convergence check.Source: §A.1 · p. 16

A.2 MERGING LORA EXPERTS

  • Each LoRA expert is an adapter on a shared frozen base; weighted parameter averaging preserves that base and combines corresponding adapter factors.The construction assumes matching adapter shapes and corresponding layers; separately trained factors need not yield the same result as averaging effective weight matrices.Source: §A.2 · p. 16

A.3 REWARD WEIGHTS AND CHOICE TEMPERATURE

  • The Bradley–Terry likelihood depends on reward weights and choice temperature through β = w/τ; constraining w to the simplex removes their joint scale ambiguity.The mapping from simplex weights and positive temperature to β is continuous and bijective over the stated domain.Source: §A.3 · p. 16
  • Identifying β from choices requires reward contrasts that span the reward space; inferring temperature also allows more decisive choice probabilities without changing contrast normalization.For random contrasts, E[dd⊤] ≻ 0 is stated as a sufficient condition.Source: §A.3 · pp. 16–17

A.4 PRIOR SPECIFICATION

This appendix specifies BPM’s priors and posterior summaries, characterizes information and concentration under pairwise feedback, and states the assumptions connecting inferred weights to merge optimality.

A.4 PRIOR SPECIFICATION

  • BPM uses shared independent Dir(s1) reward-weight priors and log τ ∼ N(log 2, 1), with task-specific s variance-matched to a reference tree.The calibrated s values are fixed across personas and feedback budgets within each task.Source: §A.4 · p. 17

A.5 POSTERIOR MEAN AND UNCERTAINTY

  • The posterior mean is a valid simplex estimate that minimizes squared error for reward weights; U measures remaining marginal uncertainty relative to the prior.Without feedback U = 1, and it need not decrease after every added comparison.Source: §A.5 · p. 18

A.6 INFORMATION PROVIDED BY REWARD CONTRASTS

  • Pairwise likelihood depends only on the informed component in the reward-contrast span, so comparisons do not directly distinguish directions orthogonal to that span.More comparisons need not add independent directions, and the dimension bound alone gives neither U nor a posterior contraction rate.Source: §A.6 · p. 18

A.7 POSTERIOR CONCENTRATION

  • Under iid comparisons and stated regularity conditions, the posterior concentrates on the unique expected-log-loss minimizer, which equals the generating parameters under Assumption 1.With misspecification, the target depends on the elicitation contrasts; the iid theorem does not establish consistency for finite shared pools.Source: §A.7 · pp. 19–20

A.8 CONNECTION TO MODEL OPTIMALITY

  • Under Assumption 2 and increasing feedback, BPM’s model converges to a maximizer of expected weighted reward; this gives no finite-sample optimality guarantee.The result also requires the preference-model conditions of Proposition 2.Source: §A.8 · p. 20

B.1 DATA ROLES AND SEPARATION

  • The data roles separate expert training, calibration, and testing; MIMIC-CXR partitions are patient-disjoint, while COCO and WritingPrompts separate roles by image and cleaned prompt.The remaining MIMIC-CXR partition is split into pool-simulation, reward-standardization, and test reports.Source: §B.1 · pp. 20–21

B.2 REWARD DIMENSIONS AND EXPERT LIBRARIES

  • The libraries contain 29 radiology, 21 image-captioning, and 16 story-generation reward experts, spanning metric, clinical, form-related, and rubric criteria.Story-generation rewards include text properties, HANNA criteria, and additional style criteria.Source: §B.2 · p. 21

B.3 EXPERT TRAINING AND COMPUTE

The appendix documents expert training and feedback-pool construction, comparator configurations, posterior sampling, and test aggregation.

B.3 EXPERT TRAINING AND COMPUTE

  • The expert-training setup includes GRPO and RLOO summarization configurations and a separate DPO summarization library; LoRA experts use rank 16, α = 32, and zero dropout.Policy training uses NVIDIA A100 80GB GPUs, while posterior inference runs on CPUs.Source: §B.3 · pp. 21–22

B.4 POOL CONSTRUCTION AND FEEDBACK SIMULATION

  • Pool C uses 20 random Dir(1) merges and samples four distinct-model pairs per input, shared across personas; persona-specific filtering determines retained feedback.Reported n counts retained comparisons from a preconstructed pool, not an online query budget. Summarization and image captioning decode greedily; story generation uses temperature 0.7 and top-p 0.95.Source: §B.4 · p. 22

B.5 COMPARED METHOD CONFIGURATIONS

  • Ridge merge fits logistic regression to standardized contrasts with an ℓ2 penalty of one before softmax; DPO trains a fresh adapter, while ICAI and Language-to-Weights use prompting-based configurations.Language-to-Weights allocates 100 points across reward dimensions and normalizes them as merge coefficients.Source: §B.5 · p. 22

B.6 POSTERIOR INFERENCE AND RANDOMIZATION

  • Posterior inference uses four NUTS chains with 1,000 warmup and 2,000 retained samples, requiring split R-hat ≤ 1.01 and at least 400 samples per weight.Feedback repetitions use nested subsets and keep the expert library fixed.Source: §B.6 · p. 23

B.7 TEST METRICS AND STATISTICAL AGGREGATION

  • Macro decided win rates weight personas equally and condition on valid, decided judgments; 95% intervals bootstrap test inputs conditional on the fixed feedback subsets and expert library.The evaluation also reports wins, losses, ties, order-inconsistent verdicts, and invalid judgments, with a supplementary score retaining valid undecided pairs.Source: §B.7 · p. 23

C PERSONA SPECIFICATIONS

This appendix section describes task-specific personas and scoring criteria for consistent pairwise judgments, including rubric-based story personas with shared criterion anchors and weak dialogue as a fault criterion. It also outlines persona-level comparisons at n = 500, averaged across three feedback subsets.

C.1 SUMMARIZATION PROMPTS

  • Six radiology personas trade off completeness, urgency, brevity, evidential support, and preservation of diagnostic certainty in summary judgments.For example, ward clinicians prioritize management-relevant omissions, emergency clinicians want urgent conclusions first and brief, and other personas focus on concise diagnosis or supported claims.Source: C PERSONA SPECIFICATIONS · p. 23, §C.1 · p. 24

C.2 IMAGE CAPTIONING PROMPTS

  • Captioning personas emphasize image retrieval, polished prose, visible evidence, and accessible descriptions, while an additional annotator persona is excluded from main averages.The prompts distinguish comprehensive object and setting coverage from evocative style, strict visual grounding, and short alt text centered on the main subject and action.Source: C PERSONA SPECIFICATIONS · p. 23, §C.2 · pp. 25–26

C PERSONA SPECIFICATIONS

  • The persona specification fixes the main evaluation set at six summarization, six captioning, and seven story-generation personas; the extra captioning Annotator is diagnostic only.Source: C PERSONA SPECIFICATIONS · p. 23

C.3 STORY GENERATION PROMPTS

  • Story-generation personas prioritize distinct editorial goals, from restrained literary prose and commercial plot momentum to child-friendly, compressed, dramatic, or restorative stories.The prompts contrast precision and emotional suggestion with action and resolution, and also include preferences for reassuring endings, compact stories, enacted scenes, and repair after wrongdoing.Source: §C.3 · pp. 26–27

D RUBRIC-BASED PERSONAS

  • Five rubric-based story personas prefer responses with higher weighted scores on their nonzero criteria; ties or missing required scores are excluded, and a controlled sweep interpolates Literary and Genre weights.Qwen3.5-27B scores eleven criteria with AutoRubric at temperature 0 and thinking disabled, using four-level anchors; endpoint rubrics remain fixed when evaluating stories.Source: Table 6 · p. 27, D RUBRIC-BASED PERSONAS · p. 27

D.1 CRITERION DEFINITIONS AND SCORE ANCHORS

  • Four-level story-quality anchors map positive criteria to level divided by three and fault criteria to one minus level divided by three, using the same definitions for both scorers.The anchor set includes criteria such as surprise and complexity, with higher levels describing a more surprising yet fitting ending or more elaborate development.Source: §D.1 · pp. 27–28
  • Weak dialogue is scored as a fault, with no dialogue receiving a score of 2 on this criterion.The scale describes 0 as alive dialogue, 1 as mostly working, 2 as weak, and 3 as unusable or unconvincing when speech was needed.Source: §D.1 · p. 29

E.1 PERSONA-LEVEL PERFORMANCE

  • At n = 500, Figure 5 reports BPM’s decided win rate against each compared method, averaged over three feedback subsets.It uses the same persona sets and decided-win-rate definition as Table 1; the supplied passages do not state the cell values.Source: Figure 5 · p. 30, §E.1 · p. 30

E.2 HUMAN FEEDBACK AND EVALUATION

Across the supplied evaluations, personalized models track instructed preferences in a human captioning proof-of-concept and simulated persona comparisons, while posterior distributions retain plausible trade-offs beyond the merge coefficients.

E.2 HUMAN FEEDBACK AND EVALUATION

  • In image captioning, both personalized models beat uniform and the preferred model changes with the instructed persona; this single-annotator study is not independent validation across human users.Feedback comprised 100 and 89 decided comparisons for the two descriptions.Source: §E.2 · p. 30, Table 7 · p. 30

E.3 LENGTH- AND READABILITY-MATCHED PREFERENCE REVERSALS

  • 54/55 versus 5/49 length-matched comparisons, and 33/33 versus 3/34 when readability was also matched, favored the Librarian model under the Librarian rather than Literary judge.Matching selects a subset of outputs and does not identify a causal effect with length removed.Source: §E.3 · p. 31

E.4 RUBRIC-BASED PREFERENCE FOLLOWING

  • Rubric-persona results report a macro advantage for BPM over every compared method at both reported settings.Table 8 reports decided win rates against each comparator, averaged over three feedback subsets.Source: §E.4 · p. 31, Table 8 · p. 31

E.5 USING INFERRED WEIGHTS FOR POLICY TRAINING

  • On Summarization, each persona preferred the policy retrained at its inferred weights over uniform-weight training and the other persona’s policy.Policies used posterior-mean weights or uniform weights under the same training recipe.Source: §E.5 · p. 31, Table 9 · p. 31

E.6 ALTERNATIVE EXPERT LIBRARIES

  • Within RLOO and DPO libraries, BPM gains over each library’s own uniform merge at both budgets, but these comparisons do not rank the libraries’ training algorithms.Results are averaged over three feedback subsets.Source: §E.6 · pp. 31–32, Table 10 · p. 32

F.1 POSTERIOR DISTRIBUTIONS AND UNCERTAINTY

  • For Story Generation at n=500, Literary posteriors emphasize show-don’t-tell, prose elegance, and imagery, while Librarian posteriors emphasize avoiding purple prose and coherence.The posterior distributions also quantify support for alternative reward weights beyond the means used as merge coefficients.Source: §F.1 · p. 32, Figure 6 · p. 32
  • Across some task-budget settings, lower posterior uncertainty is associated with higher preference win rates; the reported correlations describe between-person variation with task and budget held fixed.The correlations do not establish a within-person relationship.Source: §F.1 · p. 32

F.2 COEFFICIENT INTERVAL COVERAGE

Interval performance is assessed in simulations matched to BPM’s model and prior, with sensitivity to fitted-prior assumptions and no demonstrated feedback-driven contraction at the smallest budgets.

F.2 COEFFICIENT INTERVAL COVERAGE

  • Under the generating BPM prior, Table 11 compares coverage and width for nominal 90% intervals of β, but does not establish calibration for observed persona preferences.Coverage and width are averaged over coefficient coordinates and simulations within each task, then equally across tasks.Source: §F.2 · p. 33, Table 11 · p. 33
  • At n=10 and 25, prior-only intervals are near nominal and slightly narrower than posterior intervals, so the matched comparison does not demonstrate feedback-driven interval contraction.Posterior location and shape may change without a decrease in average interval width.Source: §F.2 · p. 33, Table 11 · p. 33
  • RM bootstrap intervals under-cover despite being wider than BPM’s, while RM Laplace intervals over-cover and are much wider in this simulation.The two tested constructions do not reproduce BPM’s interval performance here.Source: §F.2 · p. 33
  • With doubled fitted Dirichlet concentration, nominal 90% coverage for w falls to 70.6–77.1%; sparse generating weights yield 81.1–81.4% coverage on Story Generation.The reported ranges cover tested settings, not confidence intervals.Source: §F.2 · p. 33, Table 12 · p. 34
  • Halving the log-temperature prior’s standard deviation yields 87.7–90.4% coverage for w but 78.6–88.6% for β; posterior interval widths for w remain about 92–100% of prior widths.The target and fitted assumptions affect coverage differently.Source: §F.2 · p. 33, Table 12 · p. 34

F.3 POINT ESTIMATES AND COEFFICIENT STABILITY

The appendices examine coefficient stability, output preference, and alternative ways to use inferred weights; BPM’s stability is not a uniform preference advantage, and direct merging need not maximize fitted utility.

F.3 POINT ESTIMATES AND COEFFICIENT STABILITY

  • At n=25 and 100, BPM posterior means vary less than MLE in all three tasks, but comparisons with regularized estimators depend on task and budget.RM is less variable in several settings, while MAP is less variable on Story Generation at n=500.Source: §F.3 · p. 34
  • MLE and adapted SS often produce more preferred outputs than BPM, while BPM outperforms RM at both main-table budgets; coefficient stability does not uniformly maximize output preference.A diagnostic also found higher coefficient variation but slightly lower output-score variation for shrinkage on Summarization at n=100.Source: §F.3 · pp. 34–35
  • Shrinkage trades stability against preference by task: it yields more preferred outputs but less stable coefficients on Summarization, while BPM has higher output preference on Image Captioning and Story Generation at n=100.At n=25 on the latter two tasks, shrinkage selects uniform weights.Source: §F.3 · p. 35, Table 13 · p. 35
  • BPM is more stable at n=10 in all three tasks; paired intervals support lower variation through n=250 on Summarization and at every tested Story Generation budget.On Image Captioning, the stability difference is inconclusive beyond n=10.Source: §F.3 · p. 36, Table 19 · p. 38

F.4 DIRECT WEIGHTING

  • Across 126 Captioning persona-budget-feedback-draw cells, utility-based selection returns an individual expert and exceeds the BPM merge’s estimated utility in 125, without establishing persona preference.Selection uses inferred utility on held-out evaluation inputs for the diagnostic.Source: §F.4 · p. 36

F.3 POINT ESTIMATES AND COEFFICIENT STABILITY

  • Adapted Spectral Souping models are preferred in 14 of 15 task-budget comparisons; applying exact-delta merging to both estimators preserves the qualitative conclusions.The remaining interval, for Summarization at n=500, includes 50%; changing the merge operator alone does not close the gap.Source: §F.3 · p. 35, Table 17 · p. 38, Table 18 · p. 38

F.4 DIRECT WEIGHTING

  • On Summarization, direct merging achieves higher win rates against uniform than utility-based selection, though the supplied comparison does not establish a utility-preference trade-off on the same models.The experiment evaluates the methods against a common uniform-merge reference.Source: §F.4 · p. 39, Table 20 · p. 39

G.1 EVALUATION ACCOUNTING AND SCORING SENSITIVITY

The supplementary scoring preserves BPM’s advantage over uniform merging, while showing that win-rate magnitudes depend on how undecided judgments are treated. Repeatability checks find stable decided preferences, but persona wording and judge family can affect comparisons; broader human-user generalization remains untested.

G.1 EVALUATION ACCOUNTING AND SCORING SENSITIVITY

  • At n = 500, supplementary scores against uniform are 74.0% for Summarization, 73.9% for Image Captioning, and 66.4% for Story Generation.These scores retain valid ties and order-inconsistent comparisons under a central assignment of 0.5.Source: §G.1 · p. 39, Table 21 · p. 40
  • Across 42 comparisons, central supplementary scores and decided win rates agree in direction relative to 50%, but magnitudes can differ substantially.For Summarization against uniform at n = 500, the decided rate is 88.4%, the supplementary score is 74.0%, and 34.9% of verdicts are order-inconsistent.Source: §G.1 · p. 39, Table 21 · p. 40
  • The supplementary score is an assignment, not a reconstruction of missing order-level decisions; assignment bounds and sampling intervals represent different uncertainties.Invalid judgments were excluded and counted separately, with a reported rate of zero across the seven-method panel.Source: §G.1 · p. 40

G.2 JUDGE RELIABILITY AND REPLICATION

  • Repeated evaluations largely preserve decided preferences; most variation concerns whether a pair receives a decided judgment.The study re-evaluated the same 451 Summarization pairs about thirteen hours apart and 200 fixed Image Captioning pairs per persona.Source: §G.2 · p. 40, Table 22 · p. 41
  • Preference reversals persist across judge families, although comparison magnitudes can depend on the judge family.The Story Generation comparison against DPO is an exception, with the two estimates near 50%.Source: §G.2 · p. 41, Table 23 · p. 41
  • Five of six preference reversals persist after persona-description paraphrasing, but repeatability under fixed prompts does not ensure wording robustness.For Ward–ED, order-inconsistent judgments rise from 49.4% to 73.4% after paraphrasing the Ward description; broader human-user generalization remains untested.Source: §G.2 · pp. 40–41
Loading 2609.39055v1…