PaperlayerView exact revision on arXiv

Source-linked AI summary

Covariate-dependent Joint Modeling of Multivariate Ordinal Preferences and Its Connections with Comparison Models

Yujie Chen, Antik Chakraborty, Anindya Bhadra

arXiv:2610.09070v1stat.MLcs.LGstat.ME

TL;DR

Multivariate ordinal responses with covariates are often modeled separately or reduced to pairwise or list-wise comparisons, rather than modeled jointly from their full scores. The paper proposes a covariate-dependent CR–MRF and reports improved held-out prediction of comparisons over BT and PL.

  • Problem

    Common approaches model ordinal attributes separately or reduce full ordinal scores to pairwise or list-wise comparisons.

    Source: §1 · pp. 1–2

  • Method

    A covariate-dependent consecutive ratio Markov random field jointly models the multivariate ordinal scores, with maximum-likelihood inference using importance sampling for the intractable normalizer.

    Source: §1 · p. 2, §3.4 · p. 4

  • Results

    The CR–MRF improves held-out prediction over BT and PL in simulations and data analysis.

    Source: §7 · p. 8

  • Takeaways & Limitations

    BT and PL comparison models can be recovered from the joint model under restrictions, while the CR–MRF models full ordinal feedback.

    Source: §1 · pp. 1–2

    Latent-variable alternatives may leave the translation from latent dependence to observed-outcome dependence relatively opaque.

    Source: §1.1 · p. 2

Abstract

from arXiv · show

Multivariate ordinal data along with covariates are commonly collected in problems ranging from alignment of language models with human preferences, as well as in recommender systems. For example, data sets such as MovieLens contain several movies rated on a scale 1--5 by human users, along with their demographic information such as age or gender. Similarly, data sets such as HelpSteer collect human feedback on several attributes such as "helpfulness" or "verbosity" of LLM response on an ordinal scale, with covariates depending on the LLM prompt--response pairs. Unfortunately, the standard approaches for modeling these data (a) look at the attributes individually rather than jointly, and (b) often convert the data into pairwise or list-wise win--loss comparisons for fitting models such as Bradley--Terry and Plackett--Luce. Both of these lead to a coarsening of what is actually observed, which we address via a joint covariate-dependent consecutive ratio Markov random field model. We also show pairwise or listwise comparison models are obtained under restrictions of our joint model, and that joint modeling improves comparisons. We also develop a maximum likelihood inference procedure even in the presence of an intractable normalizer.

1 INTRODUCTION

The paper targets two shortcomings in multivariate ordinal-data modeling: responses are often modeled separately, or reduced to pairwise or list-wise comparisons, despite covariates and cross-attribute coupling. It develops a covariate-dependent joint CR–MRF, relates it to existing comparison models, and reports improved prediction of comparisons.

1.1 Related Works

  • Latent-variable multivariate ordinal models may scale poorly to high dimensions and can be difficult to interpret; their dependence structures may also translate opaquely to observed outcomes.The paper contrasts its covariate-dependent MRF and full-likelihood analysis with prior CR–MRF work using pseudolikelihood [Suggala et al. (2017)].Source: §1 · p. 1, §1.1 · p. 2

1 INTRODUCTION

  • The proposed covariate-dependent CR–MRF jointly models multivariate ordinal scores rather than separate responses or reduced-form comparisons.The paper also shows univariate ordinal and pairwise or list-wise comparison models arise under restrictions of the joint model.Source: §1 · pp. 1–2

1.2 Our Contributions

  • The contributions include showing BT and PL can be recovered from the CR–MRF under restrictions, clarifying how those models coarsen observations.The authors also describe an extension for missing data and report improved prediction of pairwise or list-wise comparisons on MovieLens.Source: §1.2 · p. 2

2 BACKGROUND ON ORDINAL COMPARISON MODELS

Bradley–Terry and Plackett–Luce model comparison outcomes rather than the full ordinal scores. Their extensions allow ties or weak rankings.

Bradley–Terry

  • Bradley–Terry models pairwise comparison outcomes using item strengths; the original formulation generally observes the comparison, not the underlying scores.Lower scores are assumed better, and log strength can be interpreted as item utility.Source: §2 · pp. 2–3

Plackett–Luce

  • Plackett–Luce extends Bradley–Terry to multiple items by modeling a preference ordering over the compared subset.The ordering is represented as a permutation of the items in that subset.Source: §2 · p. 3

Ties and weak rankings

  • Bradley–Terry–Davidson adds a tie propensity, while the weak Plackett–Luce model assigns a stage weight to items ranked identically.For weak PL, the propensity depends on the number of items ranked together.Source: §2 · p. 3

3 THE INTRACTABLE CR–MRF MODEL AND ML INFERENCE

The model is a joint distribution over multivariate ordinal outcomes, with node and edge parameters that can depend on covariates. Because its normalizer is intractable, inference uses importance sampling based on a tractable independent model.

3.1 The Homogeneous CR–MRF

  • The homogeneous CR–MRF defines a joint distribution for p ordinal variables on an (M+1)-level scale, with node and edge sufficient statistics.Its conditional consecutive log-odds compare adjacent categories; the normalized edge statistic lies in [0, 1] for numerical stability.Source: §3.1 · p. 3

3.2 Covariate-dependent CR–MRF

  • In the covariate-dependent model, node and edge parameters are linear functions of covariates, and zero slopes recover the homogeneous model.The joint distribution across independent individuals factorizes across individuals.Source: §3.2 · pp. 3–4

3.3 The Tractable Independence Model

  • Setting all edge parameters to zero yields a product distribution with a closed-form normalizer that can be sampled in batches.The resulting independent categorical variables provide the tractable reference model used in inference.Source: §3.3 · p. 4

3.4 Maximum Likelihood Inference

  • The paper develops stochastic gradient ascent for maximum likelihood, departing from pseudolikelihood inference used in prior CR–MRF work.The authors provide conditions for existence of the MLE and state that it is unique when it exists.Source: §3.4 · p. 4
  • Maximum-likelihood inference estimates the intractable normalizer and its gradient by importance sampling from the tractable independence model.The ratio-estimator approximates ∇log Z(Θ); in the covariate case, the procedure is repeated per individual and is batchable over individuals and samples.Source: §3.4 · pp. 4–5
  • For high-dimensional settings where the edge matrix is expected to be sparse, the method adds an ℓ1 penalty and applies soft-thresholding.The paper states a consistency result for the modified updates, subject to a unique maximizer of the penalized log-likelihood.Source: §3.4 · p. 5

4 CONNECTIONS WITH PAIRWISE AND LIST-WISE COMPARISON MODELS

The section derives pairwise and list-wise comparison models from restricted CR–MRF constructions and shows that ranking coarsens ordinal data, losing information when there are at least three rating levels.

4 CONNECTIONS WITH PAIRWISE AND LIST-WISE COMPARISON MODELS

  • An independent two-level CR–MRF induces pairwise comparisons with the BTD form, but its pair-specific tie propensity νjk is determined by the original parameters and bounded below by 2.Unlike BTD's global tie parameter, the derived tie propensity is a function of the original parameters.Source: §4 · p. 5
  • Discretizing independent exponential arrivals yields an independent CR–MRF, while the CR–MRF can also allow level-dependent consecutive ratios.The race-derived construction has consecutive ratios that do not vary across levels.Source: §4 · p. 6
  • When item strengths are equal, the discretized-race CR–MRF gives the weak PL model exactly under δr = ar−1.The CR–MRF and weak PL stage weights agree in this equal-parameter case.Source: §4 · p. 6
  • As a common rescaling sends the exponential-race strengths toward zero, the weak-ranking model approaches the original no-ties PL model.The limiting argument concerns uncensored scores and preserves relative strengths.Source: §4 · p. 6
  • With covariate-dependent exponential rates, the discretized scores remain conditionally independent geometric variables, and their induced weak ranking has the form derived for the homogeneous case.A linear dependence of log γj on covariates yields model (6).Source: §4 · p. 6

4.1 Information Loss due to Converting Ordinal Data into Ranking

  • With at least three rating levels, replacing ordinal ratings by their complete weak ranking loses Fisher information in every nonzero natural-parameter direction, even when ties are retained and Θedge ≠ 0.Complete pairwise comparisons contain exactly the same information as the complete weak ranking.Source: §4.1 · pp. 6–7

5 NUMERICAL EXPERIMENTS

The experiments evaluate out-of-sample pairwise preference prediction from simulated ordinal data, comparing covariate-dependent CR–MRF estimates with covariate-dependent BT/PL models. The reported results favor direct ordinal modeling in both approximation and calibration.

5 NUMERICAL EXPERIMENTS

  • The simulation uses five items and two covariates, with five-level and binary responses replicated ten times using 500 training and 1,000 test samples.The ordinal responses are generated from a covariate-dependent CR–MRF with all edges present and randomly signed.Source: §5 · p. 7
  • The evaluation compares test-set pairwise preference probabilities from CR–MRF MLE and MPLE estimates with probabilities from fitted BTD/strict PL models.RMSE assesses probability approximation, while the Brier score assesses calibration against observed outcomes.Source: §5 · p. 7
  • Modeling ordinal ratings directly with the CR–MRF MLE gives better approximations and calibration of pairwise preference probabilities than the comparison-model estimates.The reported metrics are RMSE and Brier score; the supplied passages do not give their numerical values.Source: §5 · p. 7, Table 1 · p. 7

6 APPLICATION TO MOVIELENS

The MovieLens application compares covariate-dependent CR–MRF and strict PL predictions for held-out reviewers. CR–MRF performs better overall, especially for ambiguous preferences, while PL is better calibrated for distinct preferences.

6 APPLICATION TO MOVIELENS

  • The comparison uses five widely rated movies and 64 held-out reviewers, with results also split by reviewer gender.The MovieLens block contains 234 reviewers who rated all five movies; 170 were used for training.Source: §6 · p. 7, Figure 1 · p. 8
  • For ambiguous preferences (close to 0.5), CR–MRF probabilities perform much better; for distinct preferences, PL estimates are better calibrated, with out-of-sample probabilities of 0.85 for males and 0.78 for females for ‘Airplane!’ versus ‘Desperately Seeking Susan’.Source: §6 · p. 8

7 CONCLUSIONS

The conclusion presents covariate-dependent CR–MRF as a joint model of full ordinal feedback and reports held-out prediction benefits over BT and PL. It identifies intractable normalization as an inference challenge and points to LLM reward-model losses as future work.

7 CONCLUSIONS

  • The authors report that simulations and data analysis show CR–MRF benefits in held-out prediction over BT and PL.They describe this as joint modeling of full ordinal feedback, in contrast to coarsened pairwise or list-wise comparisons.Source: §7 · p. 8
  • The intractable normalizing constant complicates likelihood-based inference, which the authors mitigate by building on Chen et al. (2026a).Source: §7 · p. 8
  • The authors identify testing CR–MRF loss as an alternative to BT or PL loss in LLM reward models as future work.Source: §7 · p. 8

S.1.1 Existence of MLE

The section characterizes MLE existence through sufficient statistics: the empirical mean must lie in the relative interior of the convex hull of possible statistics.

S.1.1 Existence of MLE

  • In canonical exponential-family form, the MLE matches empirical mean sufficient statistics to their model expectation.Source: §S.1.1 · p. 11
  • The MLE exists if and only if the empirical sufficient-statistic mean lies in the relative interior of the convex hull of possible statistics.Source: §S.1.1 · p. 11
  • Nonexistence occurs when a supporting hyperplane contains all observed statistics while strictly separating at least one possible statistic; identical observed statistics are a standard case.Source: §S.1.1 · p. 11

S.1.2 Node Conditional Distribution for the Homogeneous CR–MRF

The homogeneous CR–MRF has node-conditional distributions in adjacent-category logit form, which supports tractable pseudolikelihood inference.

S.1.2 Node Conditional Distribution for the Homogeneous CR–MRF

  • Subtracting adjacent-category exponents yields the consecutive log-odds ratio between categories l−1 and l.Source: §S.1.2 · p. 11
  • Each node conditional is an adjacent-category logistic regression with neighboring variables as predictors, parallel slopes across cutpoints, and category-specific intercepts.Source: §S.1.2 · p. 11
  • The predictor for neighbor k is x_k = M − y_k, and the edge parameter θ_jk is the fitted slope on x_k multiplied by M^2.Source: §S.1.2 · p. 11
  • The adjacent-category conditional form makes pseudolikelihood inference tractable.Source: §S.1.2 · p. 11

S.1.3 Node Conditional Distribution for the Covariate-dependent CR–MRF

This supplement characterizes the covariate-dependent CR–MRF’s node conditionals, comparison and information-loss properties, and missing-data handling, and presents additional simulation and MovieLens results. With randomly resolved ties, CR–MRF has lower out-of-sample total error than strict PL, while the models behave similarly with binary ratings (M = 1); the section also presents fitted MovieLens 1M edge networks and estimated node slopes.

S.1.3 Node Conditional Distribution for the Covariate-dependent CR–MRF

  • Each node conditional is an adjacent-category logistic regression: individual covariates have cutpoint-specific slopes, while neighbor values and their covariate interactions have parallel slopes shared across cutpoints.The neighbor values are M − y_i,k; the interaction slopes are also parallel across cutpoints.Source: §S.1.3 · p. 12

S.2.1 Proof of Theorem 1

  • The convergence proof uses finite-state Lipschitz gradients for log Z and compactness of the parameter space to uniformly control importance-sampling bias along the iterates.The result follows from a convergence theorem for a general class of exponential-family graphical models that includes CR–MRF.Source: §S.2.1 · p. 12

S.2.2 Proof of Proposition 1; S.2.3 Proof of Theorem 2; S.2.4 Proof of Corollary 1

  • With zero edge parameters, nodes are independent; geometric memorylessness gives the sequential tied-group recursion, and as ε → 0 the singleton transitions recover Plackett–Luce while tie probability is O(ε).The proof of the limit uses finitely many remaining sets and at most p stages.Source: §S.2.2 · p. 12, §S.2.3 · p. 13, §S.2.4 · p. 13

S.2.5 Proof of Theorem 3; S.2.6 Proof of Proposition 2

  • The MLE fails exactly when a supporting hyperplane contains all observed sufficient statistics; weak rankings lose Fisher information unless the score is constant within ranking classes.For M ≥ 2, the proof also establishes a minimal sufficient-statistic representation and positive-definite Fisher information for the full data.Source: §S.2.5 · p. 13, §S.2.6 · p. 14

S.3 INFORMATION LOSS CALCULATIONS UNDER CAPPED EXPONENTIAL RACES WITH UNKNOWN COMMON SCALE

  • At equal rates, scale and contrast are Fisher-orthogonal; when h ≠ 0, both matrices are nonsingular and efficient information for δ with ρ unknown equals its diagonal entry, a local result.Away from equal rates, off-diagonal information generally does not vanish; with fixed M as t → 0, the contrast ratio tends to one while both contrast-information quantities tend to zero, a terminal-category degeneracy, not the uncensored PL limit.Source: §S.3 · p. 15, §S.3.1 · p. 15

S.4 HANDLING MISSING DATA

  • Full CR–MRF performs better than the alternatives in missing-value MSE and pairwise-preference Brier score; both errors increase with more missing coordinates and decrease with larger training sets.The simulation varied the number of missing test-set coordinates from one to three and training-set size from 500 to 1,000.Source: §S.4 · p. 16, Table S1 · p. 17

S.5 ADDITIONAL SIMULATION RESULTS

  • Penalized MLE beats penalized pseudo-likelihood in Frobenius loss across all tested dimensions and has higher MCC for edge selection.The high-dimensional homogeneous-model comparison reports means over 10 replicates with n = 500 and M = 4.Source: §S.5 · p. 17, Table S3 · p. 18

S.6.1 CR–MRF regression

  • MovieLens estimates tracked intuitive rating shifts, and MLE and MPLE agreed in sign on gender effects and the clearest age effects.Holding other ratings at M, female reviewers’ adjacent-level odds ratios were 0.56 for When Harry Met Sally and 1.93 for Airplane!; each standard deviation of age corresponded to 0.70 for Desperately Seeking Susan and 1.25 for Ferris Bueller’s Day Off and The Breakfast Club. The only sign disagreements were near-zero age effects for When Harry Met Sally and Airplane!.Source: §S.6.1 · p. 18
  • The appendix displays fitted edge networks θjk(x) under the joint MLE and joint MPLE for men and women at ages 21, 29.5, and 47.Edge color indicates sign and line width is proportional to |θjk(x)| on a scale shared across six panels.Source: Figure S1 · p. 19, Figure S2 · p. 20
  • Table S4 reports estimated node slopes βj for MovieLens 1M under the joint MLE and node-wise MPLE, with age standardised.Source: Table S4 · p. 21

S.6.2 Comparison with preference models

  • Total error is 0.65 for CR–MRF and 0.72 for strict PL with randomly resolved ties; CR–MRF’s induced preference probabilities are overall better.The observed difference from the main-draft result is attributed to Monte Carlo variation in out-of-sample ratings sampled given the MLE and corresponding covariates.Source: Figure S3 · p. 21, §S.6.2 · p. 21
  • For binary ratings (M = 1), the covariate-dependent CR–MRF and BTD behave very similarly in out-of-sample preference prediction.Ratings above 3.5 are coded as 1 and those below 3.5 as 0; the similarity is consistent with Proposition 1.Source: §S.6.2 · p. 21, Figure S4 · p. 22
Loading 2610.09070v1…