LESS is More: Rethinking Probabilistic Models of Human Behavior

1 INTRODUCTION

• Boltzmann probability can change sharply when similar alternatives are added: four equally good trajectories, three to the right, reduce the predicted probability of going left from .5 to .25.The model assigns .25 probability to each of the four trajectories.Source: §1 · p. 2
• Continuous, bounded trajectory spaces contain alternatives with varying similarity, including trajectories near joint limits or squeezed between nearby obstacles.Such trajectories can be dissimilar from the rest of the space.Source: §1 · p. 2
• Boltzmann rationality models trajectories in a way rooted in discrete choice among distinct options, although trajectories are connected to one another.It has nevertheless succeeded in robotic inference and prediction across several domains.Source: §1 · pp. 1–2
• The paper proposes rethinking the Luce choice axiom for trajectory spaces so that similarity between trajectories influences their probabilities.The model is named LESS, for Limiting Errors due to Similar Selections.Source: §1 · p. 2
• LESS accounts for each trajectory’s feature-space similarity to alternatives, and the paper reports better human-decision prediction, more accurate simulated inference, and more robust inference on a 7DOF arm.The arm learns from user demonstrations, where ground truth is unavailable.Source: §1 · p. 2

2 METHOD

• The paper seeks an improved human-behavior model for robotics applications in prediction and reward inference.The stated motivation is to develop a theory for behavior in trajectory spaces.Source: §2 · p. 2
• The intended model is designed for trajectory spaces rather than abstract discrete decisions.The paper frames this as a theory-development goal.Source: §2 · p. 2
• The authors begin developing the theory by turning to the literature on human decision making.Source: §2 · p. 2

2.1.1 Human Decision Making.

• The Luce choice rule gives option probabilities from desirability values, with desirability modeled as exponentiated reward in the Luce-Shepard rule.The background introduces the rule as a theory of human decision making.Source: §2.1.1 · p. 2
• For trajectory choices, the Boltzmann model assigns probabilities according to exponentiated reward, with reward typically defined over trajectory features.It applies the Luce-Shepard choice rule to trajectories, which may be sequences of continuous-valued actions.Source: §2.1.1 · p. 2

2.1.2 Handling duplicates.

• Prior work identifies the duplicates problem: adding a duplicate option can unduly affect choice probabilities, prompting models that group identical or similar options.The attribute rule instead treats options as bundles of attributes while retaining desirability-based choice.Source: §2.1.2 · p. 2
• The attribute rule first selects an attribute and then an option expressing it, providing a starting point for extending duplicate handling to continuous-valued trajectory features.Attributes have desirability values, and their intensity indicates how strongly an option expresses them.Source: §2.1.2 · p. 2

2.2 The LESS Human Decision Model

• LESS extends the attribute-rule approach to continuous trajectory spaces by defining similarity between trajectories, generally in feature space.Features can range up to the trajectory itself, and the approach first treats features as attributes before softening similarity.Source: §2.2 · pp. 2–3

2.2.1 Accounting for Trajectories with Identical Features.

• When trajectories share an exact feature vector, LESS treats them as one option and distributes their selection probability equally among them.If every trajectory has a unique feature vector, this finite-sample rule recovers Boltzmann choice.Source: §2.2.1 · p. 3

2.2.2 Softening to Feature Similarity.

• LESS replaces exact attribute intensity with a soft similarity measure and extends the rule to a probability density over continuous trajectories.The measure is defined over features and trajectories.Source: §2.2.2 · p. 3
• A trajectory’s selection likelihood is inversely proportional to its feature-space similarity with other trajectories, de-weighting those similar to many alternatives.Source: §2.2.2 · p. 3

2.3 Similarity as Density

• Nearby trajectories in feature space reweight trajectory likelihoods; the integral of similarity over trajectories acts like a measure of trajectory density.The model estimates similarity using a kernel function and KDE.Source: §2.3 · p. 3
• The experiments use a radial basis function that peaks when x = ϕ(ξ) and decreases exponentially as feature-space distance grows.The function is selected as a kernel for KDE.Source: §2.3 · p. 3
• Higher bandwidth σ makes trajectories appear more similar for a given feature difference.σ determines how much that difference affects the similarity evaluation.Source: §2.3 · p. 3
• The optimal bandwidth σ* is selected by maximizing the summed log similarities of a finite trajectory sample set, equivalent to maximizing its KDE likelihood.Source: §2.3 · p. 3

2.4 Inference and Prediction with LESS

• To predict behavior under a belief b(θ), LESS marginalizes over reward parameters θ.Source: §2.4 · p. 3
• Given a human trajectory, LESS updates its belief over reward parameters using Bayesian inference.Source: §2.4 · p. 3
• When denominator integrals are intractable, experiments use discretized θ parameters and finite trajectory samples; trajectory-space sampling can significantly affect inference.Source: §2.4 · p. 3

3 LESS AS A HUMAN DECISION MODEL

• The study uses a browser-based navigation task to compare model predictions with participants’ trajectory choices.Participants were asked to make behavior decisions in a grid from start to goal.Source: §3 · p. 3, §3.1 · p. 3

3.1.1 Main Design Idea.

• The control trial measures relative preference for two dissimilar optimal routes; the experimental trial adds two optimal routes similar to the right-side option.The symmetric control routes were designed to reduce confounds such as color, turns, and obstacle distance.Source: §3.1.1 · pp. 3–4

3.1.2 Manipulated Variables.

• Adding similar right-side trajectories makes Boltzmann’s predicted left-choice probability fall, while LESS keeps it closer to the control value.For equally liked sides, Boltzmann’s left probability falls from .5 to .25; LESS accounts for similarity among the right-side trajectories.Source: §3.1.2 · p. 4

3.1.3 Dependent Measures.

• The dependent measure is each trajectory’s selection proportion in the experimental trial, used to compute agreement with model predictions.Source: §3.1.3 · p. 4

3.1.4 Subject Allocation.

• The study recruited 80 participants and excluded 3 for failing the attention test; treatment assignment was between-subjects.Source: §3.1.4 · p. 4

3.1.5 Hypotheses.

• The hypotheses predicted a difference between observed and Boltzmann proportions, but equivalence between observed and LESS proportions.Source: §3.1.5 · p. 4

3.2 Analysis

• Observed Left choice was .41; Boltzmann predicted .23 and LESS .475, and only Boltzmann differed significantly from observed Left-Right choices.The chi-square tests were X^2(1, N = 37) = 6.27, p < 0.05 for Boltzmann and X^2(1, N = 37) = 0.72, p = 0.4 for LESS.Source: §3.2 · p. 4, Figure 2 · p. 4
• Across all trajectories, at α = 0.05, minimum equivalence bounds were 0.22 for LESS and 0.39 for Boltzmann; observations differed significantly from Boltzmann (χ²(1, N = 37) = 9.72, p < 0.05), but not LESS (χ²(1, N = 37) = 5.76, p = 0.12).Source: §3.2 · p. 5

4.1 Boltzmann and LESS inference comparison

• The 2-by-2 experiment sampled demonstrations from Boltzmann or LESS across eight objectives and 150 seeds, yielding 1200 demonstration sets per sampling method.Inference was then performed using either choice model.Source: §4.1 · p. 5, §4.1.1 · p. 5, §4.1.2 · p. 5

4.1.3 Dependent Measures.

• Inference quality was measured by whether the MAP objective matched the ground truth and by posterior probability mass assigned to that objective.TrueMatch is binary; TruePosterior provides the softened measure.Source: §4.1.3 · p. 5

4.1.5 Analysis.

• Matching the inference model to the sampling model produced significantly higher TruePosterior and TrueMatch results in both sampling conditions.The sampling-by-inference interaction was significant for TruePosterior (F(1, 1199) = 965.06, p < 0.001) and TrueMatch (p < .001); post-hoc comparisons favored matching models.Source: Figure 3 · p. 5, §4.1.5 · p. 6

4.2 Qualitative analysis of LESS inference

• Boltzmann can under-learn from high-density demonstrations and over-learn from sparse ones, whereas LESS weights trajectories inversely by feature density.This weighting makes the resulting feature density approximately uniform, reducing the influence of similar alternatives on trajectory probability.Source: §4.2 · p. 6
• For ΞL, which lies in a high-density region, LESS confidently recovers the ground truth while Boltzmann retains a higher-entropy posterior.Figure 4 visualizes the contrasting inferred posteriors.Source: §4.2 · p. 6, Figure 4 · p. 6
• For ΞB, LESS supports two preference hypotheses, while Boltzmann emphasizes the sparse-region demonstrations and confidently selects one.The LESS hypotheses differ on whether to avoid both obstacles or only the top one; the selected Boltzmann hypothesis is correct in this example, though density sensitivity can also produce over- or under-learning.Source: §4.2 · pp. 6–7

4.3.1 Experimental Design.

• The experiment added average x- and y-coordinate features that affected similarity but not reward, while the robot performing inference did not know those features.The comparison used LESS or Boltzmann inference on demonstrations generated with LESS.Source: §4.3.1 · p. 7

4.3.2 Analysis.

• LESS inference significantly outperformed Boltzmann on TruePosterior and had greater odds of TrueMatch under feature misspecification (p < .001 for both).Source: §4.3.2 · p. 7
• In the worst case, an unspecified feature that completely differentiates human options can make LESS-generated behavior approach Boltzmann, under which Boltzmann inference could be superior.The authors qualify this as a worst-case scenario and report that LESS remained preferable in their practical, non-adversarial experiment.Source: §4.3.2 · p. 7

5.1.7 Analysis.

• In the table task, both methods produced the same θMAP across two 100-sample trajectory sets, but LESS posteriors varied less and had lower KLAggregate.The human task differed: its interaction was significant, but a post-hoc difference appeared only at sample size 1000; authors describe it as a more ambiguous inference problem.Source: Figure 7 · p. 7, §5.1.7 · p. 8

5.1.2 Manipulated Variables.

• Inference compared Boltzmann and LESS using trajectory sets of 10, 30, 100, 300, or 1000 samples, with ten sets per size.Source: §5.1.2 · pp. 7–8

5.1.4 Dependent Measures.

• Robustness was measured by aggregate Kullback-Leibler divergence among posterior distributions from ten trajectory sets for each task, sample size, method, and user.Source: §5.1.4 · p. 8

5.1.6 Subject Allocation.

• The real-world study used 12 participants to demonstrate three mug-carrying tasks: near the table, away from a laptop, and away from the human body.Participants interacted with a JACO 7DOF arm after training in gravity-compensation mode.Source: §5.1.6 · p. 8, §5.1.3 · p. 8

5.1.7 Analysis.

• For the laptop task, LESS had lower KLAggregate than Boltzmann at S = 10, 30, and 100, with no significant difference at S = 300 or 1000; batch inference showed the opposite pattern.In batch inference, LESS had higher divergence, hence lower robustness; Figure 6 reports the laptop and batch comparisons.Source: Figure 6 · p. 7, §5.1.7 · p. 8, §5.2 · p. 8

5.2 All demonstrations inference

• For batch inference, aggregating all users’ demonstrations made Boltzmann more robust than LESS in this study.The authors attribute this to multiple demonstrations increasing the chance that some lie in low-density regions, which can dominate Boltzmann inference when they suggest the same reward.Source: §5.2 · p. 8

• LESS better captures human decision making and attenuates inference errors that arise from similar selections, increasing accuracy and robustness.Source: §6 · p. 8
• LESS relies on pre-specified robot features for similarity selection, leaving feature-misspecification effects unclear and requiring further experiments.The authors note that better inference in Section 4.3 may reflect hypothesis H3 rather than robustness to misspecification.Source: §6 · p. 8
• In aggregate results from 12 people, LESS can yield less robust inference, and it remains unclear whether this reduces accuracy or favors Boltzmann with enough varied demonstrations.Source: §6 · p. 8
• The Mechanical Turk study uses simplistic human-choice datasets; more realistic studies would be useful but are complicated by the lack of ground-truth rewards.Source: §6 · p. 8
• The authors argue that a continuous-domain counterpart to Boltzmann rationality is needed for robot inference and prediction.Source: §6 · p. 8