Source-linked AI summary
Probabilistic coherence and proper scoring rules
Joel Predd, Robert Seiringer, Elliott H. Lieb, Daniel Osherson, Vincent Poor, Sanjeev Kulkarni
TL;DR
The paper asks how probabilistic coherence relates to domination by rival forecasts under proper scoring rules. It gives a self-contained proof of their equivalence, while leaving the generalized discontinuous case open because differentiability may be necessary.
Problem
The paper addresses whether violating probability-calculus constraints is equivalent to being dominated by a rival forecast under proper scoring rules.
Method
The authors formalize forecasts over a finite vector of events and prove the result through propositions using proper scoring rules and Bregman-divergence methods.
Results
For proper scoring rules, coherent forecasts are not weakly dominated, while incoherent forecasts are strongly dominated by coherent forecasts.
Takeaways & Limitations
The theorem connects probabilistic coherence with non-domination and provides a self-contained account relating coherence, Bregman divergences, and proper scoring rules.
Takeaways & Limitations
Whether the theorem extends to the generalized discontinuous scoring-rule case remains open because the proof generally requires differentiability of Φ.
Abstract
from arXiv · showhide
We provide self-contained proof of a theorem relating probabilistic coherence of forecasts to their non-domination by rival forecasts with respect to any proper scoring rule. The theorem appears to be new but is closely related to results achieved by other investigators.
1 Introduction
The paper identifies two defects in probabilistic forecasts—guaranteed inferiority to a rival and violation of probability constraints—and shows that proper scoring rules make these defects equivalent.
- 1 Introduction: Scoring rules assess probability estimates by assigning lower scores to probabilities closer to whether events occur.The summed scores across events are called the forecast’s penalty.
- 1 Introduction: A forecast may be defective because another forecast has a lower penalty in every possible outcome.
- 1 Introduction: A forecast may also violate probability-calculus constraints, such as assigning an included event a lower probability than the event containing it.
- 1 Introduction: The paper shows that, for a broad class of proper scoring rules, forecast incoherence and domination by a rival are equivalent.The result is presented as Theorem 1 and supported by a self-contained proof.
2 Intuitive account of concepts
The intuitive account explains proper scoring rules through expected penalties and illustrates how incoherent forecasts can be dominated, while situating the result within earlier work and its connection to Bregman divergences.
- 2 Intuitive account of concepts: With a proper scoring rule, announcing the probability matching one’s belief uniquely minimizes expected penalty.For the squared-deviation rule, a true probability p makes x = p the unique minimizer.
- 2 Intuitive account of concepts: The absolute-deviation rule is improper because expected penalty can fall from .48 to .47 when announcing .65 instead of the believed probability .6.
- 2 Intuitive account of concepts: Earlier work linked coherence and domination under quadratic or generalized scoring rules, but Lindley’s transformation complicates the interpretation.
- 2 Intuitive account of concepts: The paper offers a self-contained account connecting coherent forecasts, Bregman divergences, and domination under proper scoring rules.The treatment presupposes only elementary analysis.
3 Framework and Main Result
The framework defines forecasts, coherence, proper scoring rules, and penalty-based domination, then states that coherence exactly characterizes resistance to rival forecasts under proper scoring rules.
- Definitions: A forecast is coherent exactly when its probabilities match a probability measure on the fixed events over the sample space.
- Scoring rules: Proper scoring rules uniquely minimize expected score when the announced probability equals the underlying probability, encouraging candid reporting.
- Scoring rules: The penalty P_s sums the scores assigned to all events for a forecast and realized outcome.
- Domination: A forecast is weakly dominated when a rival never incurs a higher penalty, and strongly dominated when the rival always incurs a lower penalty.
- Main result: Theorem 1 states that coherent forecasts are not weakly dominated, whereas every incoherent forecast is strongly dominated by some coherent forecast.
- Main result: Consequently, distinct coherent forecasts cannot weakly dominate one another, and weak domination occurs exactly when strong domination by a coherent forecast occurs.
4 Three Propositions
Three propositions connect coherence with convex geometry, proper scoring rules with strictly convex functions, and Bregman divergences with projection inequalities.
- Proposition 1: The event relations can reduce the number of distinct outcome vectors below 2^n, making the convex-hull representation especially relevant.
- Proposition 1: Coherence is characterized geometrically: a forecast is coherent if and only if it lies in the convex hull of event-outcome vectors.
- Proposition 2: Each proper scoring rule corresponds to a bounded, continuous, strictly convex function, and suitable such functions conversely generate proper scoring rules.
- Bregman divergences: A Bregman divergence is nonnegative and vanishes only when its two arguments coincide.
- Proposition 3: Projection of a point onto a closed convex set yields a unique point satisfying an inequality that bounds divergence to every point in the set.
5 Proof of Theorem 1
The proof maps scoring penalties to Bregman divergences and uses convex projection to establish strict improvement for incoherent forecasts and uniqueness of undominated coherent forecasts, including unbounded scores.
- Bounded scoring rules: For bounded scoring rules, an incoherent forecast projects onto conv(V) at a coherent forecast whose divergence, and therefore penalty, is strictly lower in every outcome.
- Bounded scoring rules: If a coherent forecast were weakly dominated, divergence inequalities would force the rival to equal it, proving non-domination.
- Unbounded scoring rules: For unbounded scoring rules, the argument handles finite-gradient boundary points through a unique convex minimizer and the corresponding projection inequality.
- Unbounded scoring rules: When forecasts lie on problematic boundary faces, induction supplies an improving coherent forecast on the face before a small perturbation makes all penalties finite.
- Unbounded scoring rules: The proof of non-domination for coherent forecasts extends to boundary cases because coherent convex combinations involve only outcome vectors at finite divergence.
6 Proofs of Propositions 1–3
The proofs establish the convex-analytic structure underlying proper scoring rules and connect it to the characterization of coherent forecasts. They derive convexity, differentiability, endpoint behavior, and Bregman projection properties used in the theorem.
- Proposition 1: Coherent forecasts correspond to convex combinations of the event-pattern vectors induced by the minimal non-empty regions of the event partition.The partition provides the probability weights, while the corresponding vectors generate the convex hull containing exactly the coherent forecasts.
- Lemma 1: The technical lemma establishes finite endpoint limits for bounded convex ϕ and monotone derivative limits, including the boundary condition in Eq. (6).These limits support the endpoint arguments in the proof of Proposition 2.
- Proposition 2: Proper scoring rules induce a bounded, continuous, strictly convex function ϕ whose derivative recovers the score difference ψ(p) = s(0, p)−s(1, p).The proof uses unique minimization to obtain strict convexity and continuity of the scoring rule to establish differentiability through the interior.
- Proposition 2: Strict convexity yields the supporting-line inequality ps(1, x) + (1 −p)s(0, x) ⩾−ϕ(p), with equality only when x = p.Boundary cases are handled separately using continuity, endpoint derivative limits, and strict convexity.
- Proposition 3: For a Bregman divergence, strict convexity guarantees a unique projection πx onto any closed convex set, and directional optimality supplies the projection inequality.The proof differentiates along line segments from πx toward arbitrary points in the constraint set.
7 Generalizations
The paper extends its results to heterogeneous and non-additive penalty structures, weaker uniqueness assumptions, and discontinuous scoring rules. The discontinuous case yields a characterization result but leaves the main theorem unresolved.
- 7.1 Penalty functions: Theorem 1 extends to event-specific proper scoring rules and, through more general convex functions, to certain non-additive penalties.With distinct scores, the associated Bregman generator is Φ(x) = P_i ϕ_i(x_i).
- 7.2.1 Non-uniqueness: Without unique minimization, every incoherent forecast has a coherent forecast that weakly dominates it, but strong dominance can fail.The constant scoring rule s(i, x) ≡0 supplies a counterexample to strong dominance.
- 7.2.2 Discontinuity: For discontinuous scoring rules, Proposition 4 characterizes properness through a bounded convex function ϕ and a monotone function ψ satisfying the associated inequalities.Strict convexity corresponds exactly to strictness of the properness inequality away from x = p.
- 7.2.2 Discontinuity: Even when the component scores are discontinuous, the combined function ϕ remains continuous, constraining compensating jumps between s(0, x) and s(1, x).An upward jump in s(0, x) must be offset by a proportional downward jump in s(1, x).