Source-linked AI summary
Axiomatic properties of inconsistency indices for pairwise comparisons
Matteo Brunelli, Michele Fedrizzi
TL;DR
Pairwise comparisons represent subjective preferences, but the reliability of decisions is linked to the consistency of the judgments and the suitability of the inconsistency measure. This paper introduces five axioms for characterizing inconsistency indices and examines literature indices against them. It proves that some indices satisfy all five axioms, while others do not.
Problem
Existing inconsistency indices were introduced heuristically and independently, without a general definition or shared axiomatic properties, despite consistency being linked to decision dependability.
Method
The paper introduces five axioms for inconsistency indices and evaluates proposed literature indices against those axioms.
Results
Some literature indices satisfy all five axioms, whereas others fail to satisfy them.
Takeaways & Limitations
The axioms provide requirements for assessing whether inconsistency indices correctly evaluate deviations from consistency.
Takeaways & Limitations
The paper assumes that consistent matrices share the minimum index value and reverses indices with the opposite orientation by changing their sign.
Abstract
from arXiv · showhide
Pairwise comparisons are a well-known method for the representation of the subjective preferences of a decision maker. Evaluating their inconsistency has been a widely studied and discussed topic and several indices have been proposed in the literature to perform this task. Since an acceptable level of consistency is closely related with the reliability of preferences, a suitable choice of an inconsistency index is a crucial phase in decision making processes. The use of different methods for measuring consistency must be carefully evaluated, as it can affect the decision outcome in practical applications. In this paper, we present five axioms aimed at characterizing inconsistency indices. In addition, we prove that some of the indices proposed in the literature satisfy these axioms, while others do not, and therefore, in our view, they may fail to correctly evaluate inconsistency.
1 Introduction
Pairwise comparisons simplify preference elicitation and support deriving alternative priorities, but the reliability of resulting decisions is tied to judgment consistency. The paper addresses the lack of general axioms for inconsistency indices by introducing five properties and examining existing indices against them.
- Motivation: Pairwise comparisons reduce decision complexity by letting decision makers compare two alternatives at a time.They also support deriving a priority vector that rates the alternatives.
- Motivation: Pairwise comparisons are used in decision-analysis methods including the Analytic Hierarchy Process and its generalizations.
- Motivation: Decision dependability is assumed to be related to the consistency of pairwise judgments.Near-consistent judgments are presented as sufficiently good approximations of decision makers’ real preferences.
- Contribution: The paper introduces five axiomatic properties because inconsistency indices were developed heuristically without a general definition or shared axiomatic basis.It then evaluates literature indices and shows that some satisfy the axioms while four others do not.
2 Preliminaries
The paper models pairwise judgments as positive reciprocal matrices and distinguishes fully consistent matrices from matrices with varying inconsistency. Consistency supports priority derivation through methods such as geometric means and eigenvectors, while inconsistency indices quantify deviation from full consistency.
- Pairwise comparison matrices: A pairwise comparison matrix is a positive reciprocal square matrix whose entries estimate preferences between alternatives.Reciprocity requires aijaji = 1, and the matrix order is n > 2 in the defined matrix set.
- Consistency: A matrix is consistent when direct comparisons are confirmed by all corresponding indirect comparisons.For consistent matrices, a priority vector exists and represents the coherent preference structure.
- Priority derivation: The geometric mean and eigenvector methods can obtain priority vectors from pairwise comparison matrices.They yield the same priority vector for consistent matrices but may differ when matrices are inconsistent.
- Consistency: The set of consistent matrices is defined as a subset of the set of all pairwise comparison matrices.
- Inconsistency measurement: An inconsistency index is a real-valued function intended to indicate how much a matrix deviates from full consistency.Matrices may be assigned degrees of inconsistency rather than only classified as consistent or inconsistent.
3 Inconsistency indices
The paper reviews a broad set of inconsistency indices, including eigenvalue-, distance-, determinant-, rank-, harmonic-, max-min-, and fuzzy-scale-based approaches. These indices differ in how they quantify departures from the theoretical values or structural properties of consistent pairwise comparison matrices.
- Eigenvalue-based indices: Saaty’s Consistency Index uses the principal eigenvalue of a pairwise comparison matrix to measure inconsistency.The maximum eigenvalue equals n exactly for consistent matrices and is greater than n otherwise.
- Eigenvalue-based indices: The Consistency Ratio extends eigenvalue-based assessment with normalization by the Random Index.
- Distance-based indices: The GW index computes deviations between normalized matrix entries and theoretical ratios wi/wj.Its procedure normalizes matrix columns and the associated priority vector before calculating deviations.
- Distance-based indices: The Geometric Consistency Index and Relative Error quantify inconsistency through deviations or squared errors relative to theoretical preference ratios.The Relative Error is defined for matrices other than the all-ones matrix and is zero for that matrix.
- Alternative indices: The review also covers indices developed for fuzzy judgments, Abelian linearly ordered groups, parametric assessment, and ambiguity.
4 Axioms
The paper introduces five axioms for evaluating inconsistency indices, addressing the lack of general definitions and axiomatic criteria. The axioms constrain how indices should treat consistency, relabeling, preference intensification, localized deviations, and continuity.
- Motivation: The five axioms are proposed to narrow the general definition of inconsistency index and assess whether existing indices satisfy minimal reasonable requirements.The authors use examples and propositions to demonstrate the necessity and consequences of the axioms.
- Axiom 1: Consistency value: A1 requires all consistent matrices to receive one unique real value, taken as the index’s minimum value under the paper’s convention.This distinguishes consistent from inconsistent matrices, although some indices may require sign reversal to meet the convention.
- Axiom 2: Permutation invariance: A2 requires invariance under simultaneous row-column permutations, so inconsistency does not depend on the ordering of alternatives.Formally, the index must assign the same value to A and PAPT for any permutation matrix P.
- Axiom 3: Intensification monotonicity: A3 requires inconsistency not to decrease when preferences are intensified by the unique continuous reciprocity- and consistency-preserving transformation f(aij) = aij^b with b > 1.The transformation moves non-indifference entries farther from 1 while preserving reciprocity and consistency; therefore I(A(b)) ≥ I(A).
- Axioms 4–5: Local deviation and continuity: A4 requires inconsistency to increase monotonically when a single comparison in a consistent matrix is deviated from its original value, while A5 requires continuity.In Example 4, A′′ must satisfy I(A′′) ≥ I(A′) ≥ I(A); discontinuity can otherwise assign maximal inconsistency arbitrarily close to consistency.
- Logical status and implications: Axioms A1–A5 are logically consistent and independent, but four seemingly reasonable literature-based indices fail at least one of them.The paper gives separate constructions showing independence, including indices satisfying A1–A4 but not continuity and an index satisfying A2–A5 but not A1.
5 On the satisfaction of the axioms
The paper tests several inconsistency indices against five axioms, finding that CI, CI* and GCI satisfy all five while RE, NIσ_n, HCI and GW violate selected properties.
- CI, CI* and GCI satisfy all five axioms A1–A5.
- RE satisfies A1–A3 but fails A4 and A5, disproving its previously claimed continuity.
- NIσ_n satisfies A1, A2 and A5 but violates A4 because its value can decrease as a14 increases above 1/9.For example, NI9_4(A) at a14 = 2 is smaller than at a14 = 0.5.
- HCI satisfies A1, A2, A4 and A5 but fails A3: its value first increases, then decreases, and converges to full HCI-consistency as b grows.
- GW satisfies A1, A2 and A5, but with geometric-mean priorities it fails A3; whether it satisfies A4 remains unproved.
6 Discussion and Future Research
The discussion identifies A3 and A4 as the most demanding axioms and uses a geometric interpretation to clarify A4’s role. It concludes by noting that the relationship between inconsistency indices and priority-vector computation remains open for future research.
- Discussion: A3 and A4 are identified as the most demanding axioms among A1–A5.
- Discussion: Table 1 summarizes which proposed inconsistency indices satisfy, violate, or leave unknown the five axioms.
- Discussion: The geometric interpretation represents a consistent matrix as a point in R^n(n−1)/2 and interprets entry changes as departures from consistency.
- Future Research: The paper studies inconsistency independently of priority-vector computation, leaving their relationship for future investigation.
- Conclusions: The paper introduces axiomatic properties to organize consistency evaluation and reports that some literature indices fail to satisfy them.
Appendix
The appendix supplies proofs that several indices satisfy the proposed axioms and that others violate specific axioms. The arguments use eigenvalue properties, continuity, permutation invariance, and monotonicity analyses of perturbed matrices.
- Proof structure: The appendix also examines axiom independence and records proof strategies for the remaining proposed indices and counterexamples.
- Proofs for CI: Saaty’s CI satisfies the axioms through eigenvalue invariance under permutation, convexity under preference intensification, and continuity.
- Proofs for CI∗: CI∗ satisfies A1–A5 by analyzing its triplet-based terms, permutation symmetry, perturbation monotonicity, and continuity.
- Proofs for GCI: GCI satisfies A1 through the weight-ratio characterization of consistency and satisfies A4 through a perturbation analysis of its summed terms.
- Counterexamples: RE fails A4 because increasing one comparison does not make its inconsistency value monotonic, and it fails continuity at the all-ones matrix.
- Counterexamples: HM fails A3 because an inconsistent sequence can make HCI tend to zero, despite the initial matrices having positive inconsistency.