PaperlayerView exact revision on arXiv

Source-linked AI summary

Generalised framework for multi-criteria method selection

Jarosław Wątróbski, Jarosław Jankowski, Paweł Ziemba, Artur Karczmarczyk, Magdalena Zioło

arXiv:1810.11078v1cs.AI

TL;DR

Choosing among many MCDA methods remains insufficiently addressed despite differing results across methods. This paper develops a generalized framework that models method characteristics and uncertainty, finding recommendations consistent with expert practice and relatively robust to partial information.

  • Problem

    The diversity of MCDA methods and inconsistent choices for similar problems create a need for generalized, area-independent method-selection solutions.

  • Method

    The framework analyzes 56 MCDA methods using hierarchical method characteristics, decision-problem descriptors, rules, and uncertainty handling to support method selection.

  • Results

    Recommendations were consistent with methods used by experts, while partial uncertainty in the decision description did not significantly change the recommended method set.

  • Takeaways & Limitations

    The framework provides a generalized basis for constructing a knowledge database to support MCDA method selection from decision-problem characteristics.

  • Takeaways & Limitations

    The rules may recommend only potential methods rather than a specific method and omit contextual factors affecting MCDA method selection.

Abstract

from arXiv · show

Multi-Criteria Decision Analysis (MCDA) methods are widely used in various fields and disciplines. While most of the research has been focused on the development and improvement of new MCDA methods, relatively limited attention has been paid to their appropriate selection for the given decision problem. Their improper application decreases the quality of recommendations, as different MCDA methods deliver inconsistent results. The current paper presents a methodological and practical framework for selecting suitable MCDA methods for a particular decision situation. A set of 56 available MCDA methods was analyzed and, based on that, a hierarchical set of methods characteristics and the rule base were obtained. This analysis, rules and modelling of the uncertainty in the decision problem description allowed to build a framework supporting the selection of a MCDA method for a given decision-making situation. The practical studies indicate consistency between the methods recommended with the proposed approach and those used by the experts in reference cases. The results of the research also showed that the proposed approach can be used as a general framework for selecting an appropriate MCDA method for a given area of decision support, even in cases of data gaps in the decision-making problem description. The proposed framework was implemented within a web platform available for public use at www.mcda.it.

1. Introduction

MCDA methods address increasingly complex, uncertain, and conflicting decision problems, but different methods can produce inconsistent recommendations. This paper therefore proposes a domain-independent framework for selecting suitable MCDA methods while handling incomplete problem descriptions and validating recommendations against expert practice.

  • Motivation: Increasingly complex decision problems involve many criteria, conflicting goals, stakeholders, uncertainty, and risk across diverse application areas.MCDA methods are used to address these characteristics in political, organizational, financial, and marketing decisions.
  • Problem: Different MCDA methods can produce conflicting rankings and decisions even with identical criterion weights and variant evaluations.This inconsistency makes appropriate method selection important for decision quality.
  • Research gap: Existing method-selection approaches are often restricted to a few well-known methods, one application field, or decision-makers’ prior knowledge of formal problem characteristics.These limitations are especially problematic when the decision situation cannot be described in detail.
  • Contributions: The research proposes a formal MCDA method-selection guideline that is independent of problem domain, uses extensive method characteristics, improves recommendation accuracy, and addresses knowledge gaps.The framework selects methods through characteristics of available MCDA methods and analyzes how gaps in the decision situation description affect selection.
  • Validation: The framework handles uncertainty in decision-situation descriptions, and practical examples in sustainable logistics and transport confirmed that its recommendations were consistent with expert knowledge.The authors report that partial uncertainty in a selected aspect did not significantly affect the recommended method set.

2. Literature review

The literature characterizes MCDA methods by problem type, preference and aggregation approach, and data properties. It also emphasizes that method selection is vital but often arbitrary, while existing selection studies cover limited method sets and problem classes.

  • MCDA method characteristics: MCDA problems may be continuous or discrete, with discrete problems addressed through utility/value-function or outranking methods.Utility/value methods model indifference and preference between variants, whereas outranking methods use preference thresholds to represent uncertainty in evaluations.
  • MCDA method characteristics: MCDA methods use single-criterion aggregation, outranking aggregation, or mixed approaches combining both decision-making schools.PCCA methods exemplify mixed or indirect approaches.
  • MCDA method characteristics: Methods also differ according to data properties, including quantitative or qualitative data, measurement scale, and certainty.Data may use cardinal or ordinal scales; cardinal scales can be interval or ratio types, and ratio data express relations between values.
  • Need for method selection: Selecting the MCDA method is a vital decision-making stage because applying an unsuitable method can produce different results for the same decision problem.Method choice depends on the decision issue and the decision-maker’s operational approach.
  • Prior method-selection research: Existing selection practices are often arbitrary, while prior research has used benchmarking, MCDA-based selection, and informal or formal problem structuring.The literature also notes that existing selection works considered comparatively limited sets of methods and that some guidelines apply only to particular problem classes.

3. The proposed framework for MCDA method selection · 3.1. Main assumptions

The proposed framework combines methodological analysis with practical verification to select suitable MCDA methods for specific decision situations. It is based on 56 MCDA methods or combinations, their hierarchical characteristics, and rules that accommodate incomplete problem descriptions.

  • 3. The proposed framework for MCDA method selection: The framework has separate methodological and practical components for MCDA method selection.The methodological component supports analysis and rule-base generation, while the practical component uses collected problem descriptors to present recommended methods.
  • 3. The proposed framework for MCDA method selection: The framework’s methodological rules and database support practical verification once the relevant problem descriptors are gathered.The two framework elements are required for practical verification and recommendation of a method subset.
  • 3. The proposed framework for MCDA method selection: The methodological component defines the considered MCDA methods and analyzes their properties.Characteristics for each method are obtained and presented in Table 2.
  • 3. The proposed framework for MCDA method selection: The practical component gathers decision-problem descriptors and presents a subset of recommended MCDA methods.The recommended subset is obtained after the methodological elements and rules database have been developed.
  • 3.1. Main assumptions: The decision problem is represented by alternatives, criteria, and criterial-performance efficiency.It is expressed as the three-element set (A, G, E), where E = G(A), and performance can also be represented as E(A).
  • 3.1. Main assumptions: 56 MCDA methods or combinations form the basis of the framework.Their characteristics contain nine descriptive properties organized hierarchically.
  • 3.1. Main assumptions: The framework accounts for situations in which decision-makers cannot fully define decision-problem descriptors or their method-characteristic needs.The paper explicitly notes that decision-makers do not always have full knowledge of the given decision problem.

3.2. Proposed decision problem descriptors and MCDA methods’ properties

The framework represents MCDA methods through a property matrix and decision problems through corresponding descriptors, enabling method requirements to be matched systematically. Decision problems can use up to nine hierarchical descriptors covering weights, performance scales, uncertainty, and decision problematic.

  • Descriptor–property correspondence: An i-element MCDA method set is represented by a property matrix, while each decision problem is described by a descriptor vector whose relevant subset matches method properties.The descriptor subset may have dimension no greater than the method-property vector and is used to identify compatible methods.
  • Decision-problem descriptors: Each decision problem can be described using a maximum of nine hierarchical descriptors selected by the decision maker.The first level addresses criterion weights, performance scale, uncertainty, and decision problematic; subsequent levels refine these characteristics.
  • Descriptor hierarchy: The hierarchy further specifies weight types, uncertainty aspects, ranking type, and whether data uncertainty concerns criteria, variants, or both.Uncertainty may concern input data or decision-maker preferences, with additional descriptors distinguishing criteria from variants and partial from complete ranking.
  • Method classification: The descriptors were encoded for all considered 56 MCDA methods, and including characteristics from all hierarchy levels divides the methods into relatively few groups.Table 2 uses 0 to denote a method’s lack of ability for a characteristic.

3.3. Formal representation of the MCDA methods’ properties

The paper formalizes MCDA-method properties as descriptors for classifier-based method selection. These descriptors encode weights, comparison scales, uncertainty, decision problem types, and ranking-order requirements.

  • Descriptor framework: The proposed descriptor set identifies MCDA methods’ properties for a particular decision situation and supports classifier-based selection.The approach is presented as a formal basis for selecting an MCDA method.
  • Weights and scales: Descriptor c1 checks whether weights of any kind are used, while c1.1 distinguishes qualitative, quantitative, and relative weights.Relative weights may be represented through a pairwise-comparison matrix W sized to the criteria set.
  • Weights and scales: Descriptor c2 distinguishes qualitative, quantitative, relative, or absent scales for comparing decision variants.Variant comparisons can be represented by performance differences or relative pairwise comparisons in matrix E.
  • Uncertainty: Descriptor c3 checks for data or preference uncertainty, and c3.1 identifies whether uncertainty concerns input data, preferences, or both.Further descriptors distinguish fuzzy criteria weights, variant performance, indifference thresholds, preference thresholds, or both.
  • Decision problem and ranking: Descriptor c4 identifies the decision problem, while c4.1 distinguishes whether ranking requires a partial or full order of variants.The framework includes problematic types involving ranking, choice, sorting, and equivalence classes of variants.

3.4. Presentation of the MCDA methods’ properties using tree representation

The MCDA method-selection framework can be represented as a decision tree based on descriptors of the decision problem. It uses set intersections for complete information and unions, combined with intersections, when the problem is incompletely defined.

  • Decision-tree representation: The applied classifier is presented as a decision tree that structures the process of selecting a suitable MCDA method from decision-problem descriptors.This representation is called a method selection tree.
  • Decision-tree representation: The tree selects subsets of MCDA methods according to information known to the decision maker about the decision problem.Set algebra is used to specify subsets satisfying the descriptors.
  • Complete problem information: With full information about weights, comparison scales, uncertainty, and decision problematic, the appropriate methods are determined through intersections of descriptor sets.For c 3.1.1 = 3, methods addressing uncertainty in both criterion weights and variants are selected using S3b ∩ S3c.
  • Incomplete problem information: When the decision problem cannot be fully defined, unions of sets incorporate more general descriptors at hierarchy levels 1 and 2.This vertical approach applies when information such as the scale for expressing weights is unknown.
  • Incomplete problem information: Adapting a method to an incompletely defined problem combines the vertical approach for missing information with the horizontal approach for information that is known.The framework therefore combines unions and intersections according to the completeness of the descriptors.

3.5. Rules database generation

The rules database uses method characteristics as conditional attributes and MCDA methods as decision attributes. Among 56 methods, 31 unique characteristic sequences define the selection rules, while incomplete problem information reduces recommendation quality.

  • Rules database generation: 56 MCDA methods produced 31 unique encoded-characteristic sequences, indicating that some methods share identical characteristics and that subsets of methods can be recommended.These sequences form the foundations of the MCDA selection rules set.
  • Rules database generation: The hierarchy’s third and most detailed level defines 31 rules for selecting MCDA methods according to decision-problem properties.The rules depend on descriptor values at successive hierarchy levels.
  • Rules database generation: Lack of knowledge about a particular descriptor level decreases the quality of the resulting method recommendation.The framework can still assign methods based on available descriptor values, but missing information weakens the recommendation.
  • Rules database generation: The rule-based representation separately codes both subsets and their intersection, unlike the tree structure’s set-based assignment.The tree structure uses descriptors c_3.1, c_3.1.1, c_3.1.2 and c_4 to assign methods to disjoint sets or their intersection.

3.6. Uncertainty handling in the decision problem description

The framework models incomplete knowledge of decision-problem descriptors by generating and filtering MCDA selection rules. Results show that deeper classifier hierarchies preserve selection precision while limiting the number of matching methods under uncertainty.

  • Uncertainty modelling: 450,000 possible rules were generated, then conflicting rules were filtered using validity criteria and a four-step extraction process.Conflicts included incompatible classifier values, such as c 1 = 0 and c 1. 1 = 3.
  • Three-level hierarchy: 2.1446 average matching methods resulted when one characteristic was unknown in the 3-level framework, while two unknown descriptors produced 131 non-empty rules and 2.5191 average matched methods.With one unknown characteristic, the matching range was 1 to 12 methods.
  • Three-level hierarchy: 5.2099, 8.0400, 14.4545 and 29 average matching methods resulted when 5, 6, 7 or 8 of 9 characteristics were unknown, respectively.The growth was mapped by a 4-degree polynomial function with R 2 equal to 0.9997, while the number of rules decreased as unknown characteristics increased.
  • Hierarchy comparison: Additional hierarchy levels increased rule-set precision, with correlations of 0.97 between 1-level and 2-level or 1-level and 3-level sets, and 1 between 2-level and 3-level sets.The three rule sets could therefore be used interchangeably when higher precision was expected.
  • Implementation: A selection algorithm and online prototype were implemented to support MCDA method selection under uncertainty in the decision-problem description.The application was made available at http://www.mcda.it.

4. Practical confirmation of the framework

The framework was practically verified against expert-selected MCDA methods in sustainable transport and logistics cases, showing high conformity and satisfactory selection accuracy. Deviations and missing recommendations were attributed mainly to inadequate problem analysis or atypical method use.

  • Validation cases: The framework was validated on reference cases involving sustainable transport, logistics, supplier selection, supply chains, location choice, and related sustainability decisions.Each case compared the framework’s recommendation with an MCDA method used in the reference literature.
  • Validation results: Table 5 showed a high level of conformity between literature-selected MCDA methods and methods recommended by the framework.The reference cases treated expert recommendations as the comparison basis.
  • Validation results: Seven cases received no MCDA recommendation, while two cases were associated with an incorrect method selection.The framework could not assign a method in seven cases, whereas problem characteristics led to a wrong choice in two.
  • Limitations: Equal criterion weights were formally interpreted as missing weights, so the rule base failed to identify a suitable method for case 29.The relevant properties returned 0 instead of 1 and 2, respectively.
  • Overall assessment: The study found satisfactory selection accuracy and attributed missing or incorrect classifications mainly to inadequate problem analysis and inappropriate MCDA use.The issue also occurred with popular methods such as AHP.

5. Summary

The paper develops a generalized, domain-independent framework for selecting MCDA methods from a complete set of 56 analyzed methods. It formalizes method characteristics, decision rules, and uncertainty handling while acknowledging that the current rules may recommend potential methods rather than one specific method.

  • Framework development: A literature review identified 56 up-to-date MCDA methods, whose properties supported a complete taxonomy and formal selection framework.The framework was presented through a decision tree and descriptors.
  • Framework applicability: The domain-independent framework provides formal guidelines and rules for selecting MCDA methods across different areas of multicriteria decision making.Its intended applicability is not restricted to a particular problem domain.
  • Uncertainty handling: Hierarchical descriptors and uncertainty-aware selection rules support method selection when knowledge about the decision problem or input data is limited.The work also analyzes how uncertainty influences the final set of recommended methods.
  • Practical support: The framework establishes a basis for a knowledge database containing rules that select methods from defined options using detailed decision-problem characteristics.The contributions include practitioner guidelines, an MCDA selection algorithm, and a web-based expert-system implementation.
  • Limitations: The current rules are not always able to recommend one specific MCDA method and may instead propose a selection of potential methods.This limitation is explicitly noted in the paper’s concluding discussion.
  • Future work: Future work includes adding group decision-making methods, expanding reference cases, and developing an ontology of decision problems and MCDA methods.The ontology could support method selection based on classification results and contribute to a complete expert system.

Supplementary material

The supplementary material summarizes how representative MCDA methods model preferences, structure decision problems, and aggregate evaluations into selections, rankings, or classifications. It also describes variants using pseudo-criteria, fuzzy numbers, qualitative scales, and utility functions.

  • Preference modeling and aggregation: AHP hierarchizes the decision problem, uses nine-degree pairwise comparisons for variants and criteria weights, and additively aggregates preference vectors into a synthesized criterion for ranking.The comparison matrices are aggregated into criterial preference vectors before final synthesis.
  • Preference modeling and aggregation: ANP generalizes AHP by representing criteria–variant connections, feedback, and horizontal criterion relations in a network aggregated through a Markov-chain Supermatrix.Unlike AHP, ANP constructs a net model rather than a hierarchy.
  • Outranking-based methods: Electre methods support selection, ranking, or sorting through outranking, concordance and discordance indexes, pseudo-criteria thresholds, distillation, or comparisons with artificial profiles.Electre IV does not define criterion weights, while Electre Tri compares variants with decision-maker-defined profiles.
  • Other aggregation approaches: Other methods accommodate mixed qualitative and quantitative assessments through normalization, ordinal or quantitative weights, preference indexes, decision rules, or an analytical global utility function.These approaches include common-scale aggregation, rule reliability, criterion-by-criterion reduction, weighted averages, global preference matrices, and MAUT utility-based ranking.
  • Fuzzy methods: Fuzzy variants of AHP, ANP, Promethee, SAW, TOPSIS, and VIKOR represent evaluations, weights, or decision attributes with fuzzy numbers to model imprecise preferences and distances.Fuzzy TOPSIS uses positive and negative ideal solutions in Euclidean space, with triangular or trapezoidal fuzzy attributes.

Supplementary material

The supplementary analyses show that increasing classifier-hierarchy complexity makes selection rules more precise, while increasing unknown method properties expands the set of matching methods and reduces the number of rules. They also describe rule extraction and an algorithm for querying matching methods under uncertain classifier values.

  • Influence of classifier hierarchy: 13 rules, or 27% of 48 possible single-level rules, remain after empty rule sets are omitted, returning 4.3077 methods on average.Before omission, all 48 rules return 1.6667 methods on average.
  • Influence of classifier hierarchy: 90% of the 240 two-level rules return empty sets, while the remaining 25 rules return 2.24 methods on average, approximately 5% fewer than in the single-level hierarchy.The most general rule returns 8 methods.
  • Influence of classifier hierarchy: 31 of 960 rules, or 3%, are non-empty for nine classifiers organized into three levels, returning 1.8065 methods on average and providing the most precise rules.The analysis compares single-level, two-level, and three-level classifier hierarchies.
  • Rule extraction and selection algorithm: The supplementary material specifies extracting valid rules and provides an algorithm that returns matching MCDA methods from classifier values that may be unknown.The procedure uses a database of methods with assigned classifier values and outputs an array of matching methods with their classifier values.
  • Unknown classifier values: With one unknown property in the three-level framework, matching methods range from 0 to 12, averaging 0.1445 across 1232 rules.As unknown properties increase from 2 to 4, average matched methods rise to 0.3113, 0.6193, and 1.2147, respectively.
  • Unknown classifier values: 29 matching methods are averaged when 8 of 9 properties are unknown, while the number of rules decreases as more properties become unknown; this growth follows a geometric progression with R2 = 0.9991.For 5, 6, and 7 unknown properties, averages are 2.6709, 5.9118, and 14.4545, respectively.

Supplementary material Section 9. Detailed analysis of the most cited MCDA method selection approaches

The detailed analysis catalogs MCDA methods cited across seven reference approaches, showing that method selection literature spans classical, fuzzy, outranking, utility-based, and other decision methods.

  • Method coverage: AHP is listed across all seven reference approaches, while ANP and ARGUS appear in fewer references.The table also includes fuzzy AHP and fuzzy ANP as separate methods.
  • Method coverage: ELECTRE variants from ELECTRE II through ELECTRE TRI are repeatedly represented among the cited method-selection approaches.The listed variants include ELECTRE II, ELECTRE III, ELECTRE IS, ELECTRE IV, and ELECTRE TRI.
  • Outranking methods: The catalog includes PROMETHEE I and II, as well as additional outranking methods such as ORESTE, QUALIFLEX, REGIME, and PRAGMA.Fuzzy PROMETHEE I and II are also listed separately.
  • Additional methods: Less frequently represented methods include COMET, IDRA, MACBETH, MAPPAC, NAIADE I and II, PACMAN, PAMSSEM I and II, TACTIC, DEMATEL, and REMBRANDT.The catalog also lists lexicographic, maximax, maximin, fuzzy maximin, MELCHIOR, and methods for extracting minimum and maximum attribute values.
  • Other MCDA families: The analyzed approaches also cover additive, utility, value, and distance-based methods, including SAW, MAUT, MAVT, SMART, TOPSIS, UTA, and VIKOR.The table separately lists Fuzzy SAW and Fuzzy TOPSIS.
Loading 1810.11078v1…