Estimating the robustness of composite CBA & MCA assessments by variation of criteria importance order

Anders Vestergaard Jensen, Michael Bruhn Barfod, Steen Leleur

    Research output: Contribution to conferenceConference abstract for conferenceResearchpeer-review


    This paper discusses the concept of using rank variation concerning the stake-holder prioritising of importance criteria for exploring the sensitivity of criteria weights in multi-criteria analysis (MCA). Thereby the robustness of the MCA-based decision support can be tested. The analysis described is based on the fact that when using MCA as a decision-support tool, questions often arise about the weighting (or prioritising) of the included criteria. This part of the MCA is seen as the most subjective part and could give reasons for discussion among the decision makers or stakeholders. Furthermore, the relative weights can make a large difference in the resulting assessment of alternatives [1]. Therefore it is highly relevant to introduce a procedure for estimating the importance of criteria weights. This paper proposes a methodology for estimating the robustness of weights used in additive utility models. When assessing larger transport infrastructure projects often several non-monetised impacts could be relevant to include in the appraisal [2]. For many decision makers and stakeholders the task of setting the criteria weights for several criteria can be very difficult. To overcome this, the proposed method uses surrogate weights based on rankings of the criteria, by the use of Rank Order Distribution (ROD) weights [3]. This reduces the problem to assigning a rank order value for each criterion. A method for combining the MCA with the cost-benefit analysis (CBA) is applied as described by Salling et al. in [4]. This methodology, COSIMA, uses a calibration indicator which expresses the trade-off between the CBA and MCA part resulting in a total rate expressing the attractiveness of each alternative. However, it should be mentioned that the proposed procedure for estimating the importance of criteria weights is not limited to the ROD and COSIMA methods described above. The proposed framework is applied to the case of choosing the best corridor for a high speed railway in Sweden, between Linköping and Norrköping. This link is a part of a larger railway project called Ostlänken. 4 possible corridors have been indentified between the two cities and the preliminary studies have found 8 non-monetised criteria, which all are not included in the conducted CBA. The alternatives are compared to each other with respect to each criterion by using the REMBRANDT methodology. With 8 criteria there are 40340 (8!) possible combinations of ranking the criteria which have been made use of. The proposed method calculates all combinations and produces a set of rank variation graphs for each alternative and for different values of the trade-off indicator. This information is relatively easy to grasp for the decision-makers. The result is compared with the results from a conducted decision conference about the railway link. During the decision conference the different stakeholder preferences were unveiled by the participants who had to assign weights. The proposed method also introduces a more constrained approach. In this approach the stakeholders/decision makers have the possibility to set up some constraint to the decision problem. This could for example be that criterion XX cannot assume a rank lower than 3 or criterion XX always have to be ranked higher than criterion YY. This would mean that the outcome of the method is a subset of the total solution space. The paper finishes up with a discussion and considerations about how to present the results. The question whether to present a single decision criterion, such as the benefit-cost rate or the net present value, or instead to present graphs showing the robustness of the decision analysis is discussed. Furthermore a perspective, for estimating the robustness of weights using other MCA methodologies (and weighting methods) than the proposed framework, is discussed.
    Original languageEnglish
    Publication date2009
    Publication statusPublished - 2009
    EventThe 20th International Conference on Multiple Criteria Decision Making - Chengdu, China
    Duration: 1 Jan 2009 → …


    ConferenceThe 20th International Conference on Multiple Criteria Decision Making
    CityChengdu, China
    Period01/01/2009 → …


    • Decision-support tool
    • Stakeholder preference
    • Rank order distribution
    • COSIMA
    • Multi-criteria analysis


    Dive into the research topics of 'Estimating the robustness of composite CBA & MCA assessments by variation of criteria importance order'. Together they form a unique fingerprint.

    Cite this