Appendix 1 AHP rating scale, right Eigenvector and consistency ratio calculation
AHP rating scale
In this study, the two sided 9point ratio scale that was proposed by Saaty was used [12, 15]. While each point on the scale has an ordinal verbal interpretation to facilitate judgements, the numerical (ratio scale) values are used in the AHP weight calculations using matrix multiplication and the right Eigenvector approach.
The scale to rate pairwise comparisons looks as follows:
The verbalization of numerical judgements is as follows:
1: criterion i is equally important or preferred by the patient as criterion j
3: criterion i/j is moderately more important or preferred compared to criterion j/i
5: criterion i/j is much more important or preferred compared to criterion j/i
7: criterion i/j is very much more important or preferred compared to criterion j/i
9: criterion i/j is extremely more important or preferred compared to criterion j/i
2, 4, 6, 8: intermediate values
Matrix Multiplication / Right Eigenvector calculation
All pairwise comparisons resulting from the AHP survey are transfered to a comparison matrix A = [aij]. Values at the upper right side of the matrix‘s diagonal are the result of actual pairwise comparisons; values at the lower left side of the matrix are the reciprocal values since the AHP relies on the preference reciprocity assumption. Local importance weights in AHP are calculated using the principal right eigenvector approach as suggested by Saaty. The calculated right eigenvector represents the vector of weights (w) of included criteria / subcriteria. The right eigenvector multiplied by some matrix A is, in case of a non negative reciprocal matrix A, equal to the matrix‘s maximal eigenvalue λmax multiplied by w (A*w = λmax*w). Based on this relationship, the right eigenvector may be calculated by e.g. using the matrix multiplication method [16]. In practical terms, this process is „a simple averaging process by which the final weights are the average of all possible ways of comparing the scores on the pairwise comparisons“[11].
Matrix multiplication method
The following steps were followed to calculate the right eigenvector:
- edit the comparison matrix A
- multiply matrix A by itself
- resulting matrix: calculate sum of each matrix row
- calculate the sum of matrix rows
- normalize matrix row sums over the sum of row sums
- the result of step 5 is the right eigenvector an the matrix
- repeat the matrix multiplication procedure as described in steps 2-6 until the difference between the respective calculated eigenvectors becomes minimal (<0.0001)
- if the difference is < 0.001 – this last calculated eigenvector is the vector of weights
Aggregation of individual judgements / weights
In our study, importance weights for each individual were calculated based on individual judgements. For data aggregation, these individual weights were then averaged using arithmetic mean calculation. This preference aggregation mode – the so-called aggregation of individual priorities (AIP) - was chosen to reflect the individuality and variation in individual patient judgements. In most studies, if AHP is used as a group decision making tool, a different aggregation mode is recommended, the aggregation of individual judgements (AIJ). For the calculation of group weights, individual judgements resulting from pairwise comparisons are, according to that mode, averaged (geometric means) and group weights are then calculated based on the combined judgements. The geometric mean is – for such kinds of study - the only aggregation method that assures that the reciprocal axiom of AHP holds for the combined judgments in a matrix of combined judgments (33). However, since in our study focused on individual judgements and priorities, the aggregation of individual priorities seemed to be the more appropriate aggregation method.
Consistency Ratio Calculation
The AHP allows to calculate the logical consistency of pairwise judgements within a cluster of judgements. The concept of consistency relies on another basic assumption of the AHP besides reciprocity: the transitivity of preferences. Meaning that if A>B (A preferred to B) and B>C then A>C. However, while transitivity is a necessary condition for consistency, AHP does not require that preferences are neither transitive nor consistent. The so-called consistency ratio (CR) measures how plausible each pairwise comparison is with respect to the other comparisons in the cluster and how much the measured consistency of a matrix, the consistency index (CI of matrix A), differs from the average CI of a simulated set of reciprocal but totally random pairwise comparison matrices (so called RI – random index). The closer CI and RI are, the higher the CR and the greater is the probability that judgements in a comparison matrix are the result of a completely random decision making process. To calculate the so-called „consistency index“(CI) of a matrix, first the maximum eigenvalue of that matrix, λmax, needs to be known. If a pairwise comparison matrix consists of perfectly consistent judgements, λmax is equal to n. λmax is the average of the consistency vector, which is the weighted row sum vector of a matrix A (weighted by the calculated right eigenvector) (for details see [16, 35]). Once λmax is known, the consistency index of matrix A is calculated by the following formula:
CI =
The CI is then to be analyzed in relation to random index RI – which is the average CI of a simulated set of random reciprocal matrices having AHP scale values between 1/9 and 9. For 3 elements in a matrix, RI has been estimated to be equal to 0.58, for 4 elements 0.9 [15, 35]. The formula is CR = . The more the CI deviates from RI the smaller is the consistency ratio and the better is the consistency in observed pairwise comparisons. The aggregated consistency ratios at the criteria level (and for the overall model) were calculated using the CIs of each comparison matrix multiplied by the weights of the respective criteria (and subcriteria) and were then added to each other to calculate the average CIs. The average RI’s were calculated accordingly. Based on average CI and RI, the aggregated CRs are then calculated by dividing average CI by average RI.
1