Comparison of qualitative vs quantitative probability ratings, impact ratings, and risk scores (N = 1,144)
| Location and difference statistics | |||
|---|---|---|---|
| Parameter | Risk probability | Risk impact | Risk score |
| Mean difference (Qual – Quant) | −0.01 | 1.11 | 4.62 |
| Median difference | 0 | 1 | 5 |
| Hodges-lehmann estimate | 0 | 1 | 5 |
| 95% Bootstrap CIa | [0, 0] | [1, 1] | [5, 5] |
| Location and difference statistics | |||
|---|---|---|---|
| Parameter | Risk probability | Risk impact | Risk score |
| Mean difference (Qual – Quant) | −0.01 | 1.11 | 4.62 |
| Median difference | 0 | 1 | 5 |
| Hodges-lehmann estimate | 0 | 1 | 5 |
| 95% Bootstrap CI | [0, 0] | [1, 1] | [5, 5] |
| Paired test | |||
|---|---|---|---|
| Test | Risk probability | Risk impact | Risk score |
| Wilcoxon signed-rank (Pratt) | W = 267,649 (p = 0.538) | W = 79,274 (p < 0.001) | W = 118,278 (p < 0.001) |
| Paired test | |||
|---|---|---|---|
| Test | Risk probability | Risk impact | Risk score |
| Wilcoxon signed-rank (Pratt) | W = 267,649 ( | W = 79,274 ( | W = 118,278 ( |
| Distributional test | |||
|---|---|---|---|
| Test | Risk probability | Risk impact | Risk score |
| Kolmogorov-Smirnov | D = 0.05 (p = 0.188) | D = 0.44 (p < 0.001) | D = 0.38 (p < 0.001) |
| KS permutation p-valueb | p = 0.075 | p < 0.001 | p < 0.001 |
| Distributional test | |||
|---|---|---|---|
| Test | Risk probability | Risk impact | Risk score |
| Kolmogorov-Smirnov | D = 0.05 ( | D = 0.44 ( | D = 0.38 ( |
| KS permutation | |||
| Distance-based tests | |||
|---|---|---|---|
| Metric | Risk probability | Risk impact | Risk score |
| Wasserstein-1 (W1) | 0.08 | 1.11 | 4.62 |
| 95% bootstrap CI (paired)a | [0.05, 0.13] | [1.02, 1.19] | [4.18, 5.04] |
| Normalised W1 | 0.02c | 0.28c | 0.19d |
| 95% CI | [0.01, 0.03] | [0.26, 0.30] | [0.17, 0.21] |
| Distance-based tests | |||
|---|---|---|---|
| Metric | Risk probability | Risk impact | Risk score |
| Wasserstein-1 (W1) | 0.08 | 1.11 | 4.62 |
| 95% bootstrap CI (paired) | [0.05, 0.13] | [1.02, 1.19] | [4.18, 5.04] |
| Normalised W1 | 0.02 | 0.28 | 0.19 |
| 95% CI | [0.01, 0.03] | [0.26, 0.30] | [0.17, 0.21] |
Bootstrap confidence intervals based on 5,000 paired resamples
Permutation p-values computed using 5,000 paired-label permutations
Normalised Wasserstein distances computed using the observed 1–5 scale range
Normalised Wasserstein distances computed using the observed 1–25 scale range
Sharing content requires targeting cookies to be enabled. Please update your cookie preferences to use this feature.