2 Methods

While the CRM provides separate country-level assessments for underlying vulnerability and emerging threats, the same simple multi-step process is used in calculating scores for both:1

  • To start with, each dimension of compound risk is broken down in a series of indicators sourced from a range of internal WB risk databases and, when not available, from external sources. Indicators are grouped separately according to whether they reflect Underlying Vulnerability or Emerging Threats.

  • Each indicator is assigned a threshold deemed to represent ‘high risk’. A numeric score is then generated using a bounded min/max normalization procedure, with bounds assigned to upper (and in some cases lower) risk thresholds.2 Thresholds are typically assigned via expert consensus and elicitation. In cases where widely agreed thresholds are not available percentiles are used.

  • Next, indicators within each dimension are grouped together and aggregated. In accordance with the CRM’s flagging system, aggregation is typically done by selecting the maximum value across the suite of grouped indicators.3 In cases where there are clear differences in indicator quality, or where data has limited geographic coverage, a hierarchy of inputs is assigned. The process results in a score ranging from 0 (no risk) to 10 (high risk) for each risk dimension. Separate scores are assigned for Underlying Vulnerability and Emerging Threats.

  • An approximate measure of Overall Compound Risk is also generated for each risk dimension by combining inputs from the Underlying Vulnerability and Emerging Threat scores. This combined measure can be calculated using two separate approaches – which choice left to the user depending on needs and preferences. One approach is to aggregate the two source inputs using a geometric mean (similarly resulting in a score ranging from 0-10). Alert Levels are grouped into three categories with scores ranging from 7-10 deemed high alert, scores between 5-7 deemed medium alert status and scores between 0-5 considered low alert status. A second approach is to apply a filtering system across the two source inputs. This avoids the need to create a country-level score. Instead, alert levels are simply assigned when both inputs have reached a minimum level. For example, high overall compound risk for a given risk dimension is assigned when emerging threat and underlying vulnerability are similarly rated as high risk.

  • Country-level profiles are provided by summing the number of dimensions deemed as high (or medium) risk. Three separate measures for Underlying Vulnerability, Emerging Threats and Overall Compound Risk are provided, recognizing the diversity of user interests. Country-level scores can either be generated as the sum of risk dimensions deemed high risk or a combination of high and medium risks4. The latter is chosen as the default for the remainder of this note.

  • Finally, expert elicitation is used to validate and complement the CRM outputs with qualitative analysis. The expertise and judgement of WB sectoral experts is an essential element of the overall CRM methodology, guiding indicator selection and aggregation as well as the identification of appropriate thresholds. To help manage the complexity of the analysis, we strive for methodological simplicity, but this inevitably leads to a loss of accuracy, with the risk of not capturing all areas of interest. Expert elicitation methods can help to circumvent these limitations.

  1. Note that current changes in unemployment are featured in the MFRI heatmap↩︎

  2. Inclusion of 2020-2019 figures will be continually reviewed over time, and likely to drop in the latter quarters of 2021. Differential threshold values are also discussed in Annex Table 2↩︎

  3. Aggregation differs for each dimension of compound risk based on the quality and coverage of underlying indicators. See the Annex for further details.↩︎

  4. Note that high risk scores for underlying vulnerability and emerging threats are represented by a score of 10, while those for overall alert flags are represented by scores greater than 7 (owing to different aggregation methods). In cases where country-level scores are generated using the sum of both medium and high-risk dimensions, medium risks are assigned a half value (0.5) compared with a full value (1) for high-risks.↩︎