Using risk quantification to inform strategic decisions

By Marcos Tabacow, Governance, Risk and Assurance Executive

• The true value of risk management lies in informing real-time decisions not compliance
• Quantitative risk analyses outperform qualitative ones in decision-making, but there are limits
• Shareholders, board members and executives should challenge popular but ineffective approaches to risk management

Imagine the following situation: You are the CEO of a large organisation striving to obtain the board’s approval of next year’s budget. Bewildered by your pitch, the chair asks what your revenue forecast is, to which you unhesitatingly reply: ‘Why, it’s medium!’

It’s safe to say you won’t make it past your office on your way back.

Although what just took place is risibly far-fetched, this is typically what happens when corporate risk professionals present their work — risks are unapologetically described as medium or high or low or some other amorphous epithet.

Why are we more forgiving of risk professionals than of our fictitious CEO? For one reason or another, it seems that we have mostly come to accept that presenting risks in ambiguous qualitative language is just the way that enterprise risk management is ‘done’.

Cynics will say that such qualitative couching strikes the right balance between being seen to manage risks and the ability to wiggle out of any legal trouble that may befall an organisation or its directors in the future. If they are right, then enterprise risk management is just another euphemism for expensive compliance exercise.

But this cynical view of risk as compliance is not economical in the long term. A recent survey of 983 executives in organisations across the world, including Australia, concluded that:

‘Senior leaders and boards often fail to see the strategic value of investing in more robust and enhanced risk oversight, given a perceived lack of strategic value being provided by the risk management process.’ [1]

It is also unsustainable from a business continuity perspective. According to another study, 68 per cent of risk managers in the United States are over 40 and soon to retire, while just 6 per cent are between 20 and 30 years old[2], presaging intergenerational challenges to keep the profession alive.

To escape the embarrassment of irrelevance or oblivion, the risk profession must step up to provide valuable and unique insights for organisational decision-making. Shareholders, boards and executives should expect nothing less from such a resource intensive undertaking as enterprise risk management. The good news is that stepping up is not difficult — we just need to adopt a new mindset and abandon popular but ineffective risk assessment methods. Let’s start by unlearning the old ways.

Worse than useless

Risk management boils down to assessing, prioritising and responding to risk. But arguably the most popular risk assessment tool — the ubiquitous risk matrix — suffers from fatal design flaws that undermine risk prioritisation and response strategies. Here we will outline how a couple of unassuming design flaws can distort risk prioritisation decisions, which is the point at which the value of risk management begins to erode[3].

As you read on, consider whether your own risk matrix — assuming you are like most of us and use one — is exposed to such ills.

The following risk matrix was retrieved from a publicly available but dutifully deidentified source:

Suppose we are faced with Risks A and B outlined in the table above and wish to prioritise one over the other for treatment. By following the common practice of multiplying risk likelihood and consequence values, we see that Risk A amounts to $30.6M ($51M x 60 per cent) and Risk B to $59M ($100M x 59 per cent). Yet, although Risk B is larger than Risk A, the risk matrix forces us to reverse our priorities and label Risk A higher than Risk B. This design flaw inherent in most risk matrices is known as rank reversal error, by which higher qualitative ratings are assigned to quantitatively smaller risks, and vice-versa.[4]

A second design flaw that confounds risk priorities refers to range compression, where identical ratings are given to quantitatively very different risks. Consider the other pair of risks in the table: Risks C and D. Risk C totals $1M ($1M x 100 per cent) and Risk D $19M ($19M x 100 per cent). Even though one is 1,800 per cent larger than the other, we end up assigning equal priority by placing them in the same quadrant in the risk matrix, rating both as ‘medium’.

These and other design flaws can render risk matrices ‘worse than useless’ or ‘worse than random’, meaning that we would be better off flipping a coin when deciding which risk to prioritise.⁴ Note that such issues are endemic to risk matrices and cannot be alleviated — in fact, they can be compounded — by resorting to the usual workarounds of adding more columns to the matrix and using risk scores.

No wonder those charged with organisational oversight see scant strategic value in risk management.

A quantitative mindset

Overcoming the limitations of the risk matrix does not require a PhD in mathematics but rather the simple realisation that risks cannot generally be represented as single-point values or labels. Instead, given we are dealing with uncertainties, risks behave more like ranges of possible outcomes, where each outcome has its own chance of happening.

For instance, the risk of schedule delay for a project is typically a range that spans a high likelihood of a slight delay and a medium likelihood of a longer delay, through to a lower likelihood of a very large delay (Figure 1). The overall risk needs to account for the entire range of possible outcomes, not just a single ‘likelihood x consequence’ coordinate on a risk matrix.

Figure 1

Let us build on the graph above to illustrate how viewing risks as ranges unlocks more sophisticated insights for organisational decision-making.

Right off the bat is the ability to unambiguously capture the full effect of uncertainty on objectives. For example, by adding a few numbers to our graph (Figure 2), we can begin to articulate a richer description of risk to decision-makers, like: the project has an 80 per cent chance of slippage greater than eight weeks, a 50 per cent chance of slippage of more than 11 weeks, and a 10 per cent chance of slippage exceeding 17 weeks.[5]

Figure 2

Using this information, we can now more confidently prioritise risks by comparing the full breadth of their impact on objectives. Consider the impact of Risks A and B on the project’s timeline shown in Figure 3. Although both risks have virtually the same chance of causing a slippage of up to two weeks, delays exceeding that are more likely to come from Risk A than Risk B. For example, Risk A has a 70 per cent chance of triggering delays greater than nine weeks compared to a 30 per cent chance of such delay coming from Risk B). Due to its higher overall impact on project schedule, Risk A should be prioritised.

Figure 3

This quantitative formulation also enables risks to be properly combined to produce the total delay risk for the project (red line in Figure 3), which naturally prompts decision-makers to gauge whether such exposure lies within risk appetite or tolerance levels.

Suppose that the project board only tolerates a 20 per cent chance of project delays over 16 weeks — in other words, they accept that one in five projects might exceed that threshold, on average over a certain period (dotted line in Figure 4). Our project, however, has a 50 per cent chance of such a delay (red line in Figure 4), thus falling outside the risk appetite.

Decision-makers now face a clear choice: should the project be rejected given it exceeds the risk appetite or accepted due to some greater potential upside?

Figure 4

Weighing up risk and reward is how people intuitively make decisions so any risk analysis should be presented alongside the potential payoffs. This brings us to the main benefit of depicting risks as ranges, namely, the ability to combine both risk and opportunity in a single picture.

To illustrate, consider how delay risks may affect the project’s bottom line (Figure 5)[6]. We see that despite the risks — or because of them — the odds seem favourable with an 80 per cent chance of profit and a corresponding 20 per cent chance of loss. Given the potential payoff, the board may well be justified in approving the project even if appetite levels for specific risks (in this case, delay risks) are exceeded.

Figure 5

Not a panacea

While quantitative risk analysis can provide decision-makers with an edge to make better bets, we should note that, arguably, not all risks are quantifiable. The likelihood of rare and extreme events, dubbed Black Swans, is notoriously difficult to predict due to the nature of these phenomena and technical modelling errors.

A paradigmatic example of this issue was seen in the first half of the Global Financial Crisis, when in August 2007 the then Chief Financial Officer of Goldman Sachs, David Viniar, justified large losses at the firm by claiming that ‘we were seeing things that were 25-standard deviation moves, several days in a row.’[7]

To put this remark in perspective, a 25-standard deviation event should occur roughly once in 1.309×10¹³⁶ years[8]. Since the age of the universe is estimated to be only about 13.772×10⁹ years, what is more likely: that we witnessed something miraculously rare (several days in a row, mind you) or that the quantitative models used to estimate the likelihood of such events were wrong? Clearly, the latter.

The upshot is that under extreme uncertainty we should prepare for large deviations from the norm, rather than try to quantifiably predict when they will happen or what they will look like.

That said, the existence of extremes is no pretext for falling back on qualitative risk assessment methods. It merely implies that under such conditions any kind of modelling, including qualitative ones, is ultimately futile.

Call to action

If the risk management profession is to come of age and be of strategic value to organisational decision-making, it needs replace popular but defective qualitative risk assessment methods with sounder quantitative ones. For flawed risk analyses can only deliver risk management advice that is shaky at best or dangerous at worst.

To be on the front foot, risk professionals are urged to proactively explore the relative benefits of a quantitative mindset, whilst noting the potential limitations of risk quantification.

In parallel, shareholders, board members and executives should challenge qualitative risk assessments next time they review a risk report, starting with the design flaws of the risk matrix outlined in this article.

The alternative for the risk profession is to continue losing strategic value until it either fully degenerates into a box-ticking exercise or is eliminated altogether as a wasteful organisational experiment.

Marcos Tabacow can be contacted via LinkedIn at linkedin.com/in/marcostabacow or by email at marcos.tabacow@gmail.com.

[1] https://www.aicpa-cima.com/resources/download/2023-global-state-of-enterprise-risk-oversight-6th-edition

[2] https://www.strategic-risk-europe.com/risk-leaders/plugging-the-risk-management-talent-gap/1445897.article

[3] While a comprehensive review of the defects of the risk matrix is beyond the scope of this article, it is hoped that this brief outline will prompt producers and consumers of the risk matrix to reflect on its effectiveness.

[4] Anthony (Tony) Cox Jr, Louis. ‘What’s wrong with risk matrices?.’ Risk Analysis: An International Journal 28.2 (2008): 497-512.

[5] This graph is commonly referred to as a loss exceedance curve, which conveys the likelihood of losses (in this case, schedule delays) exceeding a certain amount over a given period.

[6] For simplicity, we are assuming that the project’s P&L is only affected by delay risks and no other issues are present. In practice, many risks affect P&L and other project performance metrics.

[7] https://www.ft.com/content/d2121cb6-49cb-11dc-9ffe-0000779fd2ac

[8] Based on a Normal or Gaussian distribution

Material published in Governance Directions is copyright and may not be reproduced without permission. The views expressed therein are those of the author and not of Governance Institute of Australia. All views and opinions are provided as general commentary only and should not be relied upon in place of specific accounting, legal or other professional advice.