Risk Assessment: Qualitative or Quantitative? Semi-Quantitative?
Sometimes I am asked about the differences between qualitative and quantitative risk assessments, so I thought I would write up a blog entry about it.
Risk assessment, as we discussed in the post from last week, is basically the process of ‘coming up with a number’ to assign to a risk. It involves identifying risks, then estimating their likelihood and severity, and finally comparing the values to something. In particular, the second step involving the estimation can be done in many different ways, which range from coming up with simple qualitative estimates to detailed probabilistic modeling and computer simulations.
The simplest type of analysis is a qualitative risk assessment. Like the term “qualitative” might imply, there really aren’t a lot of numbers or math involved. Probably the most simplistic analysis would be to use a “risk matrix” – the familiar red/yellow/green grid with likelihood along one axis and severity (or impact) along the other. The categories for likelihood and severity might be in terms of “low”, “medium”, “high” (or might include other gradations like “very low” and “very high”). Then, various risks can be placed into the different cells based on their qualitatively estimated likelihoods and severities, such as low*low, low*medium, high*high, etc. The problem comes in when it is time to compare risks – is a low-probability high-severity risk better or worse than a high-probability low-severity risk? Qualitative risk assessments have limited utility when it comes to risk evaluation. For more on the shortcomings of risk matrices, see (1) and (2).
Increasing in sophistication, we have “semi-quantitative” risk assessments. Here, instead of categories like “low”, “medium”, and “high”, we have defined ranges for likelihood and severity. For example, low probability might be defined as < 0.001. A medium impact might be a loss greater than $10,000 but less than $100,000, or it might be defined as something like “serious injury”. Semi-quantitative risk assessments often assign numeric values to each category to facilitate comparisons of risk (where the values of likelihood and severity are multiplied). While this process is better at facilitating decision making than strictly qualitative risk assessment, it still can be difficult to compare risks. For examples of semi-quantitative risk assessments, see (3) and (4).
Finally, there are quantitative risk assessments. These assessments often involve quantifying likelihoods and impacts in terms of probability distributions, and performing simulations over thousands of trials to understand the likely outcomes (5). Other methods include building complex fault trees or other probabilistic models. While these models offer more detailed analyses, they require more time, effort, and expertise to perform.
So what is the best technique to use? It depends on the nature of the analysis, and there are tradeoffs associated with each. Generally speaking, a good rule for risk modeling, assessment, and management is that the level of analysis chosen should be commensurate with the criticality of the system and associated risks (6). So for relatively unimportant decisions, maybe a full-blown quantitative analysis might not be necessary, but for mission-critical or safety-critical systems, a detailed analysis is necessary. The decision also involves the balancing the costs and benefits involved in acquiring the data, building the models, and performing the analysis. There is no one-size-fits-all solution.
At Collier Research Systems, we can help your organization select the right level of analysis for your particular decision making needs. We can build custom models to help you gain deeper insights your risks and opportunities by using the best practices and latest thinking in risk analysis. To learn more, visit www.collierresearchsystems.com.
(1) Altenbach, T.J. (1995). A comparison of risk assessment techniques from qualitative to quantitative. ASME Pressure and Piping Conference, Honolulu, Hawaii, July 23-27, 1995.
(2) Cox Jr., L.A. (2008) What’s wrong with risk matrices? Risk Analysis, 28(2), 497-512.
(3) Ross, T., Sumner, J. (2002). A simple, spreadsheet-based, food safety risk assessment tool. International Journal of Food Microbiology, 77, 39–53.
(4) Collier, Z.A., Walters, S., DiMase, D., Keisler, J.M., Linkov, I. (2014). “A semi-quantitative risk assessment standard for counterfeit electronics detection.” SAE International Journal of Aerospace, 7(1): 171-181.
(5) Vose, D. (1996). Quantitative Risk Analysis: A Guide to Monte Carlo Simulation Modeling. Wiley: Chichester, UK.
(6) Haimes, Y.Y. (2012). Systems-Based Guiding Principles for Risk Modeling, Planning, Assessment, Management, and Communication. Risk Analysis, Vol. 32, No. 9, pp. 1451-1467.