New Qualitative Model Helps Measure Security Risk Reduction

We receive many questions about risk analysis, how to demonstrate ROI for security solutions and show a solution is effective at reducing risk (an event that could result in financial loss or adverse business impact). This new Yankee Group model debunks the myth that risk analysis is a long process that requires an asset inventory, asset valuation and detailed vulnerability assessments. Qualitative risk analysis is the simplest method available for demonstrating ROI or reduction of risk (ROR).

The model, an exercise for key stakeholders, takes about an hour to complete. It is useful to vendors and providers looking to show product or service ROI, and is valuable to executives seeking a better understanding of their security risks and controls.

The Five-Step Qualitative Risk Analysis Model

Step 1: Define the scope and identify risks

Define the asset(s) you are protecting (such as your computer, application or network). Identify the risks to that asset in the areas of confidentiality, integrity, availability and accountability (these terms are defined below in bold). Prioritize each risk using a scale that makes sense to you, using designations such as high, medium and low. If you know the potential financial loss associated with that risk, you can assign a dollar amount.

Answer this question: “How critical is this risk?” rather than “Given the controls in place, how critical is this risk?”

Example 1: A small consulting company assesses the risk to their network:

• Confidentiality (information stored on my network is read without my authorization). Unauthorized access to my company’s Web site or document management system may result in loss of income or harm my company’s reputation. Information stored on internal servers and workstations or transmitted via e-mail is lower risk. Assign Medium risk or a potential loss of $60,000.
• Integrity (information stored on my network is changed without my authorization). Changes to my Web site or document management system could harm my reputation or result in loss of income. I may suffer lost productivity. Assign High risk or a potential loss of $80,000
• Availability (my network is not available for my use). If my Web site or document management system is not available, I will lose money and customers. If any part of my network is not available, my productivity will decline. Assign High risk, or a potential loss of $90,000.
• Accountability (users are not accountable for actions they perform on my network). A loss of accountability may result in lost productivity or lost income. Assign Low risk or a potential loss of $20,000

Step 2: Identify controls

List the controls that you have used to mitigate the risks identified in Step 1.

A control is any action you have taken to prevent risks. This includes policies, procedures and technical controls. These are the controls for our small network example:

• Firewall at the perimeter
• Anti-virus on the desktops
• Network access control (i.e., a login is required to access the network)
• Selective application-level encryption (i.e., some applications encrypt data)
• Selective applications that enforce access control (i.e., a login is required for some applications)
• Corporate policies that define who may access information

This network has three primary controls for ensuring integrity and availability (firewalls, anti-virus and network access control), and three primary controls for maintaining confidentiality and accountability (encryption, application access control and policy).

Step 3: Identify vulnerabilities

List the vulnerabilities of the current controls. A vulnerability is anything that reduces the effectiveness of a control or otherwise increases the likelihood of the risks occurring. Vulnerability can result from controls that are not configured correctly, controls that cannot be verified as effective, and missing controls.

In our example, these vulnerabilities underscore real-world problems with controls, such as:

• We are unsure if our firewalls are correctly configured and have no way to measure the effectiveness of this control.
• We cannot keep up with patches for our operating systems.

The vulnerabilities we identified for this network are associated with the controls we use primarily for ensuring the integrity and availability of our network. We also identified these as our greatest risks in Step 1.

Step 4: Adjust controls Identify risk-mitigation steps or opportunities for further risk reduction. In our example, we propose the following risk-mitigation steps to complement existing controls and further reduce our greatest risks:

• Vulnerability intelligence services (see Exhibit 1)
• Vulnerability assessment of the firewall

Step 5: Estimate ROR

A basic ROR calculation for vulnerability intelligence services uses the potential loss amounts from Step 1 and estimated values for control effectiveness.

The estimates for control effectiveness don’t actually affect the ROR result. ROR is a function of potential loss and the change in control effectiveness. If the proposed controls are 20 percent more effective at addressing risks, the reduction of risk is 20 percent of the potential loss amount (see Exhibit 2).

We estimate that adding a new control, vulnerability intelligence, will increase the effectiveness of our integrity and availability controls by 20 percent. This translates to a 20 percent reduction in downtime, a 20 percent decrease in virus infections, or a 20 percent reduction in time spent patching or fighting virus infection. These metrics can be used to validate this calculation and verify that we have reduced risk.

We estimate that adding the vulnerability assessment service will increase the effectiveness of our confidentiality, integrity and availability controls by 10 percent. This translates to a 10 percent reduction in the number or severity of reported vulnerabilities or a 10 percent reduction in downtime.

Conclusions
Qualitative methods are the simplest form of risk analysis. Their advantage is how quickly and easily they provide a result. Qualitative risk analysis accepts the subjectivity of risk analysis, and doesn’t require precise asset values.

In our example, we identified opportunities for risk reduction and achieved an understanding of our current state. We can continue to refine the estimates for potential loss, taking into account the probability that a particular risk will occur and the actual costs associated with it. A qualitative approach is valuable because it provides an intuitive sense of the risks and directly correlates risks with mitigating controls. This method of ROR calculation is unique because it pinpoints the security metrics we can later use to verify our decision.

Vendor Recommendations

• Use this model to conduct a qualitative risk analysis before any ROI calculation. This will enable you to understand risk, before beginning the time-consuming task of asset valuation.
• Don’t get hung up on estimating asset values and potential losses. If you are unsure what of the true value of an asset or are struggling to determine the potential losses, use a less precise scale (high, medium or low). You can always adjust these values later and see how that affects the results.

Enterprise Recommendations

• Use this model to conduct a high-level risk assessment. This will give you an overall picture of your risks and controls. Understanding your risks and identifying opportunities for risk reduction are keys to security.
• Use this model before purchasing any security product or service. This model will tell you if you are making a sensible purchase and how you can measure your success.
• Make sure key stakeholders are involved in risk analysis. Only key stakeholders, such as the owners and those responsible for maintaining the information asset, can identify and prioritize risks fully and accurately.

The Yankee Group originally published this article on 21 October 2003.