How to Use Metrics
Created by George Campbell, Security Executive Council Emeritus Faculty
The successful security executive defines his business plan and the performance of resources and services around clearly articulated measures. Those measures should be aligned with core business strategy and priorities. Figure 1 illustrates how a CSO has evaluated the importance of various security metrics, based on their relevance to business drivers such as managing costs and risks, focusing on return on investment, complying with the law and company policies, and protecting the lives and safety of employees. Note the last column on the right, which is checked every time: internal influence. Effective use of metrics that matter to business leadership, demonstrating the value of security operations, wins a security executive important capital.
2. Risk mapping: tracking security related incidents over time to identify risks
Every CSO should have half a dozen dials to watch on a regular basis. These indicators could be "survival metrics," the hot buttons on a dashboard you are expected to address that monitor the wellness of your organization or an issue of particular concern to management.
You may find that you have more than one dashboard—yours and the one your boss and a few key players expect you to watch and report on. The CFO could be an excellent resource to advise you on the presentation of dashboard metrics since this officer typically reports performance metrics to management on a regular basis.
While these dashboards view an array of priorities, you need first to identify what risks are important. One way to drill down on a particular risk and determine its priority level is through risk mapping. Risk mapping is about plotting the dynamics of the risk incident landscape. A presentation model of risk dynamics or risk profiling may be found in the risk map on Figure 2. More consequential incidents are at the top of the map, and more frequent ones are to the right.
In Figure 2, eight types of internal misconduct cases were plotted for the month, and the five highlighted all had inadequate supervision and poor policy awareness as contributing causes of the infractions. Half are high severity, indicating a need to address these vulnerabilities quickly. When presented for a specific facility, manager or organization over time, this presentation can be very instructive. If this example proved to be common over multiple samples, it's obvious that the CSO has to engage the appropriate HR resources to review the content of supervisory training and performance evaluation. A variety of risk profiles may be presented and analyzed in an Excel- based format. When contributing vulnerabilities or causes are noted in each cell, common denominators often demonstrate fundamental weaknesses in one control or another. A thorough examination of the case with an incident postmortem should yield contributing causes. There is a valuable story to be told to management, and it is particularly useful in quarterly or annual presentations to display notable trends, their contributing causes and suggestions for mitigation tactics. Work with your governance partners in this process.
And if you want to drill down on an emerging risk issue, consider engaging an audit colleague who is familiar with the targeted business process along with the process owners. Find a whiteboard and break down the business process and consider all the possibilities of how it could go wrong. Push the envelope on potential problems and solutions. You'll build a supporter in that business unit and likely head off a developing area of risk.
3. Measures Mapping: a way to identify risk mitigation strategies and evaluate their effectiveness
We are all familiar with the highway sign "Dangerous Curve, Reduce Speed Ahead." Many of the measures discussed in this story may be applied to provide the CSO and key constituents with similar caution signals. They become the earliest prompts for more in- depth analysis of trend dynamics that allow you to look at the root causes of problems, not just the symptoms.
Examples of incident trends that help diagnose risks to address include:
Such diagnostic measures identify risks. Then a CSO needs to develop a strategy to address them. Measures mapping helps you do that by looking at areas of risk, the contributing causes to those risks and actions implemented to mitigate those risks, and then measuring the effectiveness of those actions. Measures mapping, a method of analyzing specific hazards or incidents to identify potential tactics, is a modification of countermeasures mapping guidance for licensees of the Nuclear Regulatory Commission, utilized some years ago. It takes the aggravating cause results of incident lessons- learned analyses and the high- level tasks identified to mitigate the risk and postulates measures or metrics for each countermeasure.
Figure 3 takes on the issue of insider risk. In this example, the area of risk identified stems from the increased number of employees in a business unit who were the subject of misconduct cases. Investigations reveal that the problem stems in part from poor supervision of these employees. In addition, there's poor awareness on the part of employees of the company's business conduct policies. Mitigating actions involve the CSO and the security team as well as managers from human resources and legal departments.
There are several examples where measures maps are useful. It could be the need to cut security spending, the failure to respond to a security breach at the CEO's home, business interruptions caused by computer viruses or the frequency of workplace violence incidents. Measures mapping is a useful way for a CSO to brief constituents on a proposed risk mitigation strategy. And it enables status and cost updates in progress reporting.
4. Good metrics are SMART
Good metrics are "SMART"—specific, measurable, attainable, relevant and timely. (That idea originates with the engineering text Winning with Quality: Applying Quality Principles in Product Development, by John Wesner et al.) It's a CSO’s job to find the appropriate model for security measurement and reporting objectives that fits his organization. The most important data to the security executive depends on what is most important to his senior management and other stakeholders. It depends on what factors your supervisor will use to rate your performance. It depends on what you use to effectively measure the performance of your people and key vendors. It depends on what you need in your unique security environment to most effectively communicate, manage and influence.
Influence is often data- dependent. If you have a good grounding in the business and have the right radar working, you likely know things about risk, the value equation, the competition and the business risk environment that is not available or obvious from other sources.
For more information on how to begin your security metrics program see Getting Started With Security Metrics
Watch our 3-minute video to learn about how the SEC works with security leaders.
Contact us at: contact @secleader.com.
Copyright Security Executive Council. Last Updated: September 28, 2017
You can download a PDF of this page below: