What Makes a Good Metric?
Last week I participated in a two-day planning meeting attended by industry, government, and academia for a new research center related to the topic of embedded hardware and system security (1). Among the various topics of discussion was the need to develop good metrics for security. How can you measure how “secure” a system is, or compare the security of one system against another? Some things are easy to measure – height, weight, time, etc. But some things, like security, are trickier to measure.
This leads to a more general question – what makes a good metric? Metrics can be defined as “measurable properties that quantify the degree to which objectives have been achieved” (2). Given the old adage, “what gets measured gets done”, properly establishing good metrics can help your organization reach its strategic objectives.
Luckily, much has been written on the topic of what makes for a good metric (3-5). The following summarizes the various desirable properties of metrics.
1. Relevant: The metrics should link directly to the organizational strategy and its objectives.
2. Actionable: Metrics should not be merely information for the sake of information, but should instead be designed to drive specific action.
3. Unambiguous: The metric should not be vague or imprecise.
4. Direct: Metrics should address the objectives as precisely as possible instead of measuring some tangential issue.
5. Operational: The objective must be able to be quantified, and necessary data are available or can be acquired with relative ease.
6. Understandable: Metrics should be easy for users to understand and communicate to others.
7. Comprehensive: A set of metrics should address all of the relevant objectives and should not leave anything out.
8. Traceable: It should be clear who acts on the data, and what they are supposed to do.
The process of developing good metrics can aid strategic decision making by facilitating objective setting, identifying benchmarks and targets, finding data sources, and coming up with a plan for execution. Well-developed metrics can be a powerful tool in taking an organization from its current state to a better state – such as increased profitability, increased operational efficiency, or higher customer satisfaction.
Collier Research Systems can help guide you through the process of develop the winning metrics for your organization. To learn more, visit www.collierresearchsystems.com.
(2) Reichert, P., Borsuk, M., Hostmann, M., et al. 2007.Concepts of decision support for river rehabilitation. Environmental Modeling and Software, 22, 188–201.
(3) Keeney, R.L., & Gregory, R.S. 2005. Selecting attributes to measure the achievement of objectives. Operations Research, 53 (1), 1–11.
(4) Eckerson, W. W. (2009). Performance management strategies: How to create and deploy effective metrics. TDWI best practices report. Renton, WA: The Data Warehousing Institute.
(5) Neely, A., Richards, H., Mills, J., Platts, K., & Bourne, M. 1997. Designing performance measures: A structured approach. International Journal of Operations & Production Management, 17 (11), 1131–1152.