In the book Metrics: How to Improve Key Business Results, Martin Klubeck introduced a simple, but powerful tool – the Answer Key. While it is simple to use, it offers a much more comprehensive starting point than we in Business Performance Measurement have had up to now. Besides being a template for developing meaningful metrics, it also enables the user to audit an existing set of metrics. The “lock” the Answer Key opens is the root (analytical) question and the treasure behind the lock is the rich set of metrics you can develop.
Evaluating an Existing Metrics Program
When you use the Answer Key to evaluate your current metrics program, you “plug” in your current metrics to one of the four quadrants (see figure 1). To the left of the four quadrants we find “Organizational Information Needs.” This represents a high-level root question such as: “What is the organization’s overall driving need?” or “What is the overall health of the organization?” From the root question we can as a rule pick one of two tracks – Return vs. Investment (operational) or State of the Organization (strategic). This high-level split leads to more specific organizational health quadrants. The quadrants provide a view of an organization’s health from four distinct and critical perspectives. These four perspectives are the foundation of the Answer Key. Each key contributing stakeholder to an organization’s success has their own perspective:
- Product/Service Health (customer view),
- Process Health (manager view),
- Organizational Health (worker view), and
- Future Health (leadership view).
Figure 1, the four quadrants of the Answer Key
Product/Service Health – measures that can determine how well your products and services satisfy your customers. This quadrant represents the viewpoint of the most critical of stakeholders for your organization–the customers’ viewpoint. Customers are the primary determiner of the organization’s success. The customer may buy the products, use the services, or be the “purpose” of the organization. In the case of a not-for-profit, customers are the recipients of the products or services offered by the organization.
Process Health – will measure how well the organization’s processes are working. How efficiently the organization can produce its deliverables or how efficiently it can provide services. This viewpoint is the business’ perspective. Managers regard this viewpoint seriously, wanting to use metrics primarily to continuously do more with less. This tendency in itself is not a “bad” thing. Often to become more effective (improve product/service health) the organization first improves its processes. The problem occurs when the business puts profit ahead of the customer as the real purpose for its existence. If the business believes its purpose is making money and measures success accordingly, then efficiency becomes the problem rather than the solution.
Organizational/Program Health – measures that indicate the health of the workforce. The environment, the workspace, and the all-important culture comprise this essential perspective. This perspective belongs to the workforce which is the heart and soul of the organization. The use of this quadrant brings truth to the expression – our workers are our greatest assets.
Future Health – these measures are focused on the capability of the organization to grow, prosper, and adapt in the future. This perspective involves strategic planning, goal setting and attainment, and programs and projects initiated by the organization. This viewpoint resides with the leadership of the organization.
The simple (and first) test for all measures within these quadrants is to ensure that they are viewed (reported) within the proper perspective. Let’s take availability as an example. If you look at availability from the customer’s point of view (Product/Service Health) – you are only concerned with downtime in relation to the customer’s effort to access the service. If no customers attempt to access the service – then downtime is a non-player in the “customer view” quadrant. On the other hand, if you are looking at it from the “business view” (Process Health) then the amount of time down matters, even if the customer isn’t aware of it.. If you have a fully redundant system…the amount of time the primary (or secondary) system is down matters to the business viewpoint, while the customer’s viewpoint would not even consider it.
Let’s get back to assessing your current program using the Answer Key. For every metric, we look at the components of a robust program: Alignment and Comprehension. Are your measures aligned consistently with your root question? Are your measures (which comprise your metric) providing a complete answer? Are they telling the full story?
Find the perspective your metric fits within. If you have a high-level root question you may have more than one metric. Plug in what you have. Then move to the measures you collect and report for the metric. Map only one root question at a time. If your root question has more than one metric, you should map one metric at a time. Granted, this is a simplification of the effort it will take to move along the Answer Key, but the key provides a map which you should find extremely helpful.
Figure 2, The Answer Key including measures
Plug in the measures that you have collected. Then do each of the audit checks for the following:
- Alignment. Do all the measures flow cleanly back to the root question? Or are your measures spread among the four quadrants in a haphazard fashion?
- Comprehension. Does the summation of your measures provide a complete answer to the question? This includes the rule of triangulation (three or more measures making up a single viewpoint)?
If you don’t have a metric, plug in the measures. You can still do the two checks for Alignment and Comprehension. You will need (highly recommended) to determine if you have a metric – or just some disparate measures. If you believe you have a metric (a complete story for one or more perspectives) check and see if there is a root question you are answering. Perhaps it has just never been documented. If you don’t have metrics or measures, but you’ve been collecting (and perhaps analyzing) and reporting data, you are missing the context necessary to tie the data back to your root questions. You will spend most of your time “chasing data” instead of working toward solid answers to your organizational needs.
Creating Metrics from Scratch
There are many errors that can occur when we try to develop a comprehensive program of performance measures. Two prevalent ones are inadequate sampling – settling for readily available data instead of what we “need” and misaligned sampling – having data from disparate areas so that our story is disjointed and incomplete.
Both pitfalls are easily avoided by using a performance measurement matrix. Again, the “Answer Key” provides a simple and easy to follow tool for ensuring alignment, triangulation, and completeness of the measures you use to build a metric program. A very simple definition of triangulation is the use of multiple measures, methods for collecting and analyzing the measures, and multiple sources to ensure the picture painted by your metric is not dependent on only one viewpoint.
Performance measurement for a business, regardless of the size of the organization, falls into four distinct categories:
- Customer Viewpoint
- Business (manager’s) Viewpoint
- Workers’ Viewpoint
- Leadership’s Viewpoint
For each of these perspectives, we can measure different components of the organization’s overall health.
We examine the organization’s ability to deliver products and services to the customer. The question driving these measures is, “how effective are we at satisfying our customers’ needs?” We call this group of measures “effectiveness” measures.
We examine the organization’s ability to perform. Specifically, how healthy is the organization in performing the processes necessary to delivery products and services. These measures are more commonly referred to as “efficiency” metrics.
How well does the organization take care of itself? These views can be used to analyze the health of a large organization or an individual. They measure effectiveness, efficiency, and overall health. Your personal health, like the health of the organization, will help determine how well you deliver services, products, and perform the processes to do so.
Finally we look at the organizations potential for growth. Will the organization thrive or is it stuck in a rut, simply trying to survive? What are the plans for the future? Are we investing properly to meet anticipated and imminent challenges? How well is the organization prepared for the future?
Figure 3, Viewpoints vs. Health
Where to Start?
It is normally best for an organization (especially one just starting to use metrics) to focus their initial priority on the first quadrant – Customer Viewpoint–and avoid working within the Business Viewpoint. While the workforce can understand and support measures around customer satisfaction, using measures around efficiency requires a level of trust many “immature organizations” lack. Since no business can survive without customers, it is a logical place to begin. Indeed the workforce can more easily understand the customer’s viewpoint, and it is usually a safer (less contentious) place to start.
Another good starting point is the Worker Viewpoint. Leaders often claim that “our workers are our greatest asset.” By starting with the Workers Viewpoint, a leader affirms this statement. Besides directly checking on how well you please your customers, ensuring that you care for your workers is the next most important thing you can do to ensure success.
The key in this process is to help the organization identify root questions that drive the metrics. These root questions can be (literally or metaphorically) placed on the Answer Key (wherever they belong), giving you a clear picture of the type of measures you will need to provide the answer.
The Answer Key can help you check the quality of your work and ensure that you’re on the right track. And if (or when) you get stumped and you don’t know which direction to go, it can help you get on track.
Most metrics you design, if they fall on the Answer Key, will most likely start at the third tier and belong to one of four viewpoints:
- The customer’s viewpoint (effectiveness)
- The business’s viewpoint (efficiency)
- The workers’ viewpoint
- The leadership’s viewpoint
As you move from left to right on the Answer Key, you will transition from the strategic to the tactical aspects of Organizational Performance Measurement. Another way to look at this is that you move from the root question toward data that should answer it.
Regardless of where your metric (or root question) falls, you’ll have to move to the right to find the measures and data you need to answer the question. At the fourth tier you address:
- Return vs. Investment
- oProduct/Service Health – Customer View
- oProcess Health – Business View
- State of the Organization
- oOrganizational Health – Employee View
- oFuture Health – Leadership View
This tier is easily the most frequently used. It is far enough left that root questions starting here are worthy of metrics to answer, and far enough right that they are easy for most organizations to comprehend their use in improving the organization. The fifth (and any consecutive) tier, we find mostly information and measures. If we find our root question residing here, the question is probably very tactical and may not require the upper or leftmost tiers of the matrix to be sufficiently answered. Considering a Metric Development Plan, you should flesh out the metric by identifying not only the information and measures, but also document the individual data points needed. Please note, The Metrics Development Plan is another concept that is discussed in depth in the book.
The reason the Answer Key leads off the practical part of the book is that it is a great shortcut tool for you to implement metrics. It helps you put your root question into a context of organizational health. If you are forced to work without a root question, it can be used to ensure you are not trying to blend incongruous measures, as well as to help you work toward a driving need.
Working from the left, moving right, you go from the Strategic-level to the tactical. It will help you identify possible information and measures you can use to answer your root question.
Working from right to left, you can work from specific measures (or even data) back toward a driving need. Working in this direction will also help you to ensure that your metrics are logically grouped and organized as well as being aligned with your organization’s strategy. You normally don’t want information from different areas; product/service, process, organizational, or future health mixed together into one metric – as they would not be answering a single question (unless your question is a the highest levels – right-most levels of the Answer Key),
Remember that the Answer Key, while a useful shortcut, is still only a tool for helping you develop your metrics. It’s not the whole answer, and it doesn’t relieve you of the need to follow the model for developing metrics (including the identification of the Root Question).
The Answer Key, or Performance Management Framework as it is starting to be called, is not the only useful concept introduced and discussed in the book, Metrics: How to Improve Key Business Results, by Martin Klubeck. There are many other pearls contained within its pages including; the Root Question, Triangulation, Expectations, and several real-world examples of their usage. We invite you to read the book and utilize all its tools and concepts to build powerful and well-adopted metrics that can inspire your organization, company or program to thrive, increase performance and build more value within.
Authors: Martin Klubeck is a strategy and planning consultant at the University of Notre Dame and a recognized expert in the field of practical metrics. He holds a master’s degree from Webster University in human resources development and a bachelor’s in computer science from Chapman University. He is coauthor of Why Organizations Struggle So Hard to Improve So Little and numerous articles on metrics. His passion for simplifying the complex has led to the development of a simple system for developing meaningful metrics which is captured in his new book, Metrics: How to Improve Key Business Results. Klubeck is also the founder of the Consortium for the Establishment of Information Technology Performance Standards, a nonprofit organization focused on providing much-needed standards for measures.
Russ Cheesman is a senior information technology professional and consultant with experiences in all phases of the System Development Life Cycle. Much of his career had been devoted to enabling IT solutions for business problems and/or opportunities. He has served as an IT manager and practitioner in many industry sectors, including banking/financial, manufacturing, construction, retail, pharmaceutical, telecommunications, and health care. Mr. Cheesman, in recent years, has been practicing business performance measurement and management within several IT and health care organizations through the use of business strategy, balanced scorecards, metrics, key performance indicators, and business analytical systems.
© 2009 Martin Klubeck. The text of this article is licensed under the Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 license.