Monday, June 2, 2008

Discussion about metrics and KPIs

The following are snippets of a discussion that I have had with Scott Stephens (read his blog ) that started as an answer to a LinkedIn question. I think that it has some important and interesting points about metrics and KPIs.

Question from Scott:
What is your source for KPIs or business metrics? There is a challenge identifying which business measurements are important, how to measure, and how often to measure.

Me
Metrics are used to gauge and measure how a business is performing. Performance is how effectively a business is achieving its defined goals. Hence metrics only make sense in the context of business goals. In order to find the important metrics for your business you should ask yourself what are my business goals and how do I know that I am achieving or performing to the defined goals. Dr. Deming once said that: “Running a company on visible figures alone is one of the seven deadly diseases of management. Manage what you can't measure“. If you are looking for financial metrics the Balanced Scorecard methodology should be a starting place. My suggestion would be to make sure you well defined goals and then figure out how you would measure you are performing against these goals. Those will be the metrics that matter to you.
Scott: If I was specifying an SOA approach to systems, I would want to know what is the required operating spec and how would I state it. So if I know that to achieve a production schedule I need raw material availability, what is the metric to measure raw material availability? What is the definition? Is there a resource that I can share with my supplier? Would they agree that it is a “standard?” Does CAMSTAR software recognize those standard metrics? If so, where do they come from?
Me: Manufacturing KPIs are not practical to standardize. I have done some research into the matter and what I have found is that although there are some definitions of general (or meta) standard calculation their implementation differ. This phenomenon is not only related to KPIs in manufacturing but also to Manufacturing System that are notoriously heavily customized. It is my contention that a standard Manufacturing System, as well as manufacturing KPIs that can be applied broadly is an illusion. The main reasons are a direct result of the dynamic nature of the manufacturing shop floor. Even in companies that try to standardize you will find difference between production lines that manufacture the same product or product family. This is because manufacturing systems evolve; driven by the need to optimize and reduce waste. That is of course is also part of the continuous improvement philosophy. Hence KPIs that are defined today to measure performance toward a goal may not be valid tomorrow since the goal and/or environment have changed.
Scott: I agree that based on my research there is little standardization in the world of business measurements but I am not sure that this is a function of the uniqueness of the users or simply not much work in defining and developing metrics resources. I also agree with the continuous evolution of measurement and process but (and maybe this is my background in systems engineering) I believe that a top-down approach organized by high level measurements will support a results driven business and accommodate the change that you high light. Schedule performance, for example, is a metric that I think crosses multiple industries and organizations. The broad definition and method for calculation will not change although the mechanics for driving change in the measured result will.