Some Qualities of Good Metrics

Quantitative measures are important, he said, because “I want something I can see.” But they are only one kind of tool. He said that it would be impossible to predict the success of an innovative firm such as Federal Express by using traditional quantitative measures. “Measures need to have certain characteristics,” he said. “They have to be credible and make claims that can be sustained. They have to be transparent, material, contextually important. We have to be able look at our failures, assess them, and accept that it was experimentation that didn’t work. I personally object to the ‘but for’ justification: If we hadn’t done this, this wouldn’t be here. Well, you don’t know that for sure and making the claim undermines our credibility.”

“Good” metrics evolve as our expectations and aspirations grow and change. “I understand that some people in the Research Triangle are now focusing on value-added activities that grow from the world-class area we all recognize as a model of success.” The new metrics under consideration seek to retain more value-added activity in the Research Triangle area rather than exporting innovations to be implemented elsewhere.

In summary, he suggested that a good outcome measure is one that demonstrates the value proposition for stakeholders, and that we are far from having such a measure. “We owe the stakeholders a credible review of what their investment is doing. But I don’t think we’re going to find a single metric for that. Our goal is to increase our knowledge as we go along, knowing that it’s difficult to get cause and effect together. We can get improvements, so we can become more credible; we can understand what good investments are, make more of them, move forward, and expand them. Forums like this are important steps.”

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement