I was talking with some really smart folks recently about performance metrics. We started with “what” and “why” – what metrics do you collect and why (i.e. what do you do with them). After some discussion, one of the participants said:
- Measure everything, because you don’t know what will matter
- Keep data forever, because you don’t know how far back you’ll need to look
- Individual Metrics are irrelevant, they must correlate with something else
Three tenets of performance metrics, if you will.
Not earth-shattering. You might even say common-sense. But I like how clearly and succinctly they capture what we should be thinking about when deciding the “what” and “why” for performance metrics.
Number 3 is key. In the words of a CTO I once worked with, it’s the “so what”. IOW, why should I care? How does (page load time, availability, etc.) impact my business?
For example, correlation (which isn’t necessarily causation, but a good place to start) may show that 1% drop in availability resulted in a 10% drop in page views – and PVs drive revenue. Suddenly, I can quantify the monetary impact of poor performance. That’s the “so what”.