I was talking with some really smart folks recently about performance metrics. We started with “what” and “why” - what metrics do you collect and why (i.e. what do you do with them). After some discussion, one of the participants said:
I first learned of Apdex at CMG 2006 and led a pilot project within my company to understand how we might use it. I presented my experience at CMG 2007. At that conference, I ran into an acquaintance, Neil Gunther, who had begun working with Apdex and alternate visualization techniques.
Neil has now published one of the most comprehensive discussions I’ve seen on Apdex: The Apdex Index Revealed.
A little more than a year ago, AOL released PageTest to the open-source community. Since then, more than 35,000 tests have been run at webpagetest.org, and it has been downloaded more than 5,000 times from SourceForge.
Creator Pat Meenan has been busy adding some new features lately that provide even more performance information.
Over the past couple of years the study of Web page performance optimization has matured, and various rules have emerged. Steve Souders kicked it off with his original 14 Rules. Then his former team at Yahoo! expanded this to 34 Best Practices (yikes!). Microsoft has 12 Steps of their own.
Back in May, I talked about the impact of ad performance on Web page load time, and the formation of an Ad Load Performance Working Group within the IAB to try to do something about it.
The first output from this working group was announced today at the IAB Ad Operations Summit in New York.
Your Web site experienced an outage or degradation of some kind - an installation gone wrong, a server crash, a network problem, etc. How do you quantify the impact of that event?
Wouldn’t it be great to be able to assign a dollar value to an outage? That can go a long way to help build a business case for more servers, new switches, better QA procedures, etc. to reduce the likelihood of future outages.
Todd Hoff over at the High Scalability blog has assembled one of the most comprehensive collections of information on various types of latency and the impact on Web site performance that I’ve come across.
This is the kitchen sink of latency information. Highly recommended.
In an earlier post I talked about metrics for reporting Web site performance (response time). Site availability is also an important metric. And the relationship between them is often misunderstood.
I’ll be attending and speaking at the upcoming Velocity conference.
Save 20% with discount code “vel08st”.
This may seem a bit ‘back to basics’, but it’s a topic of discussions I’ve been having recently.
When presenting performance metrics, folks sometimes use mean, sometimes median, sometimes both, sometimes they include standard deviation, percentiles, etc. I’ve been looking for some concrete guidance on what metrics to use, and in what contexts.
