Comments on: Metrics for Performance Analysis http://ericgoldsmith.com/2008/05/24/metrics-for-performance-analysis/ Random thoughts on life and technology Sat, 19 May 2012 17:46:02 +0000 http://wordpress.org/?v=2.2.1 By: Philip Tellis http://ericgoldsmith.com/2008/05/24/metrics-for-performance-analysis/#comment-5341 Philip Tellis Wed, 16 Dec 2009 17:34:42 +0000 http://ericgoldsmith.com/2008/05/24/metrics-for-performance-analysis/#comment-5341 Web performance data actually follows a log-normal distribution, ie, the log (base is unimportant) of response times is normally distributed. This property of the data means that the geometric mean is a good measure of central tendency. You can get the geometric mean as exp(avg(log x)). Similarly you can get the geometric standard error. There's a paper that describes this: http://home.pacbell.net/ciemo/statistics/WhatDoYouMean.pdf There's also a question about whether you should run IQR filtering on the data before analysing it or not. IQR filtering tends to get rid of outliers, which in the case of web performance data applies mainly to excessively high numbers that come out of DNS misses (ie, an ISP's first DNS server failed and the second responded), which isn't something you can control, though you may want to know about it any way. I'll be speaking about this at confoo.ca in Montreal in March if you're interested. Web performance data actually follows a log-normal distribution, ie, the log (base is unimportant) of response times is normally distributed. This property of the data means that the geometric mean is a good measure of central tendency. You can get the geometric mean as exp(avg(log x)). Similarly you can get the geometric standard error.

There’s a paper that describes this:

http://home.pacbell.net/ciemo/statistics/WhatDoYouMean.pdf

There’s also a question about whether you should run IQR filtering on the data before analysing it or not. IQR filtering tends to get rid of outliers, which in the case of web performance data applies mainly to excessively high numbers that come out of DNS misses (ie, an ISP’s first DNS server failed and the second responded), which isn’t something you can control, though you may want to know about it any way.

I’ll be speaking about this at confoo.ca in Montreal in March if you’re interested.

]]>