Re: Mean response time question

From: Alex Rousskov (rousskov@nlanr.net)
Date: Sun Mar 07 1999 - 15:38:54 MST


Hi folks,

        We have been asked what response time measurement (mean, median,
etc.) should be reported for an experiment, including the bake-off report.
We do not have a good answer, and our current intentions are to decide after
we look at the actual bake-off results. For a detailed report, we can specify
the mean and plot the entire distribution. For a short report, it is hard to
pick the right measurement. Here is a small illustration to the
mean/percentile discussion:

                              Response time

      mean 25% 50% 75%
     ------- ----- ----- -----
     1366.24 3 9 2772
     1518.53 9 87 2972
     1796.89 30 693 3324
     1862.74 62 972 3344
     7504.14 2968 7064 10576
     8972.16 5160 8752 11568
    12003.46 7720 11344 15152
    12378.91 8672 11808 15072
    19411.09 12352 18400 23968
    21231.52 15008 20096 25632

All measurements are in milliseconds. Request rate increases with row number.
"%"s on the right are percentiles. Three second average delay on the server;
55% hit ratio.

One can easily find the request rate (row) when the box gets saturated. No
surprise that mean is close to median only after the saturation point (proxy
is the bottleneck and server response time does not matter much any more).

Alex.



This archive was generated by hypermail 2b29 : Tue Jul 10 2001 - 12:00:05 MDT