On Mon, 2005/02/14 (MST), <df@ixweb.ch> wrote:
> I am currently implementing a secure HTTP proxy for an information
> security laboratory (see http://www.infsec.ethz.ch/lab/appliedlab) and
> use web polygraph for performance testing. I want to test the proxy's
> performance a) with a realistic low request rate of .6 req/sec and b) in
> best effort mode.
> ...
> the actual test consist of a single phase during 20 minutes. I ran the
> test with different configurations of the proxy (filtering modules
> switched on or off) but always encountered a very strange behaviour:
> mean response times are _lower_ in the best effort mode than with the .6
> req/sec request rate...
>
> example from one test:
>
> best effort: 180 xAct/sec = 13.65 Mbits/sec, mean response time 5msec
> .6 req/sec: 0.60 xAct/sec = 0.05 Mbits/sec, mean response time 9 msec
There are several differences between the two tests that may explain lower
best effort response time. For example, with a single best effort robot,
the proxy handles one request at a time. There are no concurrent requests
(except for requests for embedded objects, if they exist in your
environment). With a constant mean request rate robot, you may get bursts
of concurrent requests. The proxy might slow down responses during a
burst. To [indirectly] verify this theory, study the "Concurrency Levels"
graphs produced by the "reporter" tool and increasing the number of
best-effort robots to match the concurrency level of the other test.
Another reason could be the mean rate of transactions itself. It is
possible that something in the proxy slows down isolated transactions
compared to transactions that occur without a delay. For example, the
proxy may have some kind of select(2) or idle loop optimizations that lead
to a few milliseconds of extra sleep time when a new transaction arrives
after a pause. Polygraph might have a similar problem as it is rarely used
for very low request rate testing. To [indirectly] verify this theory, you
can increase the number of rated robots or increase the single robot rate
to ensure that there is usually a few pending requests at the proxy.
Note that the difference within a few milliseconds is usually within
Polygraph test precision. A consistent difference most likely indicates
that there is something interesting going on, but any two comparable
results are still a statistical tie.
It would be great if you inform the list of your progress and, hopefully,
the actual reason behind response time difference.
Thank you,
Alex.
P.S. You can further simplify the test by removing embedded objects, but
that would be a less realistic, micro-level test, of course.
This archive was generated by hypermail 2b29 : Mon Feb 06 2006 - 12:00:28 MST