On Wed, 12 Jul 2000 brianok@us.ibm.com wrote:
> I know that there has been a limit of 1000 robots per machine, but we were
> wondering if, since our FreeBSD machines are P3 500s rather than P2 400s,
> it would be legitimate to run 1200 or 1250 robots per machine?
For custom workloads, any number of robots that your environment can
support is just fine. For standard workloads, like PolyMix-2, we
recommend that you obey the limits if you want to publish the results.
It saves you from unknowingly overloading your PCs and from extra
questions about the test validity.
Also, please note that increasing the number of robots per host
increases host memory requirements and additional OS tuning may be
needed to accommodate for more traffic.
> Also, for the third bake-off, will the robot/server machines be
> better than P2 400s, and will the number of robots allowed per
> machine change? Thanks.
We have discussed that internally, but have not reach a conclusion yet.
Clearly, any reasonable limit can be justified with some hand-waving.
Perhaps we need more input from the vendors on this.
IMO, deciding on a good upper limit is tricky for several reasons:
- under "normal" conditions, it is safe to run at
least 500 req/sec on a decent P3, but if a proxy
goes through a hick-up phase, the limit may have
to be lower for the test to survive
what may be an acceptable limit for one product
may be too high for the other
- processors get faster, but the workload is getting
more complex, requiring more CPU cycles and per-agent
RAM. Some of the workloads we use today may not survive
400/sec on an old P2.
- for simple workloads, Polygraph can saturate a
100Mbps link given enough CPU cycles, but we do
not want to have a network bottleneck in a PolyMix test
- high-end CPUs and more RAM cost more, increasing
the costs for entries under 400 req/sec (40% of the
2nd bake-off entries and, probably, the majority
of entries during customer tests).
- we do want to be on the safe side, but not ridiculously
conservative
Comments?
Thanks,
Alex.
This archive was generated by hypermail 2b29 : Tue Jul 10 2001 - 12:00:14 MDT