Re: Too many open files problem

From: Alex Rousskov (rousskov@ircache.net)
Date: Thu Dec 02 1999 - 08:10:24 MST


On Thu, 2 Dec 1999, Alan Smithee wrote:

> With 1250 robots and 500 req/sec load (0.4 req/sec per robot),
> shall open_comm_lmt be somewhere around 1?

Open_comm_lmt is totally independent from req_rate setting (but may
limit achieved per-robot request rate). Higher values of open_comm_lmt
allow one to achieve higher request rates per robot, but I cannot answer
your question because open_comm_lmt and req_rate are, again,
independent.

I suspect that, depending on your proxy response time, open_comm_lmt of
1 may not be enough for 0.4 req/sec. That is, it may limit the request
rate and/or will prevent request bursts (essentially presenting a
"best-effort" scenario).

For example, Netscape has a pool of 13 or 15 connections, with at most 4
connections opened for one host (at once). Polygraph model is not that
complex yet, we have only the former limit and do not support host
limitation (yet). IE adds a one minute timeout for idle connections. The
above stats are from Pei's student reverse engineering study available at
        http://www.cs.wisc.edu/~cao/papers/persistent-connection.html

The current setting of open_comm_lmt is 4, which is a compromise between
15/4 Netscape settings, desired per-robot request rate, and the number
of file descriptors we want to keep open per Polygraph machine.
Eventually, we will implement a more precise simulation model with two
limits. Timeouts are already supported.

Alex.



This archive was generated by hypermail 2b29 : Tue Jul 10 2001 - 12:00:10 MDT