Re: Problems with 400 Requests per Second

From: Alex Rousskov (rousskov@nlanr.net)
Date: Wed Mar 10 1999 - 08:24:06 MST


On Wed, 10 Mar 1999, Juergen Schaller wrote:

> When I try to use polygraph with more than 1500 Robots or 400 requests per
> seconds, I get the following error:
>
> 1/567 (22) Invalid argument

I suspect that you are hitting various select(2) system call limitations.

First of all, you probably want to re-define FD_SETSIZE in include/config.h
and xstd/include/xstd.h. The current limit is 4096. You should set it to the
maximum number of FDs your system supports. Ideally, it should be a job of a
configure script, but we do not have one at the moment.

> My Configuration:
>
> Sun with Solaris 2.6 + 10000 Filedescriptors:
> ./polyclt --unique_urls 1 --ports 1024:30000
> --robots 1 --launch_win 1min --goal -1:10min:0.03 --rep_cachable 100%
> --origin 192.168.11.105:8080 --proxy 192.168.11.105:8080 --req_rate 400/sec
> --verb_lvl 3

They say Sun does not support more than 1024 FDs with select(2) system call.
Polygraph uses select(2) because there is no poll(2) on FreeBSD. I do not
know how you can fix that other than waiting for us to implement a poll
support as an option.
 
> Challange L with Irix 6.5.2 and 10000 Filedescriptors:
> /polysrv --goal -1:12min:0.03 --port 8080 --xact_think norm:3sec,1.5sec

See FD_SETSIZE above. I am not sure if IRIX has select(2) limitations similar
to Solaris. Hopefully not.

Finally, look at how many OS hacks has to be applied on FreeBSD in order to
support high request rates ("Software" section):
        http://bakeoff.ircache.net/doc/logistics.html

I would be very surprised if other OSes will let you do high loads without
any tuning. If you make them work, please submit the list of tuning
parameters/hacks to us.

The last comment has nothing to do with the error you are getting though. If
you manage to fix that error, you will discover many more problems that are
irrelevant to Polygraph and must be fixed on a kernel/protocol level to
simulate high loads...

> So you see I tried a direct connection between Client and Server. The same
> error I get if I use a proxy between both.
>
> Output on the Client-Side:
> 921068816.730909| FDs: 10000 out of 10000 FDs can be used; safeguard limit:
> 9687
> 921068816.730909| run-id: 8723138.618856804:2 pid: 356
> 921068816.771804| waiting for replies; goal: -1:10.0 min:0.03
> 921068859.469635| i-warm 0 0.00 -1.00 0 5
> 921068860.469805| i-warm 21 21.00 0.00 0 395
> 921068861.471268| i-warm 97 75.89 0.00 0 739
> 921068862.437942| 1/1 (22) Invalid argument
> 921068862.437942| last error (22) was fatal; terminating...
> 921068862.438765| got 204 xactions and 0 errors
> 921068862.438765| p-warm 204 4.46 0.00 1
>
> On the serverside, everything seems to work.

The server does not get more load than a client can generate. Try running
several clients against one server, and you are likely to get the same error
on the server side as well.

Alex.



This archive was generated by hypermail 2b29 : Tue Jul 10 2001 - 12:00:05 MDT