Date: Thu, 10 Oct 2002 11:18:42 +1000 From: Christopher Smith <csmith@its.uq.edu.au> To: Luigi Rizzo <rizzo@icir.org> Cc: <hardware@freebsd.org>, <net@freebsd.org> Subject: Re: High interrupt load on firewalls Message-ID: <B9CB1292.30FD3%csmith@its.uq.edu.au> In-Reply-To: <20021009170002.A54675@carp.icir.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 10/10/02 10:00 AM, "Luigi Rizzo" <rizzo@icir.org> wrote: > On Thu, Oct 10, 2002 at 09:38:40AM +1000, Christopher Smith wrote: > ... >> With the 2.4GHz 2650 we have currently, er, "borrowed" to do some testing >> with, the load is down to 35% or so (highest I've seen it is 40%) and the >> packet loss is less than 1 packet every 5 or 10 minutes. > > quite reasonable then, i wouldn't spend too much time in trying to > improve things, engineer's time is way more expensive than a new > box -- it adds up easily. True. We need to redo the whole ruleset though, simply for manageability reasons. May as well do some performance optimising at the same time. >> The existing firewall ruleset is not particularly optimised at all, and >> rewriting the whole thing is an ongoing project of mine. I might up the >> priority of this, given your comments. I get the impression that what >> appears from top to be a high interrupt load may simply be a high "system" >> load from ipfilter processing the packets. Is this correct ? Is there any >> (reasonably easy) way of checking ? > > you might have both problems -- the network stack operates as a soft > interrupt, whether this is accounted as intr or system activity i > am not sure. I tried a quick and dirty way by just dropping the whole ruleset for ~10 seconds. The interrupt load (in top) was about a third of "normal" for that time period. So, it appears the problem is definitely in the ruleset. >> What tools do you you use to test and measure performance for things like >> this ? > > nothing special, i have a trivial c program which loops around a sendto() > trying to push udp packets out as fast as possible, i disable icmp replies > with > net.inet.udp.blackhole=1 Ok, so any of the network benching products that can spit out a stream of UDP traffic should suffice ? > and then measure the card stats with > > ns -i -w 1 -d > > (ns is picobsd's version of netstat, in /usr/src/release/picobsd/tinyware/ns) > Be warned that some cards update the stats everi 1 or 2 seconds, so you might > have to up the refresh time from 1 second ('-w 1' above) to 2 or more seconds > ( -w 2 ...) Ok. Will normal netstat do ? I tried it on one of our machines and got these results: mr2fw2# netstat -w1 -i -d input (Total) output packets errs bytes packets errs bytes colls drops 39518 0 14415117 39509 0 28791210 0 0 50231 0 18969196 50117 0 37782558 0 0 50494 0 17912692 50212 0 35471676 0 0 53685 0 20592345 53547 0 40987500 0 0 44862 0 17009437 44897 0 33974566 0 0 mr2fw2# netstat -w10 -i -d input (Total) output packets errs bytes packets errs bytes colls drops 346558 0 112637886 346876 0 224797074 0 0 392474 0 132204783 392437 0 263929848 0 0 431339 0 139024315 431086 0 277224060 0 0 428132 0 135837843 427895 0 271022908 0 0 390021 0 131117003 389502 0 261278610 0 0 This only seems to indicate ca. 80kpps, which doesn't seem to agree with the numbers I see in 'systat -ip'. Is there a counter rolling over somewhere ? >> Also, are there any cards/chipsets you would recommend for firewalling high >> speed links like this ? > > no idea, i have only tried the intel pro/1000 which seems to work > reasonably well for what i have done so far (which is very little, > basically speed testing and porting the polling support to it), > but i have not tried any other GigE card. Ok then. Apparently there's a machine coming in soon with a Broadcom 5700-based 3Com card in it. Might try and purloin that for a day or two to do some experimenting on. -- +- Christopher Smith, Systems Administrator ------------------------------+ | Server & Security Group, Information Technology Services | | The University of Queensland, Brisbane, Australia, 4072 | +- Ph +61 7 3365 4046 | email csmith@its.uq.edu.au | Fax +61 7 3365 4065 -+ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-net" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?B9CB1292.30FD3%csmith>