Date: Mon, 7 May 2001 09:22:49 +0200 (CEST) From: Luigi Rizzo <luigi@info.iet.unipi.it> To: Pekka Savola <pekkas@netcore.fi> Cc: freebsd-stable@freebsd.org Subject: Re: 4.3-S: No buffer space available [SOLVED: dummynet] Message-ID: <200105070722.JAA94039@info.iet.unipi.it> In-Reply-To: <Pine.LNX.4.33.0105070933300.24100-100000@netcore.fi> from Pekka Savola at "May 7, 2001 09:41:40 am"
next in thread | previous in thread | raw e-mail | index | archive | help
> > It would be much more efficient to do the filtering first, > > and replace all "accept" rules with a "skipto 60000" where at > > rule 60000 you have the pipe limiting outgoing traffic (and > > maybe a different one for limiting input traffic). > > This way you would also not need the net.inet.ip.fw.one_pass=0 > > setting. > > Are there significant advantages to this? Only makes this more complex > IMO. Worth a test anyway, if all else fails, I suppose. it is not more complex (it is a global sostitution in the script setting the rules), and avoids having to run through ipfw twice so you gain a little bit in terms of performance. > Why dummynet is used here, is because we've been "strongly encouraged" to > use only 20Mbit/s max; it has no other traffic shaping functionality > here. > > > Also, the 50-500 rules sound strange, wouldn't you be able to > > replace a fair amount of them with dynamic (aka stateful) rules ? > > No; perhaps I didn't state this clearly. These are not "keep-state" kind > of rules; more of "automatic IDS control" kind of rules. i realised that. Still if these rules all look similar you could have some serious performance gain by trying to implement them using keep-state rules (the latter are looked up using a hash table instead of scanning through the list, which seems to be long in your case). cheers luigi > Regards, > Pekka Savola > > > > I recompiled the kernel with 100% same options, only leaving dummynet out. > > > Everything wortked like a charm. Other things had also been tried, like > > > removing SMP support, no luck. > > > > > > The traffic being shaped to 20Mbit/s ranged from 25-35 Mbit/s (steady), > > > mostly outgoing. > > > > > > Has dummynet been tested in this kind of heavy environment? > > > > > > Is there a better value for 'queue', e.g. 1000Kbytes in this scenario? > > > > > > Regards, > > > Pekka Savola > > > > > > > Running on 4.3-S on Dual P3/866 (self-compiled for SMP, dummynet etc.): > > > > > > > > [root@xxx /root] # snmpwalk xxx yyy > > > > snmpwalk: Failure in sendto (No buffer space available) > > > > > > > > [root@xxx /root] # netstat -m > > > > 8012/8576/65536 mbufs in use (current/peak/max): > > > > 6581 mbufs allocated to data > > > > 1431 mbufs allocated to packet headers > > > > 6300/6682/16384 mbuf clusters in use (current/peak/max) > > > > 15508 Kbytes allocated to network (31% of mb_map in use) > > > > 0 requests for memory denied > > > > 0 requests for memory delayed > > > > 0 calls to protocol drain routines > > > > > > > > [root@xxx /root] # sysctl kern | grep files > > > > kern.maxfiles: 16424 > > > > kern.maxfilesperproc: 16424 > > > > kern.openfiles: 2709 > > > > > > > > --- > > > > last pid: 84147; load averages: 0.52, 0.66, 0.61 up 0+01:38:21 13:12:34 > > > > 713 processes: 12 running, 688 sleeping, 13 stopped > > > > CPU states: 14.1% user, 0.0% nice, 20.7% system, 4.0% interrupt, 61.3% idle > > > > Mem: 516M Active, 316M Inact, 134M Wired, 37M Cache, 112M Buf, 1664K Free > > > > Swap: 1024M Total, 92K Used, 1024M Free > > > > --- > > > > > > > > Pushing through 20 Mbit/s steady as we speak. > > > > > > > > This seems to have been mentioned in the beginning of April with no clear > > > > resolution. > > > > > > > > A few kernel options mentioned in the posts: > > > > --- > > > > cpu I686_CPU > > > > maxusers 512 > > > > options NMBCLUSTERS=16384 > > > > options SMP # Symmetric MultiProcessor Kernel > > > > options APIC_IO # Symmetric (APIC) I/O > > > > --- > > > > This is a system with 1 GB of RAM. Network card is: > > > > > > > > fxp0: <Intel Pro 10/100B/100+ Ethernet> port 0xccc0-0xccff mem > > > > 0xf9000000-0xf90fffff,0xf9100000-0xf9100fff irq 10 at device 8.0 on pci1 > > > > > > > > > > > > What's wrong? There definitely should be enough buffers. > > > > > > > > Also: userspace froze earlier today for some odd reason; ping and > > > > traceroute responded, ipfw worked, tcp connection could be established but > > > > the daemon listening to the port never replied. Nothing in the log or the > > > > console. Ideas? > > > > > > > > Please Cc:. > > > > > > > > > > > > > > -- > > > Pekka Savola "Tell me of difficulties surmounted, > > > Netcore Oy not those you stumble over and fall" > > > Systems. Networks. Security. -- Robert Jordan: A Crown of Swords > > > > > > > > > > > > > -- > Pekka Savola "Tell me of difficulties surmounted, > Netcore Oy not those you stumble over and fall" > Systems. Networks. Security. -- Robert Jordan: A Crown of Swords > > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200105070722.JAA94039>