Date: Mon, 7 May 2001 07:51:41 +0300 (EEST) From: Pekka Savola <pekkas@netcore.fi> To: <freebsd-stable@freebsd.org> Cc: <luigi@freebsd.org> Subject: Re: 4.3-S: No buffer space available [SOLVED: dummynet] Message-ID: <Pine.LNX.4.33.0105070740270.23569-100000@netcore.fi> In-Reply-To: <Pine.LNX.4.33.0105052014110.21251-100000@netcore.fi>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, 5 May 2001, Pekka Savola wrote:
Ok. This was caused by the dummynet rule, either directly or indirectly.
The symptoms:
1) 'No buffer space available'
2) hard crashes: does not respond to ping, closole goes blank, ethernet
link led in the switch goes off etc (about 3-4 times in 48h)
3) soft crashes: responds to ping, traceroute, tcp establishment ok, but
the userland is dead (2 times in 48h)
The first rule was:
---
$fwcmd pipe 1 config delay 0 plr 0 bw 20000Kbit/s queue 100
$fwcmd add pipe 1 ip from any to any
---
In addition, net.inet.ip.fw.one_pass=0 and 50-500 ipfw rules (assigned
using external signals).
I recompiled the kernel with 100% same options, only leaving dummynet out.
Everything wortked like a charm. Other things had also been tried, like
removing SMP support, no luck.
The traffic being shaped to 20Mbit/s ranged from 25-35 Mbit/s (steady),
mostly outgoing.
Has dummynet been tested in this kind of heavy environment?
Is there a better value for 'queue', e.g. 1000Kbytes in this scenario?
Regards,
Pekka Savola
> Running on 4.3-S on Dual P3/866 (self-compiled for SMP, dummynet etc.):
>
> [root@xxx /root] # snmpwalk xxx yyy
> snmpwalk: Failure in sendto (No buffer space available)
>
> [root@xxx /root] # netstat -m
> 8012/8576/65536 mbufs in use (current/peak/max):
> 6581 mbufs allocated to data
> 1431 mbufs allocated to packet headers
> 6300/6682/16384 mbuf clusters in use (current/peak/max)
> 15508 Kbytes allocated to network (31% of mb_map in use)
> 0 requests for memory denied
> 0 requests for memory delayed
> 0 calls to protocol drain routines
>
> [root@xxx /root] # sysctl kern | grep files
> kern.maxfiles: 16424
> kern.maxfilesperproc: 16424
> kern.openfiles: 2709
>
> ---
> last pid: 84147; load averages: 0.52, 0.66, 0.61 up 0+01:38:21 13:12:34
> 713 processes: 12 running, 688 sleeping, 13 stopped
> CPU states: 14.1% user, 0.0% nice, 20.7% system, 4.0% interrupt, 61.3% idle
> Mem: 516M Active, 316M Inact, 134M Wired, 37M Cache, 112M Buf, 1664K Free
> Swap: 1024M Total, 92K Used, 1024M Free
> ---
>
> Pushing through 20 Mbit/s steady as we speak.
>
> This seems to have been mentioned in the beginning of April with no clear
> resolution.
>
> A few kernel options mentioned in the posts:
> ---
> cpu I686_CPU
> maxusers 512
> options NMBCLUSTERS=16384
> options SMP # Symmetric MultiProcessor Kernel
> options APIC_IO # Symmetric (APIC) I/O
> ---
> This is a system with 1 GB of RAM. Network card is:
>
> fxp0: <Intel Pro 10/100B/100+ Ethernet> port 0xccc0-0xccff mem
> 0xf9000000-0xf90fffff,0xf9100000-0xf9100fff irq 10 at device 8.0 on pci1
>
>
> What's wrong? There definitely should be enough buffers.
>
> Also: userspace froze earlier today for some odd reason; ping and
> traceroute responded, ipfw worked, tcp connection could be established but
> the daemon listening to the port never replied. Nothing in the log or the
> console. Ideas?
>
> Please Cc:.
>
>
--
Pekka Savola "Tell me of difficulties surmounted,
Netcore Oy not those you stumble over and fall"
Systems. Networks. Security. -- Robert Jordan: A Crown of Swords
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-stable" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.LNX.4.33.0105070740270.23569-100000>