Date: Mon, 15 Jul 2002 17:43:24 -0400 From: =?ISO-2022-JP?B?GyRCRUQwZkQ+RzcbKEI=?= <naoyuki_tai@mac.com> To: freebsd-stable@freebsd.org Subject: mbuf clusters behavior (NMBCLUSTERS) Message-ID: <86k7nwwttv.wl@mac.com>
next in thread | raw e-mail | index | archive | help
Hello,
I'm seeing a bit of problem with my FreeBSD 4.6 stable server.
In short, how can I prevent "mbuf clusters exhaustion"?
My server: FreeBSD 4.6 Stable, Intel P3 933 / 256M Memory / a pair of
60G IDE drives (Seagate ATA IV). NIC: Intel EtherExpress 100+
NFS export.
My workstation: RH Linux 7.3 Athlon 850, NIC: 3com 950
NFS mounts the FreeBSD server's NFS
When I copy a 500M file (like iso image) from the workstation to the
server, the server starts to emit:
Jul 12 09:28:54 nile /kernel: All mbuf clusters exhausted, please see tuning(7).
So, I bumped up the nmbclusters to kern.ipc.nmbclusters=65536
I allocated 128Mbytes to the mbuf clusters, hoping that it is big enough.
But, it still shows that the same
All mbuf clusters exhausted, please see tuning(7).
Hmm.
So, I watched 'netstat -m' output while copying a file between the freebsd
server and rh client.
95/48080/262144 mbufs in use (current/peak/max):
94 mbufs allocated to data
1 mbufs allocated to packet headers
92/36424/65536 mbuf clusters in use (current/peak/max)
84868 Kbytes allocated to network (0% of mb_map in use)
0 requests for memory denied
0 requests for memory delayed
0 calls to protocol drain routines
I type
% cp a-big-file-on-the-workstation a-freebsd-servers-directory
While copying the file
92/36424/65536 mbuf clusters in use (current/peak/max)
:
8498/36424/65536 mbuf clusters in use (current/peak/max)
:
28567/36424/65536 mbuf clusters in use (current/peak/max)
I suspend the cp. The mbuf clusters stays the same for about
10 to 20 seconds. And then, it starts to come down.
----------------
If I do not suspend the copy command, the mbuf clusters hit the
max and the server starts to drop the packets. It slows down the nfs
serving severly due to its nfs retry.
How can I prevent this "mbuf clusters exhaustion"?
Increasing mbuf clusters is not an option. There is no way that I
can allocate big enough for all of files I copy.
Relating to this, I do not understand why that the mbuf clusters
are not freed fast enough. I watched "top" and it does seem to be
that CPU is not exhausted.
After all, I'm copying less than 10Mbyte/sec, probably 6 - 7 Mbytes
at most.
Hard disk is a Seagate ATA/IV 60Gbyte.The drive is hooked up to a
Promise PCI ATA/UDMA 100 controller card.
bonnie shows that it can sustain 15M - 20M bytes read/write.
Is there anything I can try?
Thank you!
--
ntai@mac.com, Naoyuki "Tai" Tai
P.S.
I sent this message to freebsd-questions, but, did not get any
response. If you've seen this message before, sorry.
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-stable" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?86k7nwwttv.wl>