Date: Tue, 28 Nov 2006 19:54:52 +0100
From: Joerg Pernfuss <elessar@bsdforen.de>
To: stable@freebsd.org
Subject: kernel statistics weirdness on RELENG_6{_1,_2} / 64bit?
Message-ID: <20061128195452.05e38596@loki.starkstrom.lan>
next in thread | raw e-mail | index | archive | help
--Sig_4ZEqzrALDhtO67Z_Xb_tHQK Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Hi, a while ago I noticed something odd on my 64bit FreeBSD 6 systems - two amd64, one alpha. I can't make head or tails of it, so I post it in hope that someone can. Box #1: elessar@snake: ~% uname -srm FreeBSD 6.1-RELEASE-p7 amd64 elessar@snake: ~% grep kern.ipc /boot/loader.conf kern.ipc.nsfbufs=3D"8192" kern.ipc.nmbclusters=3D"49152" elessar@snake: ~% sysctl kern.ipc.nsfbufs kern.ipc.nmbclusters kern.ipc.nsfbufs: 0 kern.ipc.nmbclusters: 32768 elessar@snake: ~% netstat -m 350/132/482/49152 mbuf clusters in use (current/cache/total/max) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 2864770 requests for I/O initiated by sendfile Box #2: elessar@forseti: ~% uname -srm FreeBSD 6.2-RC1 alpha elessar@forseti: ~% grep kern.ipc /boot/loader.conf kern.ipc.nsfbufs=3D"4294967300" elessar@forseti: ~% sysctl kern.ipc.nsfbufs kern.ipc.nmbclusters kern.ipc.nsfbufs: 0 kern.ipc.nmbclusters: 1664 elessar@forseti: ~% netstat -m 64/80/144/1664 mbuf clusters in use (current/cache/total/max) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile Box #3: elessar@rivendell: ~% uname -srm FreeBSD 6.1-RELEASE-p10 amd64 elessar@rivendell: ~% grep kern.ipc /boot/loader.conf kern.ipc.nsfbufs=3D"8192" kern.ipc.nmbclusters=3D"49152" elessar@rivendell: ~% sysctl kern.ipc.nsfbufs kern.ipc.nmbclusters kern.ipc.nsfbufs: 0 kern.ipc.nmbclusters: 32768 elessar@rivendell: ~% grep NSF /root/kernel/RIVENDELL options NSFBUFS=3D8192 elessar@rivendell: ~% netstat -m 64/118/182/49152 mbuf clusters in use (current/cache/total/max) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 35 requests for I/O initiated by sendfile For comparison, Box #4: elessar@web2: ~% uname -srm FreeBSD 6.1-STABLE i386 elessar@web2: ~% grep kern.ipc /boot/loader.conf kern.ipc.nsfbufs=3D"8192" elessar@web2: ~% sysctl kern.ipc.nsfbufs kern.ipc.nmbclusters kern.ipc.nsfbufs: 8192 kern.ipc.nmbclusters: 32768 elessar@web2: ~% netstat -m 104/272/376/32768 mbuf clusters in use (current/cache/total/max) 1598/2635/8192 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 8938512 requests for I/O initiated by sendfile So, looking at the `netstat -m' output, setting kern.ipc.nmbclusters in the loader.conf seems to work, but the sysctl reports the default value that would be in place, if it wasn't overridden in the loader.conf. For the sendfile buffers, the stats are completely out of whack. The sysctl reports 0 for every value (`netstat-m' just prints the sysctl fields for these values if I understood the source right (unlikely). But sendfile() works and the accounting on the number of sendfile calls seems to work too. So I believe the NSFBUFS kernel option and the loader.conf setting work, but it is quite impossible to properly tune the buffers to the workload. I did not bother trying the NMBCLUSTERS kernel option, it was removed a while ago anyway. The loader.conf kern.ipc.nsfbufs value on box #2 is ridiculously high, yes, I tried to get beyond the boundary of a u_int_32 to see if maybe something good or bad happens. It did not. At this point, I wanted to include links to two mails on amd64@ I found when I first searched for this oddity, both of which had no reply. Instead I found a posting to net@ that occured in the meantime, and a reply to it by Yar Tikhiy two days ago explaining the nsfbufs issue. There are no such buffers on amd64, ia64 and alpha as they aren't needed. http://docs.freebsd.org/cgi/getmsg.cgi?fetch=3D26091+0+current/freebsd-net Ok, that leaves the question why `sysctl kern.ipc.nmbclusters' reports a different value compared to the `netstat -m' output on box #1 and #3. Thanks for any pointers, Joerg PS: documenting the nsfbufs issue on amd64/ia64/alpha might be a good PS: idea. loader(8) seems a good place. Opinions? --=20 | /"\ ASCII ribbon | GnuPG Key ID | e86d b753 3deb e749 6c3a | | \ / campaign against | 0xbbcaad24 | 5706 1f7d 6cfd bbca ad24 | | X HTML in email | .the next sentence is true. | | / \ and news | .the previous sentence was a lie. | --Sig_4ZEqzrALDhtO67Z_Xb_tHQK Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (FreeBSD) iD8DBQFFbIX8H31s/bvKrSQRAhOyAJ9ZZ6f4Vp6KABYace3mDQBmrX5oOgCfaCBl wCb+9aoMNWpBh4YNFf05UgQ= =QRxs -----END PGP SIGNATURE----- --Sig_4ZEqzrALDhtO67Z_Xb_tHQK--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20061128195452.05e38596>