From owner-freebsd-hackers@FreeBSD.ORG  Wed Oct 27 14:14:20 2004
Return-Path: <owner-freebsd-hackers@FreeBSD.ORG>
Delivered-To: freebsd-hackers@freebsd.org
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 8C54916A4CE
	for <freebsd-hackers@freebsd.org>;
	Wed, 27 Oct 2004 14:14:20 +0000 (GMT)
Received: from ussenterprise.ufp.org (ussenterprise.ufp.org [208.185.30.210])
	by mx1.FreeBSD.org (Postfix) with ESMTP id F337243D46
	for <freebsd-hackers@freebsd.org>;
	Wed, 27 Oct 2004 14:14:19 +0000 (GMT)
	(envelope-from bicknell@ussenterprise.ufp.org)
Received: from ussenterprise.ufp.org (bicknell@localhost [127.0.0.1])
	by ussenterprise.ufp.org (8.12.9/8.12.9) with ESMTP id i9REEJaK036912
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO)
	for <freebsd-hackers@freebsd.org>;
	Wed, 27 Oct 2004 10:14:19 -0400 (EDT)
Received: (from bicknell@localhost)
	by ussenterprise.ufp.org (8.12.9/8.12.9/Submit) id i9REEJ9C036911
	for freebsd-hackers@freebsd.org; Wed, 27 Oct 2004 10:14:19 -0400 (EDT)
Date: Wed, 27 Oct 2004 10:14:19 -0400
From: Leo Bicknell <bicknell@ufp.org>
To: freebsd-hackers@freebsd.org
Message-ID: <20041027141419.GA36554@ussenterprise.ufp.org>
Mail-Followup-To: freebsd-hackers@freebsd.org
Mime-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;
	protocol="application/pgp-signature"; boundary="IJpNTDwzlM2Ie8A6"
Content-Disposition: inline
Organization: United Federation of Planets
X-PGP-Key: http://www.ufp.org/~bicknell/
Subject: Busy BIND + 5.2.1 = UDP Packet Loss
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.1
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
	<freebsd-hackers.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
	<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
	<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 27 Oct 2004 14:14:20 -0000


--IJpNTDwzlM2Ie8A6
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable


I recently upgraded a fairly busy nameserver to FreeBSD 5.2.1, and
I'm seeing packet loss from time to time on the box.  I've done
some digging, and the box seems to be dropping UDP packets.  Netstat
output:

udp:
        177604011 datagrams received
        0 with incomplete header
        7 with bad data length field
        2735233 with bad checksum
        83753 with no checksum
        205540 dropped due to no socket
        1917 broadcast/multicast datagrams dropped due to no socket
        10627437 dropped due to full socket buffers
        0 not for hashed pcb
        164033877 delivered
        169793422 datagrams output

The "dropped due to full socket buffers" seems to be the issue.  I
am also concerned by the number of packets with bad checksums, but
I have no previous data point.

I am seeing loss with DNS, but also with ping and given the few pauses
in my ssh sessions with TCP as well.  I don't see anything remarkable
with the TCP or ICMP statistics.  I don't think there's anything wrong
in MBUF land, statistics here for reference:

% netstat -m
mbuf usage:
        GEN cache:      0/256 (in use/in pool)
        CPU #0 cache:   335/672 (in use/in pool)
        Total:          335/928 (in use/in pool)
        Mbuf cache high watermark: 512
        Maximum possible: 51200
        Allocated mbuf types:
          291 mbufs allocated to data
          14 mbufs allocated to ancillary data
          16 mbufs allocated to fragment reassembly queue headers
          14 mbufs allocated to socket names and addresses
        1% of mbuf map consumed
mbuf cluster usage:
        GEN cache:      0/152 (in use/in pool)
        CPU #0 cache:   289/400 (in use/in pool)
        Total:          289/552 (in use/in pool)
        Cluster cache high watermark: 128
        Maximum possible: 25600
        2% of cluster map consumed
1336 KBytes of wired memory reserved (49% in use)
0 requests for memory denied
0 requests for memory delayed
0 calls to protocol drain routines

The only other thing I found of interest was some interrupt drops:

% sysctl -a | grep drops
net.inet.ip.intr_queue_drops: 13981

So, given the traffic profile (nameserver, heavy UDP) and the info
here can someone help point me in the right direction?  I'm not sure
where to go from here?

--=20
       Leo Bicknell - bicknell@ufp.org - CCIE 3440
        PGP keys at http://www.ufp.org/~bicknell/
Read TMBG List - tmbg-list-request@tmbg.org, www.tmbg.org

--IJpNTDwzlM2Ie8A6
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (FreeBSD)

iD8DBQFBf607Nh6mMG5yMTYRAqUsAJ9BMN6aEuXXWrJJc3+tXePO8O2J7wCaA5Ub
Hu08eain/jiTlscA5i1XyUM=
=A613
-----END PGP SIGNATURE-----

--IJpNTDwzlM2Ie8A6--