From owner-freebsd-net@FreeBSD.ORG Sun Jan 21 19:33:56 2007 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2F5DC16A400 for ; Sun, 21 Jan 2007 19:33:56 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.177]) by mx1.freebsd.org (Postfix) with ESMTP id B9B2F13C455 for ; Sun, 21 Jan 2007 19:33:55 +0000 (UTC) (envelope-from max@love2party.net) Received: from [88.66.6.102] (helo=amd64.laiers.local) by mrelayeu.kundenserver.de (node=mrelayeu5) with ESMTP (Nemesis), id 0ML25U-1H8iRo1z21-0005Fi; Sun, 21 Jan 2007 20:33:54 +0100 From: Max Laier Organization: FreeBSD To: Bruce Evans Date: Sun, 21 Jan 2007 20:32:58 +0100 User-Agent: KMail/1.9.5 References: <20070121155510.C23922@delplex.bde.org> <200701210809.27770.max@love2party.net> <20070121215054.C24780@delplex.bde.org> In-Reply-To: <20070121215054.C24780@delplex.bde.org> X-Face: ,,8R(x[kmU]tKN@>gtH1yQE4aslGdu+2]; R]*pL,U>^H?)gW@49@wdJ`H<=?utf-8?q?=25=7D*=5FBD=0A=09U=5For=3D=5CmOZf764=26nYj=3DJYbR1PW0ud?=>|!~,,CPC.1-D$FG@0h3#'5"k{V]a~.<=?utf-8?q?mZ=7D44=23Se=7Em=0A=09Fe=7E=5C=5DX5B=5D=5Fxj?=(ykz9QKMw_l0C2AQ]}Ym8)fU MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart1595927.pzSa1nX3oL"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200701212033.34914.max@love2party.net> X-Provags-ID: kundenserver.de abuse@kundenserver.de login:61c499deaeeba3ba5be80f48ecc83056 Cc: freebsd-net@freebsd.org Subject: Re: slow writes on nfs with bge devices X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Jan 2007 19:33:56 -0000 --nextPart1595927.pzSa1nX3oL Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Sunday 21 January 2007 13:25, Bruce Evans wrote: > On Sun, 21 Jan 2007, Max Laier wrote: > > On Sunday 21 January 2007 07:25, Bruce Evans wrote: > >> nfs writes much less well with bge NICs than with other NICs (sk, > >> fxp, > > > > Do you use hardware checksumming on the bge? There is an XXX in > > bge_start_locked() that looks a bit suspicious to me. > > I use the default for that. Wouldn't checksum problems show up as > errors somwhere? Did you look at the code in question? It is concerned with fragmented=20 packet chains (which NFS over UDP usually generated) and only commits to=20 sending them, if there are enough descriptors available at once. This=20 can easily explain burstyness. Can you just try to disable the delayed checksums via "ifconfig -txcsum"? = =20 Should be an easy enough test. > >>> [excessive quoting deleted] > > tcpdump shows a lot of intervals between nfs write requests of almost > exactly 10 mS (even with HZ =3D 1000, but more obvious with HZ =3D 100) > for the broken case, so the problem is apparently related to timeouts, > or timeouts are masking a larger problem. (The average interval > between write requests needs to be about 683 uS to avoid wasting any > bandwith. fxp <-> fxp on the same network averages 708 uS. fxp <-> bge > 5701 averages 3981.) See above, I really think that there is something about that if_start loop= =20 that might be causing this. =2D-=20 /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News --nextPart1595927.pzSa1nX3oL Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) iD8DBQBFs8AOXyyEoT62BG0RAhh5AJ9gf9+sPSC4Eur4cxo3Qnm1xPGmFgCdFZn7 yecaaZcGElSbDiPS6zx3USA= =4wpX -----END PGP SIGNATURE----- --nextPart1595927.pzSa1nX3oL--