From owner-freebsd-lite2 Mon Oct 7 08:02:21 1996 Return-Path: owner-lite2 Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id IAA03370 for lite2-outgoing; Mon, 7 Oct 1996 08:02:21 -0700 (PDT) Received: (from dfr@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id IAA03363 for freebsd-lite2; Mon, 7 Oct 1996 08:02:20 -0700 (PDT) Date: Mon, 7 Oct 1996 08:02:20 -0700 (PDT) From: Doug Rabson Message-Id: <199610071502.IAA03363@freefall.freebsd.org> To: freebsd-lite2 Subject: cvs commit: src/sys/i386/i386 autoconf.c Sender: owner-lite2@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk dfr 96/10/07 08:02:19 Branch: sys/i386/i386 LITE2 Modified: sys/i386/i386 autoconf.c Log: Make the cd9660 mountroot code compile. It is still busted but at least it compiles. Revision Changes Path 1.56.2000.3 +4 -2 src/sys/i386/i386/autoconf.c From owner-freebsd-lite2 Mon Oct 7 08:16:39 1996 Return-Path: owner-lite2 Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id IAA04394 for lite2-outgoing; Mon, 7 Oct 1996 08:16:39 -0700 (PDT) Received: (from dfr@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id IAA04386 for freebsd-lite2; Mon, 7 Oct 1996 08:16:37 -0700 (PDT) Date: Mon, 7 Oct 1996 08:16:37 -0700 (PDT) From: Doug Rabson Message-Id: <199610071516.IAA04386@freefall.freebsd.org> To: freebsd-lite2 Subject: cvs commit: src/sys/kern imgact_elf.c kern_lock.c kern_synch.c vfs_conf.c vfs_subr.c vfs_syscalls.c src/sys/vm vm_object.c src/sys/i386/include param.h src/sys/nfs nfs_bio.c src/sys/sys lock.h src/sys/ufs/ffs ffs_vfsops.c src/sys/ufs/ufs ufs_ihash.c ufs_vnops.c Sender: owner-lite2@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk dfr 96/10/07 08:16:36 Branch: sys/i386/include LITE2 sys/kern LITE2 sys/nfs LITE2 sys/sys LITE2 sys/ufs/ffs LITE2 sys/ufs/ufs LITE2 sys/vm LITE2 Modified: sys/i386/include param.h sys/kern imgact_elf.c kern_lock.c kern_synch.c vfs_conf.c vfs_subr.c vfs_syscalls.c sys/nfs nfs_bio.c sys/sys lock.h sys/ufs/ffs ffs_vfsops.c sys/ufs/ufs ufs_ihash.c ufs_vnops.c sys/vm vm_object.c Log: Make the simplelock debugging code dependant on SIMPLELOCK_DEBUG instead of DEBUG so that it can be used without enabling any other debugging code. Fix some (not all) simplelock problems. This does not attempt to verify that the code is MP-safe, just that it uses simplelocks consistently and does not sleep while holding a simplelock. Fix some compiler warnings in filesystem code. Remove some bogus code from NFS. Revision Changes Path 1.22.2000.2 +2 -2 src/sys/i386/include/param.h 1.8.2000.2 +2 -2 src/sys/kern/imgact_elf.c 1.1.2000.1 +7 -5 src/sys/kern/Attic/kern_lock.c 1.24.2000.1 +5 -4 src/sys/kern/kern_synch.c 1.11.2000.2 +1 -2 src/sys/kern/vfs_conf.c 1.59.2000.3 +24 -3 src/sys/kern/vfs_subr.c 1.50.2000.4 +5 -5 src/sys/kern/vfs_syscalls.c 1.24.2000.3 +1 -20 src/sys/nfs/nfs_bio.c 1.1.2000.1 +3 -3 src/sys/sys/Attic/lock.h 1.40.2000.4 +4 -2 src/sys/ufs/ffs/ffs_vfsops.c 1.5.2000.2 +1 -3 src/sys/ufs/ufs/ufs_ihash.c 1.40.2000.2 +2 -3 src/sys/ufs/ufs/ufs_vnops.c 1.79.2000.2 +3 -1 src/sys/vm/vm_object.c From owner-freebsd-lite2 Tue Oct 8 12:40:11 1996 Return-Path: owner-lite2 Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id MAA19174 for lite2-outgoing; Tue, 8 Oct 1996 12:40:11 -0700 (PDT) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id MAA19161 for ; Tue, 8 Oct 1996 12:40:06 -0700 (PDT) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.0/8.8.0) with ESMTP id DAA01695; Wed, 9 Oct 1996 03:39:53 +0800 (WST) Message-Id: <199610081939.DAA01695@spinner.DIALix.COM> To: Charles Henrich cc: freebsd-lite2@FreeBSD.org Subject: Re: NFS from Solaris Server In-reply-to: Your message of "Tue, 08 Oct 1996 14:34:30 -0400." <199610081834.OAA01563@crh.cl.msu.edu> Date: Wed, 09 Oct 1996 03:39:53 +0800 From: Peter Wemm Sender: owner-lite2@FreeBSD.org X-Loop: FreeBSD.org Precedence: bulk [Please forgive the diversion to the Lite2 list, but there's things going on there that you may be interested in] Charles Henrich wrote: > Is Hosed, on 2.1.0 NFS will write to a sun server at a whopping 100K/sec over > ethernet, but it goes along happily just slowly. > > On 2.2-961006-SNAP we also write to NFS at a whopping 100K/sec, however in 8M B > increments we write out at 600K/sec (apparently into some memory buffer??) at > which point the buffer fills, all NFS operations are locked out on the system > until the buffer is drained. This takes an eternity. How do I undo this > buffering mechanism or make it a tunable? > > Also an interesting note, a SGI to the SUN writes at 900K/sec via NFS with no > problems. The SGI is apparently doing: > > NFS v3 Proc 6, Proc 7 (Data) > > The FreeBSD box does: > > NFS v2 Proc 8 (Data) > or > NFS v3 Proc 7 (Data) > > Both are horrendously slow. Im going to attempt to figure out what the hell > Proc 6 is (everything I see says read, which doesnt make alot of sense). In > any case Im not much of a kernel hacker, so any assistance or someone with a > solution, please raise your hand! :) > > -Crh The reason I pointed you to the lite2 list, is that Doug Rabson(sp?) has been slaving away at incorporating the 4.4Lite2 NFS into the rest of the FreeBSD+Lite2 kernel code. I'm wondering if Doug would be interested in having you try the results so far? Cheers, -Peter From owner-freebsd-lite2 Tue Oct 8 12:46:22 1996 Return-Path: owner-lite2 Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id MAA19544 for lite2-outgoing; Tue, 8 Oct 1996 12:46:22 -0700 (PDT) Received: from crh.cl.msu.edu (crh.cl.msu.edu [35.8.1.24]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id MAA19535 for ; Tue, 8 Oct 1996 12:46:19 -0700 (PDT) Received: (from henrich@localhost) by crh.cl.msu.edu (8.7.6/8.7.3) id PAA01816; Tue, 8 Oct 1996 15:46:11 -0400 (EDT) From: Charles Henrich Message-Id: <199610081946.PAA01816@crh.cl.msu.edu> Subject: Re: NFS from Solaris Server To: peter@spinner.dialix.com (Peter Wemm) Date: Tue, 8 Oct 1996 15:46:10 -0400 (EDT) Cc: freebsd-lite2@FreeBSD.org In-Reply-To: <199610081939.DAA01695@spinner.DIALix.COM> from Peter Wemm at "Oct 9, 96 03:39:53 am" X-Mailer: ELM [version 2.4ME+ PL22 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-lite2@FreeBSD.org X-Loop: FreeBSD.org Precedence: bulk > The reason I pointed you to the lite2 list, is that Doug Rabson(sp?) has > been slaving away at incorporating the 4.4Lite2 NFS into the rest of the > FreeBSD+Lite2 kernel code. > > I'm wondering if Doug would be interested in having you try the results > so far? I'd be happy to give it a whirl.. -Crh Charles Henrich Michigan State University henrich@msu.edu http://pilot.msu.edu/~henrich From owner-freebsd-lite2 Wed Oct 9 03:28:23 1996 Return-Path: owner-lite2 Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id DAA12994 for lite2-outgoing; Wed, 9 Oct 1996 03:28:23 -0700 (PDT) Received: from minnow.render.com (render.demon.co.uk [158.152.30.118]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id DAA12909 for ; Wed, 9 Oct 1996 03:27:42 -0700 (PDT) Received: from minnow.render.com (minnow.render.com [193.195.178.1]) by minnow.render.com (8.6.12/8.6.9) with SMTP id LAA17057; Wed, 9 Oct 1996 11:26:08 +0100 Date: Wed, 9 Oct 1996 11:26:07 +0100 (BST) From: Doug Rabson To: Charles Henrich cc: Peter Wemm , freebsd-lite2@FreeBSD.org Subject: Re: NFS from Solaris Server In-Reply-To: <199610081946.PAA01816@crh.cl.msu.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-lite2@FreeBSD.org X-Loop: FreeBSD.org Precedence: bulk On Tue, 8 Oct 1996, Charles Henrich wrote: > > The reason I pointed you to the lite2 list, is that Doug Rabson(sp?) has > > been slaving away at incorporating the 4.4Lite2 NFS into the rest of the > > FreeBSD+Lite2 kernel code. > > > > I'm wondering if Doug would be interested in having you try the results > > so far? > > I'd be happy to give it a whirl.. I just tried to reproduce the problem between a FreeBSD-lite2 machine and an SGI Indy running IRIX 5.3 without success. I get a consistent ~700k/sec write and 1000k/sec read whatever I do to nfssvc_iod or nfs_asyncio. I don't think this is due to new code in lite2 since the NFS in lite2 is very similar to the NFS in current. They were both derived from Rick Macklems NFSv3 work. Since NFS performance is a black art and the choice of server is a big factor in read / write speeds, you might like to experiment with this patch. It modifies nfssvc_iod so that you can control the number of delayed write buffers that the iod processes before it goes back to sleep. Using this it should be possible to stop it from locking out the other iods by preventing a race with the writing process. Try different values for nfs_maxdelwri to see what works best. The patch is from a lite2 cvs tree but should apply to current sources. If you want to try out lite2 itself, you can get the cvs tree with cvsup and this supfile: src-sys release=lite2 host=sup.FreeBSD.org hostbase=/home base=/home prefix=/b/dfr/lite2 delete old use-rel-suffix Index: nfs_syscalls.c =================================================================== RCS file: /host/minnow/b/dfr/lite2/src/sys/nfs/nfs_syscalls.c,v retrieving revision 1.14.2000.1 diff -u -r1.14.2000.1 nfs_syscalls.c --- nfs_syscalls.c 1996/09/23 13:25:16 1.14.2000.1 +++ nfs_syscalls.c 1996/10/09 09:56:34 @@ -88,6 +88,7 @@ extern int nfsrvw_procrastinate; struct nfssvc_sock *nfs_udpsock, *nfs_cltpsock; static int nuidhash_max = NFS_MAXUIDHASH; +int nfs_maxdelwri = 1; static void nfsrv_zapsock __P((struct nfssvc_sock *slp)); static int nfssvc_iod __P((struct proc *)); @@ -762,19 +763,28 @@ * avoid any risk of the vnode getting vclean()'d while * we are doing the write rpc. */ + int ndelwri = 0; vp = bp->b_vp; - s = splbio(); - for (nbp = vp->v_dirtyblkhd.lh_first; nbp; - nbp = nbp->b_vnbufs.le_next) { - if ((nbp->b_flags & - (B_BUSY|B_DELWRI|B_NEEDCOMMIT|B_NOCACHE))!=B_DELWRI) - continue; - bremfree(nbp); - vfs_busy_pages(nbp, 1); - nbp->b_flags |= (B_BUSY|B_ASYNC); - break; - } - splx(s); + /* + * Limit the number of B_DELWRI buffers we handle to + * avoid locking out the other iods. + */ + if (ndelwri == nfs_maxdelwri) { + s = splbio(); + for (nbp = vp->v_dirtyblkhd.lh_first; nbp; + nbp = nbp->b_vnbufs.le_next) { + if ((nbp->b_flags & + (B_BUSY|B_DELWRI|B_NEEDCOMMIT|B_NOCACHE))!=B_DELWRI) + continue; + bremfree(nbp); + vfs_busy_pages(nbp, 1); + nbp->b_flags |= (B_BUSY|B_ASYNC); + ndelwri++; + break; + } + splx(s); + } else + nbp = NULL; /* * For the delayed write, do the first part of nfs_bwrite() * up to, but not including nfs_strategy(). -- Doug Rabson, Microsoft RenderMorphics Ltd. Mail: dfr@render.com Phone: +44 171 734 3761 FAX: +44 171 734 6426 From owner-freebsd-lite2 Wed Oct 9 03:42:01 1996 Return-Path: owner-lite2 Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id DAA14242 for lite2-outgoing; Wed, 9 Oct 1996 03:42:01 -0700 (PDT) Received: from minnow.render.com (render.demon.co.uk [158.152.30.118]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id DAA14230 for ; Wed, 9 Oct 1996 03:41:49 -0700 (PDT) Received: from minnow.render.com (minnow.render.com [193.195.178.1]) by minnow.render.com (8.6.12/8.6.9) with SMTP id LAA17085; Wed, 9 Oct 1996 11:38:50 +0100 Date: Wed, 9 Oct 1996 11:38:49 +0100 (BST) From: Doug Rabson To: Charles Henrich , Hidetoshi Shimokawa , lite2@freebsd.org Subject: Delayed write patch Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-lite2@freebsd.org X-Loop: FreeBSD.org Precedence: bulk There was a stupid mistake in the delayed write patch I just sent out. This patch should work as intended. Still makes no difference for me though. Index: nfs_syscalls.c =================================================================== RCS file: /host/minnow/b/dfr/lite2/src/sys/nfs/nfs_syscalls.c,v retrieving revision 1.14.2000.1 diff -u -r1.14.2000.1 nfs_syscalls.c --- nfs_syscalls.c 1996/09/23 13:25:16 1.14.2000.1 +++ nfs_syscalls.c 1996/10/09 10:36:28 @@ -88,6 +88,7 @@ extern int nfsrvw_procrastinate; struct nfssvc_sock *nfs_udpsock, *nfs_cltpsock; static int nuidhash_max = NFS_MAXUIDHASH; +int nfs_maxdelwri = 1; static void nfsrv_zapsock __P((struct nfssvc_sock *slp)); static int nfssvc_iod __P((struct proc *)); @@ -762,19 +763,28 @@ * avoid any risk of the vnode getting vclean()'d while * we are doing the write rpc. */ + int ndelwri = 0; vp = bp->b_vp; - s = splbio(); - for (nbp = vp->v_dirtyblkhd.lh_first; nbp; - nbp = nbp->b_vnbufs.le_next) { - if ((nbp->b_flags & - (B_BUSY|B_DELWRI|B_NEEDCOMMIT|B_NOCACHE))!=B_DELWRI) - continue; - bremfree(nbp); - vfs_busy_pages(nbp, 1); - nbp->b_flags |= (B_BUSY|B_ASYNC); - break; - } - splx(s); + /* + * Limit the number of B_DELWRI buffers we handle to + * avoid locking out the other iods. + */ + if (ndelwri < nfs_maxdelwri) { + s = splbio(); + for (nbp = vp->v_dirtyblkhd.lh_first; nbp; + nbp = nbp->b_vnbufs.le_next) { + if ((nbp->b_flags & + (B_BUSY|B_DELWRI|B_NEEDCOMMIT|B_NOCACHE))!=B_DELWRI) + continue; + bremfree(nbp); + vfs_busy_pages(nbp, 1); + nbp->b_flags |= (B_BUSY|B_ASYNC); + ndelwri++; + break; + } + splx(s); + } else + nbp = NULL; /* * For the delayed write, do the first part of nfs_bwrite() * up to, but not including nfs_strategy(). -- Doug Rabson, Microsoft RenderMorphics Ltd. Mail: dfr@render.com Phone: +44 171 734 3761 FAX: +44 171 734 6426 From owner-freebsd-lite2 Wed Oct 9 05:10:49 1996 Return-Path: owner-lite2 Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id FAA20621 for lite2-outgoing; Wed, 9 Oct 1996 05:10:49 -0700 (PDT) Received: from who.cdrom.com (who.cdrom.com [204.216.27.3]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id FAA20612 for ; Wed, 9 Oct 1996 05:10:47 -0700 (PDT) Received: from uno.sat.t.u-tokyo.ac.jp (uno.sat.t.u-tokyo.ac.jp [133.11.70.160]) by who.cdrom.com (8.7.5/8.6.11) with ESMTP id FAA10194 for ; Wed, 9 Oct 1996 05:10:46 -0700 (PDT) Received: by uno.sat.t.u-tokyo.ac.jp (8.7.3+2.6Wbeta5/8.7.3) with ESMTP id VAA01388; Wed, 9 Oct 1996 21:09:14 +0900 (JST) To: dfr@render.com Cc: henrich@crh.cl.msu.edu, lite2@freebsd.org Subject: Re: Delayed write patch In-Reply-To: Your message of "Wed, 9 Oct 1996 11:38:49 +0100 (BST)" References: X-Mailer: Mew version 1.06 on Emacs 19.28.1, Mule 2.3 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Date: Wed, 09 Oct 1996 21:09:14 +0900 Message-ID: <1386.844862954@sat.t.u-tokyo.ac.jp> From: Hidetoshi Shimokawa Sender: owner-lite2@freebsd.org X-Loop: FreeBSD.org Precedence: bulk dfr> This patch should work as intended. Still makes no difference for me dfr> though. I tried this patch, but there is no performance improvement. It's still around 300KB. tcpdump shows that only one iod is running. /\ Hidetoshi Shimokawa \/ simokawa@sat.t.u-tokyo.ac.jp PGP public key: finger -l simokawa@sat.t.u-tokyo.ac.jp From owner-freebsd-lite2 Wed Oct 9 06:17:29 1996 Return-Path: owner-lite2 Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id GAA23806 for lite2-outgoing; Wed, 9 Oct 1996 06:17:29 -0700 (PDT) Received: from minnow.render.com (render.demon.co.uk [158.152.30.118]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id GAA23744 for ; Wed, 9 Oct 1996 06:16:08 -0700 (PDT) Received: from minnow.render.com (minnow.render.com [193.195.178.1]) by minnow.render.com (8.6.12/8.6.9) with SMTP id OAA17412; Wed, 9 Oct 1996 14:11:16 +0100 Date: Wed, 9 Oct 1996 14:11:15 +0100 (BST) From: Doug Rabson To: Hidetoshi Shimokawa cc: henrich@crh.cl.msu.edu, lite2@freebsd.org Subject: Re: Delayed write patch In-Reply-To: <1386.844862954@sat.t.u-tokyo.ac.jp> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-lite2@freebsd.org X-Loop: FreeBSD.org Precedence: bulk On Wed, 9 Oct 1996, Hidetoshi Shimokawa wrote: > dfr> This patch should work as intended. Still makes no difference for me > dfr> though. > > I tried this patch, but there is no performance improvement. > It's still around 300KB. > > tcpdump shows that only one iod is running. If you change nfs_asyncio to return EIO instead of using a delayed write, how many iods are running? -- Doug Rabson, Microsoft RenderMorphics Ltd. Mail: dfr@render.com Phone: +44 171 734 3761 FAX: +44 171 734 6426 From owner-freebsd-lite2 Wed Oct 9 07:19:27 1996 Return-Path: owner-lite2 Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id HAA26869 for lite2-outgoing; Wed, 9 Oct 1996 07:19:27 -0700 (PDT) Received: from mail.crl.com (mail.crl.com [165.113.1.22]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id HAA26864 for ; Wed, 9 Oct 1996 07:19:03 -0700 (PDT) Received: from uno.sat.t.u-tokyo.ac.jp by mail.crl.com with SMTP id AA17108 (5.65c/IDA-1.5 for ); Wed, 9 Oct 1996 07:19:18 -0700 Received: by uno.sat.t.u-tokyo.ac.jp (8.7.3+2.6Wbeta5/8.7.3) with ESMTP id XAA02391; Wed, 9 Oct 1996 23:16:24 +0900 (JST) To: dfr@render.com Cc: henrich@crh.cl.msu.edu, lite2@freebsd.org Subject: Re: Delayed write patch In-Reply-To: Your message of "Wed, 9 Oct 1996 14:11:15 +0100 (BST)" References: X-Mailer: Mew version 1.06 on Emacs 19.28.1, Mule 2.3 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Date: Wed, 09 Oct 1996 23:16:23 +0900 Message-Id: <2389.844870583@sat.t.u-tokyo.ac.jp> From: Hidetoshi Shimokawa Sender: owner-lite2@freebsd.org X-Loop: FreeBSD.org Precedence: bulk dfr> > I tried this patch, but there is no performance improvement. dfr> > It's still around 300KB. dfr> > dfr> > tcpdump shows that only one iod is running. dfr> dfr> If you change nfs_asyncio to return EIO instead of using a delayed write, dfr> how many iods are running? if I start 4 iods, 4 iods are running. observing by tcpdump, 1) without delayed write (800KB/s) client server ------------------------ req(async) req(async) req(async) req(async) req(sync) rep rep rep rep rep req req req req req rep rep rep rep rep req ... 2) with delayed write (300KB/s) client server ------------------------ req req req req rep rep req req rep .... req rep req rep req rep .... The following is the patch to control delayed write by sysctl. (sorry, I don't know wether this is the right usage of sysctl) *** nfs_bio.c.orig Mon Sep 30 22:39:29 1996 --- nfs_bio.c Wed Oct 9 23:13:25 1996 *************** *** 46,51 **** --- 46,52 ---- #include #include #include + #include #include #include *************** *** 65,70 **** --- 66,74 ---- extern int nfs_numasync; extern struct nfsstats nfsstats; + int nfs_dwrite; + SYSCTL_INT(_vfs_nfs, OID_AUTO, dwrite, CTLFLAG_RW, &nfs_dwrite, 0, ""); + /* * Ifdefs for FreeBSD-current's merged VM/buffer cache. It is unfortunate * that this isn't done inside getblk() and brelse() so these calls *************** *** 758,763 **** --- 762,768 ---- * is currently doing a write for this file and will pick up the * delayed writes before going back to sleep. */ + if (!nfs_dwrite) return (EIO); bp->b_flags |= B_DELWRI; reassignbuf(bp, bp->b_vp); biodone(bp); /\ Hidetoshi Shimokawa \/ simokawa@sat.t.u-tokyo.ac.jp PGP public key: finger -l simokawa@sat.t.u-tokyo.ac.jp From owner-freebsd-lite2 Wed Oct 9 09:08:57 1996 Return-Path: owner-lite2 Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id JAA05135 for lite2-outgoing; Wed, 9 Oct 1996 09:08:57 -0700 (PDT) Received: from who.cdrom.com (who.cdrom.com [204.216.27.3]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id JAA05109 for ; Wed, 9 Oct 1996 09:08:23 -0700 (PDT) Received: from minnow.render.com (render.demon.co.uk [158.152.30.118]) by who.cdrom.com (8.7.5/8.6.11) with SMTP id JAA10495 for ; Wed, 9 Oct 1996 09:07:56 -0700 (PDT) Received: from minnow.render.com (minnow.render.com [193.195.178.1]) by minnow.render.com (8.6.12/8.6.9) with SMTP id RAA17702; Wed, 9 Oct 1996 17:05:42 +0100 Date: Wed, 9 Oct 1996 17:05:40 +0100 (BST) From: Doug Rabson To: Hidetoshi Shimokawa cc: henrich@crh.cl.msu.edu, lite2@freebsd.org Subject: Re: Delayed write patch In-Reply-To: <2389.844870583@sat.t.u-tokyo.ac.jp> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-lite2@freebsd.org X-Loop: FreeBSD.org Precedence: bulk On Wed, 9 Oct 1996, Hidetoshi Shimokawa wrote: > dfr> > I tried this patch, but there is no performance improvement. > dfr> > It's still around 300KB. > dfr> > > dfr> > tcpdump shows that only one iod is running. > dfr> > dfr> If you change nfs_asyncio to return EIO instead of using a delayed write, > dfr> how many iods are running? > if I start 4 iods, 4 iods are running. If you change nfs_maxdelwri to zero using gdb or ddb, what write performance do you see? That ought to be the same as the nfs_dwrite==0 case. I can't figure out why the other iods should get locked out. Are you sure that all the reqs are made by the same iod? -- Doug Rabson, Microsoft RenderMorphics Ltd. Mail: dfr@render.com Phone: +44 171 734 3761 FAX: +44 171 734 6426 From owner-freebsd-lite2 Wed Oct 9 11:12:20 1996 Return-Path: owner-lite2 Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id LAA17082 for lite2-outgoing; Wed, 9 Oct 1996 11:12:20 -0700 (PDT) Received: from crh.cl.msu.edu (crh.cl.msu.edu [35.8.1.24]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id LAA17073 for ; Wed, 9 Oct 1996 11:12:17 -0700 (PDT) Received: (from henrich@localhost) by crh.cl.msu.edu (8.7.6/8.7.3) id OAA07612; Wed, 9 Oct 1996 14:11:22 -0400 (EDT) From: Charles Henrich Message-Id: <199610091811.OAA07612@crh.cl.msu.edu> Subject: Re: Delayed write patch To: dfr@render.com (Doug Rabson) Date: Wed, 9 Oct 1996 14:11:22 -0400 (EDT) Cc: simokawa@sat.t.u-tokyo.ac.jp, lite2@freebsd.org In-Reply-To: from Doug Rabson at "Oct 9, 96 05:05:40 pm" X-Mailer: ELM [version 2.4ME+ PL22 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-lite2@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > If you change nfs_maxdelwri to zero using gdb or ddb, what write > performance do you see? That ought to be the same as the nfs_dwrite==0 > case. I can't figure out why the other iods should get locked out. Are > you sure that all the reqs are made by the same iod? > If we get back to the root of the problem (which isnt the buffering really, its that the data is going across sooo slowly). The packets look like so: FreeBSD -> Sun RPC Proc 8 (v2) data data data data Sun -> FreeBSD ACK FreeBSD -> Sun RPC Proc 8 (v2) data data data data Sun -> FreeBSD ACK The problem is that the Sun ACK's are taking 90ms instead of <= 1ms. Under V3 We get: FreeBSD -> Sun RPC Proc 7 data data data data Sun -> FreeBSD ACK The ACK's now come in at about half the time (~45ms), still dismal, still lousy transfer rates (sub 200ish). Now, a SGI talking to the Sun looks like (v3 I beleive): SGI -> Sun RPC Proc 6 SGI -> Sun RPC Proc 7 data data data data Sun -> SGI ACK Sun -> SGI ACK The SUN ACK's now come in at the <= 1ms they should, and we get transfer rates into the 900K/sec. The question is what is Proc 6, and why is the SGI doing it, and why is it sooo much faster? -Crh Charles Henrich Michigan State University henrich@msu.edu http://pilot.msu.edu/~henrich From owner-freebsd-lite2 Thu Oct 10 00:41:23 1996 Return-Path: owner-lite2 Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id AAA26446 for lite2-outgoing; Thu, 10 Oct 1996 00:41:23 -0700 (PDT) Received: from who.cdrom.com (who.cdrom.com [204.216.27.3]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id AAA26441 for ; Thu, 10 Oct 1996 00:41:21 -0700 (PDT) Received: from uno.sat.t.u-tokyo.ac.jp (uno.sat.t.u-tokyo.ac.jp [133.11.70.160]) by who.cdrom.com (8.7.5/8.6.11) with ESMTP id AAA11751 for ; Thu, 10 Oct 1996 00:41:16 -0700 (PDT) Received: by uno.sat.t.u-tokyo.ac.jp (8.7.3+2.6Wbeta5/8.7.3) with ESMTP id QAA01521; Thu, 10 Oct 1996 16:38:45 +0900 (JST) To: Doug Rabson Cc: henrich@crh.cl.msu.edu, lite2@freebsd.org Subject: Re: Delayed write patch X-Mailer: Mew version 1.06 on Emacs 19.28.1, Mule 2.3 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Date: Thu, 10 Oct 1996 16:38:44 +0900 Message-ID: <1519.844933124@sat.t.u-tokyo.ac.jp> From: Hidetoshi Shimokawa Sender: owner-lite2@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, I got the problem! dfr> If you change nfs_maxdelwri to zero using gdb or ddb, what write dfr> performance do you see? That ought to be the same as the nfs_dwrite==0 dfr> case. I can't figure out why the other iods should get locked out. Are dfr> you sure that all the reqs are made by the same iod? Yes, the problem was caused by.. 1) the process processing DELWRT doesn't sleep. (except when waiting packet, but at this point other iods are locked out) 2) normal iod will tsleep after processing one buffer. 3) if CPU is fast enough DELWRT process can get rcvlock after rcvunlock without process switching. (beause rcvunlock is sleep for waiting packet, it has enough time until switching) - results 1) -current (bwrite=1) 2) bwrite=0 3) bwrite=1 + my patch client server ---------------------------------- A. 100M FreeBSD ------ Solaris 2.5 --- Fast Disk Pentium Pro Sun Ultra 1) 300KB/s write 2) 800KB/s write 3) 600KB/s write B. 10M FreeBSD ------- FreeBSD ------ Slow Disk DX4 75MHz Pentium 166MHz 1) 120KB/s write 2) 140KB/s write 3) 170KB/s write I think it depends on an environment which is better bwrite=0 or bwrite=1 and it should be able to be chosen by sysctl. The following patch should fix the locking problem. (I don't know which number I should return as error code, please fix it.) I think you should remove maxdelwri patch. I would like to hear other results. *** nfs_socket.c.orig Sat Jul 27 07:08:43 1996 --- nfs_socket.c Thu Oct 10 15:37:56 1996 *************** *** 682,695 **** --- 682,700 ---- * Also necessary for connection based protocols to avoid * race conditions during a reconnect. */ + #define ALREADY_RECEIVED 1 error = nfs_rcvlock(myrep); + if (error == ALREADY_RECEIVED) + return (0); if (error) return (error); + #if 0 /* Already received, bye bye */ if (myrep->r_mrep != NULL) { nfs_rcvunlock(&nmp->nm_flag); return (0); } + #endif /* * Get the next Rpc reply off the socket */ *************** *** 1494,1499 **** --- 1499,1506 ---- *flagp |= NFSMNT_WANTRCV; (void) tsleep((caddr_t)flagp, slpflag | (PZERO - 1), "nfsrcvlk", slptimeo); + if (rep->r_mrep != NULL) + return(ALREADY_RECEIVED); if (slpflag == PCATCH) { slpflag = 0; slptimeo = 2 * hz; /\ Hidetoshi Shimokawa \/ simokawa@sat.t.u-tokyo.ac.jp PGP public key: finger -l simokawa@sat.t.u-tokyo.ac.jp From owner-freebsd-lite2 Thu Oct 10 02:24:39 1996 Return-Path: owner-lite2 Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id CAA03501 for lite2-outgoing; Thu, 10 Oct 1996 02:24:39 -0700 (PDT) Received: from minnow.render.com (render.demon.co.uk [158.152.30.118]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id CAA03472 for ; Thu, 10 Oct 1996 02:24:24 -0700 (PDT) Received: from minnow.render.com (minnow.render.com [193.195.178.1]) by minnow.render.com (8.6.12/8.6.9) with SMTP id KAA20441; Thu, 10 Oct 1996 10:21:30 +0100 Date: Thu, 10 Oct 1996 10:21:26 +0100 (BST) From: Doug Rabson To: Charles Henrich cc: simokawa@sat.t.u-tokyo.ac.jp, lite2@freebsd.org Subject: Re: Delayed write patch In-Reply-To: <199610091811.OAA07612@crh.cl.msu.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-lite2@freebsd.org X-Loop: FreeBSD.org Precedence: bulk On Wed, 9 Oct 1996, Charles Henrich wrote: > The SUN ACK's now come in at the <= 1ms they should, and we get transfer rates > into the 900K/sec. > > The question is what is Proc 6, and why is the SGI doing it, and why is it sooo > much faster? Proc 6 is the v3 read call. Proc 7 is v3 write and Proc 8 is v2 write. -- Doug Rabson, Microsoft RenderMorphics Ltd. Mail: dfr@render.com Phone: +44 171 734 3761 FAX: +44 171 734 6426 From owner-freebsd-lite2 Thu Oct 10 05:23:42 1996 Return-Path: owner-lite2 Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id FAA15987 for lite2-outgoing; Thu, 10 Oct 1996 05:23:42 -0700 (PDT) Received: from minnow.render.com (render.demon.co.uk [158.152.30.118]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id FAA15965 for ; Thu, 10 Oct 1996 05:22:38 -0700 (PDT) Received: from minnow.render.com (minnow.render.com [193.195.178.1]) by minnow.render.com (8.6.12/8.6.9) with SMTP id NAA20705; Thu, 10 Oct 1996 13:17:25 +0100 Date: Thu, 10 Oct 1996 13:17:25 +0100 (BST) From: Doug Rabson To: Hidetoshi Shimokawa cc: henrich@crh.cl.msu.edu, lite2@freebsd.org Subject: Re: Delayed write patch In-Reply-To: <1519.844933124@sat.t.u-tokyo.ac.jp> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-lite2@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Thanks for the patch. It improves performance for my SGI from ~720k/sec to ~870k/sec. I modified the patch (and the nfs_dwrite patch) slightly and added comments to try to explain what is happening. Could you check the comments to make sure they agree with your understanding of the problem and I will commit this patch to current probably tomorrow. Index: nfs_bio.c =================================================================== RCS file: /home/ncvs/src/sys/nfs/nfs_bio.c,v retrieving revision 1.25 diff -u -r1.25 nfs_bio.c --- nfs_bio.c 1996/09/19 18:20:54 1.25 +++ nfs_bio.c 1996/10/10 12:13:00 @@ -46,6 +46,7 @@ #include #include #include +#include #include #include @@ -65,6 +66,9 @@ extern int nfs_numasync; extern struct nfsstats nfsstats; +int nfs_dwrite = 1; +SYSCTL_INT(_vfs_nfs, OID_AUTO, dwrite, CTLFLAG_RW, &nfs_dwrite, 0, ""); + /* * Ifdefs for FreeBSD-current's merged VM/buffer cache. It is unfortunate * that this isn't done inside getblk() and brelse() so these calls @@ -750,6 +754,14 @@ * synchronously. */ if (bp->b_flags & (B_READ | B_WRITEINPROG | B_NOCACHE)) + return (EIO); + + /* + * Allow the administrator to override the choice of using a delayed + * write since it is a pessimization for some servers, notably some + * Solaris servers. + */ + if (!nfs_dwrite) return (EIO); /* Index: nfs_socket.c =================================================================== RCS file: /home/ncvs/src/sys/nfs/nfs_socket.c,v retrieving revision 1.17 diff -u -r1.17 nfs_socket.c --- nfs_socket.c 1996/07/11 16:32:45 1.17 +++ nfs_socket.c 1996/10/10 10:56:48 @@ -681,15 +681,17 @@ * sbwait() after someone else has received my reply for me. * Also necessary for connection based protocols to avoid * race conditions during a reconnect. + * If nfs_rcvlock() returns EALREADY, that means that + * the reply has already been recieved by another + * process and we can return immediately. In this + * case, the lock is not taken to avoid races with + * other processes. */ error = nfs_rcvlock(myrep); + if (error == EALREADY) + return (0); if (error) return (error); - /* Already received, bye bye */ - if (myrep->r_mrep != NULL) { - nfs_rcvunlock(&nmp->nm_flag); - return (0); - } /* * Get the next Rpc reply off the socket */ @@ -1494,6 +1496,14 @@ *flagp |= NFSMNT_WANTRCV; (void) tsleep((caddr_t)flagp, slpflag | (PZERO - 1), "nfsrcvlk", slptimeo); + /* + * If our reply was recieved while we were sleeping, + * then just return without taking the lock to avoid a + * situation where a single iod could 'capture' the + * recieve lock. + */ + if (rep->r_mrep != NULL) + return (EALREADY); if (slpflag == PCATCH) { slpflag = 0; slptimeo = 2 * hz; -- Doug Rabson, Microsoft RenderMorphics Ltd. Mail: dfr@render.com Phone: +44 171 734 3761 FAX: +44 171 734 6426 From owner-freebsd-lite2 Thu Oct 10 07:30:43 1996 Return-Path: owner-lite2 Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id HAA21237 for lite2-outgoing; Thu, 10 Oct 1996 07:30:43 -0700 (PDT) Received: from crh.cl.msu.edu (crh.cl.msu.edu [35.8.1.24]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id HAA21231 for ; Thu, 10 Oct 1996 07:30:41 -0700 (PDT) Received: (from henrich@localhost) by crh.cl.msu.edu (8.7.6/8.7.3) id KAA00237; Thu, 10 Oct 1996 10:29:58 -0400 (EDT) From: Charles Henrich Message-Id: <199610101429.KAA00237@crh.cl.msu.edu> Subject: Re: Delayed write patch To: dfr@render.com (Doug Rabson) Date: Thu, 10 Oct 1996 10:29:58 -0400 (EDT) Cc: simokawa@sat.t.u-tokyo.ac.jp, lite2@freebsd.org In-Reply-To: from Doug Rabson at "Oct 10, 96 01:17:25 pm" X-Mailer: ELM [version 2.4ME+ PL22 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-lite2@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > Thanks for the patch. It improves performance for my SGI from ~720k/sec > to ~870k/sec. I modified the patch (and the nfs_dwrite patch) slightly > and added comments to try to explain what is happening. Could you check > the comments to make sure they agree with your understanding of the > problem and I will commit this patch to current probably tomorrow. A few more observations. I did more performance tests with the v2 UDP, as per usual all were not good. I did notice that the system was being locked out during the tests, but not solid, I.e. after a few seconds my other transactions (to different NFS hosts) would make it out and back. I then switched the TCP v3, this didnt help at first? After about the third test everything starting running extremely well, however I do not believe the results. After completing my tests I attempted to unmount the sun, no such luck "device busy". It took a reboot (which reported "Unabl to unmount") to clear that up. Next test, after the clean boot. CP the netscape binary to the partition, that worked fine. However, when I then did an rm on the netscape binary, my system locked up dead solid. I think im going to back out those patches :) -Crh Charles Henrich Michigan State University henrich@msu.edu http://pilot.msu.edu/~henrich From owner-freebsd-lite2 Thu Oct 10 09:50:03 1996 Return-Path: owner-lite2 Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id JAA01210 for lite2-outgoing; Thu, 10 Oct 1996 09:50:03 -0700 (PDT) Received: from agora.rdrop.com (root@agora.rdrop.com [199.2.210.241]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id JAA01168 for ; Thu, 10 Oct 1996 09:49:59 -0700 (PDT) Received: from uno.sat.t.u-tokyo.ac.jp by agora.rdrop.com with smtp (Smail3.1.29.1 #17) id m0vBOIx-00093DC; Thu, 10 Oct 96 09:49 PDT Received: by uno.sat.t.u-tokyo.ac.jp (8.7.3+2.6Wbeta5/8.7.3) with ESMTP id BAA09434; Fri, 11 Oct 1996 01:49:27 +0900 (JST) To: Doug Rabson Cc: henrich@crh.cl.msu.edu, lite2@freebsd.org Subject: Re: Delayed write patch X-Mailer: Mew version 1.06 on Emacs 19.28.1, Mule 2.3 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Date: Fri, 11 Oct 1996 01:49:27 +0900 Message-ID: <9432.844966167@sat.t.u-tokyo.ac.jp> From: Hidetoshi Shimokawa Sender: owner-lite2@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, dfr> Thanks for the patch. It improves performance for my SGI from ~720k/sec dfr> to ~870k/sec. I modified the patch (and the nfs_dwrite patch) slightly I'm glad to hear this. Are you getting it under NFSv2 and 10BASE? dfr> and added comments to try to explain what is happening. Could you check dfr> the comments to make sure they agree with your understanding of the dfr> problem and I will commit this patch to current probably tomorrow. As you know, I'm not a native speaker of English, so I'm not sure that vfs.nfs.dwrite is a good notation. If you think vfs.nfs.delwri or another notation is better, I have no objection. dfr> + * If nfs_rcvlock() returns EALREADY, that means that dfr> + * the reply has already been recieved by another dfr> + * process and we can return immediately. In this dfr> + * case, the lock is not taken to avoid races with dfr> + * other processes. "In this case, the lock is not taken because it is not necessary." dfr> + /* dfr> + * If our reply was recieved while we were sleeping, dfr> + * then just return without taking the lock to avoid a dfr> + * situation where a single iod could 'capture' the dfr> + * recieve lock. dfr> + */ "..., then just return without taking the lock. It is necessary when a single iod 'captures' the recive lock. This situation often occurs while many 'delayed write buffers' are being processed" Actually, under delayed write process, single iod keeps recive lock, but other iods can process its reply without block. /\ Hidetoshi Shimokawa \/ simokawa@sat.t.u-tokyo.ac.jp PGP public key: finger -l simokawa@sat.t.u-tokyo.ac.jp From owner-freebsd-lite2 Thu Oct 10 10:29:36 1996 Return-Path: owner-lite2 Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id KAA04510 for lite2-outgoing; Thu, 10 Oct 1996 10:29:36 -0700 (PDT) Received: from who.cdrom.com (who.cdrom.com [204.216.27.3]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id KAA04502 for ; Thu, 10 Oct 1996 10:29:24 -0700 (PDT) Received: from uno.sat.t.u-tokyo.ac.jp (uno.sat.t.u-tokyo.ac.jp [133.11.70.160]) by who.cdrom.com (8.7.5/8.6.11) with ESMTP id KAA12538 for ; Thu, 10 Oct 1996 10:29:18 -0700 (PDT) Received: by uno.sat.t.u-tokyo.ac.jp (8.7.3+2.6Wbeta5/8.7.3) with ESMTP id CAA10040; Fri, 11 Oct 1996 02:26:18 +0900 (JST) To: henrich@crh.cl.msu.edu Cc: dfr@render.com, lite2@freebsd.org Subject: Re: Delayed write patch In-Reply-To: Your message of "Thu, 10 Oct 1996 10:29:58 -0400 (EDT)" References: <199610101429.KAA00237@crh.cl.msu.edu> X-Mailer: Mew version 1.06 on Emacs 19.28.1, Mule 2.3 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Date: Fri, 11 Oct 1996 02:26:18 +0900 Message-ID: <10038.844968378@sat.t.u-tokyo.ac.jp> From: Hidetoshi Shimokawa Sender: owner-lite2@freebsd.org X-Loop: FreeBSD.org Precedence: bulk >I did more performance tests with the v2 UDP, as per usual all were not good. >I did notice that the system was being locked out during the tests, but not >solid, I.e. after a few seconds my other transactions (to different NFS hosts) >would make it out and back. > >I then switched the TCP v3, this didnt help at first? After about the third >test everything starting running extremely well, however I do not believe the >results. After completing my tests I attempted to unmount the sun, no such >luck "device busy". It took a reboot (which reported "Unabl to unmount") to >clear that up. > >Next test, after the clean boot. CP the netscape binary to the partition, that >worked fine. However, when I then did an rm on the netscape binary, my system >locked up dead solid. > >I think im going to back out those patches :) Are you using NQNFS? Did you try "sysctl -w vfs.nfs.dwrite=0" ? >-Crh > > Charles Henrich Michigan State University henrich@msu.edu > > http://pilot.msu.edu/~henrich /\ Hidetoshi Shimokawa \/ simokawa@sat.t.u-tokyo.ac.jp PGP public key: finger -l simokawa@sat.t.u-tokyo.ac.jp From owner-freebsd-lite2 Fri Oct 11 02:35:35 1996 Return-Path: owner-lite2 Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id CAA03277 for lite2-outgoing; Fri, 11 Oct 1996 02:35:35 -0700 (PDT) Received: from minnow.render.com (render.demon.co.uk [158.152.30.118]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id CAA03248 for ; Fri, 11 Oct 1996 02:35:26 -0700 (PDT) Received: from minnow.render.com (minnow.render.com [193.195.178.1]) by minnow.render.com (8.6.12/8.6.9) with SMTP id KAA23835; Fri, 11 Oct 1996 10:32:06 +0100 Date: Fri, 11 Oct 1996 10:32:04 +0100 (BST) From: Doug Rabson To: Charles Henrich cc: simokawa@sat.t.u-tokyo.ac.jp, lite2@freebsd.org Subject: Re: Delayed write patch In-Reply-To: <199610101429.KAA00237@crh.cl.msu.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-lite2@freebsd.org X-Loop: FreeBSD.org Precedence: bulk On Thu, 10 Oct 1996, Charles Henrich wrote: > > Thanks for the patch. It improves performance for my SGI from ~720k/sec > > to ~870k/sec. I modified the patch (and the nfs_dwrite patch) slightly > > and added comments to try to explain what is happening. Could you check > > the comments to make sure they agree with your understanding of the > > problem and I will commit this patch to current probably tomorrow. > > A few more observations. > > I did more performance tests with the v2 UDP, as per usual all were not good. > I did notice that the system was being locked out during the tests, but not > solid, I.e. after a few seconds my other transactions (to different NFS hosts) > would make it out and back. > > I then switched the TCP v3, this didnt help at first? After about the third > test everything starting running extremely well, however I do not believe the > results. After completing my tests I attempted to unmount the sun, no such > luck "device busy". It took a reboot (which reported "Unabl to unmount") to > clear that up. > > Next test, after the clean boot. CP the netscape binary to the partition, that > worked fine. However, when I then did an rm on the netscape binary, my system > locked up dead solid. > > I think im going to back out those patches :) Could you back out all the patches except the one to nfs_socket.c to nfs_rcvlock and its caller and the nfs_bio one to use a sysctl variable vfs.nfs.dwrite to decided whether to use delayed writes. In particular, could you back out the NFSv3 fixes which I sent for nfs_vnops.c. Could you run your tests again on V2 and V3 with vfs.nfs.dwrite=0 and vfs.nfs.dwrite=1. If we get some positive results that way, we can add back the v3 fixes a piece at a time to see what is broken. -- Doug Rabson, Microsoft RenderMorphics Ltd. Mail: dfr@render.com Phone: +44 171 734 3761 FAX: +44 171 734 6426 From owner-freebsd-lite2 Fri Oct 11 02:35:52 1996 Return-Path: owner-lite2 Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id CAA03300 for lite2-outgoing; Fri, 11 Oct 1996 02:35:52 -0700 (PDT) Received: from minnow.render.com (render.demon.co.uk [158.152.30.118]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id CAA03265 for ; Fri, 11 Oct 1996 02:35:31 -0700 (PDT) Received: from minnow.render.com (minnow.render.com [193.195.178.1]) by minnow.render.com (8.6.12/8.6.9) with SMTP id KAA23839; Fri, 11 Oct 1996 10:34:20 +0100 Date: Fri, 11 Oct 1996 10:34:20 +0100 (BST) From: Doug Rabson Reply-To: Doug Rabson To: Hidetoshi Shimokawa cc: henrich@crh.cl.msu.edu, lite2@freebsd.org Subject: Re: Delayed write patch In-Reply-To: <9432.844966167@sat.t.u-tokyo.ac.jp> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-lite2@freebsd.org X-Loop: FreeBSD.org Precedence: bulk On Fri, 11 Oct 1996, Hidetoshi Shimokawa wrote: > Hi, > > dfr> Thanks for the patch. It improves performance for my SGI from ~720k/sec > dfr> to ~870k/sec. I modified the patch (and the nfs_dwrite patch) slightly > I'm glad to hear this. > Are you getting it under NFSv2 and 10BASE? > > dfr> and added comments to try to explain what is happening. Could you check > dfr> the comments to make sure they agree with your understanding of the > dfr> problem and I will commit this patch to current probably tomorrow. > As you know, I'm not a native speaker of English, so I'm not sure that > vfs.nfs.dwrite is a good notation. If you think vfs.nfs.delwri or > another notation is better, I have no objection. I think vfs.nfs.dwrite is fine. > [...] > > "In this case, the lock is not taken because it is not necessary." > > [...] > > "..., then just return without taking the lock. > It is necessary when a single iod 'captures' the recive lock. > This situation often occurs while many 'delayed write buffers' are > being processed" I have included your new wording, thanks. > > Actually, under delayed write process, single iod keeps recive lock, > but other iods can process its reply without block. I think I understand it a bit better now. -- Doug Rabson, Microsoft RenderMorphics Ltd. Mail: dfr@render.com Phone: +44 171 734 3761 FAX: +44 171 734 6426