From owner-freebsd-arch Sun Mar 24 3:45:21 2002 Delivered-To: freebsd-arch@freebsd.org Received: from storm.FreeBSD.org.uk (storm.FreeBSD.org.uk [194.242.139.170]) by hub.freebsd.org (Postfix) with ESMTP id 3D86D37B427 for ; Sun, 24 Mar 2002 03:45:08 -0800 (PST) Received: (from uucp@localhost) by storm.FreeBSD.org.uk (8.11.6/8.11.6) with UUCP id g2OBj6p35231 for arch@freebsd.org; Sun, 24 Mar 2002 11:45:07 GMT (envelope-from mark@grimreaper.grondar.za) Received: from grimreaper (localhost [127.0.0.1]) by grimreaper.grondar.org (8.12.2/8.12.2) with ESMTP id g2OBfZXl019225 for ; Sun, 24 Mar 2002 11:41:35 GMT (envelope-from mark@grimreaper.grondar.za) Message-Id: <200203241141.g2OBfZXl019225@grimreaper.grondar.org> To: arch@freebsd.org Subject: Why isn't __progname declared in a header? Date: Sun, 24 Mar 2002 11:41:35 +0000 From: Mark Murray Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi __progname is (inconsistently) declared all over the place. Why is this not in a header? Next question, what header should it go into? I'm quite happy to do the work. M -- o Mark Murray \_ O.\_ Warning: this .sig is umop ap!sdn #application/octet-stream; name=cv.pdf [Mark Murray CV PDF] cv.pdf #text/plain; name=cv.txt [Mark Murray CV Plain Text] cv.txt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 24 5:24:35 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 378FE37B400 for ; Sun, 24 Mar 2002 05:24:32 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id AAA15055; Mon, 25 Mar 2002 00:24:25 +1100 Date: Mon, 25 Mar 2002 00:24:47 +1100 (EST) From: Bruce Evans X-X-Sender: To: Mark Murray Cc: Subject: Re: Why isn't __progname declared in a header? In-Reply-To: <200203241141.g2OBfZXl019225@grimreaper.grondar.org> Message-ID: <20020325001528.B44805-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, 24 Mar 2002, Mark Murray wrote: > __progname is (inconsistently) declared all over the place. Why is > this not in a header? Because all over the place is unwarrantedly chummy with the implementation. __progname should only be used in csu/*/crt1.c, getprogname(3), setprogname(3) and perhaps in other parts of the implementation. > Next question, what header should it go into? I'm quite happy to do > the work. libc/include/libc_private.h is almost right. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 24 6:40:21 2002 Delivered-To: freebsd-arch@freebsd.org Received: from storm.FreeBSD.org.uk (storm.FreeBSD.org.uk [194.242.139.170]) by hub.freebsd.org (Postfix) with ESMTP id 7FC7D37B41A for ; Sun, 24 Mar 2002 06:40:15 -0800 (PST) Received: (from uucp@localhost) by storm.FreeBSD.org.uk (8.11.6/8.11.6) with UUCP id g2OEe9w40120; Sun, 24 Mar 2002 14:40:09 GMT (envelope-from mark@grimreaper.grondar.za) Received: from grimreaper (localhost [127.0.0.1]) by grimreaper.grondar.org (8.12.2/8.12.2) with ESMTP id g2OEbuXl083121; Sun, 24 Mar 2002 14:37:56 GMT (envelope-from mark@grimreaper.grondar.za) Message-Id: <200203241437.g2OEbuXl083121@grimreaper.grondar.org> To: Bruce Evans Cc: arch@FreeBSD.ORG Subject: Re: Why isn't __progname declared in a header? References: <20020325001528.B44805-100000@gamplex.bde.org> In-Reply-To: <20020325001528.B44805-100000@gamplex.bde.org> ; from Bruce Evans "Mon, 25 Mar 2002 00:24:47 +1100." Date: Sun, 24 Mar 2002 14:37:55 +0000 From: Mark Murray Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > On Sun, 24 Mar 2002, Mark Murray wrote: > > > __progname is (inconsistently) declared all over the place. Why is > > this not in a header? > > Because all over the place is unwarrantedly chummy with the implementation. > __progname should only be used in csu/*/crt1.c, getprogname(3), > setprogname(3) and perhaps in other parts of the implementation. OK, I've fixed that in src/lib/lib* > > Next question, what header should it go into? I'm quite happy to do > > the work. > > libc/include/libc_private.h is almost right. Hmm. It is implemented in csu/*/crt[01].c, so how about csu/common/_progname.h? M -- o Mark Murray \_ O.\_ Warning: this .sig is umop ap!sdn To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 24 8:51:19 2002 Delivered-To: freebsd-arch@freebsd.org Received: from dragon.nuxi.com (trang.nuxi.com [66.92.13.169]) by hub.freebsd.org (Postfix) with ESMTP id D90F037B41C for ; Sun, 24 Mar 2002 08:50:57 -0800 (PST) Received: from dragon.nuxi.com (obrien@localhost [127.0.0.1]) by dragon.nuxi.com (8.12.2/8.12.2) with ESMTP id g2OGovYm005589; Sun, 24 Mar 2002 08:50:57 -0800 (PST) (envelope-from obrien@dragon.nuxi.com) Received: (from obrien@localhost) by dragon.nuxi.com (8.12.2/8.12.2/Submit) id g2OGnf9C005585; Sun, 24 Mar 2002 08:49:41 -0800 (PST) Date: Sun, 24 Mar 2002 08:49:41 -0800 From: "David O'Brien" To: Mark Murray Cc: Bruce Evans , arch@FreeBSD.ORG Subject: Re: Why isn't __progname declared in a header? Message-ID: <20020324084941.B26339@dragon.nuxi.com> Reply-To: obrien@FreeBSD.ORG References: <20020325001528.B44805-100000@gamplex.bde.org> <200203241437.g2OEbuXl083121@grimreaper.grondar.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200203241437.g2OEbuXl083121@grimreaper.grondar.org>; from mark@grondar.za on Sun, Mar 24, 2002 at 02:37:55PM +0000 X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD group X-Pgp-Rsa-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Rsa-Keyid: 1024/34F9F9D5 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, Mar 24, 2002 at 02:37:55PM +0000, Mark Murray wrote: > > > Next question, what header should it go into? I'm quite happy to do > > > the work. > > > > libc/include/libc_private.h is almost right. > > Hmm. It is implemented in csu/*/crt[01].c, so how about > csu/common/_progname.h? Why create a header for just one line that is easy to duplicate in the only 4 places it is used? -- -- David (obrien@FreeBSD.org) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 24 9: 2: 4 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 947E637B404 for ; Sun, 24 Mar 2002 09:01:54 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id EAA22766; Mon, 25 Mar 2002 04:00:43 +1100 Date: Mon, 25 Mar 2002 04:01:06 +1100 (EST) From: Bruce Evans X-X-Sender: To: Mark Murray Cc: Subject: Re: Why isn't __progname declared in a header? In-Reply-To: <200203241437.g2OEbuXl083121@grimreaper.grondar.org> Message-ID: <20020325035923.I46460-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, 24 Mar 2002, Mark Murray wrote: > > On Sun, 24 Mar 2002, Mark Murray wrote: > > > > > __progname is (inconsistently) declared all over the place. Why is > > > this not in a header? > > > > Because all over the place is unwarrantedly chummy with the implementation. > > __progname should only be used in csu/*/crt1.c, getprogname(3), > > setprogname(3) and perhaps in other parts of the implementation. > > OK, I've fixed that in src/lib/lib* > > > > Next question, what header should it go into? I'm quite happy to do > > > the work. > > > > libc/include/libc_private.h is almost right. > > Hmm. It is implemented in csu/*/crt[01].c, so how about > csu/common/_progname.h? Further from being almost right. It would be a new header with very little in it, and isn't in libc's tree any more than libc's header is in csu's tree. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 24 9:10: 9 2002 Delivered-To: freebsd-arch@freebsd.org Received: from storm.FreeBSD.org.uk (storm.FreeBSD.org.uk [194.242.139.170]) by hub.freebsd.org (Postfix) with ESMTP id 820A637B417; Sun, 24 Mar 2002 09:10:03 -0800 (PST) Received: (from uucp@localhost) by storm.FreeBSD.org.uk (8.11.6/8.11.6) with UUCP id g2OHA2D92665; Sun, 24 Mar 2002 17:10:02 GMT (envelope-from mark@grimreaper.grondar.za) Received: from grimreaper (localhost [127.0.0.1]) by grimreaper.grondar.org (8.12.2/8.12.2) with ESMTP id g2OH9AXl056805; Sun, 24 Mar 2002 17:09:10 GMT (envelope-from mark@grimreaper.grondar.za) Message-Id: <200203241709.g2OH9AXl056805@grimreaper.grondar.org> To: obrien@FreeBSD.ORG Cc: arch@FreeBSD.ORG Subject: Re: Why isn't __progname declared in a header? References: <20020324084941.B26339@dragon.nuxi.com> In-Reply-To: <20020324084941.B26339@dragon.nuxi.com> ; from "David O'Brien" "Sun, 24 Mar 2002 08:49:41 PST." Date: Sun, 24 Mar 2002 17:09:10 +0000 From: Mark Murray Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > > Hmm. It is implemented in csu/*/crt[01].c, so how about > > csu/common/_progname.h? > > Why create a header for just one line that is easy to duplicate in the > only 4 places it is used? 1) To prevent it from diverging in the (currently only) 4 architectures that we have. 2) To coerce stronger type-checking in its implementation and external usage. M -- o Mark Murray \_ O.\_ Warning: this .sig is umop ap!sdn To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 24 9:15:11 2002 Delivered-To: freebsd-arch@freebsd.org Received: from storm.FreeBSD.org.uk (storm.FreeBSD.org.uk [194.242.139.170]) by hub.freebsd.org (Postfix) with ESMTP id E4CA037B419 for ; Sun, 24 Mar 2002 09:15:08 -0800 (PST) Received: (from uucp@localhost) by storm.FreeBSD.org.uk (8.11.6/8.11.6) with UUCP id g2OHF4R96158; Sun, 24 Mar 2002 17:15:04 GMT (envelope-from mark@grimreaper.grondar.za) Received: from grimreaper (localhost [127.0.0.1]) by grimreaper.grondar.org (8.12.2/8.12.2) with ESMTP id g2OHEfXl061582; Sun, 24 Mar 2002 17:14:41 GMT (envelope-from mark@grimreaper.grondar.za) Message-Id: <200203241714.g2OHEfXl061582@grimreaper.grondar.org> To: Bruce Evans Cc: arch@FreeBSD.ORG Subject: Re: Why isn't __progname declared in a header? References: <20020325035923.I46460-100000@gamplex.bde.org> In-Reply-To: <20020325035923.I46460-100000@gamplex.bde.org> ; from Bruce Evans "Mon, 25 Mar 2002 04:01:06 +1100." Date: Sun, 24 Mar 2002 17:14:41 +0000 From: Mark Murray Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > > > > Next question, what header should it go into? I'm quite happy to do > > > > the work. > > > > > > libc/include/libc_private.h is almost right. > > > > Hmm. It is implemented in csu/*/crt[01].c, so how about > > csu/common/_progname.h? > > Further from being almost right. It would be a new header with very little > in it, and isn't in libc's tree any more than libc's header is in csu's > tree. So what direction do I go in the get it closer to "right"? __progname is implemented in csu/, so it make sense for me to have its header there. Only two files in any other library (lib/libc/gen/[gs]etprogname.c) actually need this information, so it makes sense (to me anyway) for them to have some kind of special case to get it. I don't like a plain declaration, as that makes diverting declarations (more) possible. M -- o Mark Murray \_ O.\_ Warning: this .sig is umop ap!sdn To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 24 10:30:35 2002 Delivered-To: freebsd-arch@freebsd.org Received: from dragon.nuxi.com (trang.nuxi.com [66.92.13.169]) by hub.freebsd.org (Postfix) with ESMTP id F1C3337B404 for ; Sun, 24 Mar 2002 10:30:29 -0800 (PST) Received: from dragon.nuxi.com (obrien@localhost [127.0.0.1]) by dragon.nuxi.com (8.12.2/8.12.2) with ESMTP id g2OIUTYm012628; Sun, 24 Mar 2002 10:30:29 -0800 (PST) (envelope-from obrien@dragon.nuxi.com) Received: (from obrien@localhost) by dragon.nuxi.com (8.12.2/8.12.2/Submit) id g2OITCXL012622; Sun, 24 Mar 2002 10:29:12 -0800 (PST) Date: Sun, 24 Mar 2002 10:29:12 -0800 From: "David O'Brien" To: Mark Murray Cc: arch@FreeBSD.ORG Subject: Re: Why isn't __progname declared in a header? Message-ID: <20020324102912.A8262@dragon.nuxi.com> Reply-To: obrien@FreeBSD.ORG References: <20020324084941.B26339@dragon.nuxi.com> <200203241709.g2OH9AXl056805@grimreaper.grondar.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200203241709.g2OH9AXl056805@grimreaper.grondar.org>; from mark@grondar.za on Sun, Mar 24, 2002 at 05:09:10PM +0000 X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD group X-Pgp-Rsa-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Rsa-Keyid: 1024/34F9F9D5 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, Mar 24, 2002 at 05:09:10PM +0000, Mark Murray wrote: > > > Hmm. It is implemented in csu/*/crt[01].c, so how about > > > csu/common/_progname.h? > > > > Why create a header for just one line that is easy to duplicate in the > > only 4 places it is used? > > 2) To coerce stronger type-checking in its implementation and external > usage. Are you waning other consumers to include this file also? If so, we should put it in a more central location. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Mar 24 11:15:18 2002 Delivered-To: freebsd-arch@freebsd.org Received: from storm.FreeBSD.org.uk (storm.FreeBSD.org.uk [194.242.139.170]) by hub.freebsd.org (Postfix) with ESMTP id 1B94937B41B; Sun, 24 Mar 2002 11:15:08 -0800 (PST) Received: (from uucp@localhost) by storm.FreeBSD.org.uk (8.11.6/8.11.6) with UUCP id g2OJF7c97161; Sun, 24 Mar 2002 19:15:07 GMT (envelope-from mark@grimreaper.grondar.za) Received: from grimreaper (localhost [127.0.0.1]) by grimreaper.grondar.org (8.12.2/8.12.2) with ESMTP id g2OJBFXl080109; Sun, 24 Mar 2002 19:11:15 GMT (envelope-from mark@grimreaper.grondar.za) Message-Id: <200203241911.g2OJBFXl080109@grimreaper.grondar.org> To: obrien@FreeBSD.ORG Cc: arch@FreeBSD.ORG Subject: Re: Why isn't __progname declared in a header? References: <20020324102912.A8262@dragon.nuxi.com> In-Reply-To: <20020324102912.A8262@dragon.nuxi.com> ; from "David O'Brien" "Sun, 24 Mar 2002 10:29:12 PST." Date: Sun, 24 Mar 2002 19:11:14 +0000 From: Mark Murray Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > > 2) To coerce stronger type-checking in its implementation and external > > usage. > > Are you waning other consumers to include this file also? > If so, we should put it in a more central location. No - there are only two places it should be used - inside csu/*/crt[01].c and in lib/libc/gen/[gs]etprogname.c. The rest are bogus and need to be replaced with [gs]etprogname(3). M -- o Mark Murray \_ O.\_ Warning: this .sig is umop ap!sdn To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Mar 26 10:26:48 2002 Delivered-To: freebsd-arch@freebsd.org Received: from aldan.algebra.com (aldan.algebra.com [216.254.65.224]) by hub.freebsd.org (Postfix) with ESMTP id 4CEBF37B419 for ; Tue, 26 Mar 2002 10:26:45 -0800 (PST) Received: from aldan.algebra.com (localhost [127.0.0.1]) by aldan.algebra.com (8.12.2/8.12.2) with ESMTP id g2QIQ7p0011209; Tue, 26 Mar 2002 13:26:12 -0500 (EST) (envelope-from mi@aldan.algebra.com) Message-Id: <200203261826.g2QIQ7p0011209@aldan.algebra.com> Date: Tue, 26 Mar 2002 13:26:07 -0500 (EST) From: Mikhail Teterin Subject: Re: review request for bin/11294 To: des@ofug.org Cc: arch@FreeBSD.org In-Reply-To: MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 24 Mar, Dag-Erling Smorgrav wrote: > Mikhail Teterin writes: >> But this implementation has drawbacks, that mine does not, IMHO: >> >> . the configuration is system wide -- which is not neccessarily >> desirable; >> [...] > > This and all your other objections can be fixed using a global > variable that overrides the symlink (or an API function that sets the > target host). My primary objection is that the problem you are solving -- logging to some other host by default -- is different from mine -- logging to some other host at the caller's discretion. Yes, the caller can call the new API function you suggest (or set a global variable itself, which is uglier, IMHO). But the only drawback of my idea, that was pointed out so far is that it _adds a new API call_. If such an addition _can_ be tolerated -- and it looks from your response like it can -- I think, solving my problem your way is inferior to my proposed solution. Which is not to say, the solution you provided is inferior over all -- it is just for a different problem. -mi To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Mar 27 2:20: 8 2002 Delivered-To: freebsd-arch@freebsd.org Received: from matrix.eurocontrol.fr (matrix.eurocontrol.fr [147.196.254.254]) by hub.freebsd.org (Postfix) with ESMTP id 5B56D37B416 for ; Wed, 27 Mar 2002 02:19:56 -0800 (PST) Received: from caerdonn.eurocontrol.fr (caerdonn.eurocontrol.fr [147.196.51.214]) (using TLSv1 with cipher EDH-RSA-DES-CBC3-SHA (168/168 bits)) (Client CN "caerdonn.eurocontrol.fr", Issuer CN "CA ITM" (not verified)) by matrix.eurocontrol.fr (Postfix/TLS) with ESMTP id 7170C21C7; Wed, 27 Mar 2002 11:19:50 +0100 (CET) Received: by caerdonn.eurocontrol.fr (Postfix/TLS, from userid 1193) id 156A4B; Wed, 27 Mar 2002 11:19:48 +0100 (CET) Date: Wed, 27 Mar 2002 11:19:47 +0100 From: Ollivier Robert To: mckusick@mckusick.com Cc: arch@FreeBSD.org Subject: [PATCH] MFC the full filesystem softupdates fix ? Message-ID: <20020327101947.GA2322@caerdonn.eurocontrol.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.3.28i X-Operating-System: FreeBSD 5.0-CURRENT Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Following some discussions in the French fr.comp.os.bsd newsgroup about softupdates and the full filesystem problem (which is fixed in CURRENT thanks to you), here a proposed diff to merge the fix into STABLE. Can you (and -arch) review it ? It would be nice to have this fixed in STABLE as well. Thanks. Index: ffs_alloc.c =================================================================== RCS file: /home/ncvs/src/sys/ufs/ffs/ffs_alloc.c,v retrieving revision 1.64.2.2 diff -u -2 -r1.64.2.2 ffs_alloc.c --- ffs_alloc.c 21 Sep 2001 19:15:21 -0000 1.64.2.2 +++ ffs_alloc.c 27 Mar 2002 10:08:29 -0000 @@ -107,5 +107,5 @@ register struct fs *fs; ufs_daddr_t bno; - int cg; + int cg, reclaimed; #ifdef QUOTA int error; @@ -124,4 +124,6 @@ panic("ffs_alloc: missing credential"); #endif /* DIAGNOSTIC */ + reclaimed = 0; +retry: if (size == fs->fs_bsize && fs->fs_cstotal.cs_nbfree == 0) goto nospace; @@ -155,4 +157,9 @@ #endif nospace: + if (fs->fs_pendingblocks > 0 && reclaimed == 0) { + reclaimed = 1; + softdep_request_cleanup(fs, ITOV(ip)); + goto retry; + } ffs_fserr(fs, cred->cr_uid, "file system full"); uprintf("\n%s: write failed, file system is full\n", fs->fs_fsmnt); @@ -177,7 +184,7 @@ struct buf **bpp; { - register struct fs *fs; + struct fs *fs; struct buf *bp; - int cg, request, error; + int cg, request, error, reclaimed; ufs_daddr_t bprev, bno; @@ -198,4 +205,6 @@ if (cred->cr_uid != 0 && freespace(fs, fs->fs_minfree) - numfrags(fs, nsize - osize) < 0) + reclaimed = 0; +retry: goto nospace; if ((bprev = ip->i_db[lbprev]) == 0) { @@ -208,5 +217,5 @@ * Allocate the extra space in the buffer. */ - error = bread(ITOV(ip), lbprev, osize, NOCRED, &bp); + error = bread(vp, lbprev, osize, NOCRED, &bp); if (error) { brelse(bp); @@ -295,5 +304,5 @@ if (bno > 0) { bp->b_blkno = fsbtodb(fs, bno); - if (!DOINGSOFTDEP(ITOV(ip))) + if (!DOINGSOFTDEP(vp)) ffs_blkfree(ip, bprev, (long)osize); if (nsize < request) @@ -319,4 +328,9 @@ * no space available */ + if (fs->fs_pendingblocks > 0 && reclaimed == 0) { + reclaimed = 1; + softdep_request_cleanup(fs, vp); + goto retry; + } ffs_fserr(fs, cred->cr_uid, "file system full"); uprintf("\n%s: write failed, file system is full\n", fs->fs_fsmnt); Index: ffs_extern.h =================================================================== RCS file: /home/ncvs/src/sys/ufs/ffs/ffs_extern.h,v retrieving revision 1.30 diff -u -2 -r1.30 ffs_extern.h --- ffs_extern.h 9 Jan 2000 22:40:02 -0000 1.30 +++ ffs_extern.h 27 Mar 2002 10:05:03 -0000 @@ -115,4 +115,5 @@ void softdep_load_inodeblock __P((struct inode *)); void softdep_freefile __P((struct vnode *, ino_t, int)); +int softdep_request_cleanup __P((struct fs *, struct vnode *)); void softdep_setup_freeblocks __P((struct inode *, off_t)); void softdep_setup_inomapdep __P((struct buf *, struct inode *, ino_t)); Index: ffs_softdep.c =================================================================== RCS file: /home/ncvs/src/sys/ufs/ffs/ffs_softdep.c,v retrieving revision 1.57.2.11 diff -u -2 -r1.57.2.11 ffs_softdep.c --- ffs_softdep.c 5 Feb 2002 18:46:53 -0000 1.57.2.11 +++ ffs_softdep.c 27 Mar 2002 10:12:48 -0000 @@ -508,7 +508,8 @@ static struct proc *filesys_syncer; /* proc of filesystem syncer process */ static int req_clear_inodedeps; /* syncer process flush some inodedeps */ -#define FLUSH_INODES 1 +#define FLUSH_INODES 1 static int req_clear_remove; /* syncer process flush some freeblks */ -#define FLUSH_REMOVE 2 +#define FLUSH_REMOVE 2 +#define FLUSH_REMOVE_WAIT 3 /* * runtime statistics @@ -703,5 +704,5 @@ if (wk == 0) { FREE_LOCK(&lk); - return (0); + return (-1); } WORKLIST_REMOVE(wk); @@ -4561,4 +4562,39 @@ /* + * Called by the allocation routines when they are about to fail + * in the hope that we can free up some disk space. + * + * First check to see if the work list has anything on it. If it has, + * clean up entries until we successfully free some space. Because this + * process holds inodes locked, we cannot handle any remove requests + * that might block on a locked inode as that could lead to deadlock. + * If the worklist yields no free space, encourage the syncer daemon + * to help us. In no event will we try for longer than tickdelay seconds. + */ +int +softdep_request_cleanup(fs, vp) + struct fs *fs; + struct vnode *vp; +{ + long starttime, needed; + + needed = fs->fs_cstotal.cs_nbfree + fs->fs_contigsumsize; + starttime = time_second + tickdelay; + if (UFS_UPDATE(vp, 1) != 0) + return (0); + while (fs->fs_pendingblocks > 0 && fs->fs_cstotal.cs_nbfree <= needed) { + if (time_second > starttime) + return (0); + if (num_on_worklist > 0 && + process_worklist_item(NULL, LK_NOWAIT) != -1) { + stat_worklist_push += 1; + continue; + } + request_cleanup(FLUSH_REMOVE_WAIT, 0); + } + return (1); +} + +/* * If memory utilization has gotten too high, deliberately slow things * down and speed up the I/O processing. @@ -4595,4 +4631,10 @@ /* + * Next, we attempt to speed up the syncer process. If that + * is successful, then we allow the process to continue. + */ + if (speedup_syncer() && resource != FLUSH_REMOVE_WAIT) + return(0); + /* * If we are resource constrained on inode dependencies, try * flushing some dirty inodes. Otherwise, we are constrained @@ -4613,4 +4655,5 @@ case FLUSH_REMOVE: + case FLUSH_REMOVE_WAIT: stat_blk_limit_push += 1; req_clear_remove += 1; -- Ollivier ROBERT -=- Eurocontrol EEC/ITM -=- Ollivier.Robert@eurocontrol.fr FreeBSD caerdonn.eurocontrol.fr 5.0-CURRENT #46: Wed Jan 3 15:52:00 CET 2001 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Mar 27 12:20:54 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by hub.freebsd.org (Postfix) with ESMTP id 111C737B425; Wed, 27 Mar 2002 12:20:10 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc52.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020327202009.WPNJ1147.rwcrmhc52.attbi.com@InterJet.elischer.org>; Wed, 27 Mar 2002 20:20:09 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id MAA48226; Wed, 27 Mar 2002 12:10:37 -0800 (PST) Date: Wed, 27 Mar 2002 12:10:36 -0800 (PST) From: Julian Elischer To: arch@freebsd.org Cc: smp@freeBSD.org Subject: SMP safe reference counting Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG [please remove -smp from your reply] Once again on the SMP list a lock is being used to make a reference count safe. I'd like to re-raise the issue of a safe reference counting fascility. what would be the semantics? typedef volatile u_int ref_cnt; typedef void ref_free(void *); reference_add(ref_cnt *); reference_drop(ref_cnt *, ref_free *, void *); __inline void reference_add(ref_cnt *cnt) { atomic_inc(cnt); } /* Note I use the non-existing "atomic_inc()". I think we should have this as its SO COMMONLY used. */ __inline void reference_drop(ref_cnt *cnt, ref_free *fn. void * arg) { int newcount; int oldcount; do { newcount = (oldcount = *cnt) - 1; } while (atomic_cmp_and_set(cnt, oldcount, newcount) == ACS_FAILED); } /* I can't remember off the top of my head the cmp-and-set command */ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Mar 27 13:34:49 2002 Delivered-To: freebsd-arch@freebsd.org Received: from beastie.mckusick.com (beastie.mckusick.com [209.31.233.184]) by hub.freebsd.org (Postfix) with ESMTP id E9DE237B417 for ; Wed, 27 Mar 2002 13:34:33 -0800 (PST) Received: from beastie.mckusick.com (localhost [127.0.0.1]) by beastie.mckusick.com (8.11.6/8.11.6) with ESMTP id g2RLYTD97258; Wed, 27 Mar 2002 13:34:29 -0800 (PST) (envelope-from mckusick@beastie.mckusick.com) Message-Id: <200203272134.g2RLYTD97258@beastie.mckusick.com> To: Ollivier Robert Subject: Re: [PATCH] MFC the full filesystem softupdates fix ? Cc: arch@FreeBSD.ORG In-Reply-To: Your message of "Wed, 27 Mar 2002 11:19:47 +0100." <20020327101947.GA2322@caerdonn.eurocontrol.fr> Date: Wed, 27 Mar 2002 13:34:29 -0800 From: Kirk McKusick Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG If it was easy as your patch, there would be no problem in putting it in. The problem is that -current does not have the code to maintain the fs_pendingblocks variable. At a minimum you would have to add that code as well. It is a good deal more extensive, though at this point well enough tested that I am less fearful of adding it. Kirk McKusick =-=-=-=-=-= Date: Wed, 27 Mar 2002 11:19:47 +0100 From: Ollivier Robert To: mckusick@mckusick.com Cc: arch@FreeBSD.ORG Subject: [PATCH] MFC the full filesystem softupdates fix ? Following some discussions in the French fr.comp.os.bsd newsgroup about softupdates and the full filesystem problem (which is fixed in CURRENT thanks to you), here a proposed diff to merge the fix into STABLE. Can you (and -arch) review it ? It would be nice to have this fixed in STABLE as well. Thanks. Index: ffs_alloc.c =================================================================== RCS file: /home/ncvs/src/sys/ufs/ffs/ffs_alloc.c,v retrieving revision 1.64.2.2 diff -u -2 -r1.64.2.2 ffs_alloc.c --- ffs_alloc.c 21 Sep 2001 19:15:21 -0000 1.64.2.2 +++ ffs_alloc.c 27 Mar 2002 10:08:29 -0000 @@ -107,5 +107,5 @@ register struct fs *fs; ufs_daddr_t bno; - int cg; + int cg, reclaimed; #ifdef QUOTA int error; @@ -124,4 +124,6 @@ panic("ffs_alloc: missing credential"); #endif /* DIAGNOSTIC */ + reclaimed = 0; +retry: if (size == fs->fs_bsize && fs->fs_cstotal.cs_nbfree == 0) goto nospace; @@ -155,4 +157,9 @@ #endif nospace: + if (fs->fs_pendingblocks > 0 && reclaimed == 0) { + reclaimed = 1; + softdep_request_cleanup(fs, ITOV(ip)); + goto retry; + } ffs_fserr(fs, cred->cr_uid, "file system full"); uprintf("\n%s: write failed, file system is full\n", fs->fs_fsmnt); @@ -177,7 +184,7 @@ struct buf **bpp; { - register struct fs *fs; + struct fs *fs; struct buf *bp; - int cg, request, error; + int cg, request, error, reclaimed; ufs_daddr_t bprev, bno; @@ -198,4 +205,6 @@ if (cred->cr_uid != 0 && freespace(fs, fs->fs_minfree) - numfrags(fs, nsize - osize) < 0) + reclaimed = 0; +retry: goto nospace; if ((bprev = ip->i_db[lbprev]) == 0) { @@ -208,5 +217,5 @@ * Allocate the extra space in the buffer. */ - error = bread(ITOV(ip), lbprev, osize, NOCRED, &bp); + error = bread(vp, lbprev, osize, NOCRED, &bp); if (error) { brelse(bp); @@ -295,5 +304,5 @@ if (bno > 0) { bp->b_blkno = fsbtodb(fs, bno); - if (!DOINGSOFTDEP(ITOV(ip))) + if (!DOINGSOFTDEP(vp)) ffs_blkfree(ip, bprev, (long)osize); if (nsize < request) @@ -319,4 +328,9 @@ * no space available */ + if (fs->fs_pendingblocks > 0 && reclaimed == 0) { + reclaimed = 1; + softdep_request_cleanup(fs, vp); + goto retry; + } ffs_fserr(fs, cred->cr_uid, "file system full"); uprintf("\n%s: write failed, file system is full\n", fs->fs_fsmnt); Index: ffs_extern.h =================================================================== RCS file: /home/ncvs/src/sys/ufs/ffs/ffs_extern.h,v retrieving revision 1.30 diff -u -2 -r1.30 ffs_extern.h --- ffs_extern.h 9 Jan 2000 22:40:02 -0000 1.30 +++ ffs_extern.h 27 Mar 2002 10:05:03 -0000 @@ -115,4 +115,5 @@ void softdep_load_inodeblock __P((struct inode *)); void softdep_freefile __P((struct vnode *, ino_t, int)); +int softdep_request_cleanup __P((struct fs *, struct vnode *)); void softdep_setup_freeblocks __P((struct inode *, off_t)); void softdep_setup_inomapdep __P((struct buf *, struct inode *, ino_t)); Index: ffs_softdep.c =================================================================== RCS file: /home/ncvs/src/sys/ufs/ffs/ffs_softdep.c,v retrieving revision 1.57.2.11 diff -u -2 -r1.57.2.11 ffs_softdep.c --- ffs_softdep.c 5 Feb 2002 18:46:53 -0000 1.57.2.11 +++ ffs_softdep.c 27 Mar 2002 10:12:48 -0000 @@ -508,7 +508,8 @@ static struct proc *filesys_syncer; /* proc of filesystem syncer process */ static int req_clear_inodedeps; /* syncer process flush some inodedeps */ -#define FLUSH_INODES 1 +#define FLUSH_INODES 1 static int req_clear_remove; /* syncer process flush some freeblks */ -#define FLUSH_REMOVE 2 +#define FLUSH_REMOVE 2 +#define FLUSH_REMOVE_WAIT 3 /* * runtime statistics @@ -703,5 +704,5 @@ if (wk == 0) { FREE_LOCK(&lk); - return (0); + return (-1); } WORKLIST_REMOVE(wk); @@ -4561,4 +4562,39 @@ /* + * Called by the allocation routines when they are about to fail + * in the hope that we can free up some disk space. + * + * First check to see if the work list has anything on it. If it has, + * clean up entries until we successfully free some space. Because this + * process holds inodes locked, we cannot handle any remove requests + * that might block on a locked inode as that could lead to deadlock. + * If the worklist yields no free space, encourage the syncer daemon + * to help us. In no event will we try for longer than tickdelay seconds. + */ +int +softdep_request_cleanup(fs, vp) + struct fs *fs; + struct vnode *vp; +{ + long starttime, needed; + + needed = fs->fs_cstotal.cs_nbfree + fs->fs_contigsumsize; + starttime = time_second + tickdelay; + if (UFS_UPDATE(vp, 1) != 0) + return (0); + while (fs->fs_pendingblocks > 0 && fs->fs_cstotal.cs_nbfree <= needed) { + if (time_second > starttime) + return (0); + if (num_on_worklist > 0 && + process_worklist_item(NULL, LK_NOWAIT) != -1) { + stat_worklist_push += 1; + continue; + } + request_cleanup(FLUSH_REMOVE_WAIT, 0); + } + return (1); +} + +/* * If memory utilization has gotten too high, deliberately slow things * down and speed up the I/O processing. @@ -4595,4 +4631,10 @@ /* + * Next, we attempt to speed up the syncer process. If that + * is successful, then we allow the process to continue. + */ + if (speedup_syncer() && resource != FLUSH_REMOVE_WAIT) + return(0); + /* * If we are resource constrained on inode dependencies, try * flushing some dirty inodes. Otherwise, we are constrained @@ -4613,4 +4655,5 @@ case FLUSH_REMOVE: + case FLUSH_REMOVE_WAIT: stat_blk_limit_push += 1; req_clear_remove += 1; -- Ollivier ROBERT -=- Eurocontrol EEC/ITM -=- Ollivier.Robert@eurocontrol.fr FreeBSD caerdonn.eurocontrol.fr 5.0-CURRENT #46: Wed Jan 3 15:52:00 CET 2001 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Mar 27 13:52:24 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail16.speakeasy.net (mail16.speakeasy.net [216.254.0.216]) by hub.freebsd.org (Postfix) with ESMTP id 97FA537B419 for ; Wed, 27 Mar 2002 13:51:55 -0800 (PST) Received: (qmail 22095 invoked from network); 27 Mar 2002 21:51:55 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) by mail16.speakeasy.net (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 27 Mar 2002 21:51:55 -0000 Received: from laptop.baldwin.cx (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.11.6/8.11.6) with ESMTP id g2RLqav92810; Wed, 27 Mar 2002 16:52:36 -0500 (EST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.2 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Wed, 27 Mar 2002 16:51:57 -0500 (EST) From: John Baldwin To: Julian Elischer Subject: RE: SMP safe reference counting Cc: arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 27-Mar-2002 Julian Elischer wrote: > > [please remove -smp from your reply] > > Once again on the SMP list a lock is being used to make a reference count > safe. I'd like to re-raise the issue of a safe reference counting > fascility. > > what would be the semantics? I have refcount.patch :) What would be nice is to first implement atomic_fetchadd() (xadd on 486+, some hack on 386, fetchadd on ia64, similar to atomic_add on sparc64, alpha, and powerpc I believe, basically it would add a value to a memory location and return the result). You can then use taht for the reference_release (or whatever you call it). We could also use that to get rid of the really bloated debug version that uses a mutex and have a much smaller debug version that still uses atomic ops. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Mar 27 13:52:39 2002 Delivered-To: freebsd-arch@freebsd.org Received: from albatross.prod.itd.earthlink.net (albatross.mail.pas.earthlink.net [207.217.120.120]) by hub.freebsd.org (Postfix) with ESMTP id B110D37B43B for ; Wed, 27 Mar 2002 13:51:35 -0800 (PST) Received: from pool0142.cvx21-bradley.dialup.earthlink.net ([209.179.192.142] helo=mindspring.com) by albatross.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16qLKD-0004NO-00; Wed, 27 Mar 2002 13:51:22 -0800 Message-ID: <3CA23EC4.EB6E8508@mindspring.com> Date: Wed, 27 Mar 2002 13:51:00 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Kirk McKusick Cc: Ollivier Robert , arch@FreeBSD.ORG Subject: Re: [PATCH] MFC the full filesystem softupdates fix ? References: <200203272134.g2RLYTD97258@beastie.mckusick.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I think Kirk meant "-stable", not "-current"; in any case, I'd like to see it handled and brought back into -stable anyway. I look at -stable as a pruned -current. Some of the growth in -current should/will never see the light of day in a realease... -- Terry Kirk McKusick wrote: > > If it was easy as your patch, there would be no problem in > putting it in. The problem is that -current does not have > the code to maintain the fs_pendingblocks variable. At a > minimum you would have to add that code as well. It is a > good deal more extensive, though at this point well enough > tested that I am less fearful of adding it. > > Kirk McKusick > > =-=-=-=-=-= > > Date: Wed, 27 Mar 2002 11:19:47 +0100 > From: Ollivier Robert > To: mckusick@mckusick.com > Cc: arch@FreeBSD.ORG > Subject: [PATCH] MFC the full filesystem softupdates fix ? > > Following some discussions in the French fr.comp.os.bsd newsgroup about > softupdates and the full filesystem problem (which is fixed in CURRENT > thanks to you), here a proposed diff to merge the fix into STABLE. > > Can you (and -arch) review it ? It would be nice to have this fixed in > STABLE as well. > > Thanks. > > Index: ffs_alloc.c > =================================================================== > RCS file: /home/ncvs/src/sys/ufs/ffs/ffs_alloc.c,v > retrieving revision 1.64.2.2 > diff -u -2 -r1.64.2.2 ffs_alloc.c > --- ffs_alloc.c 21 Sep 2001 19:15:21 -0000 1.64.2.2 > +++ ffs_alloc.c 27 Mar 2002 10:08:29 -0000 > @@ -107,5 +107,5 @@ > register struct fs *fs; > ufs_daddr_t bno; > - int cg; > + int cg, reclaimed; > #ifdef QUOTA > int error; > @@ -124,4 +124,6 @@ > panic("ffs_alloc: missing credential"); > #endif /* DIAGNOSTIC */ > + reclaimed = 0; > +retry: > if (size == fs->fs_bsize && fs->fs_cstotal.cs_nbfree == 0) > goto nospace; > @@ -155,4 +157,9 @@ > #endif > nospace: > + if (fs->fs_pendingblocks > 0 && reclaimed == 0) { > + reclaimed = 1; > + softdep_request_cleanup(fs, ITOV(ip)); > + goto retry; > + } > ffs_fserr(fs, cred->cr_uid, "file system full"); > uprintf("\n%s: write failed, file system is full\n", fs->fs_fsmnt); > @@ -177,7 +184,7 @@ > struct buf **bpp; > { > - register struct fs *fs; > + struct fs *fs; > struct buf *bp; > - int cg, request, error; > + int cg, request, error, reclaimed; > ufs_daddr_t bprev, bno; > > @@ -198,4 +205,6 @@ > if (cred->cr_uid != 0 && > freespace(fs, fs->fs_minfree) - numfrags(fs, nsize - osize) < 0) > + reclaimed = 0; > +retry: > goto nospace; > if ((bprev = ip->i_db[lbprev]) == 0) { > @@ -208,5 +217,5 @@ > * Allocate the extra space in the buffer. > */ > - error = bread(ITOV(ip), lbprev, osize, NOCRED, &bp); > + error = bread(vp, lbprev, osize, NOCRED, &bp); > if (error) { > brelse(bp); > @@ -295,5 +304,5 @@ > if (bno > 0) { > bp->b_blkno = fsbtodb(fs, bno); > - if (!DOINGSOFTDEP(ITOV(ip))) > + if (!DOINGSOFTDEP(vp)) > ffs_blkfree(ip, bprev, (long)osize); > if (nsize < request) > @@ -319,4 +328,9 @@ > * no space available > */ > + if (fs->fs_pendingblocks > 0 && reclaimed == 0) { > + reclaimed = 1; > + softdep_request_cleanup(fs, vp); > + goto retry; > + } > ffs_fserr(fs, cred->cr_uid, "file system full"); > uprintf("\n%s: write failed, file system is full\n", fs->fs_fsmnt); > Index: ffs_extern.h > =================================================================== > RCS file: /home/ncvs/src/sys/ufs/ffs/ffs_extern.h,v > retrieving revision 1.30 > diff -u -2 -r1.30 ffs_extern.h > --- ffs_extern.h 9 Jan 2000 22:40:02 -0000 1.30 > +++ ffs_extern.h 27 Mar 2002 10:05:03 -0000 > @@ -115,4 +115,5 @@ > void softdep_load_inodeblock __P((struct inode *)); > void softdep_freefile __P((struct vnode *, ino_t, int)); > +int softdep_request_cleanup __P((struct fs *, struct vnode *)); > void softdep_setup_freeblocks __P((struct inode *, off_t)); > void softdep_setup_inomapdep __P((struct buf *, struct inode *, ino_t)); > Index: ffs_softdep.c > =================================================================== > RCS file: /home/ncvs/src/sys/ufs/ffs/ffs_softdep.c,v > retrieving revision 1.57.2.11 > diff -u -2 -r1.57.2.11 ffs_softdep.c > --- ffs_softdep.c 5 Feb 2002 18:46:53 -0000 1.57.2.11 > +++ ffs_softdep.c 27 Mar 2002 10:12:48 -0000 > @@ -508,7 +508,8 @@ > static struct proc *filesys_syncer; /* proc of filesystem syncer process */ > static int req_clear_inodedeps; /* syncer process flush some inodedeps */ > -#define FLUSH_INODES 1 > +#define FLUSH_INODES 1 > static int req_clear_remove; /* syncer process flush some freeblks */ > -#define FLUSH_REMOVE 2 > +#define FLUSH_REMOVE 2 > +#define FLUSH_REMOVE_WAIT 3 > /* > * runtime statistics > @@ -703,5 +704,5 @@ > if (wk == 0) { > FREE_LOCK(&lk); > - return (0); > + return (-1); > } > WORKLIST_REMOVE(wk); > @@ -4561,4 +4562,39 @@ > > /* > + * Called by the allocation routines when they are about to fail > + * in the hope that we can free up some disk space. > + * > + * First check to see if the work list has anything on it. If it has, > + * clean up entries until we successfully free some space. Because this > + * process holds inodes locked, we cannot handle any remove requests > + * that might block on a locked inode as that could lead to deadlock. > + * If the worklist yields no free space, encourage the syncer daemon > + * to help us. In no event will we try for longer than tickdelay seconds. > + */ > +int > +softdep_request_cleanup(fs, vp) > + struct fs *fs; > + struct vnode *vp; > +{ > + long starttime, needed; > + > + needed = fs->fs_cstotal.cs_nbfree + fs->fs_contigsumsize; > + starttime = time_second + tickdelay; > + if (UFS_UPDATE(vp, 1) != 0) > + return (0); > + while (fs->fs_pendingblocks > 0 && fs->fs_cstotal.cs_nbfree <= needed) { > + if (time_second > starttime) > + return (0); > + if (num_on_worklist > 0 && > + process_worklist_item(NULL, LK_NOWAIT) != -1) { > + stat_worklist_push += 1; > + continue; > + } > + request_cleanup(FLUSH_REMOVE_WAIT, 0); > + } > + return (1); > +} > + > +/* > * If memory utilization has gotten too high, deliberately slow things > * down and speed up the I/O processing. > @@ -4595,4 +4631,10 @@ > > /* > + * Next, we attempt to speed up the syncer process. If that > + * is successful, then we allow the process to continue. > + */ > + if (speedup_syncer() && resource != FLUSH_REMOVE_WAIT) > + return(0); > + /* > * If we are resource constrained on inode dependencies, try > * flushing some dirty inodes. Otherwise, we are constrained > @@ -4613,4 +4655,5 @@ > > case FLUSH_REMOVE: > + case FLUSH_REMOVE_WAIT: > stat_blk_limit_push += 1; > req_clear_remove += 1; > > -- > Ollivier ROBERT -=- Eurocontrol EEC/ITM -=- Ollivier.Robert@eurocontrol.fr > FreeBSD caerdonn.eurocontrol.fr 5.0-CURRENT #46: Wed Jan 3 15:52:00 CET 2001 > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Mar 27 14: 3:29 2002 Delivered-To: freebsd-arch@freebsd.org Received: from k6.locore.ca (k6.locore.ca [198.96.117.170]) by hub.freebsd.org (Postfix) with ESMTP id E6E4037B42A; Wed, 27 Mar 2002 14:03:16 -0800 (PST) Received: (from jake@localhost) by k6.locore.ca (8.11.6/8.11.6) id g2RMDqo35691; Wed, 27 Mar 2002 17:13:52 -0500 (EST) (envelope-from jake) Date: Wed, 27 Mar 2002 17:13:51 -0500 From: Jake Burkholder To: John Baldwin Cc: Julian Elischer , arch@FreeBSD.ORG Subject: Re: SMP safe reference counting Message-ID: <20020327171351.E31836@locore.ca> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from jhb@FreeBSD.ORG on Wed, Mar 27, 2002 at 04:51:57PM -0500 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Apparently, On Wed, Mar 27, 2002 at 04:51:57PM -0500, John Baldwin said words to the effect of; > > On 27-Mar-2002 Julian Elischer wrote: > > > > [please remove -smp from your reply] > > > > Once again on the SMP list a lock is being used to make a reference count > > safe. I'd like to re-raise the issue of a safe reference counting > > fascility. > > > > what would be the semantics? > > I have refcount.patch :) What would be nice is to first implement > atomic_fetchadd() (xadd on 486+, some hack on 386, fetchadd on ia64, similar to > atomic_add on sparc64, alpha, and powerpc I believe, basically it would add a > value to a memory location and return the result). You can then use taht for > the reference_release (or whatever you call it). We could also use that to get > rid of the really bloated debug version that uses a mutex and have a much > smaller debug version that still uses atomic ops. We support 386 still? > > -- > > John Baldwin <>< http://www.FreeBSD.org/~jhb/ > "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Mar 27 14:14:46 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail16.speakeasy.net (mail16.speakeasy.net [216.254.0.216]) by hub.freebsd.org (Postfix) with ESMTP id CDC8637B417 for ; Wed, 27 Mar 2002 14:14:41 -0800 (PST) Received: (qmail 4038 invoked from network); 27 Mar 2002 22:14:41 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) by mail16.speakeasy.net (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 27 Mar 2002 22:14:41 -0000 Received: from laptop.baldwin.cx (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.11.6/8.11.6) with ESMTP id g2RMFMv92936; Wed, 27 Mar 2002 17:15:23 -0500 (EST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.2 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20020327171351.E31836@locore.ca> Date: Wed, 27 Mar 2002 17:14:43 -0500 (EST) From: John Baldwin To: Jake Burkholder Subject: Re: SMP safe reference counting Cc: arch@FreeBSD.ORG, Julian Elischer Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 27-Mar-2002 Jake Burkholder wrote: > Apparently, On Wed, Mar 27, 2002 at 04:51:57PM -0500, > John Baldwin said words to the effect of; > >> >> On 27-Mar-2002 Julian Elischer wrote: >> > >> > [please remove -smp from your reply] >> > >> > Once again on the SMP list a lock is being used to make a reference count >> > safe. I'd like to re-raise the issue of a safe reference counting >> > fascility. >> > >> > what would be the semantics? >> >> I have refcount.patch :) What would be nice is to first implement >> atomic_fetchadd() (xadd on 486+, some hack on 386, fetchadd on ia64, similar >> to >> atomic_add on sparc64, alpha, and powerpc I believe, basically it would add >> a >> value to a memory location and return the result). You can then use taht >> for >> the reference_release (or whatever you call it). We could also use that to >> get >> rid of the really bloated debug version that uses a mutex and have a much >> smaller debug version that still uses atomic ops. > > We support 386 still? If you compile a custom kernel. :) -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Mar 27 14:41:49 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc54.attbi.com (rwcrmhc54.attbi.com [216.148.227.87]) by hub.freebsd.org (Postfix) with ESMTP id 04CCC37B7DB; Wed, 27 Mar 2002 14:40:47 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc54.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020327224010.ZWG1214.rwcrmhc54.attbi.com@InterJet.elischer.org>; Wed, 27 Mar 2002 22:40:10 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id OAA49064; Wed, 27 Mar 2002 14:35:00 -0800 (PST) Date: Wed, 27 Mar 2002 14:34:59 -0800 (PST) From: Julian Elischer To: Jake Burkholder Cc: John Baldwin , arch@FreeBSD.ORG Subject: Re: SMP safe reference counting In-Reply-To: <20020327171351.E31836@locore.ca> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 27 Mar 2002, Jake Burkholder wrote: > Apparently, On Wed, Mar 27, 2002 at 04:51:57PM -0500, > > We support 386 still? > We certainly support it but I think that for 386 we can ignore the concept of SMP and make versions that are not SMP safe but DO satisfy the basic functionality.. i.e. the '386 version of atomic ops need not be atomic :-) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Mar 27 16:24:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from dragon.nuxi.com (trang.nuxi.com [66.92.13.169]) by hub.freebsd.org (Postfix) with ESMTP id 0ACD537B416; Wed, 27 Mar 2002 16:24:26 -0800 (PST) Received: from dragon.nuxi.com (obrien@localhost [127.0.0.1]) by dragon.nuxi.com (8.12.2/8.12.2) with ESMTP id g2S0OPYm054024; Wed, 27 Mar 2002 16:24:25 -0800 (PST) (envelope-from obrien@dragon.nuxi.com) Received: (from obrien@localhost) by dragon.nuxi.com (8.12.2/8.12.2/Submit) id g2S0NAnk054004; Wed, 27 Mar 2002 16:23:10 -0800 (PST) Date: Wed, 27 Mar 2002 16:23:10 -0800 From: "David O'Brien" To: John Baldwin Cc: arch@FreeBSD.org Subject: Re: SMP safe reference counting Message-ID: <20020327162310.A53917@dragon.nuxi.com> Reply-To: arch@FreeBSD.org References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from jhb@FreeBSD.org on Wed, Mar 27, 2002 at 04:51:57PM -0500 X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD group X-Pgp-Rsa-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Rsa-Keyid: 1024/34F9F9D5 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Mar 27, 2002 at 04:51:57PM -0500, John Baldwin wrote: > I have refcount.patch :) Fill in the i386 and commit it. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Mar 27 17: 4:10 2002 Delivered-To: freebsd-arch@freebsd.org Received: from 12-234-96-171.client.attbi.com (12-234-96-171.client.attbi.com [12.234.96.171]) by hub.freebsd.org (Postfix) with ESMTP id 23CE337B417 for ; Wed, 27 Mar 2002 17:04:07 -0800 (PST) Received: by 12-234-96-171.client.attbi.com (Postfix, from userid 1000) id 6F5565EA1; Wed, 27 Mar 2002 17:03:08 -0800 (PST) Date: Wed, 27 Mar 2002 17:03:08 -0800 From: Jon Mini To: Julian Elischer Cc: arch@FreeBSD.ORG Subject: Re: SMP safe reference counting Message-ID: <20020327170308.I20741@stylus.haikugeek.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from julian@elischer.org on Wed, Mar 27, 2002 at 12:10:36PM -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Julian Elischer [julian@elischer.org] wrote : > > Once again on the SMP list a lock is being used to make a reference count > safe. I'd like to re-raise the issue of a safe reference counting > fascility. > > [ ... ] > > /* Note I use the non-existing "atomic_inc()". I think > we should have this as its SO COMMONLY used. */ > FWIW, I agree with Julian on both counts: a) a generic reference counting utility would be useful, however b) I frankly can't believe that FreeBSD doesn't have an atomic increment MI function. -- Jonathan Mini mini@haikugeek.com Yersterday, I was ashamed of myself. Today, I am just hungry. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Mar 27 21: 3:45 2002 Delivered-To: freebsd-arch@freebsd.org Received: from angelica.unixdaemons.com (angelica.unixdaemons.com [209.148.64.135]) by hub.freebsd.org (Postfix) with ESMTP id 97F4637B405 for ; Wed, 27 Mar 2002 21:03:41 -0800 (PST) Received: from angelica.unixdaemons.com (bmilekic@localhost.unixdaemons.com [127.0.0.1]) by angelica.unixdaemons.com (8.12.2/8.12.1) with ESMTP id g2S52fJX095453; Thu, 28 Mar 2002 00:02:41 -0500 (EST) X-Authentication-Warning: angelica.unixdaemons.com: Host bmilekic@localhost.unixdaemons.com [127.0.0.1] claimed to be angelica.unixdaemons.com Received: (from bmilekic@localhost) by angelica.unixdaemons.com (8.12.2/8.12.1/Submit) id g2S52eYf095442; Thu, 28 Mar 2002 00:02:40 -0500 (EST) (envelope-from bmilekic) Date: Thu, 28 Mar 2002 00:02:40 -0500 From: Bosko Milekic To: Julian Elischer Cc: arch@FreeBSD.ORG Subject: Re: SMP safe reference counting Message-ID: <20020328000240.A94897@unixdaemons.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: ; from julian@elischer.org on Wed, Mar 27, 2002 at 12:10:36PM -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I don't think we really need a ref. count API, per-se. I can think of several places that may need to do ref. counting but wouldn't want to do it with a bus-locked instruction because their reference counter(s) are already protected by an existing mutex. -Bosko On Wed, Mar 27, 2002 at 12:10:36PM -0800, Julian Elischer wrote: > > [please remove -smp from your reply] > > Once again on the SMP list a lock is being used to make a reference count > safe. I'd like to re-raise the issue of a safe reference counting > fascility. > > what would be the semantics? > > typedef volatile u_int ref_cnt; > typedef void ref_free(void *); > > reference_add(ref_cnt *); > reference_drop(ref_cnt *, ref_free *, void *); > > > > > > __inline void > reference_add(ref_cnt *cnt) { > atomic_inc(cnt); > } > > /* Note I use the non-existing "atomic_inc()". I think > we should have this as its SO COMMONLY used. */ > > __inline void > reference_drop(ref_cnt *cnt, ref_free *fn. void * arg) { > int newcount; > int oldcount; > > do { > newcount = (oldcount = *cnt) - 1; > } while (atomic_cmp_and_set(cnt, oldcount, newcount) == ACS_FAILED); > } > /* I can't remember off the top of my head the cmp-and-set command */ > > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > -- Bosko Milekic bmilekic@unixdaemons.com bmilekic@FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 1:41:42 2002 Delivered-To: freebsd-arch@freebsd.org Received: from tele-post-20.mail.demon.net (tele-post-20.mail.demon.net [194.217.242.20]) by hub.freebsd.org (Postfix) with ESMTP id 6CA4B37B400; Thu, 28 Mar 2002 01:41:35 -0800 (PST) Received: from mailgate.nlsystems.com ([62.49.251.130] helo=herring.nlsystems.com) by tele-post-20.mail.demon.net with esmtp (Exim 3.35 #1) id 16qWPU-000FBm-0K; Thu, 28 Mar 2002 09:41:32 +0000 Received: from salmon.nlsystems.com (salmon [10.0.0.3]) by herring.nlsystems.com (8.11.2/8.11.2) with ESMTP id g2S9eE961737; Thu, 28 Mar 2002 09:40:14 GMT (envelope-from dfr@nlsystems.com) Date: Thu, 28 Mar 2002 09:36:38 +0000 (GMT) From: Doug Rabson To: John Baldwin Cc: Julian Elischer , Subject: RE: SMP safe reference counting In-Reply-To: Message-ID: <20020328093345.Y17999-100000@salmon.nlsystems.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 27 Mar 2002, John Baldwin wrote: > > On 27-Mar-2002 Julian Elischer wrote: > > > > [please remove -smp from your reply] > > > > Once again on the SMP list a lock is being used to make a reference count > > safe. I'd like to re-raise the issue of a safe reference counting > > fascility. > > > > what would be the semantics? > > I have refcount.patch :) What would be nice is to first implement > atomic_fetchadd() (xadd on 486+, some hack on 386, fetchadd on ia64, similar to > atomic_add on sparc64, alpha, and powerpc I believe, basically it would add a > value to a memory location and return the result). You can then use taht for > the reference_release (or whatever you call it). We could also use that to get > rid of the really bloated debug version that uses a mutex and have a much > smaller debug version that still uses atomic ops. The ia64 has a fetchadd instruction which can add a constant to memory, returning the old value. It is limited to adding only the constants -16. -8, -4, -1, 1, 4, 8, 16. -- Doug Rabson Mail: dfr@nlsystems.com Phone: +44 20 8348 6160 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 3:10:18 2002 Delivered-To: freebsd-arch@freebsd.org Received: from matrix.eurocontrol.fr (matrix.eurocontrol.fr [147.196.254.254]) by hub.freebsd.org (Postfix) with ESMTP id 7758E37B419 for ; Thu, 28 Mar 2002 03:10:12 -0800 (PST) Received: from caerdonn.eurocontrol.fr (caerdonn.eurocontrol.fr [147.196.51.214]) (using TLSv1 with cipher EDH-RSA-DES-CBC3-SHA (168/168 bits)) (Client CN "caerdonn.eurocontrol.fr", Issuer CN "CA ITM" (not verified)) by matrix.eurocontrol.fr (Postfix/TLS) with ESMTP id 4510921C7; Thu, 28 Mar 2002 12:10:08 +0100 (CET) Received: by caerdonn.eurocontrol.fr (Postfix/TLS, from userid 1193) id 6A4ED20; Thu, 28 Mar 2002 12:10:07 +0100 (CET) Date: Thu, 28 Mar 2002 12:10:07 +0100 From: Ollivier Robert To: Kirk McKusick Cc: arch@FreeBSD.ORG Subject: Re: [PATCH] MFC the full filesystem softupdates fix ? Message-ID: <20020328111007.GC48070@caerdonn.eurocontrol.fr> References: <20020327101947.GA2322@caerdonn.eurocontrol.fr> <200203272134.g2RLYTD97258@beastie.mckusick.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200203272134.g2RLYTD97258@beastie.mckusick.com> User-Agent: Mutt/1.3.28i X-Operating-System: FreeBSD 5.0-CURRENT Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG According to Kirk McKusick: > If it was easy as your patch, there would be no problem in > putting it in. The problem is that -current does not have > the code to maintain the fs_pendingblocks variable. At a > minimum you would have to add that code as well. It is a > good deal more extensive, though at this point well enough I've now seen that... > tested that I am less fearful of adding it. If we take into consideration the fact that 5.0-R won't be there till October/November, do you think it would be possible to put that fix and maybe snapshots into 4-STABLE? -- Ollivier ROBERT -=- Eurocontrol EEC/ITM -=- Ollivier.Robert@eurocontrol.fr FreeBSD caerdonn.eurocontrol.fr 5.0-CURRENT #46: Wed Jan 3 15:52:00 CET 2001 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 5:30:50 2002 Delivered-To: freebsd-arch@freebsd.org Received: from srv1.cosmo-project.de (srv1.cosmo-project.de [213.83.6.106]) by hub.freebsd.org (Postfix) with ESMTP id DA57637B41A; Thu, 28 Mar 2002 05:30:44 -0800 (PST) Received: (from uucp@localhost) by srv1.cosmo-project.de (8.11.6/8.11.6) with UUCP id g2SDUQW47350; Thu, 28 Mar 2002 14:30:26 +0100 (CET) (envelope-from ticso@cicely8.cicely.de) Received: from cicely8.cicely.de (cicely8.cicely.de [10.1.1.10]) by cicely5.cicely.de (8.12.1/8.12.1) with ESMTP id g2SDKk6e020191; Thu, 28 Mar 2002 14:20:46 +0100 (CET)?g (envelope-from ticso@cicely8.cicely.de) Received: from cicely8.cicely.de (localhost [127.0.0.1]) by cicely8.cicely.de (8.12.2/8.12.2) with ESMTP id g2SDKjnU024809; Thu, 28 Mar 2002 14:20:45 +0100 (CET) (envelope-from ticso@cicely8.cicely.de) Received: (from ticso@localhost) by cicely8.cicely.de (8.12.2/8.12.2/Submit) id g2SDKgBA024808; Thu, 28 Mar 2002 14:20:42 +0100 (CET) Date: Thu, 28 Mar 2002 14:20:42 +0100 From: Bernd Walter To: Julian Elischer Cc: Jake Burkholder , John Baldwin , arch@FreeBSD.ORG Subject: Re: SMP safe reference counting Message-ID: <20020328132041.GA24734@cicely8.cicely.de> References: <20020327171351.E31836@locore.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.3.26i X-Operating-System: FreeBSD cicely8.cicely.de 5.0-CURRENT i386 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Mar 27, 2002 at 02:34:59PM -0800, Julian Elischer wrote: > > > On Wed, 27 Mar 2002, Jake Burkholder wrote: > > > Apparently, On Wed, Mar 27, 2002 at 04:51:57PM -0500, > > > > We support 386 still? > > > We certainly support it but I think that for 386 we can ignore the concept > of SMP and make versions that are not SMP safe but DO > satisfy the basic functionality.. > > i.e. the '386 version of atomic ops need not be atomic :-) They need to be atomic in the sense of preemtion. -- B.Walter COSMO-Project http://www.cosmo-project.de ticso@cicely.de Usergroup info@cosmo-project.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 6:28:38 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail15.speakeasy.net (mail15.speakeasy.net [216.254.0.215]) by hub.freebsd.org (Postfix) with ESMTP id 5C43737B41D for ; Thu, 28 Mar 2002 06:28:35 -0800 (PST) Received: (qmail 28213 invoked from network); 28 Mar 2002 14:28:34 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) by mail15.speakeasy.net (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 28 Mar 2002 14:28:34 -0000 Received: from laptop.baldwin.cx (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.11.6/8.11.6) with ESMTP id g2SETHv95725 for ; Thu, 28 Mar 2002 09:29:17 -0500 (EST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.2 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20020327162310.A53917@dragon.nuxi.com> Date: Thu, 28 Mar 2002 09:28:37 -0500 (EST) From: John Baldwin To: arch@FreeBSD.org Subject: Re: SMP safe reference counting Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 28-Mar-2002 David O'Brien wrote: > On Wed, Mar 27, 2002 at 04:51:57PM -0500, John Baldwin wrote: >> I have refcount.patch :) > > Fill in the i386 and commit it. It was objected to recently. Also, to be done cleanly, atomic_fetchadd() should be done first. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 6:28:56 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail11.speakeasy.net (mail11.speakeasy.net [216.254.0.211]) by hub.freebsd.org (Postfix) with ESMTP id 5099C37B417 for ; Thu, 28 Mar 2002 06:28:42 -0800 (PST) Received: (qmail 11366 invoked from network); 28 Mar 2002 14:28:40 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) by mail11.speakeasy.net (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 28 Mar 2002 14:28:40 -0000 Received: from laptop.baldwin.cx (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.11.6/8.11.6) with ESMTP id g2SETNv95729; Thu, 28 Mar 2002 09:29:23 -0500 (EST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.2 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20020328093345.Y17999-100000@salmon.nlsystems.com> Date: Thu, 28 Mar 2002 09:28:43 -0500 (EST) From: John Baldwin To: Doug Rabson Subject: RE: SMP safe reference counting Cc: arch@freebsd.org, Julian Elischer Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 28-Mar-2002 Doug Rabson wrote: > On Wed, 27 Mar 2002, John Baldwin wrote: > >> >> On 27-Mar-2002 Julian Elischer wrote: >> > >> > [please remove -smp from your reply] >> > >> > Once again on the SMP list a lock is being used to make a reference count >> > safe. I'd like to re-raise the issue of a safe reference counting >> > fascility. >> > >> > what would be the semantics? >> >> I have refcount.patch :) What would be nice is to first implement >> atomic_fetchadd() (xadd on 486+, some hack on 386, fetchadd on ia64, similar >> to >> atomic_add on sparc64, alpha, and powerpc I believe, basically it would add >> a >> value to a memory location and return the result). You can then use taht >> for >> the reference_release (or whatever you call it). We could also use that to >> get >> rid of the really bloated debug version that uses a mutex and have a much >> smaller debug version that still uses atomic ops. > > The ia64 has a fetchadd instruction which can add a constant to memory, > returning the old value. It is limited to adding only the constants -16. > -8, -4, -1, 1, 4, 8, 16. Ugh, I didn't know it was limited in what it could add. Hmmm, it would be nice to use that rather than a cmpxchg loop if possible. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 6:45:11 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail13.speakeasy.net (mail13.speakeasy.net [216.254.0.213]) by hub.freebsd.org (Postfix) with ESMTP id 32E6237B419 for ; Thu, 28 Mar 2002 06:45:03 -0800 (PST) Received: (qmail 614 invoked from network); 28 Mar 2002 14:45:01 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) by mail13.speakeasy.net (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 28 Mar 2002 14:45:01 -0000 Received: from laptop.baldwin.cx (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.11.6/8.11.6) with ESMTP id g2SEjhv95796; Thu, 28 Mar 2002 09:45:43 -0500 (EST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.2 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20020328000240.A94897@unixdaemons.com> Date: Thu, 28 Mar 2002 09:45:02 -0500 (EST) From: John Baldwin To: Bosko Milekic Subject: Re: SMP safe reference counting Cc: arch@FreeBSD.ORG, Julian Elischer Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 28-Mar-2002 Bosko Milekic wrote: > > I don't think we really need a ref. count API, per-se. I can think of > several places that may need to do ref. counting but wouldn't want to > do it with a bus-locked instruction because their reference counter(s) > are already protected by an existing mutex. > > -Bosko Those places wouldn't use the API then I think. However, this would be good for things like ucreds, pargs, uidinfos and others. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 10:35:44 2002 Delivered-To: freebsd-arch@freebsd.org Received: from aldan.algebra.com (aldan.algebra.com [216.254.65.224]) by hub.freebsd.org (Postfix) with ESMTP id 492C237B417 for ; Thu, 28 Mar 2002 10:35:41 -0800 (PST) Received: from aldan.algebra.com (localhost [127.0.0.1]) by aldan.algebra.com (8.12.2/8.12.2) with ESMTP id g2SIZ1kf000681; Thu, 28 Mar 2002 13:35:05 -0500 (EST) (envelope-from mi@aldan.algebra.com) Message-Id: <200203281835.g2SIZ1kf000681@aldan.algebra.com> Date: Thu, 28 Mar 2002 13:35:01 -0500 (EST) From: Mikhail Teterin Subject: Re: review request for bin/11294 To: des@ofug.org Cc: arch@FreeBSD.org In-Reply-To: MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 24 Mar, Dag-Erling Smorgrav wrote: > Mikhail Teterin writes: >> But this implementation has drawbacks, that mine does not, IMHO: >> >> . the configuration is system wide -- which is not neccessarily >> desirable; >> [...] > > This and all your other objections can be fixed using a global > variable that overrides the symlink (or an API function that sets the > target host). My primary objection is that the problem you are solving -- logging to some other host by default -- is different from mine -- logging to some other host at the caller's discretion. Yes, the caller can call the new API function you suggest (or set a global variable itself, which is uglier, IMHO). But the only drawback of my idea, that was pointed out so far is that it _adds a new API call_. If such an addition _can_ be tolerated -- and it looks from your response like it can -- I think, solving my problem your way is inferior to my proposed solution. Which is not to say, the solution you provided is inferior over all -- it is just for a different problem. -mi To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 11: 0:43 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc53.attbi.com (rwcrmhc53.attbi.com [204.127.198.39]) by hub.freebsd.org (Postfix) with ESMTP id 9425937B41B; Thu, 28 Mar 2002 11:00:20 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc53.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020328190020.WHGH2951.rwcrmhc53.attbi.com@InterJet.elischer.org>; Thu, 28 Mar 2002 19:00:20 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id KAA53417; Thu, 28 Mar 2002 10:50:03 -0800 (PST) Date: Thu, 28 Mar 2002 10:50:02 -0800 (PST) From: Julian Elischer To: Bernd Walter Cc: Jake Burkholder , John Baldwin , arch@FreeBSD.ORG Subject: Re: SMP safe reference counting In-Reply-To: <20020328132041.GA24734@cicely8.cicely.de> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, 28 Mar 2002, Bernd Walter wrote: > On Wed, Mar 27, 2002 at 02:34:59PM -0800, Julian Elischer wrote: > > > > > > On Wed, 27 Mar 2002, Jake Burkholder wrote: > > > > > Apparently, On Wed, Mar 27, 2002 at 04:51:57PM -0500, > > > > > > We support 386 still? > > > > > We certainly support it but I think that for 386 we can ignore the concept > > of SMP and make versions that are not SMP safe but DO > > satisfy the basic functionality.. > > > > i.e. the '386 version of atomic ops need not be atomic :-) > > They need to be atomic in the sense of preemtion. All instructions are atomic from preemtion but they don't need t be atomic on a bus-cycle basis. > > -- > B.Walter COSMO-Project http://www.cosmo-project.de > ticso@cicely.de Usergroup info@cosmo-project.de > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 11:29:28 2002 Delivered-To: freebsd-arch@freebsd.org Received: from green.bikeshed.org (freefall.FreeBSD.org [216.136.204.21]) by hub.freebsd.org (Postfix) with ESMTP id B62C337B41C for ; Thu, 28 Mar 2002 11:29:11 -0800 (PST) Received: from localhost (green@localhost) by green.bikeshed.org (8.11.6/8.11.6) with ESMTP id g2SJTBW04780 for ; Thu, 28 Mar 2002 14:29:11 -0500 (EST) (envelope-from green@green.bikeshed.org) Message-Id: <200203281929.g2SJTBW04780@green.bikeshed.org> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: arch@FreeBSD.org Subject: vnode::v_op bugfix / PERFORCE change 8574 for review (fwd) From: "Brian F. Feldman" Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 28 Mar 2002 14:29:11 -0500 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This seems to fix -CURRENT after the last round of bugs that UMA evidenced. Does anyone have objections to modifying this behavior of VFS? I have a copy of the patch to get between the two up at: http://green.bikeshed.org/~green/v_op-indirection.patch ------- Forwarded Message Date: Thu, 28 Mar 2002 09:09:42 -0800 (PST) Message-Id: <200203281709.g2SH9g038754@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: perforce set sender to green@freebsd.org using -f From: Brian Feldman Subject: PERFORCE change 8574 for review To: Perforce Change Reviews http://people.freebsd.org/~peter/p4db/chv.cgi?CH=8574 Change 8574 by green@green_laptop_2 on 2002/03/28 09:09:26 Turn struct vnode {}'s v_op field from a vop_t ** to a vop_t ***. Previously, it pointed to the actual vfs_init.c-generated operation vector being used. However, this vector could be redone at runtime via introduction of new vnode operations and removal of old ones; this would result in the old vnode operation vector being freed from underneath. This didn't show up before since the old kernel malloc(9) coincidentally kept the old vop_t ** in the vnodes valid. Jeff Roberson's UMA commit made this bug apparent due to differently-sized chunks of memory actually being likely to be allocated in different spots than previously allocated at even if the size was grown just by a few bytes. The new vop_t *** actually points to the operation vector pointer the kernel uses and modifies on-the-fly so that old vnodes, and new ones created with getnewvnode(), both call the correct operations. Getnewvnode()'s vop_t **vops argument changes to a vop_t *** to reflect this. -- Brian Fundakowski Feldman \'[ FreeBSD ]''''''''''\ <> green@FreeBSD.org <> bfeldman@tislabs.com \ The Power to Serve! \ Opinions expressed are my own. \,,,,,,,,,,,,,,,,,,,,,,\ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 12:14: 5 2002 Delivered-To: freebsd-arch@freebsd.org Received: from falcon.prod.itd.earthlink.net (falcon.mail.pas.earthlink.net [207.217.120.74]) by hub.freebsd.org (Postfix) with ESMTP id 3EBB437B400; Thu, 28 Mar 2002 12:13:53 -0800 (PST) Received: from pool0038.cvx40-bradley.dialup.earthlink.net ([216.244.42.38] helo=mindspring.com) by falcon.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16qgH6-0001iA-00; Thu, 28 Mar 2002 12:13:33 -0800 Message-ID: <3CA37956.2F92476B@mindspring.com> Date: Thu, 28 Mar 2002 12:13:10 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: "Brian F. Feldman" Cc: arch@FreeBSD.org Subject: Re: vnode::v_op bugfix / PERFORCE change 8574 for review (fwd) References: <200203281929.g2SJTBW04780@green.bikeshed.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "Brian F. Feldman" wrote: > > This seems to fix -CURRENT after the last round of bugs that UMA evidenced. > Does anyone have objections to modifying this behavior of VFS? I have a > copy of the patch to get between the two up at: > http://green.bikeshed.org/~green/v_op-indirection.patch The vfs_default.c code comes from the universe where Spock has a beard. I know I suggested the change made by this patch, but it's only actually a workaround to a problem that wouldn't exist, if it weren't for vfs_default.c. And it's a bad workaround, at that. 8-(. It was really meant as a single-instance workaround for the NTFS maintainer to be able to do the load/unload during developement work on the module, pending real fixes to the VFS code to bring it back in line with the design documentation from UCLA/FICUS. The only place it bites you is adding new VOPs in a loadable module, which should not bite you in the first place, and wouldn't, if the default for everything was "unimplemented", like it was designed to be, instead of having defaults that do something, like there are now. As it is, it's impossible to proxy descriptors across an interface that doesn't know about all the possible descriptors. This is just plain wrong. Unless you can add default implementations for default_vops when you add a new VOP, this whole mess doesn't make sense in any universe. Even then, it makes little sense: you have to look at it from the right perverse angle, like the joke about the tailor: "Oh, that poor man!" "Yes, but what a wonderful suit!". -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 13:13:38 2002 Delivered-To: freebsd-arch@freebsd.org Received: from beastie.mckusick.com (beastie.mckusick.com [209.31.233.184]) by hub.freebsd.org (Postfix) with ESMTP id C855637B400; Thu, 28 Mar 2002 13:13:32 -0800 (PST) Received: from beastie.mckusick.com (localhost [127.0.0.1]) by beastie.mckusick.com (8.11.6/8.11.6) with ESMTP id g2SLDWD99297; Thu, 28 Mar 2002 13:13:32 -0800 (PST) (envelope-from mckusick@beastie.mckusick.com) Message-Id: <200203282113.g2SLDWD99297@beastie.mckusick.com> To: "Brian F. Feldman" Subject: Re: vnode::v_op bugfix / PERFORCE change 8574 for review (fwd) Cc: arch@FreeBSD.ORG In-Reply-To: Your message of "Thu, 28 Mar 2002 14:29:11 EST." <200203281929.g2SJTBW04780@green.bikeshed.org> Date: Thu, 28 Mar 2002 13:13:32 -0800 From: Kirk McKusick Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I believe your proposed fix is both necessary and correct. Kirk McKusick To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 13:30:36 2002 Delivered-To: freebsd-arch@freebsd.org Received: from beastie.mckusick.com (beastie.mckusick.com [209.31.233.184]) by hub.freebsd.org (Postfix) with ESMTP id C197C37B41A for ; Thu, 28 Mar 2002 13:30:12 -0800 (PST) Received: from beastie.mckusick.com (localhost [127.0.0.1]) by beastie.mckusick.com (8.11.6/8.11.6) with ESMTP id g2SLU8D99368; Thu, 28 Mar 2002 13:30:08 -0800 (PST) (envelope-from mckusick@beastie.mckusick.com) Message-Id: <200203282130.g2SLU8D99368@beastie.mckusick.com> To: Ollivier Robert Subject: Re: [PATCH] MFC the full filesystem softupdates fix ? Cc: arch@FreeBSD.ORG In-Reply-To: Your message of "Thu, 28 Mar 2002 12:10:07 +0100." <20020328111007.GC48070@caerdonn.eurocontrol.fr> Date: Thu, 28 Mar 2002 13:30:08 -0800 From: Kirk McKusick Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Date: Thu, 28 Mar 2002 12:10:07 +0100 From: Ollivier Robert To: Kirk McKusick Cc: arch@FreeBSD.ORG Subject: Re: [PATCH] MFC the full filesystem softupdates fix ? In-Reply-To: <200203272134.g2RLYTD97258@beastie.mckusick.com> According to Kirk McKusick: > If it was easy as your patch, there would be no problem in > putting it in. The problem is that -current does not have > the code to maintain the fs_pendingblocks variable. At a > minimum you would have to add that code as well. It is a > good deal more extensive, though at this point well enough I've now seen that... > tested that I am less fearful of adding it. If we take into consideration the fact that 5.0-R won't be there till October/November, do you think it would be possible to put that fix and maybe snapshots into 4-STABLE? The is no chance that snapshots would move into -current. They touch thousands of lines of code. If you want to pull over the pending block changes, I enclose the relevent delta below. Kirk McKusick =-=-=-=-=-=-= From: Kirk McKusick Date: Tue, 8 May 2001 00:42:20 -0700 (PDT) To: cvs-committers@FreeBSD.org, cvs-all@FreeBSD.org Subject: cvs commit: src/sys/ufs/ffs fs.h softdep.h ffs_softdep.c ffs_softdep_stub.c ffs_inode.c ffs_vfsops.c src/sys/ufs/ufs ufs_extern.h inode.h ufs_inode.c X-FreeBSD-CVS-Branch: HEAD Sender: owner-cvs-committers@FreeBSD.org mckusick 2001/05/08 00:42:20 PDT Modified files: sys/ufs/ffs fs.h softdep.h ffs_softdep.c ffs_softdep_stub.c ffs_inode.c ffs_vfsops.c sys/ufs/ufs ufs_extern.h inode.h ufs_inode.c Log: When running with soft updates, track the number of blocks and files that are committed to being freed and reflect these blocks in the counts returned by statfs (and thus also by the `df' command). This change allows programs such as those that do news expiration to know when to stop if they are trying to create a certain percentage of free space. Note that this change does not solve the much harder problem of making this to-be-freed space available to applications that want it (thus on a nearly full filesystem, you may still encounter out-of-space conditions even though the free space will show up eventually). Hopefully this harder problem will be the subject of a future enhancement. Revision Changes Path 1.22 +4 -2 src/sys/ufs/ffs/fs.h 1.12 +6 -3 src/sys/ufs/ffs/softdep.h 1.94 +59 -2 src/sys/ufs/ffs/ffs_softdep.c 1.16 +10 -1 src/sys/ufs/ffs/ffs_softdep_stub.c 1.71 +3 -1 src/sys/ufs/ffs/ffs_inode.c 1.153 +37 -4 src/sys/ufs/ffs/ffs_vfsops.c 1.34 +2 -1 src/sys/ufs/ufs/ufs_extern.h 1.34 +4 -5 src/sys/ufs/ufs/inode.h 1.36 +3 -1 src/sys/ufs/ufs/ufs_inode.c To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 13:47:35 2002 Delivered-To: freebsd-arch@freebsd.org Received: from h132-197-179-27.gte.com (h132-197-179-27.gte.com [132.197.179.27]) by hub.freebsd.org (Postfix) with ESMTP id 6EC4D37B428; Thu, 28 Mar 2002 13:46:46 -0800 (PST) Received: from kanpc.gte.com (localhost [IPv6:::1]) by h132-197-179-27.gte.com (8.12.2/8.12.2) with ESMTP id g2SLkjka029503; Thu, 28 Mar 2002 16:46:45 -0500 (EST) (envelope-from ak03@kanpc.gte.com) Received: (from ak03@localhost) by kanpc.gte.com (8.12.2/8.12.2/Submit) id g2SLkjRU029502; Thu, 28 Mar 2002 16:46:45 -0500 (EST) Date: Thu, 28 Mar 2002 16:46:45 -0500 From: Alexander Kabaev To: "Brian F. Feldman" Cc: arch@FreeBSD.ORG Subject: Re: vnode::v_op bugfix / PERFORCE change 8574 for review (fwd) Message-Id: <20020328164645.790f6247.ak03@gte.com> In-Reply-To: <200203281929.g2SJTBW04780@green.bikeshed.org> References: <200203281929.g2SJTBW04780@green.bikeshed.org> Organization: Verizon Data Services X-Mailer: Sylpheed version 0.7.4claws44 (GTK+ 1.2.10; i386-portbld-freebsd5.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This fix will not work when/if the VFS layer will become preemptable. It also introduces potential performance penalty due to the additional pointer dereference and associated cache misses on each vnode operation. I have alternative patch which simple overallocates space for vop_t vectors a bit in order to avoid the need to realocate them and panics when this space becomes exausted. The amount of space kernel is preallocating is configurable through boot-time tunable. You can find patch at http://kan.dnsalias.net:8080/kan/tunable_vops.diff. This still needs some rewriting but it should be good enough to give you an idea on what I am proposing. There is also a patch which implement additional indirection the same way Brian's patch is doing, but tries to hide the change from the rest of the kernel code, so that operations like vp->v_op = ufs_specop_p need not be modified in existing code. The URL for the second patch is http://kan.dnsalias.net:8080/kan/indirect_vop_t.diff To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 13:48:12 2002 Delivered-To: freebsd-arch@freebsd.org Received: from angelica.unixdaemons.com (angelica.unixdaemons.com [209.148.64.135]) by hub.freebsd.org (Postfix) with ESMTP id 304A537B437; Thu, 28 Mar 2002 13:47:26 -0800 (PST) Received: from angelica.unixdaemons.com (bmilekic@localhost.unixdaemons.com [127.0.0.1]) by angelica.unixdaemons.com (8.12.2/8.12.1) with ESMTP id g2SLkJJX096244; Thu, 28 Mar 2002 16:46:19 -0500 (EST) X-Authentication-Warning: angelica.unixdaemons.com: Host bmilekic@localhost.unixdaemons.com [127.0.0.1] claimed to be angelica.unixdaemons.com Received: (from bmilekic@localhost) by angelica.unixdaemons.com (8.12.2/8.12.1/Submit) id g2SLkIG1096241; Thu, 28 Mar 2002 16:46:18 -0500 (EST) (envelope-from bmilekic) Date: Thu, 28 Mar 2002 16:46:18 -0500 From: Bosko Milekic To: John Baldwin Cc: arch@FreeBSD.ORG, Julian Elischer Subject: Re: SMP safe reference counting Message-ID: <20020328164618.A95605@unixdaemons.com> References: <20020328000240.A94897@unixdaemons.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: ; from jhb@FreeBSD.ORG on Thu, Mar 28, 2002 at 09:45:02AM -0500 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, Mar 28, 2002 at 09:45:02AM -0500, John Baldwin wrote: > > On 28-Mar-2002 Bosko Milekic wrote: > > > > I don't think we really need a ref. count API, per-se. I can think of > > several places that may need to do ref. counting but wouldn't want to > > do it with a bus-locked instruction because their reference counter(s) > > are already protected by an existing mutex. > > > > -Bosko > > Those places wouldn't use the API then I think. However, this would be good > for things like ucreds, pargs, uidinfos and others. Yeah, absolutely. I was just pointing out that there's little point in having a reference counting API when a lot of the reference counting implementations won't be using it. > -- > > John Baldwin <>< http://www.FreeBSD.org/~jhb/ > "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ -- Bosko Milekic bmilekic@unixdaemons.com bmilekic@FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 14: 6:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from avocet.prod.itd.earthlink.net (avocet.mail.pas.earthlink.net [207.217.120.50]) by hub.freebsd.org (Postfix) with ESMTP id DAC2437B400; Thu, 28 Mar 2002 14:06:04 -0800 (PST) Received: from pool0169.cvx22-bradley.dialup.earthlink.net ([209.179.198.169] helo=mindspring.com) by avocet.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16qi1k-000546-00; Thu, 28 Mar 2002 14:05:48 -0800 Message-ID: <3CA393A2.FB67CC2B@mindspring.com> Date: Thu, 28 Mar 2002 14:05:22 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Kirk McKusick Cc: "Brian F. Feldman" , arch@FreeBSD.ORG Subject: Re: vnode::v_op bugfix / PERFORCE change 8574 for review (fwd) References: <200203282113.g2SLDWD99297@beastie.mckusick.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Kirk McKusick wrote: > I believe your proposed fix is both necessary and correct. It should be noted that this code is the same as Alexander Kabaev's original proposed patch. Alexander's new patch, which I also suggested, is to overallocate the vector area, and then update it in place. I think that Alexander's new patch is a better workaround for the problem, both because (as he points out) the pointer fix fails when the VFS becomes preemptible, and because of the extra dereference overhead. I'm not terrifically happy about either one of them, but at least there is a much lower impact in refactoring the VOP lists (the new patch) rather than reallocating them, and since both operations are exceedingly rare (add/remove a new VOP), if one has to go in, I would prefer that it be Alexander's new one. Ideally, we would back out the default_vfs.c changes, add a "next" pointer field into the VOP list, link the list forward in order of adjacency at startup, and add new VOPs by linking them onto the end the the chain. Then none of this hackery would be necessary. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 14:20:26 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc54.attbi.com (rwcrmhc54.attbi.com [216.148.227.87]) by hub.freebsd.org (Postfix) with ESMTP id E1AAA37B41E; Thu, 28 Mar 2002 14:20:11 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc54.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020328222010.PGOA1214.rwcrmhc54.attbi.com@InterJet.elischer.org>; Thu, 28 Mar 2002 22:20:10 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id OAA54346; Thu, 28 Mar 2002 14:14:31 -0800 (PST) Date: Thu, 28 Mar 2002 14:14:31 -0800 (PST) From: Julian Elischer To: Bosko Milekic Cc: John Baldwin , arch@FreeBSD.ORG Subject: Re: SMP safe reference counting In-Reply-To: <20020328164618.A95605@unixdaemons.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, 28 Mar 2002, Bosko Milekic wrote: > > On Thu, Mar 28, 2002 at 09:45:02AM -0500, John Baldwin wrote: > > > > On 28-Mar-2002 Bosko Milekic wrote: > > > > > > I don't think we really need a ref. count API, per-se. I can think of > > > several places that may need to do ref. counting but wouldn't want to > > > do it with a bus-locked instruction because their reference counter(s) > > > are already protected by an existing mutex. > > > > > > -Bosko > > > > Those places wouldn't use the API then I think. However, this would be good > > for things like ucreds, pargs, uidinfos and others. > > Yeah, absolutely. I was just pointing out that there's little point > in having a reference counting API when a lot of the reference counting > implementations won't be using it. but a lot would.. and that's GOOD. John, can you put your sugfgested API up again? I remember it would work but was (in my mind) 'different'. I didn't think it was intuative but at least get it up for discussion.. (I posted my suggested API a few mails ago.. I'm sure it too has weaknesses though) Julian > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 14:37:34 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail14.speakeasy.net (mail14.speakeasy.net [216.254.0.214]) by hub.freebsd.org (Postfix) with ESMTP id 14DE937B417 for ; Thu, 28 Mar 2002 14:37:11 -0800 (PST) Received: (qmail 13201 invoked from network); 28 Mar 2002 22:37:09 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) by mail14.speakeasy.net (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 28 Mar 2002 22:37:09 -0000 Received: from laptop.baldwin.cx (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.11.6/8.11.6) with ESMTP id g2SMbtv97683; Thu, 28 Mar 2002 17:37:55 -0500 (EST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.2 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Thu, 28 Mar 2002 17:37:11 -0500 (EST) From: John Baldwin To: Julian Elischer Subject: Re: SMP safe reference counting Cc: arch@FreeBSD.ORG, Bosko Milekic Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 28-Mar-2002 Julian Elischer wrote: > > > On Thu, 28 Mar 2002, Bosko Milekic wrote: > >> >> On Thu, Mar 28, 2002 at 09:45:02AM -0500, John Baldwin wrote: >> > >> > On 28-Mar-2002 Bosko Milekic wrote: >> > > >> > > I don't think we really need a ref. count API, per-se. I can think of >> > > several places that may need to do ref. counting but wouldn't want to >> > > do it with a bus-locked instruction because their reference counter(s) >> > > are already protected by an existing mutex. >> > > >> > > -Bosko >> > >> > Those places wouldn't use the API then I think. However, this would be >> > good >> > for things like ucreds, pargs, uidinfos and others. >> >> Yeah, absolutely. I was just pointing out that there's little point >> in having a reference counting API when a lot of the reference counting >> implementations won't be using it. > > but a lot would.. and that's GOOD. > > John, can you put your sugfgested API up again? > I remember it would work but was (in my mind) 'different'. > I didn't think it was intuative but at least get it up for discussion.. > > (I posted my suggested API a few mails ago.. I'm sure it too has > weaknesses though) It's old but it's been at ~jhb/paches/refcount.patch for about an eternity right now. The changes I would make to it are that I would change refcount_drop() (or release or whatever it is called) to return the current value rather than a boolean about the value being 0 or not. Also, once atomic_fetchadd() is done, it will change a fair bit. There would just be a MI header, no MD stuff, and we woudln't use a mutex for the debug version that uses asserts. > Julian -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 14:37:59 2002 Delivered-To: freebsd-arch@freebsd.org Received: from beastie.mckusick.com (beastie.mckusick.com [209.31.233.184]) by hub.freebsd.org (Postfix) with ESMTP id 2506537B421; Thu, 28 Mar 2002 14:37:23 -0800 (PST) Received: from beastie.mckusick.com (localhost [127.0.0.1]) by beastie.mckusick.com (8.11.6/8.11.6) with ESMTP id g2SMbLD99685; Thu, 28 Mar 2002 14:37:21 -0800 (PST) (envelope-from mckusick@beastie.mckusick.com) Message-Id: <200203282237.g2SMbLD99685@beastie.mckusick.com> To: Terry Lambert Subject: Re: vnode::v_op bugfix / PERFORCE change 8574 for review (fwd) Cc: "Brian F. Feldman" , arch@FreeBSD.ORG In-Reply-To: Your message of "Thu, 28 Mar 2002 14:05:22 PST." <3CA393A2.FB67CC2B@mindspring.com> Date: Thu, 28 Mar 2002 14:37:21 -0800 From: Kirk McKusick Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I concur with your suggestion below that the new patch is a better approach. Your ideal solution below sounds reasonable though I have not thought it through completely. Kirk McKusick =-=-=-=-= Date: Thu, 28 Mar 2002 14:05:22 -0800 From: Terry Lambert To: Kirk McKusick CC: "Brian F. Feldman" , arch@FreeBSD.ORG Subject: Re: vnode::v_op bugfix / PERFORCE change 8574 for review (fwd) Kirk McKusick wrote: > I believe your proposed fix is both necessary and correct. It should be noted that this code is the same as Alexander Kabaev's original proposed patch. Alexander's new patch, which I also suggested, is to overallocate the vector area, and then update it in place. I think that Alexander's new patch is a better workaround for the problem, both because (as he points out) the pointer fix fails when the VFS becomes preemptible, and because of the extra dereference overhead. I'm not terrifically happy about either one of them, but at least there is a much lower impact in refactoring the VOP lists (the new patch) rather than reallocating them, and since both operations are exceedingly rare (add/remove a new VOP), if one has to go in, I would prefer that it be Alexander's new one. Ideally, we would back out the default_vfs.c changes, add a "next" pointer field into the VOP list, link the list forward in order of adjacency at startup, and add new VOPs by linking them onto the end the the chain. Then none of this hackery would be necessary. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 14:45:12 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.rpi.edu (mail.rpi.edu [128.113.22.40]) by hub.freebsd.org (Postfix) with ESMTP id ECE1F37B41C for ; Thu, 28 Mar 2002 14:44:49 -0800 (PST) Received: from [128.113.24.47] (gilead.acs.rpi.edu [128.113.24.47]) by mail.rpi.edu (8.12.1/8.12.1) with ESMTP id g2SMikuW485382; Thu, 28 Mar 2002 17:44:46 -0500 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: <200203282130.g2SLU8D99368@beastie.mckusick.com> References: <200203282130.g2SLU8D99368@beastie.mckusick.com> Date: Thu, 28 Mar 2002 17:44:45 -0500 To: Kirk McKusick From: Garance A Drosihn Subject: Re: UFS snapshots in current Cc: arch@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Scanned-By: MIMEDefang 2.3 (www dot roaringpenguin dot com slash mimedefang) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG At 1:30 PM -0800 3/28/02, Kirk McKusick wrote: >The is no chance that snapshots would move into -current. >They touch thousands of lines of code. aside: s/current/stable/ ... More useful question: what should I look at for info on using snapshots? It seems to me I should already know, but I have to admit I don't. What's worse, I think I even asked this once before... In any case, I'd like to try them out (on current) for some ideas I have. -- Garance Alistair Drosehn = gad@eclipse.acs.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 14:50:51 2002 Delivered-To: freebsd-arch@freebsd.org Received: from beastie.mckusick.com (beastie.mckusick.com [209.31.233.184]) by hub.freebsd.org (Postfix) with ESMTP id B5BA737B42B for ; Thu, 28 Mar 2002 14:50:40 -0800 (PST) Received: from beastie.mckusick.com (localhost [127.0.0.1]) by beastie.mckusick.com (8.11.6/8.11.6) with ESMTP id g2SMoDD99826; Thu, 28 Mar 2002 14:50:13 -0800 (PST) (envelope-from mckusick@beastie.mckusick.com) Message-Id: <200203282250.g2SMoDD99826@beastie.mckusick.com> To: Garance A Drosihn Subject: Re: UFS snapshots in current Cc: arch@FreeBSD.ORG In-Reply-To: Your message of "Thu, 28 Mar 2002 17:44:45 EST." Date: Thu, 28 Mar 2002 14:50:13 -0800 From: Kirk McKusick Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Date: Thu, 28 Mar 2002 17:44:45 -0500 To: Kirk McKusick From: Garance A Drosihn Subject: Re: UFS snapshots in current Cc: arch@FreeBSD.ORG At 1:30 PM -0800 3/28/02, Kirk McKusick wrote: >The is no chance that snapshots would move into -current. >They touch thousands of lines of code. aside: s/current/stable/ ... Same brain-fault twice in two days. More useful question: what should I look at for info on using snapshots? It seems to me I should already know, but I have to admit I don't. What's worse, I think I even asked this once before... In any case, I'd like to try them out (on current) for some ideas I have. -- Garance Alistair Drosehn = gad@eclipse.acs.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu General references are found at: http://www.mckusick.com/softdep/index.html The soft updates paper has a section on snapshots. The background fsck paper goes into snapshots (and their general usage) in a bit more detail, so is likely to be more useful. Kirk McKusick To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 15: 0:52 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc53.attbi.com (rwcrmhc53.attbi.com [204.127.198.39]) by hub.freebsd.org (Postfix) with ESMTP id 42E9437B429; Thu, 28 Mar 2002 15:00:30 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc53.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020328230030.EADT2951.rwcrmhc53.attbi.com@InterJet.elischer.org>; Thu, 28 Mar 2002 23:00:30 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id OAA54528; Thu, 28 Mar 2002 14:45:19 -0800 (PST) Date: Thu, 28 Mar 2002 14:45:19 -0800 (PST) From: Julian Elischer To: John Baldwin Cc: arch@FreeBSD.ORG, Bosko Milekic Subject: Re: SMP safe reference counting In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, 28 Mar 2002, John Baldwin wrote: oops I see it's there.. you just mispelled the path.. sorry > > It's old but it's been at ~jhb/paches/refcount.patch for about an eternity > right now. The changes I would make to it are that I would change > refcount_drop() (or release or whatever it is called) to return the current > value rather than a boolean about the value being 0 or not. Also, once > atomic_fetchadd() is done, it will change a fair bit. There would just be > a MI header, no MD stuff, and we woudln't use a mutex for the debug version > that uses asserts. > > > Julian > > -- > > John Baldwin <>< http://www.FreeBSD.org/~jhb/ > "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 15:27:55 2002 Delivered-To: freebsd-arch@freebsd.org Received: from green.bikeshed.org (freefall.FreeBSD.org [216.136.204.21]) by hub.freebsd.org (Postfix) with ESMTP id 673F737B416; Thu, 28 Mar 2002 15:27:51 -0800 (PST) Received: from localhost (green@localhost) by green.bikeshed.org (8.11.6/8.11.6) with ESMTP id g2SNRog05733; Thu, 28 Mar 2002 18:27:50 -0500 (EST) (envelope-from green@green.bikeshed.org) Message-Id: <200203282327.g2SNRog05733@green.bikeshed.org> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: Kirk McKusick Cc: Terry Lambert , arch@FreeBSD.ORG Subject: Re: vnode::v_op bugfix / PERFORCE change 8574 for review (fwd) In-Reply-To: Your message of "Thu, 28 Mar 2002 14:37:21 PST." <200203282237.g2SMbLD99685@beastie.mckusick.com> From: "Brian F. Feldman" Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 28 Mar 2002 18:27:49 -0500 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Kirk McKusick wrote: > I concur with your suggestion below that the new patch is a better > approach. Your ideal solution below sounds reasonable though I have > not thought it through completely. I really, really hate the idea that the machine will panic without warning if the number of vnode ops to be used becomes greather than the statically-defined limit. Isn't there some truly generic solution? -- Brian Fundakowski Feldman \'[ FreeBSD ]''''''''''\ <> green@FreeBSD.org <> bfeldman@tislabs.com \ The Power to Serve! \ Opinions expressed are my own. \,,,,,,,,,,,,,,,,,,,,,,\ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 17: 1: 5 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.rpi.edu (mail.rpi.edu [128.113.22.40]) by hub.freebsd.org (Postfix) with ESMTP id A762C37B417; Thu, 28 Mar 2002 17:01:01 -0800 (PST) Received: from [128.113.24.47] (gilead.acs.rpi.edu [128.113.24.47]) by mail.rpi.edu (8.12.1/8.12.1) with ESMTP id g2T10VuW503694; Thu, 28 Mar 2002 20:00:31 -0500 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: <200203282327.g2SNRog05733@green.bikeshed.org> References: <200203282327.g2SNRog05733@green.bikeshed.org> Date: Thu, 28 Mar 2002 20:00:30 -0500 To: "Brian F. Feldman" , Kirk McKusick From: Garance A Drosihn Subject: Re: vnode::v_op bugfix / PERFORCE change 8574 for review (fwd) Cc: Terry Lambert , arch@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Scanned-By: MIMEDefang 2.3 (www dot roaringpenguin dot com slash mimedefang) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG At 6:27 PM -0500 3/28/02, Brian F. Feldman wrote: >Kirk McKusick wrote: > > I concur with your suggestion below that the new patch > > is a better approach. Your ideal solution below sounds > > reasonable though I have not thought it through completely. > >I really, really hate the idea that the machine will panic >without warning if the number of vnode ops to be used >becomes greather than the statically-defined limit. Isn't >there some truly generic solution? A previous message said new vnode-ops are very rare. I do not know what would trigger them, but I will note that one of the things I can brag about with freebsd is that I have a freebsd machine running a production service here which has now been up for 437 consecutive days. Are these events rare enough that I would never have to worry about ending an uptime-streak because of too many of them? -- Garance Alistair Drosehn = gad@eclipse.acs.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 17:39:56 2002 Delivered-To: freebsd-arch@freebsd.org Received: from harrier.prod.itd.earthlink.net (harrier.mail.pas.earthlink.net [207.217.120.12]) by hub.freebsd.org (Postfix) with ESMTP id 0A19037B41D; Thu, 28 Mar 2002 17:39:45 -0800 (PST) Received: from pool0142.cvx40-bradley.dialup.earthlink.net ([216.244.42.142] helo=mindspring.com) by harrier.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16qlMg-000304-00; Thu, 28 Mar 2002 17:39:39 -0800 Message-ID: <3CA3C59E.9791EED9@mindspring.com> Date: Thu, 28 Mar 2002 17:38:38 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: "Brian F. Feldman" Cc: Kirk McKusick , arch@FreeBSD.ORG Subject: Re: vnode::v_op bugfix / PERFORCE change 8574 for review (fwd) References: <200203282327.g2SNRog05733@green.bikeshed.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "Brian F. Feldman" wrote: > Kirk McKusick wrote: > > I concur with your suggestion below that the new patch is a better > > approach. Your ideal solution below sounds reasonable though I have > > not thought it through completely. > > I really, really hate the idea that the machine will panic without warning > if the number of vnode ops to be used becomes greather than the > statically-defined limit. Isn't there some truly generic solution? Yes. The problem is that the number of VOPs grows, and you need to find the new VOP, potentially in the old table. The problem comes from the implied assumption that there are old and new tables. If you remove that assumption, all the bogosity falls out of the code. This is the same way that the scheduling CPU and process group affinity crap that Linux puts up with just falls out of the code, as well, when you go to per CPU run queues (then it's a decision on migration between run queues, which can be a push operation, instead of a decision by a scheduler about what needs to be run next, which is a pull operation requiring locks to be SMP safe). The easiest fix is to add a field to the struct vnodeop_desc, like so: /* * This structure describes the vnode operation taking place. */ struct vnodeop_desc { ... struct vnodeop_desc *vdesc_next; }; Then at startup, traverse the vnodeop_descs[] array, like so: struct vnodeop_desc *descp; for( descp = vnodeop_descs; descp != ; descp++) { descp->vdesc_next = descp + 1; } descp->vdesc_next = NULL; Then instead of traversing the list with an index looking at entries up to some index/list entry marker, follow the next pointer down and stop when you hit NULL. When you want to add new VOP descriptors, just link them on the end, the "hard" way: void add_new_vop_desc( struct vnodeop_desc *addp) { struct vnodeop_desc **dpp; for( dpp = &vnodeop_descs; *dpp != NULL; dpp = &(*dpp)->vdesc_next)) continue; *dpp = addp; addp->vdesc_next = NULL; } ...no reallocation required. No allocation required, either, if you drain outstanding VOPs of a particular type at the interface that dispatchd them, and unlink the entry, as part of the unload. Otherwise, you need to copy an instance so it will still be there if the module that instanced it falls out from under it due to an unload or other operation. THen when you want to dereference a VOP, VOP's which are not supported by a local media file system are supposed to fall through to EOPNOTSUPP. Currently, this doesn't happen because of the default VOPs supposedly acting as a safety net... what they really are, is an impediment. This is because when an unrecognized VOP hits a transition bounary -- whether it be an underlying VFS in a stack, a local media FS, or a proxying barrier (for user space VFS impelmentations, or for VFS proxying over a network, as a better replacement for NFS) -- it's supposed to get EOPNOTSUPP. With the vfs_default.c code, what happens instead is that there has to be an explicit "EOPNOTSUPP" entry in the list of default entries. This is incredibly bogus, but it can be fixed seperately. For now, just add a lot of additional default slots to account for EOPNOTSUPP returns for currently undefined defaults (this is much less than what you are complaining about, since the overallocation you're talking about has to happen at each FS instance, and not just in the bottm level proxy boundary from which all other FS's inherit). Both the extra dereference added by the first patch... and the dereference that happened anyway... can be eliminated fairly trivially: 1) When you start up the system, for each static FS type, sort the VOPs static declarations into the same order as the VOPs in the vnodeop_descs[] array. Do this in place in the data segment (a bubble sort works; this is not a critical performance path, and it happens once). 2) Do the same thing for VOP lists when you load modules. 3) When you instance an FS (at mount time), rather than taking a reference to the VOP list -- which requires a reverse lookup at instance time, and then a dereference at operation time -- instance a full list of VOPs known, and do it in the order of the master list. Insert the unknown entries into the instance, rather than relying on the extra dereference to find them when they are not present. 4) When you stack a VFS, "collapse" unknown entries to the underlying layer's unknown entries. While this is not strictly necessary, it will mean that an N layer nullfs ends up being O(1) instead of O(N) on dereferences to do the call down (this was suggested by Rosenthal in the Sun "Stacking Vnode Architecture" paper). 5) When you add a VOP... link it onto end of the bas list, and link it onto the entry for any VFS' that supports it (e.g. the VFS that was loaded that caused the VOP to be added in the first place). ...obviously, for this to work, the bottim end has to be treated as a proxy barrier: not a list of defaults for which entries are provided. That means getting rid of the default_vfs.c stuff for the bottom end implementation, and implementing it to a proxy that converts all unknown entries at fan-out to calls to a VOP that returns EOPNOTSUPP. THis is actually in line with John Heidemann's original design for the code, as presented in the UCLA FICUS project papers, and his Master's Thesis (and, incidently, the documentation for the FreeBSD VFS stacking architecture). Basically: o All VFS's are stacking nodes, and o All VFS's inherit from a proxying VFS boundary, rather than a default implementation. o The bottom end implementation proxy turns all VOPs into EOPNOTSUPP returning VOPs (pretend they were proxied somehwere, and that somewhere had an implementation that returned EOPNOTSUPP). It's always nice when the code does what its documentation says it does. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 17:49: 7 2002 Delivered-To: freebsd-arch@freebsd.org Received: from green.bikeshed.org (freefall.FreeBSD.org [216.136.204.21]) by hub.freebsd.org (Postfix) with ESMTP id 9B26E37B404; Thu, 28 Mar 2002 17:49:03 -0800 (PST) Received: from localhost (green@localhost) by green.bikeshed.org (8.11.6/8.11.6) with ESMTP id g2T1n2r06394; Thu, 28 Mar 2002 20:49:02 -0500 (EST) (envelope-from green@green.bikeshed.org) Message-Id: <200203290149.g2T1n2r06394@green.bikeshed.org> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: Garance A Drosihn Cc: Kirk McKusick , Terry Lambert , arch@FreeBSD.ORG Subject: Re: vnode::v_op bugfix / PERFORCE change 8574 for review (fwd) In-Reply-To: Your message of "Thu, 28 Mar 2002 20:00:30 EST." From: "Brian F. Feldman" Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 28 Mar 2002 20:49:02 -0500 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Garance A Drosihn wrote: > At 6:27 PM -0500 3/28/02, Brian F. Feldman wrote: > >Kirk McKusick wrote: > > > I concur with your suggestion below that the new patch > > > is a better approach. Your ideal solution below sounds > > > reasonable though I have not thought it through completely. > > > >I really, really hate the idea that the machine will panic > >without warning if the number of vnode ops to be used > >becomes greather than the statically-defined limit. Isn't > >there some truly generic solution? > > A previous message said new vnode-ops are very rare. I > do not know what would trigger them, but I will note that > one of the things I can brag about with freebsd is that > I have a freebsd machine running a production service > here which has now been up for 437 consecutive days. Are > these events rare enough that I would never have to worry > about ending an uptime-streak because of too many of them? It's not likely to happen, I imagine, but I'd rather to make it "impossible" to happen rather than just not likely. -- Brian Fundakowski Feldman \'[ FreeBSD ]''''''''''\ <> green@FreeBSD.org <> bfeldman@tislabs.com \ The Power to Serve! \ Opinions expressed are my own. \,,,,,,,,,,,,,,,,,,,,,,\ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 17:52:35 2002 Delivered-To: freebsd-arch@freebsd.org Received: from harrier.prod.itd.earthlink.net (harrier.mail.pas.earthlink.net [207.217.120.12]) by hub.freebsd.org (Postfix) with ESMTP id 15EF437B443; Thu, 28 Mar 2002 17:52:18 -0800 (PST) Received: from pool0142.cvx40-bradley.dialup.earthlink.net ([216.244.42.142] helo=mindspring.com) by harrier.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16qlYu-0002lw-00; Thu, 28 Mar 2002 17:52:16 -0800 Message-ID: <3CA3C8B2.D9E15AFF@mindspring.com> Date: Thu, 28 Mar 2002 17:51:46 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Garance A Drosihn Cc: "Brian F. Feldman" , Kirk McKusick , arch@FreeBSD.ORG Subject: Re: vnode::v_op bugfix / PERFORCE change 8574 for review (fwd) References: <200203282327.g2SNRog05733@green.bikeshed.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Garance A Drosihn wrote: > A previous message said new vnode-ops are very rare. I > do not know what would trigger them, but I will note that > one of the things I can brag about with freebsd is that > I have a freebsd machine running a production service > here which has now been up for 437 consecutive days. Are > these events rare enough that I would never have to worry > about ending an uptime-streak because of too many of them? That was my claim. How often do you load FS's that support operations and additional system calls that reference those operations, that weren't there before you loaded the FS? The specific problem is adding non-existant VOP placeholders to existing VOP lists for FS's that don't support them anyway, so that when they are bogusly dereferenced out, they don't end up being a derference of whatever-happens-to-be-there-in-memory after thenend of the table. The "fix" is to replace the VOP list on all existing vnodes, but doing that screws up operations in progress, since there is no draining barrier. Actually, there's a really ugly hack you could do as well, which is the same ugly hack that would allow you to enable soft updates as a mount option, rather than having to use tunefs, but I really hesitate to suggest it, because it is truly incredibly ugly, since it simulates an unmount and a mount, without invalidating any open vnodes. Conversion of read-only to R/W and vice versa do it today; the ugly part is that you have to break the routine out, add a VFSOP for it, and then pass a function to it to call in the middle of the operation. I wouldn't recommend it, unless you cleared the boards on the mount code and really worked it over (again; several of us have had our noses in there in the past, myself included). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 17:56:35 2002 Delivered-To: freebsd-arch@freebsd.org Received: from albatross.prod.itd.earthlink.net (albatross.mail.pas.earthlink.net [207.217.120.120]) by hub.freebsd.org (Postfix) with ESMTP id 1458337B419; Thu, 28 Mar 2002 17:56:28 -0800 (PST) Received: from pool0142.cvx40-bradley.dialup.earthlink.net ([216.244.42.142] helo=mindspring.com) by albatross.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16qlcw-0002Av-00; Thu, 28 Mar 2002 17:56:26 -0800 Message-ID: <3CA3C9AC.D5280D16@mindspring.com> Date: Thu, 28 Mar 2002 17:55:56 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: "Brian F. Feldman" Cc: Garance A Drosihn , Kirk McKusick , arch@FreeBSD.ORG Subject: Re: vnode::v_op bugfix / PERFORCE change 8574 for review (fwd) References: <200203290149.g2T1n2r06394@green.bikeshed.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "Brian F. Feldman" wrote: > Garance A Drosihn wrote: > > A previous message said new vnode-ops are very rare. I > > do not know what would trigger them, but I will note that > > one of the things I can brag about with freebsd is that > > I have a freebsd machine running a production service > > here which has now been up for 437 consecutive days. Are > > these events rare enough that I would never have to worry > > about ending an uptime-streak because of too many of them? > > It's not likely to happen, I imagine, but I'd rather to make it "impossible" > to happen rather than just not likely. You can make it happen by loading a module that is not statically compiled, and which adds a VOP to the list of VOPs. The one that triggered this whole thread was NTFS, which adds a VOP for no good reason that I can discern (it should be a file ioctl, IMO). The workaround is to statically compile the module so that the VOP list doesn't get added to when you mount an NTFS volume. The operation is rare, in that it can only happen when you load a module that adds a VOP. Doing _that_ is rare. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 18:12:22 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.rpi.edu (mail.rpi.edu [128.113.22.40]) by hub.freebsd.org (Postfix) with ESMTP id ACDFF37B416 for ; Thu, 28 Mar 2002 18:12:20 -0800 (PST) Received: from [128.113.24.47] (gilead.acs.rpi.edu [128.113.24.47]) by mail.rpi.edu (8.12.1/8.12.1) with ESMTP id g2T2CIuW463528; Thu, 28 Mar 2002 21:12:19 -0500 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: <3CA3C9AC.D5280D16@mindspring.com> References: <200203290149.g2T1n2r06394@green.bikeshed.org> <3CA3C9AC.D5280D16@mindspring.com> Date: Thu, 28 Mar 2002 21:12:18 -0500 To: Terry Lambert From: Garance A Drosihn Subject: Re: vnode::v_op bugfix / PERFORCE change 8574 for review (fwd) Cc: arch@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Scanned-By: MIMEDefang 2.3 (www dot roaringpenguin dot com slash mimedefang) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG At 5:55 PM -0800 3/28/02, Terry Lambert wrote: >You can make it happen by loading a module that is not statically >compiled, and which adds a VOP to the list of VOPs. The one that >triggered this whole thread was NTFS, ... Okay. That certainly doesn't sound like something I would be doing multiple times on my production server machines! -- Garance Alistair Drosehn = gad@eclipse.acs.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 19:16:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from green.bikeshed.org (freefall.FreeBSD.org [216.136.204.21]) by hub.freebsd.org (Postfix) with ESMTP id EC1AF37B41F; Thu, 28 Mar 2002 19:16:12 -0800 (PST) Received: from localhost (green@localhost) by green.bikeshed.org (8.11.6/8.11.6) with ESMTP id g2T3GBL06732; Thu, 28 Mar 2002 22:16:11 -0500 (EST) (envelope-from green@green.bikeshed.org) Message-Id: <200203290316.g2T3GBL06732@green.bikeshed.org> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: Terry Lambert Cc: Garance A Drosihn , Kirk McKusick , arch@FreeBSD.ORG Subject: Re: vnode::v_op bugfix / PERFORCE change 8574 for review (fwd) In-Reply-To: Your message of "Thu, 28 Mar 2002 17:55:56 PST." <3CA3C9AC.D5280D16@mindspring.com> From: "Brian F. Feldman" Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 28 Mar 2002 22:16:11 -0500 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Terry Lambert wrote: > "Brian F. Feldman" wrote: > > Garance A Drosihn wrote: > > > A previous message said new vnode-ops are very rare. I > > > do not know what would trigger them, but I will note that > > > one of the things I can brag about with freebsd is that > > > I have a freebsd machine running a production service > > > here which has now been up for 437 consecutive days. Are > > > these events rare enough that I would never have to worry > > > about ending an uptime-streak because of too many of them? > > > > It's not likely to happen, I imagine, but I'd rather to make it "impossible" > > to happen rather than just not likely. > > You can make it happen by loading a module that is not statically > compiled, and which adds a VOP to the list of VOPs. The one that > triggered this whole thread was NTFS, which adds a VOP for no > good reason that I can discern (it should be a file ioctl, IMO). > > The workaround is to statically compile the module so that the > VOP list doesn't get added to when you mount an NTFS volume. > > The operation is rare, in that it can only happen when you > load a module that adds a VOP. Doing _that_ is rare. I should have clarified; I meant I want that panic() Garance is referring to to be nonexistant, not that I never want a VFS module to be able to define a new vnode operation. -- Brian Fundakowski Feldman \'[ FreeBSD ]''''''''''\ <> green@FreeBSD.org <> bfeldman@tislabs.com \ The Power to Serve! \ Opinions expressed are my own. \,,,,,,,,,,,,,,,,,,,,,,\ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Mar 28 22:51:59 2002 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id 870EC37B416; Thu, 28 Mar 2002 22:51:56 -0800 (PST) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.2/8.12.2) with ESMTP id g2T6pQe7027461; Fri, 29 Mar 2002 07:51:26 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Garance A Drosihn Cc: "Brian F. Feldman" , Kirk McKusick , Terry Lambert , arch@FreeBSD.ORG Subject: Re: vnode::v_op bugfix / PERFORCE change 8574 for review (fwd) In-Reply-To: Your message of "Thu, 28 Mar 2002 20:00:30 EST." Date: Fri, 29 Mar 2002 07:51:26 +0100 Message-ID: <27460.1017384686@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message , Garance A Drosihn writes: >A previous message said new vnode-ops are very rare. I'm not sure I know of any new vnode-ops in the history of the project apart from the kqueue stuff. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 29 11:32:28 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail14.speakeasy.net (mail14.speakeasy.net [216.254.0.214]) by hub.freebsd.org (Postfix) with ESMTP id 8CA2437B416 for ; Fri, 29 Mar 2002 11:32:21 -0800 (PST) Received: (qmail 7463 invoked from network); 29 Mar 2002 19:32:19 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) by mail14.speakeasy.net (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 29 Mar 2002 19:32:19 -0000 Received: from laptop.baldwin.cx (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.11.6/8.11.6) with ESMTP id g2TJX7v01230 for ; Fri, 29 Mar 2002 14:33:07 -0500 (EST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.2 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 Date: Fri, 29 Mar 2002 14:32:22 -0500 (EST) From: John Baldwin To: arch@FreeBSD.org Subject: curthread vs. passing thread pointers around Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG During a discussion on the smp@ list about changes to the suser() API to make use of td_ucred, it was brought up that the new suser() would be assuming that the passed in thread pointer was curthread so why not just use curthread in suser() and not pass in a pointer at all. There are several places in the kernel where the same assumption is made. Thus, my question is: which general approach should we follow, and should we perhaps switch to using explicit curthread's everywhere and stop passing thread pointers around on the stack? -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 29 11:42: 2 2002 Delivered-To: freebsd-arch@freebsd.org Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by hub.freebsd.org (Postfix) with ESMTP id C3DD437B417; Fri, 29 Mar 2002 11:41:58 -0800 (PST) Received: by elvis.mu.org (Postfix, from userid 1192) id A823CAE163; Fri, 29 Mar 2002 11:41:58 -0800 (PST) Date: Fri, 29 Mar 2002 11:41:58 -0800 From: Alfred Perlstein To: John Baldwin Cc: arch@FreeBSD.org Subject: Re: curthread vs. passing thread pointers around Message-ID: <20020329194158.GX93885@elvis.mu.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.3.27i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * John Baldwin [020329 11:32] wrote: > During a discussion on the smp@ list about changes to the suser() > API to make use of td_ucred, it was brought up that the new suser() > would be assuming that the passed in thread pointer was curthread > so why not just use curthread in suser() and not pass in a pointer > at all. There are several places in the kernel where the same > assumption is made. Thus, my question is: which general approach > should we follow, and should we perhaps switch to using explicit > curthread's everywhere and stop passing thread pointers around on > the stack? Yes and no. :) For instance, some APIs are broken such that they allow the passing of a proc/thread in, but if it isn't curproc then we die a painful death. Those functions need to stop taking a proc/thread pointer and be documented that they use the current process for cred/misc processing. It also depends on how painful/good it is to use globals, if accesses to per-cpu data are expensive then we don't want curproc/curthread if they are not then we obviously want to remove register and stack space usage by using global data. So it's a judgement call isn't it? :) -- -Alfred Perlstein [alfred@freebsd.org] 'Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom.' Tax deductible donations for FreeBSD: http://www.freebsdfoundation.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 29 11:55:16 2002 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id 6B0FE37B404; Fri, 29 Mar 2002 11:55:14 -0800 (PST) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.2/8.12.2) with ESMTP id g2TJspe7072089; Fri, 29 Mar 2002 20:54:52 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Alfred Perlstein Cc: John Baldwin , arch@FreeBSD.ORG Subject: Re: curthread vs. passing thread pointers around In-Reply-To: Your message of "Fri, 29 Mar 2002 11:41:58 PST." <20020329194158.GX93885@elvis.mu.org> Date: Fri, 29 Mar 2002 20:54:51 +0100 Message-ID: <72088.1017431691@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20020329194158.GX93885@elvis.mu.org>, Alfred Perlstein writes: >* John Baldwin [020329 11:32] wrote: >> During a discussion on the smp@ list about changes to the suser() >> API to make use of td_ucred, [...] On a related note: I intend to change the open/close/ioctl interface to device drivers from a "struct thread *" to a "struct ucred *". -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 29 12:25: 6 2002 Delivered-To: freebsd-arch@freebsd.org Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by hub.freebsd.org (Postfix) with ESMTP id 3212F37B400; Fri, 29 Mar 2002 12:25:04 -0800 (PST) Received: by elvis.mu.org (Postfix, from userid 1192) id 0FCBEAE1C6; Fri, 29 Mar 2002 12:25:04 -0800 (PST) Date: Fri, 29 Mar 2002 12:25:04 -0800 From: Alfred Perlstein To: Poul-Henning Kamp Cc: John Baldwin , arch@FreeBSD.ORG Subject: Re: curthread vs. passing thread pointers around Message-ID: <20020329202504.GZ93885@elvis.mu.org> References: <20020329194158.GX93885@elvis.mu.org> <72088.1017431691@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <72088.1017431691@critter.freebsd.dk> User-Agent: Mutt/1.3.27i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Poul-Henning Kamp [020329 11:55] wrote: > In message <20020329194158.GX93885@elvis.mu.org>, Alfred Perlstein writes: > >* John Baldwin [020329 11:32] wrote: > >> During a discussion on the smp@ list about changes to the suser() > >> API to make use of td_ucred, [...] > > On a related note: I intend to change the open/close/ioctl interface > to device drivers from a "struct thread *" to a "struct ucred *". What about people (ab)?using the device driver interface for proc related stuff? -- -Alfred Perlstein [alfred@freebsd.org] 'Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom.' Tax deductible donations for FreeBSD: http://www.freebsdfoundation.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 29 12:33: 2 2002 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id 878B937B405; Fri, 29 Mar 2002 12:32:58 -0800 (PST) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.2/8.12.2) with ESMTP id g2TKWae7079086; Fri, 29 Mar 2002 21:32:36 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Alfred Perlstein Cc: John Baldwin , arch@FreeBSD.ORG Subject: Re: curthread vs. passing thread pointers around In-Reply-To: Your message of "Fri, 29 Mar 2002 12:25:04 PST." <20020329202504.GZ93885@elvis.mu.org> Date: Fri, 29 Mar 2002 21:32:36 +0100 Message-ID: <79085.1017433956@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20020329202504.GZ93885@elvis.mu.org>, Alfred Perlstein writes: >* Poul-Henning Kamp [020329 11:55] wrote: >> In message <20020329194158.GX93885@elvis.mu.org>, Alfred Perlstein writes: >> >* John Baldwin [020329 11:32] wrote: >> >> During a discussion on the smp@ list about changes to the suser() >> >> API to make use of td_ucred, [...] >> >> On a related note: I intend to change the open/close/ioctl interface >> to device drivers from a "struct thread *" to a "struct ucred *". > >What about people (ab)?using the device driver interface for proc >related stuff? The main purpose of the excercise is stop such abuse: People think they can track per instance using that argument, and _that_ just ain't going to happen until we hang devices directly under struct file and doing that will screw filesystems which use VOP's to access their device big time. There are two cases of non-abuse which I know off: streams and /dev/fd* streams I'm not sure about yet (is it even in use any more ?) /dev/fd could be solved by embedding fdescfs into devfs. Either way, it's a major patch, which can only partially be machine generated so it is probably not iminent. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 29 13:20:32 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc53.attbi.com (rwcrmhc53.attbi.com [204.127.198.39]) by hub.freebsd.org (Postfix) with ESMTP id B7DB837B400; Fri, 29 Mar 2002 13:20:14 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc53.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020329212014.SVOW2928.rwcrmhc53.attbi.com@InterJet.elischer.org>; Fri, 29 Mar 2002 21:20:14 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id MAA00923; Fri, 29 Mar 2002 12:41:47 -0800 (PST) Date: Fri, 29 Mar 2002 12:41:46 -0800 (PST) From: Julian Elischer To: Poul-Henning Kamp Cc: Alfred Perlstein , John Baldwin , arch@FreeBSD.ORG Subject: Re: curthread vs. passing thread pointers around In-Reply-To: <72088.1017431691@critter.freebsd.dk> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, 29 Mar 2002, Poul-Henning Kamp wrote: > In message <20020329194158.GX93885@elvis.mu.org>, Alfred Perlstein writes: > >* John Baldwin [020329 11:32] wrote: > >> During a discussion on the smp@ list about changes to the suser() > >> API to make use of td_ucred, [...] > > On a related note: I intend to change the open/close/ioctl interface > to device drivers from a "struct thread *" to a "struct ucred That's probably ok, though I have an uneasy feeling about it.. I'd go further and say that you should not even pass that. anyone needing it can do curthread->td_ucred. Hardly any drivers use it. Since AIO doesn't include open/close I am not sure I can think of a case when the curthread is not the thread that should be charged/authorised with the open/ioctl/close. In many cases td is only used to fead to suser*() in which case it wouldn't be needed at all. I think we could make a case for CURRENT code that we need not pass anything at all, but what are we cutting out in th epossible future? > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 29 13:30:21 2002 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id 7C8F537B419; Fri, 29 Mar 2002 13:30:11 -0800 (PST) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.2/8.12.2) with ESMTP id g2TLTke7089672; Fri, 29 Mar 2002 22:29:46 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Julian Elischer Cc: Alfred Perlstein , John Baldwin , arch@FreeBSD.ORG Subject: Re: curthread vs. passing thread pointers around In-Reply-To: Your message of "Fri, 29 Mar 2002 12:41:46 PST." Date: Fri, 29 Mar 2002 22:29:46 +0100 Message-ID: <89671.1017437386@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message , Juli an Elischer writes: >> On a related note: I intend to change the open/close/ioctl interface >> to device drivers from a "struct thread *" to a "struct ucred > >That's probably ok, though I have an uneasy feeling about it.. > >I'd go further and say that you should not even pass that. >anyone needing it can do curthread->td_ucred. Yeah, well, unless you run GEOM where the request may have been queued and you're running in context of a worker thread. (The infinite stacking issue again). >I think we could make a case for CURRENT code that we need not >pass anything at all, but what are we cutting out in th epossible future? That's what I'm afraid off too. There are several things in this, one of which is locking, the other the scale of changes needed to fix up the tree. The simple solution is probably to add three new members to cdevsw{}: sopen, sclose, sioctl (s for simple or something) which doesn't have that final argument at all. Drivers can be converted gradually. Specfs can figure out which version of the method to call, and do locking as/if needed. I really wish we had a compile-time optimized version of KOBJ we could use for devsw... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 29 13:30:59 2002 Delivered-To: freebsd-arch@freebsd.org Received: from beastie.mckusick.com (beastie.mckusick.com [209.31.233.184]) by hub.freebsd.org (Postfix) with ESMTP id AAAFF37B400; Fri, 29 Mar 2002 13:30:53 -0800 (PST) Received: from beastie.mckusick.com (localhost [127.0.0.1]) by beastie.mckusick.com (8.11.6/8.11.6) with ESMTP id g2TLUrD01863; Fri, 29 Mar 2002 13:30:53 -0800 (PST) (envelope-from mckusick@beastie.mckusick.com) Message-Id: <200203292130.g2TLUrD01863@beastie.mckusick.com> To: John Baldwin Subject: Re: curthread vs. passing thread pointers around Cc: arch@FreeBSD.ORG In-Reply-To: Your message of "Fri, 29 Mar 2002 14:32:22 EST." Date: Fri, 29 Mar 2002 13:30:52 -0800 From: Kirk McKusick Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Date: Fri, 29 Mar 2002 14:32:22 -0500 (EST) From: John Baldwin To: arch@FreeBSD.ORG Subject: curthread vs. passing thread pointers around Sender: owner-freebsd-arch@FreeBSD.ORG List-ID: List-Archive: (Web Archive) During a discussion on the smp@ list about changes to the suser() API to make use of td_ucred, it was brought up that the new suser() would be assuming that the passed in thread pointer was curthread so why not just use curthread in suser() and not pass in a pointer at all. There are several places in the kernel where the same assumption is made. Thus, my question is: which general approach should we follow, and should we perhaps switch to using explicit curthread's everywhere and stop passing thread pointers around on the stack? -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ The whole business of passing proc pointers got started in the late 1980's when we went on a big kernel cleanup campaign to get rid of all references to u.u_variable. Most of that efforts worked out well. For example, having most kernel functions return an error rather than stuffing it in u.u_error. That also explains the numerous /* XXX */ comments scattered through the kernel whenever curproc (or now curthread) gets used; the idea was that the proc (or thread) pointer should have been a parameter. A few years ago I set out to try to clean up all those global references, and found that I had to add a proc pointer parameter to nearly every function in the top half of the kernel. I concluded that the extra overhead was simply not worth it, and that we should just continue using the global pointer as we do now. There are still instances where credentials should be passed as parameters since it is not the case that using the current threads credential is the right thing to do. For example, the NFS server wants to use the credential of the incoming request, not its own in deciding filesystem access. I can also believe that suser() might have places where it would want to use a different credential than that of the current thread. So, while removing proc/thread parameters from existing functions is reasonable, we should consider whether they should be replaced with pointers to credentials. Kirk McKusick To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 29 13:38:15 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail11.speakeasy.net (mail11.speakeasy.net [216.254.0.211]) by hub.freebsd.org (Postfix) with ESMTP id 36B0137B400 for ; Fri, 29 Mar 2002 13:38:11 -0800 (PST) Received: (qmail 15683 invoked from network); 29 Mar 2002 21:38:10 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) by mail11.speakeasy.net (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 29 Mar 2002 21:38:10 -0000 Received: from laptop.baldwin.cx (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.11.6/8.11.6) with ESMTP id g2TLcwv01732; Fri, 29 Mar 2002 16:38:58 -0500 (EST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.2 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200203292130.g2TLUrD01863@beastie.mckusick.com> Date: Fri, 29 Mar 2002 16:38:12 -0500 (EST) From: John Baldwin To: Kirk McKusick Subject: Re: curthread vs. passing thread pointers around Cc: arch@FreeBSD.ORG Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 29-Mar-2002 Kirk McKusick wrote: > Date: Fri, 29 Mar 2002 14:32:22 -0500 (EST) > From: John Baldwin > To: arch@FreeBSD.ORG > Subject: curthread vs. passing thread pointers around > Sender: owner-freebsd-arch@FreeBSD.ORG > List-ID: > List-Archive: (Web Archive) > > During a discussion on the smp@ list about changes to the suser() > API to make use of td_ucred, it was brought up that the new suser() > would be assuming that the passed in thread pointer was curthread > so why not just use curthread in suser() and not pass in a pointer > at all. There are several places in the kernel where the same > assumption is made. Thus, my question is: which general approach > should we follow, and should we perhaps switch to using explicit > curthread's everywhere and stop passing thread pointers around on > the stack? > > -- > > John Baldwin <>< http://www.FreeBSD.org/~jhb/ > "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ > > The whole business of passing proc pointers got started in the late > 1980's when we went on a big kernel cleanup campaign to get rid of > all references to u.u_variable. Most of that efforts worked out > well. For example, having most kernel functions return an error > rather than stuffing it in u.u_error. That also explains the > numerous /* XXX */ comments scattered through the kernel whenever > curproc (or now curthread) gets used; the idea was that the proc > (or thread) pointer should have been a parameter. A few years ago > I set out to try to clean up all those global references, and found > that I had to add a proc pointer parameter to nearly every function > in the top half of the kernel. I concluded that the extra overhead > was simply not worth it, and that we should just continue using the > global pointer as we do now. There are still instances where > credentials should be passed as parameters since it is not the case > that using the current threads credential is the right thing to do. > For example, the NFS server wants to use the credential of the > incoming request, not its own in deciding filesystem access. I > can also believe that suser() might have places where it would > want to use a different credential than that of the current thread. > So, while removing proc/thread parameters from existing functions > is reasonable, we should consider whether they should be replaced > with pointers to credentials. Yes, in the suser() discussion on smp@ we will have at least two versions of suser(), one which uses the credential of curthread, and another which accepts a credential directly to perform the test against. > Kirk McKusick -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 29 13:40:15 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc53.attbi.com (rwcrmhc53.attbi.com [204.127.198.39]) by hub.freebsd.org (Postfix) with ESMTP id 2A3A337B416; Fri, 29 Mar 2002 13:40:08 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc53.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020329214004.TMVH2928.rwcrmhc53.attbi.com@InterJet.elischer.org>; Fri, 29 Mar 2002 21:40:04 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id MAA00932; Fri, 29 Mar 2002 12:45:30 -0800 (PST) Date: Fri, 29 Mar 2002 12:45:30 -0800 (PST) From: Julian Elischer To: Alfred Perlstein Cc: Poul-Henning Kamp , John Baldwin , arch@FreeBSD.ORG Subject: Re: curthread vs. passing thread pointers around In-Reply-To: <20020329202504.GZ93885@elvis.mu.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, 29 Mar 2002, Alfred Perlstein wrote: > * Poul-Henning Kamp [020329 11:55] wrote: > > In message <20020329194158.GX93885@elvis.mu.org>, Alfred Perlstein writes: > > >* John Baldwin [020329 11:32] wrote: > > >> During a discussion on the smp@ list about changes to the suser() > > >> API to make use of td_ucred, [...] > > > > On a related note: I intend to change the open/close/ioctl interface > > to device drivers from a "struct thread *" to a "struct ucred *". > > What about people (ab)?using the device driver interface for proc > related stuff? then they can (AB)use curthread.. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 29 15:20:58 2002 Delivered-To: freebsd-arch@freebsd.org Received: from rwcrmhc53.attbi.com (rwcrmhc53.attbi.com [204.127.198.39]) by hub.freebsd.org (Postfix) with ESMTP id A52DE37B429; Fri, 29 Mar 2002 15:20:10 -0800 (PST) Received: from InterJet.elischer.org ([12.232.206.8]) by rwcrmhc53.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with ESMTP id <20020329232010.WOTC2928.rwcrmhc53.attbi.com@InterJet.elischer.org>; Fri, 29 Mar 2002 23:20:10 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id PAA01534; Fri, 29 Mar 2002 15:02:47 -0800 (PST) Date: Fri, 29 Mar 2002 15:02:46 -0800 (PST) From: Julian Elischer To: Kirk McKusick Cc: John Baldwin , arch@FreeBSD.ORG Subject: Re: curthread vs. passing thread pointers around In-Reply-To: <200203292130.g2TLUrD01863@beastie.mckusick.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, 29 Mar 2002, Kirk McKusick wrote: > So, while removing proc/thread parameters from existing functions > is reasonable, we should consider whether they should be replaced > with pointers to credentials. For that we have suser_cred(cred,...) elready. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 29 15:52:54 2002 Delivered-To: freebsd-arch@freebsd.org Received: from scaup.prod.itd.earthlink.net (scaup.mail.pas.earthlink.net [207.217.120.49]) by hub.freebsd.org (Postfix) with ESMTP id B079437B405; Fri, 29 Mar 2002 15:52:50 -0800 (PST) Received: from pool0156.cvx40-bradley.dialup.earthlink.net ([216.244.42.156] helo=mindspring.com) by scaup.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16r6Ar-0000hw-00; Fri, 29 Mar 2002 15:52:49 -0800 Message-ID: <3CA4FE3B.C5EE8F4A@mindspring.com> Date: Fri, 29 Mar 2002 15:52:27 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Alfred Perlstein Cc: John Baldwin , arch@FreeBSD.org Subject: Re: curthread vs. passing thread pointers around References: <20020329194158.GX93885@elvis.mu.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Alfred Perlstein wrote: > For instance, some APIs are broken such that they allow the passing > of a proc/thread in, but if it isn't curproc then we die a painful > death. Those functions need to stop taking a proc/thread pointer > and be documented that they use the current process for cred/misc > processing. Heh. My reaction was that they should be fixed to not die a painful death because of an implied equality dependency that shouldn't be there in the first place... not have the passed value ripped out so that the assumption can be enshrined for all time. Tomato/tomato. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 29 15:57:52 2002 Delivered-To: freebsd-arch@freebsd.org Received: from scaup.prod.itd.earthlink.net (scaup.mail.pas.earthlink.net [207.217.120.49]) by hub.freebsd.org (Postfix) with ESMTP id 8CEE237B417; Fri, 29 Mar 2002 15:57:45 -0800 (PST) Received: from pool0156.cvx40-bradley.dialup.earthlink.net ([216.244.42.156] helo=mindspring.com) by scaup.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16r6FV-0005v2-00; Fri, 29 Mar 2002 15:57:37 -0800 Message-ID: <3CA4FF5A.A672E5E5@mindspring.com> Date: Fri, 29 Mar 2002 15:57:14 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Poul-Henning Kamp Cc: Alfred Perlstein , John Baldwin , arch@FreeBSD.ORG Subject: Re: curthread vs. passing thread pointers around References: <79085.1017433956@critter.freebsd.dk> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Poul-Henning Kamp wrote: > In message <20020329202504.GZ93885@elvis.mu.org>, Alfred Perlstein writes: > >What about people (ab)?using the device driver interface for proc > >related stuff? > > The main purpose of the excercise is stop such abuse: People think > they can track per instance using that argument, and _that_ just ain't > going to happen until we hang devices directly under struct file > and doing that will screw filesystems which use VOP's to access > their device big time. Poul's right. If you were guaranteed the ability to do this, then people would be able to use multiple sessions of VMWare on FreeBSD, only a few *YEARS* after VMWare was first ported. Then where would we be? Oh wait. That's a good thing. Poul's wrong. Never mind. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 29 20:12:21 2002 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 7765237B400; Fri, 29 Mar 2002 20:12:16 -0800 (PST) Received: from fledge.watson.org (fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.6/8.11.6) with SMTP id g2U4CCw74331; Fri, 29 Mar 2002 23:12:12 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Fri, 29 Mar 2002 23:12:12 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: John Baldwin Cc: arch@FreeBSD.org Subject: Re: curthread vs. passing thread pointers around In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, 29 Mar 2002, John Baldwin wrote: > During a discussion on the smp@ list about changes to the suser() API > to make use of td_ucred, it was brought up that the new suser() would > be assuming that the passed in thread pointer was curthread so why not > just use curthread in suser() and not pass in a pointer at all. There > are several places in the kernel where the same assumption is made. > Thus, my question is: which general approach should we follow, and > should we perhaps switch to using explicit curthread's everywhere and > stop passing thread pointers around on the stack? There are a number of situations which are rather sticky involving credentials and VFS. Basically, the set of relevant credentials is often two or three different credentials (especially for filesystems like procfs). Typically there's a cached credential associated with a persistent data naming abstraction (sich as the struct file pointing at a socket or vnode), as well as the active requesting credential. This means that low-level access control primitives almost always need to accept an explicit credential reference, although higher level primitives might make assumptions about the source of the credential. In the file system and specfs code, I've seen both the explicit VFS credential (usually sourced from a struct file, although not always), and the active requesting credential used. Frequently, incorrectly, actually. For example, there are a number of places where the cached credential is used and really should not be; likewise, there are a number of places where incorrect assumptions may be made about credentials belonging to curproc/thread. For VFS, which is a special case, I'd actually like to see both credentials passed down the stack explicitly, meaning that worker threads and processes in kernel don't have to tweak their active credential in order to make a request on behalf of another thread or process (think async io, async nfs rpc activities, etc). This actually suggests a model something like... vop_foo(vp, ..., cachedcred, activecred); for almost all VOP's. For many other simpler non-VFS/socket/... cases, moving to implicit curthread-derived credentials is probably the right choice, as it limits the exposure of specific structures in code that doesn't care about the structures. Even if we don't move to using curthread (for performance or other reasons), we should be making *extremely* heavy use of KASSERT(). Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Mar 29 20:16:31 2002 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 1D89037B41B; Fri, 29 Mar 2002 20:16:28 -0800 (PST) Received: from fledge.watson.org (fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.6/8.11.6) with SMTP id g2U4GOw74352; Fri, 29 Mar 2002 23:16:24 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Fri, 29 Mar 2002 23:16:24 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: John Baldwin Cc: arch@FreeBSD.org Subject: Re: curthread vs. passing thread pointers around In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, 29 Mar 2002, Robert Watson wrote: > For VFS, which is a special case, I'd actually like to see both > credentials passed down the stack explicitly, meaning that worker > threads and processes in kernel don't have to tweak their active > credential in order to make a request on behalf of another thread or > process (think async io, async nfs rpc activities, etc). This actually > suggests a model something like... BTW, this would also address races and problems associated with files kept open by the kernel for kernel-sponsored activies. Right now, when the kernel "saves" a credential for use with a saved vnode, it can't guarantee that all access control uses the saved credential. Some may use the active thread credential from curthread. For example, UFS will frequently use curthread->td_ucred for authorization when writing out account or quota data, which is arguably wrong. The quota and accounting code should cache two credentials for different parts of the access control decision, and both of those should be explicitly different from curthread's. This would also fix MAC and these functions, FYI :-). Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Mar 30 8:42:58 2002 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id 526FC37B416 for ; Sat, 30 Mar 2002 08:42:53 -0800 (PST) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.2/8.12.2) with ESMTP id g2UGdle7003838 for ; Sat, 30 Mar 2002 17:39:48 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: arch@freebsd.org Subject: Rewamping kernel dumps... From: Poul-Henning Kamp Date: Sat, 30 Mar 2002 17:39:47 +0100 Message-ID: <3837.1017506387@critter.freebsd.dk> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Here is what I plan to do to the kernel dumping code. Device drivers can provide a dumping function, which will act as an blocked sequential write device. This _can_ be the traditional swap-partition-on-disk, but it could also be tapes or TFTP over a network interface. The function has the following type: typedef int dumper_t( void *private, /* Private to the driver. */ void *virtual, /* Virtual (mapped) address. */ void *physical, /* Physical address of virtual. */ off_t length); /* Number of bytes to dump. */ To register a dumping function, the driver fills out a data structure and calls set_dumper(): struct dumperinfo { dumper_t *dumper; /* Dumping function. */ void *private; /* Private parts. */ u_int blocksize; /* Size of block in bytes. */ off_t mediasize; /* Space available in bytes. */ }; int set_dumper(struct dumperinfo *); Only one dumping function can be registered at a time, the current registration can be cleared by calling set_dumper(NULL); Once a dump is desired, a MD function is called: void dumpsys(struct dumperinfo *); This function must dump any magic headers and any desired memory in whatever order the format for the platform is by repeatedly calling the struct dumperinfo->dumper function. If the dumper function returns non-zero, the dump should be aborted since the hardware (presumably) failed. The configuration mechanism for disks will change so that dumpon(8) will open the device and issue an ioctl to turn dumping on this device on/off, rather than the current sysctl based mechanism. I understand that various people have some ideas about the layout of the dump on the sequential media, this is an area I will not be involved in, and I suggest people interested hash this out somehow. Comments ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Mar 30 11:25:17 2002 Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 5F5EA37B41A; Sat, 30 Mar 2002 11:25:14 -0800 (PST) Received: by flood.ping.uio.no (Postfix, from userid 2602) id 057A75346; Sat, 30 Mar 2002 20:25:11 +0100 (CET) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Poul-Henning Kamp Cc: arch@freebsd.org Subject: Re: Rewamping kernel dumps... References: <3837.1017506387@critter.freebsd.dk> From: Dag-Erling Smorgrav Date: 30 Mar 2002 20:25:10 +0100 In-Reply-To: <3837.1017506387@critter.freebsd.dk> Message-ID: Lines: 25 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.1 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Poul-Henning Kamp writes: > Here is what I plan to do to the kernel dumping code. > [...] Sounds good. > I understand that various people have some ideas about the layout of > the dump on the sequential media, this is an area I will not be involved > in, and I suggest people interested hash this out somehow. I uphold my earlier suggestion of a block of metadata formatted as text. To accomodate both sequential-access and random-access media, the size of this metadata block should be fixed (64k sound OK?), and it should be written out before and after the dump - before so you can read it and the dump off a tape without any seeking, and after so it can be found in a fixed location on a disk partition (since the dump is written at the end rather than the beginning of the partition). This eliminates the need for libkvm, and makes it possible to save a dump anywhere, anytime - currently, you have to run the kernel that crashed on the system that crashed, or an identical system, for savecore(8) to work. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Mar 30 11:29: 5 2002 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id C65B637B41A for ; Sat, 30 Mar 2002 11:29:02 -0800 (PST) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.2/8.12.2) with ESMTP id g2UJSZe7035317; Sat, 30 Mar 2002 20:28:39 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Dag-Erling Smorgrav Cc: arch@freebsd.org Subject: Re: Rewamping kernel dumps... In-Reply-To: Your message of "30 Mar 2002 20:25:10 +0100." Date: Sat, 30 Mar 2002 20:28:35 +0100 Message-ID: <35316.1017516515@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message , Dag-Erling Smorgrav writes: >Poul-Henning Kamp writes: >> Here is what I plan to do to the kernel dumping code. >> [...] > >Sounds good. > >> I understand that various people have some ideas about the layout of >> the dump on the sequential media, this is an area I will not be involved >> in, and I suggest people interested hash this out somehow. > >I uphold my earlier suggestion of a block of metadata formatted as >text. I would probably have stuck it all into tar(1) format from the beginning of the partition. One file for the msgbuf, one file for a ram snapshot etc etc. But as I said, I'm not writing that bit. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Mar 30 12:29:54 2002 Delivered-To: freebsd-arch@freebsd.org Received: from avocet.prod.itd.earthlink.net (avocet.mail.pas.earthlink.net [207.217.120.50]) by hub.freebsd.org (Postfix) with ESMTP id D93ED37B41C; Sat, 30 Mar 2002 12:29:52 -0800 (PST) Received: from pool0015.cvx22-bradley.dialup.earthlink.net ([209.179.198.15] helo=mindspring.com) by avocet.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16rPU0-0000Yr-00; Sat, 30 Mar 2002 12:29:52 -0800 Message-ID: <3CA6201D.F1FB5DD5@mindspring.com> Date: Sat, 30 Mar 2002 12:29:17 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Poul-Henning Kamp Cc: arch@freebsd.org Subject: Re: Rewamping kernel dumps... References: <3837.1017506387@critter.freebsd.dk> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Poul-Henning Kamp wrote: > Device drivers can provide a dumping function, which will act as > an blocked sequential write device. This _can_ be the traditional > swap-partition-on-disk, but it could also be tapes or TFTP over a > network interface. Did anyone else bring up "dump/restore" when Poul originally removed block devices from FreeBSD? -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Mar 30 16:19:24 2002 Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id B9E2437B405 for ; Sat, 30 Mar 2002 16:19:20 -0800 (PST) Received: by flood.ping.uio.no (Postfix, from userid 2602) id CCEB65346; Sun, 31 Mar 2002 01:19:17 +0100 (CET) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Poul-Henning Kamp Cc: arch@freebsd.org Subject: Re: Rewamping kernel dumps... References: <35316.1017516515@critter.freebsd.dk> From: Dag-Erling Smorgrav Date: 31 Mar 2002 01:19:16 +0100 In-Reply-To: <35316.1017516515@critter.freebsd.dk> Message-ID: Lines: 13 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.1 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Poul-Henning Kamp writes: > I would probably have stuck it all into tar(1) format from the > beginning of the partition. One file for the msgbuf, one file for > a ram snapshot etc etc. You can't put it at the beginning of the partition, because savecore(8) can't run before fsck(8), and fsck(8) may cause the system to swap and overwrite the core dump before savecore(8) has had a chance to save it. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Mar 30 17:18:27 2002 Delivered-To: freebsd-arch@freebsd.org Received: from pintail.mail.pas.earthlink.net (pintail.mail.pas.earthlink.net [207.217.120.122]) by hub.freebsd.org (Postfix) with ESMTP id 83BC537B416 for ; Sat, 30 Mar 2002 17:18:23 -0800 (PST) Received: from pool0290.cvx21-bradley.dialup.earthlink.net ([209.179.193.35] helo=mindspring.com) by pintail.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 16rTz3-0005l3-00; Sat, 30 Mar 2002 17:18:13 -0800 Message-ID: <3CA663BD.A29AC43@mindspring.com> Date: Sat, 30 Mar 2002 17:17:49 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Dag-Erling Smorgrav Cc: Poul-Henning Kamp , arch@freebsd.org Subject: Re: Rewamping kernel dumps... References: <35316.1017516515@critter.freebsd.dk> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Dag-Erling Smorgrav wrote: > Poul-Henning Kamp writes: > > I would probably have stuck it all into tar(1) format from the > > beginning of the partition. One file for the msgbuf, one file for > > a ram snapshot etc etc. > > You can't put it at the beginning of the partition, because > savecore(8) can't run before fsck(8), and fsck(8) may cause the system > to swap and overwrite the core dump before savecore(8) has had a > chance to save it. You kind of want swapping to be enabled with swapon on all devices, rather than implicitly enabled on one device at boot time, so that unless you explicitly enable it, you do not have *any* swap available. IMO, fsck is just as likely to fail from a lack of 5 swap devices as it is to fail for lack of 1. The code needs to run in the available memory, rather than in more than the available memory. This would resolve the problem entirely. Also making FreeBSD able to run swap-less without a panic would clean up a couple other problems, which show up when you don't configure any swap at all, or show up exceedingly rarely and are impossible to really track down under swap exhaustion. Matt would be a good person to comment, at this point, since he has done a lot of work on how the system behaves under resource exhaustion conditions. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Mar 30 20: 1:15 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.rpi.edu (mail.rpi.edu [128.113.22.40]) by hub.freebsd.org (Postfix) with ESMTP id AE68D37B41C for ; Sat, 30 Mar 2002 20:01:08 -0800 (PST) Received: from [128.113.24.47] (gilead.acs.rpi.edu [128.113.24.47]) by mail.rpi.edu (8.12.1/8.12.1) with ESMTP id g2V4154t504482; Sat, 30 Mar 2002 23:01:05 -0500 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: <200203282250.g2SMoDD99826@beastie.mckusick.com> References: <200203282250.g2SMoDD99826@beastie.mckusick.com> Date: Sat, 30 Mar 2002 23:01:04 -0500 To: Kirk McKusick From: Garance A Drosihn Subject: Re: UFS snapshots in current Cc: arch@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Scanned-By: MIMEDefang 2.3 (www dot roaringpenguin dot com slash mimedefang) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG At 2:50 PM -0800 3/28/02, Kirk McKusick wrote: > From: Garance A Drosihn > > More useful question: what should I look at for > info on using snapshots? > >General references are found at: > > http://www.mckusick.com/softdep/index.html > >The soft updates paper has a section on snapshots. The >background fsck paper goes into snapshots (and their >general usage) in a bit more detail, so is likely to >be more useful. Okay, well, I was trying this out and I had something odd happen. As I sit here waiting for my PC to return to life, I'll ask if what I was trying to do something which would be a BadIdea(tm). I wanted to use a snapshot to "freeze" the system in time before doing a large-scale set of changes. So, I created a snapshot of /usr. I then did a 'cvs update' of /usr/src, and checked how the snapshot worked. It seemed to be doing just exactly what I wanted. I then got interrupted and forgot about the snapshot. I may have also done a reboot, but I'm not sure about that. Days pass, I come back, do another 'cvs update' of the latest sources. I buildworld, buildkernel, installkernel. So far no problem (note that /usr/obj is on a separate partition). I reboot, and go to do an installworld. It goes merrily along for awhile, and then just stops. After letting it sit there for about 15 minutes, I attention out of the buildworld. It had not used a lot of cpu time. so, I tried it again. it went a few lines and then stopped again. I finally decided to just reboot. The shutdown started, and then got stuck at 'Writing entropy file'. I then hit the power button. It didn't power off! I had to unplug the machine to get it to reboot. After it came back up, I removed the checkpoint file. I forget how large it was, but it was pretty large. Before removing it, I did a 'df -k -i', and /usr had plenty of spare filespace and spare inodes. I didn't know what else I should check for. By then the system told me that I had to do an fsck of /usr by hand, so I've done that, and now have 1500 files in lost+found. It happens I have an exact duplicate of my current system in a different set of partitions, so one way or another I can get the system back to what I need. But the main question is: Could my problems have been due to that snapshot being active when I did the 'installworld'? I only *meant* it to be there during the 'cvs update', buildworld, and buildkernel, but I had forgotten it was there. It's hard to believe that this was due to any of the changes made to current in the past week. And if it is a bad idea, how hard would it be to get it to fail a little more gracefully? :-) Also, if this had worked, I probably would have duplicated this new system to my backup partitions. From what I've read of the paper, I think that would not cause a problem (other than maybe wasting disk space), because a plain copy (using pax) wouldn't add the snapshot file to the list of snapshots in the superblock of the destination partition. True? -- Garance Alistair Drosehn = gad@eclipse.acs.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Mar 30 20:18:49 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 8C34837B400; Sat, 30 Mar 2002 20:18:44 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id OAA09086; Sun, 31 Mar 2002 14:09:51 +1000 Date: Sun, 31 Mar 2002 14:10:37 +1000 (EST) From: Bruce Evans X-X-Sender: bde@gamplex.bde.org To: Poul-Henning Kamp Cc: arch@FreeBSD.ORG Subject: Re: Rewamping kernel dumps... In-Reply-To: <3837.1017506387@critter.freebsd.dk> Message-ID: <20020331133114.A4567-100000@gamplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, 30 Mar 2002, Poul-Henning Kamp wrote: > Here is what I plan to do to the kernel dumping code. > > Device drivers can provide a dumping function, which will act as > an blocked sequential write device. This _can_ be the traditional > swap-partition-on-disk, but it could also be tapes or TFTP over a > network interface. > > The function has the following type: > > typedef int dumper_t( > void *private, /* Private to the driver. */ > void *virtual, /* Virtual (mapped) address. */ > void *physical, /* Physical address of virtual. */ > off_t length); /* Number of bytes to dump. */ The type of `physical' is bogus. It should be vm_offset_t. I'm not sure if physical addresses are necessary or good here. It is only useful if the memory region is contiguous. But physical addresses can be recovered easily from virtual addresses even if the memory is discontiguous. The memory should just be contiguous to permit dump routines to be as simple as possible. The type of `length' is bogus. It should be size_t or vm_size_t. off_t is for lengths of files, not for sizes of memory. dumpstatus() has the same bogon. > To register a dumping function, the driver fills out a data > structure and calls set_dumper(): > > struct dumperinfo { > dumper_t *dumper; /* Dumping function. */ > void *private; /* Private parts. */ > u_int blocksize; /* Size of block in bytes. */ > off_t mediasize; /* Space available in bytes. */ > }; > > int set_dumper(struct dumperinfo *); > > Only one dumping function can be registered at a time, the current > registration can be cleared by calling set_dumper(NULL); Not necessary or good. The current d_dump entry point is sufficient, provided its type is changed to something like that above and its semantics is changed to always do a complete intialization and finalization for each memory region. The dump function needs to be told where to write the dump (for each memory region) so that all dump functions doesn't have to know about layers like slices and partitions. You would need something like the above to handle non-disks, but I think it should not be designed before we have experience with non-disk dump routines. Dumps to tape probably shouldn't do a finalization for each memory region... It's probably right for dump routines for non-disks to make their devices look like disks (accept memory regions that are blocked suitably for disks, and add headers and padding if necessary to compensate for unusual block sizes). > The configuration mechanism for disks will change so that dumpon(8) > will open the device and issue an ioctl to turn dumping on this device > on/off, rather than the current sysctl based mechanism. sysctls are wrong for almost everything :-). Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Mar 30 20:50:40 2002 Delivered-To: freebsd-arch@freebsd.org Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by hub.freebsd.org (Postfix) with ESMTP id 6EFF437B417 for ; Sat, 30 Mar 2002 20:50:37 -0800 (PST) Received: by elvis.mu.org (Postfix, from userid 1192) id 4A33CAE03F; Sat, 30 Mar 2002 20:50:37 -0800 (PST) Date: Sat, 30 Mar 2002 20:50:37 -0800 From: Alfred Perlstein To: Garance A Drosihn Cc: Kirk McKusick , arch@FreeBSD.ORG Subject: Re: UFS snapshots in current Message-ID: <20020331045037.GE93885@elvis.mu.org> References: <200203282250.g2SMoDD99826@beastie.mckusick.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.3.27i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Garance A Drosihn [020330 20:01] wrote: > At 2:50 PM -0800 3/28/02, Kirk McKusick wrote: > > From: Garance A Drosihn > > > > More useful question: what should I look at for > > info on using snapshots? > > > >General references are found at: > > > > http://www.mckusick.com/softdep/index.html > > > >The soft updates paper has a section on snapshots. The > >background fsck paper goes into snapshots (and their > >general usage) in a bit more detail, so is likely to > >be more useful. > > Okay, well, I was trying this out and I had something > odd happen. As I sit here waiting for my PC to return > to life, I'll ask if what I was trying to do something > which would be a BadIdea(tm). [snip] Looks like you hit one of the snapshot deadlock conditions, Dr McKusick recently introduced a fix for one of the deadlocks so this may not happen again... -Alfred To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Mar 30 22:50:21 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mail.rpi.edu (mail.rpi.edu [128.113.22.40]) by hub.freebsd.org (Postfix) with ESMTP id 9C7A237B423 for ; Sat, 30 Mar 2002 22:50:13 -0800 (PST) Received: from [128.113.24.47] (gilead.acs.rpi.edu [128.113.24.47]) by mail.rpi.edu (8.12.1/8.12.1) with ESMTP id g2V6oA4t015578; Sun, 31 Mar 2002 01:50:10 -0500 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: <20020331045037.GE93885@elvis.mu.org> References: <200203282250.g2SMoDD99826@beastie.mckusick.com> <20020331045037.GE93885@elvis.mu.org> Date: Sun, 31 Mar 2002 01:50:10 -0500 To: Alfred Perlstein From: Garance A Drosihn Subject: Re: UFS snapshots in current Cc: Kirk McKusick , arch@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Scanned-By: MIMEDefang 2.3 (www dot roaringpenguin dot com slash mimedefang) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG At 8:50 PM -0800 3/30/02, Alfred Perlstein wrote: >Looks like you hit one of the snapshot deadlock conditions, >Dr McKusick recently introduced a fix for one of the >deadlocks so this may not happen again... In the past week? The "old" current system which I had been running should have been from March 26th, when I rebuilt it to see if it would fix my vmware problems (which it did...). I don't see any commits wrt softupdates which are that recent. -- Garance Alistair Drosehn = gad@eclipse.acs.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message