From owner-freebsd-arch Sun Oct 7 9:56:59 2001 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id 5E6C437B405 for ; Sun, 7 Oct 2001 09:56:55 -0700 (PDT) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.6/8.11.6) with ESMTP id f97GuoB01361; Sun, 7 Oct 2001 18:56:51 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Brian Somers Cc: freebsd-arch@FreeBSD.ORG Subject: Re: Cloned open support In-Reply-To: Your message of "Mon, 29 Jan 2001 10:52:31 GMT." <200101291052.f0TAqVK58112@storm.FreeBSD.org.uk> Date: Sun, 07 Oct 2001 18:56:50 +0200 Message-ID: <1359.1002473810@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Uhm, Brian... We have cloned devices already... What exactly is it that you want to implement ? Poul-Henning In message <200101291052.f0TAqVK58112@storm.FreeBSD.org.uk>, Brian Somers write s: >Hi, > >I'm considering having a crack at adding support for cloned opens. >Before I start, I guess I've got two questions: > >1. Is anybody else doing this. > >2. Given that I'll need to change the d_open prototype (and > thus cdevsw), I'm going to affect every device driver in the > tree - although only changing the first arg to a (dev_t) > pointer, making things pretty easy to change and not too > unexpected for anyone that's written sysv drivers. > > Is it acceptable to simply not support old drivers ? > > I've considered trying to do things so that they're backwards > compatible, but I think the answer is that it'd be too much > of a crutch in the future. > >Ok, ok, three questions ! > >3. Should I ask for review anywhere else besides here ? > >Ta. > >-- >Brian > >Don't _EVER_ lose your sense of humour ! > > >To Unsubscribe: send mail to majordomo@FreeBSD.org >with "unsubscribe freebsd-arch" in the body of the message > -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 7 13:47:24 2001 Delivered-To: freebsd-arch@freebsd.org Received: from gull.mail.pas.earthlink.net (gull.mail.pas.earthlink.net [207.217.121.85]) by hub.freebsd.org (Postfix) with ESMTP id CE2FB37B403 for ; Sun, 7 Oct 2001 13:47:22 -0700 (PDT) Received: from mindspring.com (dialup-209.247.136.187.Dial1.SanJose1.Level3.net [209.247.136.187]) by gull.mail.pas.earthlink.net (EL-8_9_3_3/8.9.3) with ESMTP id NAA11369; Sun, 7 Oct 2001 13:46:47 -0700 (PDT) Message-ID: <3BC0BF69.1E231F9D@mindspring.com> Date: Sun, 07 Oct 2001 13:47:37 -0700 From: Terry Lambert Reply-To: tlambert2@mindspring.com X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Poul-Henning Kamp Cc: Brian Somers , freebsd-arch@FreeBSD.ORG Subject: Re: Cloned open support References: <1359.1002473810@critter.freebsd.dk> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Poul-Henning Kamp wrote: > > Uhm, Brian... > > We have cloned devices already... > > What exactly is it that you want to implement ? Probably per open instance data; he was working with the vmware2 code, wasn't he? Or was that someone else? Per open instance data is needed to support multiple instances of vmware running simultaneously, since it does not know to ask for sequentially increasing device names. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 7 14:27:48 2001 Delivered-To: freebsd-arch@freebsd.org Received: from finch-post-10.mail.demon.net (finch-post-10.mail.demon.net [194.217.242.38]) by hub.freebsd.org (Postfix) with ESMTP id A063B37B401 for ; Sun, 7 Oct 2001 14:27:45 -0700 (PDT) Received: from [62.49.251.130] (helo=herring.nlsystems.com) by finch-post-10.mail.demon.net with esmtp (Exim 2.12 #1) id 15qLRt-000AtI-0A; Sun, 7 Oct 2001 21:27:01 +0000 Received: from salmon.nlsystems.com (salmon [10.0.0.3]) by herring.nlsystems.com (8.11.2/8.11.2) with ESMTP id f97KuR790573; Sun, 7 Oct 2001 21:56:27 +0100 (BST) (envelope-from dfr@nlsystems.com) Date: Sun, 7 Oct 2001 21:55:03 +0100 (BST) From: Doug Rabson To: Terry Lambert Cc: Poul-Henning Kamp , Brian Somers , Subject: Re: Cloned open support In-Reply-To: <3BC0BF69.1E231F9D@mindspring.com> Message-ID: <20011007215318.T530-100000@salmon.nlsystems.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, 7 Oct 2001, Terry Lambert wrote: > Poul-Henning Kamp wrote: > > > > Uhm, Brian... > > > > We have cloned devices already... > > > > What exactly is it that you want to implement ? > > Probably per open instance data; he was working with the > vmware2 code, wasn't he? Or was that someone else? > > Per open instance data is needed to support multiple > instances of vmware running simultaneously, since it > does not know to ask for sequentially increasing device > names. This is already possible by substituting a new 'struct file' for the device file. See streamsopn() in dev/streams/streams.c. -- Doug Rabson Mail: dfr@nlsystems.com Phone: +44 20 8348 6160 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 7 15:17:13 2001 Delivered-To: freebsd-arch@freebsd.org Received: from peter3.wemm.org (c1315225-a.plstn1.sfba.home.com [24.14.150.180]) by hub.freebsd.org (Postfix) with ESMTP id 7582637B407 for ; Sun, 7 Oct 2001 15:17:06 -0700 (PDT) Received: from overcee.netplex.com.au (overcee.wemm.org [10.0.0.3]) by peter3.wemm.org (8.11.0/8.11.0) with ESMTP id f97MH6M70796 for ; Sun, 7 Oct 2001 15:17:06 -0700 (PDT) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by overcee.netplex.com.au (Postfix) with ESMTP id 1ABAE3809; Sun, 7 Oct 2001 15:17:06 -0700 (PDT) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 To: Poul-Henning Kamp Cc: Brian Somers , freebsd-arch@FreeBSD.ORG Subject: Re: Cloned open support In-Reply-To: <1359.1002473810@critter.freebsd.dk> Date: Sun, 07 Oct 2001 15:17:05 -0700 From: Peter Wemm Message-Id: <20011007221706.1ABAE3809@overcee.netplex.com.au> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Poul-Henning Kamp wrote: > > Uhm, Brian... > > We have cloned devices already... > > What exactly is it that you want to implement ? > > Poul-Henning Devfs name cloning feels hackish to me. Having a seperate EVENTHANDLER() for doing it feels .. just nasty. I'd much rather that we had a d_clone devsw entry and/or a D_CLONE d_flags entry. There are two types of cloning. One is to map some name "/dev/fd0135ds18h" into a device node without having to flood /dev with all possible permutations. The other is to support per-device "select next unit" style opens. Presently these are both kludged into the EVENTHANDLER interface. I think Brian wants to move the second part directly into the open handler like it is done on most other OS's that support cloning. Personally, I would be quite happy if we could do that. I realize why it is done the way it is done now though. VOP_LOOKUP() having to return a unique vnode for the device is a pain. (which is why the clone is done during lookup, so that the correct vnode is found and available). But understanding why doesn't mean that I dont wish that it could be different. :-) Doug mentions the hack in dev/streams/streams.c: td->td_dupfd = fd; return ENXIO; .. this is nasty. :-) I think the SVR4 clone driver uses something like this. It causes the original namei / open attempt to fail (thus releasing the "common" vnode) and then switching over to the *real* file/vnode at the last minute. Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Oct 7 22:43:49 2001 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id 00D0237B405 for ; Sun, 7 Oct 2001 22:43:44 -0700 (PDT) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.6/8.11.6) with ESMTP id f985hBO95928; Mon, 8 Oct 2001 07:43:37 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Peter Wemm Cc: Brian Somers , freebsd-arch@FreeBSD.ORG Subject: Re: Cloned open support In-Reply-To: Your message of "Sun, 07 Oct 2001 15:17:05 PDT." <20011007221706.1ABAE3809@overcee.netplex.com.au> Date: Mon, 08 Oct 2001 07:43:11 +0200 Message-ID: <95926.1002519791@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20011007221706.1ABAE3809@overcee.netplex.com.au>, Peter Wemm writes : >Poul-Henning Kamp wrote: >> >> Uhm, Brian... >> >> We have cloned devices already... >> >> What exactly is it that you want to implement ? >> >> Poul-Henning > >Devfs name cloning feels hackish to me. Having a seperate EVENTHANDLER() >for doing it feels .. just nasty. I'd much rather that we had a d_clone >devsw entry and/or a D_CLONE d_flags entry. Call it hackish, I call it elegant: * I didn't have to modify all the device drivers making them incompatible with anything anybody ever learned. * I didn't have to do long-haired vnode operations in cloning drivers, thus preserving the ability to do systematic SMP lock-boundaries at the cdevsw-> level. * It supports parameterized clone opens (ie: not just "/dev/pty", but also "/dev/ad0s1g" and even if somebody implemented it: "/dev/ccd,mirror,ad0s1f,ad1s1f" :-) * It is a *LOT* simpler than doing it by vnodes... The story is that by the time you reach devsw->open() you have committed the vnode and if you change the device at that time you need to unwind all the way back up to the association of the vnode with the dev_t, and wind all the way back down before the open can progress. (The fact that it is an EVENTHANDLER is just a matter of implementation, I didn't see a point to reimplementing the same functionality when EVENTHANDLERS already were available). >There are two types of cloning. One is to map some name "/dev/fd0135ds18h" >into a device node without having to flood /dev with all possible >permutations. The other is to support per-device "select next unit" style >opens. Presently these are both kludged into the EVENTHANDLER interface. Those two are actually the same kind of open Peter, semantically they both say "make me a device according to this wish: ``...'' and let me open it". >I think Brian wants to move the second part directly into the open handler >like it is done on most other OS's that support cloning. Personally, >I would be quite happy if we could do that. Most other devices have made a mess of their vnodes and drivers by doing so :-( The FreeBSD implementation completely sidesteps all the vnode hair by doing the cloning at namei() time instead of open time, this makes it much simpler and much more capable. If you do a vnode based cloning, it will not support your "/dev/fd0135ds18h" example above, unless you flood /dev with all possible entries. >I realize why it is done the way it is done now though. VOP_LOOKUP() >having to return a unique vnode for the device is a pain. (which is why >the clone is done during lookup, so that the correct vnode is found and >available). But understanding why doesn't mean that I dont wish that it >could be different. :-) Well, if devices lived at the filedescriptor level instead of at the vnode level, things would be different (but I havn't tried to implement that so I can't say for sure if it would actually be "better"...) >Doug mentions the hack in dev/streams/streams.c: > td->td_dupfd = fd; > return ENXIO; >.. this is nasty. :-) This is abuse, it should be rewritten. >I think the SVR4 clone driver uses something like this. It causes the >original namei / open attempt to fail (thus releasing the "common" vnode) >and then switching over to the *real* file/vnode at the last minute. We would have to do that as well in order to unwind the committed vnode and select another. I would like to request that nobody starts to commit a vnode based cloning (or API changes for it) until they actually have a working prototype. I've been there, done that and threw it away. The only reason I can see for adding vnode-based cloning would be if somebody can point out something they cannot do with namei-based cloning... My BSDCONey and BSDCON talks would be very good places to ask questions about this :-) Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Oct 8 1:22:34 2001 Delivered-To: freebsd-arch@freebsd.org Received: from anchor-post-34.mail.demon.net (anchor-post-34.mail.demon.net [194.217.242.92]) by hub.freebsd.org (Postfix) with ESMTP id CA14037B401 for ; Mon, 8 Oct 2001 01:22:31 -0700 (PDT) Received: from [62.49.251.130] (helo=herring.nlsystems.com) by anchor-post-34.mail.demon.net with esmtp (Exim 2.12 #1) id 15qVgD-000Pjy-0Y; Mon, 8 Oct 2001 09:22:30 +0100 Received: from salmon.nlsystems.com (salmon [10.0.0.3]) by herring.nlsystems.com (8.11.2/8.11.2) with ESMTP id f988LE717041; Mon, 8 Oct 2001 09:21:14 +0100 (BST) (envelope-from dfr@nlsystems.com) Date: Mon, 8 Oct 2001 09:19:49 +0100 (BST) From: Doug Rabson To: Poul-Henning Kamp Cc: Peter Wemm , Brian Somers , Subject: Re: Cloned open support In-Reply-To: <95926.1002519791@critter.freebsd.dk> Message-ID: <20011008091835.X530-100000@salmon.nlsystems.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, 8 Oct 2001, Poul-Henning Kamp wrote: > >Doug mentions the hack in dev/streams/streams.c: > > td->td_dupfd = fd; > > return ENXIO; > >.. this is nasty. :-) > > This is abuse, it should be rewritten. > > >I think the SVR4 clone driver uses something like this. It causes the > >original namei / open attempt to fail (thus releasing the "common" vnode) > >and then switching over to the *real* file/vnode at the last minute. > > We would have to do that as well in order to unwind the committed vnode > and select another. It might be abuse but its the only way that I know of to keep track of data per-open-file. -- Doug Rabson Mail: dfr@nlsystems.com Phone: +44 20 8348 6160 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Oct 8 1:33:53 2001 Delivered-To: freebsd-arch@freebsd.org Received: from Awfulhak.org (gw.Awfulhak.org [217.204.245.18]) by hub.freebsd.org (Postfix) with ESMTP id 4DBFE37B403 for ; Mon, 8 Oct 2001 01:33:43 -0700 (PDT) Received: from hak.lan.Awfulhak.org (root@hak.lan.Awfulhak.org [172.16.0.12]) by Awfulhak.org (8.11.6/8.11.6) with ESMTP id f988XbJ20011; Mon, 8 Oct 2001 09:33:37 +0100 (BST) (envelope-from brian@freebsd-services.com) Received: from hak.lan.Awfulhak.org (brian@localhost [127.0.0.1]) by hak.lan.Awfulhak.org (8.11.6/8.11.6) with ESMTP id f988UxT40831; Mon, 8 Oct 2001 09:30:59 +0100 (BST) (envelope-from brian@freebsd-services.com) Message-Id: <200110080830.f988UxT40831@hak.lan.Awfulhak.org> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: Poul-Henning Kamp Cc: Peter Wemm , Brian Somers , freebsd-arch@FreeBSD.ORG, brian@freebsd-services.com Subject: Re: Cloned open support In-Reply-To: Message from Poul-Henning Kamp of "Mon, 08 Oct 2001 07:43:11 +0200." <95926.1002519791@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 08 Oct 2001 09:30:59 +0100 From: Brian Somers Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi, First, let me mention that I think your response is a bit odd.... timing wise. I posted the message that you've responded to on January 29 and you responded on October 7. I implemented cloning on if_tun in May... :*P Maybe my mail server is under performing ! > In message <20011007221706.1ABAE3809@overcee.netplex.com.au>, Peter Wemm writes > : > >Poul-Henning Kamp wrote: > >> > >> Uhm, Brian... > >> > >> We have cloned devices already... > >> > >> What exactly is it that you want to implement ? > >> > >> Poul-Henning > > > >Devfs name cloning feels hackish to me. Having a seperate EVENTHANDLER() > >for doing it feels .. just nasty. I'd much rather that we had a d_clone > >devsw entry and/or a D_CLONE d_flags entry. > > Call it hackish, I call it elegant: > > * I didn't have to modify all the device drivers making them > incompatible with anything anybody ever learned. > > * I didn't have to do long-haired vnode operations in cloning > drivers, thus preserving the ability to do systematic SMP > lock-boundaries at the cdevsw-> level. > > * It supports parameterized clone opens (ie: not just "/dev/pty", > but also "/dev/ad0s1g" and even if somebody implemented it: > "/dev/ccd,mirror,ad0s1f,ad1s1f" :-) > > * It is a *LOT* simpler than doing it by vnodes... > > The story is that by the time you reach devsw->open() you have > committed the vnode and if you change the device at that time you > need to unwind all the way back up to the association of the vnode > with the dev_t, and wind all the way back down before the open can > progress. > > (The fact that it is an EVENTHANDLER is just a matter of implementation, > I didn't see a point to reimplementing the same functionality when > EVENTHANDLERS already were available). > > >There are two types of cloning. One is to map some name "/dev/fd0135ds18h" > >into a device node without having to flood /dev with all possible > >permutations. The other is to support per-device "select next unit" style > >opens. Presently these are both kludged into the EVENTHANDLER interface. > > Those two are actually the same kind of open Peter, semantically > they both say "make me a device according to this wish: ``...'' and let > me open it". > > >I think Brian wants to move the second part directly into the open handler > >like it is done on most other OS's that support cloning. Personally, > >I would be quite happy if we could do that. > > Most other devices have made a mess of their vnodes and drivers by > doing so :-( > > The FreeBSD implementation completely sidesteps all the vnode hair > by doing the cloning at namei() time instead of open time, this > makes it much simpler and much more capable. > > If you do a vnode based cloning, it will not support your > "/dev/fd0135ds18h" example above, unless you flood /dev with all > possible entries. > > >I realize why it is done the way it is done now though. VOP_LOOKUP() > >having to return a unique vnode for the device is a pain. (which is why > >the clone is done during lookup, so that the correct vnode is found and > >available). But understanding why doesn't mean that I dont wish that it > >could be different. :-) > > Well, if devices lived at the filedescriptor level instead of at the vnode > level, things would be different (but I havn't tried to implement that > so I can't say for sure if it would actually be "better"...) > > >Doug mentions the hack in dev/streams/streams.c: > > td->td_dupfd = fd; > > return ENXIO; > >.. this is nasty. :-) > > This is abuse, it should be rewritten. > > >I think the SVR4 clone driver uses something like this. It causes the > >original namei / open attempt to fail (thus releasing the "common" vnode) > >and then switching over to the *real* file/vnode at the last minute. > > We would have to do that as well in order to unwind the committed vnode > and select another. > > > I would like to request that nobody starts to commit a vnode based cloning > (or API changes for it) until they actually have a working prototype. > I've been there, done that and threw it away. > > The only reason I can see for adding vnode-based cloning would be if > somebody can point out something they cannot do with namei-based > cloning... > > > My BSDCONey and BSDCON talks would be very good places to ask questions > about this :-) > My feeling on the whole topic is that we now have a very workable system with two drawbacks: o The ``clone device'' doesn't turn up in /dev. This means that an administrator cannot treat it as a filesystem object WRT permissions - in fact, he can't even see it on the filesystem. IMHO this causes namespace problems, but this is also quite fixable. I'd like to talk about this at BSDConEurope. o The SI_CHEAPCLONE stuff is easy to get wrong, and getting it wrong opens up a bad DoS. Maybe the answer is that specinfos that are returned from make_dev() during clone() have the SI_CHEAPCLONE flag already set and a successful call to the driver's d_open() clears SI_CHEAPCLONE ? But this doesn't quite work with the tun device. The tun device abuses this flag so that it can use dev_depends() to blow away all of it's make_dev()s at module unload time.... It doesn't want to destroy_dev() them at d_close() time because I'd prefer that the administrator is able to ``touch /dev/tunX'' then ``chmod /dev/tunX'' at boot time. A partially unrelated problem is that of tracking open devices from inside a driver. I'm only mentioning this because the SI_CHEAPCLONE flag makes this more difficult - it allows devfs to destroy_dev() things when the driver isn't looking.... I don't think my SI_CHEAPCLONE abuse in if_tun is correct. Maybe the right answer is to have devfs notify the driver when it destroy_dev()s something ? > Poul-Henning > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. -- Brian http://www.freebsd-services.com/ Don't _EVER_ lose your sense of humour ! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Oct 8 3:54:27 2001 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id C9C8C37B401 for ; Mon, 8 Oct 2001 03:54:24 -0700 (PDT) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.6/8.11.6) with ESMTP id f98AqG159661; Mon, 8 Oct 2001 12:52:21 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Doug Rabson Cc: Peter Wemm , Brian Somers , freebsd-arch@FreeBSD.ORG Subject: Re: Cloned open support In-Reply-To: Your message of "Mon, 08 Oct 2001 09:19:49 BST." <20011008091835.X530-100000@salmon.nlsystems.com> Date: Mon, 08 Oct 2001 12:52:16 +0200 Message-ID: <59657.1002538336@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20011008091835.X530-100000@salmon.nlsystems.com>, Doug Rabson write s: >On Mon, 8 Oct 2001, Poul-Henning Kamp wrote: > >> >Doug mentions the hack in dev/streams/streams.c: >> > td->td_dupfd = fd; >> > return ENXIO; >> >.. this is nasty. :-) >> >> This is abuse, it should be rewritten. >> >> >I think the SVR4 clone driver uses something like this. It causes the >> >original namei / open attempt to fail (thus releasing the "common" vnode) >> >and then switching over to the *real* file/vnode at the last minute. >> >> We would have to do that as well in order to unwind the committed vnode >> and select another. > >It might be abuse but its the only way that I know of to keep track of >data per-open-file. I looked at it closer and actually that is a very interesting case: you open a device and return a socket. I don't think this can be done any other way than it is done now. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Oct 8 3:54:35 2001 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by hub.freebsd.org (Postfix) with ESMTP id B242837B401 for ; Mon, 8 Oct 2001 03:54:32 -0700 (PDT) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.6/8.11.6) with ESMTP id f98AsI160001; Mon, 8 Oct 2001 12:54:24 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Brian Somers Cc: Peter Wemm , Brian Somers , freebsd-arch@FreeBSD.ORG Subject: Re: Cloned open support In-Reply-To: Your message of "Mon, 08 Oct 2001 09:30:59 BST." <200110080830.f988UxT40831@hak.lan.Awfulhak.org> Date: Mon, 08 Oct 2001 12:54:18 +0200 Message-ID: <59999.1002538458@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <200110080830.f988UxT40831@hak.lan.Awfulhak.org>, Brian Somers write s: >Hi, > >First, let me mention that I think your response is a bit odd.... >timing wise. I posted the message that you've responded to on >January 29 and you responded on October 7. I implemented cloning on >if_tun in May... :*P > >Maybe my mail server is under performing ! Actually I think my mailer made a fool of me... Sorry... :-( -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 9 5:42:59 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 02AF637B401 for ; Tue, 9 Oct 2001 05:42:55 -0700 (PDT) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id WAA04571; Tue, 9 Oct 2001 22:42:42 +1000 Date: Tue, 9 Oct 2001 22:41:59 +1000 (EST) From: Bruce Evans X-X-Sender: To: Dag-Erling Smorgrav Cc: Peter Wemm , Subject: Re: Removing ptrace(2)'s dependency on procfs(5) In-Reply-To: Message-ID: <20011009221951.B25192-100000@delplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 6 Oct 2001, Dag-Erling Smorgrav wrote: > BTW, I'm intrigued by the following bit of code in kern_sig.c: > > do { > mtx_lock_spin(&sched_lock); > stop(p); > PROC_UNLOCK_NOSWITCH(p); > DROP_GIANT_NOSWITCH(); > p->p_stats->p_ru.ru_nivcsw++; > mi_switch(); > mtx_unlock_spin(&sched_lock); > PICKUP_GIANT(); > PROC_LOCK(p); > } while (!trace_req(p) > && p->p_flag & P_TRACED); > > It's the only reference to trace_req() (defined in sys_process.c) in > the entire tree, and trace_req() has been a constant since revision > 1.1 of sys_process.c, and true since revision 1.4 (1994/08/08), so > this loop will always execute exactly once. Is there a good reason > not to simply axe trace_req() and unroll the loop? The correctness of not looping depends on nothing except the parent making the process runnable (unless the process loses its P_TRACED attribute). This is apparently what happens, since things have worked for so long. But I don't completely understand what happens when a signal is sent to the process. The comment before this section of code suggests looping. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 9 6:27:57 2001 Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id CADA537B409 for ; Tue, 9 Oct 2001 06:27:53 -0700 (PDT) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id PAA10775; Tue, 9 Oct 2001 15:27:43 +0200 (CEST) (envelope-from des@ofug.org) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Bruce Evans Cc: Peter Wemm , Subject: Re: Removing ptrace(2)'s dependency on procfs(5) References: <20011009221951.B25192-100000@delplex.bde.org> From: Dag-Erling Smorgrav Date: 09 Oct 2001 15:27:42 +0200 In-Reply-To: <20011009221951.B25192-100000@delplex.bde.org> Message-ID: Lines: 16 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/20.7 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Bruce Evans writes: > The correctness of not looping depends on nothing except the parent > making the process runnable (unless the process loses its P_TRACED > attribute). This is apparently what happens, since things have worked > for so long. But I don't completely understand what happens when a > signal is sent to the process. The comment before this section of > code suggests looping. ...but the code will never loop, because trace_req() always returns 1. What intrigues me further is that trace_req() used to always return 0, and the log entry for the commit that changed it does not explain why it changed to returning 1. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 9 11:30: 6 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail5.speakeasy.net (mail5.speakeasy.net [216.254.0.205]) by hub.freebsd.org (Postfix) with ESMTP id 2FCA737B40B for ; Tue, 9 Oct 2001 11:29:57 -0700 (PDT) Received: (qmail 92190 invoked from network); 9 Oct 2001 18:29:56 -0000 Received: from unknown (HELO laptop.baldwin.cx) ([64.81.54.73]) (envelope-sender ) by mail5.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 9 Oct 2001 18:29:56 -0000 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 Date: Tue, 09 Oct 2001 11:29:23 -0700 (PDT) From: John Baldwin To: arch@FreeBSD.org Subject: ucred API Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hey all. I'd like to make the following changes to the ucred API in the interests of making the locking more sane. The changes will occur in 2 stages. Stage 1: - Add a new function crshared(cr) that returns true if ucred cr's refcount is greater than 1. - Change crhold(cr) to return a pointer to the ucred it bumped the reference of. Thus, the following code: cr = foo->f_cred; crhold(cr); becomes: cr = crhold(foo->f_cred); (suggested by Terry L.) - Change the crcopy() function to actually copy the contents of one ucred to another. This is much more lock friendly as having crcopy() block on malloc introduces locking problems. Before code: PROC_LOCK(p); cred = p->p_ucred); PROC_UNLOCK(p); /* XXX: race */ p->p_ucred = crcopy(cred); /* XXX: can block! */ PROC_LOCK(p); p->p_ucred = cred; ... PROC_UNLOCK(p); New code: newcred = crget(); PROC_LOCK(p); oldcred = p->p_ucred; error = suser(oldcred); if (error == 0) { crcopy(newcred, oldcred); .... p->p_ucred = newcred; newcred = oldcred; } PROC_UNLOCK(p); crfree(newcred); Which doesn't block during the critical section and can make all the access checks and changes as one "atomic" change. Stage 2: - Add a per-thread reference to the ucred that is created on syscall entry and released on syscall exit. It is also created and released if needed on trap enter/exit. It is _not_ created for interrupts since interrupts should not care about the ucred of their borrowed context. The per-thread ucred reference will then point to a ucred that won't ever change (setuid, etc. update the per-process ucred) and thus won't need any locking. Almost all references to ucreds for suser(), VOP's etc. will use the thread reference. This will ensure that a thread's ucred will be the same for an entire syscall which will close many races involving multithreaded programs and ucreds. The only place where the per-process ucred will be used for access checks is places that modify the ucred as we want to ensure there is no race of one thread making a credential change it isn't qualified to make due to it performing its access checks on a stale ucred before updating the master ucred. I've talked with Robert Watson about these already and they sound good to him. Any objections? -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 9 12:11:49 2001 Delivered-To: freebsd-arch@freebsd.org Received: from InterJet.elischer.org (c421509-a.pinol1.sfba.home.com [24.7.86.9]) by hub.freebsd.org (Postfix) with ESMTP id 2637E37B401; Tue, 9 Oct 2001 12:11:45 -0700 (PDT) Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id NAA27750; Tue, 9 Oct 2001 13:05:13 -0700 (PDT) Date: Tue, 9 Oct 2001 13:05:11 -0700 (PDT) From: Julian Elischer To: John Baldwin Cc: arch@FreeBSD.org Subject: Re: ucred API In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, 9 Oct 2001, John Baldwin wrote: > > newcred = crget(); > PROC_LOCK(p); > oldcred = p->p_ucred; > error = suser(oldcred); is suser being changed to take a cred? > if (error == 0) { > crcopy(newcred, oldcred); > > Stage 2: > - Add a per-thread reference to the ucred that is created on syscall > entry and released on syscall exit. It is also created and released > if needed on trap enter/exit. It is _not_ created for interrupts since > interrupts should not care about the ucred of their borrowed context. > The per-thread ucred reference will then point to a ucred that won't > ever change (setuid, etc. update the per-process ucred) and thus won't > need any locking. Almost all references to ucreds for suser(), VOP's > etc. will use the thread reference. This will ensure that a thread's > ucred will be the same for an entire syscall which will close many > races involving multithreaded programs and ucreds. The only place where > the per-process ucred will be used for access checks is places that > modify the ucred as we want to ensure there is no race of one thread > making a credential change it isn't qualified to make due to it performing > its access checks on a stale ucred before updating the master ucred. I assume that you say "It won't change" because if the process's cred is changed, it gets a new one and the thread's pointer still points to the old one? (ref counted) > > I've talked with Robert Watson about these already and they sound good to him. > Any objections? > no, sounds fine! > -- > > John Baldwin -- http://www.FreeBSD.org/~jhb/ > PGP Key: http://www.baldwin.cx/~john/pgpkey.asc > "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 9 12:26: 8 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail5.speakeasy.net (mail5.speakeasy.net [216.254.0.205]) by hub.freebsd.org (Postfix) with ESMTP id 23B5E37B407 for ; Tue, 9 Oct 2001 12:25:57 -0700 (PDT) Received: (qmail 34570 invoked from network); 9 Oct 2001 19:25:56 -0000 Received: from unknown (HELO laptop.baldwin.cx) ([64.81.54.73]) (envelope-sender ) by mail5.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 9 Oct 2001 19:25:56 -0000 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Tue, 09 Oct 2001 12:25:22 -0700 (PDT) From: John Baldwin To: Julian Elischer Subject: Re: ucred API Cc: arch@FreeBSD.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 09-Oct-01 Julian Elischer wrote: > > > On Tue, 9 Oct 2001, John Baldwin wrote: > >> >> newcred = crget(); >> PROC_LOCK(p); >> oldcred = p->p_ucred; >> error = suser(oldcred); > > is suser being changed to take a cred? Not now, but I think that is on Robert's plans for the future. That should be suser(p) I suppose. suser() really has nothing to do with threads or processes but only ucred's anyways. >> if (error == 0) { >> crcopy(newcred, oldcred); >> >> Stage 2: >> - Add a per-thread reference to the ucred that is created on syscall >> entry and released on syscall exit. It is also created and released >> if needed on trap enter/exit. It is _not_ created for interrupts >> since >> interrupts should not care about the ucred of their borrowed context. >> The per-thread ucred reference will then point to a ucred that won't >> ever change (setuid, etc. update the per-process ucred) and thus won't >> need any locking. Almost all references to ucreds for suser(), VOP's >> etc. will use the thread reference. This will ensure that a thread's >> ucred will be the same for an entire syscall which will close many >> races involving multithreaded programs and ucreds. The only place >> where >> the per-process ucred will be used for access checks is places that >> modify the ucred as we want to ensure there is no race of one thread >> making a credential change it isn't qualified to make due to it >> performing >> its access checks on a stale ucred before updating the master ucred. > > I assume that you say "It won't change" because if the process's cred > is changed, it gets a new one and the thread's pointer > still points to the old one? (ref counted) Right. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 9 13: 3:42 2001 Delivered-To: freebsd-arch@freebsd.org Received: from elvis.mu.org (elvis.mu.org [216.33.66.196]) by hub.freebsd.org (Postfix) with ESMTP id 9C91537B401; Tue, 9 Oct 2001 13:03:39 -0700 (PDT) Received: by elvis.mu.org (Postfix, from userid 1192) id A0B8C81D01; Tue, 9 Oct 2001 15:03:39 -0500 (CDT) Date: Tue, 9 Oct 2001 15:03:39 -0500 From: Alfred Perlstein To: John Baldwin Cc: arch@FreeBSD.org Subject: Re: ucred API Message-ID: <20011009150339.X59854@elvis.mu.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from jhb@FreeBSD.org on Tue, Oct 09, 2001 at 11:29:23AM -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * John Baldwin [011009 13:30] wrote: > Hey all. I'd like to make the following changes to the ucred API in the > interests of making the locking more sane. The changes will occur in 2 stages. > Stage 2: > - Add a per-thread reference to the ucred that is created on syscall > entry and released on syscall exit. It is also created and released > if needed on trap enter/exit. It is _not_ created for interrupts since > interrupts should not care about the ucred of their borrowed context. > The per-thread ucred reference will then point to a ucred that won't > ever change (setuid, etc. update the per-process ucred) and thus won't > need any locking. Almost all references to ucreds for suser(), VOP's > etc. will use the thread reference. This will ensure that a thread's > ucred will be the same for an entire syscall which will close many > races involving multithreaded programs and ucreds. The only place where > the per-process ucred will be used for access checks is places that > modify the ucred as we want to ensure there is no race of one thread > making a credential change it isn't qualified to make due to it performing > its access checks on a stale ucred before updating the master ucred. > > I've talked with Robert Watson about these already and they sound good to him. > Any objections? Stage 2 is bogus. You only need to reference the cred when a thread is created. In terms of KSE, what I think that means when you'd block leaving a context (lazy thread creation) you'd do your dup. -- -Alfred Perlstein [alfred@freebsd.org] 'Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom.' To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 9 13:31: 0 2001 Delivered-To: freebsd-arch@freebsd.org Received: from InterJet.elischer.org (c421509-a.pinol1.sfba.home.com [24.7.86.9]) by hub.freebsd.org (Postfix) with ESMTP id 9726037B41C; Tue, 9 Oct 2001 13:30:54 -0700 (PDT) Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id OAA28090; Tue, 9 Oct 2001 14:33:07 -0700 (PDT) Date: Tue, 9 Oct 2001 14:33:06 -0700 (PDT) From: Julian Elischer To: Alfred Perlstein Cc: John Baldwin , arch@FreeBSD.org Subject: Re: ucred API In-Reply-To: <20011009150339.X59854@elvis.mu.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG errr, Alfred.. The kernel "thread" structure is only valid when the user thread "dips" into the kernel.. The more threads "dip" into the kernel (e.g. do syscalls,) the more kernel threads are created to run on their behalf. Thus strictly speaking kernel thread structures only last for the lifetime of a syscall. (in practice they are cached but theoretically they are freed and reallocated). Since there may be many syscalls outstanding, each works with the creds that existed at the moment when it was invoked. On Tue, 9 Oct 2001, Alfred Perlstein wrote: > * John Baldwin [011009 13:30] wrote: > > Hey all. I'd like to make the following changes to the ucred API in the > > interests of making the locking more sane. The changes will occur in 2 stages. > > Stage 2: > > - Add a per-thread reference to the ucred that is created on syscall > > entry and released on syscall exit. It is also created and released > > if needed on trap enter/exit. It is _not_ created for interrupts since > > interrupts should not care about the ucred of their borrowed context. > > The per-thread ucred reference will then point to a ucred that won't > > ever change (setuid, etc. update the per-process ucred) and thus won't > > need any locking. Almost all references to ucreds for suser(), VOP's > > etc. will use the thread reference. This will ensure that a thread's > > ucred will be the same for an entire syscall which will close many > > races involving multithreaded programs and ucreds. The only place where > > the per-process ucred will be used for access checks is places that > > modify the ucred as we want to ensure there is no race of one thread > > making a credential change it isn't qualified to make due to it performing > > its access checks on a stale ucred before updating the master ucred. > > > > I've talked with Robert Watson about these already and they sound good to him. > > Any objections? > > Stage 2 is bogus. > > You only need to reference the cred when a thread is created. > > In terms of KSE, what I think that means when you'd block leaving a > context (lazy thread creation) you'd do your dup. > > -- > -Alfred Perlstein [alfred@freebsd.org] > 'Instead of asking why a piece of software is using "1970s technology," > start asking why software is ignoring 30 years of accumulated wisdom.' > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 9 13:57: 6 2001 Delivered-To: freebsd-arch@freebsd.org Received: from elvis.mu.org (elvis.mu.org [216.33.66.196]) by hub.freebsd.org (Postfix) with ESMTP id 6E62937B40A; Tue, 9 Oct 2001 13:57:03 -0700 (PDT) Received: by elvis.mu.org (Postfix, from userid 1192) id 7792681D01; Tue, 9 Oct 2001 15:57:03 -0500 (CDT) Date: Tue, 9 Oct 2001 15:57:03 -0500 From: Alfred Perlstein To: Julian Elischer Cc: John Baldwin , arch@FreeBSD.org Subject: Re: ucred API Message-ID: <20011009155703.A59854@elvis.mu.org> References: <20011009150339.X59854@elvis.mu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from julian@elischer.org on Tue, Oct 09, 2001 at 02:33:06PM -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > On Tue, 9 Oct 2001, Alfred Perlstein wrote: > > You only need to reference the cred when a thread is created. > > > > In terms of KSE, what I think that means when you'd block leaving a > > context (lazy thread creation) you'd do your dup. * Julian Elischer [011009 15:30] wrote: > errr, Alfred.. > > The kernel "thread" structure is only valid when the user thread > "dips" into the kernel.. The more threads "dip" into the kernel > (e.g. do syscalls,) the more kernel threads are created to run on their > behalf. Thus strictly speaking kernel thread structures only last for the > lifetime of a syscall. (in practice they are cached but theoretically > they are freed and reallocated). > > Since there may be many syscalls outstanding, each works with the creds > that existed at the moment when it was invoked. So it's really keyed at thread creation time (as I stated), not at syscall time, don't forget that you may have multiple threads executing in userland to achieve SMP scalability. The importance of this is that there's one per thread, not one per syscall even though that may be an N-N relationship. -Alfred To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 9 14:14:53 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 521A937B401; Tue, 9 Oct 2001 14:14:49 -0700 (PDT) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.6/8.11.5) with SMTP id f99LEjB43265; Tue, 9 Oct 2001 17:14:45 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Tue, 9 Oct 2001 17:14:44 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Julian Elischer Cc: John Baldwin , arch@FreeBSD.org Subject: Re: ucred API In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, 9 Oct 2001, Julian Elischer wrote: > On Tue, 9 Oct 2001, John Baldwin wrote: > > > newcred = crget(); > > PROC_LOCK(p); > > oldcred = p->p_ucred; > > error = suser(oldcred); > > is suser being changed to take a cred? I currently plan to make this change unless there is any substantial objection. I, like John, have reams of outstanding patches relating to this code. :-) There used to be two reasons for suser to accept a proc pointer: (1) ASU flag set on process that checks for suser (bogus) (2) prison pointer in process (now in ucred) Since both of these have gone away, it would substantially simplify things to simply always select the credential before entering suser(), and leave it as: int suser(struct ucred *cred, int flags); I also have a switch from ucred -> cred in a local tree, as well as a complete capabilities implementation, and a fairly functional MAC implementation, plus gobs of other garbage that I'll probably trickle into the tree over the next few months. > I assume that you say "It won't change" because if the process's cred is > changed, it gets a new one and the thread's pointer still points to the > old one? (ref counted) After some thinking and talking with other UNIX vendors, it does seem to me that the most 'consistent' behavior can be accomplished by locking in the credential at the start of a system call, and maintaining it until the call completes. This is especially true for VFS calls. John's thread-based solution seems to encapsulate the associated concerns extremely well, and addresses my concerns about a variety of nasty races and non-determinism that might otherwise exist. Obviously, this is something that people should think through fairly hard, but I'm very happy with the solution (and if John drank beer, I'd buy him several -- this means I should buy him Soda at BSDCon, probably). Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 9 14:50:55 2001 Delivered-To: freebsd-arch@freebsd.org Received: from InterJet.elischer.org (c421509-a.pinol1.sfba.home.com [24.7.86.9]) by hub.freebsd.org (Postfix) with ESMTP id BE39837B411; Tue, 9 Oct 2001 14:50:52 -0700 (PDT) Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id PAA28437; Tue, 9 Oct 2001 15:57:31 -0700 (PDT) Date: Tue, 9 Oct 2001 15:57:30 -0700 (PDT) From: Julian Elischer To: Alfred Perlstein Cc: John Baldwin , arch@FreeBSD.org Subject: Re: ucred API In-Reply-To: <20011009155703.A59854@elvis.mu.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG it's one per syscall. because the thread structure may be allocated to another process should it require it immediatly after the current syscall finishes. On Tue, 9 Oct 2001, Alfred Perlstein wrote: > > On Tue, 9 Oct 2001, Alfred Perlstein wrote: > > > You only need to reference the cred when a thread is created. > > > > > > In terms of KSE, what I think that means when you'd block leaving a > > > context (lazy thread creation) you'd do your dup. > > > > * Julian Elischer [011009 15:30] wrote: > > errr, Alfred.. > > > > The kernel "thread" structure is only valid when the user thread > > "dips" into the kernel.. The more threads "dip" into the kernel > > (e.g. do syscalls,) the more kernel threads are created to run on their > > behalf. Thus strictly speaking kernel thread structures only last for the > > lifetime of a syscall. (in practice they are cached but theoretically > > they are freed and reallocated). > > > > Since there may be many syscalls outstanding, each works with the creds > > that existed at the moment when it was invoked. > > So it's really keyed at thread creation time (as I stated), not at > syscall time, don't forget that you may have multiple threads > executing in userland to achieve SMP scalability. > > The importance of this is that there's one per thread, not one > per syscall even though that may be an N-N relationship. > > -Alfred > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Oct 9 15:43:25 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail12.speakeasy.net (mail12.speakeasy.net [216.254.0.212]) by hub.freebsd.org (Postfix) with ESMTP id AF12D37B403 for ; Tue, 9 Oct 2001 15:43:22 -0700 (PDT) Received: (qmail 16746 invoked from network); 9 Oct 2001 22:43:22 -0000 Received: from unknown (HELO laptop.baldwin.cx) ([64.81.54.73]) (envelope-sender ) by mail12.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 9 Oct 2001 22:43:22 -0000 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20011009150339.X59854@elvis.mu.org> Date: Tue, 09 Oct 2001 15:42:48 -0700 (PDT) From: John Baldwin To: Alfred Perlstein Subject: Re: ucred API Cc: arch@FreeBSD.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 09-Oct-01 Alfred Perlstein wrote: > * John Baldwin [011009 13:30] wrote: >> Hey all. I'd like to make the following changes to the ucred API in the >> interests of making the locking more sane. The changes will occur in 2 >> stages. >> Stage 2: >> - Add a per-thread reference to the ucred that is created on syscall >> entry and released on syscall exit. It is also created and released >> if needed on trap enter/exit. It is _not_ created for interrupts >> since >> interrupts should not care about the ucred of their borrowed context. >> The per-thread ucred reference will then point to a ucred that won't >> ever change (setuid, etc. update the per-process ucred) and thus won't >> need any locking. Almost all references to ucreds for suser(), VOP's >> etc. will use the thread reference. This will ensure that a thread's >> ucred will be the same for an entire syscall which will close many >> races involving multithreaded programs and ucreds. The only place >> where >> the per-process ucred will be used for access checks is places that >> modify the ucred as we want to ensure there is no race of one thread >> making a credential change it isn't qualified to make due to it >> performing >> its access checks on a stale ucred before updating the master ucred. >> >> I've talked with Robert Watson about these already and they sound good to >> him. >> Any objections? > > Stage 2 is bogus. No, it's not. Otherwise I have to lock the process to get a new reference to a ucred to keep it from not changing out from under me since another thread could change the ucred on another CPU. By using a thread reference, I don't have to use locks except for at the beginning of the syscall when I get the initial reference. > You only need to reference the cred when a thread is created. No. If thread A changes the ucred and later on thread B executes a syscall, it should use the new cred. > In terms of KSE, what I think that means when you'd block leaving a > context (lazy thread creation) you'd do your dup. Think of an SMP system where one thread is reading from a file while another is changing the groups, etc. Granted, that is a userland race that userland needs to handle, but we at lesat have to keep the kernel data structures from getting hosed by userland races. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 10 8:57: 3 2001 Delivered-To: freebsd-arch@freebsd.org Received: from khavrinen.lcs.mit.edu (khavrinen.lcs.mit.edu [18.24.4.193]) by hub.freebsd.org (Postfix) with ESMTP id 0A82337B401 for ; Wed, 10 Oct 2001 08:57:01 -0700 (PDT) Received: (from wollman@localhost) by khavrinen.lcs.mit.edu (8.11.4/8.11.4) id f9AFuvU63583; Wed, 10 Oct 2001 11:56:57 -0400 (EDT) (envelope-from wollman) Date: Wed, 10 Oct 2001 11:56:57 -0400 (EDT) From: Garrett Wollman Message-Id: <200110101556.f9AFuvU63583@khavrinen.lcs.mit.edu> To: Dag-Erling Smorgrav Cc: arch@FreeBSD.org Subject: Re: cvs commit: src/sys/kern kern_proc.c kern_prot.c uipc_socket.c uipc_usrreq.c src/sys/netinet raw_ip.c tcp_subr.c udp_usrreq.c In-Reply-To: References: <200110092140.f99LeVA74145@freefall.freebsd.org> <200110101522.f9AFM0S63283@khavrinen.lcs.mit.edu> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG < said: > Unless you can provide an argument showing that this is necessary to > the correct operation of a FreeBSD system, I'll simply ignore your > contribution to this discussion. And I'll simply ignore your ``contribution'' as well, if that's how you feel about it. You asked ``would people mind a lot''. I answered the question: yes, people would mind a great deal. The process table is *public information*, and has always been so in the entire history of UNIX. You are proposing a totally unacceptable POLA violation. -GAWollman To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 10 10:11:24 2001 Delivered-To: freebsd-arch@freebsd.org Received: from dragon.nuxi.com (trang.nuxi.com [66.92.13.169]) by hub.freebsd.org (Postfix) with ESMTP id B947737B40B for ; Wed, 10 Oct 2001 10:11:21 -0700 (PDT) Received: (from obrien@localhost) by dragon.nuxi.com (8.11.6/8.11.1) id f9AHArn19119; Wed, 10 Oct 2001 10:10:53 -0700 (PDT) (envelope-from obrien) Date: Wed, 10 Oct 2001 10:10:53 -0700 From: "David O'Brien" To: Dag-Erling Smorgrav , arch@FreeBSD.org Subject: Re: cvs commit: src/sys/kern kern_proc.c kern_prot.c uipc_socket.c uipc_usrreq.c src/sys/netinet raw_ip.c tcp_subr.c udp_usrreq.c Message-ID: <20011010101053.A9313@dragon.nuxi.com> Reply-To: arch@FreeBSD.org References: <200110092140.f99LeVA74145@freefall.freebsd.org> <200110101522.f9AFM0S63283@khavrinen.lcs.mit.edu> <200110101556.f9AFuvU63583@khavrinen.lcs.mit.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200110101556.f9AFuvU63583@khavrinen.lcs.mit.edu>; from wollman@khavrinen.lcs.mit.edu on Wed, Oct 10, 2001 at 11:56:57AM -0400 X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD group X-Pgp-Rsa-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Rsa-Keyid: 1024/34F9F9D5 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Oct 10, 2001 at 11:56:57AM -0400, Garrett Wollman wrote: > < said: > > > Unless you can provide an argument showing that this is necessary to > > the correct operation of a FreeBSD system, I'll simply ignore your > > contribution to this discussion. > > And I'll simply ignore your ``contribution'' as well, if that's how > you feel about it. > > You asked ``would people mind a lot''. I answered the question: yes, > people would mind a great deal. The process table is *public > information*, and has always been so in the entire history of UNIX. > You are proposing a totally unacceptable POLA violation. I agree. It would be POLA. Put something in /etc/rc* that sets it to `0' if a high SECURE_LEVEL is set, if you like. Otherwise, leave Unix alone. -- -- David (obrien@FreeBSD.org) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Oct 10 13:46:53 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 82B5437B405 for ; Wed, 10 Oct 2001 13:46:49 -0700 (PDT) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.6/8.11.5) with SMTP id f9AKkmB60522; Wed, 10 Oct 2001 16:46:48 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Wed, 10 Oct 2001 16:46:48 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: arch@FreeBSD.org Cc: Dag-Erling Smorgrav Subject: Re: cvs commit: src/sys/kern kern_proc.c kern_prot.c uipc_socket.c uipc_usrreq.c src/sys/netinet raw_ip.c tcp_subr.c udp_usrreq.c In-Reply-To: <20011010101053.A9313@dragon.nuxi.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 10 Oct 2001, David O'Brien wrote: > On Wed, Oct 10, 2001 at 11:56:57AM -0400, Garrett Wollman wrote: > > < said: > > > > > Unless you can provide an argument showing that this is necessary to > > > the correct operation of a FreeBSD system, I'll simply ignore your > > > contribution to this discussion. > > > > And I'll simply ignore your ``contribution'' as well, if that's how > > you feel about it. > > > > You asked ``would people mind a lot''. I answered the question: yes, > > people would mind a great deal. The process table is *public > > information*, and has always been so in the entire history of UNIX. > > You are proposing a totally unacceptable POLA violation. > > I agree. It would be POLA. Put something in /etc/rc* that sets it to > `0' if a high SECURE_LEVEL is set, if you like. Otherwise, leave Unix > alone. The proper solution may simply be to use /etc/sysctl.conf, and possibly to teach sysinstall how to read/edit the file properly, as well as the meanings of some popular values. Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Oct 11 11:31:59 2001 Delivered-To: freebsd-arch@freebsd.org Received: from InterJet.elischer.org (c421509-a.pinol1.sfba.home.com [24.7.86.9]) by hub.freebsd.org (Postfix) with ESMTP id 66E3737B406; Thu, 11 Oct 2001 11:31:51 -0700 (PDT) Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id MAA37376; Thu, 11 Oct 2001 12:35:59 -0700 (PDT) Date: Thu, 11 Oct 2001 12:35:58 -0700 (PDT) From: Julian Elischer To: Orran Y Krieger Cc: Bryan S Rosenburg , Marc Auslander , Paul McKenney , Greg Lehey , Matt Dillon , peter@freebsd.org, arch@freebsd.org Subject: Re: Julian Elischer: Re: FreeBSD KSE In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Quick answers.. (I'm at work) On Wed, 10 Oct 2001, Orran Y Krieger wrote: > Cool stuff. A few stream of consciousness comments, since it is late at > night :-) > > For page faults, the Exokernel system pinned resources and guaranteed that > those where the only resources used in the user-level scheduler for thread > scheduling, this seemed to tough for us since text is a problem (they did > all paging in app as well). Instead, K42 has a mode we call disabled > (logically equivalent to interrupt disabled mode in the kernel) where a bit > is shared between the kernel and app, the kernel enters the app disabled > (i.e., with the bit set) and page faults that occur when the bit is set are > handled directly by the kernel. Are you using one of these techniques for > handling page faults except when not in the user-land scheduler, or > something else? > Each KSE (think virtual processor) has a separate mailbox allocated to it in user space. Whenm the UTS (Userland Thread Scheduler) schedules a thread, the last thing it does before jumping into that thread (loading the PC) is to set a pointer in the mailbox to point to a thread control block who'se format is known to the kernel. If that pointer is NULL, then we are "IN" the UTS otherwise we are in a thread. When a thread does a system call (runnung within the KSE) the KSE takes note of the address in that pointer, and if the thread is blockled for any reason the kernel state is saved, including that pointer. The pointer is cleared and an upcall is made to the UTS, which notices that the thread has been blocked (the pointer is NULL). When the thread is restarted in the kernel, then at the beginning of the next (or present) quantam for that KSE, the syscall is first completed, and the completeion state stored within the control block who's address was stored with the blocked thread. The state of the thread is made to look as thjough the syscall completed but was immediatly (atomically) followed by a yield(). All completed threads are linked together on a linked list hanging off the KSE mailbox. After all syscalls that are completable are completed, then the remainder of the quantum is handed to the UTS in the form of an upcall. The UTS looks in the mailbox and seeing the linked list of returned threads, adds them to it's own runnable list, and then selects the next thread to run, choosing from all runnable threads, including those that just retunred from syscalls. (It's conceivable that all thios may happen on the return from a hardware interrupt if the correct KSE is already running, and the mailbox indicates that teh current thread, running in userland is pre-enpltable at this time. In this case the kernel will save it's state allong with the completed syscalls, so that the UTS may choose whether to keep running it or switch to a higher priority thread that completed a syscall. > By the way, we use the disabled mode to traverse per-processor data > structures atomically, i.e., without requiring any locks. This lets you > do various things efficiently on an MP in app space, just like disabling > interrupts in kernel does. This was important, because very cheap thread > creation that you get out of a user-level thread model was used by us to > enable cheap upcalls, event notifaction, a cool IPC facility... We use > this to move lots of other functionality out of kernel and inter app, e.g., > timers so cancelling timeout very cheap, user-level IP, user-level pipes, Basically we will use flags in the mailbox to control how the kernel treats the particular KSE. (Kernel Schedulable Entity). Obviously flags in one mailbox will not affect another KSE in the same process so they cannot be used to control inter-KSE-communications. Each KSE will however have private structures and we can use the mailbox flags to ensure for example that a thread running on a KSE will not be pre-empted by another thread if it is in a critical region. Inter KSE communications will require the usual safety measures.. > ... > > Think there is something missing in your forward progress guarantee. Lets > say that the kernel will allow 50 threads of an application to be blocked, > and then not run any more application threads. If the application has 50 > threads each communicating with each other via a huge chain of pipes (i.e., > thread 1 writes to a pipe that thread 2 reads from, thread 2 writes to a > pipe that thread 3 reads from...) and then lets assume that threads 2-51 > are all blocked on reading. Then, the kernel won't run thread 1, which is > the thread needed to unblock all the other threads. Clearly this is a > contrived example, but we could think of more realistic ones with threads > blocked on locks... In any case, with a monolithic kernel, once you > allocate threads they are guaranteed to run. With a user-level thread > model, we can have these kind of forward progress problems if the number > that can be blocked is less than the total number. The way we get around > this in K42 is to push the full thread state into application level, and > have a fixed set of resources in the kernel irrespective of the number of > threads currently blocked. That is, threads block for page faults and > system calls in their own address space. The only time a thread is blocked > in the kernel is if it faults when "disabled" and in this case the kernel > handles the page fault without reflecting any state up to the application. I haven't time to fully think about this right now, however Our scheme is that there will be SOME limit on the number of threads that can be suspended in syscalls (or page faults or whatever) at one time, and that teh Nth one will just block as would happen without KSEs enabled. (the N-1th will do an upcall with a 'danger' flag set telling the UTS to get its act together) In this case you COULD possibly make some unlikely deadlock situation but I don't think it's worth changing teh design for it.. (pilot error) > > Anyhow, just a couple of thoughts that came up when we looked at your paper > (obviously the issues we struggled with in our design). This is very cool > stuff, would be fun to bounce around designs in more detail. > -- Orran > ---------------------- Forwarded by Orran Y Krieger/Watson/IBM on > 10/10/2001 07:42 PM --------------------------- > > David Edelsohn on 10/10/2001 03:56:20 PM > > To: Orran Y Krieger/Watson/IBM@IBMUS, Bryan S Rosenburg/Watson/IBM@IBMUS, > Marc Auslander/Watson/IBM@IBMUS > cc: Paul McKenney/Beaverton/IBM@IBMUS > Subject: Julian Elischer: Re: FreeBSD KSE > > > > > ------- Forwarded Message > > Date: Wed, 10 Oct 2001 13:39:27 -0700 (PDT) > From: Julian Elischer > To: David Edelsohn > cc: Greg Lehey , Matt Dillon , > peter@freebsd.org > Subject: Re: FreeBSD KSE > In-Reply-To: <200110101805.OAA21020@makai.watson.ibm.com> > > Hi! > > On Wed, 10 Oct 2001, David Edelsohn wrote: > > > One aspect of K42 is user thread scheduling, very much like > > Schedular Activations. While reading over the FreeBSD design, two > > questions occurred to us: > > The paper you have read is somewhat out of date.. > The 'verbal' design that we are discussing has a lot of refinements > and a very different interface.. > > > 1) The design only seems to mention blocking due to system calls. > > Will KSE address threads blocking on a page faults as well? > > That is the eventual aim. (not page faults in the Usserland Thread > scheduler however).. (duh) :-) > > > 2) Does the KSE design completely ensure forward progress? > > This is not necessarily a problem in practice, but theoretically > possible. > > When the process is in the Thread scheduler, it is guaranteed to return > after any kernel entry, back to the Thread scheduler at the point it left > off. This ensures that the thread scheduler makes forward progress. > Threads are only run when the Thread scheduler runs them so it is > guaranteed to be able to make forward progress on SOME threads on each > quantum. (at least that's the way I see it..) > > I'd love to discuss some of the 'Cute' ideas that are not yet in the > paper. But I'm at work at the moment. > > Feel free to ask me about anything that catches you interest however.. > > Most of these ideas have been discussed on the 'arch' > mailing list > > e.g. > http://www.FreeBSD.org/cgi/getmsg.cgi?fetch=31777+36162+/usr/local/www/db/text/2001/freebsd-arch/20010930.freebsd-arch > > http://www.FreeBSD.org/cgi/getmsg.cgi?fetch=38716+42446+/usr/local/www/db/text/2001/freebsd-arch/20010930.freebsd-arch > > http://www.FreeBSD.org/cgi/getmsg.cgi?fetch=62467+69962+/usr/local/www/db/text/2000/freebsd-arch/20001203.freebsd-arch > > > ------- End of Forwarded Message > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Oct 11 14: 7:23 2001 Delivered-To: freebsd-arch@freebsd.org Received: from holly.dyndns.org (adsl-208-191-149-224.dsl.hstntx.swbell.net [208.191.149.224]) by hub.freebsd.org (Postfix) with ESMTP id 42E7B37B407 for ; Thu, 11 Oct 2001 14:07:21 -0700 (PDT) Received: (from chris@localhost) by holly.dyndns.org (8.11.6/8.9.3) id f9BL7pj18725 for arch@FreeBSD.org; Thu, 11 Oct 2001 16:07:52 -0500 (CDT) (envelope-from chris) Date: Thu, 11 Oct 2001 16:07:50 -0500 From: Chris Costello To: arch@FreeBSD.org Subject: Merging fdescfs. Message-ID: <20011011160750.D696@holly.calldei.com> Reply-To: chris@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG There is at least one longstanding bug in the -STABLE implementation of fdescfs at this point, and since the -CURRENT code has been around for quite some time now, has had adequate testing time, and is generally cleaner than the original code, I'm all for merging the entire thing into -STABLE now that 4.4-RELEASE has past. I'm not entirely sure, however, if POLA is a real problem in this case since so obviously few people use it, and I'd like your feedback as to whether it is or isn't. Thanks, -- +-------------------+----------------------------------------------+ | Chris Costello | Can I yell "movie" in a crowded firehouse?? | | chris@FreeBSD.org | | +-------------------+----------------------------------------------+ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Oct 11 14:34: 1 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail11.speakeasy.net (mail11.speakeasy.net [216.254.0.211]) by hub.freebsd.org (Postfix) with ESMTP id BA2BF37B408 for ; Thu, 11 Oct 2001 14:33:57 -0700 (PDT) Received: (qmail 72135 invoked from network); 11 Oct 2001 21:33:56 -0000 Received: from unknown (HELO laptop.baldwin.cx) ([64.81.54.73]) (envelope-sender ) by mail11.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 11 Oct 2001 21:33:56 -0000 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20011011160750.D696@holly.calldei.com> Date: Thu, 11 Oct 2001 14:33:20 -0700 (PDT) From: John Baldwin To: Chris Costello Subject: RE: Merging fdescfs. Cc: arch@FreeBSD.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 11-Oct-01 Chris Costello wrote: > There is at least one longstanding bug in the -STABLE > implementation of fdescfs at this point, and since the -CURRENT > code has been around for quite some time now, has had adequate > testing time, and is generally cleaner than the original code, > I'm all for merging the entire thing into -STABLE now that > 4.4-RELEASE has past. I'm not entirely sure, however, if POLA is > a real problem in this case since so obviously few people use it, > and I'd like your feedback as to whether it is or isn't. What are the changes that are visible to the user in the new fdescfs versus the old one? -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Oct 11 15: 8:37 2001 Delivered-To: freebsd-arch@freebsd.org Received: from holly.dyndns.org (adsl-208-191-149-224.dsl.hstntx.swbell.net [208.191.149.224]) by hub.freebsd.org (Postfix) with ESMTP id 51A2737B405; Thu, 11 Oct 2001 15:08:29 -0700 (PDT) Received: (from chris@localhost) by holly.dyndns.org (8.11.6/8.9.3) id f9BM90f19147; Thu, 11 Oct 2001 17:09:00 -0500 (CDT) (envelope-from chris) Date: Thu, 11 Oct 2001 17:08:59 -0500 From: Chris Costello To: John Baldwin Cc: arch@FreeBSD.org Subject: Re: Merging fdescfs. Message-ID: <20011011170859.A19044@holly.calldei.com> Reply-To: chris@FreeBSD.org References: <20011011160750.D696@holly.calldei.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from jhb@FreeBSD.org on Thu, Oct 11, 2001 at 02:33:20PM -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thursday, October 11, 2001, John Baldwin wrote: > What are the changes that are visible to the user in the new fdescfs > versus the old one? The most obvious is the fact that it no longer gets union-mounted on /dev, it's mounted directly on /dev/fd: # df . Filesystem 1K-blocks Used Avail Capacity Mounted on fdesc 1 1 0 100% /dev/fd # ls -l total 0 crw------- 1 chris tty 5, 0 Oct 11 16:59 0 crw------- 1 chris tty 5, 0 Oct 11 16:59 1 crw------- 1 chris tty 5, 0 Oct 11 16:59 2 d--------- 1 root wheel 512 Oct 11 16:56 3 d--------- 1 root wheel 512 Oct 11 16:56 4 d--------- 1 root wheel 512 Oct 11 16:56 5 This also means that there are no longer any `tty', `stdin' or `stdout' nodes in fdescfs. -- +-------------------+------------------------------------+ | Chris Costello | What this country needs | | chris@FreeBSD.org | is a good five-cent microcomputer. | +-------------------+------------------------------------+ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Oct 11 17: 8:59 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail.bobtomesford.com (w115.z064002097.dfw-tx.dsl.cnc.net [64.2.97.115]) by hub.freebsd.org (Postfix) with ESMTP id 46B1537B43B for ; Thu, 11 Oct 2001 17:08:49 -0700 (PDT) Received: from localhost (manele-femei-si-bautura.org [64.225.124.232]) by mail.bobtomesford.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21) id 4RBCL4SK; Thu, 11 Oct 2001 19:05:38 -0500 To: freebsd-arch@FreeBSD.org From: info@life-protect.com X-Mailer: Perl+Mail::Sender 0.7.08 by Jan Krynicky Subject: Protect yourself ! Message-Id: <20011012000849.46B1537B43B@hub.freebsd.org> Date: Thu, 11 Oct 2001 17:08:49 -0700 (PDT) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG When was the last tragedy you were involved in? The next one could be the last!! REMEMBER, 80% of sinister fatalities occur due to toxic gases and smoke inhalation. OUR PRODUCT CAN SAVE YOUR LIFE OR THE LIFE OF A LOVED ONE ORDER NOW ! Click HERE for more info : http://www.life-protect.com/partner.cgi?Partner=1011 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Oct 11 17:24:15 2001 Delivered-To: freebsd-arch@freebsd.org Received: from corazon.garcialks.com (w242.z064002099.sat-tx.dsl.cnc.net [64.2.99.242]) by hub.freebsd.org (Postfix) with SMTP id 92BCF37B409 for ; Thu, 11 Oct 2001 17:24:12 -0700 (PDT) Received: (qmail 25626 invoked from network); 12 Oct 2001 01:37:19 -0000 Received: from manele-femei-si-bautura.org (HELO localhost) (64.225.124.232) by w242.z064002099.sat-tx.dsl.cnc.net with SMTP; 12 Oct 2001 01:37:19 -0000 To: freebsd-arch@FreeBSD.org From: info@life-protect.com X-Mailer: Perl+Mail::Sender 0.7.08 by Jan Krynicky Subject: Protect yourself ! Message-Id: <20011012002412.92BCF37B409@hub.freebsd.org> Date: Thu, 11 Oct 2001 17:24:12 -0700 (PDT) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG When was the last tragedy you were involved in? The next one could be the last!! REMEMBER, 80% of sinister fatalities occur due to toxic gases and smoke inhalation. OUR PRODUCT CAN SAVE YOUR LIFE OR THE LIFE OF A LOVED ONE ORDER NOW ! Click HERE for more info : http://www.life-protect.com/partner.cgi?Partner=1011 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Oct 13 9:35:20 2001 Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id BDFCB37B40B; Sat, 13 Oct 2001 09:27:10 -0700 (PDT) Received: by flood.ping.uio.no (Postfix, from userid 2602) id 676B614C3E; Sat, 13 Oct 2001 18:27:09 +0200 (CEST) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: arch@freebsd.org Subject: Generating host.conf for backward compatibility References: <3BC67CBD.1DAAC031@DougBarton.net> <3BC7EA21.15F43811@DougBarton.net> From: Dag-Erling Smorgrav Date: 13 Oct 2001 18:27:08 +0200 In-Reply-To: Message-ID: Lines: 18 User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/20.7 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --=-=-= In our eagerness to switch from host.conf to nsswitch.conf, we forgot that old binaries and libraries (COMPAT_* and tons of third-party software for which sources might not even be available) still expect to find /etc/host.conf, and may not work (correctly) if it doesn't exist. The attached patch creates host.conf from nsswitch.conf if it detects the latter. The first time you boot after upgrading from -STABLE or an old -CURRENT, nsswitch.conf will be created, and host.conf won't be touched; all subsequent times, host.conf will be auto-generated from the information contained in nsswitch.conf. DES -- Dag-Erling Smorgrav - des@ofug.org --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=host_conf.diff Index: etc/rc.network =================================================================== RCS file: /home/ncvs/src/etc/rc.network,v retrieving revision 1.105 diff -u -r1.105 rc.network --- etc/rc.network 10 Oct 2001 20:36:51 -0000 1.105 +++ etc/rc.network 13 Oct 2001 16:23:35 -0000 @@ -38,16 +38,21 @@ network_pass1() { echo -n 'Doing initial network setup:' + # Generate host.conf for compatibility + # + if [ -f "/etc/nsswitch.conf" ]; then + echo '' + echo 'Generating /etc/host.conf for compatibility' + generate_host_conf /etc/nsswitch.conf /etc/host.conf + fi + # Convert host.conf to nsswitch.conf if necessary - if [ -f "/etc/host.conf" ]; then + # + if [ -f "/etc/host.conf" -a ! -f "/etc/nsswitch.conf" ]; then echo '' echo 'Warning: /etc/host.conf is no longer used' - if [ -f "/etc/nsswitch.conf" ]; then - echo ' /etc/nsswitch.conf will be used instead' - else - echo ' /etc/nsswitch.conf will be created for you' - convert_host_conf /etc/host.conf /etc/nsswitch.conf - fi + echo ' /etc/nsswitch.conf will be created for you' + convert_host_conf /etc/host.conf /etc/nsswitch.conf fi # Set the host name if it is not already set @@ -827,3 +832,25 @@ }' < $host_conf > $nsswitch_conf } +generate_host_conf() { + nsswitch_conf=$1; shift; + host_conf=$1; shift; + + awk ' +BEGIN { + xlat["files"] = "hosts"; + xlat["dns"] = "bind"; + xlat["nis"] = "nis"; +} +/^hosts:/ { + print "# Auto-generated, do not edit"; + for (n = 2; n <= NF; ++n) + if ($n in xlat) + print xlat[$n]; + quit; +} +// { + next; +} +' <$nsswitch_conf >$host_conf +} --=-=-=-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Oct 13 15:19:18 2001 Delivered-To: freebsd-arch@freebsd.org Received: from peter3.wemm.org (c1315225-a.plstn1.sfba.home.com [24.14.150.180]) by hub.freebsd.org (Postfix) with ESMTP id 5B9F137B403 for ; Sat, 13 Oct 2001 15:19:10 -0700 (PDT) Received: from overcee.netplex.com.au (overcee.wemm.org [10.0.0.3]) by peter3.wemm.org (8.11.0/8.11.0) with ESMTP id f9DMJ9M94666 for ; Sat, 13 Oct 2001 15:19:09 -0700 (PDT) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by overcee.netplex.com.au (Postfix) with ESMTP id 80C3A3810; Sat, 13 Oct 2001 15:19:09 -0700 (PDT) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 To: Dag-Erling Smorgrav Cc: arch@FreeBSD.ORG Subject: Re: Generating host.conf for backward compatibility In-Reply-To: Date: Sat, 13 Oct 2001 15:19:09 -0700 From: Peter Wemm Message-Id: <20011013221909.80C3A3810@overcee.netplex.com.au> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Dag-Erling Smorgrav wrote: > --=-=-= > > In our eagerness to switch from host.conf to nsswitch.conf, we forgot > that old binaries and libraries (COMPAT_* and tons of third-party > software for which sources might not even be available) still expect > to find /etc/host.conf, and may not work (correctly) if it doesn't > exist. I'm a bit puzzled as to why this makes a difference. The old libc resolver defaulted to "dns","files" if host.conf was missing. The file is and always was optional. Of course, that doesn't help you if you use a NIS hosts map, or if you're expecting /etc/hosts to override DNS lookups. static void init_services() { ... if ((fd = (FILE *)fopen(_PATH_HOSTCONF, "r")) == NULL) { /* make some assumptions */ service_order[0] = SERVICE_BIND; service_order[1] = SERVICE_HOSTS; service_order[2] = SERVICE_NONE; } else { while (fgets(buf, BUFSIZ, fd) != NULL && cc < SERVICE_MAX) { It has always been like this. (well, at least as far back as 1994): 1.2 (wollman 09-Aug-94): if ((fd = (FILE *)fopen(_PATH_HOSTCONF, "r")) == NULL) { 1.2 (wollman 09-Aug-94): /* make some assumptions */ 1.2 (wollman 09-Aug-94): service_order[0] = SERVICE_BIND; 1.2 (wollman 09-Aug-94): service_order[1] = SERVICE_HOSTS; 1.2 (wollman 09-Aug-94): service_order[2] = SERVICE_NONE; 1.2 (wollman 09-Aug-94): } else { 1.2 (wollman 09-Aug-94): while (fgets(buf, BUFSIZ, fd) != NULL && cc < SERVICE_MAX) { Something else must be going on. Secondly, since nsswitch.conf is the master file in the patch, it should be updated every time. Otherwise POLA will be violated if nsswitch.conf is updated and some old binaries still use old methods. Regarding COMPAT_* - none of those use host.conf, at least not in the same way that it exists for us. For example, the /compat/linux/etc/host.conf is a completely different syntax. The other COMPAT_* use different files altogether. (eg: svc.conf, service.switch, /etc/net/*, and so on). Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Oct 13 15:43:29 2001 Delivered-To: freebsd-arch@freebsd.org Received: from db-cvad-1-tmp.yahoo.com (db-cvad-1-tmp.yahoo.com [216.145.48.242]) by hub.freebsd.org (Postfix) with ESMTP id 3393C37B411 for ; Sat, 13 Oct 2001 15:43:16 -0700 (PDT) Received: from master.gorean.org (root@master.gorean.org [10.0.0.2]) by db-cvad-1-tmp.yahoo.com (8.11.6/8.11.6) with ESMTP id f9DMhFs28352; Sat, 13 Oct 2001 15:43:15 -0700 (PDT) (envelope-from DougB@FreeBSD.org) Received: from localhost (doug@localhost) by master.gorean.org (8.11.6/8.11.6) with ESMTP id f9DMhYr41744; Sat, 13 Oct 2001 15:43:34 -0700 (PDT) (envelope-from DougB@FreeBSD.org) X-Authentication-Warning: master.gorean.org: doug owned process doing -bs Date: Sat, 13 Oct 2001 15:43:34 -0700 (PDT) From: Doug Barton X-X-Sender: doug@master.gorean.org To: Peter Wemm Cc: Dag-Erling Smorgrav , Subject: Re: Generating host.conf for backward compatibility In-Reply-To: <20011013221909.80C3A3810@overcee.netplex.com.au> Message-ID: <20011013153357.R40455-100000@master.gorean.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, 13 Oct 2001, Peter Wemm wrote: > Dag-Erling Smorgrav wrote: > > --=-=-= > > > > In our eagerness to switch from host.conf to nsswitch.conf, we forgot > > that old binaries and libraries (COMPAT_* and tons of third-party > > software for which sources might not even be available) still expect > > to find /etc/host.conf, and may not work (correctly) if it doesn't > > exist. > > I'm a bit puzzled as to why this makes a difference. The old libc resolver > defaulted to "dns","files" if host.conf was missing. The file is and > always was optional. > > Of course, that doesn't help you if you use a NIS hosts map, or if you're > expecting /etc/hosts to override DNS lookups. I know that when I was running -current before the recent stdio changes made COMPAT4X mandatory, I used 4.x binaries without difficulty. The two most notable examples are cvsup and Yahoo! messenger. Once I installed the compat stuff those binaries stopped working. I haven't had a chance to test Dag-Erling's theory yet, since I've still got ATA issues to solve before I can move my workstation back to -current. Here's an ugly thought.... what would happen if /etc/host.conf existed, but its contents were missing or invalid? The more I think about this.... I vaguely recall futzing around with /etc/host.conf around the time of the nsswitch.conf change, and I may have left an empty file hanging around to test some rc* stuff. If that's the case, I apologize if this has been a waste of time... although it might be nice to build something into libc that says "If the contents of /etc/{host|nsswitch}.conf don't make any sense, fall back to the default." Looking for somewhere to hide... Doug -- "We will not tire, we will not falter, and we will not fail." - George W. Bush, President of the United States September 20, 2001 Do YOU Yahoo!? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Oct 13 21:18:53 2001 Delivered-To: freebsd-arch@freebsd.org Received: from peter3.wemm.org (c1315225-a.plstn1.sfba.home.com [24.14.150.180]) by hub.freebsd.org (Postfix) with ESMTP id D904137B403; Sat, 13 Oct 2001 21:18:47 -0700 (PDT) Received: from overcee.netplex.com.au (overcee.wemm.org [10.0.0.3]) by peter3.wemm.org (8.11.0/8.11.0) with ESMTP id f9E4IlM95379; Sat, 13 Oct 2001 21:18:47 -0700 (PDT) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by overcee.netplex.com.au (Postfix) with ESMTP id 33E0C3810; Sat, 13 Oct 2001 21:18:47 -0700 (PDT) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 To: Doug Barton Cc: Dag-Erling Smorgrav , arch@FreeBSD.org Subject: Re: Generating host.conf for backward compatibility In-Reply-To: <20011013153357.R40455-100000@master.gorean.org> Date: Sat, 13 Oct 2001 21:18:47 -0700 From: Peter Wemm Message-Id: <20011014041847.33E0C3810@overcee.netplex.com.au> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Doug Barton wrote: > On Sat, 13 Oct 2001, Peter Wemm wrote: > > > Dag-Erling Smorgrav wrote: > > > --=-=-= > > > > > > In our eagerness to switch from host.conf to nsswitch.conf, we forgot > > > that old binaries and libraries (COMPAT_* and tons of third-party > > > software for which sources might not even be available) still expect > > > to find /etc/host.conf, and may not work (correctly) if it doesn't > > > exist. > > > > I'm a bit puzzled as to why this makes a difference. The old libc resolver > > defaulted to "dns","files" if host.conf was missing. The file is and > > always was optional. > > > > Of course, that doesn't help you if you use a NIS hosts map, or if you're > > expecting /etc/hosts to override DNS lookups. > > I know that when I was running -current before the recent stdio > changes made COMPAT4X mandatory, I used 4.x binaries without difficulty. > The two most notable examples are cvsup and Yahoo! messenger. Once I > installed the compat stuff those binaries stopped working. I haven't had a > chance to test Dag-Erling's theory yet, since I've still got ATA issues to > solve before I can move my workstation back to -current. > > Here's an ugly thought.... what would happen if /etc/host.conf > existed, but its contents were missing or invalid? The more I think about > this.... I vaguely recall futzing around with /etc/host.conf around the > time of the nsswitch.conf change, and I may have left an empty file > hanging around to test some rc* stuff. This would certainly explain it if that is what happened. Your 4.x binaries would have ended up not being able to resolve anything. > If that's the case, I apologize if > this has been a waste of time... although it might be nice to build > something into libc that says "If the contents of > /etc/{host|nsswitch}.conf don't make any sense, fall back to the default." > > Looking for somewhere to hide... > > Doug > -- > "We will not tire, we will not falter, and we will not fail." > - George W. Bush, President of the United States > September 20, 2001 > > Do YOU Yahoo!? > > > Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message