From owner-freebsd-arch@FreeBSD.ORG Mon Jul 12 22:18:29 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 18A5A16A4CE; Mon, 12 Jul 2004 22:18:29 +0000 (GMT) Received: from harmony.village.org (rover.village.org [168.103.84.182]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7C38B43D39; Mon, 12 Jul 2004 22:18:27 +0000 (GMT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.11/8.12.11) with ESMTP id i6CMFZXQ040027; Mon, 12 Jul 2004 16:15:36 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Mon, 12 Jul 2004 16:15:33 -0600 (MDT) Message-Id: <20040712.161533.85418996.imp@bsdimp.com> To: phk@phk.freebsd.dk From: "M. Warner Losh" In-Reply-To: <1420.1089448716@critter.freebsd.dk> References: <20040709232355.GB1626@green.homeunix.org> <1420.1089448716@critter.freebsd.dk> X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: arch@freebsd.org Subject: Re: userland firmware loader? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jul 2004 22:18:29 -0000 In message: <1420.1089448716@critter.freebsd.dk> "Poul-Henning Kamp" writes: : In message <20040709232355.GB1626@green.homeunix.org>, Brian Fundakowski Feldma : n writes: : : >I'm writing a driver that uses a 100KB firmware file and could need to : >read it again, after initialization, to reset the device (or to load : >more). Since this device is primarily found in CardBus form, I think : >it makes a lot of sense to be able to get the firmware file at any time : >but without specific user intervention. : > : >Is this something devd should be doing? : : I would advocate a generic interface for retrieving firmware from userland. I'd advocate motherhood and apple pie too. :-) Warner From owner-freebsd-arch@FreeBSD.ORG Mon Jul 12 22:24:27 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8C4D416A4D6; Mon, 12 Jul 2004 22:24:27 +0000 (GMT) Received: from harmony.village.org (rover.village.org [168.103.84.182]) by mx1.FreeBSD.org (Postfix) with ESMTP id DF30E43D5D; Mon, 12 Jul 2004 22:24:26 +0000 (GMT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.11/8.12.11) with ESMTP id i6CMMwGN040146; Mon, 12 Jul 2004 16:22:58 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Mon, 12 Jul 2004 16:22:59 -0600 (MDT) Message-Id: <20040712.162259.21927670.imp@bsdimp.com> To: green@freebsd.org From: "M. Warner Losh" In-Reply-To: <20040710151647.GI1626@green.homeunix.org> References: <20040709232355.GB1626@green.homeunix.org> <1420.1089448716@critter.freebsd.dk> <20040710151647.GI1626@green.homeunix.org> X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: arch@freebsd.org cc: phk@phk.freebsd.dk Subject: Re: userland firmware loader? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jul 2004 22:24:27 -0000 In message: <20040710151647.GI1626@green.homeunix.org> Brian Fundakowski Feldman writes: : On Sat, Jul 10, 2004 at 10:38:36AM +0200, Poul-Henning Kamp wrote: : > In message <20040709232355.GB1626@green.homeunix.org>, Brian Fundakowski Feldma : > n writes: : > : > >I'm writing a driver that uses a 100KB firmware file and could need to : > >read it again, after initialization, to reset the device (or to load : > >more). Since this device is primarily found in CardBus form, I think : > >it makes a lot of sense to be able to get the firmware file at any time : > >but without specific user intervention. The form factor of the device should be irrelevant to the solution. When do these reset events happen? Do you introduce circular dependencies by requiring an upcall to userland to make things happen? : > >Is this something devd should be doing? : > : > I would advocate a generic interface for retrieving firmware from userland. : : What do you think about using the normal devctl_notify(9) for pushing : out firmware requests, and having a new device method that receives just : a user data pointer and length for it to copyin(9)? It wouldn't be a : very symmetrical API, but it seems like a way that might fit it into the : current structure the easiest, devd and devctl and newbus. I think its a terrible idea. It is against the whole notion that devd just reacts and doesn't hold anything up. If your device needs a reset, then devd gets the message and causes a program to run to load the firmware for the device. Does the driver block until then? What happens if the program never runs? devd reacts to events from the kernel only. It shouldn't be in a critical path or be viewed as an upcall that will return a result. It is a completely optional part of the system. I also think that firmware requests aren't well defined enough to make it into a useful interface. Better to have some way to trigger a module load from inside the driver and use the data, and then unload it when it is done w/o any interaction from userland at all. The firmware module would have a tiny bit of code that would set a pointer to the data structure necessary for the firmware load. Even better would be to allow drivers to read data from mounted file system, but that's difficult to do right now... Warner From owner-freebsd-arch@FreeBSD.ORG Tue Jul 13 10:36:51 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EB27816A4CE for ; Tue, 13 Jul 2004 10:36:51 +0000 (GMT) Received: from phuket.psconsult.nl (ps226.psconsult.nl [213.222.19.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 309D643D1D for ; Tue, 13 Jul 2004 10:36:50 +0000 (GMT) (envelope-from fb-arch@psconsult.nl) Received: from phuket.psconsult.nl (localhost [127.0.0.1]) by phuket.psconsult.nl (8.12.8p2/8.12.8) with ESMTP id i6DAamYe017568 for ; Tue, 13 Jul 2004 12:36:48 +0200 (CEST) (envelope-from fb-arch@psconsult.nl) Received: (from paul@localhost) by phuket.psconsult.nl (8.12.8p2/8.12.8/Submit) id i6DAamh3017567 for arch@freebsd.org; Tue, 13 Jul 2004 12:36:48 +0200 (CEST) Date: Tue, 13 Jul 2004 12:36:48 +0200 From: Paul Schenkeveld To: arch@freebsd.org Message-ID: <20040713103648.GA17322@psconsult.nl> Mail-Followup-To: arch@freebsd.org References: <6595.1089317548@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <6595.1089317548@critter.freebsd.dk> User-Agent: Mutt/1.5.6i Subject: Re: [RFC] kldunload -f argument. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jul 2004 10:36:52 -0000 On Thu, Jul 08, 2004 at 10:12:28PM +0200, Poul-Henning Kamp wrote: > ... > In the new world order, a new event is introduced MOD_QUIESCE[1]. > ... > [1] I hate that name, I can't spell it without looking it up. Better > suggestions are very welcome. MOD_GOIDLE ? MOD_RETIRE ? > ... > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. Paul Schenkeveld From owner-freebsd-arch@FreeBSD.ORG Tue Jul 13 23:46:01 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CD5AA16A4CE; Tue, 13 Jul 2004 23:46:01 +0000 (GMT) Received: from harmony.village.org (rover.village.org [168.103.84.182]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0772E43D39; Tue, 13 Jul 2004 23:46:00 +0000 (GMT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.11/8.12.11) with ESMTP id i6DNj8Bd055675; Tue, 13 Jul 2004 17:45:09 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Tue, 13 Jul 2004 17:45:12 -0600 (MDT) Message-Id: <20040713.174512.11239675.imp@bsdimp.com> To: rwatson@freebsd.org From: "M. Warner Losh" In-Reply-To: References: <20040713.153657.36025694.imp@bsdimp.com> X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: arch@freebsd.org cc: phk@phk.freebsd.dk Subject: newbus integration of MOD_QUIESCE (was Re: cvs commit: src/sbin/kldunload kldunload.8 kldunload.c ) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jul 2004 23:46:02 -0000 In message: Robert Watson writes: : That said, I'm not sure I see the kldunload state changes as incompatible : with the notion of having most kernel modules become newbussified, I don't either. I just wanted to point out 1) there's no committed newbus hooks 2) there are plans for similar things in newbus 3) no design effort was made to accomidate newbus (which I shouldn't have raised) and we got off in a bunch of different directions... Let's see if we can get it back on track. : I suggest creating a thread named "Newbus the world" on arch@, in which : the major classes of kernel modules and extension interfaces are : documented, along with ideas about how they could usefully turn into : Newbus-enabled pieces. Poul-Henning's list is a decent starting point, : but probably not a complete list. Remember to tell people why switching : to Newbus would be an improvement, especially for modules and module : systems that don't clearly map onto synthetic or real devices, such as : Netgraph nodes, MAC policy modules, schedulers, network protocols, file : systems, etc. I think that's likely overkill at this point. I think you are reading way more into this than I'd wanted. And you've gotten my objection backward, which is taking this discussion away from fruitful waters which I hope to make progress on the problem at hand. I don't object that everything isn't newbused. I object that newbus interaction wasn't considered enough before this was implemented. Leaving aside a he said/she said recounting, I'd like to address my frustion at hand. I'm trying to answer the question 'Now that this is in the tree, what should newbus do when one of its modules gets a MOD_QUIESCE?' The only documentation I have on this is: Add a MOD_QUIESCE event for modules. This should return error (EBUSY) of the module is in use. [[ assuming the 'of' is a typo for 'if' ]] and On unload it is first called with what set to MOD_QUIESCE . If the unload was not forced, a non-zero return will prevent the unload from happening. If the unload continues what is set to MOD_UNLOAD. If the module returns non-zero to this, the unload will not happen. The difference between MOD_QUIESCE and MOD_UNLOAD is that the module should fail MOD_QUIESCE if it is currently in use, whereas MOD_UNLOAD should only fail if it is impossible to unload the module, for instance because there are memory references to the module which cannot be revoked. I've not seen other documentation that explains the event in more detail. Nor do I see comments in the code explaining it any better. >From reading the code and the docs, MOD_QUIESCE is just a 'are you idle?' question. The nasty case I've come up with is what happens when the module is idle (not busy), but becomes busy (not idle) after the MOD_QUIESCE call? Right now newbus modules that receive a MOD_UNLOAD call attempt to detach all instances of devices contained in that module. If I have a way to poll the driver to see if it is busy (which is relatively easy to implement), then if it becomes busy after the MOD_QUIESCE call, I get a MOD_UNLOAD which would force instances to detach. Case 1: driver becomes busy MOD_QUIESCE comes in, newbus returns EBUSY Case 2: MOD_QUIESCE comes in, newbus returns 0 driver becomes busy MOD_UNLOAD comes in, newbus detaches the device In case 1 we have the correct results. In case 2 we do not unless we know newbus/the driver remembers it has been quiesced and fails the operation that causes it to become busy instead of allowing it to succeed. If this is true, this implies there's more to the MOD_QUIESCE call than just asking the question 'Are you idle?' If not, then how do we get out of the race here? Part of the problem here is that some drivers will take a detach request as a command, and others as a request. Most of them will detach unless they really really really can't unwide the state. Some will only detach if nothing is open. Network drivers generally just detach since they are never open (but may be configured, etc). So this is presently a bit of a rats nest of conflicting meanings that makes it hard to do some things people have suggested: 1) load a driver and then unload it if nothing attaches 2) unload drivers (modules) that have no attached drivers but leave the attached ones alone. MOD_QUIESCE will help these cases a lot, but I'd like to get all the different meanings in the tree in alignment, and to do that I think I need a better definition. So ignore the grumpiness that I had earlier in the day and focus on this message. Warner From owner-freebsd-arch@FreeBSD.ORG Wed Jul 14 00:00:57 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8269016A4CE for ; Wed, 14 Jul 2004 00:00:57 +0000 (GMT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1294443D31 for ; Wed, 14 Jul 2004 00:00:57 +0000 (GMT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.11/8.12.11) with ESMTP id i6E00d3S066475; Tue, 13 Jul 2004 20:00:39 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i6E00cQP066470; Tue, 13 Jul 2004 20:00:38 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Tue, 13 Jul 2004 20:00:37 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: "M. Warner Losh" In-Reply-To: <20040713.174512.11239675.imp@bsdimp.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org cc: phk@phk.freebsd.dk Subject: Re: newbus integration of MOD_QUIESCE (was Re: cvs commit: src/sbin/kldunload kldunload.8 kldunload.c ) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jul 2004 00:00:57 -0000 On Tue, 13 Jul 2004, M. Warner Losh wrote: > The nasty case I've come up with is what happens when the module is idle > (not busy), but becomes busy (not idle) after the MOD_QUIESCE call? > Right now newbus modules that receive a MOD_UNLOAD call attempt to > detach all instances of devices contained in that module. If I have a > way to poll the driver to see if it is busy (which is relatively easy to > implement), then if it becomes busy after the MOD_QUIESCE call, I get a > MOD_UNLOAD which would force instances to detach. So, it sounds like a couple of concepts are floating around: MOD_WEAKUNLOAD - Unload if you're not in use. I.e., unattached driver, unmounted file system, netgraph nodes that aren't instantiated, network protocol without any sockets, etc. Be harmlessly gone, but vetoed at a low cost. MOD_STRONGUNLOAD - Unload even though you're in use. Detach the driver, deadfs the file system, wither the geom, sever the sockets, etc. May cause disruption, but may also veto, depending on the subsystem, especially if the subsytem has no way to notify its consumers of impending doom. Can be vetoed, but try harder before vetoing. Some subsystems might always return EBUSY for this if there's really no way to express "undesirable departure" upwards. MOD_QUIESCE - Attempt MOD_WEAKUNLOAD, and if that fails, ask the module to start draining in some form. I'm a bit unclear on quite what's intended, but this seems to be less atomic notion than "unload, or don't" at various points on the spectrum. I.e., it kicks off a state transition in what is likely a slightly poorly defined state machine. Right now, the state machine is "Not loaded", "Loaded", and we use a lock to prevent intermediate states from colliding. MOD_SHUTDOWN - The system is shutting down, the module better do it too. Since there's no way to say "Um, no", most modules that don't know how to unload just ignore the event and in most cases it's harmless because the system state is toast shortly anyway. MOD_PANIC - Unload the module, regardless of the consequences. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Principal Research Scientist, McAfee Research From owner-freebsd-arch@FreeBSD.ORG Wed Jul 14 00:15:49 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3A41516A4CE; Wed, 14 Jul 2004 00:15:49 +0000 (GMT) Received: from harmony.village.org (rover.village.org [168.103.84.182]) by mx1.FreeBSD.org (Postfix) with ESMTP id B615643D31; Wed, 14 Jul 2004 00:15:48 +0000 (GMT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.11/8.12.11) with ESMTP id i6E0EGgF055972; Tue, 13 Jul 2004 18:14:16 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Tue, 13 Jul 2004 18:14:19 -0600 (MDT) Message-Id: <20040713.181419.82427717.imp@bsdimp.com> To: rwatson@freebsd.org From: "M. Warner Losh" In-Reply-To: References: <20040713.174512.11239675.imp@bsdimp.com> X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: arch@freebsd.org cc: phk@phk.freebsd.dk Subject: Re: newbus integration of MOD_QUIESCE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jul 2004 00:15:49 -0000 In message: Robert Watson writes: : MOD_WEAKUNLOAD - Unload if you're not in use. I.e., unattached driver, : unmounted file system, netgraph nodes that aren't instantiated, network : protocol without any sockets, etc. Be harmlessly gone, but vetoed : at a low cost. This is desirable. : MOD_STRONGUNLOAD - Unload even though you're in use. Detach the driver, : deadfs the file system, wither the geom, sever the sockets, etc. May : cause disruption, but may also veto, depending on the subsystem, : especially if the subsytem has no way to notify its consumers of : impending doom. Can be vetoed, but try harder before vetoing. Some : subsystems might always return EBUSY for this if there's really no way : to express "undesirable departure" upwards. This is tue current MOD_UNLOAD : MOD_QUIESCE - Attempt MOD_WEAKUNLOAD, and if that fails, ask the module to : start draining in some form. I'm a bit unclear on quite what's : intended, but this seems to be less atomic notion than "unload, or : don't" at various points on the spectrum. I.e., it kicks off a state : transition in what is likely a slightly poorly defined state machine. : Right now, the state machine is "Not loaded", "Loaded", and we use a : lock to prevent intermediate states from colliding. This is the heart of my questions about MOD_QUIESCE. Warner From owner-freebsd-arch@FreeBSD.ORG Wed Jul 14 00:23:01 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1F81516A4CE for ; Wed, 14 Jul 2004 00:23:01 +0000 (GMT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8DF5643D1F for ; Wed, 14 Jul 2004 00:23:00 +0000 (GMT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.11/8.12.11) with ESMTP id i6E0MhpD070751; Tue, 13 Jul 2004 20:22:44 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i6E0MhA2070747; Tue, 13 Jul 2004 20:22:43 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Tue, 13 Jul 2004 20:22:43 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: "M. Warner Losh" In-Reply-To: <20040713.181419.82427717.imp@bsdimp.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org cc: phk@phk.freebsd.dk Subject: Re: newbus integration of MOD_QUIESCE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jul 2004 00:23:01 -0000 On Tue, 13 Jul 2004, M. Warner Losh wrote: > : MOD_STRONGUNLOAD - Unload even though you're in use. Detach the driver, > : deadfs the file system, wither the geom, sever the sockets, etc. May > : cause disruption, but may also veto, depending on the subsystem, > : especially if the subsytem has no way to notify its consumers of > : impending doom. Can be vetoed, but try harder before vetoing. Some > : subsystems might always return EBUSY for this if there's really no way > : to express "undesirable departure" upwards. > > This is tue current MOD_UNLOAD Well, sort of. It might be well true of device drivers, but in the world of synthetic network interfaces, etc, the MOD_WEAKUNLOAD semantics apply. > : MOD_QUIESCE - Attempt MOD_WEAKUNLOAD, and if that fails, ask the module to > : start draining in some form. I'm a bit unclear on quite what's > : intended, but this seems to be less atomic notion than "unload, or > : don't" at various points on the spectrum. I.e., it kicks off a state > : transition in what is likely a slightly poorly defined state machine. > : Right now, the state machine is "Not loaded", "Loaded", and we use a > : lock to prevent intermediate states from colliding. > > This is the heart of my questions about MOD_QUIESCE. For some module systems this is free, or at least designed in. In the MAC Framework, an unload request is treated as a blocking quiesce request. A module can block unload if it wants to, but by default, the MAC Framework will wait (possibly a long time) for the framework to get to a point where it can safely unload. I.e., it assumes unload can be made atomic, if you wait long enough. With others, that's clearly not the case. You identified a couple of interesting edge cases, such as "State transition driver into 'shutting down', now do something that requires the driver", but I think there's some other big problems with the state machine. What if an application needs the functionality -- does it ask the driver to change its mind, wait for the driver to unload so it can load it again, or can the unload and load overlap? How long might it have to wait? "Drain, oh, unless I need it" may mean one thing for a driver, but something else for a file system. The nice thing about the current model is that it supports two positions: on, and off. This raises another edge case -- the case today where we load a driver, there's a linking or initialization error, and we leave it loaded in a broken state. It becomes more apparent when you load a driver with a mutex initialization in it's load routine. If you've never given it a try, try loading if_tap.ko into a kernel with the tap interface compiled in. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Principal Research Scientist, McAfee Research From owner-freebsd-arch@FreeBSD.ORG Wed Jul 14 07:51:29 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0819F16A4CE; Wed, 14 Jul 2004 07:51:29 +0000 (GMT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5759043D3F; Wed, 14 Jul 2004 07:51:28 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.11/8.12.11) with ESMTP id i6E7ow2B036245; Wed, 14 Jul 2004 09:50:59 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: "M. Warner Losh" From: "Poul-Henning Kamp" In-Reply-To: Your message of "Tue, 13 Jul 2004 17:45:12 MDT." <20040713.174512.11239675.imp@bsdimp.com> Date: Wed, 14 Jul 2004 09:50:58 +0200 Message-ID: <36244.1089791458@critter.freebsd.dk> Sender: phk@critter.freebsd.dk cc: arch@freebsd.org cc: rwatson@freebsd.org Subject: Re: newbus integration of MOD_QUIESCE (was Re: cvs commit: src/sbin/kldunload kldunload.8 kldunload.c ) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jul 2004 07:51:29 -0000 In message <20040713.174512.11239675.imp@bsdimp.com>, "M. Warner Losh" writes: >I've not seen other documentation that explains the event in more >detail. Nor do I see comments in the code explaining it any better. >>From reading the code and the docs, MOD_QUIESCE is just a 'are you >idle?' question. More like "If you are idle, prepare to be unloaded, otherwise yell". >The nasty case I've come up with is what happens when the module is >idle (not busy), but becomes busy (not idle) after the MOD_QUIESCE >call? That is for the module to handle: if it returns zero to MOD_QUIESCE it knows it will be unloaded and shouldn't accept any new business. Finegrained guidance for what a module should do in MOD_QUISCE: Network driver: If any instance is IF_UP, return EBUSY. Otherwise detach all instances and do not attach to any new instances offered. Device driver: If any instance is opened, return EBUSY. Otherwise destroy_dev all instances and do not attach to any new instances offered. GEOM module: If any provider has non-zero access counts, return EBUSY. Otherwise orphan all providers, destroy or wither all geoms + consumers and do not taste/configure new instances. MOD_UNLOAD on the other hand means: All kinds of module: Clean all instances even if they are in use. Return error only if something cannot be cleaned up, for instance memory references into the module we cannot reclaim. The fact that an in-use instance is cleaned makes other code in the kernel with insufficient error-checks fail is _not_ a reason to return error, the other code should be fixed instead. And as I said earlier: if you give -f to kldunload the MOD_QUISCE error return is ignored and MOD_UNLOAD issued anyway. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Wed Jul 14 07:54:29 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D01A516A4CE; Wed, 14 Jul 2004 07:54:29 +0000 (GMT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2D43143D2F; Wed, 14 Jul 2004 07:54:29 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.11/8.12.11) with ESMTP id i6E7sQLv036296; Wed, 14 Jul 2004 09:54:26 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Robert Watson From: "Poul-Henning Kamp" In-Reply-To: Your message of "Tue, 13 Jul 2004 20:00:37 EDT." Date: Wed, 14 Jul 2004 09:54:26 +0200 Message-ID: <36295.1089791666@critter.freebsd.dk> Sender: phk@critter.freebsd.dk cc: arch@freebsd.org Subject: Re: newbus integration of MOD_QUIESCE (was Re: cvs commit: src/sbin/kldunload kldunload.8 kldunload.c ) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jul 2004 07:54:29 -0000 In message , Robe rt Watson writes: > >On Tue, 13 Jul 2004, M. Warner Losh wrote: > >> The nasty case I've come up with is what happens when the module is idle >> (not busy), but becomes busy (not idle) after the MOD_QUIESCE call? >> Right now newbus modules that receive a MOD_UNLOAD call attempt to >> detach all instances of devices contained in that module. If I have a >> way to poll the driver to see if it is busy (which is relatively easy to >> implement), then if it becomes busy after the MOD_QUIESCE call, I get a >> MOD_UNLOAD which would force instances to detach. > >So, it sounds like a couple of concepts are floating around: I considered most of what you suggest here myself but concluded that since we had been able to get by with only -force in mount for so long, we really only needed two levels: MOD_QUIESCE "stop and prepare for unload if not in use" MOD_UNLOAD "unload unless there are memory references into the module which cannot be invalidated" I did make the kldunloadf() take a flag argument so if people want higher granularity more can be added, but remember that would come at a cost of more code in all modules. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Wed Jul 14 08:54:04 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BC26516A4CE for ; Wed, 14 Jul 2004 08:54:04 +0000 (GMT) Received: from ms-smtp-05-eri0.texas.rr.com (ms-smtp-05.texas.rr.com [24.93.47.44]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5697A43D1D for ; Wed, 14 Jul 2004 08:54:04 +0000 (GMT) (envelope-from liew@jaysern.org) Received: from [192.168.0.17] (cs243162-92.jam.rr.com [24.243.162.92]) i6BHe1Bu021567 for ; Sun, 11 Jul 2004 12:40:01 -0500 (CDT) From: Jay Sern Liew To: freebsd-arch@freebsd.org Content-Type: text/plain Organization: http://liew.jaysern.org Message-Id: <1089567603.746.45.camel@athlon> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.6 Date: Sun, 11 Jul 2004 12:40:04 -0500 Content-Transfer-Encoding: 7bit X-Virus-Scanned: Symantec AntiVirus Scan Engine Subject: increased machine precision for FreeBSD apps? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jul 2004 08:54:04 -0000 Hello guys, This is probably more of an organizational issue than an architectural one. In the event of running applications with heavy floating point arithmetic (e.g. scientific computation apps for one, encryption algorithms, compression, .. etc ), would it be a good idea to decrease rounding errors by increasing the data type size? e.g. When a programmer declares a floating point variable, the compiler/assembler allocates twice(or a multiple of) the space needed, hence increasing the precision(and better approximation?). I realize this is really just a trade off with execution speed and/or efficiency, perhaps unsuitable for real time calculations, but better for those who don't mind waiting a little for better results, with less the hardware. As a small peek into the future if this thing doesn't sound like herecy, we _could_ implement this so that it is transparent to the programmer(doing it at the compiler/assembler level), none of the ports would need to be rewritten, and could potentially benefit from the increased machine precision(where desired). Just an idea for FreeBSD to knock on the scientific community's door. (If 2.5 mil web hosts trusts FreeBSD for reliability, why can't the more mathematically inclined community?) I realize that the real scientists(read: well funded, by the big wigs) would have at their disposal computers that won't be needing this(although they could benefit from this), this would be great for home-brewed calculations, maybe students(computer scientists, mathematicians, chemists, biologists, meteorologists, .. ), or just any DIY-ist. This isn't really a new idea, perhaps comparable to x86-64; sure x86-32's may not have the 64-bit physical address space, but they can still have the 64-bit(or more) precision, right? I apologize if this suggestion has already been brought up before. (I'm not only the kind who speaks, I'd aid in the development of this). -- Jay Sern Liew liew.jaysern.org From owner-freebsd-arch@FreeBSD.ORG Wed Jul 14 08:57:17 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 603BE16A4CE for ; Wed, 14 Jul 2004 08:57:17 +0000 (GMT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id AE56043D2D for ; Wed, 14 Jul 2004 08:57:14 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.11/8.12.11) with ESMTP id i6E8vBlr037501; Wed, 14 Jul 2004 10:57:12 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Jay Sern Liew From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sun, 11 Jul 2004 12:40:04 CDT." <1089567603.746.45.camel@athlon> Date: Wed, 14 Jul 2004 10:57:11 +0200 Message-ID: <37500.1089795431@critter.freebsd.dk> Sender: phk@critter.freebsd.dk cc: freebsd-arch@freebsd.org Subject: Re: increased machine precision for FreeBSD apps? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jul 2004 08:57:17 -0000 In message <1089567603.746.45.camel@athlon>, Jay Sern Liew writes: >Hello guys, > > This is probably more of an organizational issue than an architectural >one. In the event of running applications with heavy floating point >arithmetic (e.g. scientific computation apps for one, encryption >algorithms, compression, .. etc ), would it be a good idea to decrease >rounding errors by increasing the data type size? >e.g. When a programmer declares a floating point variable, the >compiler/assembler allocates twice(or a multiple of) the space needed, >hence increasing the precision(and better approximation?). Most floating point is done in hardware so we are stuck with what the hardware offers mostly. That said "long double" to the extent it is implemented will be enough for any use inside the current physical model of reality. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Wed Jul 14 10:06:10 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 268EA16A4CF for ; Wed, 14 Jul 2004 10:06:10 +0000 (GMT) Received: from mx1.mail.ru (mx1.mail.ru [194.67.23.121]) by mx1.FreeBSD.org (Postfix) with ESMTP id CEBCE43D2F for ; Wed, 14 Jul 2004 10:06:09 +0000 (GMT) (envelope-from metal_man@mail.ru) Received: from [217.118.66.254] (port=31826 helo=METALLER) by mx1.mail.ru with smtp id 1BkgeJ-000OHu-00; Wed, 14 Jul 2004 14:06:05 +0400 Message-ID: <000101c4698a$345f4a40$ef2014ac@METALLER> From: "Dmitriy Startsev" To: "Jay Sern Liew" , References: <1089567603.746.45.camel@athlon> Date: Wed, 14 Jul 2004 14:06:11 +0400 MIME-Version: 1.0 Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 FL-Build: Fidolook 2004 (HL) 6.0.2600.32000 - 17/4/2004 06:54:21 Subject: Re: increased machine precision for FreeBSD apps? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jul 2004 10:06:10 -0000 Hello, Jay! You wrote to on Sun, 11 Jul 2004 12:40:04 -0500: JSL> This is probably more of an organizational issue than an JSL> architectural one. In the event of running applications with heavy JSL> floating point arithmetic (e.g. scientific computation apps for one, JSL> encryption algorithms, compression, .. etc ), would it be a good idea JSL> to decrease rounding errors by increasing the data type size? JSL> e.g. When a programmer declares a floating point variable, the JSL> compiler/assembler allocates twice(or a multiple of) the space needed, JSL> hence increasing the precision(and better approximation?). I realize JSL> this is really just a trade off with execution speed and/or JSL> efficiency, perhaps unsuitable for real time calculations, but better JSL> for those who don't mind waiting a little for better results, with JSL> less the hardware. As a small peek into the future if this thing JSL> doesn't sound like herecy, we _could_ implement this so that it is JSL> transparent to the programmer(doing it at the compiler/assembler JSL> level), none of the ports would need to be rewritten, and could JSL> potentially benefit from the increased machine precision(where JSL> desired). Just an idea for FreeBSD to knock on the scientific JSL> community's door. (If 2.5 mil web hosts trusts FreeBSD for JSL> reliability, why can't the more mathematically inclined community?) JSL> I realize that the real scientists(read: well funded, by the big JSL> wigs) would have at their disposal computers that won't be needing JSL> this(although they could benefit from this), this would be great for JSL> home-brewed calculations, maybe students(computer scientists, JSL> mathematicians, chemists, biologists, meteorologists, .. ), or just JSL> any DIY-ist. This isn't really a new idea, perhaps comparable to JSL> x86-64; sure x86-32's may not have the 64-bit physical address space, JSL> but they can still have the 64-bit(or more) precision, right? I JSL> apologize if this suggestion has already been brought up before. (I'm JSL> not only the kind who speaks, I'd aid in the development of this). IMHO this is a bad idea. Being a programmer myself I have to say I won't ever use a compiler that generates code different from what I expect. Even M$ compiler doesn't take THAT much upon itself:) When someone writes a program, he considers computational precision it requires and implements it using standard C types. If double is not enough, one could use long double. If even that is not enough then one have to consider using different algorithm rather than to try to increase precision. For most of the ports, increased precision will not give any benefits, but will only decrease performance and possibly introduce some serious bugs in software that relies on particular data type sizes. BTW, about x86-64: default operand size is still 32 bits on this architecture (and 64 bit types can be used on 32 bit machines as well). There are international standards related to machine data type representation, compilers, etc. Let's stick to them as much as possible. With best regards, Dmitriy Startsev. From owner-freebsd-arch@FreeBSD.ORG Wed Jul 14 15:03:58 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 82EC816A4CE; Wed, 14 Jul 2004 15:03:58 +0000 (GMT) Received: from harmony.village.org (rover.village.org [168.103.84.182]) by mx1.FreeBSD.org (Postfix) with ESMTP id DC59443D1F; Wed, 14 Jul 2004 15:03:57 +0000 (GMT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.11/8.12.11) with ESMTP id i6EF2MwO064625; Wed, 14 Jul 2004 09:02:22 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Wed, 14 Jul 2004 09:02:22 -0600 (MDT) Message-Id: <20040714.090222.36824783.imp@bsdimp.com> To: phk@phk.freebsd.dk From: "M. Warner Losh" In-Reply-To: <36244.1089791458@critter.freebsd.dk> References: <20040713.174512.11239675.imp@bsdimp.com> <36244.1089791458@critter.freebsd.dk> X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: arch@freebsd.org cc: rwatson@freebsd.org Subject: Re: newbus integration of MOD_QUIESCE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jul 2004 15:03:58 -0000 In message: <36244.1089791458@critter.freebsd.dk> "Poul-Henning Kamp" writes: : In message <20040713.174512.11239675.imp@bsdimp.com>, "M. Warner Losh" writes: : >I've not seen other documentation that explains the event in more : >detail. Nor do I see comments in the code explaining it any better. : >>From reading the code and the docs, MOD_QUIESCE is just a 'are you : >idle?' question. : : More like "If you are idle, prepare to be unloaded, otherwise yell". OK. So there is a race here. The modules have to cope with the lag between the MOD_QUIESCE call and the MOD_UNLOAD that will come later. Can the modules assume that if they get a MOD_QUIESCE that they WILL get a MOD_UNLOAD later if they return 0? : >The nasty case I've come up with is what happens when the module is : >idle (not busy), but becomes busy (not idle) after the MOD_QUIESCE : >call? : : That is for the module to handle: if it returns zero to MOD_QUIESCE : it knows it will be unloaded and shouldn't accept any new business. That's a fairly high standard to live up to. It would mean setting a flag and not accepting any new business. I thought in irc discussions that this was a 'am I busy or not' call, but it is clear that this has side effects. There's no way to ask the module if it is idle or not, so this is just a small increment over what we have now. : Finegrained guidance for what a module should do in MOD_QUISCE: : : Network driver: : : If any instance is IF_UP, return EBUSY. : Otherwise detach all instances and do not attach to : any new instances offered. This sounds a lot like what we call detach in newbus. Yet this is different than what we'd talked about in terms of the 'notify/shutdown' split that we've been designing. This makes it harder to do that. I don't like this guidance. It doesn't map to anything in newbus that we have or we're planning. So this would also act as a gate to all further probe routines returning 'good' as well? That's a lot of side effects. This makes it hard to map into other plans we have for newbus, and hard for us to provide a hook into newbus drivers for this module message. Of course whta we have today is very racy (race inserting a CardBus card and unloading the module), so things aren't made that much worse by thse change, but it doesn't help much either. : Device driver: : : If any instance is opened, return EBUSY. : Otherwise destroy_dev all instances and do not attach to : any new instances offered. Same as the above. Likely with a lot of wrinkles for bus drivers. If pccard were to receive this, does it consider itself busy because it has children drivers? Or should it pass the request down to them? Since the children drivers aren't necessarily modules, we can't just rely on them. I think it should behave the same way as if you'd called MOD_QUIESCE on the child device. Eg Give we have an ep attached to pccard. kldunload if_ep should behave the same way as kldunload pccard. If if_ep is up, then it can veto the quiesce and the unload won't happen. In addition, these sorts of recommendations fundamentally change the way that drivers unload these days. Today, kldunload of a typical network driver will detach it, even if it is up or busy. It is an extraordinary driver that does anything differently. Do you plan on fixing all the drivers/modules in the tree that doesn't do this now? Having inconsistant unload behavior is one of the problems we have in the tree, and it seems like this would make it worse. Also, in preparing this message, I noticed a number of drivers that return EINVAL instead of EOPNOSUPP for MOD_ events they don't understand. There are several that also return 0 for events that they don't understand. These should be corrected in the tree. Do you have plans to do this? : GEOM module: : : If any provider has non-zero access counts, return EBUSY. : Otherwise orphan all providers, destroy or wither all : geoms + consumers and do not taste/configure new instances. : : MOD_UNLOAD on the other hand means: : : All kinds of module: : : Clean all instances even if they are in use. : : Return error only if something cannot be cleaned up, for : instance memory references into the module we cannot reclaim. : : The fact that an in-use instance is cleaned makes other : code in the kernel with insufficient error-checks fail is : _not_ a reason to return error, the other code should be : fixed instead. : : And as I said earlier: if you give -f to kldunload the MOD_QUISCE : error return is ignored and MOD_UNLOAD issued anyway. And none of this helps us to unload idle drivers, but I guess that wasn't your goal. Finally, I'm not trying to be negative here. There are just a lot of issues that haven't been considered yet. I'm a little grumpy that I was specifically told that you didn't want to hear about newbus issues, but I'm trying to be constructive in my comments here none the less. I am looking to implement something in newbus with these messages, so I'm trying to understand it at a high level of detail. Warner From owner-freebsd-arch@FreeBSD.ORG Wed Jul 14 18:25:32 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 229BD16A4CE; Wed, 14 Jul 2004 18:25:32 +0000 (GMT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4AF4843D54; Wed, 14 Jul 2004 18:25:31 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.11/8.12.11) with ESMTP id i6EIPQIs012317; Wed, 14 Jul 2004 20:25:26 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: "M. Warner Losh" From: "Poul-Henning Kamp" In-Reply-To: Your message of "Wed, 14 Jul 2004 09:02:22 MDT." <20040714.090222.36824783.imp@bsdimp.com> Date: Wed, 14 Jul 2004 20:25:25 +0200 Message-ID: <12316.1089829525@critter.freebsd.dk> Sender: phk@critter.freebsd.dk cc: arch@freebsd.org cc: rwatson@freebsd.org Subject: Re: newbus integration of MOD_QUIESCE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jul 2004 18:25:32 -0000 In message <20040714.090222.36824783.imp@bsdimp.com>, "M. Warner Losh" writes: >: More like "If you are idle, prepare to be unloaded, otherwise yell". > >OK. So there is a race here. The modules have to cope with the lag >between the MOD_QUIESCE call and the MOD_UNLOAD that will come later. >Can the modules assume that if they get a MOD_QUIESCE that they WILL >get a MOD_UNLOAD later if they return 0? Yes, they can assume that. There is the footnote that we can have multiple modules in one kld and if one of them vetos the MOD_MODQUIESCE the MOD_UNLOAD may not happen until the user tries to kldunload again or at all. Notice that MOD_UNLOAD can also be veto'ed and that we already have this multi-modules in one KLD issue if one of the modules veto MOD_UNLOAD and the others don't. >That's a fairly high standard to live up to. It would mean setting a >flag and not accepting any new business. I'm not sure I find it a particular high standard to add code to: case MOD_QUIESCE: ... if (error == 0) sorry_we_re_close = 1; return (error) and then check the flag in the probe/attach/taste/config routines. >I thought in irc discussions >that this was a 'am I busy or not' call, but it is clear that this has >side effects. There's no way to ask the module if it is idle or not, >so this is just a small increment over what we have now. Yes. The smallest increment which allowed me to implement -f(orce). >: Finegrained guidance for what a module should do in MOD_QUISCE: >: >: Network driver: >: >: If any instance is IF_UP, return EBUSY. >: Otherwise detach all instances and do not attach to >: any new instances offered. > >This sounds a lot like what we call detach in newbus. Yet this is >different than what we'd talked about in terms of the >'notify/shutdown' split that we've been designing. This makes it >harder to do that. (I should clarify here that the detachment doesn't have to happen until MOD_UNLOAD, the important bit is determining if EBUSY should be returned and if not, prevent new business.) >I don't like this guidance. It doesn't map to >anything in newbus that we have or we're planning. MOD_QUIESCE is designed for kldunload -f, not for newbus so this is hardly surprising. >So this would also act as a gate to all further probe routines >returning 'good' as well? That's a lot of side effects. You lost me there. Once MOD_QUISCE returned zero, all probe routines in the module should fail to recognize things. I would expectat that to mean that they return 'bad', but this may just be a terminology thing. >This makes it hard to map into other plans we have for newbus, and >hard for us to provide a hook into newbus drivers for this module >message. I hate to be so blunt about this, but why is it that you think you need to do that ? Let me explain why I can't see the connection: Imagine that we have a pretty hairy driver which attaches to things on ISA, PCI and S-BUS. MOD_QUIESCE acts on all instances of this driver, no matter which bus it is attached to, and no matter what it attaches to upwards in the kernel (netgraph, geom, netstack, filesystemc, MAC etc). The user wants the kld unloaded, he asks for it, and MOD_QUISCE + MOD_UNLOAD handles this for him. If I have understood anything about what you are talking about, the newbus involvement comes from the event expressing that I pressed the "I'll yank this one" button on a hot-swap PCI slot. That would not and should not affect the instances which sit on the ISA or S-BUS. I fail to see how it can avoid having a device_t argument to tell which instance the event is going to operate on, and further more, I entirely fail to see why you would use a module event to communicate this rather than have newbus call the instance in question directly with a quiesce method. What am I missing ? How is MOD_QUIESCE any different from MOD_UNLOAD with respect to newbus ? >: If any instance is opened, return EBUSY. >: Otherwise destroy_dev all instances and do not attach to >: any new instances offered. > >Same as the above. Likely with a lot of wrinkles for bus drivers. If >pccard were to receive this, does it consider itself busy because it >has children drivers? Or should it pass the request down to them? >Since the children drivers aren't necessarily modules, we can't just >rely on them. I think you have overlooked something fundamental Warner. MOD_QUIESCE only offers the ability to split the current MOD_UNLOAD semantics in two parts so that -f(orce) can override the policy decision part of the action in the module. If pccard today implements MOD_UNLOAD, then you can split that code in two and put half of it in MOD_QUIESCE if you want to implement a -f(orce) semantic for pccard. If modules cannot implement MOD_QUISCE they don't have to. If you don't do that, then it will work exactly the way it does today (provided it returns a sensible value for an unknown events). MOD_UNLOAD has the same semantics as they already have, including the ability to simply not implement unloads because it is too hairy. >In addition, these sorts of recommendations fundamentally change the >way that drivers unload these days. Today, kldunload of a typical >network driver will detach it, even if it is up or busy. It is an >extraordinary driver that does anything differently. No, it doesn't change anything "fundamentally". It gives the module writer the ability to split the MOD_UNLOAD into two so that a -f(orce) semantic can be implemented, and if he doesn't do that, then you have the exact same situation you have today. >Do you plan on fixing all the drivers/modules in the tree that doesn't >do this now? No, MOD_QUIESCE is entirely optional. If you don't implement it there will be no difference between kldunload with or without -f(orce) and that's it. There is no difference today for any driver. >Having inconsistant unload behavior is one of the >problems we have in the tree, and it seems like this would make it >worse. If you want consistent unload behaviour, you need to start out by defining what it should be. For GEOM we currently refuse unloading if anything is busy. This is based on the behaviour of unmount which refuses if any vnodes are in use. This entire thing comes about because sometimes you may want to unload busy GEOM classes, and that's what -f(orce) gives you a lever for. If we have modules which today unload while being in use then I would personally argue that it is a POLA violation, but given the lack of the -f(orce) lever until now, I can fully understand the reason why it got that way. Hopefully now with -f(orce) we can get a consistent behaviour which IMO should be: "kldunload" fails if the module is in use. "kldunload -f" suceeds even if the module is in use. >Also, in preparing this message, I noticed a number of drivers that >return EINVAL instead of EOPNOSUPP for MOD_ events they don't >understand. There are several that also return 0 for events that they >don't understand. These should be corrected in the tree. Do you have >plans to do this? As I find them yes. Currently EOPNOTSUPP and zero are treated as "OK" return from MOD_QUIESCE. You'll notice that I put it in the man-page to define a sensible default behaviour so we can eliminate this particular inconsistency over time. >And none of this helps us to unload idle drivers, but I guess that >wasn't your goal. You mean automatically unload drivers which are not/no longer used ? No, that was nowhere near my goals. If you mean something else you'll have to explain to me what you meant. >Finally, I'm not trying to be negative here. There are just a lot of >issues that haven't been considered yet. I'm a little grumpy that I >was specifically told that you didn't want to hear about newbus >issues, but I'm trying to be constructive in my comments here none the >less. I am looking to implement something in newbus with these >messages, so I'm trying to understand it at a high level of detail. As you can see above I still totally fail to see the newbus connection in this. I can see that implementations of *both* MOD_UNLOAD and MOD_QUIESCE for bus modules could get some help from newbus, but we don't have that today for MOD_UNLOAD as far as I know, so I doubt that MOD_QUIESCE changes that. And mind you, I would far have preferred if there had been a flag argument to ->modevent() so that MOD_UNLOAD could simply have gotten a MOD_UNLOAD_FORCE flag passed in, but fixing that would require changing every single modevent handler in the system and I was not too interested in that. If Robert wants increased granularity as he argued for in one email, then the flag route would probably be necessary, but I don't think he can carry that motion. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Wed Jul 14 19:31:02 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 25EBD16A4CE; Wed, 14 Jul 2004 19:31:02 +0000 (GMT) Received: from harmony.village.org (rover.village.org [168.103.84.182]) by mx1.FreeBSD.org (Postfix) with ESMTP id BEFAC43D60; Wed, 14 Jul 2004 19:30:58 +0000 (GMT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.11/8.12.11) with ESMTP id i6EJSlov067705; Wed, 14 Jul 2004 13:28:47 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Wed, 14 Jul 2004 13:28:47 -0600 (MDT) Message-Id: <20040714.132847.23012460.imp@bsdimp.com> To: phk@phk.freebsd.dk From: "M. Warner Losh" In-Reply-To: <12316.1089829525@critter.freebsd.dk> References: <20040714.090222.36824783.imp@bsdimp.com> <12316.1089829525@critter.freebsd.dk> X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: arch@freebsd.org cc: rwatson@freebsd.org Subject: Re: newbus integration of MOD_QUIESCE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jul 2004 19:31:02 -0000 In message: <12316.1089829525@critter.freebsd.dk> "Poul-Henning Kamp" writes: : In message <20040714.090222.36824783.imp@bsdimp.com>, "M. Warner Losh" writes: : : >: More like "If you are idle, prepare to be unloaded, otherwise yell". : > : >OK. So there is a race here. The modules have to cope with the lag : >between the MOD_QUIESCE call and the MOD_UNLOAD that will come later. : >Can the modules assume that if they get a MOD_QUIESCE that they WILL : >get a MOD_UNLOAD later if they return 0? : : Yes, they can assume that. There is the footnote that we can have : multiple modules in one kld and if one of them vetos the MOD_MODQUIESCE : the MOD_UNLOAD may not happen until the user tries to kldunload again : or at all. We should document that they should. : Notice that MOD_UNLOAD can also be veto'ed and that we already have : this multi-modules in one KLD issue if one of the modules veto : MOD_UNLOAD and the others don't. True. But a failure of one of them doesn't cause us to call MOD_QUIESCE and then fail to call MOD_UNLOAD when it is zero. : >That's a fairly high standard to live up to. It would mean setting a : >flag and not accepting any new business. : : I'm not sure I find it a particular high standard to add code to: : : case MOD_QUIESCE: : ... : if (error == 0) : sorry_we_re_close = 1; : return (error) : : and then check the flag in the probe/attach/taste/config routines. Every single routine will have an extra check? and what happens if the sorry_we_re_closed is set after the check, but before the routine returns? It is unclear how to do this efficiently. Do I have to lock this somehow? Does this force a global lock for each driver? It sounds very simple, but I think that closing the races here might be hard. : >: Finegrained guidance for what a module should do in MOD_QUISCE: : >: : >: Network driver: : >: : >: If any instance is IF_UP, return EBUSY. : >: Otherwise detach all instances and do not attach to : >: any new instances offered. : > : >This sounds a lot like what we call detach in newbus. Yet this is : >different than what we'd talked about in terms of the : >'notify/shutdown' split that we've been designing. This makes it : >harder to do that. : : (I should clarify here that the detachment doesn't have to happen : until MOD_UNLOAD, the important bit is determining if EBUSY should : be returned and if not, prevent new business.) True. newbus can mark the newbus data structures : >I don't like this guidance. It doesn't map to : >anything in newbus that we have or we're planning. : : MOD_QUIESCE is designed for kldunload -f, not for newbus so this : is hardly surprising. Right. However, it was designed without making sure that newbus could do sensible things with it, but I don't want to get into that snarling match right now. : >So this would also act as a gate to all further probe routines : >returning 'good' as well? That's a lot of side effects. : : You lost me there. Once MOD_QUISCE returned zero, all probe : routines in the module should fail to recognize things. I : would expectat that to mean that they return 'bad', but this : may just be a terminology thing. We're saying the same thing: probe in the future should fail. : >This makes it hard to map into other plans we have for newbus, and : >hard for us to provide a hook into newbus drivers for this module : >message. : : I hate to be so blunt about this, but why is it that you think you : need to do that ? Leave the blunt objects at home. I want to do this because I think it is the right thing to do. we have to tell the drivers to quiesce, and it is highly desirable to have one type of quiesce rather than many different flavors that are hard to get right. : Let me explain why I can't see the connection: : : Imagine that we have a pretty hairy driver which attaches to things : on ISA, PCI and S-BUS. : : MOD_QUIESCE acts on all instances of this driver, no matter which bus : it is attached to, and no matter what it attaches to upwards in the : kernel (netgraph, geom, netstack, filesystemc, MAC etc). The user : wants the kld unloaded, he asks for it, and MOD_QUISCE + MOD_UNLOAD : handles this for him. : : If I have understood anything about what you are talking about, the : newbus involvement comes from the event expressing that I pressed : the "I'll yank this one" button on a hot-swap PCI slot. Yes. : That would not and should not affect the instances which sit on : the ISA or S-BUS. : : I fail to see how it can avoid having a device_t argument to tell : which instance the event is going to operate on, and further more, : I entirely fail to see why you would use a module event to communicate : this rather than have newbus call the instance in question directly : with a quiesce method. You are correct. There will be a device_t argument. I won't use a module event to communicate this. However, I will also map the module event to this quiesce, since it is the same concept. : What am I missing ? You are looking at it backwards. I want to map this button to some newbus method. I also want to map the MOD_QUIESCE to this same newbus method if possible to reduce the load on driver writers. They are the same thing. In the case of the pci hotswap example, only one driver would get this message, while on a MOD_QUICESE all device_t's in that module would get the quiesce call. : How is MOD_QUIESCE any different from MOD_UNLOAD with respect to : newbus ? I've already gone into that: it is a mess now and MOD_QUIESECE makes it a little messier and I'm trying to clean up the mess. : >: If any instance is opened, return EBUSY. : >: Otherwise destroy_dev all instances and do not attach to : >: any new instances offered. : > : >Same as the above. Likely with a lot of wrinkles for bus drivers. If : >pccard were to receive this, does it consider itself busy because it : >has children drivers? Or should it pass the request down to them? : >Since the children drivers aren't necessarily modules, we can't just : >rely on them. : : I think you have overlooked something fundamental Warner. : : MOD_QUIESCE only offers the ability to split the current MOD_UNLOAD : semantics in two parts so that -f(orce) can override the : policy decision part of the action in the module. : : If pccard today implements MOD_UNLOAD, then you can split that code : in two and put half of it in MOD_QUIESCE if you want to implement : a -f(orce) semantic for pccard. : : If modules cannot implement MOD_QUISCE they don't have to. : : If you don't do that, then it will work exactly the way it does : today (provided it returns a sensible value for an unknown events). which isn't the case, as I've pointed out. : MOD_UNLOAD has the same semantics as they already have, including : the ability to simply not implement unloads because it is too hairy. I'm trying to clean things up. So arguing that things aren't perfect today isn't relevant. : >In addition, these sorts of recommendations fundamentally change the : >way that drivers unload these days. Today, kldunload of a typical : >network driver will detach it, even if it is up or busy. It is an : >extraordinary driver that does anything differently. : : No, it doesn't change anything "fundamentally". : : It gives the module writer the ability to split the MOD_UNLOAD into : two so that a -f(orce) semantic can be implemented, and if he doesn't : do that, then you have the exact same situation you have today. We view the change differently then. As drivers start to implement this, you get different behavior between those that have been converted and those that haven't. Those that have been converted kldunload turns into 'unload if not busy' while those that haven't are a 'force unload'. This can lead to different behavior and a bad user experience. While MOD_QUIESCE appears to be optional, having different behavior for different modules will be a problem for our users. "when I unload fxp, it always unloads. but for dc I have to unload -f the driver for it to unload if it is attached, why the difference" is the sort of thing I'm trying to avoid. : >Do you plan on fixing all the drivers/modules in the tree that doesn't : >do this now? : : No, MOD_QUIESCE is entirely optional. If you don't implement it : there will be no difference between kldunload with or without -f(orce) : and that's it. There is no difference today for any driver. See abouve. : >Having inconsistant unload behavior is one of the : >problems we have in the tree, and it seems like this would make it : >worse. : : If you want consistent unload behaviour, you need to start out by : defining what it should be. Shouldn't you have done that before this commit? You can't put that back on me since you are the agent of this change. : For GEOM we currently refuse unloading if anything is busy. This is : based on the behaviour of unmount which refuses if any vnodes are in use. : : This entire thing comes about because sometimes you may want to : unload busy GEOM classes, and that's what -f(orce) gives you a : lever for. I understand that. : If we have modules which today unload while being in use then I would : personally argue that it is a POLA violation, but given the lack of : the -f(orce) lever until now, I can fully understand the reason : why it got that way. Yes. However, we have to fix that. Suddenly declaring all modules in the tree to be doing the wrong thing seems a bit excessive. : Hopefully now with -f(orce) we can get a consistent behaviour : which IMO should be: : : "kldunload" fails if the module is in use. : "kldunload -f" suceeds even if the module is in use. I agree that's a good goal. : >Also, in preparing this message, I noticed a number of drivers that : >return EINVAL instead of EOPNOSUPP for MOD_ events they don't : >understand. There are several that also return 0 for events that they : >don't understand. These should be corrected in the tree. Do you have : >plans to do this? : : As I find them yes. : : Currently EOPNOTSUPP and zero are treated as "OK" return from : MOD_QUIESCE. You'll notice that I put it in the man-page to define : a sensible default behaviour so we can eliminate this particular : inconsistency over time. Please do. I'd planned on making a sweep. : >And none of this helps us to unload idle drivers, but I guess that : >wasn't your goal. : : You mean automatically unload drivers which are not/no longer used ? : No, that was nowhere near my goals. If you mean something else : you'll have to explain to me what you meant. No, that's what I mean. : >Finally, I'm not trying to be negative here. There are just a lot of : >issues that haven't been considered yet. I'm a little grumpy that I : >was specifically told that you didn't want to hear about newbus : >issues, but I'm trying to be constructive in my comments here none the : >less. I am looking to implement something in newbus with these : >messages, so I'm trying to understand it at a high level of detail. : : As you can see above I still totally fail to see the newbus connection : in this. Yes. And it is very frurstrating that you don't. I'll explain it again here so you can understand what you are missing. : I can see that implementations of *both* MOD_UNLOAD and MOD_QUIESCE : for bus modules could get some help from newbus, but we don't have : that today for MOD_UNLOAD as far as I know, so I doubt that MOD_QUIESCE : changes that. You are declaring MOD_UNLOAD as doing something different than it does now by fiat. You are slowly making some drivers behave differently, and I say that is going to violate POLA. Finally, I'd like to reiterate that I want to integrate this into newbus and am willing to do the work. I find it rather frustrating that you are attacking me for pointing out the weaknesses in the resulting system after your changes and the subtle changes in semantics that are implied by the mere existance of MOD_QUIESCE. While some of them are true before, this whole thing feels rushed in and I'm trying hard to styfle my frustration at the speed it went into the tree and move forward to make things work in a sensible way. Anyway, I look forward to getting things hammered out. I see a way I can do that right now. I just want to make sure that we clearly define what MOD_QUIESCE expects from the module as well as MOD_UNLOAD so that our users have a consistant experience accross the tree. Warner From owner-freebsd-arch@FreeBSD.ORG Wed Jul 14 19:53:23 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 158DA16A4CE; Wed, 14 Jul 2004 19:53:23 +0000 (GMT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0811B43D1F; Wed, 14 Jul 2004 19:53:22 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.11/8.12.11) with ESMTP id i6EJrHP2014157; Wed, 14 Jul 2004 21:53:17 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: "M. Warner Losh" From: "Poul-Henning Kamp" In-Reply-To: Your message of "Wed, 14 Jul 2004 13:28:47 MDT." <20040714.132847.23012460.imp@bsdimp.com> Date: Wed, 14 Jul 2004 21:53:16 +0200 Message-ID: <14156.1089834796@critter.freebsd.dk> Sender: phk@critter.freebsd.dk cc: arch@freebsd.org cc: rwatson@freebsd.org Subject: Re: newbus integration of MOD_QUIESCE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jul 2004 19:53:23 -0000 In message <20040714.132847.23012460.imp@bsdimp.com>, "M. Warner Losh" writes: >: Yes, they can assume that. There is the footnote that we can have >: multiple modules in one kld and if one of them vetos the MOD_MODQUIESCE >: the MOD_UNLOAD may not happen until the user tries to kldunload again >: or at all. > >We should document that they should. The only way to fix it is to do a three-phase commit and that would be overkill, that's for sure. >: Notice that MOD_UNLOAD can also be veto'ed and that we already have >: this multi-modules in one KLD issue if one of the modules veto >: MOD_UNLOAD and the others don't. > >True. But a failure of one of them doesn't cause us to call >MOD_QUIESCE and then fail to call MOD_UNLOAD when it is zero. Actually we do. If a kld contains two modules and the second one refuses MOD_UNLOAD, the first one already completed MOD_UNLOAD and has destroyed and detached everything whereas the second is still entirely functional. >Every single routine will have an extra check? and what happens if >the sorry_we_re_closed is set after the check, but before the routine >returns? It is unclear how to do this efficiently. Do I have to lock >this somehow? Does this force a global lock for each driver? It >sounds very simple, but I think that closing the races here might be >hard. This is no different from MOD_UNLOAD warner. All these problems exist today already and we can only fix them with proper locking, newbus cannot solve that problem for us either. >You are looking at it backwards. I want to map this button to some >newbus method. I also want to map the MOD_QUIESCE to this same newbus >method if possible to reduce the load on driver writers. They are the >same thing. In the case of the pci hotswap example, only one driver >would get this message, while on a MOD_QUICESE all device_t's in that >module would get the quiesce call. That is not enough. You need to send the event to "all device_t's in all modules in the kld". And the problem with sending the existing instances a newbus event is that you still need to send another event to stop the probe from discovering new things before you do the unload. And then you still need to send MOD_QUIESCE to the module as such for the case where it doesn't use newbus. >: How is MOD_QUIESCE any different from MOD_UNLOAD with respect to >: newbus ? > >I've already gone into that: it is a mess now and MOD_QUIESECE makes >it a little messier and I'm trying to clean up the mess. I think you are trying to hard to collaps two things which are not the same. It's a mess now and MOD_QUIESCE gives us a way to clean up the mess. >We view the change differently then. As drivers start to implement >this, you get different behavior between those that have been >converted and those that haven't. Those that have been converted >kldunload turns into 'unload if not busy' while those that haven't are >a 'force unload'. This can lead to different behavior and a bad user >experience. While MOD_QUIESCE appears to be optional, having >different behavior for different modules will be a problem for our >users. Yes, that is why I added a lever we could use to make the two different but equally disirable behaviours coexist, rather than being haphazardly chosen on a module(-family) by module(-family) basis as they are today. >: If you want consistent unload behaviour, you need to start out by >: defining what it should be. > >Shouldn't you have done that before this commit? You can't put that >back on me since you are the agent of this change. I did, see: >: For GEOM we currently refuse unloading if anything is busy. This is >: based on the behaviour of unmount which refuses if any vnodes are in use. >: >: This entire thing comes about because sometimes you may want to >: unload busy GEOM classes, and that's what -f(orce) gives you a >: lever for. >: >: If we have modules which today unload while being in use then I would >: personally argue that it is a POLA violation, but given the lack of >: the -f(orce) lever until now, I can fully understand the reason >: why it got that way. > >Yes. However, we have to fix that. Suddenly declaring all modules in >the tree to be doing the wrong thing seems a bit excessive. I didn't declare a single module as doing the wrong thing, backwards compatibility is preserved. >: Hopefully now with -f(orce) we can get a consistent behaviour >: which IMO should be: >: >: "kldunload" fails if the module is in use. >: "kldunload -f" suceeds even if the module is in use. > >I agree that's a good goal. Cool, then at least we agree on one thing :-) >: >And none of this helps us to unload idle drivers, but I guess that >: >wasn't your goal. >: >: You mean automatically unload drivers which are not/no longer used ? >: No, that was nowhere near my goals. If you mean something else >: you'll have to explain to me what you meant. > >No, that's what I mean. Could you explain how you want this to work ? Is it only for device drivers or also for kld's like the linux-o-lator and GEOM classes ? Would it only happen when the hardware disappears or would it make it impossible to preload a device driver and provide the hardware later ? >While some of them are true before, this whole thing feels rushed in >and I'm trying hard to styfle my frustration at the speed it went into >the tree and move forward to make things work in a sensible way. There's nothing rushed about this. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Thu Jul 15 15:39:50 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3C89D16A4CE for ; Thu, 15 Jul 2004 15:39:50 +0000 (GMT) Received: from VARK.homeunix.com (adsl-69-107-108-110.dsl.pltn13.pacbell.net [69.107.108.110]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0BDBD43D1D for ; Thu, 15 Jul 2004 15:39:50 +0000 (GMT) (envelope-from das@FreeBSD.ORG) Received: from VARK.homeunix.com (localhost [127.0.0.1]) by VARK.homeunix.com (8.12.11/8.12.10) with ESMTP id i6FFdKFG033802; Thu, 15 Jul 2004 08:39:20 -0700 (PDT) (envelope-from das@FreeBSD.ORG) Received: (from das@localhost) by VARK.homeunix.com (8.12.11/8.12.10/Submit) id i6FFdKRS033801; Thu, 15 Jul 2004 08:39:20 -0700 (PDT) (envelope-from das@FreeBSD.ORG) Date: Thu, 15 Jul 2004 08:39:20 -0700 From: David Schultz To: Jay Sern Liew Message-ID: <20040715153920.GA33631@VARK.homeunix.com> Mail-Followup-To: Jay Sern Liew , freebsd-arch@FreeBSD.ORG References: <1089567603.746.45.camel@athlon> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1089567603.746.45.camel@athlon> cc: freebsd-arch@FreeBSD.ORG Subject: Re: increased machine precision for FreeBSD apps? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Jul 2004 15:39:50 -0000 On Sun, Jul 11, 2004, Jay Sern Liew wrote: > This is probably more of an organizational issue than an architectural > one. In the event of running applications with heavy floating point > arithmetic (e.g. scientific computation apps for one, encryption > algorithms, compression, .. etc ), would it be a good idea to decrease > rounding errors by increasing the data type size? Most programmers expect a float to be an IEEE 754 single-precision number, and they expect a double to be a double-precision number. On most modern FPUs, however, floats, doubles, and sometimes even long doubles take about the same number of clock cycles. The only good reason to use float is for things like genomics, where you're doing calculations on gigabytes of data, and the goal is to fit as much of it in memory as possible. So here's how you handle the problem. If you want to use the largest type that's at least as big as a float, use C99's float_t instead. If you want the largest type that's at least as big as a double, use double_t. If you want the largest type possible, at the expense of having to emulate it on architectures such as PowerPC, use long double. If you want to run one of those scientific apps that phk says don't exist, use an arbitrary-precision arithmetic package such as Gnu MP, Paul Rouse's public domain clone thereof, QLIB, or Mathematica. From owner-freebsd-arch@FreeBSD.ORG Fri Jul 16 01:25:50 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A539416A4CE; Fri, 16 Jul 2004 01:25:50 +0000 (GMT) Received: from out005.verizon.net (out005pub.verizon.net [206.46.170.143]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1DC9F43D1D; Fri, 16 Jul 2004 01:25:50 +0000 (GMT) (envelope-from babkin@bellatlantic.net) Received: from bellatlantic.net ([141.153.250.165]) by out005.verizon.net (InterMail vM.5.01.06.06 201-253-122-130-106-20030910) with ESMTP id <20040716012548.IQNJ3910.out005.verizon.net@bellatlantic.net>; Thu, 15 Jul 2004 20:25:48 -0500 Sender: root@FreeBSD.ORG Message-ID: <40F72E9A.D602C6EB@bellatlantic.net> Date: Thu, 15 Jul 2004 21:25:46 -0400 From: Sergey Babkin X-Mailer: Mozilla 4.7 [en] (X11; U; FreeBSD 4.7-RELEASE i386) X-Accept-Language: en, ru MIME-Version: 1.0 To: Robert Watson References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Authentication-Info: Submitted using SMTP AUTH at out005.verizon.net from [141.153.250.165] at Thu, 15 Jul 2004 20:25:47 -0500 cc: arch@freebsd.org cc: phk@phk.freebsd.dk Subject: Re: newbus integration of MOD_QUIESCE (was Re: cvs commit:src/sbin/kldunload kldunload.8 kldunload.c ) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jul 2004 01:25:50 -0000 Robert Watson wrote: > > So, it sounds like a couple of concepts are floating around: > > MOD_WEAKUNLOAD - Unload if you're not in use. I.e., unattached > > MOD_STRONGUNLOAD - Unload even though you're in use. Detach the Or add an extra argument "weak" to MOD_UNLOAD. -SB From owner-freebsd-arch@FreeBSD.ORG Fri Jul 16 20:08:47 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8173D16A4CE for ; Fri, 16 Jul 2004 20:08:47 +0000 (GMT) Received: from fidel.freesurf.fr (fidel.freesurf.fr [212.43.206.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5463B43D31 for ; Fri, 16 Jul 2004 20:08:47 +0000 (GMT) (envelope-from nsouch@free.fr) Received: from smtp.freesurf.fr (62-240-249-21.adsl.freesurf.fr [62.240.249.21]) by fidel.freesurf.fr (Postfix) with SMTP id D1A552A52E8 for ; Fri, 16 Jul 2004 22:08:45 +0200 (CEST) Received: (qmail 39241 invoked by uid 1001); 16 Jul 2004 20:08:48 -0000 Date: Fri, 16 Jul 2004 22:08:48 +0200 From: Nicolas Souchu To: freebsd-arch@freebsd.org Message-ID: <20040716220848.A35405@armor.freesurf.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i cc: rmh@debian.org Subject: some PRs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jul 2004 20:08:47 -0000 Hi, I'm planning to manage the PRs emitted by the Debian FreeBSD effort: http://www.nl.debian.org/ports/freebsd/ Patchs like: http://www.freebsd.org/cgi/query-pr.cgi?pr=68961 I'd like to work as follows: - test/review the patches - present patches to Freebsd developers on a regular basis by the way of wiki pages or/and submitions to the lists - outline patches that require discussion before commit - commit them - maintain them Depending on the patch size, some work may or may not take place on Perforce first. Opinions? Nicholas -- Nicholas Souchu - nsouch@free.fr - nsouch@FreeBSD.org http://www.freebsd.org/~nsouch/kgi4BSD From owner-freebsd-arch@FreeBSD.ORG Fri Jul 16 21:34:10 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DC25116A4CE for ; Fri, 16 Jul 2004 21:34:10 +0000 (GMT) Received: from mail.soaustin.net (mail.soaustin.net [207.200.4.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id A8AF643D53 for ; Fri, 16 Jul 2004 21:34:10 +0000 (GMT) (envelope-from linimon@lonesome.com) Received: by mail.soaustin.net (Postfix, from userid 502) id 18D11148C3; Fri, 16 Jul 2004 16:34:10 -0500 (CDT) Date: Fri, 16 Jul 2004 16:34:09 -0500 (CDT) From: Mark Linimon X-X-Sender: linimon@pancho To: Nicolas Souchu In-Reply-To: <20040716220848.A35405@armor.freesurf.fr> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: rmh@debian.org cc: freebsd-arch@freebsd.org Subject: Re: some PRs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jul 2004 21:34:11 -0000 On Fri, 16 Jul 2004, Nicolas Souchu wrote: > Depending on the patch size, some work may or may not take place > on Perforce first. > > Opinions? My only concern is that we may be getting close enough to the code freeze for 5-STABLE, which has so many other areas that need stabilization first, that we might want to consider holding off until 6-CURRENT is branched. I'm guessing, from reading some of the bullet-item summaries, that our ability to ensure stabilization is going to be sufficiently challenged as it is :-) This should probably be a re@ call. I do not remember, myself, when the freeze is scheduled for. mcl From owner-freebsd-arch@FreeBSD.ORG Fri Jul 16 21:51:24 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 946B716A4CE for ; Fri, 16 Jul 2004 21:51:24 +0000 (GMT) Received: from fidel.freesurf.fr (fidel.freesurf.fr [212.43.206.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3BA8643D53 for ; Fri, 16 Jul 2004 21:51:22 +0000 (GMT) (envelope-from nsouch@free.fr) Received: from smtp.freesurf.fr (62-240-249-21.adsl.freesurf.fr [62.240.249.21]) by fidel.freesurf.fr (Postfix) with SMTP id C92302A5449 for ; Fri, 16 Jul 2004 23:51:20 +0200 (CEST) Received: (qmail 39786 invoked by uid 1001); 16 Jul 2004 21:51:23 -0000 Date: Fri, 16 Jul 2004 23:51:23 +0200 From: Nicolas Souchu To: Mark Linimon Message-ID: <20040716235123.A39764@armor.freesurf.fr> References: <20040716220848.A35405@armor.freesurf.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from linimon@lonesome.com on Fri, Jul 16, 2004 at 04:34:09PM -0500 cc: rmh@debian.org cc: freebsd-arch@freebsd.org Subject: Re: some PRs X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jul 2004 21:51:24 -0000 Hi Mark, On Fri, Jul 16, 2004 at 04:34:09PM -0500, Mark Linimon wrote: > On Fri, 16 Jul 2004, Nicolas Souchu wrote: > > > Depending on the patch size, some work may or may not take place > > on Perforce first. > > > > Opinions? > > My only concern is that we may be getting close enough to the code > freeze for 5-STABLE, which has so many other areas that need > stabilization first, that we might want to consider holding off > until 6-CURRENT is branched. I'm guessing, from reading some of > the bullet-item summaries, that our ability to ensure stabilization > is going to be sufficiently challenged as it is :-) Indeed. > This should probably be a re@ call. I do not remember, myself, > when the freeze is scheduled for. August or so? But definitely. I'm not targeting immediate releases but later ones and at least start gathering their work in the meantime. Nicholas -- Nicholas Souchu - nsouch@free.fr - nsouch@FreeBSD.org http://www.freebsd.org/~nsouch/kgi4BSD