From owner-freebsd-arch Sun Apr 1 7:55:55 2001 Delivered-To: freebsd-arch@freebsd.org Received: from homer.softweyr.com (bsdconspiracy.net [208.187.122.220]) by hub.freebsd.org (Postfix) with ESMTP id B717937B71A for ; Sun, 1 Apr 2001 07:55:50 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from [127.0.0.1] (helo=softweyr.com ident=49497880e1b07c91fa14a3966962448d) by homer.softweyr.com with esmtp (Exim 3.16 #1) id 14jjGO-0000B1-00; Sun, 01 Apr 2001 08:55:32 -0600 Message-ID: <3AC74164.252F2166@softweyr.com> Date: Sun, 01 Apr 2001 08:55:32 -0600 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.75 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: David O'Brien Cc: Cy Schubert - ITSD Open Systems Group , Jack Rusher , Neil Blakey-Milner , Michael Lucas , "litos2001@libero.it" Subject: Re: Startup scripts a la NetBSD References: <3AC4B808.9EB5806B@integratus.com> <200103301810.f2UIAGK06787@cwsys.cwsent.com> <20010330115252.A93566@dragon.nuxi.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG David O'Brien wrote: > > On Fri, Mar 30, 2001 at 10:09:39AM -0800, Cy Schubert - ITSD Open Systems Group wrote: > > Becoming more SYSV-like is not something most people in FreeBSD are > > enamoured with. Hence I don't think it would be possible to implement > > all the changes I would like to see. > > Correct. Make that HELL NO WE WON'T GO SVR4!!! > > The BSD init + NetBSD granular rc files is a nice compromise. And thus > why I am pushing a move in that direction. And is pretty much what I've done for DoBox. We start virtually every server daemon on the box from an individual rc script. Ours handles the usual start and stop arguments, as well as init, which is run only during the very first system boot -- this allows packages that were installed in the system image to complete their post-install phase. We also have netstop and netstart dynamic events for when the internet interface goes down and comes back up, so we can reconfigure firewall settings, register dynamic dns, etc., if the IP address changes. We have three events related to system backup: backup, pre-restore, and post-restore, where subsystems can export data into a backup-able format, stop servers, and restart servers and import restored data. So far, it is working pretty well. I can't imagine trying to do something like this by editing a single rc file. We run 'stop' and 'restore' events in reverse alpha order and all other events in alphabetical order, making it simple to sequence the events. So far, we have only two dependencies, both of which need to be started early and stopped late, so we simply capitalized those two scripts and leave the rest lower-case. Producing a sorted list from FreeBSD's multiple rc.d directories is a simple enough process this shouldn't cause us any problems. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Apr 1 8:25:18 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 4AFE637B719 for ; Sun, 1 Apr 2001 08:25:16 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.1/8.11.1) with SMTP id f31FP2h74091; Sun, 1 Apr 2001 11:25:03 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Sun, 1 Apr 2001 11:25:02 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Kirk McKusick Cc: arch@freebsd.org Subject: Re: Background Fsck In-Reply-To: <200103290522.VAA06966@beastie.mckusick.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Wed, 28 Mar 2001, Kirk McKusick wrote: > The implementation will be to add a new option, -B, to the front end, > fsck. The startup scripts will invoke fsck twice, once at the current > location without -B to do all the foreground checks, and once towards > the end of the startup script with -B to start the background checks. On further thought, I'd like to propose the following alternative behavior: fsck -F # perform foreground checks fsck -B # perform background checks fsck # perform all checks I.e., the rc scripts will invoke first using fsck -F, then fsck -B (feel free to twiddle letters here). However, if a user simply invokes fsck by itself (or with other arguments, such as -p) in single-user mode, the full checks will be done. This preserves the sanity of long-lasting documentation that indicates simply running fsck or fsck -p on hitting single-user-mode is sufficient to restore a file system to normal functionality. In this way, it's possible also for the default behavior without special flags to block until the file systems are ready, the currently expected behavior. It's possible I'm misunderstanding an aspect of the way you've described a flags that makes my suggestion redundant, if so, let me know :-). Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Apr 1 16:42:25 2001 Delivered-To: freebsd-arch@freebsd.org Received: from beastie.mckusick.com (beastie.mckusick.com [209.31.233.184]) by hub.freebsd.org (Postfix) with ESMTP id 5C82137B718; Sun, 1 Apr 2001 16:42:19 -0700 (PDT) (envelope-from mckusick@mckusick.com) Received: from beastie.mckusick.com (localhost [127.0.0.1]) by beastie.mckusick.com (8.9.3/8.9.3) with ESMTP id QAA19845; Sun, 1 Apr 2001 16:42:17 -0700 (PDT) (envelope-from mckusick@beastie.mckusick.com) Message-Id: <200104012342.QAA19845@beastie.mckusick.com> To: Robert Watson Subject: Re: Background Fsck Cc: arch@freebsd.org In-Reply-To: Your message of "Sun, 01 Apr 2001 11:25:02 EDT." Date: Sun, 01 Apr 2001 16:42:17 -0700 From: Kirk McKusick Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Date: Sun, 1 Apr 2001 11:25:02 -0400 (EDT) From: Robert Watson To: Kirk McKusick cc: arch@freebsd.org Subject: Re: Background Fsck On Wed, 28 Mar 2001, Kirk McKusick wrote: > The implementation will be to add a new option, -B, to the front end, > fsck. The startup scripts will invoke fsck twice, once at the current > location without -B to do all the foreground checks, and once towards > the end of the startup script with -B to start the background checks. On further thought, I'd like to propose the following alternative behavior: fsck -F # perform foreground checks fsck -B # perform background checks fsck # perform all checks I.e., the rc scripts will invoke first using fsck -F, then fsck -B (feel free to twiddle letters here). However, if a user simply invokes fsck by itself (or with other arguments, such as -p) in single-user mode, the full checks will be done. This preserves the sanity of long-lasting documentation that indicates simply running fsck or fsck -p on hitting single-user-mode is sufficient to restore a file system to normal functionality. In this way, it's possible also for the default behavior without special flags to block until the file systems are ready, the currently expected behavior. It's possible I'm misunderstanding an aspect of the way you've described a flags that makes my suggestion redundant, if so, let me know :-). Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services Your suggest is excellent. I will do it as you suggest. Kirk To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Apr 1 18:12:32 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail.viasoft.com.cn (unknown [61.153.1.177]) by hub.freebsd.org (Postfix) with ESMTP id DA19137B719 for ; Sun, 1 Apr 2001 18:12:28 -0700 (PDT) (envelope-from bsddiy@21cn.com) Received: from William ([192.168.1.98]) by mail.viasoft.com.cn (8.9.3/8.9.3) with ESMTP id JAA22944; Mon, 2 Apr 2001 09:09:29 +0800 Date: Mon, 2 Apr 2001 09:16:18 +0800 From: David Xu X-Mailer: The Bat! (v1.48f) Personal Organization: Viasoft X-Priority: 3 (Normal) Message-ID: <1221561234.20010402091618@21cn.com> To: "David O'Brien" Cc: Cy Schubert - ITSD Open Systems Group , Jack Rusher , Neil Blakey-Milner , Michael Lucas , "litos2001@libero.it" Subject: Re[2]: Startup scripts a la NetBSD In-reply-To: <20010330115252.A93566@dragon.nuxi.com> References: <3AC4B808.9EB5806B@integratus.com> <200103301810.f2UIAGK06787@cwsys.cwsent.com> <20010330115252.A93566@dragon.nuxi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Hello David, Saturday, March 31, 2001, 3:52:52 AM, you wrote: DOB> On Fri, Mar 30, 2001 at 10:09:39AM -0800, Cy Schubert - ITSD Open Systems Group wrote: >> Becoming more SYSV-like is not something most people in FreeBSD are >> enamoured with. Hence I don't think it would be possible to implement >> all the changes I would like to see. DOB> Correct. Make that HELL NO WE WON'T GO SVR4!!! DOB> The BSD init + NetBSD granular rc files is a nice compromise. And thus DOB> why I am pushing a move in that direction. In the past, I saw lots of people refused to see rc.d style in FreeBSD, include some guys in core team, at that time, I'm very disappointed, why is it back again? is this just because of NetBSD? -- David Xu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Apr 2 9:18:39 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail.wgate.com (mail.wgate.com [38.219.83.4]) by hub.freebsd.org (Postfix) with ESMTP id 15F8337B71A for ; Mon, 2 Apr 2001 09:18:36 -0700 (PDT) (envelope-from rjesup@wgate.com) Received: from jesup.eng.tvol.net ([10.32.2.26]) by mail.wgate.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id H70XL0RA; Mon, 2 Apr 2001 12:18:34 -0400 Reply-To: Randell Jesup To: Kirk McKusick Cc: Randell Jesup , Bakul Shah , arch@FreeBSD.ORG Subject: Re: Background Fsck References: <200103302338.PAA11228@beastie.mckusick.com> From: Randell Jesup Date: 02 Apr 2001 12:21:02 -0400 In-Reply-To: Kirk McKusick's message of "Fri, 30 Mar 2001 15:38:42 -0800" Message-ID: User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.7 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Kirk McKusick writes: >As far as I can tell, you have gone through and listed every >error exit in fsck... Most of these can only arise if the logic >is broken in some fundamental way. For example: Agreed, as I stated: :From the current source - I know that many of these are internal errors :that shouldn't happen, and are correct to exit out on. However, some :(like inoinfo(), ginode(), maybe getnextinode()) should NOT just cause :a blanket exit. >Similarly, inode numbers >that are out of bounds should have been discovered at a >higher level and dealt with (e.g., an out of bounds directory >entry should have been found and zapped). That is definitely not the case however; I found that out the hard way when a coworkers disk got scribbled on. I literally had to modify the source to FSCK to ignore those in order to recover the files that hadn't been trashed. > I do not disagree >that there may be some possibilities that slip through, but >going through and wholesale getting rid of error exits is >not the right approach. In general `fsck -p' will not fix >everything, but `fsck -y' should always succeed (though >success may be an empty filesystem). "fsck -y" does not always succeed (though I agree it should). The points I listed do not ask a question and exit regardless. You're correct that quite a few cannot happen unless there's a bug in fsck somewhere, but some (especially inoinfo() and ginode()) can and do happen in the case of true corruption. -- Randell Jesup, Worldgate Communications, ex-Scala, ex-Amiga OS team ('88-94) rjesup@wgate.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Apr 2 12:37:56 2001 Delivered-To: freebsd-arch@freebsd.org Received: from srv1.cosmo-project.de (srv1.cosmo-project.de [213.83.6.106]) by hub.freebsd.org (Postfix) with ESMTP id 1644137B719 for ; Mon, 2 Apr 2001 12:37:49 -0700 (PDT) (envelope-from ticso@mail.cicely.de) Received: from mail.cicely.de (cicely20 [10.1.1.22]) by srv1.cosmo-project.de (8.11.0/8.11.0) with ESMTP id f32Jbjl36131; Mon, 2 Apr 2001 21:37:46 +0200 (CEST) Received: (from ticso@localhost) by mail.cicely.de (8.11.0/8.11.0) id f32Jc5n13611; Mon, 2 Apr 2001 21:38:05 +0200 (CEST) Date: Mon, 2 Apr 2001 21:38:04 +0200 From: Bernd Walter To: Randell Jesup Cc: Kirk McKusick , Bakul Shah , arch@FreeBSD.ORG Subject: Re: Background Fsck Message-ID: <20010402213804.A13223@cicely20.cicely.de> References: <200103302338.PAA11228@beastie.mckusick.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from rjesup@wgate.com on Mon, Apr 02, 2001 at 12:21:02PM -0400 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Mon, Apr 02, 2001 at 12:21:02PM -0400, Randell Jesup wrote: > "fsck -y" does not always succeed (though I agree it should). The > points I listed do not ask a question and exit regardless. You're correct > that quite a few cannot happen unless there's a bug in fsck somewhere, but > some (especially inoinfo() and ginode()) can and do happen in the case of > true corruption. One of the points where it fails if when it wants to put an inode into lost+found but fails to extend/create the directory. Unfortunately fsck -y doesn't do anything after such a situation happened. I usually ended in such situation with fsdb clearing the inode manualy and fsck -y finaly succeded. Well my expiriences were with early softupdates implementations and broken tagged firmware on a scsi drive but however this brokeness to the filesystem was introduced fsck -y wasn't able to fix. -- B.Walter COSMO-Project http://www.cosmo-project.de ticso@cicely.de Usergroup info@cosmo-project.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Apr 2 13:39:15 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail.wgate.com (mail.wgate.com [38.219.83.4]) by hub.freebsd.org (Postfix) with ESMTP id 6834637B724 for ; Mon, 2 Apr 2001 13:39:12 -0700 (PDT) (envelope-from rjesup@wgate.com) Received: from jesup.eng.tvol.net ([10.32.2.26]) by mail.wgate.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id H70XMBSJ; Mon, 2 Apr 2001 16:39:13 -0400 Reply-To: Randell Jesup To: Bernd Walter Cc: Kirk McKusick , Bakul Shah , arch@FreeBSD.ORG Subject: Re: Background Fsck References: <200103302338.PAA11228@beastie.mckusick.com> <20010402213804.A13223@cicely20.cicely.de> From: Randell Jesup Date: 02 Apr 2001 16:41:42 -0400 In-Reply-To: Bernd Walter's message of "Mon, 2 Apr 2001 21:38:04 +0200" Message-ID: User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.7 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Bernd Walter writes: >On Mon, Apr 02, 2001 at 12:21:02PM -0400, Randell Jesup wrote: >> "fsck -y" does not always succeed (though I agree it should). The >> points I listed do not ask a question and exit regardless. You're correct >> that quite a few cannot happen unless there's a bug in fsck somewhere, but >> some (especially inoinfo() and ginode()) can and do happen in the case of >> true corruption. > >One of the points where it fails if when it wants to put an inode into >lost+found but fails to extend/create the directory. >Unfortunately fsck -y doesn't do anything after such a situation happened. >I usually ended in such situation with fsdb clearing the inode manualy >and fsck -y finaly succeded. Exactly. Now multiply by a few thousand trashed inodes (which was what I faced - though in ginode()/etc, not in creating the lost+found, I think - I could be wrong; it was a year ago). -- Randell Jesup, Worldgate Communications, ex-Scala, ex-Amiga OS team ('88-94) rjesup@wgate.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Apr 2 15:26:45 2001 Delivered-To: freebsd-arch@freebsd.org Received: from resnet.uoregon.edu (resnet.uoregon.edu [128.223.122.47]) by hub.freebsd.org (Postfix) with ESMTP id 99E3337B71F for ; Mon, 2 Apr 2001 15:26:41 -0700 (PDT) (envelope-from dwhite@resnet.uoregon.edu) Received: from localhost (dwhite@localhost) by resnet.uoregon.edu (8.10.1/8.10.1) with ESMTP id f32MQAM38922; Mon, 2 Apr 2001 15:26:10 -0700 (PDT) Date: Mon, 2 Apr 2001 15:26:10 -0700 (PDT) From: Doug White To: David Xu Cc: "David O'Brien" , Cy Schubert - ITSD Open Systems Group , Jack Rusher , Neil Blakey-Milner , Michael Lucas , "litos2001@libero.it" Subject: Re[2]: Startup scripts a la NetBSD In-Reply-To: <1221561234.20010402091618@21cn.com> Message-ID: X-All-Your-Base: are belong to us MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Mon, 2 Apr 2001, David Xu wrote: > In the past, I saw lots of people refused to see rc.d style in > FreeBSD, include some guys in core team, at that time, I'm very > disappointed, why is it back again? is this just because of NetBSD? Because sysv-style init scripts actually make sense for both admins and installed packages? It's not traditionally BSD, but unfortunately programs are much more complex today, generally with multiple daemons and interdependencies for starting up properly. A vendor-supplied script makes sure everything is set up to start the daemons/packages correctly, and gives a nice start/stop knob for the admin to control it with. For the record, I hacked up this pile of scripts to emulate one runlevel, along with a linux-style chkconfig(8) script. You just have to call the start-rc3.sh script at some point in the boot, and stop-rc3.sh when shutting down. Since we have rc.shutdown.d you don't have to edit rc.shutdown anymore. If you really want Linnex-style /etc/rc.d/init.d/ and friends, this will give it to you. It should be in ftp://ftp.freebsd.org/pub/FreeBSD/ports/local-distfiles/dwhite/sysvrc-1.1.tar.gz, or any other MASTER_SITE_LOCAL mirror. Doug White | FreeBSD: The Power to Serve dwhite@resnet.uoregon.edu | www.FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Apr 2 17:39:56 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail.viasoft.com.cn (unknown [61.153.1.177]) by hub.freebsd.org (Postfix) with ESMTP id 45D8537B71A for ; Mon, 2 Apr 2001 17:39:53 -0700 (PDT) (envelope-from bsddiy@21cn.com) Received: from William ([192.168.1.98]) by mail.viasoft.com.cn (8.9.3/8.9.3) with ESMTP id IAA29134; Tue, 3 Apr 2001 08:36:16 +0800 Date: Tue, 3 Apr 2001 08:42:56 +0800 From: David Xu X-Mailer: The Bat! (v1.48f) Personal Organization: Viasoft X-Priority: 3 (Normal) Message-ID: <39898261.20010403084256@21cn.com> To: Doug White Cc: "David O'Brien" , Cy Schubert - ITSD Open Systems Group , Jack Rusher , Neil Blakey-Milner , Michael Lucas , "litos2001@libero.it" Subject: Re[3]: Startup scripts a la NetBSD In-reply-To: References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Hello Doug, Tuesday, April 03, 2001, 6:26:10 AM, you wrote: DW> On Mon, 2 Apr 2001, David Xu wrote: >> In the past, I saw lots of people refused to see rc.d style in >> FreeBSD, include some guys in core team, at that time, I'm very >> disappointed, why is it back again? is this just because of NetBSD? DW> Because sysv-style init scripts actually make sense for both admins and DW> installed packages? It's not traditionally BSD, but unfortunately programs DW> are much more complex today, generally with multiple daemons and DW> interdependencies for starting up properly. A vendor-supplied script DW> makes sure everything is set up to start the daemons/packages correctly, DW> and gives a nice start/stop knob for the admin to control it with. DW> For the record, I hacked up this pile of scripts to emulate one runlevel, DW> along with a linux-style chkconfig(8) script. You just have to call the DW> start-rc3.sh script at some point in the boot, and stop-rc3.sh when DW> shutting down. Since we have rc.shutdown.d you don't have to edit DW> rc.shutdown anymore. If you really want Linnex-style /etc/rc.d/init.d/ DW> and friends, this will give it to you. DW> It should be in DW> ftp://ftp.freebsd.org/pub/FreeBSD/ports/local-distfiles/dwhite/sysvrc-1.1.tar.gz, DW> or any other MASTER_SITE_LOCAL mirror. DW> Doug White | FreeBSD: The Power to Serve DW> dwhite@resnet.uoregon.edu | www.FreeBSD.org I like the idea, it has unique start/stop interface, I needn't remember obscure parameters for every daemon, it really got me. but am I dreaming? I suspect it will never be native supported in FreeBSD. -- Best regards, David Xu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Apr 3 14: 0:19 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 2570337B718; Tue, 3 Apr 2001 14:00:13 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.1/8.11.1) with SMTP id f33L0Ch22319; Tue, 3 Apr 2001 17:00:12 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Tue, 3 Apr 2001 17:00:11 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: freebsd-arch@FreeBSD.org Cc: dillon@FreeBSD.org Subject: Eliminate crget() from nfs kernel code? Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG While extending ucred in a rewrite of my mandatory access control implementation on FreeBSD, I managed to panic the system: kernel: type 12 trap, code=0 Stopped at mac_free+0x15: cmpxchgl %ecx,0x20(%ebx) db> trace mac_free(0,c1055d00,c8a52ddc,c0234686,c1055d00) at mac_free+0x15 mac_free_subj(c1055d00,0,c8a52e2c,c02ed342,c1055d00) at mac_free_subj+0xf crfree(c1055d00,c8a5a6c0,c0ff1200,c0ff124c,c89fba80) at crfree+0x82 nfs_statfs(c0ff1200,c0ff124c,c89fba80) at nfs_statfs+0x772 fstatfs(c89fba80,c8a52f80,81920c0,8156030,bfbea7ec) at fstatfs+0x44 syscall(2f,2f,2f,bfbea7ec,8156030) at syscall+0x3f5 syscall_with_err_pushed() at syscall_with_err_pushed+0x1b What I discovered was that the nfs_statfs() implementation fakes up a credential using crget() to use when invoking nfs_fsinfo(), and uses it only for the lifetime of the nfs_statfs() call. While I don't claim to have a complete understanding of the NFS code, it appears that this is done in lieu of using p->p_ucred, and that other related VFS calls are passed a ucred as an argument. This causes some problems for implementations of extensions to ucred, as it introduces a new (and fairly arbitrary) credential that must be created to satisfy the need to call crget() here. I was wondering if someone could expound on the rationale for creating a new (and fairly poorly initialized) ucred in this scenario, and perhaps comment also on two possible work-arounds: 1) Use p->p_ucred. 2) Introduce a ucred argument to vfs_statfs() to be used by file system implementations that require it. Either would be in line with current precedent for VFS call construction: vfs_sync() has a ucred passed at invocation, the others generally assume the use of p->p_ucred. Using p->p_ucred would avoid changing the VFS interface, but if it's really needed, I don't see any harm in adding a ucred argument (although this will break binary compatibility in -CURRENT, which is where I'd consider doing the change). The effect of the current code, btw, is to always make NFSPROC_STATFS() calls using uid/gid of 0 (possibly mapped to nobody on the server), rather than the calling process -- it's possible to imagine environments in which this is inappropriate, especially with the introduction of MAC. I notice that there's some similar behavior in the ncp code. Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Apr 3 14:29:56 2001 Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 07C3A37B71C; Tue, 3 Apr 2001 14:29:54 -0700 (PDT) (envelope-from dillon@earth.backplane.com) Received: (from dillon@localhost) by earth.backplane.com (8.11.2/8.9.3) id f33LTmj70619; Tue, 3 Apr 2001 14:29:48 -0700 (PDT) (envelope-from dillon) Date: Tue, 3 Apr 2001 14:29:48 -0700 (PDT) From: Matt Dillon Message-Id: <200104032129.f33LTmj70619@earth.backplane.com> To: Robert Watson Cc: freebsd-arch@FreeBSD.ORG Subject: Re: Eliminate crget() from nfs kernel code? References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :While extending ucred in a rewrite of my mandatory access control :implementation on FreeBSD, I managed to panic the system: : :... : :What I discovered was that the nfs_statfs() implementation fakes up a :credential using crget() to use when invoking nfs_fsinfo(), and uses it :only for the lifetime of the nfs_statfs() call. While I don't claim to :have a complete understanding of the NFS code, it appears that this is :done in lieu of using p->p_ucred, and that other related VFS calls are :passed a ucred as an argument. This causes some problems for :implementations of extensions to ucred, as it introduces a new (and fairly :arbitrary) credential that must be created to satisfy the need to call :crget() here. I was wondering if someone could expound on the rationale :for creating a new (and fairly poorly initialized) ucred in this scenario, :and perhaps comment also on two possible work-arounds: : : 1) Use p->p_ucred. : 2) Introduce a ucred argument to vfs_statfs() to be used by : file system implementations that require it. :... :ucred argument (although this will break binary compatibility in -CURRENT, :which is where I'd consider doing the change). The effect of the current :code, btw, is to always make NFSPROC_STATFS() calls using uid/gid of 0 :(possibly mapped to nobody on the server), rather than the calling process :-- it's possible to imagine environments in which this is inappropriate, :especially with the introduction of MAC. I notice that there's some :similar behavior in the ncp code. : :Robert N M Watson FreeBSD Core Team, TrustedBSD Project :robert@fledge.watson.org NAI Labs, Safeport Network Services Hmm. It's hard to say. The standard doesn't say anything about the credential needing to be root/wheel over the wire for statfs(), but if we change it we will probably break someone somewhere. I think you want to keep crget()/crfree() if possible, just so you don't have to worry about the data being sent over the wire breaking some poor sod. But if you want to bite the bullet and use the process ucred I'd say go for it - keep a careful watch for complaints of FreeBSD suddenly failing against various NFS servers though. The fake ucred has been in the NFS codebase from day 1. It could very well be outdated now. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Apr 3 14:42:13 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 501E037B71D; Tue, 3 Apr 2001 14:42:11 -0700 (PDT) (envelope-from bright@fw.wintelcom.net) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f33LgAL12453; Tue, 3 Apr 2001 14:42:10 -0700 (PDT) Date: Tue, 3 Apr 2001 14:42:10 -0700 From: Alfred Perlstein To: Matt Dillon Cc: Robert Watson , freebsd-arch@FreeBSD.ORG Subject: Re: Eliminate crget() from nfs kernel code? Message-ID: <20010403144210.B12164@fw.wintelcom.net> References: <200104032129.f33LTmj70619@earth.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200104032129.f33LTmj70619@earth.backplane.com>; from dillon@earth.backplane.com on Tue, Apr 03, 2001 at 02:29:48PM -0700 X-all-your-base: are belong to us. Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Matt Dillon [010403 14:30] wrote: > :While extending ucred in a rewrite of my mandatory access control > :implementation on FreeBSD, I managed to panic the system: > : > :... > : > :What I discovered was that the nfs_statfs() implementation fakes up a > :credential using crget() to use when invoking nfs_fsinfo(), and uses it > :only for the lifetime of the nfs_statfs() call. While I don't claim to > :have a complete understanding of the NFS code, it appears that this is > :done in lieu of using p->p_ucred, and that other related VFS calls are > :passed a ucred as an argument. This causes some problems for > :implementations of extensions to ucred, as it introduces a new (and fairly > :arbitrary) credential that must be created to satisfy the need to call > :crget() here. I was wondering if someone could expound on the rationale > :for creating a new (and fairly poorly initialized) ucred in this scenario, > :and perhaps comment also on two possible work-arounds: > : > : 1) Use p->p_ucred. > : 2) Introduce a ucred argument to vfs_statfs() to be used by > : file system implementations that require it. > :... > :ucred argument (although this will break binary compatibility in -CURRENT, > :which is where I'd consider doing the change). The effect of the current > :code, btw, is to always make NFSPROC_STATFS() calls using uid/gid of 0 > :(possibly mapped to nobody on the server), rather than the calling process > :-- it's possible to imagine environments in which this is inappropriate, > :especially with the introduction of MAC. I notice that there's some > :similar behavior in the ncp code. > : > :Robert N M Watson FreeBSD Core Team, TrustedBSD Project > :robert@fledge.watson.org NAI Labs, Safeport Network Services > > Hmm. It's hard to say. The standard doesn't say anything about > the credential needing to be root/wheel over the wire for statfs(), > but if we change it we will probably break someone somewhere. > > I think you want to keep crget()/crfree() if possible, just so you > don't have to worry about the data being sent over the wire breaking > some poor sod. But if you want to bite the bullet and use the process > ucred I'd say go for it - keep a careful watch for complaints of > FreeBSD suddenly failing against various NFS servers though. > > The fake ucred has been in the NFS codebase from day 1. It could very > well be outdated now. Hmm, strange there's not a static cred available for these usages. What about using process 0's ucred, does it even have one? I'm sure there's other examples of how to do this correctly in the code. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] Represent yourself, show up at BABUG http://www.babug.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Apr 3 15:50:23 2001 Delivered-To: freebsd-arch@freebsd.org Received: from Awfulhak.org (awfulhak.demon.co.uk [194.222.196.252]) by hub.freebsd.org (Postfix) with ESMTP id 03FFE37B71C; Tue, 3 Apr 2001 15:50:16 -0700 (PDT) (envelope-from brian@Awfulhak.org) Received: from hak.lan.Awfulhak.org (root@hak.lan.Awfulhak.org [172.16.0.12]) by Awfulhak.org (8.11.3/8.11.3) with ESMTP id f33MoV828960; Tue, 3 Apr 2001 23:50:31 +0100 (BST) (envelope-from brian@lan.Awfulhak.org) Received: from hak.lan.Awfulhak.org (brian@localhost [127.0.0.1]) by hak.lan.Awfulhak.org (8.11.3/8.11.3) with ESMTP id f33Mso560006; Tue, 3 Apr 2001 23:54:50 +0100 (BST) (envelope-from brian@hak.lan.Awfulhak.org) Message-Id: <200104032254.f33Mso560006@hak.lan.Awfulhak.org> X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 To: Alfred Perlstein Cc: Matt Dillon , Robert Watson , freebsd-arch@FreeBSD.ORG, brian@Awfulhak.org Subject: Re: Eliminate crget() from nfs kernel code? In-Reply-To: Message from Alfred Perlstein of "Tue, 03 Apr 2001 14:42:10 PDT." <20010403144210.B12164@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 03 Apr 2001 23:54:50 +0100 From: Brian Somers Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > * Matt Dillon [010403 14:30] wrote: > > :While extending ucred in a rewrite of my mandatory access control > > :implementation on FreeBSD, I managed to panic the system: > > : > > :... > > : > > :What I discovered was that the nfs_statfs() implementation fakes up a > > :credential using crget() to use when invoking nfs_fsinfo(), and uses it > > :only for the lifetime of the nfs_statfs() call. While I don't claim to > > :have a complete understanding of the NFS code, it appears that this is > > :done in lieu of using p->p_ucred, and that other related VFS calls are > > :passed a ucred as an argument. This causes some problems for > > :implementations of extensions to ucred, as it introduces a new (and fairly > > :arbitrary) credential that must be created to satisfy the need to call > > :crget() here. I was wondering if someone could expound on the rationale > > :for creating a new (and fairly poorly initialized) ucred in this scenario, > > :and perhaps comment also on two possible work-arounds: > > : > > : 1) Use p->p_ucred. > > : 2) Introduce a ucred argument to vfs_statfs() to be used by > > : file system implementations that require it. > > :... > > :ucred argument (although this will break binary compatibility in -CURRENT, > > :which is where I'd consider doing the change). The effect of the current > > :code, btw, is to always make NFSPROC_STATFS() calls using uid/gid of 0 > > :(possibly mapped to nobody on the server), rather than the calling process > > :-- it's possible to imagine environments in which this is inappropriate, > > :especially with the introduction of MAC. I notice that there's some > > :similar behavior in the ncp code. > > : > > :Robert N M Watson FreeBSD Core Team, TrustedBSD Project > > :robert@fledge.watson.org NAI Labs, Safeport Network Services > > > > Hmm. It's hard to say. The standard doesn't say anything about > > the credential needing to be root/wheel over the wire for statfs(), > > but if we change it we will probably break someone somewhere. > > > > I think you want to keep crget()/crfree() if possible, just so you > > don't have to worry about the data being sent over the wire breaking > > some poor sod. But if you want to bite the bullet and use the process > > ucred I'd say go for it - keep a careful watch for complaints of > > FreeBSD suddenly failing against various NFS servers though. > > > > The fake ucred has been in the NFS codebase from day 1. It could very > > well be outdated now. > > Hmm, strange there's not a static cred available for these usages. > > What about using process 0's ucred, does it even have one? > > I'm sure there's other examples of how to do this correctly in the > code. Solaris has a ``kcred'' global - wrapped with a CRED() macro AFAIR. Maybe that'd be useful here ? > -- > -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] > Represent yourself, show up at BABUG http://www.babug.org/ -- Brian Don't _EVER_ lose your sense of humour ! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Apr 3 16: 9:37 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id D2A4537B71F; Tue, 3 Apr 2001 16:09:35 -0700 (PDT) (envelope-from bright@fw.wintelcom.net) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f33N8wD15085; Tue, 3 Apr 2001 16:08:58 -0700 (PDT) Date: Tue, 3 Apr 2001 16:08:58 -0700 From: Alfred Perlstein To: Brian Somers Cc: Matt Dillon , Robert Watson , freebsd-arch@FreeBSD.ORG Subject: Re: Eliminate crget() from nfs kernel code? Message-ID: <20010403160858.L12164@fw.wintelcom.net> References: <200104032254.f33Mso560006@hak.lan.Awfulhak.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200104032254.f33Mso560006@hak.lan.Awfulhak.org>; from brian@Awfulhak.org on Tue, Apr 03, 2001 at 11:54:50PM +0100 X-all-your-base: are belong to us. Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Brian Somers [010403 15:50] wrote: > > > > Hmm, strange there's not a static cred available for these usages. > > > > What about using process 0's ucred, does it even have one? > > > > I'm sure there's other examples of how to do this correctly in the > > code. > > Solaris has a ``kcred'' global - wrapped with a CRED() macro AFAIR. > Maybe that'd be useful here ? Yes, it most likely would. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] Represent yourself, show up at BABUG http://www.babug.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Apr 3 16:35:51 2001 Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 8669F37B71F; Tue, 3 Apr 2001 16:35:49 -0700 (PDT) (envelope-from dillon@earth.backplane.com) Received: (from dillon@localhost) by earth.backplane.com (8.11.2/8.9.3) id f33NZWK73052; Tue, 3 Apr 2001 16:35:32 -0700 (PDT) (envelope-from dillon) Date: Tue, 3 Apr 2001 16:35:32 -0700 (PDT) From: Matt Dillon Message-Id: <200104032335.f33NZWK73052@earth.backplane.com> To: Alfred Perlstein Cc: Brian Somers , Robert Watson , freebsd-arch@FreeBSD.ORG Subject: Re: Eliminate crget() from nfs kernel code? References: <200104032254.f33Mso560006@hak.lan.Awfulhak.org> <20010403160858.L12164@fw.wintelcom.net> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :> > What about using process 0's ucred, does it even have one? :> > :> > I'm sure there's other examples of how to do this correctly in the :> > code. :> :> Solaris has a ``kcred'' global - wrapped with a CRED() macro AFAIR. :> Maybe that'd be useful here ? : :Yes, it most likely would. :-- :-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] That is an excellent idea. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Apr 3 20: 5:37 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 3650037B718 for ; Tue, 3 Apr 2001 20:05:35 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.1/8.11.1) with SMTP id f34342h66783; Tue, 3 Apr 2001 23:04:03 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Tue, 3 Apr 2001 23:04:02 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Matt Dillon Cc: Alfred Perlstein , Brian Somers , freebsd-arch@FreeBSD.ORG Subject: Re: Eliminate crget() from nfs kernel code? In-Reply-To: <200104032335.f33NZWK73052@earth.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, 3 Apr 2001, Matt Dillon wrote: > :> > What about using process 0's ucred, does it even have one? > :> > > :> > I'm sure there's other examples of how to do this correctly in the > :> > code. > :> > :> Solaris has a ``kcred'' global - wrapped with a CRED() macro AFAIR. > :> Maybe that'd be useful here ? > : > :Yes, it most likely would. However, it still strikes me a bit as though this is a, ``Help, I need a credential, someone find a credential'' as opposed to a, ``What credential is the one we want to use here.'' My temptation here would be to try temporarily switching to using p->p_ucred for the time being, and as Matt indicated, watch closely for reports of any interoperability problems with other implementations. Right now, the code selects to make the call using all available privilege: in a more contained environment, that might no longer be appropriate. Particularly if the ucred contains MAC integrity and confidentiality labels, which translate into labeling of the packet itself, or map into use of IPsec SAs or the like. How about I put together a patch, interop against the implementations I have on-hand (various FreeBSD, Linux, Solaris versions) and see if I'm shooting myself in the foot or not... Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Apr 3 20: 9:51 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail.noos.fr (claudel.noos.net [212.198.2.83]) by hub.freebsd.org (Postfix) with ESMTP id E4A0F37B724 for ; Tue, 3 Apr 2001 20:09:46 -0700 (PDT) (envelope-from clefevre@poboxes.com) Received: (qmail 3374680 invoked by uid 0); 4 Apr 2001 03:09:24 -0000 Received: from d165.dhcp212-198-231.noos.fr (HELO gits.dyndns.org) ([212.198.231.165]) (envelope-sender ) by claudel.noos.net (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 4 Apr 2001 03:09:24 -0000 Received: (from root@localhost) by gits.dyndns.org (8.11.3/8.11.3) id f3439h969470; Wed, 4 Apr 2001 05:09:43 +0200 (CEST) (envelope-from clefevre@poboxes.com) To: freebsd-arch@FreeBSD.ORG Subject: Re: configuration files, XML? References: <200103270440.f2R4eRr88969@katin.codeconcepts.com> X-Face: V|+c;4!|B?E%BE^{E6);aI.[<97Zd*>^#%Y5Cxv;%Y[PT-LW3;A:fRrJ8+^k"e7@+30g0YD0*^^3jgyShN7o?a]C la*Zv'5NA,=963bM%J^o]C Reply-To: Cyrille Lefevre Mail-Copies-To: never Original-Sender: clefevre-lists@noos.fr From: Cyrille Lefevre Date: 04 Apr 2001 05:09:42 +0200 In-Reply-To: <200103270440.f2R4eRr88969@katin.codeconcepts.com> Message-ID: Lines: 63 User-Agent: Gnus/5.0808 (Gnus v5.8.8) XEmacs/21.1 (Cuyahoga Valley) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG here, are some usefull links which may help : http://oss.sgi.com/projects/rhino/ http://www.opensource.apple.com/projects/darwin/1.2/projects.html (netinfo) humm! webmin ;^) well, since I didn't know anything about xml nor sgml. I'll just try to explain some sort of concept using a basic example of what it could be possible to do. public description file : public description file : (not really true) ... ... ... /whatever/passwd.xml : (nothing more) the idea is to keep data where they are in the format they are and to have some sort of configurable api which interpret them through the appropriate gui. maybe I am totally wrong about the xml possibilities ? Cyrille. -- home: mailto:clefevre@poboxes.com UNIX is user-friendly; it's just particular work: mailto:Cyrille.Lefevre@edf.fr about who it chooses to be friends with. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Apr 3 20:51:15 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail.noos.fr (zola.noos.net [212.198.2.76]) by hub.freebsd.org (Postfix) with ESMTP id 62C7F37B720 for ; Tue, 3 Apr 2001 20:51:08 -0700 (PDT) (envelope-from clefevre@poboxes.com) Received: (qmail 1390718 invoked by uid 0); 4 Apr 2001 03:51:06 -0000 Received: from d165.dhcp212-198-231.noos.fr (HELO gits.dyndns.org) ([212.198.231.165]) (envelope-sender ) by zola.noos.net (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 4 Apr 2001 03:51:06 -0000 Received: (from root@localhost) by gits.dyndns.org (8.11.3/8.11.3) id f343p4270704; Wed, 4 Apr 2001 05:51:04 +0200 (CEST) (envelope-from clefevre@poboxes.com) To: "David O'Brien" Cc: Cy Schubert - ITSD Open Systems Group , Jack Rusher , Neil Blakey-Milner , Michael Lucas , "litos2001@libero.it" Subject: Re: Startup scripts a la NetBSD References: <3AC4B808.9EB5806B@integratus.com> <200103301810.f2UIAGK06787@cwsys.cwsent.com> <20010330115252.A93566@dragon.nuxi.com> X-Face: V|+c;4!|B?E%BE^{E6);aI.[<97Zd*>^#%Y5Cxv;%Y[PT-LW3;A:fRrJ8+^k"e7@+30g0YD0*^^3jgyShN7o?a]C la*Zv'5NA,=963bM%J^o]C Reply-To: Cyrille Lefevre In-Reply-To: <20010330115252.A93566@dragon.nuxi.com> Mail-Copies-To: never Original-Sender: clefevre-lists@noos.fr From: Cyrille Lefevre Date: 04 Apr 2001 05:51:03 +0200 Message-ID: Lines: 72 User-Agent: Gnus/5.0808 (Gnus v5.8.8) XEmacs/21.1 (Cuyahoga Valley) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG --=-=-= "David O'Brien" writes: > On Fri, Mar 30, 2001 at 10:09:39AM -0800, Cy Schubert - ITSD Open Systems Group wrote: > > Becoming more SYSV-like is not something most people in FreeBSD are > > enamoured with. Hence I don't think it would be possible to implement > > all the changes I would like to see. > > Correct. Make that HELL NO WE WON'T GO SVR4!!! > > The BSD init + NetBSD granular rc files is a nice compromise. And thus > why I am pushing a move in that direction. some times ago, I've setup a NetBSD box and they init files are not yet perfect. as I remember me, you couldn't do something like /etc/rc.d/nfs.server start (or whatever) while nfs.server depends on something else. you only could do /etc/rc which start all things. also, I don't remember if they are checking if the service is already started as HP-UX does. in the same time (`ls -ld /etc/init.d` = Nov 23 :), I've began something like in attachement which allow us to start/stop all dependencies as I remember me. at least the nfs stack is working since I use it when I need it. of course, this was an experiment and need to be enhanced/reworked. the basic idea was it have a generic wrapper which source its configuration file a la HP-UX. the configuration file would be something like this (from nfs.server) : service_class= service_instance=nfs_server service_enable=YES service_proc=nfsd service_path=/sbin service_program=$service_path/$service_proc service_flags= nfsd_pre_start () { case ${nfs_reserved_port_only} in [Yy][Ee][Ss]) echo -n ' NFS on reserved port only=YES' sysctl -w vfs.nfs.nfs_privport=1 >/dev/null ;; esac } nfsd_post_start () { if [ -n "${nfs_bufpackets}" ]; then sysctl -w vfs.nfs.bufpackets=${nfs_bufpackets} >/dev/null fi } service_pre_start=nfsd_pre_start service_pre_start_depend="portmap mountd" service_post_start=nfsd_post_start service_post_start_depend="lockd statd" service_pre_stop= service_pre_stop_depend=$service_post_start_depend service_force_stop=true service_post_stop= service_post_stop_depend=mountd and the wrapper is all the rest. this isn't currently done in the attached samples. --=-=-= Content-Type: application/octet-stream Content-Disposition: attachment; filename=init.d.tgz Content-Transfer-Encoding: base64 Content-Description: init.d.tgz H4sIAEOYyjoCA+1dW2/bxhL2q/grprRgNW0kkbqiCRigDylQFCgOTh4OisRHWZFLiTBNsrxY cR39984u7xdZLmLHjjsD2Ja5s7OzuzOz83HJleM58cganzwkwUxbzudwAoK0xt/sH1hOtKW2 mC41HUDXpnPtBOYnX4GSKGYhwEno+/FtfLst5+7JsyMnnX/PjkYRD694+ABt6Nqt8z+fThfp /C8mi/lSzP9kri1PQKP5f3A6/W68drxxtFWUU/jVhnjLQw5OBAw2rr9mLkTXUcwvwfQ929kk IYsd3wPbcflLiBLzApwYHG+knCqODe9hGMKYx+bY4jZL3Dgah+ZIVIXz10K2p+QuP+rmK8oj PwlNvgrNlbgcKdytyT8iNpdmO4oi7NpBUZYTGrIwtXks86wg9E34/gXcKL3AsSLjY4A9Nz/B Z2C7C1B//+8b/exs3M9FCPaxOrgJQseLoa/vBx+VHirlgdoX9VU4V/Zli6bLosgo/nU8tDXP 5Ab2Z5V6W1HGPbZ2ufHH23dKtTHBapVXWLw1xhFOWJVpE7JLo1/lqStc8Nou26A6ipCJJXwl TD/Oum+yiEP/RqgWcqkc8vhhvPI993qPU4wd/eP6/P1bfv7+XXT+Qun1uLn1Rd8H8Psv7wCt Iq8IoiKIiqJDA2RFIzJjF4Y7uMJI46U/qINzJVgNHd6gJVyNvcR1kfv1a6XHI2biWKa6+lFc UzY1BTHqUuF1YgfMvOBxtFcLi+hqtGQ0WjVrKqDdVOaxGCujPnRthpXFA+5Zhiq6dckCuPQT L7bUkrPoitHoWgdLIc31zQsL8FpNlGzUD4zWlbxe/6DI0iZ84WVSTBwmvKlETXp+KRef9k1R +JUIE1Wr/dC/6d80zX4vy14Na8a5r9fO3KBWX3oRVmsL3Gf8FZnphX1LJ+kkh9USxXXNxJWG mNR/DgiRhRUR8n9hQzhp3Ojrio/uHjoWfr6ZVPjyy3tQlMwH80vtbgk3rHthHvfQZX5An5Qu qfab6r0Cy4mEBGukFlX4JwzdWlWC9LlMC6m3aE+aDYpGS8lHAi9D/5DtvwbLR+frV8JuEY9y sf28i/D5s9RD6Vm+x6VjF2G5ZieFUx/sIXNDzqxrkLpgDBoOIYkcbwMYmL+PXrxSQYZoHCM3 wpYqMaTVk2oQ6bWL8SrGByy5adoLlJey+c8Ubumbqznq1qTw1wOqlFEj1UX+HJqgpvN/yQxV rEWEgqN2UUSLBzaLQ7PpB7dNph/kc3nhuC5z3br0TrllxKxJjlzOA5iKj92qwtmZKMwbGv7U aksqctBc/CA4Yi4H+5oXZ21kDnDQkzw/hjDxPHQeDBd3sa17meV6KCuilNQyidgGNetrcCMt +bNoc9+IZXorlp0Q3Q3/ZenKw7RxBP/NFpNJgf/nc03gP/xE+I/w3zPFf5m3HQV/Tb4U/yVR +OUYsA1tDoOZLvByC1p5OgDlYI5KmIUwC2EWwiyEWQiz/LsxS5b/rzH7DRiGA+sR8v/lbJ7n /zN9ku7/TCeU/1P+/0zz/9LbjkKADtZ7RAGl9NZ+UDnUBVME50X469wd6RL39GEFQQqCFAQp CFIQpCBI8a+DFFn+H7IwsB4qxzya/+vz/PkvfTJbYP6vz/UF5f+U/z/T/F9629HUv851j1m/ FHxLws+FERxN9htSKM+nPJ/yfMrzKc+nPJ/y/G9m6yB//4PFD5b+3+H9j/L9n/lyKvL/6VSn /J/y/2eR/9tOyHcYA7OspuN1EHS+4y+CVJnu60UQlNnCAdV3K0Q59pSHNsOkrxpukSsN3U0m HLoPIhxvQlwfhn/C8C0M/v9eG/50/uOHUfff/qCM4qk0m4UbQx2ytnhVRPI0oNd5vW7eygJQ ZU/jvNKr56+G2kxooF/WUg+8ElIbxMMwyAnsHUEhgkIEhQgKERQiKERQiN78eDLvf6Svcj4S /tPFmQ8Z/pst5wL/4UfCf4T/ngX+63jbv4B+2SvUx8Bfg+2e4F8q9baNoE/iDZTqTlBxTsCO s4tVJoAlYgpix5S20X1WwFGgNfTU+pv/HVCrqTDtORHQIqBFQIuAFgEtAloEtIi+5Pw303Uw i3sE/LdYLMv9v4XAf5PZYkb4j/DfMz7/LfW2u5z/5vgPcQIcSr116w9VZKbJI9SUmdvq5h80 KD8GTpXHwKWVQFaC2LnkRpewlpD2WW3VCishyE/iLlkd57bl3WscG5e3JQ19fMXCsbVO73zF bC3WtyiO0Kj9Sw4oYZhidtiySC7Xa17p+6mwfdTBxlavRbFjO5gQszUqia6DA3vl+AkOg5xl iLZJbPk7dJOaDoWzeX5Ln5ew47jSC9lbkSUUNdNJ6ujAofkJA3OUXHqxTIkuiuJbjrir2cY/ A9mtoSfUTaibUDehbkLdhLoJdT851J3hP/Fw0mM9/7nUtOL8h+lCn8nnP2dzwn9fGf9lyybh wK+GAw89GXoIDNZhYAsBtmBeId/xVhc89Lib9RGnGPvuMNf5i8Ov/xHTGPNQ5BRpZiGfVeyd yvVJok0ZIMTjnLabRFt4AwXmgsmbM72Ize0WDb2Iyx2FWg7ZXJ9Zq5wh0zJdEBCMtisiHB3y PzGInOPqAxeuJepLtctloluXfHn4LR2PoslL30pcDkKOTCbFY6Z3ab9sL5X7PxaK9eVVKTkb +awBm6HLWBD7sqlR9ozqvjJZTVTebl3p1YZLmF6nkmCAXtu8TV3nr1qfIjN0grj2cG+jzMhd qWxQPjybO+IRaaNOhnydHvySj1OIwxPlEzBQivH/KG3PhcV8Pp1/VLFTqvwIFveucc7BDv1L YPgZBxX/VNsu7k0U05LNhkD1xQzZiWeKcIb+EF+/hEI3dY1o3q7rxzAsijwi9dHMUiRzrwc/ IxsqlPlhylwBKK2Zbp5rn+GkgsH1NxvUur2ljoFpMHjRHsOsQvvYfY/HI0f+Ckb2boSJytqP +C0H73fcGWhbKG3A062Ab/JWQLnIb5JilW+s72dn3437fVzmxQdkGOfL/UQs93Q3ge4m0N0E uptAdxOIvv39f/nVQo92/s90Vu7/Tyfy/J8J7f/T/v/zf/5b+t3xk4ACc1TnvM9vAojZP3wI vBMcNsQQMiRkSJvEBOsI1hGsI1j3dGFdlv/LLxh9Cvn/QkvP/9To+78o/3/++b/0uzvl/3XO e8z/peAvz/8bYij/p/yf8n/K/yn/p/yf8n/a1iEiIiIiIiIiIiIiIiIiIiIiIiIiIiIi+nr0 N+RzvHgAoAAA --=-=-= Cyrille. -- home: mailto:clefevre@poboxes.com UNIX is user-friendly; it's just particular work: mailto:Cyrille.Lefevre@edf.fr about who it chooses to be friends with. --=-=-=-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Apr 4 3:18:32 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 1D77F37B724; Wed, 4 Apr 2001 03:18:29 -0700 (PDT) (envelope-from bde@zeta.org.au) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id UAA23032; Wed, 4 Apr 2001 20:18:24 +1000 Date: Wed, 4 Apr 2001 20:17:10 +1000 (EST) From: Bruce Evans X-Sender: bde@besplex.bde.org To: Robert Watson Cc: Matt Dillon , Alfred Perlstein , Brian Somers , freebsd-arch@FreeBSD.ORG Subject: Re: Eliminate crget() from nfs kernel code? In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, 3 Apr 2001, Robert Watson wrote: > On Tue, 3 Apr 2001, Matt Dillon wrote: > > :> Solaris has a ``kcred'' global - wrapped with a CRED() macro AFAIR. > > :> Maybe that'd be useful here ? > > : > > :Yes, it most likely would. > > However, it still strikes me a bit as though this is a, ``Help, I need a > credential, someone find a credential'' as opposed to a, ``What credential > is the one we want to use here.'' My temptation here would be to try > temporarily switching to using p->p_ucred for the time being, and as Matt > indicated, watch closely for reports of any interoperability problems with > other implementations. Right now, the code selects to make the call using > all available privilege: in a more contained environment, that might no > longer be appropriate. Particularly if the ucred contains MAC integrity access() crdup()'s the p_ucred so that the privilege can be modified. Would that help? Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Apr 4 3:22:24 2001 Delivered-To: freebsd-arch@freebsd.org Received: from dire.bris.ac.uk (dire.bris.ac.uk [137.222.10.60]) by hub.freebsd.org (Postfix) with ESMTP id 5138137B729 for ; Wed, 4 Apr 2001 03:22:21 -0700 (PDT) (envelope-from Jan.Grant@bristol.ac.uk) Received: from mail.ilrt.bris.ac.uk by dire.bris.ac.uk with SMTP-PRIV with ESMTP; Wed, 4 Apr 2001 11:22:09 +0100 Received: from cmjg (helo=localhost) by mail.ilrt.bris.ac.uk with local-esmtp (Exim 3.16 #1) id 14kkP6-0002Jz-00; Wed, 04 Apr 2001 11:20:44 +0100 Date: Wed, 4 Apr 2001 11:20:44 +0100 (BST) From: Jan Grant To: Cyrille Lefevre Cc: freebsd-arch Subject: Re: configuration files, XML? In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 4 Apr 2001, Cyrille Lefevre wrote: > the idea is to keep data where they are in the format they are and to > have some sort of configurable api which interpret them through the > appropriate gui. I think it's better to keep the content separate and to generate old-format configs on demand. We do this with master.passwd and vipw already, essentially, but it needs something more general while we're waiting for the world to catch on to what a good idea XML* config files are :-/ Although this brings up another issue: stuff like chfn, passwd might be written to modify the old-format file - the synchronisation between old- and new- format config files needs to be two-way unless the system utilities are taught about XML config files too. > maybe I am totally wrong about the xml possibilities ? It's just a syntax. If you can write it down, you can write it down in XML. It might not look very pretty, but it can be done. jan * For "XML" read "any commonly applicable format" -- jan grant, ILRT, University of Bristol. http://www.ilrt.bris.ac.uk/ Tel +44(0)117 9287163 Fax +44 (0)117 9287112 RFC822 jan.grant@bris.ac.uk perl -e 's?ck?t??print:perl==pants if $_="Just Another Perl Hacker\n"' To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Apr 4 4:41:13 2001 Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 6F57337B718 for ; Wed, 4 Apr 2001 04:41:10 -0700 (PDT) (envelope-from des@ofug.org) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id NAA60458; Wed, 4 Apr 2001 13:41:09 +0200 (CEST) (envelope-from des@ofug.org) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: freebsd-arch@FreeBSD.ORG Subject: Re: configuration files, XML? References: <200103270440.f2R4eRr88969@katin.codeconcepts.com> From: Dag-Erling Smorgrav Date: 04 Apr 2001 13:41:08 +0200 In-Reply-To: Cyrille Lefevre's message of "04 Apr 2001 05:09:42 +0200" Message-ID: Lines: 9 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Cyrille Lefevre writes: > public description file : > [...] This isn't valid XML, by any stretch of imagination. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Apr 4 8: 0: 7 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 0A64D37B725; Wed, 4 Apr 2001 08:00:05 -0700 (PDT) (envelope-from bright@fw.wintelcom.net) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f34Exx207071; Wed, 4 Apr 2001 07:59:59 -0700 (PDT) Date: Wed, 4 Apr 2001 07:59:59 -0700 From: Alfred Perlstein To: Bruce Evans Cc: Robert Watson , Matt Dillon , Brian Somers , freebsd-arch@FreeBSD.ORG Subject: Re: Eliminate crget() from nfs kernel code? Message-ID: <20010404075959.S12164@fw.wintelcom.net> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from bde@zeta.org.au on Wed, Apr 04, 2001 at 08:17:10PM +1000 X-all-your-base: are belong to us. Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Bruce Evans [010404 03:18] wrote: > On Tue, 3 Apr 2001, Robert Watson wrote: > > > On Tue, 3 Apr 2001, Matt Dillon wrote: > > > :> Solaris has a ``kcred'' global - wrapped with a CRED() macro AFAIR. > > > :> Maybe that'd be useful here ? > > > : > > > :Yes, it most likely would. > > > > However, it still strikes me a bit as though this is a, ``Help, I need a > > credential, someone find a credential'' as opposed to a, ``What credential > > is the one we want to use here.'' My temptation here would be to try > > temporarily switching to using p->p_ucred for the time being, and as Matt > > indicated, watch closely for reports of any interoperability problems with > > other implementations. Right now, the code selects to make the call using > > all available privilege: in a more contained environment, that might no > > longer be appropriate. Particularly if the ucred contains MAC integrity > > access() crdup()'s the p_ucred so that the privilege can be modified. > Would that help? Yes, that's what no one else seems to get. If you want to modify your credential you must crdup() it first. You can only modify a private copy. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] Represent yourself, show up at BABUG http://www.babug.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Apr 4 8:41:53 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id A917637B71E for ; Wed, 4 Apr 2001 08:41:50 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.1/8.11.1) with SMTP id f34Femh17948; Wed, 4 Apr 2001 11:40:48 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Wed, 4 Apr 2001 11:40:47 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Matt Dillon Cc: Alfred Perlstein , Brian Somers , freebsd-arch@FreeBSD.ORG Subject: Re: Eliminate crget() from nfs kernel code? In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG For those interested in giving the p->p_ucred version a try, the simply patch is below. It works between FreeBSD boxes, and my tests against Solaris 5.5. Unfortunately, my remote NFS test box already has interop problems with Linux due to the new mount_nfs that I haven't applied patches to yet, so I can't test that case. Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services Index: nfs_vfsops.c =================================================================== RCS file: /home/ncvs/src/sys/nfs/nfs_vfsops.c,v retrieving revision 1.94 diff -u -r1.94 nfs_vfsops.c --- nfs_vfsops.c 2001/03/01 20:59:19 1.94 +++ nfs_vfsops.c 2001/04/04 03:25:36 @@ -254,7 +254,6 @@ struct nfsmount *nmp = VFSTONFS(mp); int error = 0, v3 = (nmp->nm_flag & NFSMNT_NFSV3), retattr; struct mbuf *mreq, *mrep, *md, *mb, *mb2; - struct ucred *cred; struct nfsnode *np; u_quad_t tquad; @@ -265,14 +264,12 @@ if (error) return (error); vp = NFSTOV(np); - cred = crget(); - cred->cr_ngroups = 1; if (v3 && (nmp->nm_state & NFSSTA_GOTFSINFO) == 0) - (void)nfs_fsinfo(nmp, vp, cred, p); + (void)nfs_fsinfo(nmp, vp, p->p_ucred, p); nfsstats.rpccnt[NFSPROC_FSSTAT]++; nfsm_reqhead(vp, NFSPROC_FSSTAT, NFSX_FH(v3)); nfsm_fhtom(vp, v3); - nfsm_request(vp, NFSPROC_FSSTAT, p, cred); + nfsm_request(vp, NFSPROC_FSSTAT, p, p->p_ucred); if (v3) nfsm_postop_attr(vp, retattr); if (error) { @@ -310,7 +307,6 @@ } nfsm_reqdone; vput(vp); - crfree(cred); return (error); } To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Apr 4 9:53:40 2001 Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 180D037B725; Wed, 4 Apr 2001 09:53:39 -0700 (PDT) (envelope-from dillon@earth.backplane.com) Received: (from dillon@localhost) by earth.backplane.com (8.11.2/8.9.3) id f34GrX188215; Wed, 4 Apr 2001 09:53:33 -0700 (PDT) (envelope-from dillon) Date: Wed, 4 Apr 2001 09:53:33 -0700 (PDT) From: Matt Dillon Message-Id: <200104041653.f34GrX188215@earth.backplane.com> To: Robert Watson Cc: Alfred Perlstein , Brian Somers , freebsd-arch@FreeBSD.ORG Subject: Re: Eliminate crget() from nfs kernel code? References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :For those interested in giving the p->p_ucred version a try, the simply :patch is below. It works between FreeBSD boxes, and my tests against :Solaris 5.5. Unfortunately, my remote NFS test box already has interop :problems with Linux due to the new mount_nfs that I haven't applied :patches to yet, so I can't test that case. : :Robert N M Watson FreeBSD Core Team, TrustedBSD Project :robert@fledge.watson.org NAI Labs, Safeport Network Services It occurs to me that we may have a general problem with p->p_ucred here. For KSEs to work, processes will not be able to assume that they 'own' p->p_ucred if/when they block. What happens if one thread is blocked in a system call that is using p->p_ucred directly without bumping its ref count and another thread goes in and changes the cred? -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Apr 4 10:20:31 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 4A4B037B71C for ; Wed, 4 Apr 2001 10:20:28 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.1/8.11.1) with SMTP id f34HJPh22734; Wed, 4 Apr 2001 13:19:25 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Wed, 4 Apr 2001 13:19:25 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Matt Dillon Cc: Alfred Perlstein , Brian Somers , freebsd-arch@FreeBSD.ORG Subject: Re: Eliminate crget() from nfs kernel code? In-Reply-To: <200104041653.f34GrX188215@earth.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Wed, 4 Apr 2001, Matt Dillon wrote: > :For those interested in giving the p->p_ucred version a try, the simply > :patch is below. It works between FreeBSD boxes, and my tests against > :Solaris 5.5. Unfortunately, my remote NFS test box already has interop > :problems with Linux due to the new mount_nfs that I haven't applied > :patches to yet, so I can't test that case. > > It occurs to me that we may have a general problem with p->p_ucred > here. For KSEs to work, processes will not be able to assume that > they 'own' p->p_ucred if/when they block. What happens if one thread > is blocked in a system call that is using p->p_ucred directly without > bumping its ref count and another thread goes in and changes the cred? It's fine for now pre-KSE. In some threaded operating systems, the way they handle this is to do a crcopy() of the credential when making potentially long (vis blocking) calls, freezing the credential at the time the call is instantiated. I believe Solaris does this, but haven't checked in a while. So when you make a VFSOP, you crcopy and pass the reference to the copy in so that you can release the locking on the ucred pointer rather than holding the mutex and potentially sleep. This has nicer security properties too--you don't want credentials being inconsistent during a call, our you can introduce nasty races. What this does mean is we probably need an explicit credential passed into the VFS operations, as I suggested as another possible solution to the current crget() problem. However, we can always wait on that until KSE actually starts happening (i.e., solve credential/proc locking problem first, then go apply solution). Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Apr 4 10:55:40 2001 Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 0AA1C37B71E; Wed, 4 Apr 2001 10:55:37 -0700 (PDT) (envelope-from dillon@earth.backplane.com) Received: (from dillon@localhost) by earth.backplane.com (8.11.2/8.9.3) id f34HtVK89343; Wed, 4 Apr 2001 10:55:31 -0700 (PDT) (envelope-from dillon) Date: Wed, 4 Apr 2001 10:55:31 -0700 (PDT) From: Matt Dillon Message-Id: <200104041755.f34HtVK89343@earth.backplane.com> To: Robert Watson Cc: Alfred Perlstein , Brian Somers , freebsd-arch@FreeBSD.ORG Subject: Re: Eliminate crget() from nfs kernel code? References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :It's fine for now pre-KSE. In some threaded operating systems, the way :they handle this is to do a crcopy() of the credential when making :potentially long (vis blocking) calls, freezing the credential at the time :the call is instantiated. I believe Solaris does this, but haven't :checked in a while. So when you make a VFSOP, you crcopy and pass the :reference to the copy in so that you can release the locking on the ucred :pointer rather than holding the mutex and potentially sleep. This has :nicer security properties too--you don't want credentials being :inconsistent during a call, our you can introduce nasty races. What this :does mean is we probably need an explicit credential passed into the VFS :operations, as I suggested as another possible solution to the current :crget() problem. However, we can always wait on that until KSE actually :starts happening (i.e., solve credential/proc locking problem first, then :go apply solution). : :Robert N M Watson FreeBSD Core Team, TrustedBSD Project :robert@fledge.watson.org NAI Labs, Safeport Network Services I think we could do it while avoiding the crcopy. How about this: * Any system call that uses p->p_ucred gets a reference to it via crhold(). Simple and inexpensive. We could also adjust crhold() to be an inline instead of a #define and have it return it's argument to make the code using it cleaner. fubarsyscall() { struct ucred *ucred = crhold(p->p_ucred); ... crfree(ucred); } * Any system call that modifies p->p_ucred actually detaches the existing p->p_ucred from the process structure, allocates a completely new one, and assigns the new one to the process structure. Hey, guess what! This is what we do already! Take a look at change_euid() in kern_prot.c! It's even optimized for the ref-count == 1 case. I think what this means is that we simply cleanup and use crhold() and crfree() more diligently everywhere we currently use p->p_ucred, and we're done. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Apr 4 17:23: 8 2001 Delivered-To: freebsd-arch@freebsd.org Received: from smtp10.phx.gblx.net (smtp10.phx.gblx.net [206.165.6.140]) by hub.freebsd.org (Postfix) with ESMTP id 7976737B424; Wed, 4 Apr 2001 17:23:04 -0700 (PDT) (envelope-from tlambert@usr08.primenet.com) Received: (from daemon@localhost) by smtp10.phx.gblx.net (8.9.3/8.9.3) id RAA16598; Wed, 4 Apr 2001 17:23:03 -0700 Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp10.phx.gblx.net, id smtpdro.u7a; Wed Apr 4 17:22:53 2001 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id RAA02886; Wed, 4 Apr 2001 17:22:52 -0700 (MST) From: Terry Lambert Message-Id: <200104050022.RAA02886@usr08.primenet.com> Subject: Re: Eliminate crget() from nfs kernel code? To: dillon@earth.backplane.com (Matt Dillon) Date: Thu, 5 Apr 2001 00:22:52 +0000 (GMT) Cc: rwatson@FreeBSD.ORG (Robert Watson), freebsd-arch@FreeBSD.ORG In-Reply-To: <200104032129.f33LTmj70619@earth.backplane.com> from "Matt Dillon" at Apr 03, 2001 02:29:48 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG [NFS use of crget()] > Hmm. It's hard to say. The standard doesn't say anything about > the credential needing to be root/wheel over the wire for statfs(), > but if we change it we will probably break someone somewhere. It's implicit in the fact that results caching is permitted; that means that the results you get need to be valid for everyone. The only way to assure this is to use a credential that can always get results. The only alternative is not to cache. You convert the results based on local application of the credentials to the information. Yes, I know the status of caching and stacking layer based implied caching in FreeBSD-current. It does not refute the argument. > I think you want to keep crget()/crfree() if possible, just so you > don't have to worry about the data being sent over the wire breaking > some poor sod. But if you want to bite the bullet and use the process > ucred I'd say go for it - keep a careful watch for complaints of > FreeBSD suddenly failing against various NFS servers though. Note that this means you must keep the processes from which you derive the data around, so if you have a stalled call blocked on a server reboot (for example), you cause a lot of problems. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Apr 4 17:38:17 2001 Delivered-To: freebsd-arch@freebsd.org Received: from smtp10.phx.gblx.net (smtp10.phx.gblx.net [206.165.6.140]) by hub.freebsd.org (Postfix) with ESMTP id 34B6F37B423; Wed, 4 Apr 2001 17:38:15 -0700 (PDT) (envelope-from tlambert@usr08.primenet.com) Received: (from daemon@localhost) by smtp10.phx.gblx.net (8.9.3/8.9.3) id RAA24936; Wed, 4 Apr 2001 17:38:14 -0700 Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp10.phx.gblx.net, id smtpdJ7Am7a; Wed Apr 4 17:38:13 2001 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id RAA03316; Wed, 4 Apr 2001 17:38:12 -0700 (MST) From: Terry Lambert Message-Id: <200104050038.RAA03316@usr08.primenet.com> Subject: Re: Eliminate crget() from nfs kernel code? To: rwatson@FreeBSD.ORG (Robert Watson) Date: Thu, 5 Apr 2001 00:38:11 +0000 (GMT) Cc: freebsd-arch@FreeBSD.ORG, dillon@FreeBSD.ORG In-Reply-To: from "Robert Watson" at Apr 03, 2001 05:00:11 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG [ ... crget() ... ] I am not too happy with crget() at the moment. Even discounting the fact that it calls MALLOC(), and does not check the results (the new [BAD] semantics permit this to fail under extremely low memory conditions [FOR NO GOOD REASON] instead of hanging), it is not the only place this happens in FreeBSD, so just because it's a panic waiting to happen is no real excuse to kill it, since it is in good company (e.g. crdup()). I also have a network load related panic that claims to panic in crdup() as a result of it calling crget() and getting bad data (yes, this is 4.3, and crdup() has been fixed in -current to get it's own bad data, instead of relying on crget() for it). If you "fix" crget(), you will also need to fix crdup(). There are plenty of places where crdup() is called, not just in the access() system call, where it is bogusly used to replace _only_ the initial group of the real GID, leaving the groups of the effective UID active, falsely yielding access to the file, even if the real UID would have not have contained the same group list as the effecive UID (gotta love "security" code). My recommendation is to fix the bogosities, like the non-check of the MALLOC() return, and to leave it the heck alone in the NFS case. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Apr 4 17:41:44 2001 Delivered-To: freebsd-arch@freebsd.org Received: from smtp04.primenet.com (smtp04.primenet.com [206.165.6.134]) by hub.freebsd.org (Postfix) with ESMTP id 6314937B43E for ; Wed, 4 Apr 2001 17:41:42 -0700 (PDT) (envelope-from tlambert@usr08.primenet.com) Received: (from daemon@localhost) by smtp04.primenet.com (8.9.3/8.9.3) id RAA02553; Wed, 4 Apr 2001 17:41:26 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp04.primenet.com, id smtpdAAApBaGZe; Wed Apr 4 17:41:12 2001 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id RAA03358; Wed, 4 Apr 2001 17:41:15 -0700 (MST) From: Terry Lambert Message-Id: <200104050041.RAA03358@usr08.primenet.com> Subject: Re: configuration files, XML? To: clefevre@poboxes.com Date: Thu, 5 Apr 2001 00:41:14 +0000 (GMT) Cc: freebsd-arch@FreeBSD.ORG In-Reply-To: from "Cyrille Lefevre" at Apr 04, 2001 05:09:42 AM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG [ ... stuff ... ] I like it. PS: My first reaction on seeing this was to bitch about someone posting HTML to the -arch list. 8-) 8-). Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Apr 4 17:53: 4 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 9FFFE37B440; Wed, 4 Apr 2001 17:53:00 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.1/8.11.1) with SMTP id f350qoh29611; Wed, 4 Apr 2001 20:52:50 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Wed, 4 Apr 2001 20:52:49 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Terry Lambert Cc: freebsd-arch@FreeBSD.ORG, dillon@FreeBSD.ORG Subject: Re: Eliminate crget() from nfs kernel code? In-Reply-To: <200104050038.RAA03316@usr08.primenet.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Thu, 5 Apr 2001, Terry Lambert wrote: > If you "fix" crget(), you will also need to fix crdup(). There are > plenty of places where crdup() is called, not just in the access() > system call, where it is bogusly used to replace _only_ the initial > group of the real GID, leaving the groups of the effective UID active, > falsely yielding access to the file, even if the real UID would have not > have contained the same group list as the effecive UID (gotta love > "security" code). It's arguable that the access() call is correct for the situation in which it is intended to be used: setuid and setgid programs executed as the real user. The reason is that the groups will have not have been modified by the execution of a setuid/setgid program, only the effective uid and gid, which are preserved as the real uid and gid. However, as has been discussed ad nauseum, access() is prone to races in that type of environment, and should be avoided unless the consumer really knows what they are doing. Don't know if you saw the discuss on freebsd-hackers lately, but we've been considering introducing eaccess() and feaccess() which would allow GUI's to correctly render read-only status for files in the file system without (a) opening them (bad for devices and potentially performance) or (b) replicating kernel policy for protections in userland (often impossible). Remember: the UNIX kernel policy and security primitives are quite different from the userland policy and primitives. The userland policy is layered above the kernel policy, and it's arguable that the kernel should not have to know much if anything about the userland policy (if only for layering reasons). For example, the kernel knows about processes with uids, sets of gids, and a special uid 0, which is privileged to modify these (far more complex in reality due to real/saved/... but you get the idea). This contrasts strongly with the userland policy, which involves users and groups, mapped at authentication time (and arbitrary other times) into kernel primitives. Somewhat poorly at that: the kernel policy and primitives are often a poor match for the desired userland administration policy, and reflect a desire for efficiency, not to mention changing needs over time. > My recommendation is to fix the bogosities, like the non-check of the > MALLOC() return, and to leave it the heck alone in the NFS case. I'm not sure I agree: part of the probelm with this use of crget() is that it attempts to arbitrarily create a UNIX user credential without knowledge of what that credential contains. Most credentials (in fact, all, except for that instance, and a similar instance in the ncp code) are inherited from the proc0 ucred, modulo authorized modification in the inheritence path. If you really do want to create a credential here, isn't it an NFS credential you want to arbitrarily create, not a ucred? Today's ucred has uidinfo, jail pointer, and shortly, capability state, MAC label, etc. The NFS code should not be expected to understand that: if it requires the use of a common credential, presumably it should use an arbitrary NFS credential, which the NFS code is safe assuming both the syntax and semantics of. Shared caching on a multi-user machine has always had interesting security problems associated with it: consider the case of AFS or Coda, where there's a joint cache manager handling a series of seperately authentication connections to the file server. Really, the caches should be isolated, and the processes should see namespaces determined by their credentials permitting them access to the cache. In practice, this is not done, but as a result, it's possible to apply a variety of cache-stuffing attacks. Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Apr 4 20:51:55 2001 Delivered-To: freebsd-arch@freebsd.org Received: from resnet.uoregon.edu (resnet.uoregon.edu [128.223.122.47]) by hub.freebsd.org (Postfix) with ESMTP id 60F9737B506 for ; Wed, 4 Apr 2001 20:51:43 -0700 (PDT) (envelope-from dwhite@resnet.uoregon.edu) Received: from localhost (dwhite@localhost) by resnet.uoregon.edu (8.10.1/8.10.1) with ESMTP id f353pUN50115; Wed, 4 Apr 2001 20:51:30 -0700 (PDT) Date: Wed, 4 Apr 2001 20:51:30 -0700 (PDT) From: Doug White To: David Xu Cc: "David O'Brien" , Cy Schubert - ITSD Open Systems Group , Jack Rusher , Neil Blakey-Milner , Michael Lucas , "litos2001@libero.it" Subject: Re[3]: Startup scripts a la NetBSD In-Reply-To: <39898261.20010403084256@21cn.com> Message-ID: X-All-Your-Base: are belong to us MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, 3 Apr 2001, David Xu wrote: > I like the idea, it has unique start/stop interface, I needn't > remember obscure parameters for every daemon, it really got me. > but am I dreaming? I suspect it will never be native supported > in FreeBSD. We already have it in /usr/local/etc/rc.d. Works identically to SYSV-style start/stop scripts. Use it, love it. My emulator just provides a linux-style /etc/rc.d heirarchy, a chkconfig(8)-compatible script, and start- and stop scripts for that system. This came about to placate lazy engineers and I kinda got used to it. :) Doug White | FreeBSD: The Power to Serve dwhite@resnet.uoregon.edu | www.FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Apr 4 21:50:39 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 781A837B43E for ; Wed, 4 Apr 2001 21:50:36 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.1/8.11.1) with SMTP id f354oRh32168; Thu, 5 Apr 2001 00:50:27 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Thu, 5 Apr 2001 00:50:27 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Kirk McKusick Cc: arch@freebsd.org Subject: Re: Background Fsck In-Reply-To: <200104012342.QAA19845@beastie.mckusick.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Kirk, Another usability question. Was wondering about the possibility of multiple background fsck's getting started at a time, et al, possibly due to bad behavior by the user. Can the user get shot in the foot in the following situations: 1) They unmount a file system during a background fsck -- not all that unlikely. (Assuming that fsck keeps a file open for use with the sysctl, or possibly eventually vfsop, a non-forcible unmount should get EBUSY, but not sure what a forcible unmount will do). 2) They start a second background fsck running at the same time as the first. 3) They perform a remount, possibly changing between read-only and read-write. In some of these, it may be OK for the user to be shot in the foot, but it might be worth considering strategies to prevent foot-shooting, especially in the case of remount situations. Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Apr 4 22: 7:10 2001 Delivered-To: freebsd-arch@freebsd.org Received: from rina.r.dl.itc.u-tokyo.ac.jp (rina.r.dl.itc.u-tokyo.ac.jp [133.11.199.247]) by hub.freebsd.org (Postfix) with ESMTP id 54C0537B449 for ; Wed, 4 Apr 2001 22:07:01 -0700 (PDT) (envelope-from tanimura@r.dl.itc.u-tokyo.ac.jp) Received: from rina.r.dl.itc.u-tokyo.ac.jp (localhost [127.0.0.1]) by rina.r.dl.itc.u-tokyo.ac.jp (8.11.3+3.4W/3.7W-rina.r-0.1-11.01.2000) with ESMTP id f3556Xw28400; Thu, 5 Apr 2001 14:06:35 +0900 (JST) Message-Id: <200104050506.f3556Xw28400@rina.r.dl.itc.u-tokyo.ac.jp> Date: Thu, 05 Apr 2001 14:06:33 +0900 From: Seigo Tanimura To: arch@FreeBSD.org Cc: bde@zeta.org.au, Seigo Tanimura Subject: Mmap(2) should start just below stack (was: Re: Bumping up {MAX,DFL}*SIZ in i386) In-Reply-To: <200103230517.f2N5HXx08605@rina.r.dl.itc.u-tokyo.ac.jp> References: <200103191056.f2JAuox00630@rina.r.dl.itc.u-tokyo.ac.jp> <200103230517.f2N5HXx08605@rina.r.dl.itc.u-tokyo.ac.jp> User-Agent: Wanderlust/2.4.1 (Stand By Me) SEMI/1.13.7 (Awazu) FLIM/1.13.2 (Kasanui) MULE XEmacs/21.1 (patch 14) (Cuyahoga Valley) (i386--freebsd) Organization: Digital Library Research Division, Information Techinology Centre, The University of Tokyo MIME-Version: 1.0 (generated by SEMI 1.13.7 - "Awazu") Content-Type: text/plain; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG bde and I were discussing of what is likely to be if mmap(2) is switched to start at the max of RLIMIT_DATA. As he seems to be busy now, I post this matter to -arch. Below depicts how we use a process vm space for an i386 elf executable binary: +--------------------+ 4GB | KVM | +--------------------+ 3GB | Process Stack | +--------------------+ down to 3GB - max of RLIMIT_STACK |Reserved for Process| | Stack | +--------------------+ 3GB - max of RLIMIT_STACK | Mmap(2) Heap | +--------------------+ up to 3GB - max of RLIMIT_STACK | Mmap(2)ed space | +--------------------+ MAXDSIZ | Unused | +--------------------+ end of bss + max of RLIMIT_DATA |Reserved for Malloc | +--------------------+ up to end of bss + max of RLIMIT_DATA (break) | Malloc(3) Heap | +--------------------+ end of bss | Executable binary | +--------------------+ _init? | | +--------------------+ 0 The first problem we encountered is that increasing MAXDSIZ limits the size of the address space reserved for mmap(2). Hence it is difficult to run an malloc(3)-bound application and an mmap(2)-bound one on the same machine. bde has proposed to solve this problem by starting mmap(2) at the end of bss + max of RLIMIT_DATA so that a user/application can reserve an arbitrary size of the mmap(2) heap without recompiling a kernel. The usage of a process vm space now looks something like this: | Process Stack | +--------------------+ down to 3GB - max of RLIMIT_STACK |Reserved for Process| | Stack | +--------------------+ 3GB - max of RLIMIT_STACK | Mmap(2) Heap | +--------------------+ up to 3GB - max of RLIMIT_STACK | Mmap(2)ed space | This may be fragmented. +--------------------+ end of bss + max of RLIMIT_DATA |Reserved for Malloc | +--------------------+ up to end of bss + max of RLIMIT_DATA (break) | Malloc(3) Heap | While the max of RLIMIT_DATA now seems to be able to go up, it cannot cross over any mmap(2)ed regions. Since an elf dynamic linker and shared objects are loaded by mmap(2) to reside just above the end of bss + max of RLIMIT_DATA, the max of RLIMIT_DATA cannot be increased in most cases. I propose a solution against that problem by allocating the space just below the stack of a process for mmap(2), as depicted below: | Process Stack | +--------------------+ down to 3GB - max of RLIMIT_STACK |Reserved for Process| | Stack | +--------------------+ 3GB - max of RLIMIT_STACK | Mmap(2)ed space | | (mmap(2), dynamic | This may be fragmented. | linker, shared | | objects, etc) | +--------------------+ down to end of bss + max of RLIMIT_DATA | Mmap(2) Heap | +--------------------+ end of bss + max of RLIMIT_DATA |Reserved for Malloc | +--------------------+ up to end of bss + max of RLIMIT_DATA (break) | Malloc(3) Heap | As the end of bss + max of RLIMIT_DATA gets less likely to cross over mapped regions, we can provide much better flexibility of memory usage than as of now to both malloc(3)-bound and mmap(2)-bound applications. Since we are likely to consume a much smaller size of memory for the stack than for malloc(3) and mmap(2), collision of the stack and the mmap(2)ed space should not be troublesome. Solaris seems to have already taken this method. -- Seigo Tanimura To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Apr 5 1:20: 9 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 345D037B505; Thu, 5 Apr 2001 01:20:05 -0700 (PDT) (envelope-from bde@zeta.org.au) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id SAA03015; Thu, 5 Apr 2001 18:19:59 +1000 Date: Thu, 5 Apr 2001 18:18:44 +1000 (EST) From: Bruce Evans X-Sender: bde@besplex.bde.org To: Terry Lambert Cc: Robert Watson , freebsd-arch@FreeBSD.ORG, dillon@FreeBSD.ORG Subject: Re: Eliminate crget() from nfs kernel code? In-Reply-To: <200104050038.RAA03316@usr08.primenet.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Thu, 5 Apr 2001, Terry Lambert wrote: > [ ... crget() ... ] > > I am not too happy with crget() at the moment. Even discounting > the fact that it calls MALLOC(), and does not check the results > (the new [BAD] semantics permit this to fail under extremely low > memory conditions [FOR NO GOOD REASON] instead of hanging), it is New [BAD] semantics for malloc(..., M_WAITOK) would require some dead bodies :-). I haven't seen any. > If you "fix" crget(), you will also need to fix crdup(). There > are plenty of places where crdup() is called, not just in the > access() system call, where it is bogusly used to replace _only_ > the initial group of the real GID, leaving the groups of the > effective UID active, falsely yielding access to the file, even > if the real UID would have not have contained the same group list > as the effecive UID (gotta love "security" code). This just how access() works. It checks the access that you would have setting the IDs to the real ones. Setting the IDs to the real ones has no effect on the groups list except possibly for removing/ changing the effective GID if that is on the list. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Apr 5 2:48:33 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mout1.freenet.de (mout1.freenet.de [194.97.50.132]) by hub.freebsd.org (Postfix) with ESMTP id 10B3237B423; Thu, 5 Apr 2001 02:48:30 -0700 (PDT) (envelope-from netchild@leidinger.net) Received: from [194.97.50.136] (helo=mx3.freenet.de) by mout1.freenet.de with esmtp (Exim 3.22 #1) id 14l6NQ-00007Q-00; Thu, 05 Apr 2001 11:48:28 +0200 Received: from b8114.pppool.de ([213.7.129.20] helo=Magelan.Leidinger.net) by mx3.freenet.de with esmtp (Exim 3.22 #1) id 14l6NP-0000WK-00; Thu, 05 Apr 2001 11:48:28 +0200 Received: from Leidinger.net (netchild@localhost [127.0.0.1]) by Magelan.Leidinger.net (8.11.3/8.11.3) with ESMTP id f359ctI09858; Thu, 5 Apr 2001 11:38:55 +0200 (CEST) (envelope-from netchild@Leidinger.net) Message-Id: <200104050938.f359ctI09858@Magelan.Leidinger.net> Date: Thu, 5 Apr 2001 11:38:50 +0200 (CEST) From: Alexander Leidinger Subject: Re: Background Fsck To: rwatson@FreeBSD.ORG Cc: mckusick@mckusick.com, arch@FreeBSD.ORG In-Reply-To: MIME-Version: 1.0 Content-Type: TEXT/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 5 Apr, Robert Watson wrote: > Another usability question. Was wondering about the possibility of > multiple background fsck's getting started at a time, et al, possibly due > to bad behavior by the user. Can the user get shot in the foot in the > following situations: [1-3] 4) They shutdown the machine while the background fsck is in progress. Bye, Alexander. -- Speak softly and carry a cellular phone. http://www.Leidinger.net Alexander @ Leidinger.net GPG fingerprint = C518 BC70 E67F 143F BE91 3365 79E2 9C60 B006 3FE7 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Apr 5 2:50:58 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id E701937B43E; Thu, 5 Apr 2001 02:50:55 -0700 (PDT) (envelope-from bright@fw.wintelcom.net) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f359ogo16270; Thu, 5 Apr 2001 02:50:42 -0700 (PDT) Date: Thu, 5 Apr 2001 02:50:42 -0700 From: Alfred Perlstein To: Alexander Leidinger Cc: rwatson@FreeBSD.ORG, mckusick@mckusick.com, arch@FreeBSD.ORG Subject: Re: Background Fsck Message-ID: <20010405025042.G17723@fw.wintelcom.net> References: <200104050938.f359ctI09858@Magelan.Leidinger.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200104050938.f359ctI09858@Magelan.Leidinger.net>; from Alexander@Leidinger.net on Thu, Apr 05, 2001 at 11:38:50AM +0200 X-all-your-base: are belong to us. Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Alexander Leidinger [010405 02:48] wrote: > On 5 Apr, Robert Watson wrote: > > > Another usability question. Was wondering about the possibility of > > multiple background fsck's getting started at a time, et al, possibly due > > to bad behavior by the user. Can the user get shot in the foot in the > > following situations: > > [1-3] > > 4) They shutdown the machine while the background fsck is in progress. I think it's already been explained that a restart is ok. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Apr 5 10: 6:56 2001 Delivered-To: freebsd-arch@freebsd.org Received: from smtp02.primenet.com (smtp02.primenet.com [206.165.6.132]) by hub.freebsd.org (Postfix) with ESMTP id 2428937B440; Thu, 5 Apr 2001 10:06:43 -0700 (PDT) (envelope-from tlambert@usr01.primenet.com) Received: (from daemon@localhost) by smtp02.primenet.com (8.9.3/8.9.3) id JAA09460; Thu, 5 Apr 2001 09:59:36 -0700 (MST) Received: from usr01.primenet.com(206.165.6.201) via SMTP by smtp02.primenet.com, id smtpdAAAQ7aGns; Thu Apr 5 09:59:22 2001 Received: (from tlambert@localhost) by usr01.primenet.com (8.8.5/8.8.5) id KAA24246; Thu, 5 Apr 2001 10:06:18 -0700 (MST) From: Terry Lambert Message-Id: <200104051706.KAA24246@usr01.primenet.com> Subject: Re: Eliminate crget() from nfs kernel code? To: bde@zeta.org.au (Bruce Evans) Date: Thu, 5 Apr 2001 17:06:17 +0000 (GMT) Cc: tlambert@primenet.com (Terry Lambert), rwatson@FreeBSD.ORG (Robert Watson), freebsd-arch@FreeBSD.ORG, dillon@FreeBSD.ORG In-Reply-To: from "Bruce Evans" at Apr 05, 2001 06:18:44 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > [ ... crget() ... ] > > > > I am not too happy with crget() at the moment. Even discounting > > the fact that it calls MALLOC(), and does not check the results > > (the new [BAD] semantics permit this to fail under extremely low > > memory conditions [FOR NO GOOD REASON] instead of hanging), it is > > New [BAD] semantics for malloc(..., M_WAITOK) would require some > dead bodies :-). I haven't seen any. Check every occurance of MALLOC() and the two lines immediately following it in the kernel. Some calls with M_WAITOK check for a NULL value for the assigned pointer; others do not. Someone is using it wrong. I've personally seen a panic, preceeded by several warning messages (which only occur with "INVARIANTS" turned on), under heavy network load. Specifically, write a small program that accepts and closes socket connections, while on the other end, have a client which connects as fast as possible. This is (effectively) a simulation of the highest possible HTTP connection traffic. The messages complaining that the memory on the free list has changed out from under it will occur when the system load as a result of the server hits around 98.72%. The panic can occur anywhere. However, it's easy to accelerate the panic by running a program that sleeps 1 second, runs a grep for something, and sleeps again. A lot of the complaints are about "was type cred". If you end up exacerbating the panic with the grep shell script, the pace it _invariably_ occurs is in grep calling access(2) calling crdup(), where, after the allocation, the uihold() fails on the copy of the credential with a "page not present" error on the attempted increment through indirect (incx instruction). When I check the return of the MALLOC(), I don't see a problem. Obviously, someone is either freeing something twice, using memory they are not allowed to use, or freeing something and then continuing to use it. Though there is a race condition that you could drive a truck through in close(2) (the reference count is decremented to zero, menaing that the descriptor could end up reused, prior to a lot of things being done to the now zero referencs descriptor), a gross closing of that hole indicates that it is not a close race that is the problem, and that the problem has to be occuring in the soclose() code, or in the credentials code. Since this only happens under load, I am assuming an overflow, where an allocation from the free list is not being checked for a wrap. So once again: the finger points to MALLOC().. > > If you "fix" crget(), you will also need to fix crdup(). There > > are plenty of places where crdup() is called, not just in the > > access() system call, where it is bogusly used to replace _only_ > > the initial group of the real GID, leaving the groups of the > > effective UID active, falsely yielding access to the file, even > > if the real UID would have not have contained the same group list > > as the effecive UID (gotta love "security" code). > > This just how access() works. It checks the access that you would > have setting the IDs to the real ones. Setting the IDs to the real > ones has no effect on the groups list except possibly for removing/ > changing the effective GID if that is on the list. The problem with this approach is that a priviledged process which calls setgroups(2) using its effective user ID to permit it to make the call will change the group set for the "real" credential, when the access(2) call occurs. The only way I can see around this is to keep separate cred structures for real and effective IDs, and thus keep seperate groups, and pass the "real" cred structure, instead of a hacked "effective" structure down to do the work. The problem with doing this is that there is not a [sg]etegroups(2) call. Since setgroups(2) is a BSD-ism, it seems to me that the only real way to fix this is to introduce two new BSD-isms for manipulating effective group lists, which would operate on the effective cred's grouplist, instead of on the gloabl group list.. I can't say I'm thrilled with the idea. My personal implementation choice would be to make [sg]etgroups(2) take an additional parameter, and have user space make the call via [sg]et{e}groups(3), with the second parameter selecting between effective and real. This could be made binary backward compatible using knowledge of the number of real arguments on the stack to decide on the behaviour, if necessary. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Apr 5 10:17:55 2001 Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id C3AF637B43E; Thu, 5 Apr 2001 10:17:53 -0700 (PDT) (envelope-from dillon@earth.backplane.com) Received: (from dillon@localhost) by earth.backplane.com (8.11.2/8.9.3) id f35HHhm04264; Thu, 5 Apr 2001 10:17:43 -0700 (PDT) (envelope-from dillon) Date: Thu, 5 Apr 2001 10:17:43 -0700 (PDT) From: Matt Dillon Message-Id: <200104051717.f35HHhm04264@earth.backplane.com> To: Terry Lambert Cc: bde@zeta.org.au (Bruce Evans), tlambert@primenet.com (Terry Lambert), rwatson@FreeBSD.ORG (Robert Watson), freebsd-arch@FreeBSD.ORG, dillon@FreeBSD.ORG Subject: Re: Eliminate crget() from nfs kernel code? References: <200104051706.KAA24246@usr01.primenet.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :Though there is a race condition that you could drive a truck :through in close(2) (the reference count is decremented to zero, :menaing that the descriptor could end up reused, prior to a lot :of things being done to the now zero referencs descriptor), a :gross closing of that hole indicates that it is not a close race :that is the problem, and that the problem has to be occuring in :the soclose() code, or in the credentials code. There is no race condition in close(2) that I know of. I did a pass through all the descriptor code a few months ago and I'm fairly sure that close(2) is just fine. If you believe there is a race condition, review the code and if you find it please email me and I will fix it. Keep in mind that a file pointer is completely independant of its original descriptor once it has been detached from the descriptor. The MALLOC issue seems like one that could be fixed in a few heartbeats. If that is the case, could someone please just do it? Or point out where it is and I'll do it. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Apr 5 15:17:46 2001 Delivered-To: freebsd-arch@freebsd.org Received: from dragon.nuxi.com (trang.nuxi.com [209.152.133.57]) by hub.freebsd.org (Postfix) with ESMTP id 8EF2337B422 for ; Thu, 5 Apr 2001 15:17:44 -0700 (PDT) (envelope-from obrien@NUXI.com) Received: (from obrien@localhost) by dragon.nuxi.com (8.11.3/8.11.1) id f35MHMQ30530; Thu, 5 Apr 2001 15:17:22 -0700 (PDT) (envelope-from obrien) Date: Thu, 5 Apr 2001 15:17:21 -0700 From: "David O'Brien" To: Cyrille Lefevre Cc: "David O'Brien" , Cy Schubert - ITSD Open Systems Group , Jack Rusher , Neil Blakey-Milner , Michael Lucas , "litos2001@libero.it" Subject: Re: Startup scripts a la NetBSD Message-ID: <20010405151721.I28699@dragon.nuxi.com> Reply-To: arch@FreeBSD.ORG References: <3AC4B808.9EB5806B@integratus.com> <200103301810.f2UIAGK06787@cwsys.cwsent.com> <20010330115252.A93566@dragon.nuxi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from clefevre-lists@noos.fr on Wed, Apr 04, 2001 at 05:51:03AM +0200 X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD group X-Pgp-Rsa-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Rsa-Keyid: 1024/34F9F9D5 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Wed, Apr 04, 2001 at 05:51:03AM +0200, Cyrille Lefevre wrote: > "David O'Brien" writes: > > The BSD init + NetBSD granular rc files is a nice compromise. And thus > > why I am pushing a move in that direction. > > some times ago, I've setup a NetBSD box and they init files are not yet > perfect. Speaking of the past is useless. We are talking about the NetBSD _1.5_ rc system. > as I remember me, you couldn't do something like > /etc/rc.d/nfs.server start (or whatever) while nfs.server depends on > something else. You can now. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Apr 5 15:19: 7 2001 Delivered-To: freebsd-arch@freebsd.org Received: from meow.osd.bsdi.com (meow.osd.bsdi.com [204.216.28.88]) by hub.freebsd.org (Postfix) with ESMTP id 98D0637B43F for ; Thu, 5 Apr 2001 15:19:05 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by meow.osd.bsdi.com (8.11.2/8.11.2) with ESMTP id f35MIwG90187 for ; Thu, 5 Apr 2001 15:18:58 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 Date: Thu, 05 Apr 2001 15:18:30 -0700 (PDT) From: John Baldwin To: arch@FreeBSD.org Subject: pfind(0) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Hey gang, Quick kernel question: is it ok if proc0 is inserted into the PID hash table so that pfind(0) returns proc0 instead of NULL or does this break some obscure standard some where? -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Apr 5 15:56:22 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail.noos.fr (claudel.noos.net [212.198.2.83]) by hub.freebsd.org (Postfix) with ESMTP id 27BF637B496 for ; Thu, 5 Apr 2001 15:56:19 -0700 (PDT) (envelope-from root@gits.dyndns.org) Received: (qmail 5068535 invoked by uid 0); 5 Apr 2001 22:55:55 -0000 Received: from d165.dhcp212-198-231.noos.fr (HELO gits.dyndns.org) ([212.198.231.165]) (envelope-sender ) by claudel.noos.net (qmail-ldap-1.03) with DES-CBC3-SHA encrypted SMTP for ; 5 Apr 2001 22:55:55 -0000 Received: (from root@localhost) by gits.dyndns.org (8.11.3/8.11.3) id f35MuBT62124; Fri, 6 Apr 2001 00:56:11 +0200 (CEST) (envelope-from root) From: Cyrille Lefevre Message-Id: <200104052256.f35MuBT62124@gits.dyndns.org> Subject: Re: Startup scripts a la NetBSD In-Reply-To: <20010405151721.I28699@dragon.nuxi.com> "from David O'Brien at Apr 5, 2001 03:17:21 pm" To: arch@FreeBSD.ORG Date: Fri, 6 Apr 2001 00:56:11 +0200 (CEST) Cc: Cy Schubert - ITSD Open Systems Group , Jack Rusher , Neil Blakey-Milner , Michael Lucas , "litos2001@libero.it" Reply-To: clefevre@poboxes.com Organization: ACME X-Face: V|+c;4!|B?E%BE^{E6);aI.[<97Zd*>^#%Y5Cxv;%Y[PT-LW3;A:fRrJ8+^k"e7@+30g0YD0*^^3jgyShN7o?a]C la*Zv'5NA,=963bM%J^o]C X-Mailer: ELM [version 2.4ME+ PL88 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG David O'Brien wrote: > On Wed, Apr 04, 2001 at 05:51:03AM +0200, Cyrille Lefevre wrote: > > "David O'Brien" writes: > > > The BSD init + NetBSD granular rc files is a nice compromise. And thus > > > why I am pushing a move in that direction. > > > > some times ago, I've setup a NetBSD box and they init files are not yet > > perfect. > > Speaking of the past is useless. We are talking about the NetBSD _1.5_ > rc system. I'm also talking about this release. when I say in the past, I've installed NetBSD 1.5 release on a Tadpole SparkBook 3 w/in less than 3 months. > > as I remember me, you couldn't do something like > > /etc/rc.d/nfs.server start (or whatever) while nfs.server depends on > > something else. > > You can now. I'll try again, maybe it's in -stable or -current and not in -release ? the problem is the size of the drive which is about 500 MB :( so, I don't have enough room to stay -stable on this box... Cyrille. -- home: mailto:clefevre@poboxes.com UNIX is user-friendly; it's just particular work: mailto:Cyrille.Lefevre@edf.fr about who it chooses to be friends with. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Apr 5 16:53:17 2001 Delivered-To: freebsd-arch@freebsd.org Received: from Awfulhak.org (awfulhak.demon.co.uk [194.222.196.252]) by hub.freebsd.org (Postfix) with ESMTP id 1350A37B43C for ; Thu, 5 Apr 2001 16:53:13 -0700 (PDT) (envelope-from brian@Awfulhak.org) Received: from hak.lan.Awfulhak.org (root@hak.lan.Awfulhak.org [172.16.0.12]) by Awfulhak.org (8.11.3/8.11.3) with ESMTP id f35NqSU12607; Fri, 6 Apr 2001 00:52:44 +0100 (BST) (envelope-from brian@lan.Awfulhak.org) Received: from hak.lan.Awfulhak.org (brian@localhost [127.0.0.1]) by hak.lan.Awfulhak.org (8.11.3/8.11.3) with ESMTP id f35NqOP02879; Fri, 6 Apr 2001 00:52:24 +0100 (BST) (envelope-from brian@hak.lan.Awfulhak.org) Message-Id: <200104052352.f35NqOP02879@hak.lan.Awfulhak.org> X-Mailer: exmh version 2.3.1 01/18/2001 with nmh-1.0.4 To: freebsd-arch@FreeBSD.org Cc: mckusick@mckusick.com, Brian Somers Subject: A soft-updates optimisation ? Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 06 Apr 2001 00:52:24 +0100 From: Brian Somers Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Hi, I was thinking about soft-updates a bit today and an irritating problem that I (and I assume others) am seeing. If I'm doing a build of some sort (probably make world) and I lose my machine for whatever reason, the machine almost always comes back up with either a binary or object of zero length. I guess the reason is because the write queue looks something like this (from a simplistic point of view): o Create inode with 0 size o Allocate bitmaps o Do a [clustered] write o Update the inode size and times o Repeat previous 3 steps lots of times and the optimiser must be optimising-out all of the inode updates except for the last one. *Assuming* that this is what's happening (to be honest, I haven't looked at the code), wouldn't it be infinitely better if the first step (inode creation) could be delayed 'till the end and merged with that final update ? For build scenarios, this would change the FS behaviour so that the file is either all there or not there. A ``make -DNOCLEAN buildworld'' would pick up where the previous one left off. Comments ? -- Brian Don't _EVER_ lose your sense of humour ! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Apr 5 16:56:51 2001 Delivered-To: freebsd-arch@freebsd.org Received: from aphex.newgold.net (durham0-128.dsl.gtei.net [4.3.0.128]) by hub.freebsd.org (Postfix) with ESMTP id 7715C37B443 for ; Thu, 5 Apr 2001 16:56:45 -0700 (PDT) (envelope-from jmallett@newgold.net) Received: from localhost (jmallett@localhost) by aphex.newgold.net (8.10.1/8.10.1) with ESMTP id f35NtmN24653; Thu, 5 Apr 2001 19:55:48 -0400 (EDT) Date: Thu, 5 Apr 2001 19:55:48 -0400 (EDT) From: Joseph Mallett To: Brian Somers Cc: freebsd-arch@FreeBSD.ORG, mckusick@mckusick.com Subject: Re: A soft-updates optimisation ? In-Reply-To: <200104052352.f35NqOP02879@hak.lan.Awfulhak.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG You mean like, if the data was stored in a "seperate" (as far as the FS _looks_ and _acts_) location and then put in place of the original once the update could be completed? If that's what you mean (or jut holding it in ram) then wouldn't there be overhead problems? Or am I totally missing the point? /joseph -- Joseph Mallett Security Specialist +1 919 349 2976 www.newgold.net josephm@ohsmeg.com jmallett@newgold.net xMach: The proactively unbloated microkernel 4.4BSD-like operating system. www.xMach.org On Fri, 6 Apr 2001, Brian Somers wrote: > Hi, > > I was thinking about soft-updates a bit today and an irritating > problem that I (and I assume others) am seeing. > > If I'm doing a build of some sort (probably make world) and I lose > my machine for whatever reason, the machine almost always comes back > up with either a binary or object of zero length. > > I guess the reason is because the write queue looks something like > this (from a simplistic point of view): > > o Create inode with 0 size > o Allocate bitmaps > o Do a [clustered] write > o Update the inode size and times > o Repeat previous 3 steps lots of times > > and the optimiser must be optimising-out all of the inode updates > except for the last one. > > *Assuming* that this is what's happening (to be honest, I haven't > looked at the code), wouldn't it be infinitely better if the first > step (inode creation) could be delayed 'till the end and merged with > that final update ? > > For build scenarios, this would change the FS behaviour so that the > file is either all there or not there. A ``make -DNOCLEAN > buildworld'' would pick up where the previous one left off. > > Comments ? > -- > Brian > > Don't _EVER_ lose your sense of humour ! > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Apr 5 22:31:34 2001 Delivered-To: freebsd-arch@freebsd.org Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.80]) by hub.freebsd.org (Postfix) with ESMTP id 8768C37B422 for ; Thu, 5 Apr 2001 22:31:31 -0700 (PDT) (envelope-from grog@lemis.com) Received: by wantadilla.lemis.com (Postfix, from userid 1004) id 45BD86ACB8; Fri, 6 Apr 2001 15:01:29 +0930 (CST) Date: Fri, 6 Apr 2001 15:01:29 +0930 From: Greg Lehey To: FreeBSD-arch@FreeBSD.org Subject: Making ddb a kld Message-ID: <20010406150129.O66503@wantadilla.lemis.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG A little over a year, I did some work for Sitara Networks. In the course of this work, I made ddb an lkm, in order to be able to load it on a production system if needed. I'm now in the process of merging Sitara's code, which is based on 3.2-RELEASE, into RELENG_4. Sitara have generously agreed that we can commit any fixes they have in their tree which are not vital to their commercial interests. The ddb mods would be one of them. Do we want to do this? The pros and cons I see are: Pro: 1. We often get people who say "my system crashes. Why". On checking, there's no practical way to determine why, because they don't have a debugger in the system. 2. Occasionally it's convenient to look at what's happening in a running system. Admittedly, you have to freeze the system to use ddb, but it can be of use. Con: 1. This will require moving all the ddb code, which is scattered around the system, into a separate file. I'm prepared to do the work, but it's possible that somebody would complain about the colour of the bikeshed. Here's your chance. Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Apr 5 23:39:23 2001 Delivered-To: freebsd-arch@freebsd.org Received: from gratis.grondar.za (grouter.grondar.za [196.7.18.65]) by hub.freebsd.org (Postfix) with ESMTP id 8FBC237B423 for ; Thu, 5 Apr 2001 23:39:16 -0700 (PDT) (envelope-from mark@grondar.za) Received: from grondar.za (root@gratis.grondar.za [196.7.18.133]) by gratis.grondar.za (8.11.1/8.11.1) with ESMTP id f366cmH32153; Fri, 6 Apr 2001 08:38:51 +0200 (SAST) (envelope-from mark@grondar.za) Message-Id: <200104060638.f366cmH32153@gratis.grondar.za> To: Greg Lehey Cc: FreeBSD-arch@FreeBSD.ORG Subject: Re: Making ddb a kld References: <20010406150129.O66503@wantadilla.lemis.com> In-Reply-To: <20010406150129.O66503@wantadilla.lemis.com> ; from Greg Lehey "Fri, 06 Apr 2001 15:01:29 +0930." Date: Fri, 06 Apr 2001 08:40:06 +0200 From: Mark Murray Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > I'm now in the process of merging Sitara's code, which is based on > 3.2-RELEASE, into RELENG_4. Sitara have generously agreed that we can > commit any fixes they have in their tree which are not vital to their > commercial interests. The ddb mods would be one of them. Do we want > to do this? I'd love this!! M -- Mark Murray Warning: this .sig is umop ap!sdn To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Apr 6 0:13:31 2001 Delivered-To: freebsd-arch@freebsd.org Received: from homer.softweyr.com (bsdconspiracy.net [208.187.122.220]) by hub.freebsd.org (Postfix) with ESMTP id 4F6EB37B443 for ; Fri, 6 Apr 2001 00:13:28 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from [127.0.0.1] (helo=softweyr.com ident=8c0bf464196cc755ddb191f34e9191d7) by homer.softweyr.com with esmtp (Exim 3.16 #1) id 14lQQc-0000JH-00; Fri, 06 Apr 2001 01:13:06 -0600 Message-ID: <3AC9F003.85901B0B@softweyr.com> Date: Tue, 03 Apr 2001 09:45:07 -0600 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.75 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Doug White Cc: David Xu , David O'Brien , Cy Schubert - ITSD Open Systems Group , Jack Rusher , Neil Blakey-Milner , Michael Lucas , "litos2001@libero.it" Subject: Re: Startup scripts a la NetBSD References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Doug White wrote: > > On Mon, 2 Apr 2001, David Xu wrote: > > > In the past, I saw lots of people refused to see rc.d style in > > FreeBSD, include some guys in core team, at that time, I'm very > > disappointed, why is it back again? is this just because of NetBSD? > > Because sysv-style init scripts actually make sense for both admins and > installed packages? It's not traditionally BSD, but unfortunately programs > are much more complex today, generally with multiple daemons and > interdependencies for starting up properly. A vendor-supplied script > makes sure everything is set up to start the daemons/packages correctly, > and gives a nice start/stop knob for the admin to control it with. No, it doesn't. Vendor-supplied scripts ala SysV have little chance of getting any level of interdependency right, this is one of the reasons FreeBSD developers have refused to take the SysV approach. Must discussion has ensued over how to actually solve this problem, instead of leaving it to the system administrator to magically conjure up what the dependencies are then hack in the correct order of Sxxx and Kxxx BS to make it work. If you have a solution, we're ready to see and test it. > For the record, I hacked up this pile of scripts to emulate one runlevel, > along with a linux-style chkconfig(8) script. You just have to call the > start-rc3.sh script at some point in the boot, and stop-rc3.sh when > shutting down. Since we have rc.shutdown.d you don't have to edit > rc.shutdown anymore. If you really want Linnex-style /etc/rc.d/init.d/ > and friends, this will give it to you. Ports do this, of course, by creating scripts in /usr/local/etc/rc.d or /usr/X11R6/etc/rc.d, and get called with both 'start' and 'stop' arguments. If you have a way of REALLY creating a dependency tree, please let us know. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Apr 6 0:13:37 2001 Delivered-To: freebsd-arch@freebsd.org Received: from homer.softweyr.com (bsdconspiracy.net [208.187.122.220]) by hub.freebsd.org (Postfix) with ESMTP id C12A737B506 for ; Fri, 6 Apr 2001 00:13:28 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from [127.0.0.1] (helo=softweyr.com ident=f427b9f9567ff881379d9198d53a4e97) by homer.softweyr.com with esmtp (Exim 3.16 #1) id 14lQQb-0000JF-00; Fri, 06 Apr 2001 01:13:05 -0600 Message-ID: <3AC9E86B.A8D976D4@softweyr.com> Date: Tue, 03 Apr 2001 09:12:43 -0600 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.75 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: David Xu Cc: Doug White , David O'Brien , Cy Schubert - ITSD Open Systems Group , Jack Rusher , Neil Blakey-Milner , Michael Lucas , "litos2001@libero.it" Subject: Re: Startup scripts a la NetBSD References: <39898261.20010403084256@21cn.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG David Xu wrote: > > Hello Doug, > > Tuesday, April 03, 2001, 6:26:10 AM, you wrote: > > DW> On Mon, 2 Apr 2001, David Xu wrote: > > >> In the past, I saw lots of people refused to see rc.d style in > >> FreeBSD, include some guys in core team, at that time, I'm very > >> disappointed, why is it back again? is this just because of NetBSD? > > DW> Because sysv-style init scripts actually make sense for both admins and > DW> installed packages? It's not traditionally BSD, but unfortunately programs > DW> are much more complex today, generally with multiple daemons and > DW> interdependencies for starting up properly. A vendor-supplied script > DW> makes sure everything is set up to start the daemons/packages correctly, > DW> and gives a nice start/stop knob for the admin to control it with. > > DW> For the record, I hacked up this pile of scripts to emulate one runlevel, > DW> along with a linux-style chkconfig(8) script. You just have to call the > DW> start-rc3.sh script at some point in the boot, and stop-rc3.sh when > DW> shutting down. Since we have rc.shutdown.d you don't have to edit > DW> rc.shutdown anymore. If you really want Linnex-style /etc/rc.d/init.d/ > DW> and friends, this will give it to you. > > DW> It should be in > DW> ftp://ftp.freebsd.org/pub/FreeBSD/ports/local-distfiles/dwhite/sysvrc-1.1.tar.gz, > DW> or any other MASTER_SITE_LOCAL mirror. > > DW> Doug White | FreeBSD: The Power to Serve > DW> dwhite@resnet.uoregon.edu | www.FreeBSD.org > > I like the idea, it has unique start/stop interface, I needn't > remember obscure parameters for every daemon, it really got me. > but am I dreaming? I suspect it will never be native supported > in FreeBSD. That depends on what you mean by "sysv-like". If you want runlevels 0-6 supported by init, probably not. If you want individual packages started by their own scripts, FreeBSD already has this to some extent, and even allows you to add directories. See /etc/defaults/rc.conf, which contains: local_startup="/usr/local/etc/rc.d /usr/X11R6/etc/rc.d" This allows everything you want above, for packages that are added to FreeBSD. It does not handle dependencies between packages, or state a startup order between directories in $local_startup or between the scripts in each directory. If you'd like to improve any of the above, or to move parts of the standard system startup scripts to, for instance, /etc/rc.d, with some explicit ordering, we'd like to hear it. This is FreeBSD, patches talk. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Apr 6 2:31:12 2001 Delivered-To: freebsd-arch@freebsd.org Received: from dragon.nuxi.com (trang.nuxi.com [209.152.133.57]) by hub.freebsd.org (Postfix) with ESMTP id A1A4A37B423 for ; Fri, 6 Apr 2001 02:31:10 -0700 (PDT) (envelope-from obrien@NUXI.com) Received: (from obrien@localhost) by dragon.nuxi.com (8.11.3/8.11.1) id f369QEX17839; Fri, 6 Apr 2001 02:26:14 -0700 (PDT) (envelope-from obrien) Date: Fri, 6 Apr 2001 02:26:06 -0700 From: "David O'Brien" To: Wes Peters Cc: Doug White , David Xu , "David O'Brien" , Cy Schubert - ITSD Open Systems Group , Jack Rusher , Neil Blakey-Milner , Michael Lucas , "litos2001@libero.it" Subject: Re: Startup scripts a la NetBSD Message-ID: <20010406022605.A17550@dragon.nuxi.com> Reply-To: arch@FreeBSD.ORG References: <3AC9F003.85901B0B@softweyr.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <3AC9F003.85901B0B@softweyr.com>; from wes@softweyr.com on Tue, Apr 03, 2001 at 09:45:07AM -0600 X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD group X-Pgp-Rsa-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Rsa-Keyid: 1024/34F9F9D5 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, Apr 03, 2001 at 09:45:07AM -0600, Wes Peters wrote: > Must discussion has ensued over how to actually solve this problem, instead > of leaving it to the system administrator to magically conjure up what the > dependencies are then hack in the correct order of Sxxx and Kxxx BS to make > it work. If you have a solution, we're ready to see and test it. NetBSD has a solution. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Apr 6 3: 7:47 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mailout06.sul.t-online.com (mailout06.sul.t-online.com [194.25.134.19]) by hub.freebsd.org (Postfix) with ESMTP id 022C337B43C for ; Fri, 6 Apr 2001 03:07:43 -0700 (PDT) (envelope-from alex@cichlids.cichlids.com) Received: from fwd01.aul.t-online.de by mailout06.sul.t-online.com with smtp id 14lT9a-00022y-00; Fri, 06 Apr 2001 12:07:42 +0200 Received: from neutron.cichlids.com (520050424122-0001@[217.1.53.94]) by fmrl01.sul.t-online.com with esmtp id 14lT93-1CK9aaC; Fri, 6 Apr 2001 12:07:09 +0200 Received: from cichlids.cichlids.com (cichlids.cichlids.com [192.168.0.10]) by neutron.cichlids.com (Postfix) with ESMTP id 8C762AB44; Fri, 6 Apr 2001 12:07:36 +0200 (CEST) Received: by cichlids.cichlids.com (Postfix, from userid 1001) id 47FFC14AF8; Fri, 6 Apr 2001 12:07:19 +0200 (CEST) Date: Fri, 6 Apr 2001 12:07:19 +0200 From: Alexander Langer To: Greg Lehey Cc: FreeBSD-arch@FreeBSD.org Subject: Re: Making ddb a kld Message-ID: <20010406120719.A797@cichlids.cichlids.com> References: <20010406150129.O66503@wantadilla.lemis.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010406150129.O66503@wantadilla.lemis.com>; from grog@lemis.com on Fri, Apr 06, 2001 at 03:01:29PM +0930 X-PGP-Fingerprint: 44 28 CA 4C 46 5B D3 A8 A8 E3 BA F3 4E 60 7D 7F X-PGP-at: finger alex@big.endian.de X-Verwirrung: Dieser Header dient der allgemeinen Verwirrung. X-Sender: 520050424122-0001@t-dialin.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Thus spake Greg Lehey (grog@lemis.com): > 1. This will require moving all the ddb code, which is scattered > around the system, into a separate file. I'm prepared to do the > work, but it's possible that somebody would complain about the > colour of the bikeshed. Here's your chance. This isn't really a con if you do that. However, I'd love to see that, for various, obvious reasons. Alex To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Apr 6 8:19:37 2001 Delivered-To: freebsd-arch@freebsd.org Received: from homer.softweyr.com (bsdconspiracy.net [208.187.122.220]) by hub.freebsd.org (Postfix) with ESMTP id E6E2037B43F for ; Fri, 6 Apr 2001 08:19:32 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from [127.0.0.1] (helo=softweyr.com ident=241abc72570a024ff9ff8064a8756d93) by homer.softweyr.com with esmtp (Exim 3.16 #1) id 14lY0F-000060-00; Fri, 06 Apr 2001 09:18:23 -0600 Message-ID: <3ACDDE3E.79BBBDAC@softweyr.com> Date: Fri, 06 Apr 2001 09:18:22 -0600 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.75 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Andrew Reilly Cc: Jack Rusher , jonathan@graehl.org, freebsd-arch@FreeBSD.ORG Subject: Re: configuration files, XML, Mac OS X release References: <20010325170513Z.jkh@osd.bsdi.com> <3ABEB519.CA9F1029@integratus.com> <20010326181443.A75840@gurney.reilly.home> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Andrew Reilly wrote: > > On Sun, Mar 25, 2001 at 07:18:49PM -0800, Jack Rusher wrote: > > Jordan Hubbard wrote: > > > > > > The project has a Mac (eatmorepie.freebsd.org) running the OS X > > > release candidate, though it appears to be down at the moment (I'll > > > fix that). Someone can certainly log into it at some point and > > > look this stuff over. > > > > I took a look at the release during the big pre release party at > > Apple. It looks like the only XML DTD they have in the system is a > > property list DTD designed to allow them to store groups of key/value > > pairs as the backing store for application configuration data. > > > > I remain willing to help integrate a lightweight BSD license XML based > > configuration management system into FreeBSD. Does anybody really want > > such a beast? > > I really like the idea of a uniform and configurator-friendly > config management system. I'm not crazy about XML, but my > opinion on the subject hardly matters. The problem with XML is that it is an entirely adequate language to export to and import from, but quite ugly to store in, since every setting is arbitrarily long. This isn't much different from what we have now, with rc.conf et al, but we don't really edit the existing script files programmatically. In order to change a setting using XML, you end up parsing the entire file, changing the setting in-memory, then re-writing the file, which is less than elegant. I'd sure like to see us come up with a better solution before plunging off down this path. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Apr 6 14:59:39 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 29F5C37B424 for ; Fri, 6 Apr 2001 14:59:37 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.3/8.11.3) with SMTP id f36Lxxf13869; Fri, 6 Apr 2001 17:59:59 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Fri, 6 Apr 2001 17:59:59 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Terry Lambert Cc: Matt Dillon , freebsd-arch@FreeBSD.ORG Subject: Re: Eliminate crget() from nfs kernel code? In-Reply-To: <200104050022.RAA02886@usr08.primenet.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Thu, 5 Apr 2001, Terry Lambert wrote: > It's implicit in the fact that results caching is permitted; that means > that the results you get need to be valid for everyone. The only way to > assure this is to use a credential that can always get results. The > only alternative is not to cache. My reading of nfs_statfs() was that the results aren't cached in our implementation. Is this an incorrect interpretation? My hope was to preserve the real credentials wherever possible, avoiding arbitrarily constructing credentials. If a credential must be constructed without paying attention to the creation context, then it should be an NFS credential, not a normal subject credential. My understanding also is, BTW, that uid 0 is not required to be special for servers, and so the constructed credential may not even provide magic capabilities. > > I think you want to keep crget()/crfree() if possible, just so you > > don't have to worry about the data being sent over the wire breaking > > some poor sod. But if you want to bite the bullet and use the process > > ucred I'd say go for it - keep a careful watch for complaints of > > FreeBSD suddenly failing against various NFS servers though. > > Note that this means you must keep the processes from which you derive > the data around, so if you have a stalled call blocked on a server > reboot (for example), you cause a lot of problems. Again, I'm not sure I follow. Here's my reading of how the code currently works: instantiate a credential make the NFS RPC free the credential This isn't different, from a reference count perspective from make the NFS RPC with an existing credential In both cases, there a valid held reference to the credential on calling into the implementation, and it is valid for the durection of the function call. If the NFS implementation needs to preserve the credential, it will do so by bumping the reference count. However, a quick glance at the code doesn't make it clear to me that it even needs to do that. In any case, the validity of the reference is assured until the function exits, at which point both (today) and (soon) have the same behavior: the cred could potentially be released almost immediately. Sooner, in fact, with the crget/crfree, but the same race if the bug did exist. Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Apr 6 15:22:23 2001 Delivered-To: freebsd-arch@freebsd.org Received: from smtp05.primenet.com (smtp05.primenet.com [206.165.6.135]) by hub.freebsd.org (Postfix) with ESMTP id 0DCAC37B43C for ; Fri, 6 Apr 2001 15:22:22 -0700 (PDT) (envelope-from tlambert@usr01.primenet.com) Received: (from daemon@localhost) by smtp05.primenet.com (8.9.3/8.9.3) id PAA17303; Fri, 6 Apr 2001 15:16:23 -0700 (MST) Received: from usr01.primenet.com(206.165.6.201) via SMTP by smtp05.primenet.com, id smtpdAAAtvaiWH; Fri Apr 6 15:16:14 2001 Received: (from tlambert@localhost) by usr01.primenet.com (8.8.5/8.8.5) id PAA05222; Fri, 6 Apr 2001 15:22:10 -0700 (MST) From: Terry Lambert Message-Id: <200104062222.PAA05222@usr01.primenet.com> Subject: Re: Making ddb a kld To: grog@lemis.com (Greg Lehey) Date: Fri, 6 Apr 2001 22:22:05 +0000 (GMT) Cc: FreeBSD-arch@FreeBSD.ORG In-Reply-To: <20010406150129.O66503@wantadilla.lemis.com> from "Greg Lehey" at Apr 06, 2001 03:01:29 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > 1. This will require moving all the ddb code, which is scattered > around the system, into a separate file. I'm prepared to do the > work, but it's possible that somebody would complain about the > colour of the bikeshed. Here's your chance. How does it deal with the locore.s issues? Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Apr 6 15:49:57 2001 Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 7AF1837B42C; Fri, 6 Apr 2001 15:49:53 -0700 (PDT) (envelope-from dillon@earth.backplane.com) Received: (from dillon@localhost) by earth.backplane.com (8.11.2/8.9.3) id f36Mnr448378; Fri, 6 Apr 2001 15:49:53 -0700 (PDT) (envelope-from dillon) Date: Fri, 6 Apr 2001 15:49:53 -0700 (PDT) From: Matt Dillon Message-Id: <200104062249.f36Mnr448378@earth.backplane.com> To: Robert Watson Cc: Terry Lambert , freebsd-arch@FreeBSD.ORG Subject: Re: Eliminate crget() from nfs kernel code? References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG : : :On Thu, 5 Apr 2001, Terry Lambert wrote: : :> It's implicit in the fact that results caching is permitted; that means :> that the results you get need to be valid for everyone. The only way to :> assure this is to use a credential that can always get results. The :> only alternative is not to cache. : :My reading of nfs_statfs() was that the results aren't cached in our :implementation. Is this an incorrect interpretation? My hope was to :preserve the real credentials wherever possible, avoiding arbitrarily :constructing credentials. If a credential must be constructed without :paying attention to the creation context, then it should be an NFS :credential, not a normal subject credential. My understanding also is, :BTW, that uid 0 is not required to be special for servers, and so the :constructed credential may not even provide magic capabilities. nfs_statfs() is not cached. You should be able to use the process ucred unmodified, though no testing has been done so we can't be absolutely sure it will work with a wide variety of NFS server platforms until we try it. It's worth finding out, though. I'd like to see this bit of code cleaned up. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Apr 6 16: 8:55 2001 Delivered-To: freebsd-arch@freebsd.org Received: from smtp03.primenet.com (smtp03.primenet.com [206.165.6.133]) by hub.freebsd.org (Postfix) with ESMTP id 9E32837B423; Fri, 6 Apr 2001 16:08:51 -0700 (PDT) (envelope-from tlambert@usr01.primenet.com) Received: (from daemon@localhost) by smtp03.primenet.com (8.9.3/8.9.3) id QAA05296; Fri, 6 Apr 2001 16:05:20 -0700 (MST) Received: from usr01.primenet.com(206.165.6.201) via SMTP by smtp03.primenet.com, id smtpdAAAmcaWrk; Fri Apr 6 16:05:10 2001 Received: (from tlambert@localhost) by usr01.primenet.com (8.8.5/8.8.5) id QAA05544; Fri, 6 Apr 2001 16:08:37 -0700 (MST) From: Terry Lambert Message-Id: <200104062308.QAA05544@usr01.primenet.com> Subject: Re: Eliminate crget() from nfs kernel code? To: rwatson@FreeBSD.ORG (Robert Watson) Date: Fri, 6 Apr 2001 23:08:31 +0000 (GMT) Cc: tlambert@primenet.com (Terry Lambert), dillon@earth.backplane.com (Matt Dillon), freebsd-arch@FreeBSD.ORG In-Reply-To: from "Robert Watson" at Apr 06, 2001 05:59:59 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > It's implicit in the fact that results caching is permitted; that means > > that the results you get need to be valid for everyone. The only way to > > assure this is to use a credential that can always get results. The > > only alternative is not to cache. > > My reading of nfs_statfs() was that the results aren't cached in our > implementation. Is this an incorrect interpretation? No, you are right. FreeBSD's implementation is unnecessarily slow. > My hope was to > preserve the real credentials wherever possible, avoiding arbitrarily > constructing credentials. If a credential must be constructed without > paying attention to the creation context, then it should be an NFS > credential, not a normal subject credential. My understanding also is, > BTW, that uid 0 is not required to be special for servers, and so the > constructed credential may not even provide magic capabilities. I would much prefer the "NFS credential" approach to doing this, for obvious reasons. > > > I think you want to keep crget()/crfree() if possible, just so you > > > don't have to worry about the data being sent over the wire breaking > > > some poor sod. But if you want to bite the bullet and use the process > > > ucred I'd say go for it - keep a careful watch for complaints of > > > FreeBSD suddenly failing against various NFS servers though. > > > > Note that this means you must keep the processes from which you derive > > the data around, so if you have a stalled call blocked on a server > > reboot (for example), you cause a lot of problems. > > Again, I'm not sure I follow. Here's my reading of how the code currently > works: > > instantiate a credential > make the NFS RPC > free the credential > > This isn't different, from a reference count perspective from > > make the NFS RPC with an existing credential instantiate a credential make the NFS RPC free the credential > into the implementation, and it is valid for the durection of the function > call. If the NFS implementation needs to preserve the credential, it will > do so by bumping the reference count. It was not clear to me from a casual perusal of the code whether or not holding a reference to a credential used by a process would keep it from going away, or if it always went away on the 1->0 decrement. It seems to me that credential instantiation becomes a problem, and it also seems to me that this is what you are trying to fix with your changes elsewhere. Eventually, the credential has to come from somewhere, and to come from somewhere, it has to be allocated. > In any case, > the validity of the reference is assured until the function exits, at > which point both (today) and (soon) have the same behavior: the cred could > potentially be released almost immediately. Sooner, in fact, with the > crget/crfree, but the same race if the bug did exist. Yep. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Apr 6 16:11:48 2001 Delivered-To: freebsd-arch@freebsd.org Received: from smtp04.primenet.com (smtp04.primenet.com [206.165.6.134]) by hub.freebsd.org (Postfix) with ESMTP id 74CCB37B43F; Fri, 6 Apr 2001 16:11:44 -0700 (PDT) (envelope-from tlambert@usr01.primenet.com) Received: (from daemon@localhost) by smtp04.primenet.com (8.9.3/8.9.3) id QAA02816; Fri, 6 Apr 2001 16:11:26 -0700 (MST) Received: from usr01.primenet.com(206.165.6.201) via SMTP by smtp04.primenet.com, id smtpdAAAk2aaAf; Fri Apr 6 16:11:22 2001 Received: (from tlambert@localhost) by usr01.primenet.com (8.8.5/8.8.5) id QAA05553; Fri, 6 Apr 2001 16:11:29 -0700 (MST) From: Terry Lambert Message-Id: <200104062311.QAA05553@usr01.primenet.com> Subject: Re: Eliminate crget() from nfs kernel code? To: dillon@earth.backplane.com (Matt Dillon) Date: Fri, 6 Apr 2001 23:11:23 +0000 (GMT) Cc: tlambert@primenet.com (Terry Lambert), bde@zeta.org.au (Bruce Evans), rwatson@FreeBSD.ORG (Robert Watson), freebsd-arch@FreeBSD.ORG, dillon@FreeBSD.ORG In-Reply-To: <200104051717.f35HHhm04264@earth.backplane.com> from "Matt Dillon" at Apr 05, 2001 10:17:43 AM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > The MALLOC issue seems like one that could be fixed in a few > heartbeats. If that is the case, could someone please just do it? > Or point out where it is and I'll do it. Matt, I'll send you the test programs seperately via email (no need to spam the list). Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Apr 6 16:22:27 2001 Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 7F7DD37B423; Fri, 6 Apr 2001 16:22:25 -0700 (PDT) (envelope-from dillon@earth.backplane.com) Received: (from dillon@localhost) by earth.backplane.com (8.11.2/8.9.3) id f36NMOP49124; Fri, 6 Apr 2001 16:22:24 -0700 (PDT) (envelope-from dillon) Date: Fri, 6 Apr 2001 16:22:24 -0700 (PDT) From: Matt Dillon Message-Id: <200104062322.f36NMOP49124@earth.backplane.com> To: Terry Lambert Cc: tlambert@primenet.com (Terry Lambert), bde@zeta.org.au (Bruce Evans), rwatson@FreeBSD.ORG (Robert Watson), freebsd-arch@FreeBSD.ORG Subject: Re: Eliminate crget() from nfs kernel code? References: <200104062311.QAA05553@usr01.primenet.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :> The MALLOC issue seems like one that could be fixed in a few :> heartbeats. If that is the case, could someone please just do it? :> Or point out where it is and I'll do it. : :Matt, I'll send you the test programs seperately via email (no :need to spam the list). : : : Terry Lambert : terry@lambert.org That would be great. I have some time this weekend. If I can reproduce the crash I'll fix it. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Apr 6 17:28: 8 2001 Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 05A2E37B422 for ; Fri, 6 Apr 2001 17:28:05 -0700 (PDT) (envelope-from dillon@earth.backplane.com) Received: (from dillon@localhost) by earth.backplane.com (8.11.2/8.9.3) id f370QfY50900; Fri, 6 Apr 2001 17:26:41 -0700 (PDT) (envelope-from dillon) Date: Fri, 6 Apr 2001 17:26:41 -0700 (PDT) From: Matt Dillon Message-Id: <200104070026.f370QfY50900@earth.backplane.com> To: Seigo Tanimura Cc: arch@FreeBSD.ORG, bde@zeta.org.au, Seigo Tanimura Subject: Re: Mmap(2) should start just below stack (was: Re: Bumping up {MAX,DFL}*SIZ in i386) References: <200103191056.f2JAuox00630@rina.r.dl.itc.u-tokyo.ac.jp> <200103230517.f2N5HXx08605@rina.r.dl.itc.u-tokyo.ac.jp> <200104050506.f3556Xw28400@rina.r.dl.itc.u-tokyo.ac.jp> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :I propose a solution against that problem by allocating the space just :below the stack of a process for mmap(2), as depicted below: : :| Process Stack | :+--------------------+ down to 3GB - max of RLIMIT_STACK :|Reserved for Process| :| Stack | :+--------------------+ 3GB - max of RLIMIT_STACK :| Mmap(2)ed space | :| (mmap(2), dynamic | This may be fragmented. :| linker, shared | :| objects, etc) | :+--------------------+ down to end of bss + max of RLIMIT_DATA :| Mmap(2) Heap | :+--------------------+ end of bss + max of RLIMIT_DATA :|Reserved for Malloc | :+--------------------+ up to end of bss + max of RLIMIT_DATA (break) :| Malloc(3) Heap | : :As the end of bss + max of RLIMIT_DATA gets less likely to cross over :mapped regions, we can provide much better flexibility of memory usage :... :Seigo Tanimura Hmm. Well, I like the idea of allocating from the end of the stack downwards a whole lot better then the idea of allocating from the end of RLIMIT_DATA upwards, but we have an issue in both cases that we have to deal with. suid-root programs often adjust resources upwards in order to avoid potential root compromises due to allocation failures at just the wrong time (coupled with a badly written program). Most commonly this means RLIMIT_DATA will be increased and, of course, many programs will also increase RLIMIT_DATA. However, the same problem with suid-root programs exists for RLIMIT_STACK as well. So using a RLIMIT_STACK based solution instead of RLIMIT_DATA only partially solves the problem. We also have a similar issue with fork(). Process A fork()'s and adjusts the resource limits for the child process downward or upward. -- One thing to note is that anonymous memory can trivially be allocated with mmap(), and will end up in the mmap space rather then the DATA/RSS space. Thus we have a potential solution that does not require any kernel modifications at all - use mmap() to allocate memory. Of course, at the moment, programs that allocate huge amounts of memory with malloc() would have to be aware of this fact to take advantage of it. I don't know what the correct solution is, but I am leary of making the mmap() space relative to user-adjustable resources. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Apr 6 18:17:15 2001 Delivered-To: freebsd-arch@freebsd.org Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.80]) by hub.freebsd.org (Postfix) with ESMTP id 8DF2B37B423 for ; Fri, 6 Apr 2001 18:17:10 -0700 (PDT) (envelope-from grog@lemis.com) Received: by wantadilla.lemis.com (Postfix, from userid 1004) id 35E486ACB7; Sat, 7 Apr 2001 10:47:09 +0930 (CST) Date: Sat, 7 Apr 2001 10:47:09 +0930 From: Greg Lehey To: Terry Lambert Cc: FreeBSD-arch@FreeBSD.ORG Subject: Re: Making ddb a kld Message-ID: <20010407104709.B24010@wantadilla.lemis.com> References: <20010406150129.O66503@wantadilla.lemis.com> <200104062222.PAA05222@usr01.primenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200104062222.PAA05222@usr01.primenet.com>; from tlambert@primenet.com on Fri, Apr 06, 2001 at 10:22:05PM +0000 Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Friday, 6 April 2001 at 22:22:05 +0000, Terry Lambert wrote: >> 1. This will require moving all the ddb code, which is scattered >> around the system, into a separate file. I'm prepared to do the >> work, but it's possible that somebody would complain about the >> colour of the bikeshed. Here's your chance. > > How does it deal with the locore.s issues? Which locore.s issues? Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Apr 6 18:53: 6 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id C61A637B424 for ; Fri, 6 Apr 2001 18:53:03 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.3/8.11.3) with SMTP id f371rWf16452; Fri, 6 Apr 2001 21:53:33 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Fri, 6 Apr 2001 21:53:32 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Matt Dillon Cc: Terry Lambert , freebsd-arch@FreeBSD.ORG Subject: Re: Eliminate crget() from nfs kernel code? In-Reply-To: <200104062249.f36Mnr448378@earth.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, 6 Apr 2001, Matt Dillon wrote: > nfs_statfs() is not cached. You should be able to use the > process ucred unmodified, though no testing has been done > so we can't be absolutely sure it will work with a wide > variety of NFS server platforms until we try it. It's worth > finding out, though. I'd like to see this bit of code cleaned > up. Ok, I've committed a change to 5.0-CURRENT to move to simply using the p->p_ucred rather than constructing a ucred using crget(). I've interop'd with Solaris and FreeBSD NFS servers, and it seemed to work fine. I'd appreciate it if others could do testing -- the primary test is simply whether "df /your/nfs/mount" running as non-root DTRT. I haven't had a chance yet, but sometime in the next day or two I'll get out and RPC dumper (dunno if tcpdump knows how to do this, maybe snoop from Solaris does) and look at the credentials used for NFSPROC_STATFS requests generated by other implementations. Chances are, if another already uses normal non-uid-0 non-gid-0 credentials, then we're safe. There's similar code in the ncp implementation, but I haven't followed up on that yet. Moving over to using p->p_ucred did fix my panics with an expanded ucred, so the current fix remedies the panic I was running into during my development. Dunno if Boris is reading this, but if so, perhaps he can look at a similar change in netncp. Thanks, Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Apr 7 0:39:31 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 4359E37B43C for ; Sat, 7 Apr 2001 00:39:28 -0700 (PDT) (envelope-from bde@zeta.org.au) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id RAA17940; Sat, 7 Apr 2001 17:39:21 +1000 Date: Sat, 7 Apr 2001 17:37:27 +1000 (EST) From: Bruce Evans X-Sender: bde@besplex.bde.org To: Greg Lehey Cc: FreeBSD-arch@FreeBSD.ORG Subject: Re: Making ddb a kld In-Reply-To: <20010406150129.O66503@wantadilla.lemis.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, 6 Apr 2001, Greg Lehey wrote: > A little over a year, I did some work for Sitara Networks. In the > course of this work, I made ddb an lkm, in order to be able to load it > on a production system if needed. > > I'm now in the process of merging Sitara's code, which is based on > 3.2-RELEASE, into RELENG_4. Sitara have generously agreed that we can > commit any fixes they have in their tree which are not vital to their > commercial interests. The ddb mods would be one of them. Do we want > to do this? Of course not. > The pros and cons I see are: > > Pro: > > 1. We often get people who say "my system crashes. Why". On > checking, there's no practical way to determine why, because they > don't have a debugger in the system. They can run gdb if they have a panic dump. I have thought of making ddb backtraces for panics a standard part of the kernel. People shouldn't need to know how to run ddb or ddb to produce useful crash reports. Having an loadable ddb is not much more useful than having a ddb option. It will never be there when you need it unless you always configure it. > 2. Occasionally it's convenient to look at what's happening in a > running system. Admittedly, you have to freeze the system to use > ddb, but it can be of use. Again, it is better to always configure ddb, in case you need it to look at what's happening on a hung system that can't load a module. > Con: > > 1. This will require moving all the ddb code, which is scattered > around the system, into a separate file. I'm prepared to do the > work, but it's possible that somebody would complain about the > colour of the bikeshed. Here's your chance. This would demodularise all the code that is scattered around the system. It would also give the nice version mismatch problems: e.g., ps in the kernel would tend to panic the whole system where ps in userland would just panic itself. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Apr 7 19: 1: 5 2001 Delivered-To: freebsd-arch@freebsd.org Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.80]) by hub.freebsd.org (Postfix) with ESMTP id 2996C37B422 for ; Sat, 7 Apr 2001 19:01:00 -0700 (PDT) (envelope-from grog@lemis.com) Received: by wantadilla.lemis.com (Postfix, from userid 1004) id 86D5C6ACB7; Sun, 8 Apr 2001 11:30:54 +0930 (CST) Date: Sun, 8 Apr 2001 11:30:54 +0930 From: Greg Lehey To: Bruce Evans Cc: FreeBSD-arch@FreeBSD.ORG Subject: Re: Making ddb a kld Message-ID: <20010408113054.L76422@wantadilla.lemis.com> References: <20010406150129.O66503@wantadilla.lemis.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from bde@zeta.org.au on Sat, Apr 07, 2001 at 05:37:27PM +1000 Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Saturday, 7 April 2001 at 17:37:27 +1000, Bruce Evans wrote: > On Fri, 6 Apr 2001, Greg Lehey wrote: > >> A little over a year, I did some work for Sitara Networks. In the >> course of this work, I made ddb an lkm, in order to be able to load it >> on a production system if needed. >> >> I'm now in the process of merging Sitara's code, which is based on >> 3.2-RELEASE, into RELENG_4. Sitara have generously agreed that we can >> commit any fixes they have in their tree which are not vital to their >> commercial interests. The ddb mods would be one of them. Do we want >> to do this? > > Of course not. > >> The pros and cons I see are: >> >> Pro: >> >> 1. We often get people who say "my system crashes. Why". On >> checking, there's no practical way to determine why, because they >> don't have a debugger in the system. > > They can run gdb if they have a panic dump. If they have a panic dump and a kernel with symbols. You'll recall that all attempts of mine to get the latter as default have been shot down in flames. > I have thought of making ddb backtraces for panics a standard part > of the kernel. People shouldn't need to know how to run ddb or ddb > to produce useful crash reports. Sounds good. I'd certainly agree with that. > Having an loadable ddb is not much more useful than having a ddb > option. It will never be there when you need it unless you always > configure it. Consider following scenario, on a production machine. Machine works fine for months, then starts crashing. Remote support dials in, loads the ddb module into the running system, then waits. When the machine crashes, remote support dials in again and debugs via serial interface. Not the best way to do things, admittedly, but one that some people like to do. To quote one respected member of our community: > I normally use ddb anyway -- gdb takes too long to start up and > doesn't work well for low-level stuff. >> 2. Occasionally it's convenient to look at what's happening in a >> running system. Admittedly, you have to freeze the system to use >> ddb, but it can be of use. > > Again, it is better to always configure ddb, in case you need it to > look at what's happening on a hung system that can't load a module. Then we should make it part of the default kernel. But there will always be cases where one particular debugging method doesn't work. That doesn't disqualify it completely as a method. >> Con: >> >> 1. This will require moving all the ddb code, which is scattered >> around the system, into a separate file. I'm prepared to do the >> work, but it's possible that somebody would complain about the >> colour of the bikeshed. Here's your chance. > > This would demodularise all the code that is scattered around the > system. Yes, that's what I said. > It would also give the nice version mismatch problems: e.g., ps in > the kernel would tend to panic the whole system where ps in userland > would just panic itself. Version mismatch problems are another issue, one which we should address. They don't belong in this discussion. But I thought that ddb would recover from that kind of error, just as it does if you try to display the contents of unmapped memory. Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Apr 7 22:51:50 2001 Delivered-To: freebsd-arch@freebsd.org Received: from homer.softweyr.com (bsdconspiracy.net [208.187.122.220]) by hub.freebsd.org (Postfix) with ESMTP id B5E4D37B423 for ; Sat, 7 Apr 2001 22:51:46 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from [127.0.0.1] (helo=softweyr.com ident=5dd0681e679c9f40321b02cb1fcc0028) by homer.softweyr.com with esmtp (Exim 3.16 #1) id 14m86p-0000Uw-00; Sat, 07 Apr 2001 23:51:35 -0600 Message-ID: <3ACFFC67.F99DF4E8@softweyr.com> Date: Sat, 07 Apr 2001 23:51:35 -0600 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.75 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Greg Lehey Cc: Terry Lambert , FreeBSD-arch@FreeBSD.ORG Subject: Re: Making ddb a kld References: <20010406150129.O66503@wantadilla.lemis.com> <200104062222.PAA05222@usr01.primenet.com> <20010407104709.B24010@wantadilla.lemis.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Greg Lehey wrote: > > On Friday, 6 April 2001 at 22:22:05 +0000, Terry Lambert wrote: > >> 1. This will require moving all the ddb code, which is scattered > >> around the system, into a separate file. I'm prepared to do the > >> work, but it's possible that somebody would complain about the > >> colour of the bikeshed. Here's your chance. > > > > How does it deal with the locore.s issues? > > Which locore.s issues? Crashes therein, before ddb.ko has been linked? It would make it difficult to debug the kernel linker, too. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message