From owner-freebsd-arch Sun Sep 10 11:24:26 2000 Delivered-To: freebsd-arch@freebsd.org Received: from alpha.dante.org.uk (alpha.dante.org.uk [193.63.211.19]) by hub.freebsd.org (Postfix) with ESMTP id 85AFD37B422; Sun, 10 Sep 2000 11:24:22 -0700 (PDT) Received: from localhost ([127.0.0.1] helo=dante.org.uk) by alpha.dante.org.uk with esmtp (Exim 3.12 #4) id 13YBko-0001nN-00; Sun, 10 Sep 2000 19:22:58 +0100 Message-ID: <39BBD183.D1399475@dante.org.uk> Date: Sun, 10 Sep 2000 19:22:59 +0100 From: Konstantin Chuguev Organization: Delivery of Advanced Network Technology to Europe Ltd. X-Mailer: Mozilla 4.75 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: ru, en MIME-Version: 1.0 To: "Andrey A. Chernov" , Boris Popov , freebsd-arch@FreeBSD.ORG, freebsd-i18n@FreeBSD.ORG Subject: Re: Proposal to include iconv library in the base system. References: <20000901185945.A29804@nagual.pp.ru> <39AFD666.880FE6C@dante.org.uk> Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Konstantin Chuguev wrote: > I will try my best to produce the final version 1.0 of the library and > conversion modules before Monday. > I have just submitted requests for upgrade of iconv-0.2 port to 1.0 and for creation of iconv-extra and iconv-rfc-1345 ports. Best regards, -- KC To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Sep 10 19:36: 4 2000 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id A2F0937B423 for ; Sun, 10 Sep 2000 19:36:02 -0700 (PDT) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id e8B2a2F02199 for arch@freebsd.org; Sun, 10 Sep 2000 19:36:02 -0700 (PDT) Date: Sun, 10 Sep 2000 19:36:02 -0700 From: Alfred Perlstein To: arch@freebsd.org Subject: Interruptable mutex aquires. Message-ID: <20000910193602.B12231@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG There's a lot of code in the kernel that does this: while(lock != mine) { error = tsleep(&lock, PRIBIO, "foo", PCATCH); if (error) return(error); } lock = mine; Ok, well the equivelant way to do this with mutexes would be to add a new function 'mtx_enter_interruptable()' [*] that attempts to aquire a mutex but if interrupted returns an error suitable for aborting the operation. Nice, easy, simple and fits well as a replacement tool for the current while(lock != mine) mechnism. Some people have discussed making a new flag 'MTX_CATCH', personally that sounds terrible, I'm already not really thrilled with the 'do-all' functions that have been put into place as our primatives, as they are not very intuative, it's hard to tell at a glance exactly what is a spinlock and what is a task switching mutex, let's _please_ try to keep this readable and stick away from the flags. I also think that we don't really need the overhead of a return value if at all possible. At the BSDi/FreeBSD SMP meeting I remeber asking Chuck why he implemented it this way, his respnonse was something along the lines of "it seemed like a good idea to have an object that one could spin or sleep on depending on the locatation or state of the object encapsulating the mutex, but in practice we've never used nor needed to use a mutex as a spinlock or a spinlock as a mutex" [*] yes, today's C compilers actually support identifiers longer than 6 characters! if mtx_enter_interruptable is too offensive I could live with mtx_enter_intr(). thanks, -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Sep 10 20:36:32 2000 Delivered-To: freebsd-arch@freebsd.org Received: from pcnet1.pcnet.com (pcnet1.pcnet.com [204.213.232.3]) by hub.freebsd.org (Postfix) with ESMTP id 5719B37B423 for ; Sun, 10 Sep 2000 20:36:30 -0700 (PDT) Received: (from eischen@localhost) by pcnet1.pcnet.com (8.8.7/PCNet) id XAA14835; Sun, 10 Sep 2000 23:36:12 -0400 (EDT) Date: Sun, 10 Sep 2000 23:36:11 -0400 (EDT) From: Daniel Eischen To: Alfred Perlstein Cc: arch@FreeBSD.ORG Subject: Re: Interruptable mutex aquires. In-Reply-To: <20000910193602.B12231@fw.wintelcom.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sun, 10 Sep 2000, Alfred Perlstein wrote: > > There's a lot of code in the kernel that does this: > > while(lock != mine) { > error = tsleep(&lock, PRIBIO, "foo", PCATCH); > if (error) > return(error); > } > lock = mine; > > Ok, well the equivelant way to do this with mutexes would be > to add a new function 'mtx_enter_interruptable()' [*] > that attempts to aquire a mutex but if interrupted returns > an error suitable for aborting the operation. I don't think we want to equate tsleep() with mtx_enter(). I'd rather see us use msleep_sig() until condition variables are added and then use cv_wait_sig() (or something like that). Mutexes should be taken for very short times, and we shouldn't worry about them being interrupted unless they surround a tsleep()/msleep(). > > Nice, easy, simple and fits well as a replacement tool for > the current while(lock != mine) mechnism. > > Some people have discussed making a new flag 'MTX_CATCH', personally > that sounds terrible, I'm already not really thrilled with the > 'do-all' functions that have been put into place as our primatives, > as they are not very intuative, it's hard to tell at a glance > exactly what is a spinlock and what is a task switching mutex, > let's _please_ try to keep this readable and stick away from the > flags. I sort of agree but can see why the flags were used instead of relying on mtx_init() to set the type of mutex. I wonder if we are too concerned about trying to squeeze an extra .5% performance for recursive mutexes, than with trying to create a simple and understandable interface. > > I also think that we don't really need the overhead of a > return value if at all possible. > > At the BSDi/FreeBSD SMP meeting I remeber asking Chuck why he > implemented it this way, his respnonse was something along the > lines of "it seemed like a good idea to have an object that one > could spin or sleep on depending on the locatation or state of the > object encapsulating the mutex, but in practice we've never used > nor needed to use a mutex as a spinlock or a spinlock as a mutex" > > [*] yes, today's C compilers actually support identifiers longer > than 6 characters! if mtx_enter_interruptable is too offensive > I could live with mtx_enter_intr(). I prefer *_sig() -- Dan Eischen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Sep 10 21:10:51 2000 Delivered-To: freebsd-arch@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 9B16937B422 for ; Sun, 10 Sep 2000 21:10:48 -0700 (PDT) Received: (qmail 28867 invoked by uid 1142); 11 Sep 2000 04:10:47 -0000 Date: 10 Sep 2000 21:10:47 -0700 Date: Sun, 10 Sep 2000 21:10:42 -0700 From: Jason Evans To: Alfred Perlstein Cc: arch@freebsd.org Subject: Re: Interruptable mutex aquires. Message-ID: <20000910211042.T19447@blitz.canonware.com> References: <20000910193602.B12231@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: <20000910193602.B12231@fw.wintelcom.net>; from bright@wintelcom.net on Sun, Sep 10, 2000 at 07:36:02PM -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sun, Sep 10, 2000 at 07:36:02PM -0700, Alfred Perlstein wrote: > > There's a lot of code in the kernel that does this: > > while(lock != mine) { > error = tsleep(&lock, PRIBIO, "foo", PCATCH); > if (error) > return(error); > } > lock = mine; I assume you're suggesting this become: if (mtx_enter_interruptable(&lock) == EINTR) return EINTR; /* We own the lock. */ Following are some other possibilities. -------------------- (1) Gross way: while (mtx_try_enter(lock, MTX_DEF)) ; /* We own the lock. */ -------------------- (2) Or, to come closer to the original example: while (mtx_try_enter(lock, MTX_DEF)) { if (someone_signaled_us) return EINTR; } /* We own the lock. */ -------------------- (3) Even more gross, but essentially the same as the original: while (mtx_try_enter(lock, MTX_DEF)) { error = tsleep(lock, ?, "foo", 0); if (error) return(error); } /* We own the lock. */ -------------------- (4) Re-design: mtx_enter(lock, MTX_DEF); while (some_state != WHAT_WE_WANT) { cv_wait(condition, lock); if (some_state == BAD_DEAL) { mtx_exit(lock); return(error); } } /* * We own lock. Drop it if it isn't protecting anything we need * to read or write. Otherwise, keep it as long as necessary. * * It may be that an entirely different lock actually protects the * data we need to access. If so, drop lock then acquire whatever * mutex we really need. */ .... mtx_exit(lock); ------- Somewhere else: some_state = WHAT_WE_WANT; cv_signal(condition); -------------------- The first three possibilities are basically equivalent to the original. (4) appears very different, but it is actually just an explicit form of your original. Your original acquires a lock, unless told to abort the acquisition (i.e. it's signaled). Example (4) explicitly implements a three state system: {NOT_READY, WHAT_WE_WANT, BAD_DEAL}. The code starts out in state NOT_READY, and progresses to WHAT_WE_WANT, unless something bad happens, in which case it's a BAD_DEAL. BAD_DEAL corresponds to being signaled. You may argue that this is way harder than using mtx_enter_interruptable(), and so it indeed appears. However, I would argue that the real problem lies with the original, in that it muddles two separate concepts, state and locking. Yes, we're currently stuck with code that does this. However, as seen above, there is more than one way of dealing with that (IMO complex) coding style without adding mtx_enter_interruptable(). Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Sep 10 22:50: 5 2000 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 4FA7637B423 for ; Sun, 10 Sep 2000 22:49:55 -0700 (PDT) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.8.7/8.8.7) with ESMTP id QAA13968; Mon, 11 Sep 2000 16:49:36 +1100 Date: Mon, 11 Sep 2000 16:49:35 +1100 (EST) From: Bruce Evans X-Sender: bde@besplex.bde.org To: Boris Popov Cc: freebsd-arch@FreeBSD.ORG Subject: Re: Shared kernel code In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, 1 Sep 2000, Boris Popov wrote: I haven't seen any public replies to this. > Currently I'm preparing smbfs bits for import into main tree. > Since both nwfs and smbfs contains common parts I've tried to unify them > in order to share code. Currently there are two common parts: kernel side > of iconv library and encapsulation of mbuf management functions. Both can > be statically compiled in the kernel or loaded as KLDs. > > But there is an interesting question arises - where to put source > code for those modules which don't represent any device and may consists > from multiple source files ? > > Historically such files were placed in the kern and libkern > directories. But it seems that both aren't suitable for newly added > interfaces because of the flat name space. So, my suggestion is to create Why not? I like flat name spaces :-), or at least flat directory trees. > sys/lib directory which should be organized as src/lib directory. Eg: > > sys/lib > libiconv/ > libmbuf/ > etc... > > Basically, sys/libkern also can be moved under lib directory. I think libkern as an actual library should be brought back. I made linkern object files normal kernel object files 5 years ago to avoid configuration and linkage complications, especially for modules. I think libraries can now be handled reasonably by the kernel linker. Use libkern.a (and possibly other libraries) in the usual way for static linkage. This would result in unreferenced objects not being linked into the kernel. Turn libkern.a into a module (libkern.ko) to satisfy references from modules that aren't satisfied by the kernel proper. Most modules would depend on libkern.ko. Use other modules containing special purpose objects to avoid bloating libkern.ko. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Sep 10 23:42: 4 2000 Delivered-To: freebsd-arch@freebsd.org Received: from relay.butya.kz (butya-gw.butya.kz [212.154.129.94]) by hub.freebsd.org (Postfix) with ESMTP id 68EA437B422 for ; Sun, 10 Sep 2000 23:41:58 -0700 (PDT) Received: by relay.butya.kz (Postfix, from userid 1000) id 1784D2876C; Mon, 11 Sep 2000 13:41:53 +0700 (ALMST) Received: from localhost (localhost [127.0.0.1]) by relay.butya.kz (Postfix) with ESMTP id 0312228766; Mon, 11 Sep 2000 13:41:52 +0700 (ALMST) Date: Mon, 11 Sep 2000 13:41:52 +0700 (ALMST) From: Boris Popov To: Bruce Evans Cc: freebsd-arch@FreeBSD.ORG Subject: Re: Shared kernel code In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Mon, 11 Sep 2000, Bruce Evans wrote: > > But there is an interesting question arises - where to put source > > code for those modules which don't represent any device and may consists > > from multiple source files ? > > > > Historically such files were placed in the kern and libkern > > directories. But it seems that both aren't suitable for newly added > > interfaces because of the flat name space. So, my suggestion is to create > > Why not? I like flat name spaces :-), or at least flat directory trees. Looks good for me if each set of files have a prefix. Eg, iconv_*, mbuf_* (sys/kern as example). > > Basically, sys/libkern also can be moved under lib directory. > > I think libkern as an actual library should be brought back. I made > linkern object files normal kernel object files 5 years ago to avoid > configuration and linkage complications, especially for modules. I > think libraries can now be handled reasonably by the kernel linker. > Use libkern.a (and possibly other libraries) in the usual way for > static linkage. This would result in unreferenced objects not being > linked into the kernel. Turn libkern.a into a module (libkern.ko) > to satisfy references from modules that aren't satisfied by the kernel > proper. Most modules would depend on libkern.ko. Use other modules > containing special purpose objects to avoid bloating libkern.ko. Agreed. I think ongoing Peter's work on KLDs will handle it very well. The only disadvantage is that libkern.ko will load all functions despite of the fact some of them already compiled statically. So, time for MODULE_VERSION(libkern, X) and MODULE_VERSION(kernel, X) :) -- Boris Popov http://www.butya.kz/~bp/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 8: 0:32 2000 Delivered-To: freebsd-arch@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id 34C2B37B423; Mon, 11 Sep 2000 08:00:27 -0700 (PDT) Received: from beppo.feral.com (beppo [192.67.166.79]) by feral.com (8.9.3/8.9.3) with ESMTP id IAA10432; Mon, 11 Sep 2000 08:00:27 -0700 Date: Mon, 11 Sep 2000 08:00:04 -0700 (PDT) From: Matthew Jacob Reply-To: mjacob@feral.com To: Poul-Henning Kamp Cc: arch@FreeBSD.ORG, smp@FreeBSD.ORG Subject: Re: Device probing... In-Reply-To: <89354.968526601@critter> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG The only theoretical problem with this is if you ever want to do a BOOT driver based autoload mechanism. Currently FreeBSD has the ability to load (preload) modules. If, in the future, you want to load modules after loading a primary kernel object but before configuring your root device's bus and driver stack, you need to do callbacks to a PROM driver. It's even harder to do this if you've enabled interrupts (or even changed MMU mappings, btw). So- if there's any chance you want to ever do a load as you go model, having interrupts enabled might make it more difficult. -matt p.s.: FWIW, I'm *for* you doing this. I *hate* having to support polled mode in drivers. On Sat, 9 Sep 2000, Poul-Henning Kamp wrote: > > I would really like to se us move all the device probe/attach code > to run in "normal context", ie enable interrupts before we probe > everything. > > The main argument for this is the drivers can then use the full > complement of kernel infrastructure and interrupts during probe > attach. > > Drivers needs to be able to function in this environment anyway > if they support removeable devices (pccard, usb, etc). > > Are there reasons to not do this I have overlooked ? > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD coreteam member | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-smp" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 11:15:46 2000 Delivered-To: freebsd-arch@freebsd.org Received: from berserker.bsdi.com (berserker.twistedbit.com [199.79.183.1]) by hub.freebsd.org (Postfix) with ESMTP id A99AA37B423 for ; Mon, 11 Sep 2000 11:15:43 -0700 (PDT) Received: from berserker.bsdi.com (cp@[127.0.0.1]) by berserker.bsdi.com (8.9.3/8.9.3) with ESMTP id MAA21525; Mon, 11 Sep 2000 12:15:35 -0600 (MDT) Message-Id: <200009111815.MAA21525@berserker.bsdi.com> To: Alfred Perlstein Cc: arch@freebsd.org Subject: Re: Interruptable mutex aquires. From: Chuck Paterson Date: Mon, 11 Sep 2000 12:15:35 -0600 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In the followng the lock which is acquired is often held for long periods, including while an IO is in progress, where as mutexs don't want to be held across an io. Chuck Alfred Perlstein wrote on: Sun, 10 Sep 2000 19:36:02 PDT } }There's a lot of code in the kernel that does this: } }while(lock != mine) { } error = tsleep(&lock, PRIBIO, "foo", PCATCH); } if (error) } return(error); }} }lock = mine; } }Ok, well the equivelant way to do this with mutexes would be }to add a new function 'mtx_enter_interruptable()' [*] }that attempts to aquire a mutex but if interrupted returns }an error suitable for aborting the operation. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 11:47:51 2000 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id A2C5A37B423 for ; Mon, 11 Sep 2000 11:47:49 -0700 (PDT) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id e8BIlkc25944; Mon, 11 Sep 2000 11:47:46 -0700 (PDT) Date: Mon, 11 Sep 2000 11:47:46 -0700 From: Alfred Perlstein To: Chuck Paterson Cc: arch@freebsd.org Subject: Re: Interruptable mutex aquires. Message-ID: <20000911114746.G12231@fw.wintelcom.net> References: <200009111815.MAA21525@berserker.bsdi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i In-Reply-To: <200009111815.MAA21525@berserker.bsdi.com>; from cp@bsdi.com on Mon, Sep 11, 2000 at 12:15:35PM -0600 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Chuck Paterson [000911 11:15] wrote: > Alfred Perlstein wrote on: Sun, 10 Sep 2000 19:36:02 PDT > } > }There's a lot of code in the kernel that does this: > } > }while(lock != mine) { > } error = tsleep(&lock, PRIBIO, "foo", PCATCH); > } if (error) > } return(error); > }} > }lock = mine; > } > }Ok, well the equivelant way to do this with mutexes would be > }to add a new function 'mtx_enter_interruptable()' [*] > }that attempts to aquire a mutex but if interrupted returns > }an error suitable for aborting the operation. > > In the followng the lock which is acquired is > often held for long periods, including while an IO is in > progress, where as mutexs don't want to be held across > an io. can you have a look here: http://people.freebsd.org/~alfred/mpsafe/mpsafestack.diff I'm wondering if what I'm doing with socketbuffers is 'right' basically the socketbuffer is still protected by a sleep+flags loop, however the flags are actually protected by a mutex on the socket itself and msleep is called after setting the flags to request a wakeup with PCATCH set. or from my journal at http://people.freebsd.org/~alfred/mpsafe/stackjournal.txt : Initially I wanted to place mutex locks in both the socket and socketbuffer structures, that proved to be too painful, instead use a lock on the socket and keep the old sleep/flags locking on the socketbuffer, there isn't a race because the socketbuffer flags are protected by my socket lock and the newly added msleep() function allows me to maninpulate the flags and sleep on them safely with my socket mutex interlocked. It's a bit confusing to me because mutexes can be used for long term mutual exclusion and one might still want to be able to grab a signal and return an error if things go to pot. thanks, -Alfred To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 12:37:32 2000 Delivered-To: freebsd-arch@freebsd.org Received: from mass.osd.bsdi.com (adsl-63-202-177-115.dsl.snfc21.pacbell.net [63.202.177.115]) by hub.freebsd.org (Postfix) with ESMTP id 5963937B423; Mon, 11 Sep 2000 12:37:27 -0700 (PDT) Received: from mass.osd.bsdi.com (localhost [127.0.0.1]) by mass.osd.bsdi.com (8.9.3/8.9.3) with ESMTP id MAA14055; Mon, 11 Sep 2000 12:36:41 -0700 (PDT) (envelope-from msmith@mass.osd.bsdi.com) Message-Id: <200009111936.MAA14055@mass.osd.bsdi.com> X-Mailer: exmh version 2.1.1 10/15/1999 To: mjacob@feral.com Cc: Poul-Henning Kamp , arch@FreeBSD.ORG, smp@FreeBSD.ORG Subject: Re: Device probing... In-reply-to: Your message of "Mon, 11 Sep 2000 08:00:04 PDT." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 11 Sep 2000 12:36:41 -0700 From: Mike Smith Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > The only theoretical problem with this is if you ever want to do a BOOT > driver based autoload mechanism. Currently FreeBSD has the ability to load > (preload) modules. If, in the future, you want to load modules after > loading a primary kernel object but before configuring your root device's > bus and driver stack, you need to do callbacks to a PROM driver. It's even > harder to do this if you've enabled interrupts (or even changed MMU > mappings, btw). Doing this on a PC is so problematic that the various PnP specifications that try to offload all of the resource management work onto the OS still talk about allocating resources for the "boot path". It's actually *extremely* difficult to do this, unless you have absolute knowledge of the mapping between "PROM device" and "kernel-space device". Consider the situation where you have initialised your disk driver, but want to use the "PROM driver" interface still to fetch another driver. Because (in the PC environment) you simply can't tell that the disk driver you've already initialised has taken over the hardware that the "PROM driver" path would use, you're stuck. In reality, this is less of an issue simply because most probing we do these days is entirely metainformation-based. How many of our probe routines actually do more than passive metainformation comparisons? > p.s.: FWIW, I'm *for* you doing this. I *hate* having to support polled mode > in drivers. I'm inclined to agree with this here. > On Sat, 9 Sep 2000, Poul-Henning Kamp wrote: > > > I would really like to se us move all the device probe/attach code > > to run in "normal context", ie enable interrupts before we probe > > everything. Most of the reasons for not doing this have gone away. We do need to re-order a few things though. Note that this will make it "interesting" if we want to support the interrupt hardware with "normal" drivers. > > The main argument for this is the drivers can then use the full > > complement of kernel infrastructure and interrupts during probe > > attach. > > > > Drivers needs to be able to function in this environment anyway > > if they support removeable devices (pccard, usb, etc). > > > > Are there reasons to not do this I have overlooked ? The biggest one that's left is that a lot of drivers will end up printing things in interrupt context, leading to *incredibly* messy console output. We'll need a cleaner way of presenting driver messages. -- ... every activity meets with opposition, everyone who acts has his rivals and unfortunately opponents also. But not because people want to be opponents, rather because the tasks and relationships force people to take different points of view. [Dr. Fritz Todt] To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 16:48: 0 2000 Delivered-To: freebsd-arch@freebsd.org Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.80]) by hub.freebsd.org (Postfix) with ESMTP id 80B7037B422; Mon, 11 Sep 2000 16:47:25 -0700 (PDT) Received: (from grog@localhost) by wantadilla.lemis.com (8.11.0/8.9.3) id e8BNl5281326; Tue, 12 Sep 2000 09:17:05 +0930 (CST) (envelope-from grog) Date: Tue, 12 Sep 2000 09:17:05 +0930 From: Greg Lehey To: John Baldwin Cc: Mark Murray , FreeBSD-arch@FreeBSD.org Subject: Re: cvs commit: src/sys/conf files src/sys/sys random.h src/sys/dev/randomdev hash.c hash.h harvest.c randomdev.c yarrow.c yarro Message-ID: <20000912091705.O19431@wantadilla.lemis.com> References: <20000911093457.H15703@wantadilla.lemis.com> <200009110111.SAA29487@pike.osd.bsdi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <200009110111.SAA29487@pike.osd.bsdi.com>; from jhb@pike.osd.bsdi.com on Sun, Sep 10, 2000 at 06:11:25PM -0700 Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG [following up to -arch] On Sunday, 10 September 2000 at 18:11:25 -0700, John Baldwin wrote: > Greg Lehey wrote: >> On Sunday, 10 September 2000 at 6:52:19 -0700, Mark Murray wrote: >>> markm 2000/09/10 06:52:19 PDT >>> >>> Log: >>> Large upgrade to the entropy device; mainly inspired by feedback >>> from many folk. >>> >>> o The reseed process is now a kthread. With SMPng, kthreads are >>> pre-emptive, so the annoying jerkiness of the mouse is gone. >>> >>> o The data structures are protected by mutexes now, not splfoo()/splx(). >> >> The last thing I heard was that we were getting worried about putting >> in too many mutexes. How was this resolved? > > IIRC, Mark's code doesn't tsleep() with a mutex, which was one of > the problems that the malloc() mutex patch had. Although, in my > opinion, we'd probably be better off starting with large subsystem > locks as the first step of the lock pushdown, and then successively > push down the locks within each subsystem and sub-subsystem. I > think trying to add a bunch of small locks in the beginning will > just give us massive amounts of headaches. That's exactly my point, though I'd be inclined to replace "headache" with something stronger. We really need to come up with a strategy for adding mutexes, or we're going to be in trouble. Take a look at BSD/OS's sys/proc.h for an example of how complicated things can get: /* * Description of a process. * * This structure contains the information needed to manage a thread of * control, known in UN*X as a process; it has references to substructures * containing descriptions of things that the process uses, but may share * with related processes. The process structure and the substructures * are always addressible except for those marked "(PROC ONLY)" below, * which might be addressible only on a processor on which the process * is running. * * * - not yet protected * a - only touched by curproc or parent during fork/wait * b - created at fork, never chagnes * c - locked by proc mtx * d - locked by allproc lock * e - locked by proc tree lock * f - session mtx * g - process group mtx * h - pid hash table mtx * i - by curproc or the master session mtx * j - locked by sched mtx * k - either by curproc or a lock which prevents the lock from * going way, such a (d,e). * l - the attaching proc or attaching proc parent. * m - Giant * n - not locked, lazy * o - time lock */ I've been wondering whether we shouldn't associate mutexes with data structures rather than code. It's possible that it would make it easier to avoid deadlocks. Thoughts? Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 16:57:51 2000 Delivered-To: freebsd-arch@freebsd.org Received: from mail-relay.eunet.no (mail-relay.eunet.no [193.71.71.242]) by hub.freebsd.org (Postfix) with ESMTP id B96F237B42C; Mon, 11 Sep 2000 16:57:47 -0700 (PDT) Received: from login-1.eunet.no (login-1.eunet.no [193.75.110.2]) by mail-relay.eunet.no (8.9.3/8.9.3/GN) with ESMTP id BAA89671; Tue, 12 Sep 2000 01:57:45 +0200 (CEST) (envelope-from mbendiks@eunet.no) Received: from localhost (mbendiks@localhost) by login-1.eunet.no (8.9.3/8.8.8) with ESMTP id BAA78528; Tue, 12 Sep 2000 01:57:45 +0200 (CEST) (envelope-from mbendiks@eunet.no) X-Authentication-Warning: login-1.eunet.no: mbendiks owned process doing -bs Date: Tue, 12 Sep 2000 01:57:45 +0200 (CEST) From: Marius Bendiksen To: Mike Smith Cc: mjacob@feral.com, Poul-Henning Kamp , arch@FreeBSD.ORG, smp@FreeBSD.ORG Subject: Re: Device probing... In-Reply-To: <200009111936.MAA14055@mass.osd.bsdi.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > The biggest one that's left is that a lot of drivers will end up printing > things in interrupt context, leading to *incredibly* messy console > output. We'll need a cleaner way of presenting driver messages. Couldn't this be resolved by adding a console buffer which you could spool lines to, rather than printing them directly? The buffer would then sort a certain number of lines at a time, and print them grouped by driver. Marius To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 16:59:21 2000 Delivered-To: freebsd-arch@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id 9C9BB37B422; Mon, 11 Sep 2000 16:59:17 -0700 (PDT) Received: from zeppo.feral.com (IDENT:mjacob@zeppo [192.67.166.71]) by feral.com (8.9.3/8.9.3) with ESMTP id QAA12781; Mon, 11 Sep 2000 16:59:12 -0700 Date: Mon, 11 Sep 2000 16:55:21 -0700 (PDT) From: Matthew Jacob Reply-To: mjacob@feral.com To: Marius Bendiksen Cc: Mike Smith , Poul-Henning Kamp , arch@FreeBSD.ORG, smp@FreeBSD.ORG Subject: Re: Device probing... In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Good idea, except it's hard to debug unless you have a working ddb or gdb or something (which is very rare these days for the machines I usually work on). > > The biggest one that's left is that a lot of drivers will end up printing > > things in interrupt context, leading to *incredibly* messy console > > output. We'll need a cleaner way of presenting driver messages. > > Couldn't this be resolved by adding a console buffer which you could spool > lines to, rather than printing them directly? The buffer would then sort a > certain number of lines at a time, and print them grouped by driver. > > > Marius > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 17:13:22 2000 Delivered-To: freebsd-arch@freebsd.org Received: from smtp04.primenet.com (smtp04.primenet.com [206.165.6.134]) by hub.freebsd.org (Postfix) with ESMTP id A072937B423; Mon, 11 Sep 2000 17:13:19 -0700 (PDT) Received: (from daemon@localhost) by smtp04.primenet.com (8.9.3/8.9.3) id RAA08638; Mon, 11 Sep 2000 17:11:04 -0700 (MST) Received: from usr09.primenet.com(206.165.6.209) via SMTP by smtp04.primenet.com, id smtpdAAAOeaqGq; Mon Sep 11 17:10:47 2000 Received: (from tlambert@localhost) by usr09.primenet.com (8.8.5/8.8.5) id RAA25062; Mon, 11 Sep 2000 17:12:52 -0700 (MST) From: Terry Lambert Message-Id: <200009120012.RAA25062@usr09.primenet.com> Subject: Re: cvs commit: src/sys/conf files src/sys/sys random.h src/sys/dev/randomdev hash.c hash.h harvest.c randomdev.c yarrow.c yarro To: grog@lemis.com (Greg Lehey) Date: Tue, 12 Sep 2000 00:12:52 +0000 (GMT) Cc: jhb@pike.osd.bsdi.com (John Baldwin), markm@FreeBSD.ORG (Mark Murray), FreeBSD-arch@FreeBSD.ORG In-Reply-To: <20000912091705.O19431@wantadilla.lemis.com> from "Greg Lehey" at Sep 12, 2000 09:17:05 AM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > [following up to -arch] [ ... ] > I've been wondering whether we shouldn't associate mutexes with data > structures rather than code. It's possible that it would make it > easier to avoid deadlocks. Thoughts? I think that deadlock detection and back-off beats deadlock avoidance, particularly if you are trying to debug something hairy. The ability to take an EWOULDBLOCK response and decide to print out the cycle that would result in the deadlock, not only for the caller trying to assert a new lock/mutex, but for the other player(s) in the deadlock situation, is invaluable. To achieve this, you really want to associate all locks into an overall hierarchical relationship. This lets you keep metadata on the locks; if you are familiar with Sedgewick, what you will be doing is keeping a running tally of the transitive closure of arcs (one per locking thread) over a directed acyclic graph. When you want to assert a new lock, you imply an edge from the lock location (as a leaf node) to the root. Then you can do a linear traversal to the root; this will immediately point out any cycles which would be caused by the new lock, should the assertion go through. This is normally a small number of compares, equal to the depth of your tree at the location where the new lock would be, plus one. Effectively, this is an O(1) method of finding Hamiltonian cycles in your proposed lock list. Creation and deletion of lock objects in the graph is a rare event, and so can be permitted to be relatively expensive (though it's on the order of 20,000 transactions a second on a P60, per work done at Novell in 1994). SIX mode locking can further increase concurrency. It's easy to see that you could have, for example, a process table that you wanted to be able to simultaneously access. You could have a top level "entrire process table" lock, and then divide the table into 6 chunks with inferior locks hung off the top level lock. This would let you traverse 5/6ths of the table without contention, with one writer holding a lock for creation of a new table entry. You could take this to a per slot locking granularity, if you felt it were necessary for the task at hand. The main point is that, the closer you get to the root, the less time you have to hold the lock to get an inferior lock, since the inferior (leaf) lock is what's protecting your data -- non-leaf locks only protect access to their inferior lists. Dynix used a similar approach for some of its highly contended kernel structures and resource pools, though it only went one deep in most places. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 17:18:29 2000 Delivered-To: freebsd-arch@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 5B3CE37B424 for ; Mon, 11 Sep 2000 17:18:27 -0700 (PDT) Received: (qmail 69229 invoked by uid 1142); 12 Sep 2000 00:18:26 -0000 Date: 11 Sep 2000 17:18:26 -0700 Date: Mon, 11 Sep 2000 17:18:19 -0700 From: Jason Evans To: Greg Lehey Cc: John Baldwin , Mark Murray , FreeBSD-arch@FreeBSD.org Subject: Re: cvs commit: src/sys/conf files src/sys/sys random.h src/sys/dev/randomdev hash.c hash.h harvest.c randomdev.c yarrow.c yarro Message-ID: <20000911171819.A31089@blitz.canonware.com> References: <20000911093457.H15703@wantadilla.lemis.com> <200009110111.SAA29487@pike.osd.bsdi.com> <20000912091705.O19431@wantadilla.lemis.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: <20000912091705.O19431@wantadilla.lemis.com>; from grog@lemis.com on Tue, Sep 12, 2000 at 09:17:05AM +0930 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, Sep 12, 2000 at 09:17:05AM +0930, Greg Lehey wrote: > [following up to -arch] > On Sunday, 10 September 2000 at 18:11:25 -0700, John Baldwin wrote: > > Greg Lehey wrote: > >> On Sunday, 10 September 2000 at 6:52:19 -0700, Mark Murray wrote: > >>> markm 2000/09/10 06:52:19 PDT > >>> > >>> Log: > >>> Large upgrade to the entropy device; mainly inspired by feedback > >>> from many folk. > >>> > >>> o The reseed process is now a kthread. With SMPng, kthreads are > >>> pre-emptive, so the annoying jerkiness of the mouse is gone. > >>> > >>> o The data structures are protected by mutexes now, not splfoo()/splx(). > >> > >> The last thing I heard was that we were getting worried about putting > >> in too many mutexes. How was this resolved? > > > > IIRC, Mark's code doesn't tsleep() with a mutex, which was one of > > the problems that the malloc() mutex patch had. Although, in my > > opinion, we'd probably be better off starting with large subsystem > > locks as the first step of the lock pushdown, and then successively > > push down the locks within each subsystem and sub-subsystem. I > > think trying to add a bunch of small locks in the beginning will > > just give us massive amounts of headaches. Mark's changes to randomdev are very straight-forward, and they use a mutex exactly as a mutex should be used (though a condition variable will help make things even cleaner, once they're available). A general discussion of high level locking strategy is in order, but in my opinion, Mark's code is not an appropriate harbinger, since there's absolutely nothing wrong with the approach he took, regardless of the results of high level discussion. The issue we need to decide on is how to avoid bad interactions (read deadlock/livelock) between mutex hierarchies, but the randomdev mutex basically stands alone, so isn't subject to those discussions. Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 18: 1:43 2000 Delivered-To: freebsd-arch@freebsd.org Received: from tinker.exit.com (exit-gw.power.net [207.151.46.196]) by hub.freebsd.org (Postfix) with ESMTP id 440F837B424; Mon, 11 Sep 2000 18:01:41 -0700 (PDT) Received: from realtime.exit.com (realtime [206.223.0.5]) by tinker.exit.com (8.11.0/8.11.0) with ESMTP id e8C11Va22409; Mon, 11 Sep 2000 18:01:31 -0700 (PDT) (envelope-from frank@exit.com) Received: (from frank@localhost) by realtime.exit.com (8.11.0/8.11.0) id e8C11nN56928; Mon, 11 Sep 2000 18:01:49 -0700 (PDT) (envelope-from frank) From: Frank Mayhar Message-Id: <200009120101.e8C11nN56928@realtime.exit.com> Subject: Re: cvs commit: src/sys/conf files src/sys/sys random.h src/sys/dev/randomdev hash.c hash.h harvest.c randomdev.c yarrow.c yarro In-Reply-To: <20000912091705.O19431@wantadilla.lemis.com> from Greg Lehey at "Sep 12, 2000 09:17:05 am" To: Greg Lehey Date: Mon, 11 Sep 2000 18:01:49 -0700 (PDT) Cc: John Baldwin , Mark Murray , FreeBSD-arch@FreeBSD.ORG Reply-To: frank@exit.com Organization: Exit Consulting X-Copyright0: Copyright 2000 Frank Mayhar. All Rights Reserved. X-Copyright1: Permission granted for electronic reproduction as Usenet News or email only. X-Mailer: ELM [version 2.4ME+ PL68 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Greg Lehey wrote: > I've been wondering whether we shouldn't associate mutexes with data > structures rather than code. It's possible that it would make it > easier to avoid deadlocks. Thoughts? Speaking as a BSD/OS (and former Unixware) developer: YES! :-) -- Frank Mayhar frank@exit.com http://www.exit.com/ Exit Consulting http://store.exit.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 18: 6:34 2000 Delivered-To: freebsd-arch@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id A269F37B42C; Mon, 11 Sep 2000 18:06:32 -0700 (PDT) Received: from zeppo.feral.com (IDENT:mjacob@zeppo [192.67.166.71]) by feral.com (8.9.3/8.9.3) with ESMTP id SAA13068; Mon, 11 Sep 2000 18:06:18 -0700 Date: Mon, 11 Sep 2000 18:02:26 -0700 (PDT) From: Matthew Jacob Reply-To: mjacob@feral.com To: Frank Mayhar Cc: Greg Lehey , John Baldwin , Mark Murray , FreeBSD-arch@FreeBSD.ORG Subject: Re: cvs commit: src/sys/conf files src/sys/sys random.h src/sys/dev/randomdev hash.c hash.h harvest.c randomdev.c yarrow.c yarro In-Reply-To: <200009120101.e8C11nN56928@realtime.exit.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Greg Lehey wrote: > > I've been wondering whether we shouldn't associate mutexes with data > > structures rather than code. It's possible that it would make it > > easier to avoid deadlocks. Thoughts? > > Speaking as a BSD/OS (and former Unixware) developer: YES! Hmm. I would rather have assumed that this is what mutexes are about. Semaphores gate entry in code. Mutexes provide locking on data. Simple enough. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 18:22:24 2000 Delivered-To: freebsd-arch@freebsd.org Received: from tinker.exit.com (exit-gw.power.net [207.151.46.196]) by hub.freebsd.org (Postfix) with ESMTP id 562D637B424 for ; Mon, 11 Sep 2000 18:22:22 -0700 (PDT) Received: from realtime.exit.com (realtime [206.223.0.5]) by tinker.exit.com (8.11.0/8.11.0) with ESMTP id e8C1MLa22490 for ; Mon, 11 Sep 2000 18:22:21 -0700 (PDT) (envelope-from frank@exit.com) Received: (from frank@localhost) by realtime.exit.com (8.11.0/8.11.0) id e8C1MdI57055 for FreeBSD-arch@FreeBSD.ORG; Mon, 11 Sep 2000 18:22:39 -0700 (PDT) (envelope-from frank) From: Frank Mayhar Message-Id: <200009120122.e8C1MdI57055@realtime.exit.com> Subject: Re: cvs commit: src/sys/conf files src/sys/sys random.h src/sys/dev/randomdev hash.c hash.h harvest.c randomdev.c yarrow.c yarro In-Reply-To: from Matthew Jacob at "Sep 11, 2000 06:02:26 pm" To: mjacob@feral.com Date: Mon, 11 Sep 2000 18:17:24 -0700 (PDT) Cc: Frank Mayhar , Greg Lehey , John Baldwin , Mark Murray , FreeBSD-arch@FreeBSD.ORGG Reply-To: frank@exit.com Organization: Exit Consulting X-Copyright0: Copyright 2000 Frank Mayhar. All Rights Reserved. X-Copyright1: Permission granted for electronic reproduction as Usenet News or email only. X-Mailer: ELM [version 2.4ME+ PL68 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Matthew Jacob wrote: > Frank Mayahr wrote: > > Greg Lehey wrote: > > > I've been wondering whether we shouldn't associate mutexes with data > > > structures rather than code. It's possible that it would make it > > > easier to avoid deadlocks. Thoughts? > > Speaking as a BSD/OS (and former Unixware) developer: YES! > Hmm. I would rather have assumed that this is what mutexes are about. > Semaphores gate entry in code. Mutexes provide locking on data. Simple enough. This is my take on it exactly and is how I've always approached adding locking. It's a lot simpler to do per-data structure locking than code- path locking, IMHO, as you can see all the places the data structure is used (whereas tracking down all the code paths can be a real nightmare) and one can avoid using locks where they're not needed. The latter is at least as important as putting locking in in the first place, since unnecessary locking can eat your performance alive. -- Frank Mayhar frank@exit.com http://www.exit.com/ Exit Consulting http://store.exit.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 18:48:59 2000 Delivered-To: freebsd-arch@freebsd.org Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.80]) by hub.freebsd.org (Postfix) with ESMTP id 5479037B424 for ; Mon, 11 Sep 2000 18:48:53 -0700 (PDT) Received: (from grog@localhost) by wantadilla.lemis.com (8.11.0/8.9.3) id e8C1mgH96238; Tue, 12 Sep 2000 11:18:42 +0930 (CST) (envelope-from grog) Date: Tue, 12 Sep 2000 11:18:42 +0930 From: Greg Lehey To: Alfred Perlstein Cc: arch@FreeBSD.ORG Subject: Re: Interruptable mutex aquires. Message-ID: <20000912111842.F88615@wantadilla.lemis.com> References: <20000910193602.B12231@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <20000910193602.B12231@fw.wintelcom.net>; from bright@wintelcom.net on Sun, Sep 10, 2000 at 07:36:02PM -0700 Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sunday, 10 September 2000 at 19:36:02 -0700, Alfred Perlstein wrote: > > There's a lot of code in the kernel that does this: > > while(lock != mine) { > error = tsleep(&lock, PRIBIO, "foo", PCATCH); > if (error) > return(error); > } > lock = mine; > > Ok, well the equivelant way to do this with mutexes would be > to add a new function 'mtx_enter_interruptable()' [*] > that attempts to aquire a mutex but if interrupted returns > an error suitable for aborting the operation. > > Nice, easy, simple and fits well as a replacement tool for > the current while(lock != mine) mechnism. > > Some people have discussed making a new flag 'MTX_CATCH', personally > that sounds terrible, I'm already not really thrilled with the > 'do-all' functions that have been put into place as our primatives, > as they are not very intuative, it's hard to tell at a glance > exactly what is a spinlock and what is a task switching mutex, let's > _please_ try to keep this readable and stick away from the flags. I suppose there's a case for simpler terminology, but have you looked at the mutex code? There's a *lot* of code in there, and apart from the duplication, maintenance would be terrible. I'd suggest the flag approach, but there's nothing stopping a macro encapsulation with a name like mtx_enter_interruptable. > At the BSDi/FreeBSD SMP meeting I remeber asking Chuck why he > implemented it this way, his respnonse was something along the lines > of "it seemed like a good idea to have an object that one could spin > or sleep on depending on the locatation or state of the object > encapsulating the mutex, but in practice we've never used nor needed > to use a mutex as a spinlock or a spinlock as a mutex" Ugh. Another use of the word "mutex". I thought we were using mutex to cover both terms. Don't forget that BSD/OS 5 hasn't been released yet. Rather amusingly, ours is available before theirs :-) But it's quite possible that they will find a use for this kind of mutex. > [*] yes, today's C compilers actually support identifiers longer > than 6 characters! if mtx_enter_interruptable is too offensive > I could live with mtx_enter_intr(). This isn't a question of identifier length, it's a question of style(9). As long as we have 8 character indentation and only 80 characters text width, we're damned to using short identifiers. Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 18:59: 3 2000 Delivered-To: freebsd-arch@freebsd.org Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.80]) by hub.freebsd.org (Postfix) with ESMTP id 2D95237B422 for ; Mon, 11 Sep 2000 18:59:00 -0700 (PDT) Received: (from grog@localhost) by wantadilla.lemis.com (8.11.0/8.9.3) id e8C1wAw97393; Tue, 12 Sep 2000 11:28:10 +0930 (CST) (envelope-from grog) Date: Tue, 12 Sep 2000 11:28:10 +0930 From: Greg Lehey To: Chuck Paterson Cc: Alfred Perlstein , arch@FreeBSD.ORG Subject: Re: Interruptable mutex aquires. Message-ID: <20000912112810.G88615@wantadilla.lemis.com> References: <200009111815.MAA21525@berserker.bsdi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <200009111815.MAA21525@berserker.bsdi.com>; from cp@bsdi.com on Mon, Sep 11, 2000 at 12:15:35PM -0600 Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Monday, 11 September 2000 at 12:15:35 -0600, Chuck Paterson wrote: > Alfred Perlstein wrote on: Sun, 10 Sep 2000 19:36:02 PDT >> >> There's a lot of code in the kernel that does this: >> >> while(lock != mine) { >> error = tsleep(&lock, PRIBIO, "foo", PCATCH); >> if (error) >> return(error); >>> >> lock = mine; >> >> Ok, well the equivelant way to do this with mutexes would be >> to add a new function 'mtx_enter_interruptable()' [*] >> that attempts to aquire a mutex but if interrupted returns >> an error suitable for aborting the operation. > > In the followng the lock which is acquired is often held for > long periods, including while an IO is in progress, where as mutexs > don't want to be held across an io. We discussed this in June, but I still don't understand what there is in the nature of mutexes which makes it undesirable to hold them for long times. This seems to be more of a usage thing: the mutexes we (and BSD/OS) currently have want to be released quickly. Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 19: 0:37 2000 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 3E23737B423 for ; Mon, 11 Sep 2000 19:00:35 -0700 (PDT) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id e8C20To08937; Mon, 11 Sep 2000 19:00:29 -0700 (PDT) Date: Mon, 11 Sep 2000 19:00:29 -0700 From: Alfred Perlstein To: Greg Lehey Cc: arch@FreeBSD.ORG Subject: Re: Interruptable mutex aquires. Message-ID: <20000911190029.B12231@fw.wintelcom.net> References: <20000910193602.B12231@fw.wintelcom.net> <20000912111842.F88615@wantadilla.lemis.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i In-Reply-To: <20000912111842.F88615@wantadilla.lemis.com>; from grog@lemis.com on Tue, Sep 12, 2000 at 11:18:42AM +0930 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Greg Lehey [000911 18:48] wrote: > On Sunday, 10 September 2000 at 19:36:02 -0700, Alfred Perlstein wrote: > > > > Some people have discussed making a new flag 'MTX_CATCH', personally > > that sounds terrible, I'm already not really thrilled with the > > 'do-all' functions that have been put into place as our primatives, > > as they are not very intuative, it's hard to tell at a glance > > exactly what is a spinlock and what is a task switching mutex, let's > > _please_ try to keep this readable and stick away from the flags. > > I suppose there's a case for simpler terminology, but have you looked > at the mutex code? There's a *lot* of code in there, and apart from > the duplication, maintenance would be terrible. I'd suggest the flag > approach, but there's nothing stopping a macro encapsulation with a > name like mtx_enter_interruptable. That could work. > > At the BSDi/FreeBSD SMP meeting I remeber asking Chuck why he > > implemented it this way, his respnonse was something along the lines > > of "it seemed like a good idea to have an object that one could spin > > or sleep on depending on the locatation or state of the object > > encapsulating the mutex, but in practice we've never used nor needed > > to use a mutex as a spinlock or a spinlock as a mutex" > > Ugh. Another use of the word "mutex". I thought we were using mutex > to cover both terms. > > Don't forget that BSD/OS 5 hasn't been released yet. Rather > amusingly, ours is available before theirs :-) But it's quite possible > that they will find a use for this kind of mutex. I find it annoying to have to wrap what should be a simple interruptable mutex with a very complex code, nothing that Jason presented looked easy to parse. Honestly though, since my initial request for this feature and subsequent workaround that _was_ needed (mtx_enter_intr was not an option actually) I haven't really come across anything that really needs it. > > [*] yes, today's C compilers actually support identifiers longer > > than 6 characters! if mtx_enter_interruptable is too offensive > > I could live with mtx_enter_intr(). > > This isn't a question of identifier length, it's a question of > style(9). As long as we have 8 character indentation and only 80 > characters text width, we're damned to using short identifiers. The Linux core kernel seems able to produce more readable code than ours constantly. In fact our in house coding practices strictly follow style 9 with the exception of almost always using {} for surrounding control blocks, we usually don't have any problems with descriptive identifiers. Then again we set our tab stops to 4 characters, as I do with any kernel code that I introduce. I think the golden rule is: If it takes you more than 5 minutes to think of a cute and 'descriptive' way to abbreviate something, then it's probably not descriptive at all. :) -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 19:12: 4 2000 Delivered-To: freebsd-arch@freebsd.org Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.80]) by hub.freebsd.org (Postfix) with ESMTP id E599137B422 for ; Mon, 11 Sep 2000 19:12:00 -0700 (PDT) Received: (from grog@localhost) by wantadilla.lemis.com (8.11.0/8.9.3) id e8C2Bsq99036; Tue, 12 Sep 2000 11:41:54 +0930 (CST) (envelope-from grog) Date: Tue, 12 Sep 2000 11:41:54 +0930 From: Greg Lehey To: Alfred Perlstein Cc: arch@FreeBSD.ORG Subject: Re: Interruptable mutex aquires. Message-ID: <20000912114154.H88615@wantadilla.lemis.com> References: <200009111815.MAA21525@berserker.bsdi.com> <20000911114746.G12231@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <20000911114746.G12231@fw.wintelcom.net>; from bright@wintelcom.net on Mon, Sep 11, 2000 at 11:47:46AM -0700 Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Monday, 11 September 2000 at 11:47:46 -0700, Alfred Perlstein wrote: > can you have a look here: > http://people.freebsd.org/~alfred/mpsafe/mpsafestack.diff > > I'm wondering if what I'm doing with socketbuffers is 'right' > basically the socketbuffer is still protected by a sleep+flags > loop, however the flags are actually protected by a mutex > on the socket itself and msleep is called after setting the > flags to request a wakeup with PCATCH set. > > or from my journal at > http://people.freebsd.org/~alfred/mpsafe/stackjournal.txt : > > Initially I wanted to place mutex locks in both the socket and > socketbuffer structures, that proved to be too painful, instead > use a lock on the socket and keep the old sleep/flags locking on > the socketbuffer, there isn't a race because the socketbuffer > flags are protected by my socket lock and the newly added msleep() > function allows me to maninpulate the flags and sleep on them > safely with my socket mutex interlocked. > > It's a bit confusing to me because mutexes can be used for long > term mutual exclusion and one might still want to be able to > grab a signal and return an error if things go to pot. I think we need to come to some kind of consensus about how we are going to structure locking before we go into this much detail. At the moment we don't even agree whether we can hold on to (blocking) mutexes for long periods of time. Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 19:24:33 2000 Delivered-To: freebsd-arch@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 03E5E37B423 for ; Mon, 11 Sep 2000 19:24:32 -0700 (PDT) Received: (qmail 72922 invoked by uid 1142); 12 Sep 2000 02:24:31 -0000 Date: 11 Sep 2000 19:24:31 -0700 Date: Mon, 11 Sep 2000 19:24:25 -0700 From: Jason Evans To: Greg Lehey Cc: arch@FreeBSD.ORG Subject: Long-term mutex ownership (was Re: Interruptable mutex aquires.) Message-ID: <20000911192425.B31089@blitz.canonware.com> References: <200009111815.MAA21525@berserker.bsdi.com> <20000911114746.G12231@fw.wintelcom.net> <20000912114154.H88615@wantadilla.lemis.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: <20000912114154.H88615@wantadilla.lemis.com>; from grog@lemis.com on Tue, Sep 12, 2000 at 11:41:54AM +0930 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, Sep 12, 2000 at 11:41:54AM +0930, Greg Lehey wrote: > I think we need to come to some kind of consensus about how we are > going to structure locking before we go into this much detail. At the > moment we don't even agree whether we can hold on to (blocking) > mutexes for long periods of time. I don't recall the original argument against holding mutexes for long periods. From an abstract point of view, there's nothing wrong with such practice, and in fact it makes sense for many problems. Is there an issue with our implementation? If so, can someone please explain it? Thanks, Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 19:26:19 2000 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id AFA6B37B423 for ; Mon, 11 Sep 2000 19:26:16 -0700 (PDT) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id e8C2QAa09517; Mon, 11 Sep 2000 19:26:10 -0700 (PDT) Date: Mon, 11 Sep 2000 19:26:10 -0700 From: Alfred Perlstein To: Jason Evans Cc: Greg Lehey , arch@FreeBSD.ORG Subject: Re: Long-term mutex ownership (was Re: Interruptable mutex aquires.) Message-ID: <20000911192610.D12231@fw.wintelcom.net> References: <200009111815.MAA21525@berserker.bsdi.com> <20000911114746.G12231@fw.wintelcom.net> <20000912114154.H88615@wantadilla.lemis.com> <20000911192425.B31089@blitz.canonware.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i In-Reply-To: <20000911192425.B31089@blitz.canonware.com>; from jasone@canonware.com on Mon, Sep 11, 2000 at 07:24:25PM -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Jason Evans [000911 19:24] wrote: > On Tue, Sep 12, 2000 at 11:41:54AM +0930, Greg Lehey wrote: > > I think we need to come to some kind of consensus about how we are > > going to structure locking before we go into this much detail. At the > > moment we don't even agree whether we can hold on to (blocking) > > mutexes for long periods of time. > > I don't recall the original argument against holding mutexes for long > periods. From an abstract point of view, there's nothing wrong with such > practice, and in fact it makes sense for many problems. Is there an issue > with our implementation? If so, can someone please explain it? You are currently unable to abort if a signal comes in. :) Conditional variables add overhead and complexity where it's not needed. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 19:38:45 2000 Delivered-To: freebsd-arch@freebsd.org Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.80]) by hub.freebsd.org (Postfix) with ESMTP id F29AA37B423; Mon, 11 Sep 2000 19:38:39 -0700 (PDT) Received: (from grog@localhost) by wantadilla.lemis.com (8.11.0/8.9.3) id e8C2cFI02425; Tue, 12 Sep 2000 12:08:15 +0930 (CST) (envelope-from grog) Date: Tue, 12 Sep 2000 12:08:15 +0930 From: Greg Lehey To: Terry Lambert Cc: John Baldwin , Mark Murray , FreeBSD-arch@FreeBSD.ORG Subject: SMP lock ordering (was: cvs commit: src/sys/conf files src/sys/sys random.h src/sys/dev/randomdev hash.c hash.h harvest.c randomdev.c yarrow.c yarro) Message-ID: <20000912120815.I88615@wantadilla.lemis.com> References: <20000912091705.O19431@wantadilla.lemis.com> <200009120012.RAA25062@usr09.primenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <200009120012.RAA25062@usr09.primenet.com>; from tlambert@primenet.com on Tue, Sep 12, 2000 at 12:12:52AM +0000 Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tuesday, 12 September 2000 at 0:12:52 +0000, Terry Lambert wrote: >> [following up to -arch] > [ ... ] >> I've been wondering whether we shouldn't associate mutexes with data >> structures rather than code. It's possible that it would make it >> easier to avoid deadlocks. Thoughts? > > I think that deadlock detection and back-off beats deadlock > avoidance, particularly if you are trying to debug something hairy. > The ability to take an EWOULDBLOCK response and decide to print out > the cycle that would result in the deadlock, not only for the caller > trying to assert a new lock/mutex, but for the other player(s) in > the deadlock situation, is invaluable. I believe the WITNESS code does this. Take a look at sys/kern/kern_mutex.c. > To achieve this, you really want to associate all locks into an > overall hierarchical relationship. This lets you keep metadata on > the locks; if you are familiar with Sedgewick, what you will be > doing is keeping a running tally of the transitive closure of arcs > (one per locking thread) over a directed acyclic graph. > > When you want to assert a new lock, you imply an edge from the lock > location (as a leaf node) to the root. Then you can do a linear > traversal to the root; this will immediately point out any cycles > which would be caused by the new lock, should the assertion go > through. This is normally a small number of compares, equal to the > depth of your tree at the location where the new lock would be, plus > one. Effectively, this is an O(1) method of finding Hamiltonian > cycles in your proposed lock list. Right, approaches like this have been tried before. Doesn't SGI do it this way? Obviously you don't want an iterative approach to getting the lock, but ensuring that you don't get a lock of lower level than the ones you're currently holding should be OK. The real issue I see here is that it will be *very* difficult to establish the relationships between the locks. > Creation and deletion of lock objects in the graph is a rare event, > and so can be permitted to be relatively expensive (though it's on > the order of 20,000 transactions a second on a P60, per work done at > Novell in 1994). I'd guess that this would be acceptable. Of course, it's dangerous to make assumptions based on other operating systems. Can you come up with code to back this kind of strategy? So far, nobody else has been able to. > SIX mode locking can further increase concurrency. > > It's easy to see that you could have, for example, a process table > that you wanted to be able to simultaneously access. You could have > a top level "entrire process table" lock, and then divide the table > into 6 chunks with inferior locks hung off the top level lock. This > would let you traverse 5/6ths of the table without contention, with > one writer holding a lock for creation of a new table entry. You > could take this to a per slot locking granularity, if you felt it > were necessary for the task at hand. Take a look at the way BSD/OS does it (see my earlier message). I don't think that just arbitrarily chopping up the struct makes sense; this method would mean that you might need to get more than one of these locks because of the geographic location of the data. Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 19:41:33 2000 Delivered-To: freebsd-arch@freebsd.org Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.80]) by hub.freebsd.org (Postfix) with ESMTP id 2D72637B424; Mon, 11 Sep 2000 19:41:29 -0700 (PDT) Received: (from grog@localhost) by wantadilla.lemis.com (8.11.0/8.9.3) id e8C2f5E02783; Tue, 12 Sep 2000 12:11:05 +0930 (CST) (envelope-from grog) Date: Tue, 12 Sep 2000 12:11:05 +0930 From: Greg Lehey To: Matthew Jacob Cc: Frank Mayhar , John Baldwin , Mark Murray , FreeBSD-arch@FreeBSD.ORG Subject: Re: cvs commit: src/sys/conf files src/sys/sys random.h src/sys/dev/randomdev hash.c hash.h harvest.c randomdev.c yarrow.c yarro Message-ID: <20000912121105.J88615@wantadilla.lemis.com> References: <200009120101.e8C11nN56928@realtime.exit.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from mjacob@feral.com on Mon, Sep 11, 2000 at 06:02:26PM -0700 Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Monday, 11 September 2000 at 18:02:26 -0700, Matt Jacob wrote: >> Greg Lehey wrote: >>> I've been wondering whether we shouldn't associate mutexes with data >>> structures rather than code. It's possible that it would make it >>> easier to avoid deadlocks. Thoughts? >> >> Speaking as a BSD/OS (and former Unixware) developer: YES! > > Hmm. I would rather have assumed that this is what mutexes are > about. Semaphores gate entry in code. Mutexes provide locking on > data. Simple enough. That's a matter of definition. The big difference I see between a semaphore and a blocking "mutex" is that there's no count associated with the blocking "mutex": it's a degenerate case of a semaphore. At Tandem, we used semaphores exclusively (well, we had a mutex instruction, but it was really interrupt lockout). As far as I can recall, the semaphore counter was always 1, so the effect was identical to the current blocking "mutexes". Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 19:53:47 2000 Delivered-To: freebsd-arch@freebsd.org Received: from cs.waikato.ac.nz (taupo.cs.waikato.ac.nz [130.217.248.134]) by hub.freebsd.org (Postfix) with ESMTP id F00D537B42C; Mon, 11 Sep 2000 19:53:40 -0700 (PDT) Received: (from joerg@localhost) by cs.waikato.ac.nz (8.9.3/8.9.3) id OAA69567; Tue, 12 Sep 2000 14:52:55 +1200 (NZST) (envelope-from joerg) Date: Tue, 12 Sep 2000 14:52:55 +1200 From: Joerg Micheel To: Greg Lehey Cc: Matthew Jacob , Frank Mayhar , John Baldwin , Mark Murray , FreeBSD-arch@freebsd.org, joerg@cs.waikato.ac.nz Subject: Re: cvs commit: src/sys/conf files src/sys/sys random.h src/sys/dev/randomdev hash.c hash.h harvest.c randomdev.c yarrow.c yarro Message-ID: <20000912145255.A41113@cs.waikato.ac.nz> References: <200009120101.e8C11nN56928@realtime.exit.com> <20000912121105.J88615@wantadilla.lemis.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20000912121105.J88615@wantadilla.lemis.com>; from grog@lemis.com on Tue, Sep 12, 2000 at 12:11:05PM +0930 Organization: Dept of Computer Science, University of Waikato, Hamilton, New Zealand Project: WAND - Waikato Applied Network Dynamics, DAG Operating-System: ... powered by FreeBSD Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, Sep 12, 2000 at 12:11:05PM +0930, Greg Lehey wrote: > On Monday, 11 September 2000 at 18:02:26 -0700, Matt Jacob wrote: > >> Greg Lehey wrote: > >>> I've been wondering whether we shouldn't associate mutexes with data > >>> structures rather than code. It's possible that it would make it > >>> easier to avoid deadlocks. Thoughts? > >> > >> Speaking as a BSD/OS (and former Unixware) developer: YES! > > > > Hmm. I would rather have assumed that this is what mutexes are > > about. Semaphores gate entry in code. Mutexes provide locking on > > data. Simple enough. > > That's a matter of definition. The big difference I see between a > semaphore and a blocking "mutex" is that there's no count associated > with the blocking "mutex": it's a degenerate case of a semaphore. > > At Tandem, we used semaphores exclusively (well, we had a mutex > instruction, but it was really interrupt lockout). As far as I can > recall, the semaphore counter was always 1, so the effect was > identical to the current blocking "mutexes". I liked the model Sun chose for Solaris. They have mutex', rw_locks, condition variables. I don't like semaphores. Mutexes are for short locks. Condition variables are for long-term waits, they are associated with a mutex. You can only sleep/wakeup a CV when holding the associated with it, which prevents races. When having to sleep on a CV the kernel would unlock the mutex and reaquire it for the running thread before returning. Joerg -- Joerg B. Micheel Email: Waikato Applied Network Dynamics Phone: +64 7 8384794 The University of Waikato, CompScience Fax: +64 7 8585095 Private Bag 3105 Pager: +64 868 38222 Hamilton, New Zealand Plan: TINE and the DAG's To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 20: 2: 6 2000 Delivered-To: freebsd-arch@freebsd.org Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.80]) by hub.freebsd.org (Postfix) with ESMTP id 3CCFB37B622; Mon, 11 Sep 2000 20:02:00 -0700 (PDT) Received: (from grog@localhost) by wantadilla.lemis.com (8.11.0/8.9.3) id e8C31E505290; Tue, 12 Sep 2000 12:31:14 +0930 (CST) (envelope-from grog) Date: Tue, 12 Sep 2000 12:31:14 +0930 From: Greg Lehey To: Joerg Micheel Cc: Matthew Jacob , Frank Mayhar , John Baldwin , Mark Murray , FreeBSD-arch@freebsd.org Subject: Mutexes and semaphores (was: cvs commit: src/sys/conf files src/sys/sys random.h src/sys/dev/randomdev hash.c hash.h harvest.c randomdev.c yarrow.c yarro) Message-ID: <20000912123114.K88615@wantadilla.lemis.com> References: <200009120101.e8C11nN56928@realtime.exit.com> <20000912121105.J88615@wantadilla.lemis.com> <20000912145255.A41113@cs.waikato.ac.nz> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <20000912145255.A41113@cs.waikato.ac.nz>; from joerg@cs.waikato.ac.nz on Tue, Sep 12, 2000 at 02:52:55PM +1200 Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tuesday, 12 September 2000 at 14:52:55 +1200, Joerg Micheel wrote: > On Tue, Sep 12, 2000 at 12:11:05PM +0930, Greg Lehey wrote: >> On Monday, 11 September 2000 at 18:02:26 -0700, Matt Jacob wrote: >>>> Greg Lehey wrote: >>>>> I've been wondering whether we shouldn't associate mutexes with data >>>>> structures rather than code. It's possible that it would make it >>>>> easier to avoid deadlocks. Thoughts? >>>> >>>> Speaking as a BSD/OS (and former Unixware) developer: YES! >>> >>> Hmm. I would rather have assumed that this is what mutexes are >>> about. Semaphores gate entry in code. Mutexes provide locking on >>> data. Simple enough. >> >> That's a matter of definition. The big difference I see between a >> semaphore and a blocking "mutex" is that there's no count associated >> with the blocking "mutex": it's a degenerate case of a semaphore. >> >> At Tandem, we used semaphores exclusively (well, we had a mutex >> instruction, but it was really interrupt lockout). As far as I can >> recall, the semaphore counter was always 1, so the effect was >> identical to the current blocking "mutexes". > > I liked the model Sun chose for Solaris. They have mutex', rw_locks, > condition variables. I don't like semaphores. What's the difference between a mutex and a semaphore? > Mutexes are for short locks. Condition variables are for long-term > waits, they are associated with a mutex. You can only sleep/wakeup a > CV when holding the associated with it, which prevents races. When > having to sleep on a CV the kernel would unlock the mutex and > reaquire it for the running thread before returning. Yes, that's pretty much what msleep() does. We're still discussing whether we should have real condition variables. Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 20: 8:16 2000 Delivered-To: freebsd-arch@freebsd.org Received: from pcnet1.pcnet.com (pcnet1.pcnet.com [204.213.232.3]) by hub.freebsd.org (Postfix) with ESMTP id 0DD0337B422 for ; Mon, 11 Sep 2000 20:08:14 -0700 (PDT) Received: (from eischen@localhost) by pcnet1.pcnet.com (8.8.7/PCNet) id XAA14349; Mon, 11 Sep 2000 23:07:56 -0400 (EDT) Date: Mon, 11 Sep 2000 23:07:56 -0400 (EDT) From: Daniel Eischen To: Alfred Perlstein Cc: Jason Evans , Greg Lehey , arch@FreeBSD.ORG Subject: Re: Long-term mutex ownership (was Re: Interruptable mutex aquires.) In-Reply-To: <20000911192610.D12231@fw.wintelcom.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Mon, 11 Sep 2000, Alfred Perlstein wrote: > * Jason Evans [000911 19:24] wrote: > > On Tue, Sep 12, 2000 at 11:41:54AM +0930, Greg Lehey wrote: > > > I think we need to come to some kind of consensus about how we are > > > going to structure locking before we go into this much detail. At the > > > moment we don't even agree whether we can hold on to (blocking) > > > mutexes for long periods of time. > > > > I don't recall the original argument against holding mutexes for long > > periods. From an abstract point of view, there's nothing wrong with such > > practice, and in fact it makes sense for many problems. Is there an issue > > with our implementation? If so, can someone please explain it? > > You are currently unable to abort if a signal comes in. :) > > Conditional variables add overhead and complexity where it's > not needed. On the contrary. Our current mutex scheme with all the flags is not as simple and as easy to understand as other OS kernel mutexes I've dealt with. Having flags to mtx_enter() makes it more error prone and I'd rather just use the mutex as it was mtx_init()'d. I don't even want recursive mutexes and think that simple mutexes with separate count fields should be used (I would probably lose this argument though). Condition variables are used when you plan on waiting for something, like I/O completion. You can wake one or all threads waiting on condition variables. This is not typical of mutexes. Our current splXXX()/tsleep()/splx() scheme corresponds very nicely with mtx_enter()/cv_wait()/mtx_exit(). I don't recommend obfuscating mutexes even more by creating yet another variant to be used in place of tsleep() or cv_wait(). -- Dan Eischen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 20:51:59 2000 Delivered-To: freebsd-arch@freebsd.org Received: from phnxpop3.phnx.uswest.net (phnxpop3.phnx.uswest.net [206.80.192.3]) by hub.freebsd.org (Postfix) with SMTP id 2752337B422 for ; Mon, 11 Sep 2000 20:51:52 -0700 (PDT) Received: (qmail 97669 invoked by alias); 12 Sep 2000 03:51:48 -0000 Delivered-To: fixup-FreeBSD-arch@FreeBSD.ORG@fixme Received: (qmail 97507 invoked by uid 0); 12 Sep 2000 03:51:45 -0000 Received: from 63-224-154-90.customers.uswest.net (HELO pinyon.org) (63.224.154.90) by phnxpop3.phnx.uswest.net with SMTP; 12 Sep 2000 03:51:45 -0000 Received: from chomsky.Pinyon.ORG (localhost [127.0.0.1]) by pinyon.org (Postfix) with ESMTP id 44B1868 for ; Mon, 11 Sep 2000 20:51:45 -0700 (MST) X-Mailer: exmh version 2.1.1 10/15/1999 To: FreeBSD-arch@FreeBSD.ORG Subject: Re: cvs commit: src/sys/conf files src/sys/sys random.h src/sys/dev/randomdev hash.c hash.h harvest.c randomdev.c yarrow.c yarro In-Reply-To: Message from Joerg Micheel of "Tue, 12 Sep 2000 14:52:55 +1200." <20000912145255.A41113@cs.waikato.ac.nz> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 11 Sep 2000 20:51:45 -0700 From: "Russell L. Carter" Message-Id: <20000912035145.44B1868@pinyon.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG %On Tue, Sep 12, 2000 at 12:11:05PM +0930, Greg Lehey wrote: %> On Monday, 11 September 2000 at 18:02:26 -0700, Matt Jacob wrote: %> >> Greg Lehey wrote: %> >>> I've been wondering whether we shouldn't associate mutexes with data %> >>> structures rather than code. It's possible that it would make it %> >>> easier to avoid deadlocks. Thoughts? %> >> %> >> Speaking as a BSD/OS (and former Unixware) developer: YES! %> > %> > Hmm. I would rather have assumed that this is what mutexes are %> > about. Semaphores gate entry in code. Mutexes provide locking on %> > data. Simple enough. %> %> That's a matter of definition. The big difference I see between a %> semaphore and a blocking "mutex" is that there's no count associated %> with the blocking "mutex": it's a degenerate case of a semaphore. %> %> At Tandem, we used semaphores exclusively (well, we had a mutex %> instruction, but it was really interrupt lockout). As far as I can %> recall, the semaphore counter was always 1, so the effect was %> identical to the current blocking "mutexes". % %I liked the model Sun chose for Solaris. They have mutex', rw_locks, %condition variables. I don't like semaphores. Mutexes are for short %locks. Condition variables are for long-term waits, they are associated %with a mutex. You can only sleep/wakeup a CV when holding the associated %with it, which prevents races. When having to sleep on a CV the kernel %would unlock the mutex and reaquire it for the running thread before %returning. CVs are the natural pattern for an environment with worker thread(s) and delegator(s). You can't avoid a mutex hierarchy if you want to provide at least multithreaded R access to necessary data structures. I spend probably 50% of my time working out architectures that allow R and W access to complex data structures accessed by a slew of atomic ops. It's hard, but fun. So it's cool to see it here. An OS is a lot harder... Russell % % Joerg %-- %Joerg B. Micheel Email: %Waikato Applied Network Dynamics Phone: +64 7 8384794 %The University of Waikato, CompScience Fax: +64 7 8585095 %Private Bag 3105 Pager: +64 868 38222 %Hamilton, New Zealand Plan: TINE and the DAG's % % %To Unsubscribe: send mail to majordomo@FreeBSD.org %with "unsubscribe freebsd-arch" in the body of the message % To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 21: 4:57 2000 Delivered-To: freebsd-arch@freebsd.org Received: from berserker.bsdi.com (berserker.twistedbit.com [199.79.183.1]) by hub.freebsd.org (Postfix) with ESMTP id 88E3B37B423 for ; Mon, 11 Sep 2000 21:04:54 -0700 (PDT) Received: from berserker.bsdi.com (cp@[127.0.0.1]) by berserker.bsdi.com (8.9.3/8.9.3) with ESMTP id WAA25503; Mon, 11 Sep 2000 22:04:28 -0600 (MDT) Message-Id: <200009120404.WAA25503@berserker.bsdi.com> To: Jason Evans Cc: Greg Lehey , arch@freebsd.org Subject: Re: Long-term mutex ownership (was Re: Interruptable mutex aquires.) From: Chuck Paterson Date: Mon, 11 Sep 2000 22:04:28 -0600 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG If you hold a mutex across a sleep call then release of the mutex becomes dependent upon another process. This makes it impossible (at least from a practical point of view) to prove that there are not deadlocks. From my experience I was use to mutexs being held across sleep (async event waits) calls. Eric Varsanyi whose background was Cray, rather than Sun, thought we should do without them until they were needed. It turned out that we didn't really need them at all so we decided to go without and not have to fight the dead lock issues. It wasn't as if we could do with out them, it was not an issue at all. Mutex are for protecting data there are other mechanism for doing the other types of synchronization. Chuck } }I don't recall the original argument against holding mutexes for long }periods. From an abstract point of view, there's nothing wrong with such }practice, and in fact it makes sense for many problems. Is there an issue }with our implementation? If so, can someone please explain it? } }Thanks, }Jason } } }To Unsubscribe: send mail to majordomo@FreeBSD.org }with "unsubscribe freebsd-arch" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 21: 8:30 2000 Delivered-To: freebsd-arch@freebsd.org Received: from smtp01.primenet.com (smtp01.primenet.com [206.165.6.131]) by hub.freebsd.org (Postfix) with ESMTP id 3059437B422; Mon, 11 Sep 2000 21:08:27 -0700 (PDT) Received: (from daemon@localhost) by smtp01.primenet.com (8.9.3/8.9.3) id VAA29313; Mon, 11 Sep 2000 21:07:50 -0700 (MST) Received: from usr07.primenet.com(206.165.6.207) via SMTP by smtp01.primenet.com, id smtpdAAAaRaGh5; Mon Sep 11 21:07:38 2000 Received: (from tlambert@localhost) by usr07.primenet.com (8.8.5/8.8.5) id VAA19788; Mon, 11 Sep 2000 21:08:10 -0700 (MST) From: Terry Lambert Message-Id: <200009120408.VAA19788@usr07.primenet.com> Subject: Re: SMP lock ordering (was: cvs commit: src/sys/conf files src/sys/sys random.h src/sys/dev/randomdev hash.c hash.h harvest.c random To: grog@lemis.com (Greg Lehey) Date: Tue, 12 Sep 2000 04:08:10 +0000 (GMT) Cc: tlambert@primenet.com (Terry Lambert), jhb@pike.osd.bsdi.com (John Baldwin), markm@FreeBSD.ORG (Mark Murray), FreeBSD-arch@FreeBSD.ORG In-Reply-To: <20000912120815.I88615@wantadilla.lemis.com> from "Greg Lehey" at Sep 12, 2000 12:08:15 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG [ ... hierarchical locks with up front state costing ... ] > Right, approaches like this have been tried before. Doesn't SGI do it > this way? Obviously you don't want an iterative approach to getting > the lock, but ensuring that you don't get a lock of lower level than > the ones you're currently holding should be OK. I don't know what SGI does; that's one UNIX kernel I haven't been into the guts of yet. > The real issue I see here is that it will be *very* difficult to > establish the relationships between the locks. It's complex, but it's not really that difficult, given the correct structures. > > Creation and deletion of lock objects in the graph is a rare event, > > and so can be permitted to be relatively expensive (though it's on > > the order of 20,000 transactions a second on a P60, per work done at > > Novell in 1994). > > I'd guess that this would be acceptable. Of course, it's dangerous to > make assumptions based on other operating systems. That was 20,000 transactions a second in a user space process acting as a lock manager. The first attempt was a spinlock, followed by a heavier weight semaphore acquisition on a collision. > Can you come up with code to back this kind of strategy? So far, > nobody else has been able to. I have some code. It tries to optimize cycles by inheritance, when someone holds some locks, and goes onto a wait list for a lock held by someone else. This adds an additional dimension to the pot, resulting in a 5 dimensional graph, but it's not that hard. You just have to think in terms of things that can be locked, things that can do locking, groups of things that can do locking (as in multiple threads in a single process), the set of locks held by the group (an arc), and the base graph in which the hierarchy of lockables are created. Mostly, it's just pointer frobbing and some nasty data structures. > > SIX mode locking can further increase concurrency. [ ... bad example of where you would increase granularity ... ] > Take a look at the way BSD/OS does it (see my earlier message). I > don't think that just arbitrarily chopping up the struct makes sense; > this method would mean that you might need to get more than one of > these locks because of the geographic location of the data. OK, my example was bad, but the idea is sound; a better example would be a system wide and per processor resource pools that get filled/drained from/to the system pools for things like vnodes and other objects. The point was that the amount of real contention would be reduced. The "process" structure I described was really a per client state structure for a file server, in the work back in 1994, but I didn't want to complicate things. I guess it's best to stick with first principles... the pools example might have one processor stalling another in an 8 processor system in order to do page stealing, while the other 6 processors march merrily along. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 21:25:37 2000 Delivered-To: freebsd-arch@freebsd.org Received: from cs.waikato.ac.nz (taupo.cs.waikato.ac.nz [130.217.248.134]) by hub.freebsd.org (Postfix) with ESMTP id 4DE2937B424; Mon, 11 Sep 2000 21:25:32 -0700 (PDT) Received: (from joerg@localhost) by cs.waikato.ac.nz (8.9.3/8.9.3) id QAA69871; Tue, 12 Sep 2000 16:25:06 +1200 (NZST) (envelope-from joerg) Date: Tue, 12 Sep 2000 16:25:06 +1200 From: Joerg Micheel To: Greg Lehey Cc: Matthew Jacob , Frank Mayhar , John Baldwin , Mark Murray , FreeBSD-arch@freebsd.org, joerg@cs.waikato.ac.nz Subject: Re: Mutexes and semaphores (was: cvs commit: src/sys/conf files src/sys/sys random.h src/sys/dev/randomdev hash.c hash.h harvest.c randomdev.c yarrow.c yarro) Message-ID: <20000912162506.C41113@cs.waikato.ac.nz> References: <200009120101.e8C11nN56928@realtime.exit.com> <20000912121105.J88615@wantadilla.lemis.com> <20000912145255.A41113@cs.waikato.ac.nz> <20000912123114.K88615@wantadilla.lemis.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20000912123114.K88615@wantadilla.lemis.com>; from grog@lemis.com on Tue, Sep 12, 2000 at 12:31:14PM +0930 Organization: Dept of Computer Science, University of Waikato, Hamilton, New Zealand Project: WAND - Waikato Applied Network Dynamics, DAG Operating-System: ... powered by FreeBSD Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, Sep 12, 2000 at 12:31:14PM +0930, Greg Lehey wrote: > On Tuesday, 12 September 2000 at 14:52:55 +1200, Joerg Micheel wrote: > > On Tue, Sep 12, 2000 at 12:11:05PM +0930, Greg Lehey wrote: > >> On Monday, 11 September 2000 at 18:02:26 -0700, Matt Jacob wrote: > >>>> Greg Lehey wrote: > >>>>> I've been wondering whether we shouldn't associate mutexes with data > >>>>> structures rather than code. It's possible that it would make it > >>>>> easier to avoid deadlocks. Thoughts? > >>>> > >>>> Speaking as a BSD/OS (and former Unixware) developer: YES! > >>> > >>> Hmm. I would rather have assumed that this is what mutexes are > >>> about. Semaphores gate entry in code. Mutexes provide locking on > >>> data. Simple enough. > >> > >> That's a matter of definition. The big difference I see between a > >> semaphore and a blocking "mutex" is that there's no count associated > >> with the blocking "mutex": it's a degenerate case of a semaphore. > >> > >> At Tandem, we used semaphores exclusively (well, we had a mutex > >> instruction, but it was really interrupt lockout). As far as I can > >> recall, the semaphore counter was always 1, so the effect was > >> identical to the current blocking "mutexes". > > > > I liked the model Sun chose for Solaris. They have mutex', rw_locks, > > condition variables. I don't like semaphores. > > What's the difference between a mutex and a semaphore? None, if it comes to the actual usage and implementation. Usually, people will use semaphores for locking blocks of code, mutexes for data structures. Not much of a difference in the end result, but locking data structures seems more natural to me. Terms are used interchangeably, so it may not be a difference at all to someone else. I think the point I was making was that the syncronization primitives in Solaris present a consistent view to the programmer, it is very clear what to do to achieve your bit of syncronization (which method to use). I cannot comment on how difficult (and expensive in terms of overhead) it is to use the model. Joerg -- Joerg B. Micheel Email: Waikato Applied Network Dynamics Phone: +64 7 8384794 The University of Waikato, CompScience Fax: +64 7 8585095 Private Bag 3105 Pager: +64 868 38222 Hamilton, New Zealand Plan: TINE and the DAG's To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 22: 9:17 2000 Delivered-To: freebsd-arch@freebsd.org Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.80]) by hub.freebsd.org (Postfix) with ESMTP id 680EF37B423 for ; Mon, 11 Sep 2000 22:09:13 -0700 (PDT) Received: (from grog@localhost) by wantadilla.lemis.com (8.11.0/8.9.3) id e8C58td20847; Tue, 12 Sep 2000 14:38:55 +0930 (CST) (envelope-from grog) Date: Tue, 12 Sep 2000 14:38:55 +0930 From: Greg Lehey To: Chuck Paterson Cc: Jason Evans , arch@freebsd.org Subject: Re: Long-term mutex ownership (was Re: Interruptable mutex aquires.) Message-ID: <20000912143855.O88615@wantadilla.lemis.com> References: <200009120404.WAA25503@berserker.bsdi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <200009120404.WAA25503@berserker.bsdi.com>; from cp@bsdi.com on Mon, Sep 11, 2000 at 10:04:28PM -0600 Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Monday, 11 September 2000 at 22:04:28 -0600, Chuck Paterson wrote: >> I don't recall the original argument against holding mutexes for long >> periods. From an abstract point of view, there's nothing wrong with such >> practice, and in fact it makes sense for many problems. Is there an issue >> with our implementation? If so, can someone please explain it? > > If you hold a mutex across a sleep call then release of the > mutex becomes dependent upon another process. This makes it > impossible (at least from a practical point of view) to prove that > there are not deadlocks. That depends on what locks you're holding, and who else would be interested in them. > From my experience I was use to mutexs being held across sleep > (async event waits) calls. Eric Varsanyi whose background was Cray, > rather than Sun, thought we should do without them until they were > needed. That's definitely a good idea :-) > It turned out that we didn't really need them at all Well, that depends on how satisfied you are with tsleep. > so we decided to go without and not have to fight the dead lock > issues. It wasn't as if we could do with out them, it was not an > issue at all. The general consensus (which I currently don't yet share) is that we should use condition variables for things like async event waits. I'm still looking for a consistent definition of condition variables, and how they differ from "mutexes". > Mutex are for protecting data there are other mechanism for doing > the other types of synchronization. In the current system, this is correct. That doesn't mean it's necessarily the optimum. The thing I'm looking at most here is tsleep (which we have renamed msleep because of the additional mutex parameter). The real problem is wakeup: we need to search a list of processes, which is slow, and wakeup itself needs to go to the bitter end. Yes, we can use wakeup_one (which stops at the first proc with the right p_wchan), but we still need to search an average of half the chain. We have 128 hash buckets, but that's not many for a system which could have thousands of processes active. The obvious alternative is an object on which you can queue waiting processes, whether we want to call it a mutex, semaphore, condition variable or turnstile. I really need to think out some simple examples. Everything we've been doing lately looks so complicated, and though we can understand the complexity relatively well, I think it's making it difficult to gain an overview. I think our arguments are showing more our individual perspective than any real difference in viewpoint. Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 22:16:11 2000 Delivered-To: freebsd-arch@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 9D92B37B422 for ; Mon, 11 Sep 2000 22:16:09 -0700 (PDT) Received: (from jhb@localhost) by pike.osd.bsdi.com (8.9.3/8.9.3) id WAA78736; Mon, 11 Sep 2000 22:15:25 -0700 (PDT) (envelope-from jhb) From: John Baldwin Message-Id: <200009120515.WAA78736@pike.osd.bsdi.com> Subject: Re: Long-term mutex ownership (was Re: Interruptable mutex aquires.) In-Reply-To: <20000912143855.O88615@wantadilla.lemis.com> from Greg Lehey at "Sep 12, 2000 02:38:55 pm" To: Greg Lehey Date: Mon, 11 Sep 2000 22:15:25 -0700 (PDT) Cc: Chuck Paterson , Jason Evans , arch@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL68 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Greg Lehey wrote: > The general consensus (which I currently don't yet share) is that we > should use condition variables for things like async event waits. I'm > still looking for a consistent definition of condition variables, and > how they differ from "mutexes". Go grab Andrew S. Tannebaum's (sp?) _Modern_Operating_Systems_. One of the first chapters gives execellent treatment to comparing/contrasting mutexes, semaphores, condition variables, and sleep/wakeup. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.cslab.vt.edu/~jobaldwi/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 22:56:58 2000 Delivered-To: freebsd-arch@freebsd.org Received: from smtp02.iafrica.com (smtp02.iafrica.com [196.7.0.140]) by hub.freebsd.org (Postfix) with ESMTP id 0652637B424 for ; Mon, 11 Sep 2000 22:56:54 -0700 (PDT) Received: from [196.7.18.138] (helo=grimreaper.grondar.za ident=root) by smtp02.iafrica.com with esmtp (Exim 1.92 #1) id 13Yj3o-000Pjg-00; Tue, 12 Sep 2000 07:56:49 +0200 Received: from grimreaper.grondar.za (mark@localhost [127.0.0.1]) by grimreaper.grondar.za (8.11.0/8.11.0) with ESMTP id e8C5ubX72439; Tue, 12 Sep 2000 07:56:38 +0200 (SAST) (envelope-from mark@grimreaper.grondar.za) Message-Id: <200009120556.e8C5ubX72439@grimreaper.grondar.za> To: Greg Lehey Cc: John Baldwin , FreeBSD-arch@FreeBSD.org Subject: Re: cvs commit: src/sys/conf files src/sys/sys random.h src/sys/dev/randomdev hash.c hash.h harvest.c randomdev.c yarrow.c yarro References: <20000912091705.O19431@wantadilla.lemis.com> In-Reply-To: <20000912091705.O19431@wantadilla.lemis.com> ; from Greg Lehey "Tue, 12 Sep 2000 09:17:05 +0930." Date: Tue, 12 Sep 2000 07:56:36 +0200 From: Mark Murray Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > I've been wondering whether we shouldn't associate mutexes with data > structures rather than code. It's possible that it would make it > easier to avoid deadlocks. Thoughts? That was exactly what I did with the 2 mutexes in the /dev/random driver. One mutex protects the entropy-harvesting FIFO, and the other protects the internal state structure during reseeds. M -- Mark Murray Join the anti-SPAM movement: http://www.cauce.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 22:57:52 2000 Delivered-To: freebsd-arch@freebsd.org Received: from cs.waikato.ac.nz (taupo.cs.waikato.ac.nz [130.217.248.134]) by hub.freebsd.org (Postfix) with ESMTP id CDB5037B424 for ; Mon, 11 Sep 2000 22:57:48 -0700 (PDT) Received: (from joerg@localhost) by cs.waikato.ac.nz (8.9.3/8.9.3) id RAA70181; Tue, 12 Sep 2000 17:57:25 +1200 (NZST) (envelope-from joerg) Date: Tue, 12 Sep 2000 17:57:25 +1200 From: Joerg Micheel To: John Baldwin Cc: Greg Lehey , Chuck Paterson , Jason Evans , arch@freebsd.org, joerg@cs.waikato.ac.nz Subject: Re: Long-term mutex ownership (was Re: Interruptable mutex aquires.) Message-ID: <20000912175725.A70000@cs.waikato.ac.nz> References: <20000912143855.O88615@wantadilla.lemis.com> <200009120515.WAA78736@pike.osd.bsdi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200009120515.WAA78736@pike.osd.bsdi.com>; from jhb@pike.osd.bsdi.com on Mon, Sep 11, 2000 at 10:15:25PM -0700 Organization: Dept of Computer Science, University of Waikato, Hamilton, New Zealand Project: WAND - Waikato Applied Network Dynamics, DAG Operating-System: ... powered by FreeBSD Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Mon, Sep 11, 2000 at 10:15:25PM -0700, John Baldwin wrote: > Greg Lehey wrote: > > The general consensus (which I currently don't yet share) is that we > > should use condition variables for things like async event waits. I'm > > still looking for a consistent definition of condition variables, and > > how they differ from "mutexes". > > Go grab Andrew S. Tannebaum's (sp?) _Modern_Operating_Systems_. One of > the first chapters gives execellent treatment to comparing/contrasting > mutexes, semaphores, condition variables, and sleep/wakeup. ... and once you are done with it find out that it is all the same just offering different views onto something that can be implemented and used in exactly the same way. I've just gone through this - trying to explain it to 239 second years students - I can't see any difference. Joerg -- Joerg B. Micheel Email: Waikato Applied Network Dynamics Phone: +64 7 8384794 The University of Waikato, CompScience Fax: +64 7 8585095 Private Bag 3105 Pager: +64 868 38222 Hamilton, New Zealand Plan: TINE and the DAG's To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 23: 1:28 2000 Delivered-To: freebsd-arch@freebsd.org Received: from smtp02.iafrica.com (smtp02.iafrica.com [196.7.0.140]) by hub.freebsd.org (Postfix) with ESMTP id 9770137B423 for ; Mon, 11 Sep 2000 23:01:24 -0700 (PDT) Received: from [196.7.18.138] (helo=grimreaper.grondar.za ident=root) by smtp02.iafrica.com with esmtp (Exim 1.92 #1) id 13Yj80-000Pph-00; Tue, 12 Sep 2000 08:01:09 +0200 Received: from grimreaper.grondar.za (mark@localhost [127.0.0.1]) by grimreaper.grondar.za (8.11.0/8.11.0) with ESMTP id e8C611X75116; Tue, 12 Sep 2000 08:01:01 +0200 (SAST) (envelope-from mark@grimreaper.grondar.za) Message-Id: <200009120601.e8C611X75116@grimreaper.grondar.za> To: Jason Evans Cc: Greg Lehey , John Baldwin , FreeBSD-arch@FreeBSD.org Subject: Re: cvs commit: src/sys/conf files src/sys/sys random.h src/sys/dev/randomdev hash.c hash.h harvest.c randomdev.c yarrow.c yarro References: <20000911171819.A31089@blitz.canonware.com> In-Reply-To: <20000911171819.A31089@blitz.canonware.com> ; from Jason Evans "Mon, 11 Sep 2000 17:18:19 MST." Date: Tue, 12 Sep 2000 08:01:00 +0200 From: Mark Murray Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Mark's changes to randomdev are very straight-forward, and they use a mutex > exactly as a mutex should be used (though a condition variable will help > make things even cleaner, once they're available). This is good to know; thanks! :-) > A general discussion of high level locking strategy is in order, but in my > opinion, Mark's code is not an appropriate harbinger, since there's > absolutely nothing wrong with the approach he took, regardless of the > results of high level discussion. The issue we need to decide on is how to > avoid bad interactions (read deadlock/livelock) between mutex hierarchies, > but the randomdev mutex basically stands alone, so isn't subject to those > discussions. I kinda winged it on this one, but I had the assistance of folk online. It would be good to have some hardcore documentation. M -- Mark Murray Join the anti-SPAM movement: http://www.cauce.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 23: 9:17 2000 Delivered-To: freebsd-arch@freebsd.org Received: from cs.waikato.ac.nz (taupo.cs.waikato.ac.nz [130.217.248.134]) by hub.freebsd.org (Postfix) with ESMTP id 7C6F737B422 for ; Mon, 11 Sep 2000 23:09:14 -0700 (PDT) Received: (from joerg@localhost) by cs.waikato.ac.nz (8.9.3/8.9.3) id SAA70225; Tue, 12 Sep 2000 18:09:04 +1200 (NZST) (envelope-from joerg) Date: Tue, 12 Sep 2000 18:09:03 +1200 From: Joerg Micheel To: Greg Lehey Cc: Chuck Paterson , Jason Evans , arch@freebsd.org, joerg@cs.waikato.ac.nz Subject: Re: Long-term mutex ownership (was Re: Interruptable mutex aquires.) Message-ID: <20000912180903.B70000@cs.waikato.ac.nz> References: <200009120404.WAA25503@berserker.bsdi.com> <20000912143855.O88615@wantadilla.lemis.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20000912143855.O88615@wantadilla.lemis.com>; from grog@lemis.com on Tue, Sep 12, 2000 at 02:38:55PM +0930 Organization: Dept of Computer Science, University of Waikato, Hamilton, New Zealand Project: WAND - Waikato Applied Network Dynamics, DAG Operating-System: ... powered by FreeBSD Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, Sep 12, 2000 at 02:38:55PM +0930, Greg Lehey wrote: > The general consensus (which I currently don't yet share) is that we > should use condition variables for things like async event waits. I'm > still looking for a consistent definition of condition variables, and > how they differ from "mutexes". There is no memory associated with condition variables. Someone can grab a mutex. The condition variable is a rendevous thing - you signal on it. If someone is waiting - fine, they get a ring. > The thing I'm looking at most here is tsleep (which we have renamed > msleep because of the additional mutex parameter). The real problem > is wakeup: we need to search a list of processes, which is slow, and > wakeup itself needs to go to the bitter end. Yes, we can use > wakeup_one (which stops at the first proc with the right p_wchan), but > we still need to search an average of half the chain. We have 128 > hash buckets, but that's not many for a system which could have > thousands of processes active. The obvious alternative is an object > on which you can queue waiting processes, whether we want to call it a > mutex, semaphore, condition variable or turnstile. You are discussing implementation here. The mutex is a replacement for splx() locking. The condition variable is a replacement for sleep()/wakeup() on multiprocessors. Likewise, for a rendevous on a cv, you need to hold a mutex, much like having to raise the splx() level before going on sleep(). It is not necessarily the best model to think of it, but something that everyone beeing around in UNIX for the past 20 years can understand with not too much effort. The rw_lock is a new smart way to improve parallism, it is much the same as a mutex, only you need to tell what you are going to do and that determines whether you are allowed to enter. Apart from that it is identical to the mutex. > I really need to think out some simple examples. Everything we've > been doing lately looks so complicated, and though we can understand > the complexity relatively well, I think it's making it difficult to > gain an overview. I think our arguments are showing more our > individual perspective than any real difference in viewpoint. That's the programmers (users) perspective. I think you really just have two different "things": mutex and cond variable (the rw_lock being sugar coating). As you said, the FreeBSD term for condition variable is the msleep() mechanism. You need mutexes in addition. Do you think this is too much ? Too little structure ? Joerg -- Joerg B. Micheel Email: Waikato Applied Network Dynamics Phone: +64 7 8384794 The University of Waikato, CompScience Fax: +64 7 8585095 Private Bag 3105 Pager: +64 868 38222 Hamilton, New Zealand Plan: TINE and the DAG's To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 23:32:18 2000 Delivered-To: freebsd-arch@freebsd.org Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.80]) by hub.freebsd.org (Postfix) with ESMTP id 2573437B422 for ; Mon, 11 Sep 2000 23:32:13 -0700 (PDT) Received: (from grog@localhost) by wantadilla.lemis.com (8.11.0/8.9.3) id e8C6Vl424063; Tue, 12 Sep 2000 16:01:47 +0930 (CST) (envelope-from grog) Date: Tue, 12 Sep 2000 16:01:47 +0930 From: Greg Lehey To: Joerg Micheel Cc: John Baldwin , Chuck Paterson , Jason Evans , arch@freebsd.org Subject: Re: Long-term mutex ownership (was Re: Interruptable mutex aquires.) Message-ID: <20000912160147.A23948@wantadilla.lemis.com> References: <20000912143855.O88615@wantadilla.lemis.com> <200009120515.WAA78736@pike.osd.bsdi.com> <20000912175725.A70000@cs.waikato.ac.nz> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <20000912175725.A70000@cs.waikato.ac.nz>; from joerg@cs.waikato.ac.nz on Tue, Sep 12, 2000 at 05:57:25PM +1200 Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tuesday, 12 September 2000 at 17:57:25 +1200, Joerg Micheel wrote: > On Mon, Sep 11, 2000 at 10:15:25PM -0700, John Baldwin wrote: >> Greg Lehey wrote: >>> The general consensus (which I currently don't yet share) is that we >>> should use condition variables for things like async event waits. I'm >>> still looking for a consistent definition of condition variables, and >>> how they differ from "mutexes". >> >> Go grab Andrew S. Tannebaum's (sp?) _Modern_Operating_Systems_. One of >> the first chapters gives execellent treatment to comparing/contrasting >> mutexes, semaphores, condition variables, and sleep/wakeup. > > ... and once you are done with it find out that it is all the same > just offering different views onto something that can be implemented > and used in exactly the same way. I've just gone through this - > trying to explain it to 239 second years students - I can't see any > difference. My point exactly. Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 23:32:18 2000 Delivered-To: freebsd-arch@freebsd.org Received: from mercury.Sun.COM (mercury.Sun.COM [192.9.25.1]) by hub.freebsd.org (Postfix) with ESMTP id 6F04237B424; Mon, 11 Sep 2000 23:32:14 -0700 (PDT) Received: from ms-emuc07-01.Germany.Sun.COM ([129.157.128.14]) by mercury.Sun.COM (8.9.3+Sun/8.9.3) with ESMTP id XAA28676; Mon, 11 Sep 2000 23:32:05 -0700 (PDT) Received: from germany.sun.com (hacker [129.157.133.195]) by ms-emuc07-01.Germany.Sun.COM (8.9.3+Sun/8.9.3/ENSMAIL,v1.9) with ESMTP id IAA05431; Tue, 12 Sep 2000 08:32:04 +0200 (MEST) Message-ID: <39BDCDB2.B50BBB52@germany.sun.com> Date: Tue, 12 Sep 2000 08:31:14 +0200 From: Michael Schuster - Sun Germany Organization: Sun Microsystems, Inc. X-Mailer: Mozilla 4.73 [en] (X11; I; SunOS 5.8 sun4u) X-Accept-Language: en MIME-Version: 1.0 To: Joerg Micheel Cc: Greg Lehey , Matthew Jacob , Frank Mayhar , John Baldwin , Mark Murray , FreeBSD-arch@FreeBSD.ORG Subject: Re: Mutexes and semaphores (was: cvs commit: src/sys/conf files src/sys/sys random.h src/sys/dev/randomdev hash.c hash.h harvest.c randomdev.c yarrow.c yarro) References: <200009120101.e8C11nN56928@realtime.exit.com> <20000912121105.J88615@wantadilla.lemis.com> <20000912145255.A41113@cs.waikato.ac.nz> <20000912123114.K88615@wantadilla.lemis.com> <20000912162506.C41113@cs.waikato.ac.nz> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Joerg Micheel wrote: > > What's the difference between a mutex and a semaphore? > > None, if it comes to the actual usage and implementation. Usually, > people will use semaphores for locking blocks of code, mutexes for > data structures. Not much of a difference in the end result, but > locking data structures seems more natural to me. well, (in Solaris) there is a difference, mainly in implementation, which affects debugability (is that a correct word? :-) amongs others: Mutexes have ownership whereas semaphores don't. This makes detection of recursive mutex enter (which is not allowed in Solaris, I'm happy to say) and illegal freeing of mutexes trivial. Also, semaphores are multi-valued, and therefore lend themselves to some uses mutexes aren't meant for (I think the dining philosophers were a classical example of that). Performancewise, mutexes are fastest, as they're implemented directly on top of (Sparc's) CAS instruction (in the simple case). > Joerg cheers Michael -- Michael Schuster / Michael.Schuster@sun.com Sun Microsystems GmbH / Sonnenallee 1, D-85551 Heimstetten (+49 89) 46008-2974 / x62974 Recursion, n.: see 'Recursion' To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 23:35:37 2000 Delivered-To: freebsd-arch@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id 787B137B422; Mon, 11 Sep 2000 23:35:35 -0700 (PDT) Received: from beppo.feral.com (beppo [192.67.166.79]) by feral.com (8.9.3/8.9.3) with ESMTP id XAA13810; Mon, 11 Sep 2000 23:35:14 -0700 Date: Mon, 11 Sep 2000 23:34:55 -0700 (PDT) From: Matthew Jacob Reply-To: mjacob@feral.com To: Joerg Micheel Cc: Greg Lehey , Frank Mayhar , John Baldwin , Mark Murray , FreeBSD-arch@FreeBSD.ORG Subject: Re: cvs commit: src/sys/conf files src/sys/sys random.h src/sys/dev/randomdev hash.c hash.h harvest.c randomdev.c yarrow.c yarro In-Reply-To: <20000912145255.A41113@cs.waikato.ac.nz> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > At Tandem, we used semaphores exclusively (well, we had a mutex > > instruction, but it was really interrupt lockout). As far as I can > > recall, the semaphore counter was always 1, so the effect was > > identical to the current blocking "mutexes". > > I liked the model Sun chose for Solaris. They have mutex', rw_locks, > condition variables. I don't like semaphores. Mutexes are for short > locks. Condition variables are for long-term waits, they are associated > with a mutex. You can only sleep/wakeup a CV when holding the associated > with it, which prevents races. When having to sleep on a CV the kernel > would unlock the mutex and reaquire it for the running thread before > returning. It encouraged very sloppy programming. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 23:45:29 2000 Delivered-To: freebsd-arch@freebsd.org Received: from cs.waikato.ac.nz (taupo.cs.waikato.ac.nz [130.217.248.134]) by hub.freebsd.org (Postfix) with ESMTP id F3D9D37B423; Mon, 11 Sep 2000 23:45:24 -0700 (PDT) Received: (from joerg@localhost) by cs.waikato.ac.nz (8.9.3/8.9.3) id SAA70493; Tue, 12 Sep 2000 18:45:11 +1200 (NZST) (envelope-from joerg) Date: Tue, 12 Sep 2000 18:45:11 +1200 From: Joerg Micheel To: Michael Schuster - Sun Germany Cc: Greg Lehey , Matthew Jacob , Frank Mayhar , John Baldwin , Mark Murray , FreeBSD-arch@freebsd.org, joerg@cs.waikato.ac.nz Subject: Re: Mutexes and semaphores (was: cvs commit: src/sys/conf files src/sys/sys random.h src/sys/dev/randomdev hash.c hash.h harvest.c randomdev.c yarrow.c yarro) Message-ID: <20000912184511.C70000@cs.waikato.ac.nz> References: <200009120101.e8C11nN56928@realtime.exit.com> <20000912121105.J88615@wantadilla.lemis.com> <20000912145255.A41113@cs.waikato.ac.nz> <20000912123114.K88615@wantadilla.lemis.com> <20000912162506.C41113@cs.waikato.ac.nz> <39BDCDB2.B50BBB52@germany.sun.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <39BDCDB2.B50BBB52@germany.sun.com>; from michael.schuster@germany.sun.com on Tue, Sep 12, 2000 at 08:31:14AM +0200 Organization: Dept of Computer Science, University of Waikato, Hamilton, New Zealand Project: WAND - Waikato Applied Network Dynamics, DAG Operating-System: ... powered by FreeBSD Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, Sep 12, 2000 at 08:31:14AM +0200, Michael Schuster - Sun Germany wrote: > well, (in Solaris) there is a difference, mainly in implementation, which > affects debugability (is that a correct word? :-) amongs others: Mutexes > have ownership whereas semaphores don't. This makes detection of recursive > mutex enter (which is not allowed in Solaris, I'm happy to say) and illegal > freeing of mutexes trivial. Also, semaphores are multi-valued, and > therefore lend themselves to some uses mutexes aren't meant for (I think > the dining philosophers were a classical example of that). Ok, a semaphore could have an initial value of 2 and being entered by 2 threads before blocking would occur. This really is a academic case, you can implement it with more essential mechanisms - mutexes. Your definition of a semaphore seems to suggest an implementation detail. For a mutex, it is only important if it is held by someone or not. For a semaphore there is additional information on who holds it. It would allow recursive enter of the same lock already being held - something useful at times for getting the locking code right (with multiple locks and data dependencies). It smells somewhat of a bug - harder to implement and debug and perhaps unnecessary. I think the goal is to define the minimal set of primitives and get it right - fast, efficient, cheap, whatever. > Performancewise, mutexes are fastest, as they're implemented directly on > top of (Sparc's) CAS instruction (in the simple case). Exactly. Joerg -- Joerg B. Micheel Email: Waikato Applied Network Dynamics Phone: +64 7 8384794 The University of Waikato, CompScience Fax: +64 7 8585095 Private Bag 3105 Pager: +64 868 38222 Hamilton, New Zealand Plan: TINE and the DAG's To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 23:49:29 2000 Delivered-To: freebsd-arch@freebsd.org Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.80]) by hub.freebsd.org (Postfix) with ESMTP id 3A8CE37B43C for ; Mon, 11 Sep 2000 23:49:24 -0700 (PDT) Received: (from grog@localhost) by wantadilla.lemis.com (8.11.0/8.9.3) id e8C6mv124109; Tue, 12 Sep 2000 16:18:57 +0930 (CST) (envelope-from grog) Date: Tue, 12 Sep 2000 16:18:57 +0930 From: Greg Lehey To: Mark Murray Cc: Jason Evans , John Baldwin , FreeBSD-arch@FreeBSD.org Subject: Re: cvs commit: src/sys/conf files src/sys/sys random.h src/sys/dev/randomdev hash.c hash.h harvest.c randomdev.c yarrow.c yarro Message-ID: <20000912161857.B23948@wantadilla.lemis.com> References: <20000911171819.A31089@blitz.canonware.com> <200009120601.e8C611X75116@grimreaper.grondar.za> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <200009120601.e8C611X75116@grimreaper.grondar.za>; from mark@grondar.za on Tue, Sep 12, 2000 at 08:01:00AM +0200 Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tuesday, 12 September 2000 at 8:01:00 +0200, Mark Murray wrote: >> A general discussion of high level locking strategy is in order, but in my >> opinion, Mark's code is not an appropriate harbinger, since there's >> absolutely nothing wrong with the approach he took, regardless of the >> results of high level discussion. The issue we need to decide on is how to >> avoid bad interactions (read deadlock/livelock) between mutex hierarchies, >> but the randomdev mutex basically stands alone, so isn't subject to those >> discussions. > > I kinda winged it on this one, but I had the assistance of folk online. > It would be good to have some hardcore documentation. Indeed. I'm thinking of doing something, but don't mistake thinking for promising :-) Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Sep 11 23:49:54 2000 Delivered-To: freebsd-arch@freebsd.org Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.80]) by hub.freebsd.org (Postfix) with ESMTP id BF00E37B424; Mon, 11 Sep 2000 23:49:50 -0700 (PDT) Received: (from grog@localhost) by wantadilla.lemis.com (8.11.0/8.9.3) id e8C6nT624128; Tue, 12 Sep 2000 16:19:29 +0930 (CST) (envelope-from grog) Date: Tue, 12 Sep 2000 16:19:29 +0930 From: Greg Lehey To: Matthew Jacob Cc: Joerg Micheel , Frank Mayhar , John Baldwin , Mark Murray , FreeBSD-arch@FreeBSD.ORG Subject: Re: cvs commit: src/sys/conf files src/sys/sys random.h src/sys/dev/randomdev hash.c hash.h harvest.c randomdev.c yarrow.c yarro Message-ID: <20000912161928.C23948@wantadilla.lemis.com> References: <20000912145255.A41113@cs.waikato.ac.nz> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from mjacob@feral.com on Mon, Sep 11, 2000 at 11:34:55PM -0700 Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Monday, 11 September 2000 at 23:34:55 -0700, Matt Jacob wrote: >>> At Tandem, we used semaphores exclusively (well, we had a mutex >>> instruction, but it was really interrupt lockout). As far as I can >>> recall, the semaphore counter was always 1, so the effect was >>> identical to the current blocking "mutexes". >> >> I liked the model Sun chose for Solaris. They have mutex', rw_locks, >> condition variables. I don't like semaphores. Mutexes are for short >> locks. Condition variables are for long-term waits, they are associated >> with a mutex. You can only sleep/wakeup a CV when holding the associated >> with it, which prevents races. When having to sleep on a CV the kernel >> would unlock the mutex and reaquire it for the running thread before >> returning. > > It encouraged very sloppy programming. That begs the question "can you elaborate?". Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 0:29:12 2000 Delivered-To: freebsd-arch@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id 10E2137B422; Tue, 12 Sep 2000 00:29:09 -0700 (PDT) Received: from beppo.feral.com (beppo [192.67.166.79]) by feral.com (8.9.3/8.9.3) with ESMTP id AAA13939; Tue, 12 Sep 2000 00:28:43 -0700 Date: Tue, 12 Sep 2000 00:28:24 -0700 (PDT) From: Matthew Jacob Reply-To: mjacob@feral.com To: Greg Lehey Cc: Joerg Micheel , Frank Mayhar , John Baldwin , Mark Murray , FreeBSD-arch@FreeBSD.ORG Subject: Re: cvs commit: src/sys/conf files src/sys/sys random.h src/sys/dev/randomdev hash.c hash.h harvest.c randomdev.c yarrow.c yarro In-Reply-To: <20000912161928.C23948@wantadilla.lemis.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > >> I liked the model Sun chose for Solaris. They have mutex', rw_locks, > >> condition variables. I don't like semaphores. Mutexes are for short > >> locks. Condition variables are for long-term waits, they are associated > >> with a mutex. You can only sleep/wakeup a CV when holding the associated > >> with it, which prevents races. When having to sleep on a CV the kernel > >> would unlock the mutex and reaquire it for the running thread before > >> returning. > > > > It encouraged very sloppy programming. > > That begs the question "can you elaborate?". Sure. Rather than think about how locks and locking and interactions with code flow can affect system performance, a plethora of r/w locks and lock upgrades and downgrades, and cv signals *and* broadcasts, and so on and so and so on were made available. As a consequence, the original 1988 internal paper which described the architecture took until at least 1996 or so (about Solaris 2.6) until one could say that performance was back to decent levels. Some of the performance buyback was at least 3 distinct re-implementations of locks, but some was just a lot of steady cleanup of code over time. The point here is that a plethora of different locks fuzzes the fundamental issue which is how to write code that doesn't step on itself and doesn't have really peculiar performance affect. In contrast to Sun, Kubota had a spinlock in h/w with a 30us timeout which would blow the machine up if exceeded. The spinlock was either used as a mutex on data structures, or within a semaphore (to guard the count field) to gate threads into kernel code segments. Everything that could be done inside the Solaris kernel with literally an order of magnitude more locking primitives was able to be successfully done in the Kubota kernel (including, I may say, processor affinity and interrupt threads, which Solaris took years to get even close to implementing). This was with a fairly standard early SVr4 based kernel with some BSD additions (I worked mostly on IPI, SCSI and some buffer cache delwri cleanups). "There's nothing that increases the zest for living so much as the imminence of death". There's nothing that increases respect for SMP more than having the machine blow up on you because you didn't think the code flow through- you don't grab a lock and wait for an external device to do something. Bad Things Happen(tm). I believe that, *so far*, FreeBSD is taking a very good middle ground approach between these two extremes. In particular, the witness code approach from BSDi is avoiding the absolute chaotic nightmare that the three way clustercoitus that streams mux modules, non-recursive Mutexes (which is what Solaris locks are) and the 'unsafe_driver' global mutex to allow for non-SMP ready modules to exist with SMP-ready modules made of Solaris. A mechanism of locks in addition to something close the current sleep/wakeup mechanism should be sufficient. Please don't allow 'creeping featurism/comp sci 101' in. Rather than add in reader/writer locks- make the people asking for them *really* justify why they need the expense of a locking mechanism that will be around for years. What problem does it solve? Can the problem be solved by redoing the subsystem in question to be more SMP friendly? IMO, etc... I'm not trying to start a flame war here. I think everyone working on this stuff gets extra special credits. -matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 1:28:11 2000 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id C881237B43E for ; Tue, 12 Sep 2000 01:28:08 -0700 (PDT) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id e8C8S1J17679; Tue, 12 Sep 2000 01:28:01 -0700 (PDT) Date: Tue, 12 Sep 2000 01:28:01 -0700 From: Alfred Perlstein To: Daniel Eischen Cc: Jason Evans , Greg Lehey , arch@FreeBSD.ORG Subject: Re: Long-term mutex ownership (was Re: Interruptable mutex aquires.) Message-ID: <20000912012801.G12231@fw.wintelcom.net> References: <20000911192610.D12231@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i In-Reply-To: ; from eischen@vigrid.com on Mon, Sep 11, 2000 at 11:07:56PM -0400 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Daniel Eischen [000911 20:08] wrote: > On Mon, 11 Sep 2000, Alfred Perlstein wrote: > > * Jason Evans [000911 19:24] wrote: > > > On Tue, Sep 12, 2000 at 11:41:54AM +0930, Greg Lehey wrote: > > > > I think we need to come to some kind of consensus about how we are > > > > going to structure locking before we go into this much detail. At the > > > > moment we don't even agree whether we can hold on to (blocking) > > > > mutexes for long periods of time. > > > > > > I don't recall the original argument against holding mutexes for long > > > periods. From an abstract point of view, there's nothing wrong with such > > > practice, and in fact it makes sense for many problems. Is there an issue > > > with our implementation? If so, can someone please explain it? > > > > You are currently unable to abort if a signal comes in. :) > > > > Conditional variables add overhead and complexity where it's > > not needed. > > On the contrary. Our current mutex scheme with all the flags is > not as simple and as easy to understand as other OS kernel mutexes > I've dealt with. Having flags to mtx_enter() makes it more error > prone and I'd rather just use the mutex as it was mtx_init()'d. > I don't even want recursive mutexes and think that simple mutexes > with separate count fields should be used (I would probably lose > this argument though). > > Condition variables are used when you plan on waiting for something, > like I/O completion. You can wake one or all threads waiting on > condition variables. This is not typical of mutexes. Our current > splXXX()/tsleep()/splx() scheme corresponds very nicely with > mtx_enter()/cv_wait()/mtx_exit(). I don't recommend obfuscating > mutexes even more by creating yet another variant to be used > in place of tsleep() or cv_wait(). It's amazing what a rum and coke can do for one's concentration, you guys are right and I wish someone had looked at the code I was trying to work with, shook me and said "Alfred, you dumbass!" but then again I only have myself to blame here. :) The socketbuffer stuff is a prime example of what needs conditional variables and I didn't realize I was rolling my own using flags and msleep. *sigh* :) I am going to grep the Linux kernel and see if they have any examples of where an interruptable mutex is needed, but for now I don't need it and conditional variables would seem to be a nice thing. -Alfred To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 2: 7:38 2000 Delivered-To: freebsd-arch@freebsd.org Received: from peach.ocn.ne.jp (peach.ocn.ne.jp [210.145.254.87]) by hub.freebsd.org (Postfix) with ESMTP id 0A74237B423 for ; Tue, 12 Sep 2000 02:07:36 -0700 (PDT) Received: from newsguy.com (p19-dn03kiryunisiki.gunma.ocn.ne.jp [210.232.224.148]) by peach.ocn.ne.jp (8.9.1a/OCN/) with ESMTP id SAA04145; Tue, 12 Sep 2000 18:06:52 +0900 (JST) Message-ID: <39BDF208.2A9F65D0@newsguy.com> Date: Tue, 12 Sep 2000 18:06:16 +0900 From: "Daniel C. Sobral" X-Mailer: Mozilla 4.7 [en] (Win98; I) X-Accept-Language: en,pt-BR MIME-Version: 1.0 To: Joerg Micheel Cc: John Baldwin , Greg Lehey , Chuck Paterson , Jason Evans , arch@FreeBSD.ORG Subject: Re: Long-term mutex ownership (was Re: Interruptable mutex aquires.) References: <20000912143855.O88615@wantadilla.lemis.com> <200009120515.WAA78736@pike.osd.bsdi.com> <20000912175725.A70000@cs.waikato.ac.nz> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Joerg Micheel wrote: > > > Go grab Andrew S. Tannebaum's (sp?) _Modern_Operating_Systems_. One of > > the first chapters gives execellent treatment to comparing/contrasting > > mutexes, semaphores, condition variables, and sleep/wakeup. > > ... and once you are done with it find out that it is all the same > just offering different views onto something that can be implemented > and used in exactly the same way. I've just gone through this - trying > to explain it to 239 second years students - I can't see any difference. Really, the difference between a mutex and a semaphore is simple. Mutex is mutual exclusion. It serves solely to garantee exclusive access to something for some time. Semaphore, which was originally described like we describe mutex today, has become a rendez-vous mechanism. It controls resource availability. It can be used either to "consume" resources, of which the mutex is the trivial case, or used to signal resource availability, which is more akin to a cv. In the first case, you expect to enter, and freeze if someone is in (it's an n-exclusion, not mutual exclusion, problem, though). The the latter case, you expect to freeze, and enter when something has been produced. OTOH, I'm not going to defend the difference between cv and sleep/wakeup. :) -- Daniel C. Sobral (8-DCS) dcs@newsguy.com dcs@freebsd.org capo@white.bunnies.bsdconspiracy.net OK, so the solar flares are my fault.. I am sorry, ok?!?! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 3:34:42 2000 Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 443C837B423; Tue, 12 Sep 2000 03:34:38 -0700 (PDT) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id MAA21588; Tue, 12 Sep 2000 12:34:33 +0200 (CEST) (envelope-from des@ofug.org) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: mjacob@feral.com Cc: Marius Bendiksen , Mike Smith , Poul-Henning Kamp , arch@FreeBSD.ORG, smp@FreeBSD.ORG Subject: Re: Device probing... References: From: Dag-Erling Smorgrav Date: 12 Sep 2000 12:34:32 +0200 In-Reply-To: Matthew Jacob's message of "Mon, 11 Sep 2000 16:55:21 -0700 (PDT)" Message-ID: Lines: 9 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Matthew Jacob writes: > Good idea, except it's hard to debug unless you have a working ddb or gdb or > something (which is very rare these days for the machines I usually work on). Make the message spooler conditional on DEBUG not being defined. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 4:26:57 2000 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 2CDBE37B422 for ; Tue, 12 Sep 2000 04:26:55 -0700 (PDT) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id e8CBQsN21789 for arch@freebsd.org; Tue, 12 Sep 2000 04:26:54 -0700 (PDT) Date: Tue, 12 Sep 2000 04:26:54 -0700 From: Alfred Perlstein To: arch@freebsd.org Subject: what to do with softinterrupts? Message-ID: <20000912042654.L12231@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG There's a problem with softinterrupts: we have only one. I'm trying to figure out what to do with the situation because unless I can schedule more than one concurrantly I can't really test much of what I'm working on, there's also making the input path concurrant and responsive. After thinking about it for some time I came up with a solution for network drivers. The idea is to keep several spare interrupt threads available for the network interrupts. Instead of ether_input() figuring out the type of packet and schduling a soft interrupt it will: Swap out its repsonsibility with another spare thread to handle the interface's hardware interrupt Perform a callback to the driver to re-enable interrupts Jump directly into what is now a pseudo-soft-interrupt This will happen as long as it can reserve a spare thread, and we're not pre-empting a softinterrupt running(*) in the same protocol on the same CPU. Otherwise it simply queues and schedules like before. (*) running == not blocked on a mutex, if it's blocked on a mutex we allow another jump from hardware to software to gain concurrancy. When the softinterrupt is complete it will then recheck its input queue for mbufs, if there is none it will return to it's state as an idle interrupt thread making itself free for consumption to service other devices or stacks. From my point of view this will give: better concurrancy, by making the stack less likely to stall because of blocking while accessing potentially locked objects less context switching because there's no need to schedule a soft interrupt unless we are seriously bogged down Poul-Henning Kamp thought that investigating atomic ops and keeping most of the current scheme of things would also work. We'd have to have a softinterrupt bound to each CPU, but as long as they could proceed without blocking we'd probably be ok. The problem I see with this is that it sort of tries to get around the whole interrupt thread idea by going back to non-blocking interrupts instead of taking the advantage (or performance hit) of the threads system we have in place. I'm not dead set on either idea, but I wanted to know what other solutions people could come up with to address this problem, any takers? any existing solutions? thanks, -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 4:34: 4 2000 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 183DA37B422 for ; Tue, 12 Sep 2000 04:34:02 -0700 (PDT) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.0/8.9.3) with ESMTP id e8CBXvN73660; Tue, 12 Sep 2000 13:33:57 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Alfred Perlstein Cc: arch@FreeBSD.ORG Subject: Re: what to do with softinterrupts? In-Reply-To: Your message of "Tue, 12 Sep 2000 04:26:54 PDT." <20000912042654.L12231@fw.wintelcom.net> Date: Tue, 12 Sep 2000 13:33:56 +0200 Message-ID: <73658.968758436@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <20000912042654.L12231@fw.wintelcom.net>, Alfred Perlstein writes: >Poul-Henning Kamp thought that investigating atomic ops and >keeping most of the current scheme of things would also work. Well, that's not *quite* what I meant. First we should exploit atomic ops all the places we can. Atomics are cheaper than anything we can ever do with threads. What we do next is the good question. I think something like the lazy-switch we (intend to) do for hw-interrupts sounds promising: foo_input() runs as far as it can without blocking, when it needs to block, it does so in a another thread and continues input processing for other packets. We could either have one thread per PCB socket standby for this use, but that's probably a waste of threads, or we can snatch them from a pool of threads cached for that purpose. I seriously don't think we can decide which way to go until we have basically tried all the candidates in practice. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD coreteam member | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 7:14:19 2000 Delivered-To: freebsd-arch@freebsd.org Received: from urban.iinet.net.au (urban.iinet.net.au [203.59.24.231]) by hub.freebsd.org (Postfix) with ESMTP id CFA4537B443 for ; Tue, 12 Sep 2000 07:14:14 -0700 (PDT) Received: from muzak.iinet.net.au (muzak.iinet.net.au [203.59.24.237]) by urban.iinet.net.au (8.8.7/8.8.7) with ESMTP id WAA31508; Tue, 12 Sep 2000 22:14:10 +0800 Received: from jules.elischer.org ([203.59.169.234]) by muzak.iinet.net.au (8.8.5/8.8.5) with SMTP id WAA28846; Tue, 12 Sep 2000 22:14:06 +0800 Message-ID: <39BE3A2A.167EB0E7@elischer.org> Date: Tue, 12 Sep 2000 07:14:02 -0700 From: Julian Elischer X-Mailer: Mozilla 3.04Gold (X11; I; FreeBSD 5.0-CURRENT i386) MIME-Version: 1.0 To: Poul-Henning Kamp Cc: Alfred Perlstein , arch@FreeBSD.ORG Subject: Re: what to do with softinterrupts? References: <73658.968758436@critter> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Poul-Henning Kamp wrote: > > > > I think something like the lazy-switch we (intend to) do for > hw-interrupts sounds promising: foo_input() runs as far as it can > without blocking, when it needs to block, it does so in a another > thread and continues input processing for other packets. this is for soft interrupts, right? > > We could either have one thread per PCB socket standby for > this use, but that's probably a waste of threads, or we can > snatch them from a pool of threads cached for that purpose. the second is probably enough. but each "subsystem" may need to pre-declare how many threads it wants in its pool. > > I seriously don't think we can decide which way to go until we > have basically tried all the candidates in practice. > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD coreteam member | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message -- __--_|\ Julian Elischer / \ julian@elischer.org ( OZ ) World tour 2000 ---> X_.---._/ presently in: Perth v To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 7:17:22 2000 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id B3F2237B424 for ; Tue, 12 Sep 2000 07:17:19 -0700 (PDT) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.0/8.9.3) with ESMTP id e8CEHBN78539; Tue, 12 Sep 2000 16:17:11 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Julian Elischer Cc: Alfred Perlstein , arch@FreeBSD.ORG Subject: Re: what to do with softinterrupts? In-Reply-To: Your message of "Tue, 12 Sep 2000 07:14:02 PDT." <39BE3A2A.167EB0E7@elischer.org> Date: Tue, 12 Sep 2000 16:17:11 +0200 Message-ID: <78537.968768231@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <39BE3A2A.167EB0E7@elischer.org>, Julian Elischer writes: >Poul-Henning Kamp wrote: >> >> I think something like the lazy-switch we (intend to) do for >> hw-interrupts sounds promising: foo_input() runs as far as it can >> without blocking, when it needs to block, it does so in a another >> thread and continues input processing for other packets. > >this is for soft interrupts, right? See subject :-) >> We could either have one thread per PCB socket standby for >> this use, but that's probably a waste of threads, or we can >> snatch them from a pool of threads cached for that purpose. > >the second is probably enough. >but each "subsystem" may need to pre-declare how many threads >it wants in its pool. Hmm, well uhm. I really like things which dimension themselves, so if we can avoid more "maxuser" like junk if possible I would really like it. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD coreteam member | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 7:31: 9 2000 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 42AA137B423; Tue, 12 Sep 2000 07:31:02 -0700 (PDT) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.0/8.9.3) with ESMTP id e8CEV1N78948; Tue, 12 Sep 2000 16:31:01 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: smp@freebsd.org, arch@freebsd.org Subject: system initialization order. From: Poul-Henning Kamp Date: Tue, 12 Sep 2000 16:31:01 +0200 Message-ID: <78946.968769061@critter> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG With SMPng I think we need to take a good hard stare at the order in which we initialize the system, a lot of the reasons behind the current order are invalid, and some new reasons for a new order are not honoured. Roughly speaking, I think we need something like this order: init console print copyright initialize VM/malloc(9) init other stuff needed for: setup proc0 setup proc1 (park it on a semaphore for now) setup idle procs enable scheduler init hardclock enable hardclock interrupt initialize timecounters This should now represent a sufficiently "normal" environment that the order of the rest doesn't really matter very much: init network protocols init pseudodrivers probe/attach real drivers ata/scsi delayed probing needs thought about here. locate & mount root let proc1 loose Comments ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD coreteam member | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 8: 7:58 2000 Delivered-To: freebsd-arch@freebsd.org Received: from gidora.zeta.org.au (gidora.zeta.org.au [203.26.10.25]) by hub.freebsd.org (Postfix) with SMTP id 793AE37B422 for ; Tue, 12 Sep 2000 08:07:53 -0700 (PDT) Received: (qmail 24516 invoked from network); 12 Sep 2000 15:07:49 -0000 Received: from unknown (HELO bde.zeta.org.au) (203.2.228.102) by gidora.zeta.org.au with SMTP; 12 Sep 2000 15:07:49 -0000 Date: Wed, 13 Sep 2000 02:07:47 +1100 (EST) From: Bruce Evans X-Sender: bde@besplex.bde.org To: Poul-Henning Kamp Cc: smp@FreeBSD.ORG, arch@FreeBSD.ORG Subject: Re: system initialization order. In-Reply-To: <78946.968769061@critter> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, 12 Sep 2000, Poul-Henning Kamp wrote: > With SMPng I think we need to take a good hard stare at the > order in which we initialize the system, a lot of the reasons > behind the current order are invalid, and some new reasons for > a new order are not honoured. > > Roughly speaking, I think we need something like this order: > > init console > print copyright > initialize VM/malloc(9) > init other stuff needed for: > setup proc0 > setup proc1 (park it on a semaphore for now) > setup idle procs > enable scheduler > init hardclock > enable hardclock interrupt Should be softclock (scheduler doesn't use hardclock). > initialize timecounters > > This should now represent a sufficiently "normal" environment that > the order of the rest doesn't really matter very much: I think this mainly moves clock initialization earlier. OK with me. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 8: 8:27 2000 Delivered-To: freebsd-arch@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id 5CC1A37B424; Tue, 12 Sep 2000 08:08:23 -0700 (PDT) Received: from beppo.feral.com (beppo [192.67.166.79]) by feral.com (8.9.3/8.9.3) with ESMTP id IAA15394; Tue, 12 Sep 2000 08:08:16 -0700 Date: Tue, 12 Sep 2000 08:07:55 -0700 (PDT) From: Matthew Jacob Reply-To: mjacob@feral.com To: Dag-Erling Smorgrav Cc: Marius Bendiksen , Mike Smith , Poul-Henning Kamp , arch@FreeBSD.ORG, smp@FreeBSD.ORG Subject: Re: Device probing... In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG One could do this. On 12 Sep 2000, Dag-Erling Smorgrav wrote: > Matthew Jacob writes: > > Good idea, except it's hard to debug unless you have a working ddb or gdb or > > something (which is very rare these days for the machines I usually work on). > > Make the message spooler conditional on DEBUG not being defined. > > DES > -- > Dag-Erling Smorgrav - des@ofug.org > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 8:15: 9 2000 Delivered-To: freebsd-arch@freebsd.org Received: from berserker.bsdi.com (berserker.twistedbit.com [199.79.183.1]) by hub.freebsd.org (Postfix) with ESMTP id C801037B424 for ; Tue, 12 Sep 2000 08:15:07 -0700 (PDT) Received: from berserker.bsdi.com (cp@localhost [127.0.0.1]) by berserker.bsdi.com (8.9.3/8.9.3) with ESMTP id JAA29329; Tue, 12 Sep 2000 09:15:04 -0600 (MDT) Message-Id: <200009121515.JAA29329@berserker.bsdi.com> To: Alfred Perlstein Cc: arch@freebsd.org Subject: Re: what to do with softinterrupts? From: Chuck Paterson Date: Tue, 12 Sep 2000 09:15:04 -0600 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Before you can run soft interrupts on multiple processors I believe there is a stuff in the tcp/ip stack that can't be fixed by simple locking. For instance ip_srcrt and tcp_saveipgen come to mind. Chuck To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 8:21:55 2000 Delivered-To: freebsd-arch@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id 9491037B424; Tue, 12 Sep 2000 08:21:49 -0700 (PDT) Received: from beppo.feral.com (beppo [192.67.166.79]) by feral.com (8.9.3/8.9.3) with ESMTP id IAA15455; Tue, 12 Sep 2000 08:21:49 -0700 Date: Tue, 12 Sep 2000 08:21:29 -0700 (PDT) From: Matthew Jacob Reply-To: mjacob@feral.com To: Poul-Henning Kamp Cc: smp@FreeBSD.ORG, arch@FreeBSD.ORG Subject: Re: system initialization order. In-Reply-To: <78946.968769061@critter> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > With SMPng I think we need to take a good hard stare at the > order in which we initialize the system, a lot of the reasons > behind the current order are invalid, and some new reasons for > a new order are not honoured. > > Roughly speaking, I think we need something like this order: > > init console You can't init a real console w/o h/w. This has been the awful grotesque ugliness of the at least the alpha NetBSD && FreeBSD ports. Despite what Mike said in a different context, I have to say you will init a console early- using some kind of boot or throwaway driver- possibly even just a buffer to store output, until the real console can be initialized after, in *proper* order, the h/w for the real console is there. > print copyright > initialize VM/malloc(9) > init other stuff needed for: > setup proc0 > setup proc1 (park it on a semaphore for now) > setup idle procs > enable scheduler > init hardclock > enable hardclock interrupt > initialize timecounters > Fine, fine... > This should now represent a sufficiently "normal" environment that > the order of the rest doesn't really matter very much: > > init network protocols > init pseudodrivers In some cases pseudo drivers depend on real drivers, don't they? > probe/attach real drivers > > ata/scsi delayed probing needs thought about here. Yes. SCSI/FC more than ATA. Some thought might be given to avoiding complete evaluation here and doing what Solaris does instead which would allow you to parse a boot path and try and guess which specific device you need to get initialized. You don't to wait for 200 disks to probe if you can help it. > > locate & mount root > let proc1 loose > > Comments ? > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD coreteam member | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 9:11:25 2000 Delivered-To: freebsd-arch@freebsd.org Received: from gidora.zeta.org.au (gidora.zeta.org.au [203.26.10.25]) by hub.freebsd.org (Postfix) with SMTP id 1824737B423 for ; Tue, 12 Sep 2000 09:11:19 -0700 (PDT) Received: (qmail 29081 invoked from network); 12 Sep 2000 16:11:15 -0000 Received: from unknown (HELO bde.zeta.org.au) (203.2.228.102) by gidora.zeta.org.au with SMTP; 12 Sep 2000 16:11:15 -0000 Date: Wed, 13 Sep 2000 03:11:12 +1100 (EST) From: Bruce Evans X-Sender: bde@besplex.bde.org To: Poul-Henning Kamp Cc: smp@FreeBSD.ORG, arch@FreeBSD.ORG Subject: Re: system initialization order. In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Wed, 13 Sep 2000, Bruce Evans wrote: > On Tue, 12 Sep 2000, Poul-Henning Kamp wrote: > >... > > init hardclock > > enable hardclock interrupt > > Should be softclock (scheduler doesn't use hardclock). Oops, statclock. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 9:16:50 2000 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 6E4B937B42C; Tue, 12 Sep 2000 09:16:44 -0700 (PDT) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.0/8.9.3) with ESMTP id e8CGGVN82449; Tue, 12 Sep 2000 18:16:31 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Bruce Evans Cc: smp@FreeBSD.ORG, arch@FreeBSD.ORG Subject: Re: system initialization order. In-Reply-To: Your message of "Wed, 13 Sep 2000 03:11:12 +1100." Date: Tue, 12 Sep 2000 18:16:31 +0200 Message-ID: <82447.968775391@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message , Bruce Evan s writes: >On Wed, 13 Sep 2000, Bruce Evans wrote: > >> On Tue, 12 Sep 2000, Poul-Henning Kamp wrote: >> >... >> > init hardclock >> > enable hardclock interrupt >> >> Should be softclock (scheduler doesn't use hardclock). > >Oops, statclock. Uhm, the point was to get the timeouts working, not to enable forced rescheduling... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD coreteam member | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 10:44:16 2000 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id B327037B424 for ; Tue, 12 Sep 2000 10:44:14 -0700 (PDT) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id e8CHi8001567; Tue, 12 Sep 2000 10:44:08 -0700 (PDT) Date: Tue, 12 Sep 2000 10:44:08 -0700 From: Alfred Perlstein To: Chuck Paterson Cc: arch@freebsd.org Subject: Re: what to do with softinterrupts? Message-ID: <20000912104408.M12231@fw.wintelcom.net> References: <200009121515.JAA29329@berserker.bsdi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i In-Reply-To: <200009121515.JAA29329@berserker.bsdi.com>; from cp@bsdi.com on Tue, Sep 12, 2000 at 09:15:04AM -0600 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Chuck Paterson [000912 08:15] wrote: > > Before you can run soft interrupts on multiple processors > I believe there is a stuff in the tcp/ip stack that can't be fixed > by simple locking. For instance ip_srcrt and tcp_saveipgen come > to mind. At a glance they just seem to be globals that ought to wind uo on the stack and passed down through the protocol levels/functions. I understand that they contain state that must be saved for the packet processing run thought ip_input->tcp_input. I do need to study the interaction more carefully, but is this not the case? -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 11:11:52 2000 Delivered-To: freebsd-arch@freebsd.org Received: from berserker.bsdi.com (berserker.twistedbit.com [199.79.183.1]) by hub.freebsd.org (Postfix) with ESMTP id 1E74437B42C for ; Tue, 12 Sep 2000 11:11:50 -0700 (PDT) Received: from berserker.bsdi.com (cp@localhost [127.0.0.1]) by berserker.bsdi.com (8.9.3/8.9.3) with ESMTP id MAA00974; Tue, 12 Sep 2000 12:11:43 -0600 (MDT) Message-Id: <200009121811.MAA00974@berserker.bsdi.com> To: Alfred Perlstein Cc: arch@freebsd.org Subject: Re: what to do with softinterrupts? From: Chuck Paterson Date: Tue, 12 Sep 2000 12:11:43 -0600 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Alfred Perlstein wrote on: Tue, 12 Sep 2000 10:44:08 PDT }At a glance they just seem to be globals that ought to wind uo on }the stack and passed down through the protocol levels/functions. }I understand that they contain state that must be saved for the }packet processing run thought ip_input->tcp_input. I do need to }study the interaction more carefully, but is this not the case? } I believe that you are fundamentally correct. My specific knowledge is limited to noting the existance of these globals. Chuck To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 11:34:58 2000 Delivered-To: freebsd-arch@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id C438237B42C for ; Tue, 12 Sep 2000 11:34:53 -0700 (PDT) Received: (qmail 2178 invoked by uid 1142); 12 Sep 2000 18:34:52 -0000 Date: 12 Sep 2000 11:34:52 -0700 Date: Tue, 12 Sep 2000 11:34:45 -0700 From: Jason Evans To: Matthew Jacob Cc: arch@FreeBSD.ORG Subject: Necessary synchronization primitives (was Re: cvs commit: [...] yarrow.c [...]) Message-ID: <20000912113445.F31089@blitz.canonware.com> References: <20000912161928.C23948@wantadilla.lemis.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: ; from mjacob@feral.com on Tue, Sep 12, 2000 at 12:28:24AM -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, Sep 12, 2000 at 12:28:24AM -0700, Matthew Jacob wrote: > I believe that, *so far*, FreeBSD is taking a very good middle ground approach > between these two extremes. In particular, the witness code approach from BSDi > is avoiding the absolute chaotic nightmare that the three way clustercoitus > that streams mux modules, non-recursive Mutexes (which is what Solaris locks > are) and the 'unsafe_driver' global mutex to allow for non-SMP ready modules > to exist with SMP-ready modules made of Solaris. > > Please don't allow 'creeping featurism/comp sci 101' in. Rather than add > in reader/writer locks- make the people asking for them *really* justify why > they need the expense of a locking mechanism that will be around for years. > What problem does it solve? Can the problem be solved by redoing the subsystem > in question to be more SMP friendly? Executive summary: My experience has indicated that 1) mutexes, 2) condition variables, 3) barriers, and 4) message queues are an adequate set of locking primitives for almost any problem. I've been drooling over threads since being introduced to OS/2 in 1992. I actually started using threads in significant ways in about 1996. The last 5 years have taught me a few lessons about what is useful versus what sounds good on paper. To make it clear where this email is going, I'll start off by saying "reader/writer locks are rarely useful", and later on will add support to that opinion. Here's a laundry list of synchronization primitives, in approximately increasing order of complexity: * mutex {blocking, spinning, adaptive, recursive} Simple mutual exclusion lock, in many flavors. Only one thread can own a mutex at a time. - blocking If a thread tries to acquire a mutex that is already owned, the thread will block until it is granted the mutex (i.e. the previous owner releases the mutex). - spinning If a thread tries to acquire a mutex that is already owned, the threads spins in a tight loop, trying again and again to acquire the mutex until it succeeds. Spin mutexes tend to be very dangerous and difficult to use correctly. In userland, spinning mutexes are almost always a bad idea (though not *always*). In the kernel, and in threading library implementations, there are decidedly legitimate reasons for using spinning mutexes. - adaptive A spinning mutex that blocks after a certain amount of spinning. In userland programming, these pretty much remove any need for pure spinning mutexes. - recursive A thread can acquire the same mutex more than once, recursively. Until the SMP work, I never used recursive mutexes. In my opinion, if recursive mutexes are needed to solve a programming problem, then the problem needs to be re-thought. That is, recursive mutexes are a crutch, not a necessity. However, given that we're converting an old piece of code to being threaded, there are a number of places where recursive mutexes save us from having to rip major portions of the kernel to pieces. Hopefully we can keep recursive mutex use to a minimum, but from a pragmatic point of view, we need them, at least for now. * condition variable The name is pretty descriptive. There are two basic operations on a condition variable: waiting and signaling. A thread can wait for a condition to occur, and another thread can signal that the condition has occurred (or broadcast, in order to wake all threads waiting on the condition). Condition variables are useful for waiting for state changes to occur. Note that the recently added msleep() is for all practical purposes a condition variable implementation. The rest of the primitives can be constructed using mutexes and condition variables as building blocks. * Dijkstra semaphore I've gone through my library (including Knuth and various other books) and the best I've been able to determine is that Dijkstra semaphores have two operations: P (passeren: Dutch for "to pass") and V (vrijgeven: Dutch for "to give free"). It is unclear to me given the references at hand whether Dijkstra semaphores are generalized enough to include a count like counting semaphores (described below), but I suspect so. Perhaps Greg can shed some light on that. * (counting) semaphore A number is associated with a semaphore, and the two main operations are to post (increment) and wait (decrement). Posting always succeeds (somewhat analogous to unlocking a mutex), but waiting will cause a thread to block if the decrement operation would cause the semaphore value to go below zero. Semaphores can be degenerately used as a mutex, by constraining the value of the semaphore to always be 0 or 1. One major difference though is that semaphores can be waited on, then posted, by entirely different threads, whereas a mutex must be locked, then unlocked, by the same thread. Semaphores can also be degenerately used as a condition variable, by waiting until another thread posts (signals) that the condition has occurred. * reader/writer lock Multiple readers can simultaneously own read locks, but only one writer (and no readers) can own a write lock at a time. The idea behind reader/writer locks is to allow concurrent access to a data structure that has many potential readers, in cases where the data structure is rarely modified. * barrier Multiple threads can wait on a barrier, and they all wait until a predetermined threshold (number of waiters) is reached, at which time the barrier breaks (the waiting threads can all resume running). * message queue Messages can be written and read. Message queues are useful for decoupling complex subsystems, in that they conceptually create an asynchronous link between separate state machines. ----------- I've personally found mutexes, condition variables, barriers, and message queues to be useful. In reality, I've emulated barriers with semaphores, which is the only thing I've used semaphores for, but in retrospect, a barrier primitive would have been a better way to go. That leaves out semaphores and reader/writer locks. As mentioned above, semaphores can be degenerately used as mutexes or condition variables, but semaphores can also be avoided by using mutexes and condition variables in combination. The omission of reader/writer locks requires a more in-depth explanation. In fact, I have used reader/writer locks. However, in every case where I've done so, later analysis of the code has led me to conclude either that: * The additional overhead of reader/writer locks as opposed to plain mutexes has not been worth it. That is, reader/writer locks were not increasing parallelism enough to compensate for the fact that they're more expensive to use. This is an important point; people (including me) tend to try to optimize a problem by using reader/writer locks, when in reality it usually harms performance. * The problem should have been solved in a different way that did not involve the need for reader/writer locks. There's one situation where I've used a form of reader/writer locks: in combination with a job queue. In that case, a job queue consisted of jobs that needed to either read or write a shared data structure. Only one write job could be dispatched at a time, but all contiguous read jobs could be dispatched in parallel (I say contiguous, because in this case, ordering of operations was important). So, this was a situation in which reader/writer locking made sense, but it was in a context that a reader/writer lock primitive was useless. There may be places where a reader/writer locking primitive truly makes sense, but I've never seen one. Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 12:54:43 2000 Delivered-To: freebsd-arch@freebsd.org Received: from pcnet1.pcnet.com (pcnet1.pcnet.com [204.213.232.3]) by hub.freebsd.org (Postfix) with ESMTP id 89C9A37B423 for ; Tue, 12 Sep 2000 12:54:38 -0700 (PDT) Received: (from eischen@localhost) by pcnet1.pcnet.com (8.8.7/PCNet) id PAA27821; Tue, 12 Sep 2000 15:54:17 -0400 (EDT) Date: Tue, 12 Sep 2000 15:54:17 -0400 (EDT) From: Daniel Eischen To: Jason Evans Cc: Matthew Jacob , arch@FreeBSD.ORG Subject: Re: Necessary synchronization primitives (was Re: cvs commit: [...] yarrow.c [...]) In-Reply-To: <20000912113445.F31089@blitz.canonware.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 12 Sep 2000, Jason Evans wrote: > Executive summary: My experience has indicated that 1) mutexes, 2) > condition variables, 3) barriers, and 4) message queues are an adequate set > of locking primitives for almost any problem. > > I've been drooling over threads since being introduced to OS/2 in 1992. I > actually started using threads in significant ways in about 1996. The last > 5 years have taught me a few lessons about what is useful versus what > sounds good on paper. To make it clear where this email is going, I'll > start off by saying "reader/writer locks are rarely useful", and later on > will add support to that opinion. > > Here's a laundry list of synchronization primitives, in approximately > increasing order of complexity: > > * mutex {blocking, spinning, adaptive, recursive} > > Simple mutual exclusion lock, in many flavors. Only one thread can own a > mutex at a time. > > - blocking > > If a thread tries to acquire a mutex that is already owned, the thread > will block until it is granted the mutex (i.e. the previous owner > releases the mutex). > > - spinning > > If a thread tries to acquire a mutex that is already owned, the threads > spins in a tight loop, trying again and again to acquire the mutex > until it succeeds. Spin mutexes tend to be very dangerous and > difficult to use correctly. In userland, spinning mutexes are almost > always a bad idea (though not *always*). In the kernel, and in > threading library implementations, there are decidedly legitimate > reasons for using spinning mutexes. > > - adaptive > > A spinning mutex that blocks after a certain amount of spinning. In > userland programming, these pretty much remove any need for pure > spinning mutexes. > > - recursive > > A thread can acquire the same mutex more than once, recursively. Until > the SMP work, I never used recursive mutexes. In my opinion, if > recursive mutexes are needed to solve a programming problem, then the > problem needs to be re-thought. That is, recursive mutexes are a > crutch, not a necessity. However, given that we're converting an old > piece of code to being threaded, there are a number of places where > recursive mutexes save us from having to rip major portions of the > kernel to pieces. Hopefully we can keep recursive mutex use to a > minimum, but from a pragmatic point of view, we need them, at least > for now. I agree with everything you've said here, elsewhere in this Email, and in recommending the Butenhof book. I also want to add that I hate recursive mutexes :-) Perhaps now is not the right time, but I'd like to see our kernel mutex (mtx) not support recursve mutexes. We could provide a different set of functions (and perhaps a different data structure) that would be optimized explicitly for recursive mutexes (still discouraging their use). This would hopefully help us in eliminating the hideous flags needed for mtx_enter(), instead relying on the flags provided at mtx_init() time. -- Dan Eischen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 14:19:56 2000 Delivered-To: freebsd-arch@freebsd.org Received: from peach.ocn.ne.jp (peach.ocn.ne.jp [210.145.254.87]) by hub.freebsd.org (Postfix) with ESMTP id 95DC137B424 for ; Tue, 12 Sep 2000 14:19:53 -0700 (PDT) Received: from newsguy.com (p21-dn01kiryunisiki.gunma.ocn.ne.jp [211.0.245.22]) by peach.ocn.ne.jp (8.9.1a/OCN/) with ESMTP id GAA18570; Wed, 13 Sep 2000 06:19:47 +0900 (JST) Message-ID: <39BE9DCE.857D2310@newsguy.com> Date: Wed, 13 Sep 2000 06:19:10 +0900 From: "Daniel C. Sobral" X-Mailer: Mozilla 4.7 [en] (Win98; I) X-Accept-Language: en,pt-BR MIME-Version: 1.0 To: Jason Evans Cc: Matthew Jacob , arch@FreeBSD.ORG Subject: Re: Necessary synchronization primitives (was Re: cvs commit: [...] yarrow.c [...]) References: <20000912161928.C23948@wantadilla.lemis.com> <20000912113445.F31089@blitz.canonware.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Jason Evans wrote: > > * Dijkstra semaphore > > I've gone through my library (including Knuth and various other books) > and the best I've been able to determine is that Dijkstra semaphores have > two operations: P (passeren: Dutch for "to pass") and V (vrijgeven: Dutch > for "to give free"). It is unclear to me given the references at hand > whether Dijkstra semaphores are generalized enough to include a count > like counting semaphores (described below), but I suspect so. Perhaps > Greg can shed some light on that. The original Dijkstra algorithm is a mutex, pure and simple. The counting semaphores comes from elsewhere. -- Daniel C. Sobral (8-DCS) dcs@newsguy.com dcs@freebsd.org capo@white.bunnies.bsdconspiracy.net OK, so the solar flares are my fault.. I am sorry, ok?!?! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 17: 4:47 2000 Delivered-To: freebsd-arch@freebsd.org Received: from winston.osd.bsdi.com (winston.osd.bsdi.com [204.216.27.229]) by hub.freebsd.org (Postfix) with ESMTP id 6DEE437B43C for ; Tue, 12 Sep 2000 17:04:45 -0700 (PDT) Received: from winston.osd.bsdi.com (jkh@localhost [127.0.0.1]) by winston.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id e8D04bU85918; Tue, 12 Sep 2000 17:04:38 -0700 (PDT) (envelope-from jkh@winston.osd.bsdi.com) To: Shaun Jurrens Cc: arch@FreeBSD.ORG Subject: Re: sysinstall specification (was: build tools as...) In-Reply-To: Message from Shaun Jurrens of "Fri, 01 Sep 2000 02:40:57 +0200." <20000901024057.B3659@dakota.priv.shamz.net> Date: Tue, 12 Sep 2000 17:04:37 -0700 Message-ID: <85914.968803477@winston.osd.bsdi.com> From: Jordan Hubbard Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > A generally recognized method of solving complex problems is to > subdivide them into manageable parts. Modularity in programming, > design, and problem solving have enough aspects in common, that this > little treatise might help. My suggestion is simply that those > "fathers" of the current sysinstaller make the design of a new > sysinstaller simpler to those not possessing the years of experience > with the pitfalls of the gradual "monster" now known as sysinstall A good suggestion, and one I've just taken to heart in a 5000+ word document I just posted to hackers@freebsd.org and cc'd to committers@freebsd.org. If others would like to see it here and/or would like a personal copy, just let me know - I'm loath to post such a large document to too many places and arch has always had a somewhat limited readership. - Jordan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 17:31:31 2000 Delivered-To: freebsd-arch@freebsd.org Received: from Awfulhak.org (tun.AwfulHak.org [194.242.139.173]) by hub.freebsd.org (Postfix) with ESMTP id E11C737B424; Tue, 12 Sep 2000 17:31:13 -0700 (PDT) Received: from hak.lan.Awfulhak.org (root@hak.lan.awfulhak.org [172.16.0.12]) by Awfulhak.org (8.11.0/8.11.0) with ESMTP id e8D0ROv08469; Wed, 13 Sep 2000 01:27:24 +0100 (BST) (envelope-from brian@hak.lan.Awfulhak.org) Received: from hak.lan.Awfulhak.org (brian@localhost [127.0.0.1]) by hak.lan.Awfulhak.org (8.11.0/8.11.0) with ESMTP id e8D0RrH91664; Wed, 13 Sep 2000 01:27:53 +0100 (BST) (envelope-from brian@hak.lan.Awfulhak.org) Message-Id: <200009130027.e8D0RrH91664@hak.lan.Awfulhak.org> X-Mailer: exmh version 2.1.1 10/15/1999 To: Joerg Micheel Cc: Greg Lehey , Matthew Jacob , Frank Mayhar , John Baldwin , Mark Murray , FreeBSD-arch@FreeBSD.org, brian@Awfulhak.org Subject: Re: Mutexes and semaphores (was: cvs commit: src/sys/conf files src/sys/sys random.h src/sys/dev/randomdev hash.c hash.h harvest.c randomdev.c yarrow.c yarro) In-Reply-To: Message from Joerg Micheel of "Tue, 12 Sep 2000 16:25:06 +1200." <20000912162506.C41113@cs.waikato.ac.nz> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 13 Sep 2000 01:27:53 +0100 From: Brian Somers Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > What's the difference between a mutex and a semaphore? > > None, if it comes to the actual usage and implementation. Usually, > people will use semaphores for locking blocks of code, mutexes for > data structures. Not much of a difference in the end result, but > locking data structures seems more natural to me. I would tend to disagree. A mutex is a way of guaranteeing that a resource isn't used more than once. A semaphore is a mechanism for queuing a notification to a (potentially) waiting process. A mutex is just a binary semaphore, but I don't think that's relevant. -- Brian Don't _EVER_ lose your sense of humour ! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 18: 8:21 2000 Delivered-To: freebsd-arch@freebsd.org Received: from mail-relay.eunet.no (mail-relay.eunet.no [193.71.71.242]) by hub.freebsd.org (Postfix) with ESMTP id 7255A37B43E; Tue, 12 Sep 2000 18:08:17 -0700 (PDT) Received: from login-1.eunet.no (login-1.eunet.no [193.75.110.2]) by mail-relay.eunet.no (8.9.3/8.9.3/GN) with ESMTP id DAA55937; Wed, 13 Sep 2000 03:08:15 +0200 (CEST) (envelope-from mbendiks@eunet.no) Received: from localhost (mbendiks@localhost) by login-1.eunet.no (8.9.3/8.8.8) with ESMTP id DAA85526; Wed, 13 Sep 2000 03:08:15 +0200 (CEST) (envelope-from mbendiks@eunet.no) X-Authentication-Warning: login-1.eunet.no: mbendiks owned process doing -bs Date: Wed, 13 Sep 2000 03:08:15 +0200 (CEST) From: Marius Bendiksen To: Dag-Erling Smorgrav Cc: arch@FreeBSD.ORG, smp@FreeBSD.ORG Subject: Re: Device probing... In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > Good idea, except it's hard to debug unless you have a working ddb or gdb or > > something (which is very rare these days for the machines I usually work on). > Make the message spooler conditional on DEBUG not being defined. Rather, I think, make the buffering and sorting code conditional. Use the spooler anyway, but have it act mostly as a printf() passthrough when DEBUG is defined. Marius To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 22: 2:32 2000 Delivered-To: freebsd-arch@freebsd.org Received: from winston.osd.bsdi.com (winston.osd.bsdi.com [204.216.27.229]) by hub.freebsd.org (Postfix) with ESMTP id 9716F37B42C for ; Tue, 12 Sep 2000 22:02:30 -0700 (PDT) Received: from winston.osd.bsdi.com (jkh@localhost [127.0.0.1]) by winston.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id e8D51nU11498; Tue, 12 Sep 2000 22:01:49 -0700 (PDT) (envelope-from jkh@winston.osd.bsdi.com) To: "Francisco Reyes" Cc: arch@freebsd.org Subject: Re: sysinstall specification (was: build tools as...) In-Reply-To: Message from "Francisco Reyes" of "Wed, 13 Sep 2000 00:17:21 EDT." <200009130448.AAA32757@sanson.reyes.somos.net> Date: Tue, 12 Sep 2000 22:01:49 -0700 Message-ID: <11494.968821309@winston.osd.bsdi.com> From: Jordan Hubbard Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > >A good suggestion, and one I've just taken to heart in a 5000+ word > >document I just posted to hackers@freebsd.org and cc'd to > >committers@freebsd.org. If others would like to see it here and/or > >would like a personal copy, just let me know - I'm loath to post such > >a large document to too many places and arch has always had a somewhat > >limited readership. > > Why not put it on a web page? A fine idea - please find the latest version at: http://people.freebsd.org/~jkh/package-and-install.txt - Jordan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Sep 12 22:15:26 2000 Delivered-To: freebsd-arch@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (Postfix) with ESMTP id AFD0A37B422; Tue, 12 Sep 2000 22:15:22 -0700 (PDT) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.9.3/8.9.3) with ESMTP id XAA63052; Tue, 12 Sep 2000 23:15:21 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id XAA85122; Tue, 12 Sep 2000 23:14:54 -0600 (MDT) Message-Id: <200009130514.XAA85122@harmony.village.org> To: Poul-Henning Kamp Subject: Re: system initialization order. Cc: smp@FreeBSD.ORG, arch@FreeBSD.ORG In-reply-to: Your message of "Tue, 12 Sep 2000 16:31:01 +0200." <78946.968769061@critter> References: <78946.968769061@critter> Date: Tue, 12 Sep 2000 23:14:54 -0600 From: Warner Losh Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <78946.968769061@critter> Poul-Henning Kamp writes: : probe/attach real drivers : : ata/scsi delayed probing needs thought about here. It would be highly desirable to allow for pccard/cardbus cards to become active here. : locate & mount root : let proc1 loose : : Comments ? It would be desirable if there's nothing to mount root on, that additional choices can become available. For example, if you are being prompted for the root device a card can be plugged in while you are waiting... It would be nice if the new system allows for this. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Sep 13 0:45:22 2000 Delivered-To: freebsd-arch@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (Postfix) with ESMTP id 30F0337B42C; Wed, 13 Sep 2000 00:45:19 -0700 (PDT) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.9.3/8.9.3) with ESMTP id BAA63430; Wed, 13 Sep 2000 01:45:15 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id BAA86075; Wed, 13 Sep 2000 01:44:45 -0600 (MDT) Message-Id: <200009130744.BAA86075@harmony.village.org> To: Joerg Micheel Subject: Re: cvs commit: src/sys/conf files src/sys/sys random.h src/sys/dev/randomdev hash.c hash.h harvest.c randomdev.c yarrow.c yarro Cc: Greg Lehey , Matthew Jacob , Frank Mayhar , John Baldwin , Mark Murray , FreeBSD-arch@FreeBSD.ORG In-reply-to: Your message of "Tue, 12 Sep 2000 14:52:55 +1200." <20000912145255.A41113@cs.waikato.ac.nz> References: <20000912145255.A41113@cs.waikato.ac.nz> <200009120101.e8C11nN56928@realtime.exit.com> <20000912121105.J88615@wantadilla.lemis.com> Date: Wed, 13 Sep 2000 01:44:45 -0600 From: Warner Losh Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <20000912145255.A41113@cs.waikato.ac.nz> Joerg Micheel writes: : I liked the model Sun chose for Solaris. They have mutex', rw_locks, : condition variables. I don't like semaphores. Mutexes are for short : locks. Condition variables are for long-term waits, they are associated : with a mutex. You can only sleep/wakeup a CV when holding the associated : with it, which prevents races. When having to sleep on a CV the kernel : would unlock the mutex and reaquire it for the running thread before : returning. I've found that these are about as easy to use as spl/splx. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Sep 13 8:23:49 2000 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 4735037B42C for ; Wed, 13 Sep 2000 08:23:36 -0700 (PDT) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.9.3/8.9.3) with SMTP id LAA32757 for ; Wed, 13 Sep 2000 11:23:35 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Wed, 13 Sep 2000 11:23:35 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: freebsd-arch@FreeBSD.org Subject: thread related papers, et al. Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Some concern has been expressed, and I admit to expressing this concern myself, that we may be rushing headlong into threading the kernel without thinking through it very much, and an eager desire to plop mutexes wherever we can put them. So far, not problems, but it's a problem worth keeping in mind, especially since there seems to be a lot of confusion about the types of synchronization primitives and how to use them correctly (and efficiently). I thought I'd post a couple of paper references that might be interesting for people to look at if this is a topic that interests them. I think it would also be very helpful if, once the basic framework for SMPng is finished (interrupt threads, synchronization primitives, etc), we could assemble a guide for threaded kernel programming, including a combination of suggestions, guidelines, and directives for the correct approaches to take in the kernel. A number of existing kernel programmers may not have direct or extensive experience with kernel programming, and helping to short-cut the learning curve would probably be much appreciated, both by them and by the consumers of FreeBSD :-). Birrell89: Andrew Birrell. An introduction to programming with threads. DEC technical report TR-35, DEC/SRC, January, 1989. Hauser93: Carl Hauser, Christian Jacobi, Marvin Theimer, Brent Welch, Mark Weiser. Using threads in interactive systems: A case study. ACM Symp. on Operating Systems Principles (SOSP-14), December 1993. Savage97: Stefan Savage, Michael Burrows, Greg Nelson, Patrick Sobalvarro, Thomas Anderson. Eraser: A Dynamic Data Race Detector for Multi-Threaded Programs. ACM Symp. on Operating Systems Principles (SOSP-16), October 1997. The last is pretty researchy, and as such is probably not something you want to consider a directive on how to program, but it's worth learning from their experiences, and understanding both the problems they need to deal with, and solutions they have in mind (as well as the limitations of that solution). Darren Reed also recently pointed me at a Solaris kernel programmers glossary online that I found very useful to read, to see how others have done this. It's probably already been posted, but the URL is: http://docs.sun.com:80/ab2/coll.40.6/REFMAN9F/@Ab2TocView?Ab2Lang=C&Ab2Enc=iso-8859-1 One priority Darren has expressed is getting the lock manager working properly with SMPng, as mutexes don't provide the semantics he needs in terms of read vs. write locks; this seems like a reasonable viewpoint, and possibly it's just a question of cleaning up the lock manager a little, and wrapping locks in mutexes (to a first approximation). That said, the Solaris lock calls seem a lot cleaner, so maybe we should adopt something like their APIs and semantics, which would also give Solaris kernel module programmers a starting point if they want to port third party modules to FreeBSD :-). Robert N M Watson robert@fledge.watson.org http://www.watson.org/~robert/ PGP key fingerprint: AF B5 5F FF A6 4A 79 37 ED 5F 55 E9 58 04 6A B1 TIS Labs at Network Associates, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Sep 13 8:47:20 2000 Delivered-To: freebsd-arch@freebsd.org Received: from berserker.bsdi.com (berserker.twistedbit.com [199.79.183.1]) by hub.freebsd.org (Postfix) with ESMTP id 1A3CA37B423; Wed, 13 Sep 2000 08:47:18 -0700 (PDT) Received: from berserker.bsdi.com (cp@localhost [127.0.0.1]) by berserker.bsdi.com (8.9.3/8.9.3) with ESMTP id JAA09262; Wed, 13 Sep 2000 09:47:17 -0600 (MDT) Message-Id: <200009131547.JAA09262@berserker.bsdi.com> To: Robert Watson Cc: freebsd-arch@freebsd.org Subject: Re: thread related papers, et al. From: Chuck Paterson Date: Wed, 13 Sep 2000 09:47:17 -0600 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG The BSD/OS lockmanager code is on builder. The changes are straight forward. A few minutes staring at it should make it all clear. I do agree this needs to be done right away. Until it is done biodone must be called holding the Giant lock, and therefore device drivers can't really be made mp safe. Chuck } }One priority Darren has expressed is getting the lock manager working }properly with SMPng, as mutexes don't provide the semantics he needs in }terms of read vs. write locks; this seems like a reasonable viewpoint, and }possibly it's just a question of cleaning up the lock manager a little, }and wrapping locks in mutexes (to a first approximation). That said, the }Solaris lock calls seem a lot cleaner, so maybe we should adopt something }like their APIs and semantics, which would also give Solaris kernel module }programmers a starting point if they want to port third party modules to }FreeBSD :-). } } Robert N M Watson To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Sep 13 13:29:50 2000 Delivered-To: freebsd-arch@freebsd.org Received: from wall.polstra.com (rtrwan160.accessone.com [206.213.115.74]) by hub.freebsd.org (Postfix) with ESMTP id B547D37B443; Wed, 13 Sep 2000 13:29:32 -0700 (PDT) Received: from vashon.polstra.com (vashon.polstra.com [206.213.73.13]) by wall.polstra.com (8.9.3/8.9.3) with ESMTP id NAA28901; Wed, 13 Sep 2000 13:29:31 -0700 (PDT) (envelope-from jdp@polstra.com) From: John Polstra Received: (from jdp@localhost) by vashon.polstra.com (8.9.3/8.9.1) id NAA00620; Wed, 13 Sep 2000 13:29:31 -0700 (PDT) (envelope-from jdp@polstra.com) Date: Wed, 13 Sep 2000 13:29:31 -0700 (PDT) Message-Id: <200009132029.NAA00620@vashon.polstra.com> To: arch@FreeBSD.ORG Reply-To: arch@FreeBSD.ORG Cc: rwatson@FreeBSD.ORG Subject: Re: thread related papers, et al. In-Reply-To: References: Organization: Polstra & Co., Seattle, WA Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In article , Robert Watson wrote: > > I thought I'd post a couple of paper references that might be interesting > for people to look at if this is a topic that interests them. [...] > Birrell89: Andrew Birrell. An introduction to programming with threads. > DEC technical report TR-35, DEC/SRC, January, 1989. I'm a big fan of this paper, so here's the URL to make it easier for folks to get it: http://gatekeeper.dec.com/pub/DEC/SRC/research-reports/abstracts/src-rr-035.html John -- John Polstra jdp@polstra.com John D. Polstra & Co., Inc. Seattle, Washington USA "Disappointment is a good sign of basic intelligence." -- Chögyam Trungpa To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Sep 14 12:15:58 2000 Delivered-To: freebsd-arch@freebsd.org Received: from segfault.kiev.ua (segfault.kiev.ua [193.193.193.4]) by hub.freebsd.org (Postfix) with ESMTP id EA20437B422 for ; Thu, 14 Sep 2000 12:15:54 -0700 (PDT) Received: (from netch@localhost) by segfault.kiev.ua (8) id WFY83868; Thu, 14 Sep 2000 22:15:29 +0300 (EEST) (envelope-from netch) Date: Thu, 14 Sep 2000 22:15:28 +0300 From: Valentin Nechayev To: Wes Peters Cc: freebsd-arch@freebsd.org Subject: Re: Request for review: nsswitch Message-ID: <20000914221528.B66058@netch.kiev.ua> Reply-To: netch@segfault.kiev.ua References: <200009050320.UAA04806@usr02.primenet.com> <39B48B6E.158195C4@softweyr.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <39B48B6E.158195C4@softweyr.com>; from wes@softweyr.com on Tue, Sep 05, 2000 at 05:58:00AM +0000 X-42: On Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Hello Wes Peters! Tue, Sep 05, 2000 at 05:58:00, wes wrote about "Re: Request for review: nsswitch": > One of the UNIX systems I've used over the years, probably SunOS, allowed > you to add the name of a host as a {sym,}link to rlogin; the executable > would check argv[0] and if it wasn't a recognized pattern try it as a > hostname. The common usage was to add links to your favorite hosts in > /hosts/name and add that to your PATH. > > My vote would be to add this feature to ssh. I consider this feature as evil. /netch To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Sep 14 12:30:50 2000 Delivered-To: freebsd-arch@freebsd.org Received: from soda.csua.Berkeley.edu (soda.CSUA.Berkeley.EDU [128.32.43.52]) by hub.freebsd.org (Postfix) with ESMTP id BC74537B424 for ; Thu, 14 Sep 2000 12:30:48 -0700 (PDT) Received: from soda.csua.Berkeley.edu (localhost [127.0.0.1]) by soda.csua.Berkeley.edu (8.8.8/) via ESMTP id MAA20323 for ; Thu, 14 Sep 2000 12:30:47 -0700 (PDT) env-from (ranga@CSUA.Berkeley.EDU) Message-Id: <200009141930.MAA20323@soda.csua.Berkeley.edu> To: freebsd-arch@FreeBSD.ORG Subject: Re: Request for review: nsswitch In-Reply-To: Your message of "Thu, 14 Sep 2000 22:15:28 +0300." <20000914221528.B66058@netch.kiev.ua> Date: Thu, 14 Sep 2000 12:30:47 -0700 From: Sriranga Veeraraghavan Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > One of the UNIX systems I've used over the years, probably SunOS, > > allowed you to add the name of a host as a {sym,}link to rlogin; > > the executable would check argv[0] and if it wasn't a recognized > > pattern try it as a hostname. The common usage was to add links > > to your favorite hosts in /hosts/name and add that to your PATH. > > > > My vote would be to add this feature to ssh. > > I consider this feature as evil. Hi, I don't think it is a good idea to add such a feature (of dubious benefit to most users) to a program like ssh. If you really want something like this create a script like the following in $HOME/bin: #!/bin/sh exec ssh `basename "$0"` $@ and symlink it to the hostnames you want. Then add $HOME/bin to PATH and you have basically the same functionality. ----ranga To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 9:23:36 2000 Delivered-To: freebsd-arch@freebsd.org Received: from androcles.com (androcles.com [204.57.240.10]) by hub.freebsd.org (Postfix) with ESMTP id D178237B43C for ; Fri, 15 Sep 2000 09:23:29 -0700 (PDT) Received: (from dhh@localhost) by androcles.com (8.9.3/8.9.3) id JAA87065; Fri, 15 Sep 2000 09:23:08 -0700 (PDT) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit MIME-Version: 1.0 In-Reply-To: <20000914221528.B66058@netch.kiev.ua> Date: Fri, 15 Sep 2000 09:23:08 -0700 (PDT) From: "Duane H. Hesser" To: Valentin Nechayev Subject: Re: Request for review: nsswitch Cc: freebsd-arch@FreeBSD.ORG, Wes Peters Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG It was a feature of rsh and rlogin in 4.2BSD, and persists to this day. Check your sources for rsh.c and rlogin.c. Please explain why it is `evil'. On 14-Sep-00 Valentin Nechayev wrote: > Hello Wes Peters! > > Tue, Sep 05, 2000 at 05:58:00, wes wrote about "Re: Request for review: nsswitch": > >> One of the UNIX systems I've used over the years, probably SunOS, allowed >> you to add the name of a host as a {sym,}link to rlogin; the executable >> would check argv[0] and if it wasn't a recognized pattern try it as a >> hostname. The common usage was to add links to your favorite hosts in >> /hosts/name and add that to your PATH. >> >> My vote would be to add this feature to ssh. > > I consider this feature as evil. > > > /netch > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > -------------- Duane H. Hesser dhh@androcles.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 9:27: 0 2000 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id DF45E37B423 for ; Fri, 15 Sep 2000 09:26:57 -0700 (PDT) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.9.3/8.9.3) with SMTP id MAA45840 for ; Fri, 15 Sep 2000 12:26:56 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Fri, 15 Sep 2000 12:26:56 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: freebsd-arch@FreeBSD.org Subject: sysctl MIB and kernel internals Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG As I'm sure you've frequently experienced, getting your userland and kernel out of synch can break all kinds of things, including common utilities such as ps. One reason for this is a tight coupling between the internal implementation structures in the kernel, and the exported management interface to userland. For example, sysctl(3) exposes, effectively verbatim, a direct concatenation of struct proc, the related credential and process structures, et al. This is a problem, as any trivial change to those structures, even one unrelated to the data visible to userland, will change the MIB interface breaking any compiled binaries relying on it. Also, it means exposing many kernel interneals throughout userland. For example, as part of the capalities support described in my BSDcon paper, I added a reference to struct cap in struct ucred, creating a dependency on sys/capability.h if struct ucred is to be used. The result was that the following userland source files had to be fixed with the added include: RCS file: /home/ncvs/src/contrib/amd/include/am_defs.h,v RCS file: /home/ncvs/src/include/rpcsvc/bootparam_prot.x,v RCS file: /home/ncvs/src/lib/libc/gen/getmntinfo.c,v RCS file: /home/ncvs/src/lib/libkvm/kvm_getswapinfo.c,v RCS file: /home/ncvs/src/usr.bin/find/function.c,v RCS file: /home/ncvs/src/usr.bin/kdump/Makefile,v RCS file: /home/ncvs/src/usr.sbin/inetd/builtins.c,v RCS file: /home/ncvs/src/usr.sbin/pstat/pstat.c,v RCS file: /home/ncvs/src/usr.sbin/rpc.umntall/rpc.umntall.c,v So there are several problems relating to this. I'd like to see us consider moving to an alternative model, divorcing the implementation internals of various kernel objects (processes, et al) from the MIB interface retrieving management data about them. I.e., struct proc would continue to be used in kernel, but relevant fields would be copied to struct export_proc for export via sysctl. In addition, it would be worth prefixing these exported structures with a version number allowing the caller to determine if they support an appropriate version of the interface, allowing a more comprehensible error. Only fields desirable to export would be in export_proc, so if an extra pointer is added to struct ucred (recent resource control changes, capabilities), an extra pointer to struct proc (jail), etc won't needless break userland tools. This would also have the effect of allowing us to document the MIB, rather than just say, ``err. go see the kernel source''. Just a thought. Robert N M Watson robert@fledge.watson.org http://www.watson.org/~robert/ PGP key fingerprint: AF B5 5F FF A6 4A 79 37 ED 5F 55 E9 58 04 6A B1 TIS Labs at Network Associates, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 9:36:37 2000 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 6D77937B424; Fri, 15 Sep 2000 09:36:34 -0700 (PDT) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.0/8.9.3) with ESMTP id e8FGaXN64487; Fri, 15 Sep 2000 18:36:33 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Robert Watson Cc: freebsd-arch@FreeBSD.ORG Subject: Re: sysctl MIB and kernel internals In-Reply-To: Your message of "Fri, 15 Sep 2000 12:26:56 EDT." Date: Fri, 15 Sep 2000 18:36:33 +0200 Message-ID: <64485.969035793@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message , Robe rt Watson writes: >I'd like to see us consider moving to an alternative model, divorcing the >implementation internals of various kernel objects (processes, et al) from >the MIB interface retrieving management data about them. Yes (please!) -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD coreteam member | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 10:21:50 2000 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 67EFA37B424; Fri, 15 Sep 2000 10:21:48 -0700 (PDT) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id e8FHLjx03896; Fri, 15 Sep 2000 10:21:45 -0700 (PDT) Date: Fri, 15 Sep 2000 10:21:45 -0700 From: Alfred Perlstein To: Robert Watson Cc: freebsd-arch@FreeBSD.ORG Subject: Re: sysctl MIB and kernel internals Message-ID: <20000915102144.E12231@fw.wintelcom.net> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i In-Reply-To: ; from rwatson@FreeBSD.ORG on Fri, Sep 15, 2000 at 12:26:56PM -0400 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Robert Watson [000915 09:27] wrote: > > I'd like to see us consider moving to an alternative model, divorcing the > implementation internals of various kernel objects (processes, et al) from > the MIB interface retrieving management data about them. I.e., struct > proc would continue to be used in kernel, but relevant fields would be > copied to struct export_proc for export via sysctl. In addition, it would > be worth prefixing these exported structures with a version number > allowing the caller to determine if they support an appropriate version of > the interface, allowing a more comprehensible error. Only fields > desirable to export would be in export_proc, so if an extra pointer is > added to struct ucred (recent resource control changes, capabilities), an > extra pointer to struct proc (jail), etc won't needless break userland > tools. > Look at what NetBSD did: http://www.netbsd.org/Changes/#new_sysctls -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 12:25:33 2000 Delivered-To: freebsd-arch@freebsd.org Received: from radon.gryphonsoft.com (mcut-b-167.resnet.purdue.edu [128.211.209.167]) by hub.freebsd.org (Postfix) with ESMTP id 4998D37B424 for ; Fri, 15 Sep 2000 12:25:29 -0700 (PDT) Received: by radon.gryphonsoft.com (Postfix, from userid 1000) id B2B961AAB; Fri, 15 Sep 2000 14:23:05 -0500 (EST) Date: Fri, 15 Sep 2000 14:23:05 -0500 From: Will Andrews To: "Duane H. Hesser" Cc: Valentin Nechayev , freebsd-arch@FreeBSD.ORG, Wes Peters Subject: Re: Request for review: nsswitch Message-ID: <20000915142305.L40658@radon.gryphonsoft.com> Reply-To: Will Andrews References: <20000914221528.B66058@netch.kiev.ua> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from dhh@androcles.com on Fri, Sep 15, 2000 at 09:23:08AM -0700 X-Operating-System: FreeBSD 4.1-STABLE i386 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, Sep 15, 2000 at 09:23:08AM -0700, Duane H. Hesser wrote: > It was a feature of rsh and rlogin in 4.2BSD, and persists to this day. > Check your sources for rsh.c and rlogin.c. > > Please explain why it is `evil'. I don't necessarily think it's `evil', but rather an unneeded feature. As someone else demonstrated, its functionality can be equated with a really simple shell script. I would prefer to go against "creeping featurism" in this case and ask that this feature not be merged into ssh from rsh. Rsh is evil and I don't give a flying rat's patootie about it. -- Will Andrews GCS/E/S @d- s+:+ a--- C++ UB++++$ P+ L- E--- W+ N-- !o ?K w--- O- M+ V- PS+ PE++ Y+ PGP+>+++ t++ 5 X+ R+ tv+ b++ DI+++ D+ G++ e>++++ h! r- y? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 12:37:43 2000 Delivered-To: freebsd-arch@freebsd.org Received: from radon.gryphonsoft.com (mcut-b-167.resnet.purdue.edu [128.211.209.167]) by hub.freebsd.org (Postfix) with ESMTP id 03EFB37B423 for ; Fri, 15 Sep 2000 12:37:41 -0700 (PDT) Received: by radon.gryphonsoft.com (Postfix, from userid 1000) id 8D4851AAB; Fri, 15 Sep 2000 14:35:15 -0500 (EST) Date: Fri, 15 Sep 2000 14:35:15 -0500 From: Will Andrews To: arch@FreeBSD.org Subject: Rsh/Rlogin/Rcmd & friends Message-ID: <20000915143515.N40658@radon.gryphonsoft.com> Reply-To: Will Andrews Mail-Followup-To: Will Andrews , arch@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i X-Operating-System: FreeBSD 4.1-STABLE i386 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Ok people. I want a reason why we shouldn't move rsh/rlogin/rcmd out of the base system and into ports, now that we can support SSH connections out of the box. And, ahead of the pack of y'all, POLA be damned! -- Will Andrews GCS/E/S @d- s+:+ a--- C++ UB++++$ P+ L- E--- W+ N-- !o ?K w--- O- M+ V- PS+ PE++ Y+ PGP+>+++ t++ 5 X+ R+ tv+ b++ DI+++ D+ G++ e>++++ h! r- y? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 12:52:34 2000 Delivered-To: freebsd-arch@freebsd.org Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.208.78.105]) by hub.freebsd.org (Postfix) with ESMTP id 69E8237B423 for ; Fri, 15 Sep 2000 12:52:33 -0700 (PDT) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.9.3/8.9.3) id MAA75808; Fri, 15 Sep 2000 12:56:10 -0700 (PDT) (envelope-from sgk) From: Steve Kargl Message-Id: <200009151956.MAA75808@troutmask.apl.washington.edu> Subject: Re: Rsh/Rlogin/Rcmd & friends In-Reply-To: <20000915143515.N40658@radon.gryphonsoft.com> from Will Andrews at "Sep 15, 2000 02:35:15 pm" To: Will Andrews Date: Fri, 15 Sep 2000 12:56:10 -0700 (PDT) Cc: arch@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL61 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Will Andrews wrote: > Ok people. I want a reason why we shouldn't move rsh/rlogin/rcmd out of > the base system and into ports, now that we can support SSH connections > out of the box. > > And, ahead of the pack of y'all, POLA be damned! > What are the consequences of your proposal with the use of rdump/rrestore from another (non-FreeBSD) machine into a tape drive equipped FreeBSD box? -- Steve To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 12:56: 3 2000 Delivered-To: freebsd-arch@freebsd.org Received: from radon.gryphonsoft.com (mcut-b-167.resnet.purdue.edu [128.211.209.167]) by hub.freebsd.org (Postfix) with ESMTP id 2752737B423 for ; Fri, 15 Sep 2000 12:56:01 -0700 (PDT) Received: by radon.gryphonsoft.com (Postfix, from userid 1000) id BB14C1AAB; Fri, 15 Sep 2000 14:53:37 -0500 (EST) Date: Fri, 15 Sep 2000 14:53:37 -0500 From: Will Andrews To: Steve Kargl Cc: Will Andrews , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends Message-ID: <20000915145337.Q40658@radon.gryphonsoft.com> Reply-To: Will Andrews Mail-Followup-To: Will Andrews , Steve Kargl , arch@FreeBSD.ORG References: <20000915143515.N40658@radon.gryphonsoft.com> <200009151956.MAA75808@troutmask.apl.washington.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200009151956.MAA75808@troutmask.apl.washington.edu>; from sgk@troutmask.apl.washington.edu on Fri, Sep 15, 2000 at 12:56:10PM -0700 X-Operating-System: FreeBSD 4.1-STABLE i386 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, Sep 15, 2000 at 12:56:10PM -0700, Steve Kargl wrote: > What are the consequences of your proposal with the use of > rdump/rrestore from another (non-FreeBSD) machine into a > tape drive equipped FreeBSD box? What consequences? Remember, we'll still have ports for these things. It only matters as far as new installations go. Post-install operations are unimportant. -- Will Andrews GCS/E/S @d- s+:+ a--- C++ UB++++$ P+ L- E--- W+ N-- !o ?K w--- O- M+ V- PS+ PE++ Y+ PGP+>+++ t++ 5 X+ R+ tv+ b++ DI+++ D+ G++ e>++++ h! r- y? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 13:24:44 2000 Delivered-To: freebsd-arch@freebsd.org Received: from pcnet1.pcnet.com (pcnet1.pcnet.com [204.213.232.3]) by hub.freebsd.org (Postfix) with ESMTP id D0BA537B424 for ; Fri, 15 Sep 2000 13:24:42 -0700 (PDT) Received: (from eischen@localhost) by pcnet1.pcnet.com (8.8.7/PCNet) id QAA25647; Fri, 15 Sep 2000 16:24:23 -0400 (EDT) Date: Fri, 15 Sep 2000 16:24:23 -0400 (EDT) From: Daniel Eischen To: Will Andrews Cc: Steve Kargl , Will Andrews , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends In-Reply-To: <20000915145337.Q40658@radon.gryphonsoft.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, 15 Sep 2000, Will Andrews wrote: > On Fri, Sep 15, 2000 at 12:56:10PM -0700, Steve Kargl wrote: > > What are the consequences of your proposal with the use of > > rdump/rrestore from another (non-FreeBSD) machine into a > > tape drive equipped FreeBSD box? > > What consequences? Remember, we'll still have ports for these things. > It only matters as far as new installations go. Post-install operations > are unimportant. Wrong. If that were true tcsh wouldn't be in the base system today. -- Dan Eischen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 13:34:24 2000 Delivered-To: freebsd-arch@freebsd.org Received: from point.osg.gov.bc.ca (point.osg.gov.bc.ca [142.32.102.44]) by hub.freebsd.org (Postfix) with ESMTP id C970537B424 for ; Fri, 15 Sep 2000 13:34:21 -0700 (PDT) Received: (from daemon@localhost) by point.osg.gov.bc.ca (8.8.7/8.8.8) id NAA07362; Fri, 15 Sep 2000 13:33:49 -0700 Received: from passer.osg.gov.bc.ca(142.32.110.29) via SMTP by point.osg.gov.bc.ca, id smtpda07360; Fri Sep 15 13:33:41 2000 Received: (from uucp@localhost) by passer.osg.gov.bc.ca (8.9.3/8.9.1) id NAA59599; Fri, 15 Sep 2000 13:33:40 -0700 (PDT) Received: from cwsys9.cwsent.com(10.2.2.1), claiming to be "cwsys.cwsent.com" via SMTP by passer9.cwsent.com, id smtpdr59588; Fri Sep 15 13:32:42 2000 Received: (from uucp@localhost) by cwsys.cwsent.com (8.11.0/8.9.1) id e8FKWfC25897; Fri, 15 Sep 2000 13:32:41 -0700 (PDT) Message-Id: <200009152032.e8FKWfC25897@cwsys.cwsent.com> Received: from localhost.cwsent.com(127.0.0.1), claiming to be "cwsys" via SMTP by localhost.cwsent.com, id smtpdH25866; Fri Sep 15 13:31:52 2000 X-Mailer: exmh version 2.1.1 10/15/1999 Reply-To: Cy Schubert - ITSD Open Systems Group From: Cy Schubert - ITSD Open Systems Group X-OS: FreeBSD 4.1-RELEASE X-Sender: cy To: Will Andrews Cc: arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends In-reply-to: Your message of "Fri, 15 Sep 2000 14:35:15 CDT." <20000915143515.N40658@radon.gryphonsoft.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 15 Sep 2000 13:31:52 -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <20000915143515.N40658@radon.gryphonsoft.com>, Will Andrews writes: > Ok people. I want a reason why we shouldn't move rsh/rlogin/rcmd out of > the base system and into ports, now that we can support SSH connections > out of the box. > > And, ahead of the pack of y'all, POLA be damned! Yet another reason why everything, not just ports, should be installed using packages. Then install just what you want. Given that packages for parts of the base system are not here today, I would add telnet and ftp to this list too. They should not be installed by default. SSH only. Other apps that should be in ports are NFS, NIS, sendmail (we already have qmail and postfix in ports), and possibly even inetd itself -- who needs inetd when you've got SSH. If we're going to do it let's do it right the first time. Regards, Phone: (250)387-8437 Cy Schubert Fax: (250)387-5766 Team Leader, Sun/DEC Team Internet: Cy.Schubert@osg.gov.bc.ca Open Systems Group, ITSD, ISTA Province of BC To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 13:38:28 2000 Delivered-To: freebsd-arch@freebsd.org Received: from radon.gryphonsoft.com (mcut-b-167.resnet.purdue.edu [128.211.209.167]) by hub.freebsd.org (Postfix) with ESMTP id 52EDF37B422 for ; Fri, 15 Sep 2000 13:38:26 -0700 (PDT) Received: by radon.gryphonsoft.com (Postfix, from userid 1000) id 2D5921ACC; Fri, 15 Sep 2000 15:36:02 -0500 (EST) Date: Fri, 15 Sep 2000 15:36:02 -0500 From: Will Andrews To: Daniel Eischen Cc: Will Andrews , Steve Kargl , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends Message-ID: <20000915153602.S40658@radon.gryphonsoft.com> Reply-To: Will Andrews Mail-Followup-To: Will Andrews , Daniel Eischen , Steve Kargl , arch@FreeBSD.ORG References: <20000915145337.Q40658@radon.gryphonsoft.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from eischen@vigrid.com on Fri, Sep 15, 2000 at 04:24:23PM -0400 X-Operating-System: FreeBSD 4.1-STABLE i386 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, Sep 15, 2000 at 04:24:23PM -0400, Daniel Eischen wrote: > > What consequences? Remember, we'll still have ports for these things. > > It only matters as far as new installations go. Post-install operations > > are unimportant. > > Wrong. If that were true tcsh wouldn't be in the base system today. You misinterpreted me. I meant in this specific case, post-install operation doesn't matter. People can use ssh to get in the machines to do things rsh/rlogin/rcmd offer. -- Will Andrews GCS/E/S @d- s+:+ a--- C++ UB++++$ P+ L- E--- W+ N-- !o ?K w--- O- M+ V- PS+ PE++ Y+ PGP+>+++ t++ 5 X+ R+ tv+ b++ DI+++ D+ G++ e>++++ h! r- y? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 13:52:24 2000 Delivered-To: freebsd-arch@freebsd.org Received: from point.osg.gov.bc.ca (point.osg.gov.bc.ca [142.32.102.44]) by hub.freebsd.org (Postfix) with ESMTP id 973CF37B423 for ; Fri, 15 Sep 2000 13:52:21 -0700 (PDT) Received: (from daemon@localhost) by point.osg.gov.bc.ca (8.8.7/8.8.8) id NAA07685; Fri, 15 Sep 2000 13:51:16 -0700 Received: from passer.osg.gov.bc.ca(142.32.110.29) via SMTP by point.osg.gov.bc.ca, id smtpda07683; Fri Sep 15 13:51:10 2000 Received: (from uucp@localhost) by passer.osg.gov.bc.ca (8.9.3/8.9.1) id NAA60269; Fri, 15 Sep 2000 13:51:10 -0700 (PDT) Received: from cwsys9.cwsent.com(10.2.2.1), claiming to be "cwsys.cwsent.com" via SMTP by passer9.cwsent.com, id smtpdz60267; Fri Sep 15 13:50:48 2000 Received: (from uucp@localhost) by cwsys.cwsent.com (8.11.0/8.9.1) id e8FKojS25996; Fri, 15 Sep 2000 13:50:45 -0700 (PDT) Message-Id: <200009152050.e8FKojS25996@cwsys.cwsent.com> Received: from localhost.cwsent.com(127.0.0.1), claiming to be "cwsys" via SMTP by localhost.cwsent.com, id smtpdC25989; Fri Sep 15 13:50:02 2000 X-Mailer: exmh version 2.1.1 10/15/1999 Reply-To: Cy Schubert - ITSD Open Systems Group From: Cy Schubert - ITSD Open Systems Group X-OS: FreeBSD 4.1-RELEASE X-Sender: cy To: Will Andrews Cc: Daniel Eischen , Steve Kargl , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends In-reply-to: Your message of "Fri, 15 Sep 2000 15:36:02 CDT." <20000915153602.S40658@radon.gryphonsoft.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 15 Sep 2000 13:50:02 -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <20000915153602.S40658@radon.gryphonsoft.com>, Will Andrews writes: > On Fri, Sep 15, 2000 at 04:24:23PM -0400, Daniel Eischen wrote: > > > What consequences? Remember, we'll still have ports for these things. > > > It only matters as far as new installations go. Post-install operations > > > are unimportant. > > > > Wrong. If that were true tcsh wouldn't be in the base system today. > > You misinterpreted me. I meant in this specific case, post-install > operation doesn't matter. People can use ssh to get in the machines to > do things rsh/rlogin/rcmd offer. They can also use ssh to get to machines to do things that telnet (IMO similar function as rlogin). Sftp can be used to replace ftp. Not including telnet and ftp in this discussion is inconsistent. Regards, Phone: (250)387-8437 Cy Schubert Fax: (250)387-5766 Team Leader, Sun/DEC Team Internet: Cy.Schubert@osg.gov.bc.ca Open Systems Group, ITSD, ISTA Province of BC To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 13:53: 1 2000 Delivered-To: freebsd-arch@freebsd.org Received: from piano.mahoroba.org (piano-wi.calm.imasy.or.jp [202.227.26.45]) by hub.freebsd.org (Postfix) with ESMTP id 6F8B837B449 for ; Fri, 15 Sep 2000 13:52:56 -0700 (PDT) Received: from localhost (IDENT:AqGspW3uJ6PcejEtvu/uh8wUY8x3dM6lmF41nEQLXWcX8ulEs/hiV7UyLGWh/MM3@localhost [::1]) by piano.mahoroba.org (8.11.0/8.11.0/piano) with ESMTP id e8FKqQg04847; Sat, 16 Sep 2000 05:52:26 +0900 (JST) (envelope-from ume@mahoroba.org) To: will@physics.purdue.edu Cc: arch@FreeBSD.org Subject: Re: Rsh/Rlogin/Rcmd & friends In-Reply-To: <20000915143515.N40658@radon.gryphonsoft.com> References: <20000915143515.N40658@radon.gryphonsoft.com> X-Mailer: xcite1.20> Mew version 1.94.2 on Emacs 20.7 / Mule 4.0 =?iso-2022-jp?B?KBskQjJWMWMbKEIp?= X-PGP-Public-Key: http://www.imasy.org/~ume/publickey.asc X-PGP-Fingerprint: 6B 0C 53 FC 5D D0 37 91 05 D0 B3 EF 36 9B 6A BC X-URL: http://www.imasy.org/~ume/ X-OS: FreeBSD 4.1-STABLE Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-Id: <20000916055225C.ume@mahoroba.org> Date: Sat, 16 Sep 2000 05:52:25 +0900 From: Hajimu UMEMOTO X-Dispatcher: imput version 20000414(IM141) Lines: 35 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG >>>>> On Fri, 15 Sep 2000 14:35:15 -0500 >>>>> Will Andrews said: will> Ok people. I want a reason why we shouldn't move rsh/rlogin/rcmd out of will> the base system and into ports, now that we can support SSH connections will> out of the box. Some programs call rsh. For example, GNU tar calls rsh. How about this patch. I'm using it for a long time. --- gnu/usr.bin/tar/rtapelib.c.orig Mon Nov 4 19:07:44 1996 +++ gnu/usr.bin/tar/rtapelib.c Tue Jul 27 15:22:32 1999 @@ -372,6 +372,8 @@ if (*login) { + execl ("/usr/bin/ssh", "ssh", "-l", login, system, + "/etc/rmt", (char *) 0); execl ("/usr/bin/rsh", "rsh", "-l", login, system, "/etc/rmt", (char *) 0); execlp ("rsh", "rsh", "-l", login, system, @@ -379,6 +381,8 @@ } else { + execl ("/usr/bin/ssh", "ssh", system, + "/etc/rmt", (char *) 0); execl ("/usr/bin/rsh", "rsh", system, "/etc/rmt", (char *) 0); execlp ("rsh", "rsh", system, -- Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan ume@mahoroba.org ume@bisd.hitachi.co.jp ume@FreeBSD.org http://www.imasy.org/~ume/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 13:56:19 2000 Delivered-To: freebsd-arch@freebsd.org Received: from radon.gryphonsoft.com (mcut-b-167.resnet.purdue.edu [128.211.209.167]) by hub.freebsd.org (Postfix) with ESMTP id 63D1037B422 for ; Fri, 15 Sep 2000 13:56:17 -0700 (PDT) Received: by radon.gryphonsoft.com (Postfix, from userid 1000) id 5DCCF1ACC; Fri, 15 Sep 2000 15:53:54 -0500 (EST) Date: Fri, 15 Sep 2000 15:53:54 -0500 From: Will Andrews To: Cy Schubert - ITSD Open Systems Group Cc: Will Andrews , Daniel Eischen , Steve Kargl , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends Message-ID: <20000915155354.U40658@radon.gryphonsoft.com> Reply-To: Will Andrews Mail-Followup-To: Will Andrews , Cy Schubert - ITSD Open Systems Group , Daniel Eischen , Steve Kargl , arch@FreeBSD.ORG References: <20000915153602.S40658@radon.gryphonsoft.com> <200009152050.e8FKojS25996@cwsys.cwsent.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200009152050.e8FKojS25996@cwsys.cwsent.com>; from Cy.Schubert@uumail.gov.bc.ca on Fri, Sep 15, 2000 at 01:50:02PM -0700 X-Operating-System: FreeBSD 4.1-STABLE i386 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, Sep 15, 2000 at 01:50:02PM -0700, Cy Schubert - ITSD Open Systems Group wrote: > Not including telnet and ftp in this discussion is inconsistent. They are included.. see subject. -- Will Andrews GCS/E/S @d- s+:+ a--- C++ UB++++$ P+ L- E--- W+ N-- !o ?K w--- O- M+ V- PS+ PE++ Y+ PGP+>+++ t++ 5 X+ R+ tv+ b++ DI+++ D+ G++ e>++++ h! r- y? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 13:56:58 2000 Delivered-To: freebsd-arch@freebsd.org Received: from radon.gryphonsoft.com (mcut-b-167.resnet.purdue.edu [128.211.209.167]) by hub.freebsd.org (Postfix) with ESMTP id E03E937B423 for ; Fri, 15 Sep 2000 13:56:56 -0700 (PDT) Received: by radon.gryphonsoft.com (Postfix, from userid 1000) id 2A8941B23; Fri, 15 Sep 2000 15:54:34 -0500 (EST) Date: Fri, 15 Sep 2000 15:54:34 -0500 From: Will Andrews To: Hajimu UMEMOTO Cc: will@physics.purdue.edu, arch@FreeBSD.org Subject: Re: Rsh/Rlogin/Rcmd & friends Message-ID: <20000915155434.V40658@radon.gryphonsoft.com> Reply-To: Will Andrews Mail-Followup-To: Will Andrews , Hajimu UMEMOTO , arch@FreeBSD.org References: <20000915143515.N40658@radon.gryphonsoft.com> <20000916055225C.ume@mahoroba.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20000916055225C.ume@mahoroba.org>; from ume@mahoroba.org on Sat, Sep 16, 2000 at 05:52:25AM +0900 X-Operating-System: FreeBSD 4.1-STABLE i386 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sat, Sep 16, 2000 at 05:52:25AM +0900, Hajimu UMEMOTO wrote: > Some programs call rsh. For example, GNU tar calls rsh. > How about this patch. I'm using it for a long time. Yeah, that looks good. I guess I'll have to prod around and find any dependencies on rsh. -- Will Andrews GCS/E/S @d- s+:+ a--- C++ UB++++$ P+ L- E--- W+ N-- !o ?K w--- O- M+ V- PS+ PE++ Y+ PGP+>+++ t++ 5 X+ R+ tv+ b++ DI+++ D+ G++ e>++++ h! r- y? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 14: 2: 2 2000 Delivered-To: freebsd-arch@freebsd.org Received: from radon.gryphonsoft.com (mcut-b-167.resnet.purdue.edu [128.211.209.167]) by hub.freebsd.org (Postfix) with ESMTP id 1EEC937B424 for ; Fri, 15 Sep 2000 14:02:01 -0700 (PDT) Received: by radon.gryphonsoft.com (Postfix, from userid 1000) id 4DE511B23; Fri, 15 Sep 2000 15:59:38 -0500 (EST) Date: Fri, 15 Sep 2000 15:59:38 -0500 From: Will Andrews To: Cy Schubert - ITSD Open Systems Group Cc: Will Andrews , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends Message-ID: <20000915155938.W40658@radon.gryphonsoft.com> Reply-To: Will Andrews Mail-Followup-To: Will Andrews , Cy Schubert - ITSD Open Systems Group , arch@FreeBSD.ORG References: <20000915143515.N40658@radon.gryphonsoft.com> <200009152032.e8FKWfC25897@cwsys.cwsent.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200009152032.e8FKWfC25897@cwsys.cwsent.com>; from Cy.Schubert@uumail.gov.bc.ca on Fri, Sep 15, 2000 at 01:31:52PM -0700 X-Operating-System: FreeBSD 4.1-STABLE i386 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, Sep 15, 2000 at 01:31:52PM -0700, Cy Schubert - ITSD Open Systems Group wrote: > Yet another reason why everything, not just ports, should be installed > using packages. Then install just what you want. I don't agree that *everything* should be installed using packages.. but certainly some things should be. > Other apps that should be in ports are NFS, NIS, sendmail (we already > have qmail and postfix in ports), and possibly even inetd itself -- who > needs inetd when you've got SSH. > > If we're going to do it let's do it right the first time. Agreed.. -- Will Andrews GCS/E/S @d- s+:+ a--- C++ UB++++$ P+ L- E--- W+ N-- !o ?K w--- O- M+ V- PS+ PE++ Y+ PGP+>+++ t++ 5 X+ R+ tv+ b++ DI+++ D+ G++ e>++++ h! r- y? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 14: 2:23 2000 Delivered-To: freebsd-arch@freebsd.org Received: from point.osg.gov.bc.ca (point.osg.gov.bc.ca [142.32.102.44]) by hub.freebsd.org (Postfix) with ESMTP id 4E07237B424 for ; Fri, 15 Sep 2000 14:02:20 -0700 (PDT) Received: (from daemon@localhost) by point.osg.gov.bc.ca (8.8.7/8.8.8) id OAA07740; Fri, 15 Sep 2000 14:01:36 -0700 Received: from passer.osg.gov.bc.ca(142.32.110.29) via SMTP by point.osg.gov.bc.ca, id smtpda07738; Fri Sep 15 14:01:26 2000 Received: (from uucp@localhost) by passer.osg.gov.bc.ca (8.9.3/8.9.1) id OAA60430; Fri, 15 Sep 2000 14:01:26 -0700 (PDT) Received: from cwsys9.cwsent.com(10.2.2.1), claiming to be "cwsys.cwsent.com" via SMTP by passer9.cwsent.com, id smtpdJ60428; Fri Sep 15 14:00:50 2000 Received: (from uucp@localhost) by cwsys.cwsent.com (8.11.0/8.9.1) id e8FL0ne26068; Fri, 15 Sep 2000 14:00:49 -0700 (PDT) Message-Id: <200009152100.e8FL0ne26068@cwsys.cwsent.com> Received: from localhost.cwsent.com(127.0.0.1), claiming to be "cwsys" via SMTP by localhost.cwsent.com, id smtpdP26051; Fri Sep 15 13:59:56 2000 X-Mailer: exmh version 2.1.1 10/15/1999 Reply-To: Cy Schubert - ITSD Open Systems Group From: Cy Schubert - ITSD Open Systems Group X-OS: FreeBSD 4.1-RELEASE X-Sender: cy To: Hajimu UMEMOTO Cc: will@physics.purdue.edu, arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends In-reply-to: Your message of "Sat, 16 Sep 2000 05:52:25 +0900." <20000916055225C.ume@mahoroba.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 15 Sep 2000 13:59:53 -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <20000916055225C.ume@mahoroba.org>, Hajimu UMEMOTO writes: > >>>>> On Fri, 15 Sep 2000 14:35:15 -0500 > >>>>> Will Andrews said: > > will> Ok people. I want a reason why we shouldn't move rsh/rlogin/rcmd out o > f > will> the base system and into ports, now that we can support SSH connections > will> out of the box. > > Some programs call rsh. For example, GNU tar calls rsh. > How about this patch. I'm using it for a long time. > > --- gnu/usr.bin/tar/rtapelib.c.orig Mon Nov 4 19:07:44 1996 > +++ gnu/usr.bin/tar/rtapelib.c Tue Jul 27 15:22:32 1999 > @@ -372,6 +372,8 @@ > > if (*login) > { > + execl ("/usr/bin/ssh", "ssh", "-l", login, system, > + "/etc/rmt", (char *) 0); > execl ("/usr/bin/rsh", "rsh", "-l", login, system, > "/etc/rmt", (char *) 0); > execlp ("rsh", "rsh", "-l", login, system, > @@ -379,6 +381,8 @@ > } > else > { > + execl ("/usr/bin/ssh", "ssh", system, > + "/etc/rmt", (char *) 0); > execl ("/usr/bin/rsh", "rsh", system, > "/etc/rmt", (char *) 0); > execlp ("rsh", "rsh", system, I would rather we remove rsh entirely. If you want to use rsh install the GNU tar port. Regards, Phone: (250)387-8437 Cy Schubert Fax: (250)387-5766 Team Leader, Sun/DEC Team Internet: Cy.Schubert@osg.gov.bc.ca Open Systems Group, ITSD, ISTA Province of BC To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 14: 3:44 2000 Delivered-To: freebsd-arch@freebsd.org Received: from point.osg.gov.bc.ca (point.osg.gov.bc.ca [142.32.102.44]) by hub.freebsd.org (Postfix) with ESMTP id 6F85037B422 for ; Fri, 15 Sep 2000 14:03:41 -0700 (PDT) Received: (from daemon@localhost) by point.osg.gov.bc.ca (8.8.7/8.8.8) id OAA07762; Fri, 15 Sep 2000 14:02:36 -0700 Received: from passer.osg.gov.bc.ca(142.32.110.29) via SMTP by point.osg.gov.bc.ca, id smtpda07756; Fri Sep 15 14:02:29 2000 Received: (from uucp@localhost) by passer.osg.gov.bc.ca (8.9.3/8.9.1) id OAA60441; Fri, 15 Sep 2000 14:02:28 -0700 (PDT) Received: from cwsys9.cwsent.com(10.2.2.1), claiming to be "cwsys.cwsent.com" via SMTP by passer9.cwsent.com, id smtpdZ60433; Fri Sep 15 14:01:49 2000 Received: (from uucp@localhost) by cwsys.cwsent.com (8.11.0/8.9.1) id e8FL1mJ26079; Fri, 15 Sep 2000 14:01:48 -0700 (PDT) Message-Id: <200009152101.e8FL1mJ26079@cwsys.cwsent.com> Received: from localhost.cwsent.com(127.0.0.1), claiming to be "cwsys" via SMTP by localhost.cwsent.com, id smtpdi26075; Fri Sep 15 14:01:39 2000 X-Mailer: exmh version 2.1.1 10/15/1999 Reply-To: Cy Schubert - ITSD Open Systems Group From: Cy Schubert - ITSD Open Systems Group X-OS: FreeBSD 4.1-RELEASE X-Sender: cy To: Will Andrews Cc: Cy Schubert - ITSD Open Systems Group , Daniel Eischen , Steve Kargl , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends In-reply-to: Your message of "Fri, 15 Sep 2000 15:53:54 CDT." <20000915155354.U40658@radon.gryphonsoft.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 15 Sep 2000 14:01:38 -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <20000915155354.U40658@radon.gryphonsoft.com>, Will Andrews writes: > On Fri, Sep 15, 2000 at 01:50:02PM -0700, Cy Schubert - ITSD Open Systems Gro > up wrote: > > Not including telnet and ftp in this discussion is inconsistent. > > They are included.. see subject. Excellent!!! Next question. Why even support them as ports? We already have a KRB5 port. Regards, Phone: (250)387-8437 Cy Schubert Fax: (250)387-5766 Team Leader, Sun/DEC Team Internet: Cy.Schubert@osg.gov.bc.ca Open Systems Group, ITSD, ISTA Province of BC To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 14:12: 2 2000 Delivered-To: freebsd-arch@freebsd.org Received: from post.mail.nl.demon.net (post-10.mail.nl.demon.net [194.159.73.20]) by hub.freebsd.org (Postfix) with ESMTP id 8AA0E37B422 for ; Fri, 15 Sep 2000 14:11:59 -0700 (PDT) Received: from [212.238.54.101] (helo=freebie.demon.nl) by post.mail.nl.demon.net with smtp (Exim 3.14 #2) id 13a2m6-0003SW-00; Fri, 15 Sep 2000 21:11:58 +0000 Received: (from wkb@localhost) by freebie.demon.nl (8.11.0/8.11.0) id e8FLD8r05486; Fri, 15 Sep 2000 23:13:08 +0200 (CEST) (envelope-from wkb) Date: Fri, 15 Sep 2000 23:13:08 +0200 From: Wilko Bulte To: Will Andrews Cc: Cy Schubert - ITSD Open Systems Group , arch@freebsd.org Subject: Re: Rsh/Rlogin/Rcmd & friends Message-ID: <20000915231308.A5459@freebie.demon.nl> References: <20000915143515.N40658@radon.gryphonsoft.com> <200009152032.e8FKWfC25897@cwsys.cwsent.com> <20000915155938.W40658@radon.gryphonsoft.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: <20000915155938.W40658@radon.gryphonsoft.com>; from will@physics.purdue.edu on Fri, Sep 15, 2000 at 03:59:38PM -0500 X-OS: FreeBSD 4.1-STABLE X-PGP: finger wilko@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, Sep 15, 2000 at 03:59:38PM -0500, Will Andrews wrote: > On Fri, Sep 15, 2000 at 01:31:52PM -0700, Cy Schubert - ITSD Open Systems Group wrote: > > Yet another reason why everything, not just ports, should be installed > > using packages. Then install just what you want. > > I don't agree that *everything* should be installed using packages.. but > certainly some things should be. Like the /usr/games stuff? > > Other apps that should be in ports are NFS, NIS, sendmail (we already > > have qmail and postfix in ports), and possibly even inetd itself -- who > > needs inetd when you've got SSH. > > > > If we're going to do it let's do it right the first time. Which is, IMHO, not having to select 10^6 different items during sysinstall. That would smell like Linux, right? -- Wilko Bulte wilko@freebsd.org Arnhem, the Netherlands To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 14:18:32 2000 Delivered-To: freebsd-arch@freebsd.org Received: from pcnet1.pcnet.com (pcnet1.pcnet.com [204.213.232.3]) by hub.freebsd.org (Postfix) with ESMTP id 8823237B422 for ; Fri, 15 Sep 2000 14:18:30 -0700 (PDT) Received: (from eischen@localhost) by pcnet1.pcnet.com (8.8.7/PCNet) id RAA03480; Fri, 15 Sep 2000 17:18:12 -0400 (EDT) Date: Fri, 15 Sep 2000 17:18:12 -0400 (EDT) From: Daniel Eischen To: Will Andrews Cc: Will Andrews , Steve Kargl , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends In-Reply-To: <20000915153602.S40658@radon.gryphonsoft.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, 15 Sep 2000, Will Andrews wrote: > On Fri, Sep 15, 2000 at 04:24:23PM -0400, Daniel Eischen wrote: > > > What consequences? Remember, we'll still have ports for these things. > > > It only matters as far as new installations go. Post-install operations > > > are unimportant. > > > > Wrong. If that were true tcsh wouldn't be in the base system today. > > You misinterpreted me. I meant in this specific case, post-install > operation doesn't matter. People can use ssh to get in the machines to > do things rsh/rlogin/rcmd offer. No, you haven't proven to me that removal of rsh/rlogin/rcmd doesn't break anything like remote backups. As Steve Kargl wrote: > > What are the consequences of your proposal with the use of > > rdump/rrestore from another (non-FreeBSD) machine into a > > tape drive equipped FreeBSD box? To me that means that something that use to work "out of the box" will not work without adding the necessary port(s). Sure, you can argue that you can easily install the port, but the same could be said to folks that wanted tcsh as their default shell. -- Dan Eischen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 14:18:57 2000 Delivered-To: freebsd-arch@freebsd.org Received: from freefall.freebsd.org (freefall.FreeBSD.org [216.136.204.21]) by hub.freebsd.org (Postfix) with ESMTP id D95AD37B423; Fri, 15 Sep 2000 14:18:55 -0700 (PDT) Received: from localhost (kris@localhost) by freefall.freebsd.org (8.9.3/8.9.2) with ESMTP id OAA70101; Fri, 15 Sep 2000 14:18:55 -0700 (PDT) (envelope-from kris@FreeBSD.org) X-Authentication-Warning: freefall.freebsd.org: kris owned process doing -bs Date: Fri, 15 Sep 2000 14:18:55 -0700 (PDT) From: Kris Kennaway To: Hajimu UMEMOTO Cc: will@physics.purdue.edu, arch@FreeBSD.org Subject: Re: Rsh/Rlogin/Rcmd & friends In-Reply-To: <20000916055225C.ume@mahoroba.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sat, 16 Sep 2000, Hajimu UMEMOTO wrote: > Some programs call rsh. For example, GNU tar calls rsh. > How about this patch. I'm using it for a long time. It would probably be better to check and evironment variable TAR_RSH and use that (like CVS_RSH and others) Kris -- In God we Trust -- all others must submit an X.509 certificate. -- Charles Forsythe To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 14:25:34 2000 Delivered-To: freebsd-arch@freebsd.org Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.208.78.105]) by hub.freebsd.org (Postfix) with ESMTP id E088737B424 for ; Fri, 15 Sep 2000 14:25:32 -0700 (PDT) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.9.3/8.9.3) id OAA76593; Fri, 15 Sep 2000 14:29:09 -0700 (PDT) (envelope-from sgk) From: Steve Kargl Message-Id: <200009152129.OAA76593@troutmask.apl.washington.edu> Subject: Re: Rsh/Rlogin/Rcmd & friends In-Reply-To: <20000915145337.Q40658@radon.gryphonsoft.com> from Will Andrews at "Sep 15, 2000 02:53:37 pm" To: Will Andrews Date: Fri, 15 Sep 2000 14:29:09 -0700 (PDT) Cc: arch@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL61 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Will Andrews wrote: > On Fri, Sep 15, 2000 at 12:56:10PM -0700, Steve Kargl wrote: > > What are the consequences of your proposal with the use of > > rdump/rrestore from another (non-FreeBSD) machine into a > > tape drive equipped FreeBSD box? > > What consequences? Remember, we'll still have ports for these things. > It only matters as far as new installations go. Post-install operations > are unimportant. > rdump/rrestore (on at least Tru64 Unix) expect to find /etc/rmt on the remote machine. Other non-FreeBSD Unices probably also expect to find /etc/rmt. troutmask:kargl[268] ls -l /etc/rmt l--------- 1 root wheel 13 Nov 17 1998 /etc/rmt@ -> /usr/sbin/rmt Are you advocating that your port will create the proper symlink? /etc/rmt@ -> /usr/local/sbin/rmt Isn't this against the principles of "The Ports Collection". Note rmt(8) states "Rmt is normally started up with an rexec(3) or rcmd(3) call." -- Steve To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 14:27: 0 2000 Delivered-To: freebsd-arch@freebsd.org Received: from point.osg.gov.bc.ca (point.osg.gov.bc.ca [142.32.102.44]) by hub.freebsd.org (Postfix) with ESMTP id 954CC37B422 for ; Fri, 15 Sep 2000 14:26:57 -0700 (PDT) Received: (from daemon@localhost) by point.osg.gov.bc.ca (8.8.7/8.8.8) id OAA08144; Fri, 15 Sep 2000 14:26:23 -0700 Received: from passer.osg.gov.bc.ca(142.32.110.29) via SMTP by point.osg.gov.bc.ca, id smtpda08142; Fri Sep 15 14:26:14 2000 Received: (from uucp@localhost) by passer.osg.gov.bc.ca (8.9.3/8.9.1) id OAA61191; Fri, 15 Sep 2000 14:26:14 -0700 (PDT) Received: from cwsys9.cwsent.com(10.2.2.1), claiming to be "cwsys.cwsent.com" via SMTP by passer9.cwsent.com, id smtpdv61175; Fri Sep 15 14:25:49 2000 Received: (from uucp@localhost) by cwsys.cwsent.com (8.11.0/8.9.1) id e8FLPmt26231; Fri, 15 Sep 2000 14:25:48 -0700 (PDT) Message-Id: <200009152125.e8FLPmt26231@cwsys.cwsent.com> Received: from localhost.cwsent.com(127.0.0.1), claiming to be "cwsys" via SMTP by localhost.cwsent.com, id smtpdi26227; Fri Sep 15 14:25:31 2000 X-Mailer: exmh version 2.1.1 10/15/1999 Reply-To: Cy Schubert - ITSD Open Systems Group From: Cy Schubert - ITSD Open Systems Group X-OS: FreeBSD 4.1-RELEASE X-Sender: cy To: Wilko Bulte Cc: Will Andrews , Cy Schubert - ITSD Open Systems Group , arch@FreeBSD.ORG Subject: Packages (was: Re: Rsh/Rlogin/Rcmd & friends) In-reply-to: Your message of "Fri, 15 Sep 2000 23:13:08 +0200." <20000915231308.A5459@freebie.demon.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 15 Sep 2000 14:25:31 -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <20000915231308.A5459@freebie.demon.nl>, Wilko Bulte writes: > On Fri, Sep 15, 2000 at 03:59:38PM -0500, Will Andrews wrote: > > On Fri, Sep 15, 2000 at 01:31:52PM -0700, Cy Schubert - ITSD Open Systems G > roup wrote: > > > Yet another reason why everything, not just ports, should be installed > > > using packages. Then install just what you want. > > > > I don't agree that *everything* should be installed using packages.. but > > certainly some things should be. > > Like the /usr/games stuff? > > > > Other apps that should be in ports are NFS, NIS, sendmail (we already > > > have qmail and postfix in ports), and possibly even inetd itself -- who > > > needs inetd when you've got SSH. > > > > > > If we're going to do it let's do it right the first time. > > Which is, IMHO, not having to select 10^6 different items during > sysinstall. That would smell like Linux, right? What would be wrong with that? Solaris is installed via packages. So are the most widely distributed mainframe operating systems (MVS, VM, and VSE). AIX installs everything using packages. The fact that RedHat has adopted this approach, which has been used for years before Linus even dreamed of Lunux, doesn't mean it's the Linux way of doing things. I think the opposite is true: Not using packages is the BSD approach, then there's the package approach that the rest of the world uses. Regards, Phone: (250)387-8437 Cy Schubert Fax: (250)387-5766 Team Leader, Sun/DEC Team Internet: Cy.Schubert@osg.gov.bc.ca Open Systems Group, ITSD, ISTA Province of BC To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 14:35:24 2000 Delivered-To: freebsd-arch@freebsd.org Received: from po3.glue.umd.edu (po3.glue.umd.edu [128.8.10.123]) by hub.freebsd.org (Postfix) with ESMTP id ABF8237B443 for ; Fri, 15 Sep 2000 14:35:15 -0700 (PDT) Received: from z.glue.umd.edu (IDENT:root@z.glue.umd.edu [128.8.10.71]) by po3.glue.umd.edu (8.10.1/8.10.1) with ESMTP id e8FLZ8Y22074; Fri, 15 Sep 2000 17:35:09 -0400 (EDT) Received: from z.glue.umd.edu (IDENT:sendmail@localhost [127.0.0.1]) by z.glue.umd.edu (8.9.3/8.9.3) with SMTP id RAA04636; Fri, 15 Sep 2000 17:35:08 -0400 (EDT) Received: from localhost (howardjp@localhost) by z.glue.umd.edu (8.9.3/8.9.3) with ESMTP id RAA04631; Fri, 15 Sep 2000 17:35:08 -0400 (EDT) X-Authentication-Warning: z.glue.umd.edu: howardjp owned process doing -bs Date: Fri, 15 Sep 2000 17:35:07 -0400 (EDT) From: James Howard To: Cy Schubert - ITSD Open Systems Group Cc: Wilko Bulte , Will Andrews , arch@FreeBSD.ORG Subject: Re: Packages (was: Re: Rsh/Rlogin/Rcmd & friends) In-Reply-To: <200009152125.e8FLPmt26231@cwsys.cwsent.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, 15 Sep 2000, Cy Schubert - ITSD Open Systems Group wrote: > things. I think the opposite is true: Not using packages is the BSD > approach, then there's the package approach that the rest of the world > uses. On the other hand, this is one of the reasons I prefer FreeBSD. J~ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 14:38:50 2000 Delivered-To: freebsd-arch@freebsd.org Received: from point.osg.gov.bc.ca (point.osg.gov.bc.ca [142.32.102.44]) by hub.freebsd.org (Postfix) with ESMTP id E480337B422 for ; Fri, 15 Sep 2000 14:38:46 -0700 (PDT) Received: (from daemon@localhost) by point.osg.gov.bc.ca (8.8.7/8.8.8) id OAA08215; Fri, 15 Sep 2000 14:37:43 -0700 Received: from passer.osg.gov.bc.ca(142.32.110.29) via SMTP by point.osg.gov.bc.ca, id smtpda08213; Fri Sep 15 14:37:31 2000 Received: (from uucp@localhost) by passer.osg.gov.bc.ca (8.9.3/8.9.1) id OAA61373; Fri, 15 Sep 2000 14:37:31 -0700 (PDT) Received: from cwsys9.cwsent.com(10.2.2.1), claiming to be "cwsys.cwsent.com" via SMTP by passer9.cwsent.com, id smtpdz61355; Fri Sep 15 14:36:51 2000 Received: (from uucp@localhost) by cwsys.cwsent.com (8.11.0/8.9.1) id e8FLaou26312; Fri, 15 Sep 2000 14:36:50 -0700 (PDT) Message-Id: <200009152136.e8FLaou26312@cwsys.cwsent.com> Received: from localhost.cwsent.com(127.0.0.1), claiming to be "cwsys" via SMTP by localhost.cwsent.com, id smtpdX26303; Fri Sep 15 14:35:51 2000 X-Mailer: exmh version 2.1.1 10/15/1999 Reply-To: Cy Schubert - ITSD Open Systems Group From: Cy Schubert - ITSD Open Systems Group X-OS: FreeBSD 4.1-RELEASE X-Sender: cy To: Daniel Eischen Cc: Will Andrews , Steve Kargl , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends In-reply-to: Your message of "Fri, 15 Sep 2000 17:18:12 EDT." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 15 Sep 2000 14:35:50 -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message , Daniel Ei schen writes: > On Fri, 15 Sep 2000, Will Andrews wrote: > > On Fri, Sep 15, 2000 at 04:24:23PM -0400, Daniel Eischen wrote: > > > > What consequences? Remember, we'll still have ports for these things. > > > > It only matters as far as new installations go. Post-install operation > s > > > > are unimportant. > > > > > > Wrong. If that were true tcsh wouldn't be in the base system today. > > > > You misinterpreted me. I meant in this specific case, post-install > > operation doesn't matter. People can use ssh to get in the machines to > > do things rsh/rlogin/rcmd offer. > > No, you haven't proven to me that removal of rsh/rlogin/rcmd doesn't > break anything like remote backups. As Steve Kargl wrote: > > > > What are the consequences of your proposal with the use of > > > rdump/rrestore from another (non-FreeBSD) machine into a > > > tape drive equipped FreeBSD box? > > To me that means that something that use to work "out of the box" will > not work without adding the necessary port(s). Sure, you can argue that > you can easily install the port, but the same could be said to folks > that wanted tcsh as their default shell. So what! That's the price of security. I believe that the telnet/ftp/"r" commands shouldn't even be ports. We need to make it difficult to install unsafe software on the system. That way the admin would have to go to all the trouble to find the source for unsafe software somewhere on the Net, port it, and install it. Then it's not FreeBSD's fault if that admin's system is compromised. Regards, Phone: (250)387-8437 Cy Schubert Fax: (250)387-5766 Team Leader, Sun/DEC Team Internet: Cy.Schubert@osg.gov.bc.ca Open Systems Group, ITSD, ISTA Province of BC To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 14:44:52 2000 Delivered-To: freebsd-arch@freebsd.org Received: from winston.osd.bsdi.com (winston.osd.bsdi.com [204.216.27.229]) by hub.freebsd.org (Postfix) with ESMTP id 48F8337B423 for ; Fri, 15 Sep 2000 14:44:49 -0700 (PDT) Received: from winston.osd.bsdi.com (jkh@localhost [127.0.0.1]) by winston.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id e8FLif697881; Fri, 15 Sep 2000 14:44:42 -0700 (PDT) (envelope-from jkh@winston.osd.bsdi.com) To: Will Andrews Cc: Cy Schubert - ITSD Open Systems Group , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends In-Reply-To: Message from Will Andrews of "Fri, 15 Sep 2000 15:59:38 CDT." <20000915155938.W40658@radon.gryphonsoft.com> Date: Fri, 15 Sep 2000 14:44:41 -0700 Message-ID: <97878.969054281@winston.osd.bsdi.com> From: Jordan Hubbard Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > I don't agree that *everything* should be installed using packages.. but > certainly some things should be. Hmmm. So that begs the question: What would you argue should _not_ be packages and what would be your grounds for such argument in those cases? - Jordan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 14:46:38 2000 Delivered-To: freebsd-arch@freebsd.org Received: from mail.nyct.net (bsd4.nyct.net [216.139.128.6]) by hub.freebsd.org (Postfix) with ESMTP id 356FA37B422 for ; Fri, 15 Sep 2000 14:46:36 -0700 (PDT) Received: from bsd1.nyct.net (efutch@bsd1.nyct.net [216.139.128.3]) by mail.nyct.net (8.9.3/8.8.7) with ESMTP id RAA46367; Fri, 15 Sep 2000 17:46:18 -0400 (EDT) (envelope-from efutch@nyct.net) Date: Fri, 15 Sep 2000 17:46:30 -0400 (EDT) From: "Eric D. Futch" To: Cy Schubert - ITSD Open Systems Group Cc: Daniel Eischen , Will Andrews , Steve Kargl , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends In-Reply-To: <200009152136.e8FLaou26312@cwsys.cwsent.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, 15 Sep 2000, Cy Schubert - ITSD Open Systems Group wrote: >In message , > >So what! That's the price of security. I believe that the >telnet/ftp/"r" commands shouldn't even be ports. We need to make it >difficult to install unsafe software on the system. That way the admin >would have to go to all the trouble to find the source for unsafe >software somewhere on the Net, port it, and install it. Then it's not >FreeBSD's fault if that admin's system is compromised. > Then we can just call ourselves OpenBSD and be done with it :). > >Regards, Phone: (250)387-8437 >Cy Schubert Fax: (250)387-5766 >Team Leader, Sun/DEC Team Internet: Cy.Schubert@osg.gov.bc.ca >Open Systems Group, ITSD, ISTA >Province of BC > -- Eric Futch New York Connect.Net, Ltd. efutch@nyct.net Technical Support Staff http://www.nyct.net (212) 293-2620 "Bringing New York The Internet Access It Deserves" KNYC: 15-Sep-00 16:51 EDT: 70.0 F (21.1 C), partly cloudy, humidity 47% To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 14:57:32 2000 Delivered-To: freebsd-arch@freebsd.org Received: from point.osg.gov.bc.ca (point.osg.gov.bc.ca [142.32.102.44]) by hub.freebsd.org (Postfix) with ESMTP id BAA2337B42C for ; Fri, 15 Sep 2000 14:57:29 -0700 (PDT) Received: (from daemon@localhost) by point.osg.gov.bc.ca (8.8.7/8.8.8) id OAA08286; Fri, 15 Sep 2000 14:56:44 -0700 Received: from passer.osg.gov.bc.ca(142.32.110.29) via SMTP by point.osg.gov.bc.ca, id smtpda08284; Fri Sep 15 14:56:41 2000 Received: (from uucp@localhost) by passer.osg.gov.bc.ca (8.9.3/8.9.1) id OAA61520; Fri, 15 Sep 2000 14:56:41 -0700 (PDT) Received: from cwsys9.cwsent.com(10.2.2.1), claiming to be "cwsys.cwsent.com" via SMTP by passer9.cwsent.com, id smtpdr61512; Fri Sep 15 14:55:53 2000 Received: (from uucp@localhost) by cwsys.cwsent.com (8.11.0/8.9.1) id e8FLtr926417; Fri, 15 Sep 2000 14:55:53 -0700 (PDT) Message-Id: <200009152155.e8FLtr926417@cwsys.cwsent.com> Received: from localhost.cwsent.com(127.0.0.1), claiming to be "cwsys" via SMTP by localhost.cwsent.com, id smtpdF26408; Fri Sep 15 14:55:51 2000 X-Mailer: exmh version 2.1.1 10/15/1999 Reply-To: Cy Schubert - ITSD Open Systems Group From: Cy Schubert - ITSD Open Systems Group X-OS: FreeBSD 4.1-RELEASE X-Sender: cy To: James Howard Cc: Cy Schubert - ITSD Open Systems Group , Wilko Bulte , Will Andrews , arch@FreeBSD.ORG Subject: Re: Packages (was: Re: Rsh/Rlogin/Rcmd & friends) In-reply-to: Your message of "Fri, 15 Sep 2000 17:35:07 EDT." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 15 Sep 2000 14:55:51 -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message , James Howard writes: > On Fri, 15 Sep 2000, Cy Schubert - ITSD Open Systems Group wrote: > > > things. I think the opposite is true: Not using packages is the BSD > > approach, then there's the package approach that the rest of the world > > uses. > > On the other hand, this is one of the reasons I prefer FreeBSD. I prefer FreeBSD because it's quality software. The fact that it doesn't use packages is IMO a negative, however the quality of the software far outweighs this significant negative. If FreeBSD was installed using packages I'd consider myself having my cake and eating it too. What RedHat does, requiring you to select each individual RPM is silly. IBM's approach (I spent 19 years of my career as an MVS Systems Programmer so I draw upon that experience as my preference) where the Systems Programmer would install the package and with the appropriate option SMP/E (MVS's package manager) would install all pre-requisites and co-requisites. I would normally run SMP/E with the "check" option first to see what it would install for me, then run it for real to actually do the install. Some packages were distributed as binary only and others were installed as source and binary. The MVS systems programmer would apply patches (PTF's or APAR fixes) to packages (Functions or FMID's). If I as the System's Programmer wanted to modify a piece of the O/S I would build a USERMOD. The hierarchical nature of SMP/E would discard my USERMODS, notifying me of course, when any of my USERMODS were being updated by an FMID (package), PTF (patch), APAR (temporary patch), or another USERMOD (specifically written to supercede a prior USERMOD). Implementing packages into FreeBSD would radically change the way releases are created so if we do something like this, it needs to be carefully thought out well in advance of implementation, e.g. 99% of all the issues need to be hashed out long before the first piece of code is written. Implementation cannot be taken lightly. Regards, Phone: (250)387-8437 Cy Schubert Fax: (250)387-5766 Team Leader, Sun/DEC Team Internet: Cy.Schubert@osg.gov.bc.ca Open Systems Group, ITSD, ISTA Province of BC To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 14:58:41 2000 Delivered-To: freebsd-arch@freebsd.org Received: from h132-197-97-45.gte.com (h132-197-97-45.gte.com [132.197.97.45]) by hub.freebsd.org (Postfix) with ESMTP id F057737B422 for ; Fri, 15 Sep 2000 14:58:38 -0700 (PDT) Received: (from ak03@localhost) by h132-197-97-45.gte.com (8.11.0/8.11.0) id e8FLwY306459; Fri, 15 Sep 2000 17:58:34 -0400 (EDT) (envelope-from ak03) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200009152136.e8FLaou26312@cwsys.cwsent.com> Date: Fri, 15 Sep 2000 17:58:34 -0400 (EDT) Organization: GTE Laboratories Inc. From: "Alexander N. Kabaev" To: Cy Schubert - ITSD Open Systems Group Subject: Re: Rsh/Rlogin/Rcmd & friends Cc: freebsd-arch@FreeBSD.ORG, Steve Kargl , Will Andrews , Daniel Eischen Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > So what! That's the price of security. I believe that the > telnet/ftp/"r" commands shouldn't even be ports. We need to make it > difficult to install unsafe software on the system. That way the admin > would have to go to all the trouble to find the source for unsafe > software somewhere on the Net, port it, and install it. Then it's not > FreeBSD's fault if that admin's system is compromised. Disabling corresponding inetd.conf entries will serve that purpose as well. Some of us work in places full of Unixes of all kinds and flavours and out-of-the-box interoperability argument presented by Steve makes a lot of sense in such an environment. "Everything is the package" idea is OK as long as we will retain the option of complete system upgrade using CVS/buildworld path. Forcing users to deal with every package separately will destroy that fuzzy feeling of completeness and tightly OS components integration FreeBSD was always famous for and IMHO will do more damage than good. Please count my voice as "strongly against" proposed code removal. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 15: 3:30 2000 Delivered-To: freebsd-arch@freebsd.org Received: from winston.osd.bsdi.com (winston.osd.bsdi.com [204.216.27.229]) by hub.freebsd.org (Postfix) with ESMTP id 8C31737B422 for ; Fri, 15 Sep 2000 15:03:27 -0700 (PDT) Received: from winston.osd.bsdi.com (jkh@localhost [127.0.0.1]) by winston.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id e8FM3F697980; Fri, 15 Sep 2000 15:03:15 -0700 (PDT) (envelope-from jkh@winston.osd.bsdi.com) To: Cy Schubert - ITSD Open Systems Group Cc: Wilko Bulte , Will Andrews , arch@FreeBSD.ORG Subject: Re: Packages (was: Re: Rsh/Rlogin/Rcmd & friends) In-Reply-To: Message from Cy Schubert - ITSD Open Systems Group of "Fri, 15 Sep 2000 14:25:31 PDT." <200009152125.e8FLPmt26231@cwsys.cwsent.com> Date: Fri, 15 Sep 2000 15:03:15 -0700 Message-ID: <97976.969055395@winston.osd.bsdi.com> From: Jordan Hubbard Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > Which is, IMHO, not having to select 10^6 different items during > > sysinstall. That would smell like Linux, right? > > What would be wrong with that? Solaris is installed via packages. So > are the most widely distributed mainframe operating systems (MVS, VM, > and VSE). AIX installs everything using packages. The fact that > RedHat has adopted this approach, which has been used for years before I think people are drawing a false association between "organization" and "packaging", two things which are currently linked together in FreeBSD for historical rather than practical reasons. Let's take FreeBSD's bindist for example. It started life as a simple sweeping together of everything in "/etc, {/usr,}{/bin,/sbin}" and that was that. Then people griped that they wanted to not have to install catpages, so those got scooped out into another tarball before the bindist packaging phase got run and now the contents of the aformetioned directories were magically smaller. Then we did the same with the proflibs, then the manpages (I may have the ordering wrong here, sue me), etc etc. The eventual result was that you had a "flat but occasionally deep" set of distributions like: bin proflibs manpages catpages ports ... Where some things like "bin" were still very big and not further sub-divided at all (still contained docs tools, compilers, etc) and other things, by collective whim, were removed and shoved up to the same level in the hierarchy. What people wanted but didn't actually get was something more like: [X] bin [X] base [ ] proflibs [ ] manpages [ ] catpages [ ] ports ... And what they'd *really* like is something more like: [X] bin [X] base [X] compiler tools [X] perl language [X] documentation formatters [X] printing [ ] proflibs [ ] manpages [ ] catpages ... [ ] ports ... Which actually subdivides bin into all the logical pieces which comprise it. You'll notice that I also checked some of the items by default, something which I think it's up to the *installer* to do ahead of time in response to an earlier "user profile" question. There's no reason why, if the user selects "Standard", that they shouldn't get a default set of packages selected which results in the *exact same* FreeBSD footprint which we all(*) know and love(**) today. Similarly, if the user knows exactly what they're doing and they really really hate Perl (just to pick a completely arbitrary and unbiased example), they can de-select that or simply pick "Custom" instead of "Standard" in the first place. If these things are also furthermore packages, then one of us admin types can come along later during the process of trying to debug this person's systems and use pkg_info (or whatever follows it) to get an immediate report of what's installed and go "aha, you didn't install the compiler tools, no wonder your kernel builds are falling over." - Jordan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 15: 9:10 2000 Delivered-To: freebsd-arch@freebsd.org Received: from point.osg.gov.bc.ca (point.osg.gov.bc.ca [142.32.102.44]) by hub.freebsd.org (Postfix) with ESMTP id 8BF2837B422 for ; Fri, 15 Sep 2000 15:09:06 -0700 (PDT) Received: (from daemon@localhost) by point.osg.gov.bc.ca (8.8.7/8.8.8) id PAA08375; Fri, 15 Sep 2000 15:09:04 -0700 Received: from passer.osg.gov.bc.ca(142.32.110.29) via SMTP by point.osg.gov.bc.ca, id smtpda08373; Fri Sep 15 15:08:59 2000 Received: (from uucp@localhost) by passer.osg.gov.bc.ca (8.9.3/8.9.1) id PAA61752; Fri, 15 Sep 2000 15:08:58 -0700 (PDT) Received: from cwsys9.cwsent.com(10.2.2.1), claiming to be "cwsys.cwsent.com" via SMTP by passer9.cwsent.com, id smtpdP61746; Fri Sep 15 15:08:56 2000 Received: (from uucp@localhost) by cwsys.cwsent.com (8.11.0/8.9.1) id e8FM8tZ26513; Fri, 15 Sep 2000 15:08:55 -0700 (PDT) Message-Id: <200009152208.e8FM8tZ26513@cwsys.cwsent.com> Received: from localhost.cwsent.com(127.0.0.1), claiming to be "cwsys" via SMTP by localhost.cwsent.com, id smtpdw26507; Fri Sep 15 15:08:49 2000 X-Mailer: exmh version 2.1.1 10/15/1999 Reply-To: Cy Schubert - ITSD Open Systems Group From: Cy Schubert - ITSD Open Systems Group X-OS: FreeBSD 4.1-RELEASE X-Sender: cy To: "Alexander N. Kabaev" Cc: Cy Schubert - ITSD Open Systems Group , freebsd-arch@FreeBSD.ORG, Steve Kargl , Will Andrews , Daniel Eischen Subject: Re: Rsh/Rlogin/Rcmd & friends In-reply-to: Your message of "Fri, 15 Sep 2000 17:58:34 EDT." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 15 Sep 2000 15:08:49 -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message , "Alexander N. Kabaev" writes: > > So what! That's the price of security. I believe that the > > telnet/ftp/"r" commands shouldn't even be ports. We need to make it > > difficult to install unsafe software on the system. That way the admin > > would have to go to all the trouble to find the source for unsafe > > software somewhere on the Net, port it, and install it. Then it's not > > FreeBSD's fault if that admin's system is compromised. > > Disabling corresponding inetd.conf entries will serve that purpose as well. > Some of us work in places full of Unixes of all kinds and flavours and > out-of-the-box interoperability argument presented by Steve makes a lot of > sense in such an environment. > > "Everything is the package" idea is OK as long as we will retain the option o > f > complete system upgrade using CVS/buildworld path. Forcing users to deal with > every package separately will destroy that fuzzy feeling of completeness and > tightly OS components integration FreeBSD was always famous for and IMHO will > do more damage than good. > > Please count my voice as "strongly against" proposed code removal. How about a compromise? Like an option or options in make.conf? Then put telnet/ftp/"r" commands into an rcmddist distribution? Regards, Phone: (250)387-8437 Cy Schubert Fax: (250)387-5766 Team Leader, Sun/DEC Team Internet: Cy.Schubert@osg.gov.bc.ca Open Systems Group, ITSD, ISTA Province of BC To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 15:11:55 2000 Delivered-To: freebsd-arch@freebsd.org Received: from radon.gryphonsoft.com (mcut-b-167.resnet.purdue.edu [128.211.209.167]) by hub.freebsd.org (Postfix) with ESMTP id 1D1BA37B423 for ; Fri, 15 Sep 2000 15:11:54 -0700 (PDT) Received: by radon.gryphonsoft.com (Postfix, from userid 1000) id 420FC1B5A; Fri, 15 Sep 2000 17:09:31 -0500 (EST) Date: Fri, 15 Sep 2000 17:09:31 -0500 From: Will Andrews To: Jordan Hubbard Cc: Will Andrews , Cy Schubert - ITSD Open Systems Group , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends Message-ID: <20000915170931.X40658@radon.gryphonsoft.com> Reply-To: Will Andrews Mail-Followup-To: Will Andrews , Jordan Hubbard , Cy Schubert - ITSD Open Systems Group , arch@FreeBSD.ORG References: <97878.969054281@winston.osd.bsdi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <97878.969054281@winston.osd.bsdi.com>; from jkh@winston.osd.bsdi.com on Fri, Sep 15, 2000 at 02:44:41PM -0700 X-Operating-System: FreeBSD 4.1-STABLE i386 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, Sep 15, 2000 at 02:44:41PM -0700, Jordan Hubbard wrote: > Hmmm. So that begs the question: What would you argue should _not_ be > packages and what would be your grounds for such argument in those > cases? Basic stuff like /bin/cp and other /bin/*, /sbin/* and so forth, with certain exceptions. Of course, the only reason why I'd say this is because: wouldn't "packages" be more difficult to maintain than the source? Or am I on crack again? Perhaps it's not that hard at all, and everything should be packaged. -- Will Andrews GCS/E/S @d- s+:+ a--- C++ UB++++$ P+ L- E--- W+ N-- !o ?K w--- O- M+ V- PS+ PE++ Y+ PGP+>+++ t++ 5 X+ R+ tv+ b++ DI+++ D+ G++ e>++++ h! r- y? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 15:22: 8 2000 Delivered-To: freebsd-arch@freebsd.org Received: from pcnet1.pcnet.com (pcnet1.pcnet.com [204.213.232.3]) by hub.freebsd.org (Postfix) with ESMTP id DC95537B43F for ; Fri, 15 Sep 2000 15:22:05 -0700 (PDT) Received: (from eischen@localhost) by pcnet1.pcnet.com (8.8.7/PCNet) id SAA12340; Fri, 15 Sep 2000 18:21:47 -0400 (EDT) Date: Fri, 15 Sep 2000 18:21:46 -0400 (EDT) From: Daniel Eischen To: Cy Schubert - ITSD Open Systems Group Cc: Will Andrews , Steve Kargl , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends In-Reply-To: <200009152136.e8FLaou26312@cwsys.cwsent.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, 15 Sep 2000, Cy Schubert - ITSD Open Systems Group wrote: > In message , > Daniel Ei > schen writes: > > On Fri, 15 Sep 2000, Will Andrews wrote: > > > On Fri, Sep 15, 2000 at 04:24:23PM -0400, Daniel Eischen wrote: > > > > > What consequences? Remember, we'll still have ports for these things. > > > > > It only matters as far as new installations go. Post-install operation > > s > > > > > are unimportant. > > > > > > > > Wrong. If that were true tcsh wouldn't be in the base system today. > > > > > > You misinterpreted me. I meant in this specific case, post-install > > > operation doesn't matter. People can use ssh to get in the machines to > > > do things rsh/rlogin/rcmd offer. > > > > No, you haven't proven to me that removal of rsh/rlogin/rcmd doesn't > > break anything like remote backups. As Steve Kargl wrote: > > > > > > What are the consequences of your proposal with the use of > > > > rdump/rrestore from another (non-FreeBSD) machine into a > > > > tape drive equipped FreeBSD box? > > > > To me that means that something that use to work "out of the box" will > > not work without adding the necessary port(s). Sure, you can argue that > > you can easily install the port, but the same could be said to folks > > that wanted tcsh as their default shell. > > So what! That's the price of security. I believe that the > telnet/ftp/"r" commands shouldn't even be ports. We need to make it > difficult to install unsafe software on the system. That way the admin > would have to go to all the trouble to find the source for unsafe > software somewhere on the Net, port it, and install it. Then it's not > FreeBSD's fault if that admin's system is compromised. It was difficult enough to get our users comfortable enough with even using telnet and ftp, and I don't want to waste any more of my software engineering time in user education. If you want a anally secure box by default, run OpenBSD. But you don't need to _remove_ telnet,ftp,r* from the src tree to get a secure system from installation. You could easily have an install option that removes (or doesn't install) your unsafe programs. And I am against 10^6 install options, regardless of whether Linux, Solaris, or any other UN*X does it that way. -- Dan Eischen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 15:31:30 2000 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id E579F37B422 for ; Fri, 15 Sep 2000 15:31:28 -0700 (PDT) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.9.3/8.9.3) with SMTP id SAA66969; Fri, 15 Sep 2000 18:31:21 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Fri, 15 Sep 2000 18:31:21 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Will Andrews Cc: arch@FreeBSD.org Subject: Re: Rsh/Rlogin/Rcmd & friends In-Reply-To: <20000915143515.N40658@radon.gryphonsoft.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, 15 Sep 2000, Will Andrews wrote: > Ok people. I want a reason why we shouldn't move rsh/rlogin/rcmd out of > the base system and into ports, now that we can support SSH connections > out of the box. > > And, ahead of the pack of y'all, POLA be damned! So what's wrong with my encrypted, Kerberos-enabled r* tools? BTW, I suspect that throwing thing into "ports" as a grave is not really the right answer. Keeping things in the CVS repo is fine, we can just disable them in the build, or relegate them to a "network compatibility" distribution or something. My understand is that ports is intended for maintaining and applying patches to software not maintained as part of our distribution or in our source repository. People need to distinguish between building/installing something by default, and the home of the source code. Robert N M Watson robert@fledge.watson.org http://www.watson.org/~robert/ PGP key fingerprint: AF B5 5F FF A6 4A 79 37 ED 5F 55 E9 58 04 6A B1 TIS Labs at Network Associates, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 15:36: 8 2000 Delivered-To: freebsd-arch@freebsd.org Received: from point.osg.gov.bc.ca (point.osg.gov.bc.ca [142.32.102.44]) by hub.freebsd.org (Postfix) with ESMTP id B5DED37B422 for ; Fri, 15 Sep 2000 15:36:05 -0700 (PDT) Received: (from daemon@localhost) by point.osg.gov.bc.ca (8.8.7/8.8.8) id PAA08482; Fri, 15 Sep 2000 15:35:25 -0700 Received: from passer.osg.gov.bc.ca(142.32.110.29) via SMTP by point.osg.gov.bc.ca, id smtpda08480; Fri Sep 15 15:35:12 2000 Received: (from uucp@localhost) by passer.osg.gov.bc.ca (8.9.3/8.9.1) id PAA61977; Fri, 15 Sep 2000 15:35:12 -0700 (PDT) Received: from cwsys9.cwsent.com(10.2.2.1), claiming to be "cwsys.cwsent.com" via SMTP by passer9.cwsent.com, id smtpdE61972; Fri Sep 15 15:34:57 2000 Received: (from uucp@localhost) by cwsys.cwsent.com (8.11.0/8.9.1) id e8FMYu326648; Fri, 15 Sep 2000 15:34:56 -0700 (PDT) Message-Id: <200009152234.e8FMYu326648@cwsys.cwsent.com> Received: from localhost.cwsent.com(127.0.0.1), claiming to be "cwsys" via SMTP by localhost.cwsent.com, id smtpdM26641; Fri Sep 15 15:34:01 2000 X-Mailer: exmh version 2.1.1 10/15/1999 Reply-To: Cy Schubert - ITSD Open Systems Group From: Cy Schubert - ITSD Open Systems Group X-OS: FreeBSD 4.1-RELEASE X-Sender: cy To: Will Andrews Cc: Jordan Hubbard , Cy Schubert - ITSD Open Systems Group , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends In-reply-to: Your message of "Fri, 15 Sep 2000 17:09:31 CDT." <20000915170931.X40658@radon.gryphonsoft.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 15 Sep 2000 15:34:01 -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <20000915170931.X40658@radon.gryphonsoft.com>, Will Andrews writes: > On Fri, Sep 15, 2000 at 02:44:41PM -0700, Jordan Hubbard wrote: > > Hmmm. So that begs the question: What would you argue should _not_ be > > packages and what would be your grounds for such argument in those > > cases? > > Basic stuff like /bin/cp and other /bin/*, /sbin/* and so forth, with > certain exceptions. > > Of course, the only reason why I'd say this is because: wouldn't > "packages" be more difficult to maintain than the source? > > Or am I on crack again? Perhaps it's not that hard at all, and > everything should be packaged. If you are then I must be too. All I've had is coffee. Maybe there's something in the coffee. :) The CVS tree would remain as it is right how. There would be an extract process to build the packages out of the CVS tree, similar to what make release does, just more to it. As I sense that there's no consensus here right now, I'm willing to concede at this moment, though I'm willing to be involved in any long-term discussions about this. As BSDi and FreeBSD have plans to merge their source trees over time, I'm sure that this issue will be revisited. Regards, Phone: (250)387-8437 Cy Schubert Fax: (250)387-5766 Team Leader, Sun/DEC Team Internet: Cy.Schubert@osg.gov.bc.ca Open Systems Group, ITSD, ISTA Province of BC To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 15:42:33 2000 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id B1FAB37B423 for ; Fri, 15 Sep 2000 15:42:30 -0700 (PDT) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.9.3/8.9.3) with SMTP id SAA67839; Fri, 15 Sep 2000 18:42:09 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Fri, 15 Sep 2000 18:42:09 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Cy Schubert - ITSD Open Systems Group Cc: Will Andrews , Daniel Eischen , Steve Kargl , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends In-Reply-To: <200009152050.e8FKojS25996@cwsys.cwsent.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, 15 Sep 2000, Cy Schubert - ITSD Open Systems Group wrote: > > You misinterpreted me. I meant in this specific case, post-install > > operation doesn't matter. People can use ssh to get in the machines to > > do things rsh/rlogin/rcmd offer. > > They can also use ssh to get to machines to do things that telnet (IMO > similar function as rlogin). Sftp can be used to replace ftp. FTP is a widely deployed software distribution service, which even the FreeBSD project uses. There's no anonymous sftp, and such a service would be undesirable for performance and security reasons, due to ssh's adoption of keys in the keyfile to connect to a host -- accepting a key for anonymous access should not be the same as accepting it for authenticated access, but there is no trust level service in the SSH keyfile. Also, last I checked, out sftp port did something stupid with regards to path handling -- it relies on sftpserv being in the path, accessible by the ssh daemon. We hard-coded the path making it non-portable to other platforms, where /usr/libexec is *not* the location of sftpserv. Attempting to sftp to a platform with sftpserv in a different path fails as a result of this modification. Sftpserv is intended to reside in the default path for sshd. Not to mention that sftp is not a standized protocol. > Not including telnet and ftp in this discussion is inconsistent. Not including either would be a mistake -- just disable them in inetd.conf. We have SSH, which means we support secure service. Telling the rest of the world to be damned and removing countless tightly integrated and well-supported services would be foolish. Remember: tools not policy. I use telnet and ftp all the time with perfect safety. Robert N M Watson robert@fledge.watson.org http://www.watson.org/~robert/ PGP key fingerprint: AF B5 5F FF A6 4A 79 37 ED 5F 55 E9 58 04 6A B1 TIS Labs at Network Associates, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 16:15: 7 2000 Delivered-To: freebsd-arch@freebsd.org Received: from palrel3.hp.com (palrel3.hp.com [156.153.255.226]) by hub.freebsd.org (Postfix) with ESMTP id 9A61B37B423 for ; Fri, 15 Sep 2000 16:15:05 -0700 (PDT) Received: from adlmail.cup.hp.com (adlmail.cup.hp.com [15.0.100.30]) by palrel3.hp.com (Postfix) with ESMTP id EC898B62; Fri, 15 Sep 2000 16:15:04 -0700 (PDT) Received: from cup.hp.com (gauss.cup.hp.com [15.28.97.152]) by adlmail.cup.hp.com (8.9.3 (PHNE_18546)/8.9.3 SMKit7.02) with ESMTP id QAA09084; Fri, 15 Sep 2000 16:15:04 -0700 (PDT) Message-ID: <39C2AD78.F9AAD85C@cup.hp.com> Date: Fri, 15 Sep 2000 19:15:04 -0400 From: Marcel Moolenaar Organization: Hewlett-Packard X-Mailer: Mozilla 4.73 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: "Eric D. Futch" Cc: Cy Schubert - ITSD Open Systems Group , Daniel Eischen , Will Andrews , Steve Kargl , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG "Eric D. Futch" wrote: > > >So what! That's the price of security. I believe that the > >telnet/ftp/"r" commands shouldn't even be ports. We need to make it > >difficult to install unsafe software on the system. That way the admin > >would have to go to all the trouble to find the source for unsafe > >software somewhere on the Net, port it, and install it. Then it's not > >FreeBSD's fault if that admin's system is compromised. > > > Then we can just call ourselves OpenBSD and be done with it :). We can do better than that: remove networking, consoles and the ability to run processes. As an optimization we can even avoid loading the kernel. Yep, that's the price of security :-) -- Marcel Moolenaar mail: marcel@cup.hp.com / marcel@FreeBSD.org tel: (408) 447-4222 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 17: 3:27 2000 Delivered-To: freebsd-arch@freebsd.org Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.208.78.105]) by hub.freebsd.org (Postfix) with ESMTP id 6617237B423 for ; Fri, 15 Sep 2000 17:03:25 -0700 (PDT) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.9.3/8.9.3) id RAA77706; Fri, 15 Sep 2000 17:06:58 -0700 (PDT) (envelope-from sgk) From: Steve Kargl Message-Id: <200009160006.RAA77706@troutmask.apl.washington.edu> Subject: Re: Rsh/Rlogin/Rcmd & friends In-Reply-To: <200009152136.e8FLaou26312@cwsys.cwsent.com> from Cy Schubert - ITSD Open Systems Group at "Sep 15, 2000 02:35:50 pm" To: Cy Schubert - ITSD Open Systems Group Date: Fri, 15 Sep 2000 17:06:57 -0700 (PDT) Cc: Daniel Eischen , Will Andrews , arch@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL61 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Cy Schubert - ITSD Open Systems Group wrote: > > So what! That's the price of security. I believe that the > telnet/ftp/"r" commands shouldn't even be ports. We need to make it > difficult to install unsafe software on the system. That way the admin > would have to go to all the trouble to find the source for unsafe > software somewhere on the Net, port it, and install it. Then it's not > FreeBSD's fault if that admin's system is compromised. > This is a somewhat myoptic view of the world. If I didn't read your sig, I would have thought you worked with only FreeBSD boxes. When I got the OSF1 box several years ago, I intended to use amanda for remote back-ups. Well, amanda wasn't 64-bit clean, and I had an helluva time trying to get OSF1 and FreeBSD talking. I haven't tried amanda lately so may be it works nows, but having rdump/rrstore on OSF1 and /etc/rmt of FreeBSD made it straight forward to back-up the OSF1 system. FreeBSD provides the bullets. It up to the admin to shoot his foot or not. -- Steve To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 17:40:38 2000 Delivered-To: freebsd-arch@freebsd.org Received: from Awfulhak.org (tun.AwfulHak.org [194.242.139.173]) by hub.freebsd.org (Postfix) with ESMTP id 1A15137B423; Fri, 15 Sep 2000 17:40:31 -0700 (PDT) Received: from hak.lan.Awfulhak.org (root@hak.lan.awfulhak.org [172.16.0.12]) by Awfulhak.org (8.11.0/8.11.0) with ESMTP id e8G0b3v29208; Sat, 16 Sep 2000 01:37:03 +0100 (BST) (envelope-from brian@hak.lan.Awfulhak.org) Received: from hak.lan.Awfulhak.org (brian@localhost [127.0.0.1]) by hak.lan.Awfulhak.org (8.11.0/8.11.0) with ESMTP id e8G0b2n00679; Sat, 16 Sep 2000 01:37:02 +0100 (BST) (envelope-from brian@hak.lan.Awfulhak.org) Message-Id: <200009160037.e8G0b2n00679@hak.lan.Awfulhak.org> X-Mailer: exmh version 2.1.1 10/15/1999 To: Robert Watson Cc: freebsd-arch@FreeBSD.org, brian@Awfulhak.org Subject: Re: sysctl MIB and kernel internals In-Reply-To: Message from Robert Watson of "Fri, 15 Sep 2000 12:26:56 EDT." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sat, 16 Sep 2000 01:37:02 +0100 From: Brian Somers Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > I'd like to see us consider moving to an alternative model, divorcing the > implementation internals of various kernel objects (processes, et al) from > the MIB interface retrieving management data about them. I.e., struct > proc would continue to be used in kernel, but relevant fields would be > copied to struct export_proc for export via sysctl. In addition, it would > be worth prefixing these exported structures with a version number > allowing the caller to determine if they support an appropriate version of > the interface, allowing a more comprehensible error. Only fields > desirable to export would be in export_proc, so if an extra pointer is > added to struct ucred (recent resource control changes, capabilities), an > extra pointer to struct proc (jail), etc won't needless break userland > tools. > > This would also have the effect of allowing us to document the MIB, rather > than just say, ``err. go see the kernel source''. One other thing worth adding - a length if it's not there: struct something_exported { int version; size_t len; field1; field2; ... }; This way, extra user-land fields can be added to the end with len increased but version not increased. FWIW Solaris has taken this one step further with their kstat stuff. A driver can add and remove statistics of different types to the kstat tree on-the-fly. The types can be lists of named variables, for example module.instance.name.variable = value that can be read (or written) generically, or ``raw data'' (still having a module, instance and name) with a size and repetition count - treated as a known structure by the kernel & app. There are other types, but I can't remember what they are.... The nice thing about the structured data is that it's easy to add more stuff without breaking user-land. I don't think it's *that* useful though (not useful enough to suggest we add anything to our MIB interface), but worth knowing. > Just a thought. > > Robert N M Watson > > robert@fledge.watson.org http://www.watson.org/~robert/ > PGP key fingerprint: AF B5 5F FF A6 4A 79 37 ED 5F 55 E9 58 04 6A B1 > TIS Labs at Network Associates, Safeport Network Services -- Brian Don't _EVER_ lose your sense of humour ! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 18:10:47 2000 Delivered-To: freebsd-arch@freebsd.org Received: from mail.rpi.edu (mail.rpi.edu [128.113.100.7]) by hub.freebsd.org (Postfix) with ESMTP id BC0AE37B423 for ; Fri, 15 Sep 2000 18:10:44 -0700 (PDT) Received: from [128.113.24.47] (gilead.acs.rpi.edu [128.113.24.47]) by mail.rpi.edu (8.9.3/8.9.3) with ESMTP id VAA193074 for ; Fri, 15 Sep 2000 21:10:43 -0400 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: <200009160006.RAA77706@troutmask.apl.washington.edu> References: <200009160006.RAA77706@troutmask.apl.washington.edu> Date: Fri, 15 Sep 2000 21:11:18 -0400 To: arch@FreeBSD.ORG From: Garance A Drosihn Subject: Re: Rsh/Rlogin/Rcmd & friends Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG To take a different topic for awhile, what things should we change to "default to" ssh instead of rsh, rlogin, etc? For instance, I use cvs to connect to remote repositories. I am used to setting: export CVS_RSH=ssh I also do something (I forget what) so my rsync scripts default to using ssh instead of rsh. is that the kind of thing we should be doing? If so, where would we do it? Modifying each of the ports? Putting the environment variables in the default .cshrc (.bashrc, etc) files? --- Garance Alistair Drosehn = gad@eclipse.acs.rpi.edu Senior Systems Programmer or drosih@rpi.edu Rensselaer Polytechnic Institute To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 18:14:22 2000 Delivered-To: freebsd-arch@freebsd.org Received: from winston.osd.bsdi.com (winston.osd.bsdi.com [204.216.27.229]) by hub.freebsd.org (Postfix) with ESMTP id 3259537B423 for ; Fri, 15 Sep 2000 18:14:20 -0700 (PDT) Received: from winston.osd.bsdi.com (jkh@localhost [127.0.0.1]) by winston.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id e8G1EC698766; Fri, 15 Sep 2000 18:14:12 -0700 (PDT) (envelope-from jkh@winston.osd.bsdi.com) To: Will Andrews , Cy Schubert - ITSD Open Systems Group , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends In-Reply-To: Message from Will Andrews of "Fri, 15 Sep 2000 17:09:31 CDT." <20000915170931.X40658@radon.gryphonsoft.com> Date: Fri, 15 Sep 2000 18:14:12 -0700 Message-ID: <98763.969066852@winston.osd.bsdi.com> From: Jordan Hubbard Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Basic stuff like /bin/cp and other /bin/*, /sbin/* and so forth, with > certain exceptions. All sounds like part of the "base" package myself. > Of course, the only reason why I'd say this is because: wouldn't > "packages" be more difficult to maintain than the source? Nope! In fact, one might expect it to be quite a bit easier. Source upgrades are fraught with manually-navigated rapids of peril. Binary upgrades can contain scripts which at least automate a user's decent into hell instead. > Or am I on crack again? Perhaps it's not that hard at all, and > everything should be packaged. It's not that hard at all, and everything should be packaged. Pass the pipe. :-) - Jordan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 18:14:45 2000 Delivered-To: freebsd-arch@freebsd.org Received: from point.osg.gov.bc.ca (point.osg.gov.bc.ca [142.32.102.44]) by hub.freebsd.org (Postfix) with ESMTP id 45D3037B42C for ; Fri, 15 Sep 2000 18:14:42 -0700 (PDT) Received: (from daemon@localhost) by point.osg.gov.bc.ca (8.8.7/8.8.8) id SAA08958; Fri, 15 Sep 2000 18:13:36 -0700 Received: from passer.osg.gov.bc.ca(142.32.110.29) via SMTP by point.osg.gov.bc.ca, id smtpda08956; Fri Sep 15 18:13:15 2000 Received: (from uucp@localhost) by passer.osg.gov.bc.ca (8.9.3/8.9.1) id SAA62793; Fri, 15 Sep 2000 18:13:12 -0700 (PDT) Received: from cwsys9.cwsent.com(10.2.2.1), claiming to be "cwsys.cwsent.com" via SMTP by passer9.cwsent.com, id smtpdY62787; Fri Sep 15 18:13:00 2000 Received: (from uucp@localhost) by cwsys.cwsent.com (8.11.0/8.9.1) id e8G1D0627243; Fri, 15 Sep 2000 18:13:00 -0700 (PDT) Message-Id: <200009160113.e8G1D0627243@cwsys.cwsent.com> Received: from localhost.cwsent.com(127.0.0.1), claiming to be "cwsys" via SMTP by localhost.cwsent.com, id smtpdK27232; Fri Sep 15 18:12:08 2000 X-Mailer: exmh version 2.1.1 10/15/1999 Reply-To: Cy Schubert - ITSD Open Systems Group From: Cy Schubert - ITSD Open Systems Group X-OS: FreeBSD 4.1-RELEASE X-Sender: cy To: Steve Kargl Cc: Cy Schubert - ITSD Open Systems Group , Daniel Eischen , Will Andrews , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends In-reply-to: Your message of "Fri, 15 Sep 2000 17:06:57 PDT." <200009160006.RAA77706@troutmask.apl.washington.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 15 Sep 2000 18:12:07 -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <200009160006.RAA77706@troutmask.apl.washington.edu>, Steve Kargl wr ites: > Cy Schubert - ITSD Open Systems Group wrote: > > > > So what! That's the price of security. I believe that the > > telnet/ftp/"r" commands shouldn't even be ports. We need to make it > > difficult to install unsafe software on the system. That way the admin > > would have to go to all the trouble to find the source for unsafe > > software somewhere on the Net, port it, and install it. Then it's not > > FreeBSD's fault if that admin's system is compromised. > > > > This is a somewhat myoptic view of the world. If I didn't > read your sig, I would have thought you worked with only > FreeBSD boxes. Being that I am consulted UNIX security issues across the BC Government, I advise what an auditor would tell me. My advice is normally conservative from a security auditor's point of view, e.g. disable or remove all services and use or install only what you will use. This advice normally reduces any chance of culpability should something unfortunate happen. Looks like I've been working for government too long. :) > [deleted] > > FreeBSD provides the bullets. It up to the admin to shoot > his foot or not. Something I've been thinking about over for a while is to create a script that would either disable (and re-enable) services or applications via config files and permissions or optionally just delete (no turning back) services and applications -- the admin would choose which mode it would run. Something like this could be distributed in /etc or as a port and could be run by an admin just after install. Would there be any interest in this or would it be a waste of my time? Regards, Phone: (250)387-8437 Cy Schubert Fax: (250)387-5766 Team Leader, Sun/DEC Team Internet: Cy.Schubert@osg.gov.bc.ca Open Systems Group, ITSD, ISTA Province of BC To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 19:21:21 2000 Delivered-To: freebsd-arch@freebsd.org Received: from mppsystems.com (mppsystems.com [208.210.148.205]) by hub.freebsd.org (Postfix) with ESMTP id B5DC337B424 for ; Fri, 15 Sep 2000 19:21:18 -0700 (PDT) Received: (from mpp@localhost) by mppsystems.com (8.9.3/8.9.3) id VAA43306; Fri, 15 Sep 2000 21:20:34 -0500 (CDT) (envelope-from mpp) Date: Fri, 15 Sep 2000 21:20:34 -0500 From: Mike Pritchard To: Cy Schubert - ITSD Open Systems Group Cc: Will Andrews , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends Message-ID: <20000915212034.A43097@mppsystems.com> References: <20000915143515.N40658@radon.gryphonsoft.com> <200009152032.e8FKWfC25897@cwsys.cwsent.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200009152032.e8FKWfC25897@cwsys.cwsent.com>; from Cy.Schubert@uumail.gov.bc.ca on Fri, Sep 15, 2000 at 01:31:52PM -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, Sep 15, 2000 at 01:31:52PM -0700, Cy Schubert - ITSD Open Systems Group wrote: > In message <20000915143515.N40658@radon.gryphonsoft.com>, Will Andrews > writes: > > Ok people. I want a reason why we shouldn't move rsh/rlogin/rcmd out of > > the base system and into ports, now that we can support SSH connections > > out of the box. > > > > And, ahead of the pack of y'all, POLA be damned! > > Yet another reason why everything, not just ports, should be installed > using packages. Then install just what you want. > > Given that packages for parts of the base system are not here today, I > would add telnet and ftp to this list too. They should not be > installed by default. SSH only. I wouldn't add telnet to that list. I can't ssh into any of my routers. Using telnet to access them works. I can alway access them via the serial cable, but sometimes that isn't always practical (sometimes the cable is wired up to a dumb terminal in the computer room). I do always require ssh for external access to any of my machines, so I am pro-ssh. But unless someone makes it easy to install these other utilities via sysinstall, I'm against it. If during installation, I had to option to install/not install telnet, ftp, and the r* commands, it wouldn't be too big of a deal. But, someone has to step forward and make the changes to sysinstall in order to make this happen. If your network gets screwed up, ssh isn't very forgiving about not being able to resolve addresses and the like. telnet, ftp, and the r* commands don't care as much, and you can at least do some recovery and usually get yourself going again. Jordan has said this before. It would be nice to be able to pick and choose exactly what gets installed on your system (and re-built/installed when you do a "make world"), but no one has actually come up with the code to make it happen. -Mike -- Mike Pritchard mpp@FreeBSD.org or mpp@mppsystems.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 19:40:41 2000 Delivered-To: freebsd-arch@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (Postfix) with ESMTP id 2572737B42C for ; Fri, 15 Sep 2000 19:40:38 -0700 (PDT) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.9.3/8.9.3) with ESMTP id UAA77910; Fri, 15 Sep 2000 20:40:35 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id UAA09702; Fri, 15 Sep 2000 20:40:33 -0600 (MDT) Message-Id: <200009160240.UAA09702@harmony.village.org> To: Steve Kargl Subject: Re: Rsh/Rlogin/Rcmd & friends Cc: Will Andrews , arch@FreeBSD.ORG In-reply-to: Your message of "Fri, 15 Sep 2000 12:56:10 PDT." <200009151956.MAA75808@troutmask.apl.washington.edu> References: <200009151956.MAA75808@troutmask.apl.washington.edu> Date: Fri, 15 Sep 2000 20:40:33 -0600 From: Warner Losh Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <200009151956.MAA75808@troutmask.apl.washington.edu> Steve Kargl writes: : What are the consequences of your proposal with the use of : rdump/rrestore from another (non-FreeBSD) machine into a : tape drive equipped FreeBSD box? I have a set of patches to make rdump work over ssh or any other transport program you want. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 19:42:56 2000 Delivered-To: freebsd-arch@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (Postfix) with ESMTP id CAEF437B43E for ; Fri, 15 Sep 2000 19:42:53 -0700 (PDT) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.9.3/8.9.3) with ESMTP id UAA77917; Fri, 15 Sep 2000 20:42:52 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id UAA09717; Fri, 15 Sep 2000 20:42:49 -0600 (MDT) Message-Id: <200009160242.UAA09717@harmony.village.org> To: Cy Schubert - ITSD Open Systems Group Subject: Re: Rsh/Rlogin/Rcmd & friends Cc: Hajimu UMEMOTO , will@physics.purdue.edu, arch@FreeBSD.ORG In-reply-to: Your message of "Fri, 15 Sep 2000 13:59:53 PDT." <200009152100.e8FL0ne26068@cwsys.cwsent.com> References: <200009152100.e8FL0ne26068@cwsys.cwsent.com> Date: Fri, 15 Sep 2000 20:42:49 -0600 From: Warner Losh Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <200009152100.e8FL0ne26068@cwsys.cwsent.com> Cy Schubert - ITSD Open Systems Group writes: : I would rather we remove rsh entirely. If you want to use rsh install : the GNU tar port. Actually, I think we should use rcmd that I'm putting into dump. More general and consistant. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 20:50:39 2000 Delivered-To: freebsd-arch@freebsd.org Received: from peach.ocn.ne.jp (peach.ocn.ne.jp [210.145.254.87]) by hub.freebsd.org (Postfix) with ESMTP id C48CA37B42C for ; Fri, 15 Sep 2000 20:50:33 -0700 (PDT) Received: from newsguy.com (p12-dn02kiryunisiki.gunma.ocn.ne.jp [211.0.245.77]) by peach.ocn.ne.jp (8.9.1a/OCN/) with ESMTP id MAA23703; Sat, 16 Sep 2000 12:49:28 +0900 (JST) Message-ID: <39C2EB20.73116B08@newsguy.com> Date: Sat, 16 Sep 2000 12:38:08 +0900 From: "Daniel C. Sobral" X-Mailer: Mozilla 4.7 [en] (Win98; I) X-Accept-Language: en,pt-BR MIME-Version: 1.0 To: Cy Schubert - ITSD Open Systems Group Cc: Daniel Eischen , Will Andrews , Steve Kargl , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends References: <200009152136.e8FLaou26312@cwsys.cwsent.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Cy Schubert - ITSD Open Systems Group wrote: > > So what! That's the price of security. I believe that the > telnet/ftp/"r" commands shouldn't even be ports. We need to make it > difficult to install unsafe software on the system. That way the admin > would have to go to all the trouble to find the source for unsafe > software somewhere on the Net, port it, and install it. Then it's not > FreeBSD's fault if that admin's system is compromised. Excuse me, but we don't need to make "difficult" anything. You go admin YOUR machines the way you want, but you don't f*cking come tell me how I should admin mine. I'm all for secure *defaults*, I'm all for warning the user of possibly security risks, but DON'T TREAT ME LIKE MICROSOFT DOES. Period. -- Daniel C. Sobral (8-DCS) dcs@newsguy.com dcs@freebsd.org capo@the.secret.bsdconspiracy.net "I demand that my picture show a handsome face, even if it doesn't look like me." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 23: 3: 1 2000 Delivered-To: freebsd-arch@freebsd.org Received: from orthanc.ab.ca (207-167-15-66.dsl.worldgate.ca [207.167.15.66]) by hub.freebsd.org (Postfix) with ESMTP id 780F737B424 for ; Fri, 15 Sep 2000 23:02:59 -0700 (PDT) Received: from orthanc.ab.ca (localhost [127.0.0.1]) by orthanc.ab.ca (8.11.0.Beta3/8.11.0.Beta3) with ESMTP id e8G62lZ88974; Sat, 16 Sep 2000 00:02:47 -0600 (MDT) Message-Id: <200009160602.e8G62lZ88974@orthanc.ab.ca> To: Will Andrews Cc: arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends In-reply-to: Your message of "Fri, 15 Sep 2000 14:35:15 CDT." <20000915143515.N40658@radon.gryphonsoft.com> Date: Sat, 16 Sep 2000 00:02:46 -0600 From: Lyndon Nerenberg Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG >>>>> "Will" == Will Andrews writes: Will> Ok people. I want a reason why we shouldn't move Will> rsh/rlogin/rcmd out of the base system and into ports, now Will> that we can support SSH connections out of the box. Because rsh/rlogin (I haven't look at rexec()) support kerberos. --lyndon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 23:15: 7 2000 Delivered-To: freebsd-arch@freebsd.org Received: from orthanc.ab.ca (207-167-15-66.dsl.worldgate.ca [207.167.15.66]) by hub.freebsd.org (Postfix) with ESMTP id CE36737B422 for ; Fri, 15 Sep 2000 23:15:05 -0700 (PDT) Received: from orthanc.ab.ca (localhost [127.0.0.1]) by orthanc.ab.ca (8.11.0.Beta3/8.11.0.Beta3) with ESMTP id e8G6EqZ89121; Sat, 16 Sep 2000 00:14:52 -0600 (MDT) Message-Id: <200009160614.e8G6EqZ89121@orthanc.ab.ca> To: Warner Losh Cc: arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends In-reply-to: Your message of "Fri, 15 Sep 2000 20:40:33 MDT." <200009160240.UAA09702@harmony.village.org> Date: Sat, 16 Sep 2000 00:14:51 -0600 From: Lyndon Nerenberg Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG >>>>> "Warner" == Warner Losh writes: Warner> I have a set of patches to make rdump work over ssh or any Warner> other transport program you want. These patches fix the (binary ONLY) dump commands on: Linux AIX HP/UX Tru 64 Solaris IRIX ? (Un)fortunately, the entire universe does not (yet) run FreeBSD. Me, I have to do a lot of backups with rsh. --lyndon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Sep 15 23:27: 2 2000 Delivered-To: freebsd-arch@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (Postfix) with ESMTP id D087A37B424 for ; Fri, 15 Sep 2000 23:26:58 -0700 (PDT) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.9.3/8.9.3) with ESMTP id AAA79004; Sat, 16 Sep 2000 00:26:56 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id AAA11742; Sat, 16 Sep 2000 00:26:54 -0600 (MDT) Message-Id: <200009160626.AAA11742@harmony.village.org> To: Lyndon Nerenberg Subject: Re: Rsh/Rlogin/Rcmd & friends Cc: arch@FreeBSD.ORG In-reply-to: Your message of "Sat, 16 Sep 2000 00:14:51 MDT." <200009160614.e8G6EqZ89121@orthanc.ab.ca> References: <200009160614.e8G6EqZ89121@orthanc.ab.ca> Date: Sat, 16 Sep 2000 00:26:54 -0600 From: Warner Losh Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <200009160614.e8G6EqZ89121@orthanc.ab.ca> Lyndon Nerenberg writes: : These patches fix the (binary ONLY) dump commands on: : : Linux : AIX : HP/UX : Tru 64 : Solaris : IRIX : : ? No. They just allow one to specify an alternative transport. It allows for easy secure transport for remote dumps. I don't see it as a reason to remove rsh/rshd, but do admit that it would be one step closer to that. There's a lot of reasons to move towards this, but we're not close to being there yet. : (Un)fortunately, the entire universe does not (yet) run FreeBSD. Me, I : have to do a lot of backups with rsh. I know. that's why at the very least we should have the ability to install rsh. I'm not sure that eliminating rsh, rlogin and telnet are a desriable thing, but at the same time I'd like to see them die. ftp is used for anonymous ftp, so it should likely remain. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Sep 16 0:11:57 2000 Delivered-To: freebsd-arch@freebsd.org Received: from smtp02.iafrica.com (smtp02.iafrica.com [196.7.0.140]) by hub.freebsd.org (Postfix) with ESMTP id 0EC2F37B423 for ; Sat, 16 Sep 2000 00:11:53 -0700 (PDT) Received: from [196.7.18.138] (helo=grimreaper.grondar.za ident=root) by smtp02.iafrica.com with esmtp (Exim 1.92 #1) id 13aC8a-000ArF-00; Sat, 16 Sep 2000 09:11:48 +0200 Received: from grimreaper.grondar.za (mark@localhost [127.0.0.1]) by grimreaper.grondar.za (8.11.0/8.11.0) with ESMTP id e8G7Bjn05695; Sat, 16 Sep 2000 09:11:45 +0200 (SAST) (envelope-from mark@grimreaper.grondar.za) Message-Id: <200009160711.e8G7Bjn05695@grimreaper.grondar.za> To: Will Andrews Cc: arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends References: <20000915143515.N40658@radon.gryphonsoft.com> In-Reply-To: <20000915143515.N40658@radon.gryphonsoft.com> ; from Will Andrews "Fri, 15 Sep 2000 14:35:15 EST." Date: Sat, 16 Sep 2000 09:11:43 +0200 From: Mark Murray Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Ok people. I want a reason why we shouldn't move rsh/rlogin/rcmd out of > the base system and into ports, now that we can support SSH connections > out of the box. You have my vote! I am of the opinion that thew r-utils should be lead outside and subjected to the bullet-in-the-back-of-the-head style of justice as meted out by certain small governments. M -- Mark Murray Join the anti-SPAM movement: http://www.cauce.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Sep 16 7: 1: 2 2000 Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id EFC9C37B42C for ; Sat, 16 Sep 2000 07:00:59 -0700 (PDT) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id QAA45237; Sat, 16 Sep 2000 16:00:51 +0200 (CEST) (envelope-from des@ofug.org) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Cy Schubert - ITSD Open Systems Group Cc: Daniel Eischen , Will Andrews , Steve Kargl , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends References: <200009152136.e8FLaou26312@cwsys.cwsent.com> From: Dag-Erling Smorgrav Date: 16 Sep 2000 16:00:51 +0200 In-Reply-To: Cy Schubert - ITSD Open Systems Group's message of "Fri, 15 Sep 2000 14:35:50 -0700" Message-ID: Lines: 13 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Cy Schubert - ITSD Open Systems Group writes: > So what! That's the price of security. I believe that the > telnet/ftp/"r" commands shouldn't even be ports. We need to make it > difficult to install unsafe software on the system. That way the admin > would have to go to all the trouble to find the source for unsafe > software somewhere on the Net, port it, and install it. Then it's not > FreeBSD's fault if that admin's system is compromised. I'm glad you're not in charge. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Sep 16 7: 7:38 2000 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 6497D37B42C for ; Sat, 16 Sep 2000 07:07:34 -0700 (PDT) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.0/8.9.3) with ESMTP id e8GE7LN73293; Sat, 16 Sep 2000 16:07:21 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Dag-Erling Smorgrav Cc: Cy Schubert - ITSD Open Systems Group , Daniel Eischen , Will Andrews , Steve Kargl , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends In-Reply-To: Your message of "16 Sep 2000 16:00:51 +0200." Date: Sat, 16 Sep 2000 16:07:21 +0200 Message-ID: <73291.969113241@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message , Dag-Erling Smorgrav writes: >Cy Schubert - ITSD Open Systems Group writes: >> So what! That's the price of security. I believe that the >> telnet/ftp/"r" commands shouldn't even be ports. We need to make it >> difficult to install unsafe software on the system. That way the admin >> would have to go to all the trouble to find the source for unsafe >> software somewhere on the Net, port it, and install it. Then it's not >> FreeBSD's fault if that admin's system is compromised. > >I'm glad you're not in charge. > I can only agree with DES. It is not FreeBSD job to tell people which tools they can or cannot use. FreeBSDs job is to deliver a comprehensive tool-chest. FreeBSD: Tools, not policies. Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD coreteam member | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Sep 16 7:12: 2 2000 Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 0888737B43C; Sat, 16 Sep 2000 07:12:00 -0700 (PDT) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id QAA45275; Sat, 16 Sep 2000 16:11:57 +0200 (CEST) (envelope-from des@ofug.org) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Kris Kennaway Cc: Hajimu UMEMOTO , will@physics.purdue.edu, arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends References: From: Dag-Erling Smorgrav Date: 16 Sep 2000 16:11:56 +0200 In-Reply-To: Kris Kennaway's message of "Fri, 15 Sep 2000 14:18:55 -0700 (PDT)" Message-ID: Lines: 9 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Kris Kennaway writes: > It would probably be better to check and evironment variable TAR_RSH and > use that (like CVS_RSH and others) # ln -s /usr/bin/ssh /bin/rsh DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Sep 16 8: 3:46 2000 Delivered-To: freebsd-arch@freebsd.org Received: from post.mail.nl.demon.net (post-10.mail.nl.demon.net [194.159.73.20]) by hub.freebsd.org (Postfix) with ESMTP id 31E7837B423 for ; Sat, 16 Sep 2000 08:03:44 -0700 (PDT) Received: from [212.238.54.101] (helo=freebie.demon.nl) by post.mail.nl.demon.net with smtp (Exim 3.14 #2) id 13aJVG-0005kK-00; Sat, 16 Sep 2000 15:03:42 +0000 Received: (from wkb@localhost) by freebie.demon.nl (8.11.0/8.11.0) id e8GF4s441157; Sat, 16 Sep 2000 17:04:54 +0200 (CEST) (envelope-from wkb) Date: Sat, 16 Sep 2000 17:04:54 +0200 From: Wilko Bulte To: Dag-Erling Smorgrav Cc: Cy Schubert - ITSD Open Systems Group , Daniel Eischen , Will Andrews , Steve Kargl , arch@freebsd.org Subject: Re: Rsh/Rlogin/Rcmd & friends Message-ID: <20000916170454.A40993@freebie.demon.nl> References: <200009152136.e8FLaou26312@cwsys.cwsent.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: ; from des@ofug.org on Sat, Sep 16, 2000 at 04:00:51PM +0200 X-OS: FreeBSD 4.1-STABLE X-PGP: finger wilko@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sat, Sep 16, 2000 at 04:00:51PM +0200, Dag-Erling Smorgrav wrote: > Cy Schubert - ITSD Open Systems Group writes: > > So what! That's the price of security. I believe that the > > telnet/ftp/"r" commands shouldn't even be ports. We need to make it > > difficult to install unsafe software on the system. That way the admin > > would have to go to all the trouble to find the source for unsafe > > software somewhere on the Net, port it, and install it. Then it's not > > FreeBSD's fault if that admin's system is compromised. > > I'm glad you're not in charge. Extremely glad, I might add.. -- Wilko Bulte wilko@freebsd.org Arnhem, the Netherlands To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Sep 16 8:35:43 2000 Delivered-To: freebsd-arch@freebsd.org Received: from point.osg.gov.bc.ca (point.osg.gov.bc.ca [142.32.102.44]) by hub.freebsd.org (Postfix) with ESMTP id A5B3337B43E for ; Sat, 16 Sep 2000 08:35:39 -0700 (PDT) Received: (from daemon@localhost) by point.osg.gov.bc.ca (8.8.7/8.8.8) id IAA10992; Sat, 16 Sep 2000 08:34:17 -0700 Received: from passer.osg.gov.bc.ca(142.32.110.29) via SMTP by point.osg.gov.bc.ca, id smtpda10990; Sat Sep 16 08:34:17 2000 Received: (from uucp@localhost) by passer.osg.gov.bc.ca (8.9.3/8.9.1) id IAA71670; Sat, 16 Sep 2000 08:34:15 -0700 (PDT) Received: from cwsys9.cwsent.com(10.2.2.1), claiming to be "cwsys.cwsent.com" via SMTP by passer9.cwsent.com, id smtpdJ71668; Sat Sep 16 08:34:14 2000 Received: (from uucp@localhost) by cwsys.cwsent.com (8.11.0/8.9.1) id e8GFYDc32614; Sat, 16 Sep 2000 08:34:13 -0700 (PDT) Message-Id: <200009161534.e8GFYDc32614@cwsys.cwsent.com> Received: from localhost.cwsent.com(127.0.0.1), claiming to be "cwsys" via SMTP by localhost.cwsent.com, id smtpds32608; Sat Sep 16 08:33:17 2000 X-Mailer: exmh version 2.1.1 10/15/1999 Reply-To: Cy Schubert - ITSD Open Systems Group From: Cy Schubert - ITSD Open Systems Group X-OS: FreeBSD 4.1-RELEASE X-Sender: cy To: Mark Murray Cc: Will Andrews , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends In-reply-to: Your message of "Sat, 16 Sep 2000 09:11:43 +0200." <200009160711.e8G7Bjn05695@grimreaper.grondar.za> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sat, 16 Sep 2000 08:33:17 -0700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <200009160711.e8G7Bjn05695@grimreaper.grondar.za>, Mark Murray write s: > > Ok people. I want a reason why we shouldn't move rsh/rlogin/rcmd out of > > the base system and into ports, now that we can support SSH connections > > out of the box. > > You have my vote! > > I am of the opinion that thew r-utils should be lead outside and > subjected to the bullet-in-the-back-of-the-head style of justice > as meted out by certain small governments. It's obvious from the comments I've seen on this list, the comments I've received privately, your comment, and my own comment from yesterday, which by the way sparked a number of negative comments against me personally, that this has become an emotional issue, maybe even a religious issue. I must admit the mistake I made was not to offer a single technical reason for my vote. Looking back at the comments made over the last 24 hours, the majority of the votes made the same mistake. Being IMO a reasonable person -- hey I'm not a jerk all the time, believe me -- :) -- I would like to change my vote, as I've been convinced by those with clearer heads than mine. Even though I've changed my vote in regards to this subject, I do think we can offer a compromise if everyone is willing to step back and look at the bigger picture (sorry, I wasn't looking at the bigger picture -- definitely not management material yesterday). If there is a significant number of people who want this we might be able to put together a port (and I'll volunteer to do it) that will: 1. Remove telent, ftp, and "r" services from inetd.conf, by commenting them out so when the port is deinstalled, the services re-apper. 2. The port would create an mtree file (not replace any existing) that would change the permissions of the telnet, ftp, and "r" commands daemon binaries and client binaries. Those interested in removing the said services would then install the port after every FreeBSD upgrade or make installworld. Now for the vote. As per the rule of argument, I will offer at least three options: 1. Will Andrews' initial proposal. 2. The compromise port just described above. 3. A simple script disable the appropriate services in inetd.conf. I'll donate mine if people want it. 4. Status quo. My vote: If enough people want it, I'd be willing to support options 2 or 3. If not, I've seen enough reasonable arguments than mine to change my vote to option 4. (The bullying and insults didn't change my mind. The few reasonable arguments -- Robert Watson's note comes to mind -- is what did it). Regards, Phone: (250)387-8437 Cy Schubert Fax: (250)387-5766 Team Leader, Sun/DEC Team Internet: Cy.Schubert@osg.gov.bc.ca Open Systems Group, ITSD, ISTA Province of BC To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Sep 16 9:26:34 2000 Delivered-To: freebsd-arch@freebsd.org Received: from peace.mahoroba.org (peace.calm.imasy.or.jp [202.227.26.34]) by hub.freebsd.org (Postfix) with ESMTP id 2CB1037B422; Sat, 16 Sep 2000 09:26:21 -0700 (PDT) Received: from localhost (localhost [::1]) (authenticated) by peace.mahoroba.org (8.11.0/8.11.0/peace) with ESMTP/inet6 id e8GGPjM01016; Sun, 17 Sep 2000 01:25:45 +0900 (JST) (envelope-from ume@mahoroba.org) Date: Sun, 17 Sep 2000 01:25:42 +0900 (JST) Message-Id: <20000917.012542.78700152.ume@mahoroba.org> To: kris@FreeBSD.org Cc: will@physics.purdue.edu, arch@FreeBSD.org Subject: Re: Rsh/Rlogin/Rcmd & friends From: Hajimu UMEMOTO In-Reply-To: References: <20000916055225C.ume@mahoroba.org> X-Mailer: xcite1.20> Mew version 1.95b38 on Emacs 20.7 / Mule 4.0 =?iso-2022-jp?B?KBskQjJWMWMbKEIp?= X-PGP-Public-Key: http://www.imasy.org/~ume/publickey.asc X-PGP-Fingerprint: 6B 0C 53 FC 5D D0 37 91 05 D0 B3 EF 36 9B 6A BC X-URL: http://www.imasy.org/~ume/ X-OS: FreeBSD 5.0-CURRENT Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG >>>>> On Fri, 15 Sep 2000 14:18:55 -0700 (PDT) >>>>> Kris Kennaway said: kris> It would probably be better to check and evironment variable TAR_RSH and kris> use that (like CVS_RSH and others) Yes. I rewrote to use TAR_RSH. Index: gnu/usr.bin/tar/Makefile diff -u gnu/usr.bin/tar/Makefile.orig gnu/usr.bin/tar/Makefile --- gnu/usr.bin/tar/Makefile.orig Tue Sep 22 02:20:08 1998 +++ gnu/usr.bin/tar/Makefile Sun Sep 17 01:03:20 2000 @@ -6,7 +6,7 @@ CFLAGS+= -DHAVE_GETGRGID=1 -DHAVE_GETPWUID=1 -DHAVE_STRING_H=1 CFLAGS+= -DHAVE_LIMITS_H=1 -DHAVE_STRSTR=1 -DHAVE_VALLOC=1 -DHAVE_MKDIR=1 CFLAGS+= -DHAVE_MKNOD=1 -DHAVE_RENAME=1 -DHAVE_FTRUNCATE=1 -DHAVE_GETCWD=1 -CFLAGS+= -DBSD42=1 -DHAVE_VPRINTF=1 -DNEEDPAD -I${.CURDIR} +CFLAGS+= -DBSD42=1 -DHAVE_VPRINTF=1 -DNEEDPAD -DSTDC_HEADERS=1 -I${.CURDIR} CFLAGS+= -DDEF_AR_FILE=\"/dev/rsa0\" -DDEFBLOCKING=20 YFLAGS= NOSHARED?=yes Index: gnu/usr.bin/tar/rtapelib.c diff -u gnu/usr.bin/tar/rtapelib.c.orig gnu/usr.bin/tar/rtapelib.c --- gnu/usr.bin/tar/rtapelib.c.orig Sun Nov 3 23:47:52 1996 +++ gnu/usr.bin/tar/rtapelib.c Sun Sep 17 00:56:22 2000 @@ -276,6 +276,7 @@ char device[CMDBUFSIZE]; /* The remote device name. */ char login[CMDBUFSIZE]; /* The remote user name. */ char *sys, *dev, *user; /* For copying into the above buffers. */ + char *tar_rsh; sys = system; dev = device; @@ -370,19 +371,31 @@ setuid (getuid ()); setgid (getgid ()); + tar_rsh = getenv("TAR_RSH"); + if (*login) { - execl ("/usr/bin/rsh", "rsh", "-l", login, system, - "/etc/rmt", (char *) 0); - execlp ("rsh", "rsh", "-l", login, system, - "/etc/rmt", (char *) 0); + if (tar_rsh) { + execlp (tar_rsh, tar_rsh, "-l", login, system, + "/etc/rmt", (char *) 0); + } else { + execl ("/usr/bin/rsh", "rsh", "-l", login, system, + "/etc/rmt", (char *) 0); + execlp ("rsh", "rsh", "-l", login, system, + "/etc/rmt", (char *) 0); + } } else { - execl ("/usr/bin/rsh", "rsh", system, - "/etc/rmt", (char *) 0); - execlp ("rsh", "rsh", system, - "/etc/rmt", (char *) 0); + if (tar_rsh) { + execlp (tar_rsh, tar_rsh, system, + "/etc/rmt", (char *) 0); + } else { + execl ("/usr/bin/rsh", "rsh", system, + "/etc/rmt", (char *) 0); + execlp ("rsh", "rsh", system, + "/etc/rmt", (char *) 0); + } } /* Bad problems if we get here. */ -- Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan ume@mahoroba.org ume@bisd.hitachi.co.jp ume@FreeBSD.org http://www.imasy.org/~ume/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Sep 16 9:32:13 2000 Delivered-To: freebsd-arch@freebsd.org Received: from mail.inka.de (quechua.inka.de [212.227.14.2]) by hub.freebsd.org (Postfix) with ESMTP id 7173637B422 for ; Sat, 16 Sep 2000 09:32:11 -0700 (PDT) Received: from ganerc.mips.inka.de (uucp@) by mail.inka.de with local-bsmtp id 13aKsr-0001ma-01; Sat, 16 Sep 2000 18:32:09 +0200 Received: (from daemon@localhost) by ganerc.mips.inka.de (8.11.0/8.11.0) id e8GFuLG51313 for freebsd-arch@freebsd.org; Sat, 16 Sep 2000 17:56:21 +0200 (CEST) (envelope-from daemon) From: naddy@mips.inka.de (Christian Weisgerber) Subject: Re: Rsh/Rlogin/Rcmd & friends Date: 16 Sep 2000 17:56:20 +0200 Message-ID: <8q0574$1i33$1@ganerc.mips.inka.de> References: <200009151956.MAA75808@troutmask.apl.washington.edu> <200009160240.UAA09702@harmony.village.org> To: freebsd-arch@freebsd.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Warner Losh wrote: > I have a set of patches to make rdump work over ssh or any other > transport program you want. Strange. When I suggested something like this in PR #15830 the response from the committers was that it was useless and people could just use "dump -f -|ssh" anyway. Make sure to start the transport program in its own process group, otherwise ^C will kill it and dump's offer to continue will become meaningless. -- Christian "naddy" Weisgerber naddy@mips.inka.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Sep 16 9:59:51 2000 Delivered-To: freebsd-arch@freebsd.org Received: from ns1.sunesi.net (ns1.sunesi.net [196.15.192.194]) by hub.freebsd.org (Postfix) with ESMTP id F2DD837B422 for ; Sat, 16 Sep 2000 09:59:46 -0700 (PDT) Received: from nbm by ns1.sunesi.net with local (Exim 3.03 #1) id 13aLJV-000EMM-00; Sat, 16 Sep 2000 18:59:41 +0200 Date: Sat, 16 Sep 2000 18:59:41 +0200 From: Neil Blakey-Milner To: Christian Weisgerber Cc: freebsd-arch@freebsd.org Subject: Re: Rsh/Rlogin/Rcmd & friends Message-ID: <20000916185941.A55093@mithrandr.moria.org> References: <200009151956.MAA75808@troutmask.apl.washington.edu> <200009160240.UAA09702@harmony.village.org> <8q0574$1i33$1@ganerc.mips.inka.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: <8q0574$1i33$1@ganerc.mips.inka.de>; from naddy@mips.inka.de on Sat, Sep 16, 2000 at 05:56:20PM +0200 Organization: Sunesi Clinical Systems X-Operating-System: FreeBSD 3.3-RELEASE i386 X-URL: http://rucus.ru.ac.za/~nbm/ Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sat 2000-09-16 (17:56), Christian Weisgerber wrote: > Warner Losh wrote: > > > I have a set of patches to make rdump work over ssh or any other > > transport program you want. > > Strange. When I suggested something like this in PR #15830 the > response from the committers was that it was useless and people > could just use "dump -f -|ssh" anyway. Looking in freebsd-bugs archive, and the problem report feedback, there was no such response to the PR. If you could point me to where you got negative response from "the committers", I'll be happy to investigate. Warner suggested it was too specific to dump, and suggested the rshcmd from OpenBSD last month. You had since changed your email address, and I presume his response bounced like Sheldon's change of status to your PR bounced. Checking back, we've been forced to close PRs of yours since we've been unable to get feedback from you due to this address change. You might want to investigate the PR database from the web site to determine if these closures were satisfactory to you. If some haven't been closed but have an old address, kindly respond to the PR with your new address. Neil -- Neil Blakey-Milner Sunesi Clinical Systems nbm@mithrandr.moria.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Sep 16 14:24:28 2000 Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 02A2137B422 for ; Sat, 16 Sep 2000 14:24:26 -0700 (PDT) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.9.3/8.9.3) with SMTP id RAA04430; Sat, 16 Sep 2000 17:23:46 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Sat, 16 Sep 2000 17:23:46 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Cy Schubert - ITSD Open Systems Group Cc: Mark Murray , Will Andrews , arch@FreeBSD.ORG Subject: Re: Rsh/Rlogin/Rcmd & friends In-Reply-To: <200009161534.e8GFYDc32614@cwsys.cwsent.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG The port idea is an interesting one -- I actually played with some similar ideas for the FreeBSD hardening project I was working on a couple of years ago. A port would provide an interface to select its behavior, and then strip down components of the system reflecting the needs of the administrator. For example, disable services, setuid binaries, clamp down on directory permissions (especially the user skeleton directory impacting account creation properties). I'd like to see the following, with that idea in mind: 1) inetd is disabled in /etc/defaults/rc.conf (probably already is that way) 2) It can be enabled in sysinstall by turning on Internet network servers in some or another configuration location. 3) Creation of a ports/security/harden port which provides a dialog-like front end for managing mtree application to the base system, allowing disabling of unneeded services. 4) In the long run, moving towards a more granular installation of base system components -- i.e., keep services such as rsh/telnet/etc in the CVS repo and "part of FreeBSD" but allow them to be installed/not installed via sysinstall, and toggled via matching make.conf entries. And we need to think of a name for these services, such as "Cryptography-free Internet Servers" and "Cryptography-free Internet Clients" :-). Robert N M Watson robert@fledge.watson.org http://www.watson.org/~robert/ PGP key fingerprint: AF B5 5F FF A6 4A 79 37 ED 5F 55 E9 58 04 6A B1 TIS Labs at Network Associates, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message