From owner-freebsd-arch Sun Jan 14 6:44:20 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 80F1E37B401 for ; Sun, 14 Jan 2001 06:43:57 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f0EEhu902269 for arch@freebsd.org; Sun, 14 Jan 2001 06:43:56 -0800 (PST) Date: Sun, 14 Jan 2001 06:43:56 -0800 From: Alfred Perlstein To: arch@freebsd.org Subject: kthread_* api pretty rough Message-ID: <20010114064356.Q7240@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I was pretty excited to see that we're using kthread_ stuff, however actually using it is a bit more complex than I'd hoped. There's a couple of questions that I didn't see answered in the manpage: 1) how to actually start the thread? kern/kern_idle.c and kern/kern_intr.c both have somewhat different things that they fiddle with in the proc struct returned by kthread_create. 2) is simply calling kthread_exit(); enough to clean up the thread? 3) what are the state of the locks when the thread runs initially? what about what state they need to be in before calling kthread_exit()? 4) why we need to fiddle with the proc flags after starting a kthread instead of having kthread_create do it for us. 5) do i really want to be using kproc_start instead of kthread_create? I just want to run a couple of threads to do some microbenchmarks on some code I'm testing. 6) shouldn't the stuff to OR with the process flags be made into an arg to kthread_create? thanks, -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 15 10:38:16 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 9B4B937B402; Mon, 15 Jan 2001 10:37:57 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f0FIbvh11454; Mon, 15 Jan 2001 10:37:57 -0800 (PST) Date: Mon, 15 Jan 2001 10:37:57 -0800 From: Alfred Perlstein To: Hajimu UMEMOTO Cc: arch@FreeBSD.org Subject: dynamic vs static sysctls? Message-ID: <20010115103757.B7240@fw.wintelcom.net> References: <20010116.025742.74757685.ume@FreeBSD.org> <20010115100618.Y7240@fw.wintelcom.net> <20010116.033215.41625863.ume@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010116.033215.41625863.ume@FreeBSD.org>; from ume@FreeBSD.org on Tue, Jan 16, 2001 at 03:32:15AM +0900 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG [moved to -arch] * Hajimu UMEMOTO [010115 10:33] wrote: > Hi, > > >>>>> On Mon, 15 Jan 2001 10:06:18 -0800 > >>>>> Alfred Perlstein said: > > bright> * Hajimu UMEMOTO [010115 10:00] wrote: > > Hi, > > > > I wish to obtain number of processes forked since boot from userland. > > So, I made a patch to intend to commit. > > Any comment? > > bright> Why not just use a dynamic sysctl for this? > > I think dynamic sysctl is useful for dynamic context. But, here is > just static and it seems there is no advantage. Isn't it? Well all the sysctl's I've added have been dynamic, I think the only reason for the 'static' sysctls is to give sysctl() a numeric way to get at the sysctls, which isn't very useful when we have getsysctlbyname(). Using a dynamic sysctl It would reduce the delta by quite a bit. The stuff your patch does seems to allow programs to use the old (IMHO) depricated sysctl() versus getsysctlbyname() function. No one I know wants to use sysctl instead of getsysctlbyname afaik, however, I would like to know if my opinions are the what we're aiming for. Any other comments? Besideds the way the sysctl is done, the change is pretty nice to see, but will need mplocking later. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 15 10:51:50 2001 Delivered-To: freebsd-arch@freebsd.org Received: from peace.mahoroba.org (peace.calm.imasy.or.jp [202.227.26.34]) by hub.freebsd.org (Postfix) with ESMTP id 2CE0D37B404 for ; Mon, 15 Jan 2001 10:51:30 -0800 (PST) Received: from localhost (IDENT:Rcdf20X0AKvR4gDa7Zt14EsizVWdyDUXZ73FUGhIMnJGkApY3D5ZQK0IHVFppA61@localhost [::1]) (authenticated) by peace.mahoroba.org (8.11.2/8.11.2/peace) with ESMTP/inet6 id f0FIo9w01770; Tue, 16 Jan 2001 03:50:09 +0900 (JST) (envelope-from ume@FreeBSD.org) Date: Tue, 16 Jan 2001 03:50:09 +0900 (JST) Message-Id: <20010116.035009.71081161.ume@FreeBSD.org> To: bright@wintelcom.net Cc: arch@FreeBSD.org Subject: Re: dynamic vs static sysctls? From: Hajimu UMEMOTO In-Reply-To: <20010115103757.B7240@fw.wintelcom.net> References: <20010115100618.Y7240@fw.wintelcom.net> <20010116.033215.41625863.ume@FreeBSD.org> <20010115103757.B7240@fw.wintelcom.net> X-Mailer: xcite1.38> Mew version 1.95b97 on Emacs 20.7 / Mule 4.0 =?iso-2022-jp?B?KBskQjJWMWMbKEIp?= X-PGP-Public-Key: http://www.imasy.org/~ume/publickey.asc X-PGP-Fingerprint: 6B 0C 53 FC 5D D0 37 91 05 D0 B3 EF 36 9B 6A BC X-URL: http://www.imasy.org/~ume/ X-OS: FreeBSD 5.0-CURRENT Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG >>>>> On Mon, 15 Jan 2001 10:37:57 -0800 >>>>> Alfred Perlstein said: bright> Well all the sysctl's I've added have been dynamic, I think the bright> only reason for the 'static' sysctls is to give sysctl() a bright> numeric way to get at the sysctls, which isn't very useful bright> when we have getsysctlbyname(). You mean dynamic sysctl is just use of OID_AUTO, right? I thought SYSCTL_ADD_INT(). bright> Using a dynamic sysctl It would reduce the delta by quite a bit. Indeed. bright> The stuff your patch does seems to allow programs to use the old bright> (IMHO) depricated sysctl() versus getsysctlbyname() function. bright> No one I know wants to use sysctl instead of getsysctlbyname afaik, bright> however, I would like to know if my opinions are the what we're bright> aiming for. Okey, I'll change to use OID_AUTO. bright> Any other comments? One question. How can I detect newly added sysctl? In static, I can just do by #ifdef OID. bright> Besideds the way the sysctl is done, the change is pretty nice bright> to see, but will need mplocking later. Do you mean ALLPROC_LOCK(AP_EXCLUSIVE) / ALLPROC_LOCK(AP_RELEASE) ? If so, I'll move nforks++ between them. -- Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan ume@mahoroba.org ume@bisd.hitachi.co.jp ume@{,jp.}FreeBSD.org http://www.imasy.org/~ume/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 15 11:14:57 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 8F6A037B401; Mon, 15 Jan 2001 11:14:39 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f0FJEdx12466; Mon, 15 Jan 2001 11:14:39 -0800 (PST) Date: Mon, 15 Jan 2001 11:14:39 -0800 From: Alfred Perlstein To: Hajimu UMEMOTO Cc: arch@FreeBSD.org Subject: Re: dynamic vs static sysctls? Message-ID: <20010115111439.C7240@fw.wintelcom.net> References: <20010115100618.Y7240@fw.wintelcom.net> <20010116.033215.41625863.ume@FreeBSD.org> <20010115103757.B7240@fw.wintelcom.net> <20010116.035009.71081161.ume@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010116.035009.71081161.ume@FreeBSD.org>; from ume@FreeBSD.org on Tue, Jan 16, 2001 at 03:50:09AM +0900 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Hajimu UMEMOTO [010115 10:51] wrote: > >>>>> On Mon, 15 Jan 2001 10:37:57 -0800 > >>>>> Alfred Perlstein said: > > bright> Well all the sysctl's I've added have been dynamic, I think the > bright> only reason for the 'static' sysctls is to give sysctl() a > bright> numeric way to get at the sysctls, which isn't very useful > bright> when we have getsysctlbyname(). > > You mean dynamic sysctl is just use of OID_AUTO, right? I thought > SYSCTL_ADD_INT(). > > bright> Using a dynamic sysctl It would reduce the delta by quite a bit. > > Indeed. > > bright> The stuff your patch does seems to allow programs to use the old > bright> (IMHO) depricated sysctl() versus getsysctlbyname() function. > > bright> No one I know wants to use sysctl instead of getsysctlbyname afaik, > bright> however, I would like to know if my opinions are the what we're > bright> aiming for. > > Okey, I'll change to use OID_AUTO. Just do this: static unsigned int nforks = 0; SYSCTL_UINT(_kern, OID_AUTO, nforks, CTLFLAG_RD, &nforks, 0, "number of fork()s since boot"); document the sysctl. :) > > bright> Any other comments? > > One question. How can I detect newly added sysctl? In static, I can > just do by #ifdef OID. Oh, you mean in a kernel module? I think you need to use kernel_sysctl() (from kern/kern_sysctl.c), I'm not sure if it's documented anywhere, if not it'd be nice to have a sysctl(9) manpage. :) > bright> Besideds the way the sysctl is done, the change is pretty nice > bright> to see, but will need mplocking later. > > Do you mean ALLPROC_LOCK(AP_EXCLUSIVE) / ALLPROC_LOCK(AP_RELEASE) ? > If so, I'll move nforks++ between them. No, don't do that, we'll figure out how to lock it later, you may want to use the mp_fixme (mpfixme?) macro near the sysctl variable though. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 15 11:44:45 2001 Delivered-To: freebsd-arch@freebsd.org Received: from peace.mahoroba.org (peace.calm.imasy.or.jp [202.227.26.34]) by hub.freebsd.org (Postfix) with ESMTP id A307537B400 for ; Mon, 15 Jan 2001 11:44:25 -0800 (PST) Received: from localhost (IDENT:7SJytFdI1vPyN6QO893Uhj5vlanE7URwFVWeh5EGKSt1dQTpLCkxQLNJxYAlPD/v@localhost [::1]) (authenticated) by peace.mahoroba.org (8.11.2/8.11.2/peace) with ESMTP/inet6 id f0FJh6w02406; Tue, 16 Jan 2001 04:43:06 +0900 (JST) (envelope-from ume@FreeBSD.org) Date: Tue, 16 Jan 2001 04:43:05 +0900 (JST) Message-Id: <20010116.044305.78700228.ume@FreeBSD.org> To: bright@wintelcom.net Cc: arch@FreeBSD.org Subject: Re: dynamic vs static sysctls? From: Hajimu UMEMOTO In-Reply-To: <20010115111439.C7240@fw.wintelcom.net> References: <20010115103757.B7240@fw.wintelcom.net> <20010116.035009.71081161.ume@FreeBSD.org> <20010115111439.C7240@fw.wintelcom.net> X-Mailer: xcite1.38> Mew version 1.95b97 on Emacs 20.7 / Mule 4.0 =?iso-2022-jp?B?KBskQjJWMWMbKEIp?= X-PGP-Public-Key: http://www.imasy.org/~ume/publickey.asc X-PGP-Fingerprint: 6B 0C 53 FC 5D D0 37 91 05 D0 B3 EF 36 9B 6A BC X-URL: http://www.imasy.org/~ume/ X-OS: FreeBSD 5.0-CURRENT Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG >>>>> On Mon, 15 Jan 2001 11:14:39 -0800 >>>>> Alfred Perlstein said: bright> Just do this: bright> static unsigned int nforks = 0; bright> SYSCTL_UINT(_kern, OID_AUTO, nforks, CTLFLAG_RD, &nforks, 0, bright> "number of fork()s since boot"); bright> document the sysctl. :) Okey. Thanks. bright> Oh, you mean in a kernel module? I think you need to use bright> kernel_sysctl() (from kern/kern_sysctl.c), I'm not sure if it's bright> documented anywhere, if not it'd be nice to have a sysctl(9) manpage. bright> :) No. I just want to distinguish the version from userland. > Do you mean ALLPROC_LOCK(AP_EXCLUSIVE) / ALLPROC_LOCK(AP_RELEASE) ? > If so, I'll move nforks++ between them. bright> No, don't do that, we'll figure out how to lock it later, you may bright> want to use the mp_fixme (mpfixme?) macro near the sysctl variable bright> though. Okey, I see. -- Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan ume@mahoroba.org ume@bisd.hitachi.co.jp ume@{,jp.}FreeBSD.org http://www.imasy.org/~ume/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 15 12:17:14 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id B6F5C37B698; Mon, 15 Jan 2001 12:16:56 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f0FKGuh14523; Mon, 15 Jan 2001 12:16:56 -0800 (PST) Date: Mon, 15 Jan 2001 12:16:56 -0800 From: Alfred Perlstein To: Hajimu UMEMOTO Cc: arch@FreeBSD.ORG Subject: Re: dynamic vs static sysctls? Message-ID: <20010115121656.D7240@fw.wintelcom.net> References: <20010115103757.B7240@fw.wintelcom.net> <20010116.035009.71081161.ume@FreeBSD.org> <20010115111439.C7240@fw.wintelcom.net> <20010116.044305.78700228.ume@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010116.044305.78700228.ume@FreeBSD.org>; from ume@FreeBSD.ORG on Tue, Jan 16, 2001 at 04:43:05AM +0900 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Hajimu UMEMOTO [010115 11:47] wrote: > >>>>> On Mon, 15 Jan 2001 11:14:39 -0800 > >>>>> Alfred Perlstein said: > > bright> Just do this: > > bright> static unsigned int nforks = 0; > bright> SYSCTL_UINT(_kern, OID_AUTO, nforks, CTLFLAG_RD, &nforks, 0, > bright> "number of fork()s since boot"); > > bright> document the sysctl. :) > > Okey. Thanks. > > bright> Oh, you mean in a kernel module? I think you need to use > bright> kernel_sysctl() (from kern/kern_sysctl.c), I'm not sure if it's > bright> documented anywhere, if not it'd be nice to have a sysctl(9) manpage. > bright> :) > > No. I just want to distinguish the version from userland. I think sysctl probably returns something like ENOENT if getsysctlbyname fails. If not it should. > > Do you mean ALLPROC_LOCK(AP_EXCLUSIVE) / ALLPROC_LOCK(AP_RELEASE) ? > > If so, I'll move nforks++ between them. > > bright> No, don't do that, we'll figure out how to lock it later, you may > bright> want to use the mp_fixme (mpfixme?) macro near the sysctl variable > bright> though. > > Okey, I see. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 15 14:17:37 2001 Delivered-To: freebsd-arch@freebsd.org Received: from meow.osd.bsdi.com (meow.osd.bsdi.com [204.216.28.88]) by hub.freebsd.org (Postfix) with ESMTP id 0871B37B401; Mon, 15 Jan 2001 14:17:17 -0800 (PST) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by meow.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id f0FMFk194839; Mon, 15 Jan 2001 14:15:46 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20010115103757.B7240@fw.wintelcom.net> Date: Mon, 15 Jan 2001 14:17:19 -0800 (PST) From: John Baldwin To: Alfred Perlstein Subject: RE: dynamic vs static sysctls? Cc: arch@FreeBSD.org, Hajimu UMEMOTO Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 15-Jan-01 Alfred Perlstein wrote: > [moved to -arch] > > * Hajimu UMEMOTO [010115 10:33] wrote: >> Hi, >> >> >>>>> On Mon, 15 Jan 2001 10:06:18 -0800 >> >>>>> Alfred Perlstein said: >> >> bright> * Hajimu UMEMOTO [010115 10:00] wrote: >> > Hi, >> > >> > I wish to obtain number of processes forked since boot from userland. >> > So, I made a patch to intend to commit. >> > Any comment? >> >> bright> Why not just use a dynamic sysctl for this? >> >> I think dynamic sysctl is useful for dynamic context. But, here is >> just static and it seems there is no advantage. Isn't it? > > Well all the sysctl's I've added have been dynamic, I think the > only reason for the 'static' sysctls is to give sysctl() a > numeric way to get at the sysctls, which isn't very useful > when we have getsysctlbyname(). Actually, he means dynamic as in SYSCTL_ADD_*. I.e., sysctls created at runtime, not compile time like SYSCTL_*. One should use static sysctls if possible as they don't need malloc() at runtime, etc. Dynamic sysctls are good for providing sysctl access to variables that are dynamically generated. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 15 14:17:37 2001 Delivered-To: freebsd-arch@freebsd.org Received: from meow.osd.bsdi.com (meow.osd.bsdi.com [204.216.28.88]) by hub.freebsd.org (Postfix) with ESMTP id A151C37B400 for ; Mon, 15 Jan 2001 14:17:16 -0800 (PST) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by meow.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id f0FMFk194835; Mon, 15 Jan 2001 14:15:46 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20010114064356.Q7240@fw.wintelcom.net> Date: Mon, 15 Jan 2001 14:17:19 -0800 (PST) From: John Baldwin To: Alfred Perlstein Subject: RE: kthread_* api pretty rough Cc: arch@FreeBSD.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 14-Jan-01 Alfred Perlstein wrote: > I was pretty excited to see that we're using kthread_ stuff, however > actually using it is a bit more complex than I'd hoped. > > There's a couple of questions that I didn't see answered in the > manpage: > > 1) how to actually start the thread? kern/kern_idle.c and > kern/kern_intr.c both have somewhat different things that they > fiddle with in the proc struct returned by kthread_create. They are special. Just use kthread_create(). > 2) is simply calling kthread_exit(); enough to clean up the thread? Yes. Note that right now you need to have Giant before you call it. > 3) what are the state of the locks when the thread runs initially? > what about what state they need to be in before calling kthread_exit()? Nothing is held when it starts. Giant and only Giant should be held when calling exit. > 4) why we need to fiddle with the proc flags after starting a kthread > instead of having kthread_create do it for us. You don't. > 5) do i really want to be using kproc_start instead of kthread_create? > I just want to run a couple of threads to do some microbenchmarks > on some code I'm testing. kproc_start() is a function used in SYSINIT()'s to start up things like bufdaemon, etc. (Did you read the manpage? :) > 6) shouldn't the stuff to OR with the process flags be made into > an arg to kthread_create? Most users don't set anything in the process flags that I know of.. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 15 20:23:54 2001 Delivered-To: freebsd-arch@freebsd.org Received: from beastie.mckusick.com (beastie.mckusick.com [209.31.233.184]) by hub.freebsd.org (Postfix) with ESMTP id D8C8A37B401 for ; Mon, 15 Jan 2001 20:23:36 -0800 (PST) Received: from beastie.mckusick.com (localhost [127.0.0.1]) by beastie.mckusick.com (8.9.3/8.9.3) with ESMTP id PAA22257; Mon, 15 Jan 2001 15:45:35 -0800 (PST) (envelope-from mckusick@beastie.mckusick.com) Message-Id: <200101152345.PAA22257@beastie.mckusick.com> To: Alfred Perlstein Subject: Re: dynamic vs static sysctls? Cc: arch@FreeBSD.ORG In-Reply-To: Your message of "Mon, 15 Jan 2001 10:37:57 PST." <20010115103757.B7240@fw.wintelcom.net> Date: Mon, 15 Jan 2001 15:45:35 -0800 From: Kirk McKusick Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Date: Mon, 15 Jan 2001 10:37:57 -0800 From: Alfred Perlstein To: Hajimu UMEMOTO Cc: arch@FreeBSD.ORG Subject: dynamic vs static sysctls? [moved to -arch] ... No one I know wants to use sysctl instead of getsysctlbyname afaik, however, I would like to know if my opinions are the what we're aiming for. ... -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." I have an example where sysctl is still useful. In my work on a background version of fsck, I have used sysctl to allow me to pass information into the kernel that I want to have updated in the filesystem. In the case of lost blocks, there may be hundreds or even thousands of blocks that need to be put back into the bit maps. Each non-contiguous block is passed in separately which results in hundreds or thousands of sysctl calls. I want to do one call to sysctlnametomib (a new, but obviously trivial function) to return the numeric mib, and then use that mib on all the subsequent sysctl calls. That dramatically cuts down on the time it takes to return the blocks since I do not have to translate the same name repeatedly (which takes far longer than the block release itself). I would think that this might be an issue for any application that uses sysctl to get/set the same information repeatedly. Kirk McKusick To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 15 22:34: 2 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 3A05B37B400 for ; Mon, 15 Jan 2001 22:33:43 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f0G6XgG00716; Mon, 15 Jan 2001 22:33:42 -0800 (PST) Date: Mon, 15 Jan 2001 22:33:42 -0800 From: Alfred Perlstein To: Kirk McKusick Cc: arch@FreeBSD.ORG Subject: Re: dynamic vs static sysctls? Message-ID: <20010115223342.L7240@fw.wintelcom.net> References: <20010115103757.B7240@fw.wintelcom.net> <200101152345.PAA22257@beastie.mckusick.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200101152345.PAA22257@beastie.mckusick.com>; from mckusick@mckusick.com on Mon, Jan 15, 2001 at 03:45:35PM -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Kirk McKusick [010115 20:23] wrote: > Date: Mon, 15 Jan 2001 10:37:57 -0800 > From: Alfred Perlstein > To: Hajimu UMEMOTO > Cc: arch@FreeBSD.ORG > Subject: dynamic vs static sysctls? > > [moved to -arch] > > ... > > No one I know wants to use sysctl instead of getsysctlbyname afaik, > however, I would like to know if my opinions are the what we're > aiming for. > > ... > > -- > -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] > "I have the heart of a child; I keep it in a jar on my desk." > > I have an example where sysctl is still useful. In my work on a > background version of fsck, I have used sysctl to allow me to > pass information into the kernel that I want to have updated in > the filesystem. In the case of lost blocks, there may be hundreds > or even thousands of blocks that need to be put back into the > bit maps. Each non-contiguous block is passed in separately > which results in hundreds or thousands of sysctl calls. I want > to do one call to sysctlnametomib (a new, but obviously trivial > function) to return the numeric mib, and then use that mib on all > the subsequent sysctl calls. That dramatically cuts down on the > time it takes to return the blocks since I do not have to translate > the same name repeatedly (which takes far longer than the block > release itself). I would think that this might be an issue for > any application that uses sysctl to get/set the same information > repeatedly. I like this optimization, the only problem is that there's a chance that a dynamic sysctl may disapear out from under you, and even worse, replaced by an unrelated writeable one. Perhaps you should include a generation count within the sysctl to prevent this sort of potential problem? So besideds passing the sysctl number, you would include a generation count and error out if both numbers didn't match? -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 15 23:12:18 2001 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id DCB3D37B400 for ; Mon, 15 Jan 2001 23:11:59 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id f0G7BoZ43116; Tue, 16 Jan 2001 08:11:50 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Kirk McKusick Cc: Alfred Perlstein , arch@FreeBSD.ORG Subject: Re: dynamic vs static sysctls? In-Reply-To: Your message of "Mon, 15 Jan 2001 15:45:35 PST." <200101152345.PAA22257@beastie.mckusick.com> Date: Tue, 16 Jan 2001 08:11:50 +0100 Message-ID: <43114.979629110@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <200101152345.PAA22257@beastie.mckusick.com>, Kirk McKusick writes: > No one I know wants to use sysctl instead of getsysctlbyname afaik, > however, I would like to know if my opinions are the what we're > aiming for. > >I want >to do one call to sysctlnametomib (a new, but obviously trivial >function) to return the numeric mib, and then use that mib on all >the subsequent sysctl calls. As new-sysctl "designer" I think this is a good idea. It's a trivial thing to implement, so are there any warm and able bodies who want to try their hands at it and send the patch to me for review ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jan 15 23:13:52 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mass.osd.bsdi.com (mass.dis.org [216.240.45.41]) by hub.freebsd.org (Postfix) with ESMTP id C0A8B37B401 for ; Mon, 15 Jan 2001 23:13:35 -0800 (PST) Received: from mass.osd.bsdi.com (localhost [127.0.0.1]) by mass.osd.bsdi.com (8.11.1/8.11.1) with ESMTP id f0G7Rss00920; Mon, 15 Jan 2001 23:27:54 -0800 (PST) (envelope-from msmith@mass.osd.bsdi.com) Message-Id: <200101160727.f0G7Rss00920@mass.osd.bsdi.com> X-Mailer: exmh version 2.1.1 10/15/1999 To: Kirk McKusick Cc: arch@FreeBSD.ORG Subject: Re: dynamic vs static sysctls? In-reply-to: Your message of "Mon, 15 Jan 2001 15:45:35 PST." <200101152345.PAA22257@beastie.mckusick.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 15 Jan 2001 23:27:54 -0800 From: Mike Smith Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > I have an example where sysctl is still useful. In my work on a > background version of fsck, I have used sysctl to allow me to > pass information into the kernel that I want to have updated in > the filesystem. In the case of lost blocks, there may be hundreds > or even thousands of blocks that need to be put back into the > bit maps. Each non-contiguous block is passed in separately > which results in hundreds or thousands of sysctl calls. I'm not convinced that sysctl is the "right" way to go about doing this, really. But I can't think of a better one. 8) > I want > to do one call to sysctlnametomib (a new, but obviously trivial > function) to return the numeric mib, and then use that mib on all > the subsequent sysctl calls. This functionality already exists; it's simply undocumented and doesn't have a convenient library wrapper function. See eg. the sysctlbyname implementation, kenv, ifconfig, etc. > That dramatically cuts down on the > time it takes to return the blocks since I do not have to translate > the same name repeatedly (which takes far longer than the block > release itself). I would think that this might be an issue for > any application that uses sysctl to get/set the same information > repeatedly. It is. It's also useful in the case where you use trailing fields in the MIB to pass arguments to the sysctl handler. -- ... every activity meets with opposition, everyone who acts has his rivals and unfortunately opponents also. But not because people want to be opponents, rather because the tasks and relationships force people to take different points of view. [Dr. Fritz Todt] V I C T O R Y N O T V E N G E A N C E To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 16 6:18:48 2001 Delivered-To: freebsd-arch@freebsd.org Received: from ringworld.nanolink.com (ringworld.nanolink.com [195.24.48.189]) by hub.freebsd.org (Postfix) with SMTP id EDDBF37B699 for ; Tue, 16 Jan 2001 06:18:24 -0800 (PST) Received: (qmail 8033 invoked by uid 1000); 16 Jan 2001 14:17:04 -0000 Date: Tue, 16 Jan 2001 16:17:03 +0200 From: Peter Pentchev To: arch@FreeBSD.org Subject: no newgroup/newgrp in FreeBSD? Message-ID: <20010116161703.H364@ringworld.oblivion.bg> Mail-Followup-To: arch@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Heyas, Is there a reason there is no newgroup/newgrp(1) in FreeBSD, besides the fact that the kernel can support a simultaneous set of groups? Still, it would be nice for compatibility (and SUSv2 compliance) to have some kind of newgrp/newgroup implementation, albeit even a null one. On http://ringwraith.online.bg/~roam/devel/c/sysutils/newgrp-1.0.tar.gz there is something that just might do the trick (although I'm not quite clear on the handling of supplementary groups according to SUSv2..) G'luck, Peter PS. And yes, I've heard before that FreeBSD is still quite far from SUSv2 compliance; still, I had to write this for a friend with some very weird shell/web hosting requirements, who did not want to change his existing scripts' modus operandi.. and then I thought, what the hell, somebody might be interested in that, or just tell me why I've wasted my time :) PPS. And yes, I do realize that in FreeBSD, the supplementary groups act much the same as the primary group.. but still.. :) -- If this sentence were in Chinese, it would say something else. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 16 6:23:57 2001 Delivered-To: freebsd-arch@freebsd.org Received: from ringworld.nanolink.com (ringworld.nanolink.com [195.24.48.189]) by hub.freebsd.org (Postfix) with SMTP id 923D837B404 for ; Tue, 16 Jan 2001 06:23:37 -0800 (PST) Received: (qmail 8138 invoked by uid 1000); 16 Jan 2001 14:22:19 -0000 Date: Tue, 16 Jan 2001 16:22:19 +0200 From: Peter Pentchev To: arch@FreeBSD.org Subject: Re: no newgroup/newgrp in FreeBSD? Message-ID: <20010116162219.I364@ringworld.oblivion.bg> Mail-Followup-To: arch@FreeBSD.org References: <20010116161703.H364@ringworld.oblivion.bg> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010116161703.H364@ringworld.oblivion.bg>; from roam@orbitel.bg on Tue, Jan 16, 2001 at 04:17:03PM +0200 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, Jan 16, 2001 at 04:17:03PM +0200, Peter Pentchev wrote: > Heyas, > > Is there a reason there is no newgroup/newgrp(1) in FreeBSD, besides > the fact that the kernel can support a simultaneous set of groups? > Still, it would be nice for compatibility (and SUSv2 compliance) > to have some kind of newgrp/newgroup implementation, albeit even > a null one. > > On http://ringwraith.online.bg/~roam/devel/c/sysutils/newgrp-1.0.tar.gz Uhm. Oops. Remove the c/ part. http://ringwraith.online.bg/~roam/devel/sysutils/newgrp-1.0.tar.gz Sorry, and thanks to Chris Faulhaber for pointing this out. G'luck, Peter -- Thit sentence is not self-referential because "thit" is not a word. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 16 6:41:53 2001 Delivered-To: freebsd-arch@freebsd.org Received: from puck.firepipe.net (mcut-b-167.resnet.purdue.edu [128.211.209.167]) by hub.freebsd.org (Postfix) with ESMTP id 911CC37B402 for ; Tue, 16 Jan 2001 06:41:34 -0800 (PST) Received: by puck.firepipe.net (Postfix, from userid 1000) id E6A1618F0; Tue, 16 Jan 2001 09:41:33 -0500 (EST) Date: Tue, 16 Jan 2001 09:41:33 -0500 From: Will Andrews To: Peter Pentchev Cc: arch@FreeBSD.ORG Subject: Re: no newgroup/newgrp in FreeBSD? Message-ID: <20010116094133.B1858@puck.firepipe.net> Reply-To: Will Andrews Mail-Followup-To: Will Andrews , Peter Pentchev , arch@FreeBSD.ORG References: <20010116161703.H364@ringworld.oblivion.bg> <20010116162219.I364@ringworld.oblivion.bg> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="+g7M9IMkV8truYOl" Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010116162219.I364@ringworld.oblivion.bg>; from roam@orbitel.bg on Tue, Jan 16, 2001 at 04:22:19PM +0200 X-Operating-System: FreeBSD 4.2-STABLE i386 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG --+g7M9IMkV8truYOl Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jan 16, 2001 at 04:22:19PM +0200, Peter Pentchev wrote: > http://ringwraith.online.bg/~roam/devel/sysutils/newgrp-1.0.tar.gz [..] Sorry for being ignorant, but what's the difference between this and the pw(8) group operations? --=20 wca --+g7M9IMkV8truYOl Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (FreeBSD) Comment: For info see http://www.gnupg.org iD8DBQE6ZF2dF47idPgWcsURAu59AJ4yVa/FN3nTQlP+HK7obskfCk4SeACdHygY XeO9jAuOy8TPb6jERi9tbeE= =oSFQ -----END PGP SIGNATURE----- --+g7M9IMkV8truYOl-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 16 6:58:34 2001 Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id C90DB37B400 for ; Tue, 16 Jan 2001 06:58:09 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id PAA37513; Tue, 16 Jan 2001 15:57:53 +0100 (CET) (envelope-from des@ofug.org) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Peter Pentchev Cc: arch@FreeBSD.ORG Subject: Re: no newgroup/newgrp in FreeBSD? References: <20010116161703.H364@ringworld.oblivion.bg> From: Dag-Erling Smorgrav Date: 16 Jan 2001 15:57:52 +0100 In-Reply-To: Peter Pentchev's message of "Tue, 16 Jan 2001 16:17:03 +0200" Message-ID: Lines: 12 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Peter Pentchev writes: > On http://ringwraith.online.bg/~roam/devel/c/sysutils/newgrp-1.0.tar.gz > there is something that just might do the trick (although I'm not > quite clear on the handling of supplementary groups according to > SUSv2..) Just add links to pw(8) with the appropriate names, and modify pw(8) do DTRT if invoked with a different basename that "pw". DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 16 7:30:31 2001 Delivered-To: freebsd-arch@freebsd.org Received: from point.osg.gov.bc.ca (point.osg.gov.bc.ca [142.32.102.44]) by hub.freebsd.org (Postfix) with ESMTP id 38ED937B401 for ; Tue, 16 Jan 2001 07:30:12 -0800 (PST) Received: (from daemon@localhost) by point.osg.gov.bc.ca (8.8.7/8.8.8) id HAA03067; Tue, 16 Jan 2001 07:29:21 -0800 Received: from passer.osg.gov.bc.ca(142.32.110.29) via SMTP by point.osg.gov.bc.ca, id smtpda03064; Tue Jan 16 07:29:13 2001 Received: (from uucp@localhost) by passer.osg.gov.bc.ca (8.11.2/8.9.1) id f0GFT8j13755; Tue, 16 Jan 2001 07:29:08 -0800 (PST) Received: from cwsys9.cwsent.com(10.2.2.1), claiming to be "cwsys.cwsent.com" via SMTP by passer9.cwsent.com, id smtpdc13750; Tue Jan 16 07:28:26 2001 Received: (from uucp@localhost) by cwsys.cwsent.com (8.11.2/8.9.1) id f0GFSPE22987; Tue, 16 Jan 2001 07:28:25 -0800 (PST) Message-Id: <200101161528.f0GFSPE22987@cwsys.cwsent.com> Received: from localhost.cwsent.com(127.0.0.1), claiming to be "cwsys" via SMTP by localhost.cwsent.com, id smtpdy22975; Tue Jan 16 07:28:04 2001 X-Mailer: exmh version 2.2 06/23/2000 with nmh-1.0.4 Reply-To: Cy Schubert - ITSD Open Systems Group From: Cy Schubert - ITSD Open Systems Group X-OS: FreeBSD 4.2-RELEASE X-Sender: cy To: Will Andrews Cc: Peter Pentchev , arch@FreeBSD.ORG Subject: Re: no newgroup/newgrp in FreeBSD? In-reply-to: Your message of "Tue, 16 Jan 2001 09:41:33 EST." <20010116094133.B1858@puck.firepipe.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 16 Jan 2001 07:28:04 -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <20010116094133.B1858@puck.firepipe.net>, Will Andrews writes: > On Tue, Jan 16, 2001 at 04:22:19PM +0200, Peter Pentchev wrote: > > http://ringwraith.online.bg/~roam/devel/sysutils/newgrp-1.0.tar.gz > [..] > > Sorry for being ignorant, but what's the difference between this and the > pw(8) group operations? Newgrp(1) is a SYSV thing and doesn't fit in the Berkeley paradigm. Under SYSV files/directories are created with the primary group that a process is running under. Under BSD (and SYSV under a directory with the sticky bit set) the group ownership of a created file/directory is determined by the group of the directory under which the file/directory is to be created. In short, newgrp is not needed on a BSD system, as it would accomplish nothing in regards file creation group ownership criteria, which is opposite to SYSV semantics. Regards, Phone: (250)387-8437 Cy Schubert Fax: (250)387-5766 Team Leader, Sun/Alpha Team Internet: Cy.Schubert@osg.gov.bc.ca Open Systems Group, ITSD, ISTA Province of BC To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 16 12:10:45 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail.rpi.edu (mail.rpi.edu [128.113.100.7]) by hub.freebsd.org (Postfix) with ESMTP id EBFE437B401 for ; Tue, 16 Jan 2001 12:10:27 -0800 (PST) Received: from [128.113.24.47] (gilead.acs.rpi.edu [128.113.24.47]) by mail.rpi.edu (8.9.3/8.9.3) with ESMTP id PAA322366; Tue, 16 Jan 2001 15:10:24 -0500 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: <20010116161703.H364@ringworld.oblivion.bg> References: <20010116161703.H364@ringworld.oblivion.bg> Date: Tue, 16 Jan 2001 15:10:22 -0500 To: Peter Pentchev , arch@FreeBSD.ORG From: Garance A Drosihn Subject: Re: no newgroup/newgrp in FreeBSD? Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG At 4:17 PM +0200 1/16/01, Peter Pentchev wrote: >PS. And yes, I've heard before that FreeBSD is still quite far > from SUSv2 compliance; still, I had to write this for a friend > with some very weird shell/web hosting requirements, who did > not want to change his existing scripts' modus operandi.. If your friend is worried about SingleUnixSpec V2, he should read this quote from the description of newgrp in that specification: The newgrp command is intended only for use from an interactive terminal. It does not offer a useful interface for the support of applications. The exit status of newgrp is generally inapplicable. If newgrp is used in a script, in most cases it will successfully invoke a new shell and the rest of the original shell script will be bypassed when the new shell exits. So while it might be debated whether FreeBSD should have any newgrp command at all, the specification makes it clear that your friend should not be using it in any script. Thus, tell your friend to change his scripts, "in order to comply with SUSv2". -- Garance Alistair Drosehn = gad@eclipse.acs.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 16 14:10:47 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mobile.wemm.org (c1315225-a.plstn1.sfba.home.com [65.0.135.147]) by hub.freebsd.org (Postfix) with ESMTP id B673B37B69C for ; Tue, 16 Jan 2001 14:10:24 -0800 (PST) Received: from netplex.com.au (localhost [127.0.0.1]) by mobile.wemm.org (8.11.1/8.11.1) with ESMTP id f0GMAFs17644; Tue, 16 Jan 2001 14:10:15 -0800 (PST) (envelope-from peter@netplex.com.au) Message-Id: <200101162210.f0GMAFs17644@mobile.wemm.org> X-Mailer: exmh version 2.2 06/23/2000 with nmh-1.0.4 To: Will Andrews Cc: Peter Pentchev , arch@FreeBSD.ORG Subject: Re: no newgroup/newgrp in FreeBSD? In-Reply-To: <20010116094133.B1858@puck.firepipe.net> Date: Tue, 16 Jan 2001 14:10:15 -0800 From: Peter Wemm Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Will Andrews wrote: > On Tue, Jan 16, 2001 at 04:22:19PM +0200, Peter Pentchev wrote: > > http://ringwraith.online.bg/~roam/devel/sysutils/newgrp-1.0.tar.gz > [..] > > Sorry for being ignorant, but what's the difference between this and the > pw(8) group operations? > > --=20 > wca No, newgrp/newgroup is like su, but for systems with one gid only. It is setuid-root and sometimes also called sg(1). Suppose you are listed in /etc/ passwd as having a primary group of "foo" and you are also listed in /etc/ group as a member of 'bar'... You could then do a 'newgrp bar' and you would get a new shell that had a gid of 'bar' rather than 'foo'. This functionality does not have any place in FreeBSD as "all groups in the groups vector are equal". We could simply provide a non-setuid wrapper for running a new command with no changes... That would be compliant with the interface.. Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 16 14:36:12 2001 Delivered-To: freebsd-arch@freebsd.org Received: from alive.znep.com (sense-sea-MegaSub-1-500.oz.net [216.39.145.246]) by hub.freebsd.org (Postfix) with ESMTP id 158E537B402 for ; Tue, 16 Jan 2001 14:35:55 -0800 (PST) Received: from localhost (marcs@localhost) by alive.znep.com (8.9.3/8.9.1) with ESMTP id OAA13197; Tue, 16 Jan 2001 14:35:38 -0800 (PST) (envelope-from marcs@znep.com) Date: Tue, 16 Jan 2001 14:35:37 -0800 (PST) From: Marc Slemko To: Peter Wemm Cc: Will Andrews , Peter Pentchev , arch@FreeBSD.ORG Subject: Re: no newgroup/newgrp in FreeBSD? In-Reply-To: <200101162210.f0GMAFs17644@mobile.wemm.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, 16 Jan 2001, Peter Wemm wrote: > This functionality does not have any place in FreeBSD as "all groups in > the groups vector are equal". We could simply provide a non-setuid wrapper > for running a new command with no changes... That would be compliant with > the interface.. newgrp is also of use when your group membership in /etc/groups has been changed after you logged in and were setup with the appropriate group list. You login, are not a member of group freeporn, then someone adds you to group freeporn, and "newgrp freeporn" will let you get free porn without logging in again, etc. No? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 16 14:44:23 2001 Delivered-To: freebsd-arch@freebsd.org Received: from meow.osd.bsdi.com (meow.osd.bsdi.com [204.216.28.88]) by hub.freebsd.org (Postfix) with ESMTP id B97A137B698 for ; Tue, 16 Jan 2001 14:44:04 -0800 (PST) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by meow.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id f0GMfpL20477; Tue, 16 Jan 2001 14:41:51 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Tue, 16 Jan 2001 14:43:40 -0800 (PST) From: John Baldwin To: Marc Slemko Subject: Re: no newgroup/newgrp in FreeBSD? Cc: arch@FreeBSD.org, Peter Pentchev , Will Andrews , Peter Wemm Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 16-Jan-01 Marc Slemko wrote: > On Tue, 16 Jan 2001, Peter Wemm wrote: > >> This functionality does not have any place in FreeBSD as "all groups in >> the groups vector are equal". We could simply provide a non-setuid wrapper >> for running a new command with no changes... That would be compliant with >> the interface.. > > newgrp is also of use when your group membership in /etc/groups has been > changed after you logged in and were setup with the appropriate group > list. > > You login, are not a member of group freeporn, then someone adds you to > group freeporn, and "newgrp freeporn" will let you get free porn without > logging in again, etc. > > No? /usr/bin/login -f ${LOGNAME} -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 16 14:51: 0 2001 Delivered-To: freebsd-arch@freebsd.org Received: from alive.znep.com (sense-sea-MegaSub-1-500.oz.net [216.39.145.246]) by hub.freebsd.org (Postfix) with ESMTP id 54B7E37B6A4; Tue, 16 Jan 2001 14:50:42 -0800 (PST) Received: from localhost (marcs@localhost) by alive.znep.com (8.9.3/8.9.1) with ESMTP id OAA13309; Tue, 16 Jan 2001 14:50:41 -0800 (PST) (envelope-from marcs@znep.com) Date: Tue, 16 Jan 2001 14:50:41 -0800 (PST) From: Marc Slemko To: John Baldwin Cc: arch@FreeBSD.org Subject: Re: no newgroup/newgrp in FreeBSD? In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, 16 Jan 2001, John Baldwin wrote: > > On 16-Jan-01 Marc Slemko wrote: > > On Tue, 16 Jan 2001, Peter Wemm wrote: > > > >> This functionality does not have any place in FreeBSD as "all groups in > >> the groups vector are equal". We could simply provide a non-setuid wrapper > >> for running a new command with no changes... That would be compliant with > >> the interface.. > > > > newgrp is also of use when your group membership in /etc/groups has been > > changed after you logged in and were setup with the appropriate group > > list. > > > > You login, are not a member of group freeporn, then someone adds you to > > group freeporn, and "newgrp freeporn" will let you get free porn without > > logging in again, etc. > > > > No? > > /usr/bin/login -f ${LOGNAME} A non setuid wrapper would still not provide the same functionality that newgrp does, which is part of what I'm repsonding to. And login -f does not preserve things like environment, cwd, etc. like newgrp does which matters when you have environment variables that vary (eg. ssh authentication agent). And you are then logged in twice. Sure, there are lots of other ways to do nearly the same thing. Including simply logging out and logging in again. The point is simply that newgrp would not be a noop on freebsd if it were implemented and does have some useful, if minor, functionality. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 16 15: 9:45 2001 Delivered-To: freebsd-arch@freebsd.org Received: from beastie.mckusick.com (machine.qwest.net [204.154.239.5]) by hub.freebsd.org (Postfix) with ESMTP id 3E23537B400 for ; Tue, 16 Jan 2001 15:09:25 -0800 (PST) Received: from beastie.mckusick.com (localhost [127.0.0.1]) by beastie.mckusick.com (8.9.3/8.9.3) with ESMTP id PAA23554; Tue, 16 Jan 2001 15:06:18 -0800 (PST) (envelope-from mckusick@beastie.mckusick.com) Message-Id: <200101162306.PAA23554@beastie.mckusick.com> To: Poul-Henning Kamp Subject: Re: dynamic vs static sysctls? Cc: Alfred Perlstein , arch@FreeBSD.ORG In-Reply-To: Your message of "Tue, 16 Jan 2001 08:11:50 +0100." <43114.979629110@critter> Date: Tue, 16 Jan 2001 15:06:18 -0800 From: Kirk McKusick Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG To: Kirk McKusick Subject: Re: dynamic vs static sysctls? In-Reply-To: Your message of "Mon, 15 Jan 2001 15:45:35 PST." <200101152345.PAA22257@beastie.mckusick.com> Date: Tue, 16 Jan 2001 08:11:50 +0100 From: Poul-Henning Kamp >I want >to do one call to sysctlnametomib (a new, but obviously trivial >function) to return the numeric mib, and then use that mib on all >the subsequent sysctl calls. As new-sysctl "designer" I think this is a good idea. It's a trivial thing to implement, so are there any warm and able bodies who want to try their hands at it and send the patch to me for review ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Here is my proposal (along with a test program for it). Kirk =-=-=-=-=-=-=-=-= #include #include int sysctlnametomib __P((char *name, int *oidp, size_t *sizep)); /* * This function uses a presently undocumented interface to the kernel * to walk the tree and get the type so it can print the value. * This interface is under work and consideration, and should probably * be killed with a big axe by the first person who can find the time. * (be aware though, that the proper interface isn't as obvious as it * may seem, there are various conflicting requirements. */ int sysctlnametomib(char *name, int *oidp, size_t *sizep) { int oid[2]; int error; oid[0] = 0; oid[1] = 3; *sizep *= sizeof (int); error = sysctl(oid, 2, oidp, sizep, name, strlen(name)); *sizep /= sizeof (int); if (error < 0) return (error); return (0); } /* * Test program for sysctlnametomib. * * Usage: sysctlnametomib name */ main(ac, av) int ac; char *av[]; { int i, mib[CTL_MAXNAME]; size_t size; if (ac < 2) { printf("Usage: %s name\n", av[0]); exit(1); } size = CTL_MAXNAME; if (sysctlnametomib(av[1], mib, &size) < 0) { perror("sysctlnametomib"); exit(1); } for (i = 0; i < size; i++) printf("mib[%d] = %d\n", i, mib[i]); exit(0); } To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 16 15:10:45 2001 Delivered-To: freebsd-arch@freebsd.org Received: from point.osg.gov.bc.ca (point.osg.gov.bc.ca [142.32.102.44]) by hub.freebsd.org (Postfix) with ESMTP id B370237B698; Tue, 16 Jan 2001 15:10:25 -0800 (PST) Received: (from daemon@localhost) by point.osg.gov.bc.ca (8.8.7/8.8.8) id PAA04373; Tue, 16 Jan 2001 15:09:53 -0800 Received: from passer.osg.gov.bc.ca(142.32.110.29) via SMTP by point.osg.gov.bc.ca, id smtpda04369; Tue Jan 16 15:09:35 2001 Received: (from uucp@localhost) by passer.osg.gov.bc.ca (8.11.2/8.9.1) id f0GN9U616447; Tue, 16 Jan 2001 15:09:30 -0800 (PST) Received: from cwsys9.cwsent.com(10.2.2.1), claiming to be "cwsys.cwsent.com" via SMTP by passer9.cwsent.com, id smtpdj16445; Tue Jan 16 15:08:34 2001 Received: (from uucp@localhost) by cwsys.cwsent.com (8.11.2/8.9.1) id f0GN8Xt24503; Tue, 16 Jan 2001 15:08:33 -0800 (PST) Message-Id: <200101162308.f0GN8Xt24503@cwsys.cwsent.com> Received: from localhost.cwsent.com(127.0.0.1), claiming to be "cwsys" via SMTP by localhost.cwsent.com, id smtpdE24499; Tue Jan 16 15:08:21 2001 X-Mailer: exmh version 2.2 06/23/2000 with nmh-1.0.4 Reply-To: Cy Schubert - ITSD Open Systems Group From: Cy Schubert - ITSD Open Systems Group X-OS: FreeBSD 4.2-RELEASE X-Sender: cy To: Marc Slemko Cc: John Baldwin , arch@FreeBSD.ORG Subject: Re: no newgroup/newgrp in FreeBSD? In-reply-to: Your message of "Tue, 16 Jan 2001 14:50:41 PST." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 16 Jan 2001 15:08:21 -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message , Marc Slemk o writes: > On Tue, 16 Jan 2001, John Baldwin wrote: > > > > > On 16-Jan-01 Marc Slemko wrote: > > > On Tue, 16 Jan 2001, Peter Wemm wrote: > > > > > >> This functionality does not have any place in FreeBSD as "all groups in > > >> the groups vector are equal". We could simply provide a non-setuid wrap > per > > >> for running a new command with no changes... That would be compliant wi > th > > >> the interface.. > > > > > > newgrp is also of use when your group membership in /etc/groups has been > > > changed after you logged in and were setup with the appropriate group > > > list. > > > > > > You login, are not a member of group freeporn, then someone adds you to > > > group freeporn, and "newgrp freeporn" will let you get free porn without > > > logging in again, etc. > > > > > > No? > > > > /usr/bin/login -f ${LOGNAME} > > A non setuid wrapper would still not provide the same functionality that > newgrp does, which is part of what I'm repsonding to. > > And login -f does not preserve things like environment, cwd, etc. like > newgrp does which matters when you have environment variables that vary > (eg. ssh authentication agent). And you are then logged in twice. > > Sure, there are lots of other ways to do nearly the same thing. > Including simply logging out and logging in again. The point is simply > that newgrp would not be a noop on freebsd if it were implemented and does > have some useful, if minor, functionality. I fail to see how login -f or logging out and in again would provide what the originator of this thread wants to do. Nor do I see how a noop would fill the bill either. Simply put, newgrp is a SYSVism that cannot emulated under BSD. I suppose we could implement the sgid bit for directories to implement SYSV behaviour when during file/directory creation -- the reverse of SYSV's sgid bit for directories -- and along with it a newgrp program. Then again, why? If I wanted to use a SYSV system instead of a BSD system I'd use Solaris x86 instead of FreeBSD. Linux uses SYSV semantics as well so it would be an alternative to Solaris. Regards, Phone: (250)387-8437 Cy Schubert Fax: (250)387-5766 Team Leader, Sun/Alpha Team Internet: Cy.Schubert@osg.gov.bc.ca Open Systems Group, ITSD, ISTA Province of BC To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 16 15:29:26 2001 Delivered-To: freebsd-arch@freebsd.org Received: from tao.org.uk (genesis.tao.org.uk [194.242.131.94]) by hub.freebsd.org (Postfix) with ESMTP id D449537B400; Tue, 16 Jan 2001 15:29:08 -0800 (PST) Received: by tao.org.uk (Postfix, from userid 100) id A06E031BA; Tue, 16 Jan 2001 23:29:08 +0000 (GMT) Date: Tue, 16 Jan 2001 23:29:08 +0000 From: Josef Karthauser To: Marc Slemko Cc: John Baldwin , arch@FreeBSD.ORG Subject: Re: no newgroup/newgrp in FreeBSD? Message-ID: <20010116232908.D1731@tao.org.uk> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from marcs@znep.com on Tue, Jan 16, 2001 at 02:50:41PM -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, Jan 16, 2001 at 02:50:41PM -0800, Marc Slemko wrote: > > A non setuid wrapper would still not provide the same functionality that > newgrp does, which is part of what I'm repsonding to. > > And login -f does not preserve things like environment, cwd, etc. like > newgrp does which matters when you have environment variables that vary > (eg. ssh authentication agent). And you are then logged in twice. I've often wished for this, when in sysadmin mode and logged into a remote server over a difficult to replicate method :) (like ssh over ppp over brian;). Joe > > Sure, there are lots of other ways to do nearly the same thing. > Including simply logging out and logging in again. The point is simply > that newgrp would not be a noop on freebsd if it were implemented and does > have some useful, if minor, functionality. > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 16 17:24:26 2001 Delivered-To: freebsd-arch@freebsd.org Received: from smtp03.primenet.com (smtp03.primenet.com [206.165.6.133]) by hub.freebsd.org (Postfix) with ESMTP id 3B63137B69D for ; Tue, 16 Jan 2001 17:24:07 -0800 (PST) Received: (from daemon@localhost) by smtp03.primenet.com (8.9.3/8.9.3) id SAA26728; Tue, 16 Jan 2001 18:21:24 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp03.primenet.com, id smtpdAAAToayj0; Tue Jan 16 18:21:18 2001 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id SAA19503; Tue, 16 Jan 2001 18:23:57 -0700 (MST) From: Terry Lambert Message-Id: <200101170123.SAA19503@usr08.primenet.com> Subject: Re: dynamic vs static sysctls? To: mckusick@mckusick.com (Kirk McKusick) Date: Wed, 17 Jan 2001 01:23:57 +0000 (GMT) Cc: bright@wintelcom.net (Alfred Perlstein), arch@FreeBSD.ORG In-Reply-To: <200101152345.PAA22257@beastie.mckusick.com> from "Kirk McKusick" at Jan 15, 2001 03:45:35 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG ... > I want > to do one call to sysctlnametomib (a new, but obviously trivial > function) to return the numeric mib, and then use that mib on all > the subsequent sysctl calls. That dramatically cuts down on the > time it takes to return the blocks since I do not have to translate > the same name repeatedly (which takes far longer than the block > release itself). I would think that this might be an issue for > any application that uses sysctl to get/set the same information > repeatedly. Dynamic content isn't the only thing, though. I've used a dynamically generated sysctl for a while to pass information out from the kernel to user space for statistical purposes. In general, this would be a better way of exposing things like load average, assuming you could get the tools converted. In my static implementation, I used a user space catalog file to do the name/OID translation, since I didn't want to put all that stuff in the kernel anyway. I'm a bit concerned about the transient vs. permanent information (a different axis from the static vs. dynamic, IMO). The only thing I could think of was to do what the NetWare bindery emulation in NDS does, which is to reset the transient portion of the data each time the system reboots, and have a "transient" flag in the compiled database, and use the compiled database for all my lookups (you need both indices anyway, so flat files lose anyway). I'm really not happy with the approach, due to Cached Data Considered Harmful... This smacks too much of cached data (e.g. the IP addresses of interfaces in sendmail, which don't get updated when they are changed out from under it via ifconfig, like they should be, or the cached data in DNS that gets there via the protocol, but bind doesn't notice third party config file changes, like cron is wont to do). But I really can't think of a nicer way that doesn't have me shoving string data that's not a filesystem path across the user/kernel boundary, in both directions. 8-(. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 16 17:26:10 2001 Delivered-To: freebsd-arch@freebsd.org Received: from smtp02.primenet.com (smtp02.primenet.com [206.165.6.132]) by hub.freebsd.org (Postfix) with ESMTP id 9D8FA37B69F for ; Tue, 16 Jan 2001 17:25:52 -0800 (PST) Received: (from daemon@localhost) by smtp02.primenet.com (8.9.3/8.9.3) id SAA23983; Tue, 16 Jan 2001 18:20:22 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp02.primenet.com, id smtpdAAA67aqWU; Tue Jan 16 18:20:15 2001 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id SAA19549; Tue, 16 Jan 2001 18:25:43 -0700 (MST) From: Terry Lambert Message-Id: <200101170125.SAA19549@usr08.primenet.com> Subject: Re: dynamic vs static sysctls? To: bright@wintelcom.net (Alfred Perlstein) Date: Wed, 17 Jan 2001 01:25:43 +0000 (GMT) Cc: mckusick@mckusick.com (Kirk McKusick), arch@FreeBSD.ORG In-Reply-To: <20010115223342.L7240@fw.wintelcom.net> from "Alfred Perlstein" at Jan 15, 2001 10:33:42 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > I like this optimization, the only problem is that there's a chance > that a dynamic sysctl may disapear out from under you, and even > worse, replaced by an unrelated writeable one. > > Perhaps you should include a generation count within the sysctl to > prevent this sort of potential problem? So besideds passing the > sysctl number, you would include a generation count and error out > if both numbers didn't match? Dynamically created OIDs should: 1) Monotonically increase 2) Be illegal to cache over reboots 3) Reset on reboots Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 16 17:29:20 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 3A6BF37B69F for ; Tue, 16 Jan 2001 17:29:01 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f0H1Sxx00994; Tue, 16 Jan 2001 17:28:59 -0800 (PST) Date: Tue, 16 Jan 2001 17:28:59 -0800 From: Alfred Perlstein To: Terry Lambert Cc: Kirk McKusick , arch@FreeBSD.ORG Subject: Re: dynamic vs static sysctls? Message-ID: <20010116172859.B7240@fw.wintelcom.net> References: <20010115223342.L7240@fw.wintelcom.net> <200101170125.SAA19549@usr08.primenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200101170125.SAA19549@usr08.primenet.com>; from tlambert@primenet.com on Wed, Jan 17, 2001 at 01:25:43AM +0000 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Terry Lambert [010116 17:25] wrote: > > I like this optimization, the only problem is that there's a chance > > that a dynamic sysctl may disapear out from under you, and even > > worse, replaced by an unrelated writeable one. > > > > Perhaps you should include a generation count within the sysctl to > > prevent this sort of potential problem? So besideds passing the > > sysctl number, you would include a generation count and error out > > if both numbers didn't match? > > Dynamically created OIDs should: > > 1) Monotonically increase They don't seem to, this is the main problem with doing any translation, it looks like: /* * If this oid has a number OID_AUTO, give it a number which * is greater than any current oid. Make sure it is at least * 100 to leave space for pre-assigned oid numbers. */ if (oidp->oid_number == OID_AUTO) { /* First, find the highest oid in the parent list >99 */ n = 99; SLIST_FOREACH(p, parent, oid_link) { if (p->oid_number > n) n = p->oid_number; } oidp->oid_number = n + 1; } Could cause some issues because it seems that if a sysctl were to "go away" the next one to apear under the same level will replace its number. > > 2) Be illegal to cache over reboots of course! > > 3) Reset on reboots ditto. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 16 17:30:32 2001 Delivered-To: freebsd-arch@freebsd.org Received: from smtp10.phx.gblx.net (smtp10.phx.gblx.net [206.165.6.140]) by hub.freebsd.org (Postfix) with ESMTP id 0BB7437B401; Tue, 16 Jan 2001 17:30:15 -0800 (PST) Received: (from daemon@localhost) by smtp10.phx.gblx.net (8.9.3/8.9.3) id SAA133208; Tue, 16 Jan 2001 18:30:06 -0700 Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp10.phx.gblx.net, id smtpdeScfya; Tue Jan 16 18:30:05 2001 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id SAA19626; Tue, 16 Jan 2001 18:30:12 -0700 (MST) From: Terry Lambert Message-Id: <200101170130.SAA19626@usr08.primenet.com> Subject: Re: dynamic vs static sysctls? To: msmith@FreeBSD.ORG (Mike Smith) Date: Wed, 17 Jan 2001 01:30:12 +0000 (GMT) Cc: mckusick@mckusick.com (Kirk McKusick), arch@FreeBSD.ORG In-Reply-To: <200101160727.f0G7Rss00920@mass.osd.bsdi.com> from "Mike Smith" at Jan 15, 2001 11:27:54 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > I would think that this might be an issue for > > any application that uses sysctl to get/set the same information > > repeatedly. > > It is. It's also useful in the case where you use trailing fields in the > MIB to pass arguments to the sysctl handler. It's also useful for externalizing a sub-arc opaquely for standard sub-arcs. For example, the Network Interface Monitoring MIB contents, being externalized via systctl to a user space SNMP implementation, and recast in generic form, not dependent on the FrBSD OID-arc. This also has the benefit of standardizing SNMP consoles, as well as letting us standardize between BSD's, without playing "dueling OID-arc origins" (e.g. "BSDI" vs. "FreeBSD" vs. "NetBSD" vs. "XXX" is not a problem, if you can just point at an RFC and smile). Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 16 17:44:55 2001 Delivered-To: freebsd-arch@freebsd.org Received: from smtp02.primenet.com (smtp02.primenet.com [206.165.6.132]) by hub.freebsd.org (Postfix) with ESMTP id 08ED137B401 for ; Tue, 16 Jan 2001 17:44:38 -0800 (PST) Received: (from daemon@localhost) by smtp02.primenet.com (8.9.3/8.9.3) id SAA00871; Tue, 16 Jan 2001 18:39:07 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp02.primenet.com, id smtpdAAAY2aGKb; Tue Jan 16 18:38:57 2001 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id SAA20025; Tue, 16 Jan 2001 18:44:21 -0700 (MST) From: Terry Lambert Message-Id: <200101170144.SAA20025@usr08.primenet.com> Subject: Re: dynamic vs static sysctls? To: bright@wintelcom.net (Alfred Perlstein) Date: Wed, 17 Jan 2001 01:44:20 +0000 (GMT) Cc: tlambert@primenet.com (Terry Lambert), mckusick@mckusick.com (Kirk McKusick), arch@FreeBSD.ORG In-Reply-To: <20010116172859.B7240@fw.wintelcom.net> from "Alfred Perlstein" at Jan 16, 2001 05:28:59 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > Dynamically created OIDs should: > > > > 1) Monotonically increase > > They don't seem to, this is the main problem with doing any translation, > it looks like: > > /* > * If this oid has a number OID_AUTO, give it a number which > * is greater than any current oid. Make sure it is at least > * 100 to leave space for pre-assigned oid numbers. > */ This is really broken. Common usage for OID subarcs is to take your base OID, and then use the top level next tuple to assign it to an application, then use the next tuple to do things like "dynamic", "static", "transient", and so on. Really, these things should be under a different top level "oid_number" value entirely. > Could cause some issues because it seems that if a sysctl were to > "go away" the next one to apear under the same level will replace > its number. It's worse than that. You can't dictate semantics, except for on subbranches. It seems to me that I'm perfectly OK to cache the thing in a text file of my own application-specific choosing, unless it specifically states that I'm not allowed to do that because there's a non-cacheable semantic in effect. I'm going back into lurk-mode now; OIDs just push my hot button today, for some reason. 8-p. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jan 16 23:29:17 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id C1FDE37B400 for ; Tue, 16 Jan 2001 23:28:56 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id SAA24533; Wed, 17 Jan 2001 18:28:37 +1100 Date: Wed, 17 Jan 2001 18:28:43 +1100 (EST) From: Bruce Evans X-Sender: bde@besplex.bde.org To: Kirk McKusick Cc: Alfred Perlstein , arch@FreeBSD.ORG Subject: Re: dynamic vs static sysctls? In-Reply-To: <200101152345.PAA22257@beastie.mckusick.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Mon, 15 Jan 2001, Kirk McKusick wrote: > I have an example where sysctl is still useful. In my work on a > background version of fsck, I have used sysctl to allow me to > pass information into the kernel that I want to have updated in > the filesystem. In the case of lost blocks, there may be hundreds > or even thousands of blocks that need to be put back into the > bit maps. Each non-contiguous block is passed in separately > which results in hundreds or thousands of sysctl calls. I want > to do one call to sysctlnametomib (a new, but obviously trivial > function) to return the numeric mib, and then use that mib on all > the subsequent sysctl calls. That dramatically cuts down on the > time it takes to return the blocks since I do not have to translate > the same name repeatedly (which takes far longer than the block > release itself). I would think that this might be an issue for > any application that uses sysctl to get/set the same information > repeatedly. There is no reason why numeric mibs should be significantly faster than string mibs. Syscall overhead would dominate string lookup time if the implementation was even moderately efficient. However, out current implementation is inefficient for numeric mibs and worse for string mibs. The enclosed test program gives the following times on a Celeron522 for related lookups via getppid(), sysctl(), sysctlbyname() and open()/read()/close(). GETPPID: 1.3 usec/call SYSCTL: 8.6 usec/call SYSCTLBYNAME: 21.9 usec/call FS: 22.0 usec/call (reading /COPYRIGHT) FS: 34.5 usec/call (reading /proc/curproc/regs) getppid() isn't closely related, but it performs the same amount of useful work as the other operations -- it copies a single integer from the kernel to the application. --- #include #include #include #include main() { int fd; int hw_float; int i; size_t len; int mib[2]; for (i = 0; i < 1000000; ++i) { len = sizeof(hw_float); #ifdef GETPPID (void)getppid(); #endif #ifdef SYSCTL mib[0] = CTL_HW; mib[1] = HW_FLOATINGPT; if (sysctl(mib, 2, &hw_float, &len, (void *)0, 0) == -1) err(1, "sysctl"); #endif #ifdef SYSCTLBYNAME if (sysctlbyname("hw.floatingpoint", &hw_float, &len, (void *)0, 0) == -1) err(1, "sysctl"); #endif #ifdef FS if ((fd = open("/COPYRIGHT", 0)) == -1) err(1, "open"); if (read(fd, &hw_float, sizeof(hw_float)) != sizeof(hw_float)) err(1, "read"); if (close(fd) != 0) err(1, "close"); #endif } } --- Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 3:16:37 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 5659937B400; Wed, 17 Jan 2001 03:16:19 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f0HBGJC14809; Wed, 17 Jan 2001 03:16:19 -0800 (PST) Date: Wed, 17 Jan 2001 03:16:19 -0800 From: Alfred Perlstein To: arch@freebsd.org, current@freebsd.org Subject: HEADS-UP: await/asleep removal imminent Message-ID: <20010117031618.K7240@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Please trim cc's as appropriate. I have a patch here that removes await/asleep from the kernel API. http://people.freebsd.org/~alfred/noasleep.diff Matt Dillon implemented alseep/await quite some time ago and the only thing that's using it is ata. In order to clean up some of the schduler and vm system I'm removing support for it. Peter Wemm and I suspect that ata doesn't need it. Right now I'm running several make -j128 buildworlds and buildkernels with this patch to catch any ata problems. Now would be a good time to test this delta for stability reasons. I plan on applying the delta within a few days. thanks, -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 3:38:58 2001 Delivered-To: freebsd-arch@freebsd.org Received: from freebsd.dk (freebsd.dk [212.242.42.178]) by hub.freebsd.org (Postfix) with ESMTP id 038F637B400; Wed, 17 Jan 2001 03:38:38 -0800 (PST) Received: (from sos@localhost) by freebsd.dk (8.9.3/8.9.1) id MAA11834; Wed, 17 Jan 2001 12:38:33 +0100 (CET) (envelope-from sos) From: Soren Schmidt Message-Id: <200101171138.MAA11834@freebsd.dk> Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: <20010117031618.K7240@fw.wintelcom.net> from Alfred Perlstein at "Jan 17, 2001 03:16:19 am" To: bright@wintelcom.net (Alfred Perlstein) Date: Wed, 17 Jan 2001 12:38:33 +0100 (CET) Cc: arch@FreeBSD.ORG, current@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL54 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG It seems Alfred Perlstein wrote: > Please trim cc's as appropriate. > > I have a patch here that removes await/asleep from the kernel API. > > http://people.freebsd.org/~alfred/noasleep.diff > > Matt Dillon implemented alseep/await quite some time ago and the > only thing that's using it is ata. In order to clean up some of > the schduler and vm system I'm removing support for it. > > Peter Wemm and I suspect that ata doesn't need it. Right now I'm > running several make -j128 buildworlds and buildkernels with this > patch to catch any ata problems. I'll commit the ata part as soon as I have it tested here.... -Søren To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 8:15:17 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail.wgate.com (mail.wgate.com [38.219.83.4]) by hub.freebsd.org (Postfix) with ESMTP id 127FC37B400; Wed, 17 Jan 2001 08:14:56 -0800 (PST) Received: from jesup.eng.tvol.net ([10.32.2.26]) by mail.wgate.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21) id CX5HP616; Wed, 17 Jan 2001 11:14:54 -0500 Reply-To: Randell Jesup To: Soren Schmidt Cc: bright@wintelcom.net (Alfred Perlstein), arch@FreeBSD.ORG, current@FreeBSD.ORG Subject: Re: HEADS-UP: await/asleep removal imminent References: <200101171138.MAA11834@freebsd.dk> From: Randell Jesup Date: 17 Jan 2001 11:17:29 -0500 In-Reply-To: Soren Schmidt's message of "Wed, 17 Jan 2001 12:38:33 +0100 (CET)" Message-ID: User-Agent: Gnus/5.0807 (Gnus v5.8.7) Emacs/20.7 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG >It seems Alfred Perlstein wrote: >> I have a patch here that removes await/asleep from the kernel API. >> >> http://people.freebsd.org/~alfred/noasleep.diff >> >> Matt Dillon implemented alseep/await quite some time ago and the >> only thing that's using it is ata. In order to clean up some of >> the schduler and vm system I'm removing support for it. >> >> Peter Wemm and I suspect that ata doesn't need it. Right now I'm >> running several make -j128 buildworlds and buildkernels with this >> patch to catch any ata problems. Ummmm... It seems to me from reading the man page for asleep/await that they have significant utility, and that the real issue would be one of code not using them, especially as people work to remove the Giant lock for SMP. Or is the discussion in the man page wrong in some way? -- Randell Jesup, Worldgate Communications, ex-Scala, ex-Amiga OS team ('88-94) rjesup@wgate.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 9:23: 0 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id EBF0237B402; Wed, 17 Jan 2001 09:22:29 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f0HHL9A23321; Wed, 17 Jan 2001 09:21:09 -0800 (PST) Date: Wed, 17 Jan 2001 09:21:09 -0800 From: Alfred Perlstein To: Randell Jesup Cc: Soren Schmidt , arch@FreeBSD.ORG, current@FreeBSD.ORG Subject: Re: HEADS-UP: await/asleep removal imminent Message-ID: <20010117092109.O7240@fw.wintelcom.net> References: <200101171138.MAA11834@freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from rjesup@wgate.com on Wed, Jan 17, 2001 at 11:17:29AM -0500 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Randell Jesup [010117 08:14] wrote: > >It seems Alfred Perlstein wrote: > >> I have a patch here that removes await/asleep from the kernel API. > >> > >> http://people.freebsd.org/~alfred/noasleep.diff > >> > >> Matt Dillon implemented alseep/await quite some time ago and the > >> only thing that's using it is ata. In order to clean up some of > >> the schduler and vm system I'm removing support for it. > >> > >> Peter Wemm and I suspect that ata doesn't need it. Right now I'm > >> running several make -j128 buildworlds and buildkernels with this > >> patch to catch any ata problems. > > Ummmm... > > It seems to me from reading the man page for asleep/await that > they have significant utility, and that the real issue would be one of > code not using them, especially as people work to remove the Giant > lock for SMP. > > Or is the discussion in the man page wrong in some way? The manpage is correct, but we've yet to see it used properly in the code with the exception of ata, and even with ata we're not sure if it's needed. Basically the idea is that at a certain depth into the kernel you start using the M_AWAIT parameter when requesting certain resources. Since that removes the possiblity of blocking you can feel safe holding a mutex over these lower parts of the code, then only releasing your mutex later. If a resource shortage is detected then you can call await and retry later. 1) from sleep(9): to actually wait for the previously queued wait condition. If asleep() is called several times, only the most recent call is effective. asleep() may be called with an ident value of NULL to remove any previ- ously queued condition. Basically, there's a chance you may clobber previous calls to asleep, this is somewhat unlikely since most shortages or blocking when some 'NOBLOCK' flag is passed in should immediately return. Somewhat of a concern, but it really is unlikely to be a problem. 2) most things you'd use mawait for would be under a lock already, since it's under a lock, you must context switch anyway, there's no "optimized out" sleep, because first the interrupt will block on your mutex, then you'll get to run, when you drop your mutex the interrupt will then run. If you do this with msleep or masleep the effect is the same. 3) no one uses it! :) 4) most anything you need asleep for could probably be done with mutex/cv's. I'm not going to axe it for a few days, this is a really amazing API that Matt added, the problem is utility and useage over code complexity. It's just a proposal. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 10: 2:34 2001 Delivered-To: freebsd-arch@freebsd.org Received: from freebsd.dk (freebsd.dk [212.242.42.178]) by hub.freebsd.org (Postfix) with ESMTP id DDBC637B699; Wed, 17 Jan 2001 10:02:12 -0800 (PST) Received: (from sos@localhost) by freebsd.dk (8.9.3/8.9.1) id TAA02435; Wed, 17 Jan 2001 19:02:06 +0100 (CET) (envelope-from sos) From: Soren Schmidt Message-Id: <200101171802.TAA02435@freebsd.dk> Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: <20010117092109.O7240@fw.wintelcom.net> from Alfred Perlstein at "Jan 17, 2001 09:21:09 am" To: bright@wintelcom.net (Alfred Perlstein) Date: Wed, 17 Jan 2001 19:02:06 +0100 (CET) Cc: rjesup@wgate.com (Randell Jesup), arch@FreeBSD.ORG, current@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL54 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG It seems Alfred Perlstein wrote: > > >> Peter Wemm and I suspect that ata doesn't need it. Right now I'm > > >> running several make -j128 buildworlds and buildkernels with this > > >> patch to catch any ata problems. > > > > Ummmm... > > > > It seems to me from reading the man page for asleep/await that > > they have significant utility, and that the real issue would be one of > > code not using them, especially as people work to remove the Giant > > lock for SMP. > > > > Or is the discussion in the man page wrong in some way? > > The manpage is correct, but we've yet to see it used properly in > the code with the exception of ata, and even with ata we're not > sure if it's needed. Uhm, well I tried removing it here, and now -current (on SMP HW) fails in new "interesting" ways. The problem here is that -current is not stable on SMP HW so the question is if this change in behavior is to the better or to the worse... I suggest creative manpower is used to stabilize -current, instead of fine trimming which API's should stay or not... -Søren To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 10: 5:45 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 7658C37B404; Wed, 17 Jan 2001 10:05:18 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f0HI5HD24831; Wed, 17 Jan 2001 10:05:17 -0800 (PST) Date: Wed, 17 Jan 2001 10:05:17 -0800 From: Alfred Perlstein To: Randell Jesup , dillon@freebsd.org Cc: Soren Schmidt , arch@freebsd.org, current@freebsd.org Subject: Re: HEADS-UP: await/asleep removal imminent Message-ID: <20010117100516.Q7240@fw.wintelcom.net> References: <200101171138.MAA11834@freebsd.dk> <20010117092109.O7240@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010117092109.O7240@fw.wintelcom.net>; from bright@wintelcom.net on Wed, Jan 17, 2001 at 09:21:09AM -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Alfred Perlstein [010117 09:24] wrote: > > I'm not going to axe it for a few days, this is a really amazing > API that Matt added, the problem is utility and useage over code > complexity. > > It's just a proposal. I found several places where it may be useful, but I'm not sure if the benefits outweigh the gains. In a copy of the tree I've locked down the socket layer (not the entire stack, just sockets :) ) there's code like this: SOCKBUF_UNLOCK(&so->so_snd, 0); if (top == 0) { MGETHDR(m, M_TRYWAIT, MT_DATA); if (m == NULL) { error = ENOBUFS; SOCKBUF_LOCK(&so->so_snd, 0); goto release; } mlen = MHLEN; ... SOCKBUF_LOCK(&so->so_snd, 0); /* XXX */ The lock must be unwound becasue we're calling MGETHDR with M_TRYWAIT. If wae used M_TRY'A'WAIT the code would probably look something like this: /* SOCKBUF_UNLOCK(&so->so_snd, 0); */ again: if (top == 0) { MGETHDR(m, M_TRYWAIT, MT_DATA); if (m == NULL) { error = mawait(&so->so_snd.sb_mtx, -1, -1); if (error) { if (error == EWOULDBLOCK) error = ENOBUFS; goto release; } goto again; /* SOCKBUF_LOCK(&so->so_snd, 0); */ } mlen = MHLEN; ... /* SOCKBUF_LOCK(&so->so_snd, 0); */ /* XXX */ Which means we don't have to drop the lock over the socket unless we'd block on allocation. Matt, is this what you intended for it to do? So far I've only seen it used to avoid races, but not to optimize out mutex aquire/release. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 10:10:34 2001 Delivered-To: freebsd-arch@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 6E76F37B6A3 for ; Wed, 17 Jan 2001 10:10:14 -0800 (PST) Received: (qmail 19297 invoked by uid 1142); 17 Jan 2001 18:10:10 -0000 Date: 17 Jan 2001 10:10:10 -0800 Date: Wed, 17 Jan 2001 10:09:54 -0800 From: Jason Evans To: Alfred Perlstein Cc: Randell Jesup , arch@FreeBSD.ORG Subject: Re: HEADS-UP: await/asleep removal imminent Message-ID: <20010117100954.S61852@canonware.com> References: <200101171138.MAA11834@freebsd.dk> <20010117092109.O7240@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010117092109.O7240@fw.wintelcom.net>; from bright@wintelcom.net on Wed, Jan 17, 2001 at 09:21:09AM -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG [-current trimmed] On Wed, Jan 17, 2001 at 09:21:09AM -0800, Alfred Perlstein wrote: > 3) no one uses it! :) > > 4) most anything you need asleep for could probably be done with > mutex/cv's. > > I'm not going to axe it for a few days, this is a really amazing > API that Matt added, the problem is utility and useage over code > complexity. I support its removal. I would argue that not only _could_ most uses of asleep instead use mutexes and condition variables, but they _should_. The fact that asleep()/await() aren't used to any significant degree, along with the fact that there are other ways to accomplish the same thing, means that we have an easy opportunity to reduce the complexity of our kernel by removing unused code. At the time that Matt wrote the code, it made sense, and if we hadn't gone the direction of using BSD/OS's SMP architecture, it probably would have been used quite a bit. Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 10:15:23 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 9C93C37B6C5; Wed, 17 Jan 2001 10:15:01 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f0HIDgn25079; Wed, 17 Jan 2001 10:13:42 -0800 (PST) Date: Wed, 17 Jan 2001 10:13:42 -0800 From: Alfred Perlstein To: Soren Schmidt Cc: Randell Jesup , arch@FreeBSD.ORG, current@FreeBSD.ORG Subject: Re: HEADS-UP: await/asleep removal imminent Message-ID: <20010117101342.R7240@fw.wintelcom.net> References: <20010117092109.O7240@fw.wintelcom.net> <200101171802.TAA02435@freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200101171802.TAA02435@freebsd.dk>; from sos@freebsd.dk on Wed, Jan 17, 2001 at 07:02:06PM +0100 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Soren Schmidt [010117 10:02] wrote: > It seems Alfred Perlstein wrote: > > > >> Peter Wemm and I suspect that ata doesn't need it. Right now I'm > > > >> running several make -j128 buildworlds and buildkernels with this > > > >> patch to catch any ata problems. > > > > > > Ummmm... > > > > > > It seems to me from reading the man page for asleep/await that > > > they have significant utility, and that the real issue would be one of > > > code not using them, especially as people work to remove the Giant > > > lock for SMP. > > > > > > Or is the discussion in the man page wrong in some way? > > > > The manpage is correct, but we've yet to see it used properly in > > the code with the exception of ata, and even with ata we're not > > sure if it's needed. > > Uhm, well I tried removing it here, and now -current (on SMP HW) > fails in new "interesting" ways. The problem here is that > -current is not stable on SMP HW so the question is if this > change in behavior is to the better or to the worse... > > I suggest creative manpower is used to stabilize -current, instead > of fine trimming which API's should stay or not... I started a loop of make -j128 buildworld and buildkernel last night, I still haven't seen anything odd happen on my hardware. You and Poul-Henning have to figure out what's going on, no one else is able to reproduce this instability you're talking about. There has to be a way for you guys to get us some reasonable tracebacks or diagnostics instead of just saying "it's broke". Perhaps you can explain how you're able to trigger this instability with a test script? Poul-Henning told me he just needed to do a make -j256 world, I did 10 of them without a problem... I'd also like to see what hardware you guys are running on and what kernel config. I'm pretty sure that running with a weird value for HZ causes lockups on -stable, dunno about current. Basically if you're expecting me or the SMP team to figure out what's going on without more info, you're pretty much out of luck. ...wondering if the box Paul Saab gave me is actually SMP... :) -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 10:26: 2 2001 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 0B16C37B6B7; Wed, 17 Jan 2001 10:25:39 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id f0HIPSZ53777; Wed, 17 Jan 2001 19:25:28 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Alfred Perlstein Cc: Soren Schmidt , Randell Jesup , arch@FreeBSD.ORG, current@FreeBSD.ORG Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: Your message of "Wed, 17 Jan 2001 10:13:42 PST." <20010117101342.R7240@fw.wintelcom.net> Date: Wed, 17 Jan 2001 19:25:28 +0100 Message-ID: <53775.979755928@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG >Perhaps you can explain how you're able to trigger this instability >with a test script? Poul-Henning told me he just needed to do a >make -j256 world, I did 10 of them without a problem... Then you misunderstood me, I don't have anything in the dept of SMP hw which can trigger it. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 10:43: 0 2001 Delivered-To: freebsd-arch@freebsd.org Received: from freebsd.dk (freebsd.dk [212.242.42.178]) by hub.freebsd.org (Postfix) with ESMTP id 77C3937B6CD; Wed, 17 Jan 2001 10:42:33 -0800 (PST) Received: (from sos@localhost) by freebsd.dk (8.9.3/8.9.1) id TAA12276; Wed, 17 Jan 2001 19:42:27 +0100 (CET) (envelope-from sos) From: Soren Schmidt Message-Id: <200101171842.TAA12276@freebsd.dk> Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: <20010117101342.R7240@fw.wintelcom.net> from Alfred Perlstein at "Jan 17, 2001 10:13:42 am" To: bright@wintelcom.net (Alfred Perlstein) Date: Wed, 17 Jan 2001 19:42:26 +0100 (CET) Cc: rjesup@wgate.com (Randell Jesup), arch@FreeBSD.ORG, current@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL54 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG It seems Alfred Perlstein wrote: > > > > I suggest creative manpower is used to stabilize -current, instead > > of fine trimming which API's should stay or not... > > I started a loop of make -j128 buildworld and buildkernel last > night, I still haven't seen anything odd happen on my hardware. > > You and Poul-Henning have to figure out what's going on, no one > else is able to reproduce this instability you're talking about. Oohh you dont read the mailing lists then, there has been plenty of reports of hanging -current boxen since SMPng... > There has to be a way for you guys to get us some reasonable > tracebacks or diagnostics instead of just saying "it's broke". Its close to impossible, the two symptoms I see here are either spontanous reboots, or solid hangs where only a reset can get you out, so I cant say much other than "it's broke". > Perhaps you can explain how you're able to trigger this instability > with a test script? Poul-Henning told me he just needed to do a > make -j256 world, I did 10 of them without a problem... Hmm, with a -current kernel from today 1200 CET i just need to do a make depend on a GENERIC kernel, and wham it locks up. > I'd also like to see what hardware you guys are running on and what > kernel config. I'm pretty sure that running with a weird value > for HZ causes lockups on -stable, dunno about current. Nothing special, GENERIC kernel with SMP defined will do nicely, running without SMP improves matters but on the fastet machine I'm still getting lockups, but they are rare... Hardware it hangs on here include: 2*PPro@200 192MB FX chipset ATA disks on onboard controller (PIIX3) 2*PII@350 512MB BX chipset SCSI disks on NCR controller 2*PIII@1G 512MB ServerWorks chipset ATA disks on onboard + HPT controller. It seems the faster the machine the faster the lockup/hang.. Need I mention that they all work just fine(tm) under -stable and -current back on PRE_SMPNG... So, we (phk & I) are trying to figure out what is going on, but there is little to go on but hunch... So there is nothing special to it guys, you just have to try.. Oh btw using a ccd/vinum/ATA-raid thingy makes the problem worse, probably due to the higher interrupt rates. > Basically if you're expecting me or the SMP team to figure out > what's going on without more info, you're pretty much out of luck. See above, not really possible, we have been trying to find some (affordable) HW that could be used to preserve a log over a boot, but so far I havn't been able to find anything that works, and is fast enough to not effect the system too much... > ...wondering if the box Paul Saab gave me is actually SMP... :) Yup, that would explain things :) -Søren To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 10:46:59 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id C550A37B6C9; Wed, 17 Jan 2001 10:46:37 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f0HIhIT26159; Wed, 17 Jan 2001 10:43:18 -0800 (PST) Date: Wed, 17 Jan 2001 10:43:18 -0800 From: Alfred Perlstein To: Poul-Henning Kamp Cc: Soren Schmidt , Randell Jesup , arch@FreeBSD.ORG, current@FreeBSD.ORG Subject: Re: HEADS-UP: await/asleep removal imminent Message-ID: <20010117104318.U7240@fw.wintelcom.net> References: <20010117101342.R7240@fw.wintelcom.net> <53775.979755928@critter> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <53775.979755928@critter>; from phk@critter.freebsd.dk on Wed, Jan 17, 2001 at 07:25:28PM +0100 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Poul-Henning Kamp [010117 10:25] wrote: > > >Perhaps you can explain how you're able to trigger this instability > >with a test script? Poul-Henning told me he just needed to do a > >make -j256 world, I did 10 of them without a problem... > > Then you misunderstood me, I don't have anything in the dept > of SMP hw which can trigger it. If you could be more vague it would assist me greatly in figuring out what's broken. thanks, -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 10:53:42 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 6915537B6B1; Wed, 17 Jan 2001 10:53:14 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f0HIqQX26581; Wed, 17 Jan 2001 10:52:26 -0800 (PST) Date: Wed, 17 Jan 2001 10:52:26 -0800 From: Alfred Perlstein To: Soren Schmidt Cc: Randell Jesup , arch@FreeBSD.ORG, current@FreeBSD.ORG Subject: Re: HEADS-UP: await/asleep removal imminent Message-ID: <20010117105226.V7240@fw.wintelcom.net> References: <20010117101342.R7240@fw.wintelcom.net> <200101171842.TAA12276@freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200101171842.TAA12276@freebsd.dk>; from sos@freebsd.dk on Wed, Jan 17, 2001 at 07:42:26PM +0100 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Soren Schmidt [010117 10:43] wrote: > It seems Alfred Perlstein wrote: > > > > > > I suggest creative manpower is used to stabilize -current, instead > > > of fine trimming which API's should stay or not... > > > > I started a loop of make -j128 buildworld and buildkernel last > > night, I still haven't seen anything odd happen on my hardware. > > > > You and Poul-Henning have to figure out what's going on, no one > > else is able to reproduce this instability you're talking about. > > Oohh you dont read the mailing lists then, there has been plenty > of reports of hanging -current boxen since SMPng... Yes, but none with anything useful. :( > > There has to be a way for you guys to get us some reasonable > > tracebacks or diagnostics instead of just saying "it's broke". > > Its close to impossible, the two symptoms I see here are either > spontanous reboots, or solid hangs where only a reset can get > you out, so I cant say much other than "it's broke". You probably have a much better understanding of low level programming than I do, you _should_ be able to figure out what's going on. > > Perhaps you can explain how you're able to trigger this instability > > with a test script? Poul-Henning told me he just needed to do a > > make -j256 world, I did 10 of them without a problem... > > Hmm, with a -current kernel from today 1200 CET i just need to > do a make depend on a GENERIC kernel, and wham it locks up. Odd, doesn't hang for me. > > I'd also like to see what hardware you guys are running on and what > > kernel config. I'm pretty sure that running with a weird value > > for HZ causes lockups on -stable, dunno about current. > > Nothing special, GENERIC kernel with SMP defined will do nicely, running > without SMP improves matters but on the fastet machine I'm still getting > lockups, but they are rare... > > Hardware it hangs on here include: > > 2*PPro@200 192MB FX chipset ATA disks on onboard controller (PIIX3) > > 2*PII@350 512MB BX chipset SCSI disks on NCR controller > > 2*PIII@1G 512MB ServerWorks chipset ATA disks on onboard + HPT controller. > > It seems the faster the machine the faster the lockup/hang.. > > Need I mention that they all work just fine(tm) under -stable and > -current back on PRE_SMPNG... > > So, we (phk & I) are trying to figure out what is going on, but > there is little to go on but hunch... > So there is nothing special to it guys, you just have to try.. > Oh btw using a ccd/vinum/ATA-raid thingy makes the problem worse, > probably due to the higher interrupt rates. I will try stacking a vinum over vn striped setup later tonight to see if this still locks up. You're still not telling me what combination of vn/vinum does this, so I guess I'll have to stumble around in the dark for a bit until I find the magic combination to find the Danish panic/lockup? I think phk just told me that you need a UP kernel to find this, but he's being pretty vague about it so I don't know. > > Basically if you're expecting me or the SMP team to figure out > > what's going on without more info, you're pretty much out of luck. > > See above, not really possible, we have been trying to find some > (affordable) HW that could be used to preserve a log over a boot, > but so far I havn't been able to find anything that works, and > is fast enough to not effect the system too much... > > > ...wondering if the box Paul Saab gave me is actually SMP... :) > > Yup, that would explain things :) Well, I do see processes migrating from CPU to CPU and there's the dmesg: FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 0, version: 0x00040011, at 0xfee00000 cpu1 (AP): apic id: 1, version: 0x00040011, at 0xfee00000 io0 (APIC): apic id: 4, version: 0x000f0011, at 0xfec00000 io1 (APIC): apic id: 5, version: 0x000f0011, at 0xfec01000 SMP: AP CPU #1 Launched! SMP: CPU1 apic_initialize(): lint0: 0x00010700 lint1: 0x00010400 TPR: 0x00000010 SVR: 0x000001ff start_init: trying /sbin/init Dual 750mhz, 1GB RAM, atapci0: dual disks: ad0: ATA-5 disk at ata0-master -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 10:55:59 2001 Delivered-To: freebsd-arch@freebsd.org Received: from meow.osd.bsdi.com (meow.osd.bsdi.com [204.216.28.88]) by hub.freebsd.org (Postfix) with ESMTP id CF2EB37B6C8 for ; Wed, 17 Jan 2001 10:55:37 -0800 (PST) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by meow.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id f0HIrGL38304; Wed, 17 Jan 2001 10:53:16 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20010117100954.S61852@canonware.com> Date: Wed, 17 Jan 2001 10:55:09 -0800 (PST) From: John Baldwin To: Jason Evans Subject: Re: HEADS-UP: await/asleep removal imminent Cc: arch@FreeBSD.org, Randell Jesup , Alfred Perlstein Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 17-Jan-01 Jason Evans wrote: > [-current trimmed] > > On Wed, Jan 17, 2001 at 09:21:09AM -0800, Alfred Perlstein wrote: >> 3) no one uses it! :) >> >> 4) most anything you need asleep for could probably be done with >> mutex/cv's. >> >> I'm not going to axe it for a few days, this is a really amazing >> API that Matt added, the problem is utility and useage over code >> complexity. > > I support its removal. I would argue that not only _could_ most uses of > asleep instead use mutexes and condition variables, but they _should_. The > fact that asleep()/await() aren't used to any significant degree, along > with the fact that there are other ways to accomplish the same thing, means > that we have an easy opportunity to reduce the complexity of our kernel by > removing unused code. > > At the time that Matt wrote the code, it made sense, and if we hadn't gone > the direction of using BSD/OS's SMP architecture, it probably would have > been used quite a bit. Well, it will be unused if we axe all tsleep's in favor of cv's which does incur extra overhead, as each cv has to be init'd and destroy'd and carries a linked list around with it. The extra storage overhead doesn't outweight the speed increase (from lack of the hash lookup) in all cases I think, so I'm not sure we want to axe tsleep() just yet. If you axe tsleep() then asleep() can be emulated by either passing cv's around between functions. > Jason -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 10:56:48 2001 Delivered-To: freebsd-arch@freebsd.org Received: from meow.osd.bsdi.com (meow.osd.bsdi.com [204.216.28.88]) by hub.freebsd.org (Postfix) with ESMTP id B3DDE37B401; Wed, 17 Jan 2001 10:56:27 -0800 (PST) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by meow.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id f0HIrFL38300; Wed, 17 Jan 2001 10:53:15 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200101171842.TAA12276@freebsd.dk> Date: Wed, 17 Jan 2001 10:55:08 -0800 (PST) From: John Baldwin To: Soren Schmidt Subject: Re: HEADS-UP: await/asleep removal imminent Cc: current@FreeBSD.org, arch@FreeBSD.org, (Randell Jesup) , (Alfred Perlstein) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 17-Jan-01 Soren Schmidt wrote: > Nothing special, GENERIC kernel with SMP defined will do nicely, running > without SMP improves matters but on the fastet machine I'm still getting > lockups, but they are rare... AHA! Useful info!! GENERIC is quite close to bloated, so the fact that it is GENERIC + SMP maybe an edge case. I'll try and test this out now that you have actually provided some useful info. :-P If you want a stable box, try trimming the kernel config down to what you need. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 10:57:34 2001 Delivered-To: freebsd-arch@freebsd.org Received: from meow.osd.bsdi.com (meow.osd.bsdi.com [204.216.28.88]) by hub.freebsd.org (Postfix) with ESMTP id E060737B400; Wed, 17 Jan 2001 10:57:12 -0800 (PST) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by meow.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id f0HIrGL38308; Wed, 17 Jan 2001 10:53:16 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <53775.979755928@critter> Date: Wed, 17 Jan 2001 10:55:09 -0800 (PST) From: John Baldwin To: Poul-Henning Kamp Subject: Re: HEADS-UP: await/asleep removal imminent Cc: current@FreeBSD.org, arch@FreeBSD.org, Randell Jesup , Soren Schmidt , Alfred Perlstein Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 17-Jan-01 Poul-Henning Kamp wrote: > >>Perhaps you can explain how you're able to trigger this instability >>with a test script? Poul-Henning told me he just needed to do a >>make -j256 world, I did 10 of them without a problem... > > Then you misunderstood me, I don't have anything in the dept > of SMP hw which can trigger it. Ok, what _does_ trigger this then? And what kernel config are you using: GENERIC? GENERIC is rather plagued with problems right now due to its bloated size. There have been reports of people not being able to boot a freshly installed system because of the kernel bloat, so you really probably want to be using a custom kernel until that bug is found and uncovered. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 11: 6:16 2001 Delivered-To: freebsd-arch@freebsd.org Received: from freebsd.dk (freebsd.dk [212.242.42.178]) by hub.freebsd.org (Postfix) with ESMTP id 592E937B6CA; Wed, 17 Jan 2001 11:05:48 -0800 (PST) Received: (from sos@localhost) by freebsd.dk (8.9.3/8.9.1) id UAA18132; Wed, 17 Jan 2001 20:05:39 +0100 (CET) (envelope-from sos) From: Soren Schmidt Message-Id: <200101171905.UAA18132@freebsd.dk> Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: <20010117105226.V7240@fw.wintelcom.net> from Alfred Perlstein at "Jan 17, 2001 10:52:26 am" To: bright@wintelcom.net (Alfred Perlstein) Date: Wed, 17 Jan 2001 20:05:38 +0100 (CET) Cc: rjesup@wgate.com (Randell Jesup), arch@FreeBSD.ORG, current@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL54 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG It seems Alfred Perlstein wrote: > > > > > > You and Poul-Henning have to figure out what's going on, no one > > > else is able to reproduce this instability you're talking about. > > > > Oohh you dont read the mailing lists then, there has been plenty > > of reports of hanging -current boxen since SMPng... > > Yes, but none with anything useful. :( Check, but thats because the breakage is the way it is, there *IS* no useful info to pass on, other than it locks up, is that so hard to understand ?? > > > There has to be a way for you guys to get us some reasonable > > > tracebacks or diagnostics instead of just saying "it's broke". > > > > Its close to impossible, the two symptoms I see here are either > > spontanous reboots, or solid hangs where only a reset can get > > you out, so I cant say much other than "it's broke". > > You probably have a much better understanding of low level programming > than I do, you _should_ be able to figure out what's going on. Ohh, thanks for the roses, but I havn't been following the SMP work that closely I thought it was in good hands, thats why I have told you guys that its broken for the last couble of months :) > > So there is nothing special to it guys, you just have to try.. > > Oh btw using a ccd/vinum/ATA-raid thingy makes the problem worse, > > probably due to the higher interrupt rates. > > I will try stacking a vinum over vn striped setup later tonight > to see if this still locks up. > > You're still not telling me what combination of vn/vinum does this, > so I guess I'll have to stumble around in the dark for a bit until > I find the magic combination to find the Danish panic/lockup? I didn't mention vn above, I use the ata-raid thingy with src & obj put on two striped disks each. The main point here being that this changes the way IO's are done... > I think phk just told me that you need a UP kernel to find this, > but he's being pretty vague about it so I don't know. You need sleep :) I said that SMP lockups quickly, UP runs pretty well and only hangs on rare occasions... -Søren To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 11: 8: 6 2001 Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (placeholder-dcat-1076843399.broadbandoffice.net [64.47.83.135]) by hub.freebsd.org (Postfix) with ESMTP id 926F837B6CF; Wed, 17 Jan 2001 11:07:32 -0800 (PST) Received: (from dillon@localhost) by earth.backplane.com (8.11.1/8.9.3) id f0HJ7Qe48680; Wed, 17 Jan 2001 11:07:26 -0800 (PST) (envelope-from dillon) Date: Wed, 17 Jan 2001 11:07:26 -0800 (PST) From: Matt Dillon Message-Id: <200101171907.f0HJ7Qe48680@earth.backplane.com> To: Alfred Perlstein Cc: Randell Jesup , Soren Schmidt , arch@FreeBSD.ORG, current@FreeBSD.ORG Subject: Re: HEADS-UP: await/asleep removal imminent References: <200101171138.MAA11834@freebsd.dk> <20010117092109.O7240@fw.wintelcom.net> <20010117100516.Q7240@fw.wintelcom.net> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :* Alfred Perlstein [010117 09:24] wrote: :> :> I'm not going to axe it for a few days, this is a really amazing :> API that Matt added, the problem is utility and useage over code :> complexity. :> :> It's just a proposal. : :I found several places where it may be useful, but I'm not sure if the :benefits outweigh the gains. :... : :The lock must be unwound becasue we're calling MGETHDR with M_TRYWAIT. :If wae used M_TRY'A'WAIT the code would probably look something like :this: The basic premis of using asleep()/await() is to allow you to propogate a 'blocking condition' back up to a higher level rather then blocking deep in side the kernel. The original reasoning was to deal with memory allocation blockages. For example, lets say you have three subsystem layers calling each other. The top layer wishes to implement a non-blocking API but the bottom layer might do an allocation that could block. The bottom layer could do a non-blocking allocation and return NULL, but how does the top layer (or an even higher layer) know when to try again? The original idea with asleep()/await() was for the bottom layer to call asleep() on the resource that would block and then return NULL. NULL would tell the higher layer(s) that someone down below couldn't get some resource and that an asleep() has been setup. The higher layers can then decide what to do with the situation.. they can abort the operation entirely, they can do the blocking (await() call) themselves, or they can propogate the condition to their own callers. This way you can hold more then one lock (now mutex) through a number of program layers without having to worry about them blocking on you. -- For the c urrent SMP system, asleep()/await() could be used to deal with complex situations where you (A) do not want to release a mutex through a call to another subsystem (like the memory allocator), or (B) do not know if the code calling you is already holding some mutex X and you want to hold mutex Y while you make a call to another subsystem. So, in that regard, you example: : /* SOCKBUF_UNLOCK(&so->so_snd, 0); */ :again: : if (top == 0) { : MGETHDR(m, M_TRYWAIT, MT_DATA); : if (m == NULL) { : error = mawait(&so->so_snd.sb_mtx, -1, -1); : if (error) { : if (error == EWOULDBLOCK) : error = ENOBUFS; : goto release; : } : goto again; : /* SOCKBUF_LOCK(&so->so_snd, 0); */ : } : mlen = MHLEN; :... : /* SOCKBUF_LOCK(&so->so_snd, 0); */ /* XXX */ : :Which means we don't have to drop the lock over the socket unless :we'd block on allocation. Works exactly as I originally intended. :Matt, is this what you intended for it to do? So far I've only :seen it used to avoid races, but not to optimize out mutex :aquire/release. : :-- :-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] I've never liked the BSDI mutex rules, because subsystems have to have major knowledge as to how other systems operate (in reagrds to whether they can block or not), and the callee must have intimate knowledge of the callers to know that it can hold a mutex and that be the only mutex it is holding. This makes for extremely fragile and complex coding. So, this said... I'm still on the fence as to whether await()/asleep() can be used effectively. As you said, there are not too many cases at the moment and await()/asleep() does introduce significant code complexity to the scheduler, and for it to really shine it needs to be optimized to not require any sort of mutex at all in the calls to asleep(). In order to get rid of the overhead, asleep() needs to simply initialize the curproc fields and not try to actually queue the process to the sleep queue. Right now asleep() queues the process to the sleep queue (see kern/kern_synch.c) in order to support the ability for the system to asynchronously 'wake the process up again' before it actually goes to sleep (which causes the later await() to become a NOP). i.e. the situation that caused the potential blockage might be resolved before the process has a chance to sleep. Traditionally we have used SPL levels (and now mutexes) to prevent the possibility of a condition being satisfied between the test and the tsleep(). ---- Proposal Revamp asleep/await to be based on state variables rather then tsleep's traditionaln 'fake' addresses. Rather then have a traditional sleep/wakeup we instead have a state variable that asleep/await operate on. For example lets say we have a memory allocator. When the memory allocator finds it would block, it utilizes a global state structure representing the blockage and clears the state (blah.state = 0;). Then it calls asleep(&blah). asleep simply stores the pointer to &blah in the process structure, it does not try to queue it or do anything else. Thus no locking or mutexes or interrupt disablement is required *at all*. The routine is entirely passive and safe to call from anywhere. Later, some event makes more memory available for allocation. That even is asynchronous and simply sets the state variable to 1 and wakes up anybody on the sleep queue for that condition variable. This event will NOT catch the guy in the previous paragraph who has not yet called await(), however, since the call to asleep() does not actually enqueue the process (which would require a mutex). Later, the process that called asleep() finally decides to try to go to sleep for real. await() checks p_state->state and if it is zero await() places the process on the sleep queue for real and actually goes to sleep. If p_state->state is non-zero, await() simply clears the pointer (proc->p_state = NULL;) and returns (without sleeping). I believe that this conforms to the state of the SMP system much much better then my original asleep()/await() implementation, and has the advantage of extremely *LOW* overhead (virtually none in many cases). You could then use it to give the 'power of blocking' to the caller rather then the callee. This in turn gives you much greater flexibility in regards to who can hold mutexes when and who can hold mutexes through procedure calls to other subsystems. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 11:10:22 2001 Delivered-To: freebsd-arch@freebsd.org Received: from freebsd.dk (freebsd.dk [212.242.42.178]) by hub.freebsd.org (Postfix) with ESMTP id ACB6E37B6A8; Wed, 17 Jan 2001 11:10:00 -0800 (PST) Received: (from sos@localhost) by freebsd.dk (8.9.3/8.9.1) id UAA19215; Wed, 17 Jan 2001 20:09:59 +0100 (CET) (envelope-from sos) From: Soren Schmidt Message-Id: <200101171909.UAA19215@freebsd.dk> Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: from John Baldwin at "Jan 17, 2001 10:55:08 am" To: jhb@FreeBSD.org (John Baldwin) Date: Wed, 17 Jan 2001 20:09:59 +0100 (CET) Cc: current@FreeBSD.org, arch@FreeBSD.org, rjesup@wgate.com, bright@wintelcom.net X-Mailer: ELM [version 2.4ME+ PL54 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG It seems John Baldwin wrote: > > On 17-Jan-01 Soren Schmidt wrote: > > Nothing special, GENERIC kernel with SMP defined will do nicely, running > > without SMP improves matters but on the fastet machine I'm still getting > > lockups, but they are rare... > > AHA! Useful info!! GENERIC is quite close to bloated, so the fact that it is > GENERIC + SMP maybe an edge case. I'll try and test this out now that you have > actually provided some useful info. :-P If you want a stable box, try > trimming the kernel config down to what you need. You read too much into that, I dont normally run a GENERIC kernel, I run one stripped down to what I need, I gave you guys a generic example of what fails, and what can easily be reproduced. Anyhow, I have asked before to have you guys supply me with a kernel that has been compiled "the right way" and I'll test it out here just to make sure I dont do anything stupid.. Just a bare bones kernel, fxp & ata drivers will do nicely for the serverworks thingy, give me an URL I can get it from or put it on ftp.freebsd.dk/incoming and give my a ping when its there... -Søren To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 11:21:49 2001 Delivered-To: freebsd-arch@freebsd.org Received: from meow.osd.bsdi.com (meow.osd.bsdi.com [204.216.28.88]) by hub.freebsd.org (Postfix) with ESMTP id 51BD637B754; Wed, 17 Jan 2001 11:21:22 -0800 (PST) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by meow.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id f0HJJML38924; Wed, 17 Jan 2001 11:19:22 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200101171907.f0HJ7Qe48680@earth.backplane.com> Date: Wed, 17 Jan 2001 11:21:15 -0800 (PST) From: John Baldwin To: Matt Dillon Subject: Re: HEADS-UP: await/asleep removal imminent Cc: current@FreeBSD.org, arch@FreeBSD.org, Soren Schmidt , Randell Jesup , Alfred Perlstein Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 17-Jan-01 Matt Dillon wrote: >:* Alfred Perlstein [010117 09:24] wrote: >:> >:> I'm not going to axe it for a few days, this is a really amazing >:> API that Matt added, the problem is utility and useage over code >:> complexity. >:> >:> It's just a proposal. >: >:I found several places where it may be useful, but I'm not sure if the >:benefits outweigh the gains. >:... >: >:The lock must be unwound becasue we're calling MGETHDR with M_TRYWAIT. >:If wae used M_TRY'A'WAIT the code would probably look something like >:this: > > The basic premis of using asleep()/await() is to allow you to > propogate a 'blocking condition' back up to a higher level rather > then blocking deep in side the kernel. Gotcha. Given that, if we switch to using condition variables for this, why not have the low level code return a pointer to a cv to wait on? This would do the same thing using an existing API (now) and I think would serve the same purposes as your API. Returning NULL for a cv would mean that no sleep was required. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 11:22:39 2001 Delivered-To: freebsd-arch@freebsd.org Received: from meow.osd.bsdi.com (meow.osd.bsdi.com [204.216.28.88]) by hub.freebsd.org (Postfix) with ESMTP id 07D6A37B698; Wed, 17 Jan 2001 11:22:12 -0800 (PST) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by meow.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id f0HJJML38928; Wed, 17 Jan 2001 11:19:22 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200101171909.UAA19215@freebsd.dk> Date: Wed, 17 Jan 2001 11:21:16 -0800 (PST) From: John Baldwin To: Soren Schmidt Subject: Re: HEADS-UP: await/asleep removal imminent Cc: bright@wintelcom.net, rjesup@wgate.com, arch@FreeBSD.org, current@FreeBSD.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 17-Jan-01 Soren Schmidt wrote: > It seems John Baldwin wrote: >> >> On 17-Jan-01 Soren Schmidt wrote: >> > Nothing special, GENERIC kernel with SMP defined will do nicely, running >> > without SMP improves matters but on the fastet machine I'm still getting >> > lockups, but they are rare... >> >> AHA! Useful info!! GENERIC is quite close to bloated, so the fact that it >> is >> GENERIC + SMP maybe an edge case. I'll try and test this out now that you >> have >> actually provided some useful info. :-P If you want a stable box, try >> trimming the kernel config down to what you need. > > You read too much into that, I dont normally run a GENERIC kernel, > I run one stripped down to what I need, I gave you guys a generic > example of what fails, and what can easily be reproduced. > > Anyhow, I have asked before to have you guys supply me with > a kernel that has been compiled "the right way" and I'll test > it out here just to make sure I dont do anything stupid.. > > Just a bare bones kernel, fxp & ata drivers will do nicely for the > serverworks thingy, give me an URL I can get it from or put it > on ftp.freebsd.dk/incoming and give my a ping when its there... Ok, will do. I'll even compile it on an SMP box. :-P I'm also adding in several debugging options that hopefully will casue it to have an informative panic rather than a hang or reboot. > -Søren -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 11:23: 4 2001 Delivered-To: freebsd-arch@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id EB28637B6A4 for ; Wed, 17 Jan 2001 11:22:44 -0800 (PST) Received: (qmail 22327 invoked by uid 1142); 17 Jan 2001 19:22:43 -0000 Date: 17 Jan 2001 11:22:43 -0800 Date: Wed, 17 Jan 2001 11:22:32 -0800 From: Jason Evans To: John Baldwin Cc: arch@FreeBSD.org Subject: Re: HEADS-UP: await/asleep removal imminent Message-ID: <20010117112232.U61852@canonware.com> References: <20010117100954.S61852@canonware.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from jhb@FreeBSD.org on Wed, Jan 17, 2001 at 10:55:09AM -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Wed, Jan 17, 2001 at 10:55:09AM -0800, John Baldwin wrote: > > On 17-Jan-01 Jason Evans wrote: > Well, it will be unused if we axe all tsleep's in favor of cv's which does > incur extra overhead, as each cv has to be init'd and destroy'd and carries a > linked list around with it. The extra storage overhead doesn't outweight the > speed increase (from lack of the hash lookup) in all cases I think, so I'm not > sure we want to axe tsleep() just yet. If you axe tsleep() then asleep() can > be emulated by either passing cv's around between functions. I didn't say anything about axing tsleep() right now. That may happen in the future, but not before 5.0. Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 11:29:36 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 3EDFA37B6C1 for ; Wed, 17 Jan 2001 11:28:52 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f0HJSp227908; Wed, 17 Jan 2001 11:28:51 -0800 (PST) Date: Wed, 17 Jan 2001 11:28:50 -0800 From: Alfred Perlstein To: Matt Dillon Cc: arch@FreeBSD.ORG Subject: Re: HEADS-UP: await/asleep removal imminent Message-ID: <20010117112850.X7240@fw.wintelcom.net> References: <200101171138.MAA11834@freebsd.dk> <20010117092109.O7240@fw.wintelcom.net> <20010117100516.Q7240@fw.wintelcom.net> <200101171907.f0HJ7Qe48680@earth.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200101171907.f0HJ7Qe48680@earth.backplane.com>; from dillon@earth.backplane.com on Wed, Jan 17, 2001 at 11:07:26AM -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Matt Dillon [010117 11:07] wrote: > :* Alfred Perlstein [010117 09:24] wrote: > :> > :> I'm not going to axe it for a few days, this is a really amazing > :> API that Matt added, the problem is utility and useage over code > :> complexity. > :> > :> It's just a proposal. > : > :I found several places where it may be useful, but I'm not sure if the > :benefits outweigh the gains. > :... > : > :The lock must be unwound becasue we're calling MGETHDR with M_TRYWAIT. > :If wae used M_TRY'A'WAIT the code would probably look something like > :this: > > The basic premis of using asleep()/await() is to allow you to > propogate a 'blocking condition' back up to a higher level rather > then blocking deep in side the kernel. > > The original reasoning was to deal with memory allocation blockages. > > For example, lets say you have three subsystem layers calling each > other. The top layer wishes to implement a non-blocking API but the > bottom layer might do an allocation that could block. > > The bottom layer could do a non-blocking allocation and return NULL, > but how does the top layer (or an even higher layer) know when to try > again? > > The original idea with asleep()/await() was for the bottom layer to > call asleep() on the resource that would block and then return NULL. > NULL would tell the higher layer(s) that someone down below couldn't > get some resource and that an asleep() has been setup. > > The higher layers can then decide what to do with the situation.. they > can abort the operation entirely, they can do the blocking (await() call) > themselves, or they can propogate the condition to their own callers. > > This way you can hold more then one lock (now mutex) through a number > of program layers without having to worry about them blocking on you. > > -- > > For the c urrent SMP system, asleep()/await() could be used to deal with > complex situations where you (A) do not want to release a mutex through > a call to another subsystem (like the memory allocator), or (B) do not > know if the code calling you is already holding some mutex X and you > want to hold mutex Y while you make a call to another subsystem. > > So, in that regard, you example: > > : /* SOCKBUF_UNLOCK(&so->so_snd, 0); */ > :again: > : if (top == 0) { > : MGETHDR(m, M_TRYWAIT, MT_DATA); > : if (m == NULL) { > : error = mawait(&so->so_snd.sb_mtx, -1, -1); > : if (error) { > : if (error == EWOULDBLOCK) > : error = ENOBUFS; > : goto release; > : } > : goto again; > : /* SOCKBUF_LOCK(&so->so_snd, 0); */ > : } > : mlen = MHLEN; > :... > : /* SOCKBUF_LOCK(&so->so_snd, 0); */ /* XXX */ > : > :Which means we don't have to drop the lock over the socket unless > :we'd block on allocation. > > Works exactly as I originally intended. Yup, less overhead. > :Matt, is this what you intended for it to do? So far I've only > :seen it used to avoid races, but not to optimize out mutex > :aquire/release. > : > :-- > :-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] > > I've never liked the BSDI mutex rules, because subsystems have to have > major knowledge as to how other systems operate (in reagrds to whether > they can block or not), and the callee must have intimate knowledge of > the callers to know that it can hold a mutex and that be the only mutex > it is holding. > > This makes for extremely fragile and complex coding. Also the need to drop/reaquire locks in a somewhat obtuse manner as my example shows. > So, this said... I'm still on the fence as to whether await()/asleep() > can be used effectively. As you said, there are not too many cases > at the moment and await()/asleep() does introduce significant code > complexity to the scheduler, and for it to really shine it needs to be > optimized to not require any sort of mutex at all in the calls to > asleep(). > > In order to get rid of the overhead, asleep() needs to simply initialize > the curproc fields and not try to actually queue the process to the > sleep queue. Right now asleep() queues the process to the sleep queue > (see kern/kern_synch.c) in order to support the ability for the system > to asynchronously 'wake the process up again' before it actually goes to > sleep (which causes the later await() to become a NOP). i.e. the > situation that caused the potential blockage might be resolved before > the process has a chance to sleep. Traditionally we have used SPL levels > (and now mutexes) to prevent the possibility of a condition being > satisfied between the test and the tsleep(). > > ---- > Proposal This is confusing: > > Revamp asleep/await to be based on state variables rather then > tsleep's traditionaln 'fake' addresses. Rather then > have a traditional sleep/wakeup we instead have a state variable that > asleep/await operate on. For example lets say we have a memory allocator. > When the memory allocator finds it would block, it utilizes a global > state structure representing the blockage and clears the state > (blah.state = 0;). Then it calls asleep(&blah). asleep simply stores > the pointer to &blah in the process structure, it does not try to queue > it or do anything else. Thus no locking or mutexes or interrupt > disablement is required *at all*. The routine is entirely passive and > safe to call from anywhere. It's not queued _anywhere_? > Later, some event makes more memory available for allocation. That > even is asynchronous and simply sets the state variable to 1 > and wakes up anybody on the sleep queue for that condition variable. > This event will NOT catch the guy in the previous paragraph who has > not yet called await(), however, since the call to asleep() does not > actually enqueue the process (which would require a mutex). again, it's not queued _anywhere_? > Later, the process that called asleep() finally decides to try > to go to sleep for real. await() checks p_state->state and if it > is zero await() places the process on the sleep queue for real and actually > goes to sleep. If p_state->state is non-zero, await() simply clears > the pointer (proc->p_state = NULL;) and returns (without sleeping). This is a bit weird, basically await doesn't enqueue the process, so there's a weird race going on such that all "freeing" need to signal rather than just when the pool is exhausted, in fact the pool may not be exhausted but you still block because no one signal's after you await. > I believe that this conforms to the state of the SMP system much much > better then my original asleep()/await() implementation, and has the > advantage of extremely *LOW* overhead (virtually none in many cases). > > You could then use it to give the 'power of blocking' to the caller > rather then the callee. This in turn gives you much greater flexibility > in regards to who can hold mutexes when and who can hold mutexes through > procedure calls to other subsystems. Jason Evans just added his conditional variables, you could change asleep/await into cv_attach/cv_await or something, that would give you less contention on the sleep/run queues, but you'd still need a spinlock or mutex on these variables. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 11:41:51 2001 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 919BE37B6A5; Wed, 17 Jan 2001 11:41:29 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id f0HJfIZ54249; Wed, 17 Jan 2001 20:41:18 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Alfred Perlstein Cc: Soren Schmidt , Randell Jesup , arch@FreeBSD.ORG, current@FreeBSD.ORG Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: Your message of "Wed, 17 Jan 2001 10:43:18 PST." <20010117104318.U7240@fw.wintelcom.net> Date: Wed, 17 Jan 2001 20:41:18 +0100 Message-ID: <54247.979760478@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <20010117104318.U7240@fw.wintelcom.net>, Alfred Perlstein writes: >* Poul-Henning Kamp [010117 10:25] wrote: >> >> >Perhaps you can explain how you're able to trigger this instability >> >with a test script? Poul-Henning told me he just needed to do a >> >make -j256 world, I did 10 of them without a problem... >> >> Then you misunderstood me, I don't have anything in the dept >> of SMP hw which can trigger it. > >If you could be more vague it would assist me greatly in figuring >out what's broken. I have been as clear as possible at all times, so if anything here has been vague, it gotta be in your end. The only reason why I'm in this fracas is to get people to take the anguished cries of sos seriously so that we don't loose our ATA maintainer... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 11:42:32 2001 Delivered-To: freebsd-arch@freebsd.org Received: from meow.osd.bsdi.com (meow.osd.bsdi.com [204.216.28.88]) by hub.freebsd.org (Postfix) with ESMTP id 3565337B6A8 for ; Wed, 17 Jan 2001 11:42:05 -0800 (PST) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by meow.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id f0HJe4L39662; Wed, 17 Jan 2001 11:40:04 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20010117112850.X7240@fw.wintelcom.net> Date: Wed, 17 Jan 2001 11:41:58 -0800 (PST) From: John Baldwin To: Alfred Perlstein Subject: Re: HEADS-UP: await/asleep removal imminent Cc: arch@FreeBSD.org, Matt Dillon Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 17-Jan-01 Alfred Perlstein wrote: > > Jason Evans just added his conditional variables, you could change > asleep/await into cv_attach/cv_await or something, that would give > you less contention on the sleep/run queues, but you'd still need > a spinlock or mutex on these variables. A condvar already contains the state information that asleep() caches in p_aleep right now, so just passing condvar pointers around would accomplish this task using an existing API and is conceptually cleaner: int my_lowlevel_func(struct cv **cv) { if (bad_things) { *cv = &my_condition; return (some_error); } *cv = NULL; return (0); } ... struct cv *cv; ... if (my_lowlevel_func(&cv) != 0) { KASSERT(cv != NULL, (__FUNCTION__ ": nothing to sleep on!")); cv_wait(cv); } -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 11:43:42 2001 Delivered-To: freebsd-arch@freebsd.org Received: from meow.osd.bsdi.com (meow.osd.bsdi.com [204.216.28.88]) by hub.freebsd.org (Postfix) with ESMTP id E385F37B6BA; Wed, 17 Jan 2001 11:43:21 -0800 (PST) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by meow.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id f0HJe3L39648; Wed, 17 Jan 2001 11:40:03 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200101171909.UAA19215@freebsd.dk> Date: Wed, 17 Jan 2001 11:41:56 -0800 (PST) From: John Baldwin To: Soren Schmidt Subject: Re: HEADS-UP: await/asleep removal imminent Cc: bright@wintelcom.net, rjesup@wgate.com, arch@FreeBSD.org, current@FreeBSD.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 17-Jan-01 Soren Schmidt wrote: > It seems John Baldwin wrote: >> >> On 17-Jan-01 Soren Schmidt wrote: >> > Nothing special, GENERIC kernel with SMP defined will do nicely, running >> > without SMP improves matters but on the fastet machine I'm still getting >> > lockups, but they are rare... >> >> AHA! Useful info!! GENERIC is quite close to bloated, so the fact that it >> is >> GENERIC + SMP maybe an edge case. I'll try and test this out now that you >> have >> actually provided some useful info. :-P If you want a stable box, try >> trimming the kernel config down to what you need. > > You read too much into that, I dont normally run a GENERIC kernel, > I run one stripped down to what I need, I gave you guys a generic > example of what fails, and what can easily be reproduced. > > Anyhow, I have asked before to have you guys supply me with > a kernel that has been compiled "the right way" and I'll test > it out here just to make sure I dont do anything stupid.. > > Just a bare bones kernel, fxp & ata drivers will do nicely for the > serverworks thingy, give me an URL I can get it from or put it > on ftp.freebsd.dk/incoming and give my a ping when its there... http://www.FreeBSD.org/~jhb/kernel.sos It was compiled wtih INCLUDE_CONFIG_FILE, so you can grab the config file from that. > -Søren -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 11:46:32 2001 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id A295B37B6B0; Wed, 17 Jan 2001 11:46:10 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id f0HJk3Z54316; Wed, 17 Jan 2001 20:46:03 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Alfred Perlstein Cc: Soren Schmidt , Randell Jesup , arch@FreeBSD.ORG, current@FreeBSD.ORG Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: Your message of "Wed, 17 Jan 2001 10:52:26 PST." <20010117105226.V7240@fw.wintelcom.net> Date: Wed, 17 Jan 2001 20:46:03 +0100 Message-ID: <54314.979760763@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <20010117105226.V7240@fw.wintelcom.net>, Alfred Perlstein writes: >> > There has to be a way for you guys to get us some reasonable >> > tracebacks or diagnostics instead of just saying "it's broke". >> >> Its close to impossible, the two symptoms I see here are either >> spontanous reboots, or solid hangs where only a reset can get >> you out, so I cant say much other than "it's broke". > >You probably have a much better understanding of low level programming >than I do, you _should_ be able to figure out what's going on. Excuse me! There is no fucking way you guys can expect anybody else to be able to debug at the rate things are being changed right now. >> Hmm, with a -current kernel from today 1200 CET i just need to >> do a make depend on a GENERIC kernel, and wham it locks up. > >Odd, doesn't hang for me. "Absense of evidence is not evidence of absense". Remember that. >I think phk just told me that you need a UP kernel to find this, >but he's being pretty vague about it so I don't know. No I told you that I have not been able to reproduce this because my SMP boxes are too slow to matter. Or maybe because they have to few and too ancient disks. Or maybe because I don't have time to really give them a beating these days. I don't know. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 11:48:20 2001 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 30E3037B402; Wed, 17 Jan 2001 11:47:51 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id f0HJlmZ54346; Wed, 17 Jan 2001 20:47:48 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: John Baldwin Cc: current@FreeBSD.org, arch@FreeBSD.org, Randell Jesup , Soren Schmidt , Alfred Perlstein Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: Your message of "Wed, 17 Jan 2001 10:55:09 PST." Date: Wed, 17 Jan 2001 20:47:48 +0100 Message-ID: <54340.979760868@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message , John Baldwin writes: > >On 17-Jan-01 Poul-Henning Kamp wrote: >> >>>Perhaps you can explain how you're able to trigger this instability >>>with a test script? Poul-Henning told me he just needed to do a >>>make -j256 world, I did 10 of them without a problem... >> >> Then you misunderstood me, I don't have anything in the dept >> of SMP hw which can trigger it. > >Ok, what _does_ trigger this then? My theory is that it's related to disk-IO. My guess is that it may be related to calling a *_strategy() routine from an interrupt routine. That is what sets ccd, ata-raid and vinum apart from normal disk-IO. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 11:50:54 2001 Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (placeholder-dcat-1076843399.broadbandoffice.net [64.47.83.135]) by hub.freebsd.org (Postfix) with ESMTP id C450137B6EC; Wed, 17 Jan 2001 11:50:29 -0800 (PST) Received: (from dillon@localhost) by earth.backplane.com (8.11.1/8.9.3) id f0HJoT149144; Wed, 17 Jan 2001 11:50:29 -0800 (PST) (envelope-from dillon) Date: Wed, 17 Jan 2001 11:50:29 -0800 (PST) From: Matt Dillon Message-Id: <200101171950.f0HJoT149144@earth.backplane.com> To: John Baldwin Cc: current@FreeBSD.org, arch@FreeBSD.org, Soren Schmidt , Randell Jesup , Alfred Perlstein Subject: Re: HEADS-UP: await/asleep removal imminent References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :>:... :>: :>:The lock must be unwound becasue we're calling MGETHDR with M_TRYWAIT. :>:If wae used M_TRY'A'WAIT the code would probably look something like :>:this: :> :> The basic premis of using asleep()/await() is to allow you to :> propogate a 'blocking condition' back up to a higher level rather :> then blocking deep in side the kernel. : :Gotcha. Given that, if we switch to using condition variables for this, why :not have the low level code return a pointer to a cv to wait on? This would do :the same thing using an existing API (now) and I think would serve the same :purposes as your API. Returning NULL for a cv would mean that no sleep was :required. : :-- : :John Baldwin -- http://www.FreeBSD.org/~jhb/ You could but that would require massive changes to the API for many of the calls. It's probably easier for the low level code to passively install the pointer into the proc structure and return 'NULL' - which is more likely to already be supported or mostly supported by the API. Don't get me wrong, the caller needs to know what is going on, but there is something to be said for not passing yet another argument to half the procedures in the kernel :-) One of the best things about a passive pointer in the proc structure as per my proposal is that the caller can decide to ignore it, can NULL it out, can leave it in place and propogate the failure upwards, or can replace it with another condition pointer. This gives the caller an incredible amount of flexibility in how it decides to handle the blocking condition, all without introducing any overhead. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 11:52:40 2001 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 596D937B6F0; Wed, 17 Jan 2001 11:52:18 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id f0HJqFZ54517; Wed, 17 Jan 2001 20:52:15 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: John Baldwin Cc: Soren Schmidt , bright@wintelcom.net, rjesup@wgate.com, arch@FreeBSD.ORG, current@FreeBSD.ORG Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: Your message of "Wed, 17 Jan 2001 11:41:56 PST." Date: Wed, 17 Jan 2001 20:52:15 +0100 Message-ID: <54515.979761135@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG John, It might be nice with a "How to provide useful info on SMPng problems." FAQ somewhere. I have no idea which of all the WITNESS and other options do what and when they are useful... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 11:55:55 2001 Delivered-To: freebsd-arch@freebsd.org Received: from freebsd.dk (freebsd.dk [212.242.42.178]) by hub.freebsd.org (Postfix) with ESMTP id 7C17737B699; Wed, 17 Jan 2001 11:55:30 -0800 (PST) Received: (from sos@localhost) by freebsd.dk (8.9.3/8.9.1) id UAA30462; Wed, 17 Jan 2001 20:55:29 +0100 (CET) (envelope-from sos) From: Soren Schmidt Message-Id: <200101171955.UAA30462@freebsd.dk> Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: from John Baldwin at "Jan 17, 2001 11:41:56 am" To: jhb@FreeBSD.ORG (John Baldwin) Date: Wed, 17 Jan 2001 20:55:28 +0100 (CET) Cc: bright@wintelcom.net, rjesup@wgate.com, arch@FreeBSD.ORG, current@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL54 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG It seems John Baldwin wrote: > > Anyhow, I have asked before to have you guys supply me with > > a kernel that has been compiled "the right way" and I'll test > > it out here just to make sure I dont do anything stupid.. > > > > Just a bare bones kernel, fxp & ata drivers will do nicely for the > > serverworks thingy, give me an URL I can get it from or put it > > on ftp.freebsd.dk/incoming and give my a ping when its there... > > http://www.FreeBSD.org/~jhb/kernel.sos > > It was compiled wtih INCLUDE_CONFIG_FILE, so you can grab the config file from > that. Roger, I'll put it on as soon as the box finishes what its doing now, I'll get back to you... -Søren To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 11:59:26 2001 Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (placeholder-dcat-1076843399.broadbandoffice.net [64.47.83.135]) by hub.freebsd.org (Postfix) with ESMTP id F268D37B6A5 for ; Wed, 17 Jan 2001 11:59:02 -0800 (PST) Received: (from dillon@localhost) by earth.backplane.com (8.11.1/8.9.3) id f0HJwv649221; Wed, 17 Jan 2001 11:58:57 -0800 (PST) (envelope-from dillon) Date: Wed, 17 Jan 2001 11:58:57 -0800 (PST) From: Matt Dillon Message-Id: <200101171958.f0HJwv649221@earth.backplane.com> To: Alfred Perlstein Cc: arch@FreeBSD.ORG Subject: Re: HEADS-UP: await/asleep removal imminent References: <200101171138.MAA11834@freebsd.dk> <20010117092109.O7240@fw.wintelcom.net> <20010117100516.Q7240@fw.wintelcom.net> <200101171907.f0HJ7Qe48680@earth.backplane.com> <20010117112850.X7240@fw.wintelcom.net> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :This is confusing: : :> :> Revamp asleep/await to be based on state variables rather then :> tsleep's traditionaln 'fake' addresses. Rather then :> have a traditional sleep/wakeup we instead have a state variable that :> asleep/await operate on. For example lets say we have a memory allocator. :> When the memory allocator finds it would block, it utilizes a global :> state structure representing the blockage and clears the state :> (blah.state = 0;). Then it calls asleep(&blah). asleep simply stores :> the pointer to &blah in the process structure, it does not try to queue :> it or do anything else. Thus no locking or mutexes or interrupt :> disablement is required *at all*. The routine is entirely passive and :> safe to call from anywhere. : :It's not queued _anywhere_? Right. Not queued anywhere... the await() (or tsleep or whatever) code would check for the existance of the pointer in the proc structure at the time the process tries to go to sleep. This is why you need a condition variable... because there is no way for an asynchronous event to 'wakeup' the process before it has tried to go to sleep for real the process needs some way to detect that the event has occured, hence the necessity of a condition variable. The traditional tsleep/wakeup code has *NO* persistent state associated with it, which means the process is required to place itself on the sleep queue atomically in order to avoid the event occuring between the resource allocation failure (or whatever) and the tsleep() call. :> to go to sleep for real. await() checks p_state->state and if it :> is zero await() places the process on the sleep queue for real and actually :> goes to sleep. If p_state->state is non-zero, await() simply clears :> the pointer (proc->p_state = NULL;) and returns (without sleeping). : :This is a bit weird, basically await doesn't enqueue the process, so :there's a weird race going on such that all "freeing" need to signal :rather than just when the pool is exhausted, in fact the pool may not be :exhausted but you still block because no one signal's after you await. await() in its current form doesn't enqueue the process because asleep() has already done it (in order for async events to be able to dequeue the process which turns the later await() into a NOP). That's the current state of the code... but the problem here is that in order for await() to enqueue the process await() needs to grab a mutex, which is expensive and nasty. So my proposal is to use a condition variable that holds the state of whether something will block or not persistently and have asleep() just store a reference to the condition variable. asleep() would NOT enqueue anything. await() would be responsible for checking the status of the condition variable and queueing the process to the appropriate sleep queue if the condition variable is zero. The exact reverse of the original implementation resulting in huge benefits in efficiency. If you think about it, it makes sense... you don't have to enqueue anything until you actually try to go to sleep for real (await()). The condition variable itself does not need to know that a process has asleep()'d on it until that process actually tries to await(). (though for safety you might want to have a simple ref count to track it. But it isn't required for the algorithm to work). -Matt :> I believe that this conforms to the state of the SMP system much much :> better then my original asleep()/await() implementation, and has the :> advantage of extremely *LOW* overhead (virtually none in many cases). :> :> You could then use it to give the 'power of blocking' to the caller :> rather then the callee. This in turn gives you much greater flexibility :> in regards to who can hold mutexes when and who can hold mutexes through :> procedure calls to other subsystems. : :Jason Evans just added his conditional variables, you could change :asleep/await into cv_attach/cv_await or something, that would give :you less contention on the sleep/run queues, but you'd still need :a spinlock or mutex on these variables. : :-- :-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] :"I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 12: 1:11 2001 Delivered-To: freebsd-arch@freebsd.org Received: from meow.osd.bsdi.com (meow.osd.bsdi.com [204.216.28.88]) by hub.freebsd.org (Postfix) with ESMTP id 109B337B6AD; Wed, 17 Jan 2001 12:00:53 -0800 (PST) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by meow.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id f0HJuqL40169; Wed, 17 Jan 2001 11:56:52 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <54515.979761135@critter> Date: Wed, 17 Jan 2001 11:58:46 -0800 (PST) From: John Baldwin To: Poul-Henning Kamp Subject: Re: HEADS-UP: await/asleep removal imminent Cc: current@FreeBSD.org, arch@FreeBSD.org, rjesup@wgate.com, bright@wintelcom.net, Soren Schmidt Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 17-Jan-01 Poul-Henning Kamp wrote: > > John, > > It might be nice with a "How to provide useful info on SMPng > problems." FAQ somewhere. > > I have no idea which of all the WITNESS and other options > do what and when they are useful... Fair enough. One thing on my todo list is a ktr.9 manpage documenting its use as it is one of the things I've found useful so far. Some other general docco I'll try and come up with as well. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 12:11:12 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id D318F37B6A2 for ; Wed, 17 Jan 2001 12:10:51 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f0HKAp029397; Wed, 17 Jan 2001 12:10:51 -0800 (PST) Date: Wed, 17 Jan 2001 12:10:51 -0800 From: Alfred Perlstein To: Matt Dillon Cc: arch@FreeBSD.ORG Subject: Re: HEADS-UP: await/asleep removal imminent Message-ID: <20010117121051.Z7240@fw.wintelcom.net> References: <200101171138.MAA11834@freebsd.dk> <20010117092109.O7240@fw.wintelcom.net> <20010117100516.Q7240@fw.wintelcom.net> <200101171907.f0HJ7Qe48680@earth.backplane.com> <20010117112850.X7240@fw.wintelcom.net> <200101171958.f0HJwv649221@earth.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200101171958.f0HJwv649221@earth.backplane.com>; from dillon@earth.backplane.com on Wed, Jan 17, 2001 at 11:58:57AM -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Matt Dillon [010117 11:59] wrote: > > :This is confusing: > : > :> > :> Revamp asleep/await to be based on state variables rather then > :> tsleep's traditionaln 'fake' addresses. Rather then > :> have a traditional sleep/wakeup we instead have a state variable that > :> asleep/await operate on. For example lets say we have a memory allocator. > :> When the memory allocator finds it would block, it utilizes a global > :> state structure representing the blockage and clears the state > :> (blah.state = 0;). Then it calls asleep(&blah). asleep simply stores > :> the pointer to &blah in the process structure, it does not try to queue > :> it or do anything else. Thus no locking or mutexes or interrupt > :> disablement is required *at all*. The routine is entirely passive and > :> safe to call from anywhere. > : > :It's not queued _anywhere_? > > Right. Not queued anywhere... the await() (or tsleep or whatever) > code would check for the existance of the pointer in the proc > structure at the time the process tries to go to sleep. > > This is why you need a condition variable... because there is no > way for an asynchronous event to 'wakeup' the process before it has > tried to go to sleep for real the process needs some way to detect > that the event has occured, hence the necessity of a condition > variable. > > The traditional tsleep/wakeup code has *NO* persistent state associated > with it, which means the process is required to place itself on the > sleep queue atomically in order to avoid the event occuring between the > resource allocation failure (or whatever) and the tsleep() call. I think you guys may be confused, I think you're mistaking our conditional variables with counting semaphores which act as sort of turnstiles(sp?) where you put in N tokens and only N procs can pass through them before blocking. The conditional variables as implemented in FreeBSD seem to only wake up something sleeping on them, they don't have the abilities you're looking for. -Alfred To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 12:15:42 2001 Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (placeholder-dcat-1076843399.broadbandoffice.net [64.47.83.135]) by hub.freebsd.org (Postfix) with ESMTP id 930BB37B400; Wed, 17 Jan 2001 12:15:24 -0800 (PST) Received: (from dillon@localhost) by earth.backplane.com (8.11.1/8.9.3) id f0HKEta49433; Wed, 17 Jan 2001 12:14:55 -0800 (PST) (envelope-from dillon) Date: Wed, 17 Jan 2001 12:14:55 -0800 (PST) From: Matt Dillon Message-Id: <200101172014.f0HKEta49433@earth.backplane.com> To: Jason Evans Cc: John Baldwin , arch@FreeBSD.ORG Subject: Re: HEADS-UP: await/asleep removal imminent References: <20010117100954.S61852@canonware.com> <20010117112232.U61852@canonware.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG One advantage to passing a condition variable back explicitly rather then storing it indirectly in curproc, aka something like this: condition *cv = NULL; rc = majorsubsystemroutine( ...., &cv) Is that it explicitly defines the API. If the procedure takes a condition variable, then you know the procedure might have a blocking condition associated with it which you must handle. If the procedure doesn't take a condition variable, then you know that the procedure is non-blocking(1) AND does *not* produce a blocking condition. In the indirect 'store in curproc' case you can't easily tell. So in that respect I really like the idea of passing the condition variable back through the subroutine layers explicitly, even though I don't like the idea of polluting existing APIs with the additional argument. One thing is absolutely crystal clear to me though: You cannot under any circumstances enqueue the process on the condition variable at the time it is passed back. You can only enqueue the process when you actually intend to go to sleep. Otherwise it wouldn't be an efficient enough interfafce. note(1): well, you don't know that it is non-blocking, because obviously many high level procedures like system calls will block inherently and not return any sort of condition variable. But for LOW LEVEL procedures the definition can hold true and would go a long way to reducing the confusion in the codebase. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 12:22:19 2001 Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 09FBC37B404; Wed, 17 Jan 2001 12:22:00 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id VAA44246; Wed, 17 Jan 2001 21:19:50 +0100 (CET) (envelope-from des@ofug.org) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Poul-Henning Kamp Cc: John Baldwin , Soren Schmidt , bright@wintelcom.net, rjesup@wgate.com, arch@FreeBSD.ORG, current@FreeBSD.ORG Subject: Re: HEADS-UP: await/asleep removal imminent References: <54515.979761135@critter> From: Dag-Erling Smorgrav Date: 17 Jan 2001 21:19:49 +0100 In-Reply-To: Poul-Henning Kamp's message of "Wed, 17 Jan 2001 20:52:15 +0100" Message-ID: Lines: 14 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Poul-Henning Kamp writes: > It might be nice with a "How to provide useful info on SMPng > problems." FAQ somewhere. I'd also like a "KTR for fun and profit" FAQ... > I have no idea which of all the WITNESS and other options > do what and when they are useful... WITNESS, at least, is fairly well documented in NOTES. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 12:30:21 2001 Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (placeholder-dcat-1076843399.broadbandoffice.net [64.47.83.135]) by hub.freebsd.org (Postfix) with ESMTP id 8BE3137B6DC for ; Wed, 17 Jan 2001 12:29:52 -0800 (PST) Received: (from dillon@localhost) by earth.backplane.com (8.11.1/8.9.3) id f0HKTk649628; Wed, 17 Jan 2001 12:29:46 -0800 (PST) (envelope-from dillon) Date: Wed, 17 Jan 2001 12:29:46 -0800 (PST) From: Matt Dillon Message-Id: <200101172029.f0HKTk649628@earth.backplane.com> To: Alfred Perlstein Cc: arch@FreeBSD.ORG Subject: Re: HEADS-UP: await/asleep removal imminent References: <200101171138.MAA11834@freebsd.dk> <20010117092109.O7240@fw.wintelcom.net> <20010117100516.Q7240@fw.wintelcom.net> <200101171907.f0HJ7Qe48680@earth.backplane.com> <20010117112850.X7240@fw.wintelcom.net> <200101171958.f0HJwv649221@earth.backplane.com> <20010117121051.Z7240@fw.wintelcom.net> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :> :It's not queued _anywhere_? :> :> Right. Not queued anywhere... the await() (or tsleep or whatever) :> code would check for the existance of the pointer in the proc :> structure at the time the process tries to go to sleep. :> :> This is why you need a condition variable... because there is no :> way for an asynchronous event to 'wakeup' the process before it has :> tried to go to sleep for real the process needs some way to detect :> that the event has occured, hence the necessity of a condition :> variable. :> :> The traditional tsleep/wakeup code has *NO* persistent state associated :> with it, which means the process is required to place itself on the :> sleep queue atomically in order to avoid the event occuring between the :> resource allocation failure (or whatever) and the tsleep() call. : :I think you guys may be confused, I think you're mistaking our :conditional variables with counting semaphores which act as sort :of turnstiles(sp?) where you put in N tokens and only N procs can :pass through them before blocking. : :The conditional variables as implemented in FreeBSD seem to only :wake up something sleeping on them, they don't have the abilities :you're looking for. : :-Alfred Well, I haven't read the code so I don't know... what I'm saying though is that it isn't a hard thing to do at all. If you are going to have a persistent structure, you can have persistent state in that structure. One of the reasons why the original asleep/await interface is so expensive is because it is based on non-persistent information and requires queueing from the get-go to deal with wakeups. Here is an example of what I am suggesting. I'll call these 'state' variables rather then condition variables to avoid confusion: typedef struct StateVar { int State; /* 0= resource not avail, 1=available */ List WaitQ; /* processes waiting for 0->1 transition */ Mutex *MutexNeeded; } StateVar; StateVar MemState = { 1, ... , &memory_mutex }; void * memory_allocate(int bytes, StateVar **sv) { Obtain memory mutex ... if (no memory available) { MemState.State = 0; *sv = &MemState; ptr = NULL; /* NULL return indicates wait condition */ } Release memory mutex return(ptr); } void memory_free(void *ptr) { Obtain memory mutex ... do free operation if (reasonable memory free based on hysteresis) { MemState.State = 1; WakeupWaiters(&MemState); } Release memory mutex } ------------------------------------------- somefunction() { StateVar *sv = NULL; while (memory_allocate(bytes, &sv) == NULL) { .... window of opportunity where some other process may release enough memory to set State variable to 1. This case is handled by the implementation ..... await(sv); /* or await(&sv) and await sets it to NULL */ sv = NULL; } } ---------------------------------------------- await(StateVar *sv) { if (sv->State == 0) { ObtainMutex(sv->sv_MutexNeeded); while (sv->State == 0) { Enqueue process on sv->WaitQ Go to sleep for real } ReleaseMutex(sv->sv_MutexNeeded); } } To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 13:19:46 2001 Delivered-To: freebsd-arch@freebsd.org Received: from meow.osd.bsdi.com (meow.osd.bsdi.com [204.216.28.88]) by hub.freebsd.org (Postfix) with ESMTP id 6129237B401; Wed, 17 Jan 2001 13:19:27 -0800 (PST) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by meow.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id f0HLGwL42089; Wed, 17 Jan 2001 13:16:58 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Wed, 17 Jan 2001 13:18:53 -0800 (PST) From: John Baldwin To: Dag-Erling Smorgrav Subject: Re: HEADS-UP: await/asleep removal imminent Cc: current@FreeBSD.org, arch@FreeBSD.org, rjesup@wgate.com, bright@wintelcom.net, Soren Schmidt , Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 17-Jan-01 Dag-Erling Smorgrav wrote: > Poul-Henning Kamp writes: >> It might be nice with a "How to provide useful info on SMPng >> problems." FAQ somewhere. > > I'd also like a "KTR for fun and profit" FAQ... ktr.9 is next on my todo list of manpages to write. I have several sitting in the queue pending review right now, but that is the next one I can start writing, hopefully sometime this week. Now that my release build box is stable and reliable again, I'm testing out releases with debug options turned on by default in GENERIC. I'm also playing with having a kernel.debug in the kernel snap by default as well. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 14:35:44 2001 Delivered-To: freebsd-arch@freebsd.org Received: from VL-MS-MR002.sc1.videotron.ca (relais.videotron.ca [24.201.245.36]) by hub.freebsd.org (Postfix) with ESMTP id 857C837B401; Wed, 17 Jan 2001 14:35:22 -0800 (PST) Received: from jehovah ([24.201.144.31]) by VL-MS-MR002.sc1.videotron.ca (Netscape Messaging Server 4.15) with SMTP id G7BW2W00.MIA; Wed, 17 Jan 2001 17:35:20 -0500 Message-ID: <012a01c080d5$fc532270$1f90c918@jehovah> From: "Bosko Milekic" To: "Alfred Perlstein" , "Randell Jesup" , Cc: "Soren Schmidt" , , References: <200101171138.MAA11834@freebsd.dk> <20010117092109.O7240@fw.wintelcom.net> <20010117100516.Q7240@fw.wintelcom.net> Subject: Re: HEADS-UP: await/asleep removal imminent Date: Wed, 17 Jan 2001 17:36:50 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2919.6700 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Alfred Perlstein wrote: > * Alfred Perlstein [010117 09:24] wrote: > > > > I'm not going to axe it for a few days, this is a really amazing > > API that Matt added, the problem is utility and useage over code > > complexity. > > > > It's just a proposal. > > I found several places where it may be useful, but I'm not sure if the > benefits outweigh the gains. > > In a copy of the tree I've locked down the socket layer (not the entire > stack, just sockets :) ) there's code like this: > > SOCKBUF_UNLOCK(&so->so_snd, 0); > if (top == 0) { > MGETHDR(m, M_TRYWAIT, MT_DATA); > if (m == NULL) { > error = ENOBUFS; > SOCKBUF_LOCK(&so->so_snd, 0); > goto release; > } > mlen = MHLEN; > ... > SOCKBUF_LOCK(&so->so_snd, 0); /* XXX */ > > The lock must be unwound becasue we're calling MGETHDR with M_TRYWAIT. > If wae used M_TRY'A'WAIT the code would probably look something like > this: > > /* SOCKBUF_UNLOCK(&so->so_snd, 0); */ > again: > if (top == 0) { > MGETHDR(m, M_TRYWAIT, MT_DATA); > if (m == NULL) { > error = mawait(&so->so_snd.sb_mtx, -1, -1); > if (error) { > if (error == EWOULDBLOCK) > error = ENOBUFS; > goto release; > } > goto again; > /* SOCKBUF_LOCK(&so->so_snd, 0); */ > } > mlen = MHLEN; > ... > /* SOCKBUF_LOCK(&so->so_snd, 0); */ /* XXX */ > > Which means we don't have to drop the lock over the socket unless > we'd block on allocation. No. You'd still have to drop it for now. Remember? (Last commit to uipc_mbuf.c). You have to drop it because of the problem you may have if Giant is gotten before your sockbuf/socket lock. In the allocation, you may be acquiring Giant again. I don't know the exact semantics, but if you at some point may grab the sockbuf/socket lock without already holding Giant and call the allocation routine, you're opening the door for deadlock. > Matt, is this what you intended for it to do? So far I've only > seen it used to avoid races, but not to optimize out mutex > aquire/release. I've only seen it to be useful to avoid races. If you're holding a lock and you need to sleep but if you drop the lock before you actually switch you may get woken up and never find out, thus still going to sleep. With the asleep you could hold the lock and place yourself on the sleep queue such that when you drop the lock and call await, you'll find out if you've gotten awoken (you'll be removed from the sleep queue). With the interlocking with sched_lock now down in msleep(), this "feature" of asleep/await is useless. > -- > -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] > "I have the heart of a child; I keep it in a jar on my desk." Regards, Bosko. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 15:23:11 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id ECA8937B400; Wed, 17 Jan 2001 15:22:52 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f0HNMoQ05449; Wed, 17 Jan 2001 15:22:50 -0800 (PST) Date: Wed, 17 Jan 2001 15:22:50 -0800 From: Alfred Perlstein To: Bosko Milekic Cc: Randell Jesup , dillon@FreeBSD.ORG, Soren Schmidt , arch@FreeBSD.ORG, current@FreeBSD.ORG Subject: Re: HEADS-UP: await/asleep removal imminent Message-ID: <20010117152250.D7240@fw.wintelcom.net> References: <200101171138.MAA11834@freebsd.dk> <20010117092109.O7240@fw.wintelcom.net> <20010117100516.Q7240@fw.wintelcom.net> <012a01c080d5$fc532270$1f90c918@jehovah> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <012a01c080d5$fc532270$1f90c918@jehovah>; from bmilekic@technokratis.com on Wed, Jan 17, 2001 at 05:36:50PM -0500 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Bosko Milekic [010117 14:35] wrote: > > Alfred Perlstein wrote: > > > Which means we don't have to drop the lock over the socket unless > > we'd block on allocation. > > No. You'd still have to drop it for now. Remember? (Last commit to > uipc_mbuf.c). You have to drop it because of the problem you may have if > Giant is gotten before your sockbuf/socket lock. In the allocation, you may > be acquiring Giant again. I don't know the exact semantics, but if you at > some point may grab the sockbuf/socket lock without already holding Giant and > call the allocation routine, you're opening the door for deadlock. The rest of the patches unwind giant on entry into the socket layer so grabbing Giant shouldn't be a problem. > > Matt, is this what you intended for it to do? So far I've only > > seen it used to avoid races, but not to optimize out mutex > > aquire/release. > > I've only seen it to be useful to avoid races. If you're holding a lock > and you need to sleep but if you drop the lock before you actually switch you > may get woken up and never find out, thus still going to sleep. With the > asleep you could hold the lock and place yourself on the sleep queue such > that when you drop the lock and call await, you'll find out if you've gotten > awoken (you'll be removed from the sleep queue). With the interlocking with > sched_lock now down in msleep(), this "feature" of asleep/await is useless. Read the rest of my posts, the ones not cc'd to everyone, just Matt and -arch. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 19: 8: 1 2001 Delivered-To: freebsd-arch@freebsd.org Received: from relay.butya.kz (butya-gw.butya.kz [212.154.129.94]) by hub.freebsd.org (Postfix) with ESMTP id 14A6437B400; Wed, 17 Jan 2001 19:07:39 -0800 (PST) Received: by relay.butya.kz (Postfix, from userid 1000) id 1CF5728811; Thu, 18 Jan 2001 09:07:32 +0600 (ALMT) Received: from localhost (localhost [127.0.0.1]) by relay.butya.kz (Postfix) with ESMTP id 0D76328804; Thu, 18 Jan 2001 09:07:32 +0600 (ALMT) Date: Thu, 18 Jan 2001 09:07:31 +0600 (ALMT) From: Boris Popov To: Alfred Perlstein Cc: arch@freebsd.org, current@freebsd.org Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: <20010117031618.K7240@fw.wintelcom.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Wed, 17 Jan 2001, Alfred Perlstein wrote: > I have a patch here that removes await/asleep from the kernel API. > > http://people.freebsd.org/~alfred/noasleep.diff > > Matt Dillon implemented alseep/await quite some time ago and the > only thing that's using it is ata. In order to clean up some of > the schduler and vm system I'm removing support for it. Hmm, I've found this API usable for synchronization between two kernel threads/processes: alseep(&somewhere); wakeup(&anotherprocess); await(&somewhere); where 'anotherprocess' do some job and call wakeup(&somewhere). I suspect that this can be emulated with mutexes and/or conditional variables. So, it is possible to keep the API, but change its implementation to allow mentioned cleanup of scheduler ? P.S. it is already too hard to maintain external kernel modules even within various releng_4s, not to mention -current and releng_3... -- Boris Popov http://www.butya.kz/~bp/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 19:30:55 2001 Delivered-To: freebsd-arch@freebsd.org Received: from smtp10.phx.gblx.net (smtp10.phx.gblx.net [206.165.6.140]) by hub.freebsd.org (Postfix) with ESMTP id 836D737B400; Wed, 17 Jan 2001 19:30:38 -0800 (PST) Received: (from daemon@localhost) by smtp10.phx.gblx.net (8.9.3/8.9.3) id UAA100182; Wed, 17 Jan 2001 20:30:16 -0700 Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp10.phx.gblx.net, id smtpdzDbmEa; Wed Jan 17 20:30:12 2001 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id UAA25782; Wed, 17 Jan 2001 20:30:19 -0700 (MST) From: Terry Lambert Message-Id: <200101180330.UAA25782@usr08.primenet.com> Subject: Re: HEADS-UP: await/asleep removal imminent To: jhb@FreeBSD.ORG (John Baldwin) Date: Thu, 18 Jan 2001 03:30:19 +0000 (GMT) Cc: jasone@canonware.com (Jason Evans), arch@FreeBSD.ORG, rjesup@wgate.com (Randell Jesup), bright@wintelcom.net (Alfred Perlstein) In-Reply-To: from "John Baldwin" at Jan 17, 2001 10:55:09 AM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Well, it will be unused if we axe all tsleep's in favor of cv's which does > incur extra overhead, as each cv has to be init'd and destroy'd and carries a > linked list around with it. The extra storage overhead doesn't outweight the > speed increase (from lack of the hash lookup) in all cases I think, so I'm not > sure we want to axe tsleep() just yet. If you axe tsleep() then asleep() can > be emulated by either passing cv's around between functions. I'll ask the same question I asked the POSIX committee about mutex initialization: Why is a non-default initialization required? Why explicitly choose an implementation that doesn't permit static instances to be declared and used, without explicit initialization? What is the freaking problem with "zero" not being a perfectly valid number in this warped philosophy? Pick a "just initialized" state that equals "all zeros". Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 22:27: 7 2001 Delivered-To: freebsd-arch@freebsd.org Received: from hand.dotat.at (sfo-gw.covalent.net [207.44.198.62]) by hub.freebsd.org (Postfix) with ESMTP id 40C9837B404; Wed, 17 Jan 2001 22:26:51 -0800 (PST) Received: from fanf by hand.dotat.at with local (Exim 3.15 #3) id 14J8Wy-000JJe-00; Thu, 18 Jan 2001 06:26:44 +0000 Date: Thu, 18 Jan 2001 06:26:44 +0000 From: Tony Finch To: Mike Smith Cc: Kirk McKusick , arch@FreeBSD.ORG Subject: Re: dynamic vs static sysctls? Message-ID: <20010118062644.D30538@hand.dotat.at> References: <200101152345.PAA22257@beastie.mckusick.com> <200101160727.f0G7Rss00920@mass.osd.bsdi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: <200101160727.f0G7Rss00920@mass.osd.bsdi.com> Organization: Covalent Technologies, Inc Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Mike Smith wrote: > >> In my work on a background version of fsck, I have used sysctl to >> allow me to pass information into the kernel that I want to have >> updated in the filesystem. > >I'm not convinced that sysctl is the "right" way to go about doing this, >really. But I can't think of a better one. 8) Why not an ioctl on the disk device? You could arrange to pass in an array of free blocks to reduce the number of syscalls. Tony. -- f.a.n.finch fanf@covalent.net dot@dotat.at "Then they attacked a town. A small town, I'll admit. But nevertheless a town of people. People who died." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 22:40: 4 2001 Delivered-To: freebsd-arch@freebsd.org Received: from VL-MS-MR002.sc1.videotron.ca (relais.videotron.ca [24.201.245.36]) by hub.freebsd.org (Postfix) with ESMTP id 76ECE37B69E; Wed, 17 Jan 2001 22:39:42 -0800 (PST) Received: from jehovah ([24.201.144.31]) by VL-MS-MR002.sc1.videotron.ca (Netscape Messaging Server 4.15) with SMTP id G7CII505.GOU; Thu, 18 Jan 2001 01:39:41 -0500 Message-ID: <001d01c08119$a62482b0$1f90c918@jehovah> From: "Bosko Milekic" To: Cc: "John Baldwin" , Subject: mtx_init: MTX_RECURSE and potential SMP troublespots Date: Thu, 18 Jan 2001 01:41:12 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2919.6700 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Hello, This patch: http://people.freebsd.org/~bmilekic/code/recur_witn.diff introduces MTX_RECURSE (the old MTX_RECURSE state bit is renamed to MTX_RECURSED) and makes passing the MTX_RECURSE bit to mtx_init() get registered in the witness code such that during execution, the WITNESS portion of mtx_enter/mtx_exit (witness_enter/witness_exit) will call panic if a mutex is recursed but did not have this bit in its flags during initialization. This is a first change for debugging purposes mainly in preparation for a modified, cleaner, mtx_*() interface which is likely to be coming to -CURRENT eventually. The diff also cleans up some mtx_recurse != 0 checks to use the already provided mtx_recursed() macros. Final reviews are welcome. (LINT builds with this, by the way, and -CURRENT on my machine seems to be going fine, so far). Now, the important bit. The patch attempts to add MTX_RECURSE to existing mutex locks that are known to recurse. Amongst these, we have: eventhandler, Giant, callout, sched_lock, and a whole bunch of network card drivers in pci/* and company, as well as some others which I was able to find out recurse. If you are a maintainer or have added locks to some subsystem, please glance at the patch and make sure that if your locks recurse, that their mtx_init() has been changed to include the MTX_RECURSE bit. If this is not the case, please let us know ASAP. Furthermore, if you've added a lock that you know/think should not be recursing and you see its mtx_init() call was modified in the patch to include the MTX_RECURSE bit, also please let us know. While going around the code, I've spotted some definite problems, and some potential problems. There are likely more which I've missed, so if you've added any locks, please make sure to review for similar problems. sys/dev/acpica/Osd/OsdSynch.c probably has a bug where AcpiOsDeleteSemaphore() does not mtx_destroy() on its previously mtx_init()ed mutex locks. sys/dev/ichsmb passes MTX_NORECURSE to mtx_init(), when it's only supposed to be passed to mtx_exit(). Apparently, this is not in mutex(9) [yet?] sys/dev/isp probably calls tsleep() with a mutex held and has a wakeup() for that sleep surrounded by the mutex lock's acquire/drop. sys/dev/pccbb seems to protect wakeup() calls w/ a mutex enter/exit. This is odd because just code is typically not supposed to be protected with a mutex. This may be a problem because I believe I saw this code drop the mutex before calling tsleep() which may be a race because in this case, a wakeup() can come before the actual tsleep() is entered (or before the switch happens) and therefore the thread will go to sleep even if a wakeup() was already issued just prior to. sys/dev/yarrow.c "may" be violating lock order (there _may_ be a lock order reversal); the fact that printf()s are surrounded with Giant lock enter/exit and that some of the routines containing the printf() calls are called with the driver's lock held and that some of them aren't may be a problem if those that are not do attempt to acquire the driver lock and are entered with Giant already held (from somewhere). There may be a lock reversal; but I haven't been able to investigate sufficiently to be able to claim this with absolute certainty. Regards, Bosko. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 23: 6:42 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 69AB937B402; Wed, 17 Jan 2001 23:06:26 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f0I76MJ18369; Wed, 17 Jan 2001 23:06:22 -0800 (PST) Date: Wed, 17 Jan 2001 23:06:22 -0800 From: Alfred Perlstein To: Tony Finch Cc: Mike Smith , Kirk McKusick , arch@FreeBSD.ORG Subject: Re: dynamic vs static sysctls? Message-ID: <20010117230622.K7240@fw.wintelcom.net> References: <200101152345.PAA22257@beastie.mckusick.com> <200101160727.f0G7Rss00920@mass.osd.bsdi.com> <20010118062644.D30538@hand.dotat.at> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010118062644.D30538@hand.dotat.at>; from dot@dotat.at on Thu, Jan 18, 2001 at 06:26:44AM +0000 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Tony Finch [010117 22:27] wrote: > Mike Smith wrote: > > > >> In my work on a background version of fsck, I have used sysctl to > >> allow me to pass information into the kernel that I want to have > >> updated in the filesystem. > > > >I'm not convinced that sysctl is the "right" way to go about doing this, > >really. But I can't think of a better one. 8) > > Why not an ioctl on the disk device? You could arrange to pass in an > array of free blocks to reduce the number of syscalls. It's not a disk action, it's an FS action, an fsctl call might be handy, or a completely static sysctl, but not a disk device ioctl. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 23:30:27 2001 Delivered-To: freebsd-arch@freebsd.org Received: from moby.geekhouse.net (moby.geekhouse.net [64.81.6.36]) by hub.freebsd.org (Postfix) with ESMTP id 3F07C37B401 for ; Wed, 17 Jan 2001 23:30:09 -0800 (PST) Received: from laptop.baldwin.cx (john@dhcp150.geekhouse.net [192.168.1.150]) by moby.geekhouse.net (8.11.0/8.9.3) with ESMTP id f0I7Xos67077; Wed, 17 Jan 2001 23:33:58 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200101180330.UAA25782@usr08.primenet.com> Date: Wed, 17 Jan 2001 23:29:47 -0800 (PST) From: John Baldwin To: Terry Lambert Subject: Re: HEADS-UP: await/asleep removal imminent Cc: (Alfred Perlstein) , (Randell Jesup) , arch@FreeBSD.org, (Jason Evans) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 18-Jan-01 Terry Lambert wrote: >> Well, it will be unused if we axe all tsleep's in favor of cv's which does >> incur extra overhead, as each cv has to be init'd and destroy'd and carries >> a >> linked list around with it. The extra storage overhead doesn't outweight >> the >> speed increase (from lack of the hash lookup) in all cases I think, so I'm >> not >> sure we want to axe tsleep() just yet. If you axe tsleep() then asleep() >> can >> be emulated by either passing cv's around between functions. > > I'll ask the same question I asked the POSIX committee about > mutex initialization: > > Why is a non-default initialization required? > > Why explicitly choose an implementation that doesn't permit > static instances to be declared and used, without explicit > initialization? > > > What is the freaking problem with "zero" not being a perfectly > valid number in this warped philosophy? > > > Pick a "just initialized" state that equals "all zeros". Well, you could do an evil ugly gross hack trick thing where on the first action on the cv you check if it is all zero's and init it if not, but that is just gross. What is wrong with properly keeping track of your data structures anyways? -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 23:30:33 2001 Delivered-To: freebsd-arch@freebsd.org Received: from moby.geekhouse.net (moby.geekhouse.net [64.81.6.36]) by hub.freebsd.org (Postfix) with ESMTP id BD4C237B402; Wed, 17 Jan 2001 23:30:11 -0800 (PST) Received: from laptop.baldwin.cx (john@dhcp150.geekhouse.net [192.168.1.150]) by moby.geekhouse.net (8.11.0/8.9.3) with ESMTP id f0I7Y0s67081; Wed, 17 Jan 2001 23:34:05 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <001d01c08119$a62482b0$1f90c918@jehovah> Date: Wed, 17 Jan 2001 23:29:57 -0800 (PST) From: John Baldwin To: Bosko Milekic Subject: RE: mtx_init: MTX_RECURSE and potential SMP troublespots Cc: jasone@FreeBSD.org, arch@FreeBSD.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 18-Jan-01 Bosko Milekic wrote: > sys/dev/ichsmb passes MTX_NORECURSE to mtx_init(), when it's only supposed to > be passed to mtx_exit(). Apparently, this is not in mutex(9) [yet?] Also, MTX_NORECURSE is only useful on spin mutexes. If you compile a kernel with MUTEX_DEBUG and ichsmb, it panics during boot. All the MTX_NORECURSE flags just need to die here. > sys/dev/isp probably calls tsleep() with a mutex held and has a wakeup() for > that sleep surrounded by the mutex lock's acquire/drop. It should use msleep() instead then. The mutex around wakeup() isn't wrong, though not necessarily optimal. > sys/dev/pccbb seems to protect wakeup() calls w/ a mutex enter/exit. This is > odd because just code is typically not supposed to be protected with a mutex. > This may be a problem because I believe I saw this code drop the mutex before > calling tsleep() which may be a race because in this case, a wakeup() can > come before the actual tsleep() is entered (or before the switch happens) and > therefore the thread will go to sleep even if a wakeup() was already issued > just prior to. You have to use msleep() instead of mtx_exit/tsleep(). I have several other changes to the pccbb mutexes in my tree however. :) > sys/dev/yarrow.c "may" be violating lock order (there _may_ be a lock order > reversal); the fact that printf()s are surrounded with Giant lock enter/exit > and that some of the routines containing the printf() calls are called with > the driver's lock held and that some of them aren't may be a problem if those > that are not do attempt to acquire the driver lock and are entered with Giant > already held (from somewhere). There may be a lock reversal; but I haven't > been able to investigate sufficiently to be able to claim this with absolute > certainty. Hmm, haven't looked at this yet. > Regards, > Bosko. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jan 17 23:39:36 2001 Delivered-To: freebsd-arch@freebsd.org Received: from smtp04.primenet.com (smtp04.primenet.com [206.165.6.134]) by hub.freebsd.org (Postfix) with ESMTP id A78A537B402; Wed, 17 Jan 2001 23:39:16 -0800 (PST) Received: (from daemon@localhost) by smtp04.primenet.com (8.9.3/8.9.3) id AAA03583; Thu, 18 Jan 2001 00:34:19 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp04.primenet.com, id smtpdAAAgDai_g; Thu Jan 18 00:34:16 2001 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id AAA00872; Thu, 18 Jan 2001 00:39:11 -0700 (MST) From: Terry Lambert Message-Id: <200101180739.AAA00872@usr08.primenet.com> Subject: Re: dynamic vs static sysctls? To: bright@wintelcom.net (Alfred Perlstein) Date: Thu, 18 Jan 2001 07:39:11 +0000 (GMT) Cc: dot@dotat.at (Tony Finch), msmith@FreeBSD.ORG (Mike Smith), mckusick@mckusick.com (Kirk McKusick), arch@FreeBSD.ORG In-Reply-To: <20010117230622.K7240@fw.wintelcom.net> from "Alfred Perlstein" at Jan 17, 2001 11:06:22 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > >> In my work on a background version of fsck, I have used sysctl to > > >> allow me to pass information into the kernel that I want to have > > >> updated in the filesystem. > > > > > >I'm not convinced that sysctl is the "right" way to go about doing this, > > >really. But I can't think of a better one. 8) > > > > Why not an ioctl on the disk device? You could arrange to pass in an > > array of free blocks to reduce the number of syscalls. > > It's not a disk action, it's an FS action, an fsctl call might be handy, > or a completely static sysctl, but not a disk device ioctl. FWIW, this really depends on whose job you think it is to keep track of bad blocks and virtually "fix" them. SVR4.2 and above, for example, do the bad-block fixing and retries at or below the device driver level. Using this approach, bad blocks would never be known to the FS under any circumstances. There was a nice bug in the SVR4.2 Veritas implementation that used to make /usr disappear back when UnixWare was at "SVR4.2-prerelease", where soft errors were reported outside the device driver to the FS code, instead of "soft fixed" before the FS code ever saw them. Something to gloat about when the Linux Veritas port is released, until they fix their drivers to only report up unrecoverable hard errors, too, I guess, and a barrier to entry for a BSD Veritas port, until BSD does the same, or Veritas becomes irrelevent (may be quite a while). Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 0: 2:21 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 404A637B402; Thu, 18 Jan 2001 00:02:04 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f0I822t19689; Thu, 18 Jan 2001 00:02:02 -0800 (PST) Date: Thu, 18 Jan 2001 00:02:02 -0800 From: Alfred Perlstein To: Terry Lambert Cc: Tony Finch , Mike Smith , Kirk McKusick , arch@FreeBSD.ORG Subject: Re: dynamic vs static sysctls? Message-ID: <20010118000202.L7240@fw.wintelcom.net> References: <20010117230622.K7240@fw.wintelcom.net> <200101180739.AAA00872@usr08.primenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200101180739.AAA00872@usr08.primenet.com>; from tlambert@primenet.com on Thu, Jan 18, 2001 at 07:39:11AM +0000 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Terry Lambert [010117 23:39] wrote: > > > >> In my work on a background version of fsck, I have used sysctl to > > > >> allow me to pass information into the kernel that I want to have > > > >> updated in the filesystem. > > > > > > > >I'm not convinced that sysctl is the "right" way to go about doing this, > > > >really. But I can't think of a better one. 8) > > > > > > Why not an ioctl on the disk device? You could arrange to pass in an > > > array of free blocks to reduce the number of syscalls. > > > > It's not a disk action, it's an FS action, an fsctl call might be handy, > > or a completely static sysctl, but not a disk device ioctl. > > FWIW, this really depends on whose job you think it is to > keep track of bad blocks and virtually "fix" them. We're talking about block changes that relate in in-core filesystem data, not just bad-block remapping. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 0: 3:17 2001 Delivered-To: freebsd-arch@freebsd.org Received: from freebsd.dk (freebsd.dk [212.242.42.178]) by hub.freebsd.org (Postfix) with ESMTP id 2D7F137B400; Thu, 18 Jan 2001 00:02:57 -0800 (PST) Received: (from sos@localhost) by freebsd.dk (8.9.3/8.9.1) id JAA03496; Thu, 18 Jan 2001 09:02:55 +0100 (CET) (envelope-from sos) From: Soren Schmidt Message-Id: <200101180802.JAA03496@freebsd.dk> Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: from John Baldwin at "Jan 17, 2001 11:41:56 am" To: jhb@FreeBSD.ORG (John Baldwin) Date: Thu, 18 Jan 2001 09:02:55 +0100 (CET) Cc: bright@wintelcom.net, rjesup@wgate.com, arch@FreeBSD.ORG, current@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL54 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG It seems John Baldwin wrote: > > Anyhow, I have asked before to have you guys supply me with > > a kernel that has been compiled "the right way" and I'll test > > it out here just to make sure I dont do anything stupid.. > > > > Just a bare bones kernel, fxp & ata drivers will do nicely for the > > serverworks thingy, give me an URL I can get it from or put it > > on ftp.freebsd.dk/incoming and give my a ping when its there... > > http://www.FreeBSD.org/~jhb/kernel.sos > > It was compiled wtih INCLUDE_CONFIG_FILE, so you can grab the config file from > that. Now this is "interesting", I booted on your kernel, and it has run through make world 4 times in a row... So I felt lucky, checked out a new fresh src/sys tree, and made a new kernel from the config file you used, reboot, starts test, and *HANG* Hmm!! Now I have tried a sys tree from today (0800 CET) two days ago and a week ago, and all 3 of them produces the lockup. So the only thing I can think of is that you guys have something in your src trees that cvs & I dont... Now what ? -Søren To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 0: 9: 1 2001 Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id C705A37B402; Thu, 18 Jan 2001 00:08:38 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id f0I87M419862; Thu, 18 Jan 2001 00:07:22 -0800 (PST) Date: Thu, 18 Jan 2001 00:07:22 -0800 From: Alfred Perlstein To: Soren Schmidt Cc: John Baldwin , rjesup@wgate.com, arch@FreeBSD.ORG, current@FreeBSD.ORG Subject: Re: HEADS-UP: await/asleep removal imminent Message-ID: <20010118000721.M7240@fw.wintelcom.net> References: <200101180802.JAA03496@freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200101180802.JAA03496@freebsd.dk>; from sos@freebsd.dk on Thu, Jan 18, 2001 at 09:02:55AM +0100 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Soren Schmidt [010118 00:03] wrote: > It seems John Baldwin wrote: > > > Anyhow, I have asked before to have you guys supply me with > > > a kernel that has been compiled "the right way" and I'll test > > > it out here just to make sure I dont do anything stupid.. > > > > > > Just a bare bones kernel, fxp & ata drivers will do nicely for the > > > serverworks thingy, give me an URL I can get it from or put it > > > on ftp.freebsd.dk/incoming and give my a ping when its there... > > > > http://www.FreeBSD.org/~jhb/kernel.sos > > > > It was compiled wtih INCLUDE_CONFIG_FILE, so you can grab the config file from > > that. > > Now this is "interesting", I booted on your kernel, and it has run > through make world 4 times in a row... > > So I felt lucky, checked out a new fresh src/sys tree, and made a new > kernel from the config file you used, reboot, starts test, and *HANG* > > Hmm!! > > Now I have tried a sys tree from today (0800 CET) two days ago and a > week ago, and all 3 of them produces the lockup. > > So the only thing I can think of is that you guys have something in > your src trees that cvs & I dont... > > Now what ? What are the compile flags you are using? -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 0:16:10 2001 Delivered-To: freebsd-arch@freebsd.org Received: from freebsd.dk (freebsd.dk [212.242.42.178]) by hub.freebsd.org (Postfix) with ESMTP id CC10D37B402; Thu, 18 Jan 2001 00:15:47 -0800 (PST) Received: (from sos@localhost) by freebsd.dk (8.9.3/8.9.1) id JAA06438; Thu, 18 Jan 2001 09:15:30 +0100 (CET) (envelope-from sos) From: Soren Schmidt Message-Id: <200101180815.JAA06438@freebsd.dk> Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: <20010118000721.M7240@fw.wintelcom.net> from Alfred Perlstein at "Jan 18, 2001 00:07:22 am" To: bright@wintelcom.net (Alfred Perlstein) Date: Thu, 18 Jan 2001 09:15:28 +0100 (CET) Cc: jhb@FreeBSD.ORG (John Baldwin), rjesup@wgate.com, arch@FreeBSD.ORG, current@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL54 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG It seems Alfred Perlstein wrote: > > Now this is "interesting", I booted on your kernel, and it has run > > through make world 4 times in a row... > > > > So I felt lucky, checked out a new fresh src/sys tree, and made a new > > kernel from the config file you used, reboot, starts test, and *HANG* > > > > Hmm!! > > > > Now I have tried a sys tree from today (0800 CET) two days ago and a > > week ago, and all 3 of them produces the lockup. > > > > So the only thing I can think of is that you guys have something in > > your src trees that cvs & I dont... > > > > Now what ? > > What are the compile flags you are using? Run of the mill std /etc/defaults/make.conf nothing else, just copied from /usr/src/defaults/make.conf... And a clean env also btw... -Søren To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 0:33:53 2001 Delivered-To: freebsd-arch@freebsd.org Received: from moby.geekhouse.net (moby.geekhouse.net [64.81.6.36]) by hub.freebsd.org (Postfix) with ESMTP id 36FA537B400; Thu, 18 Jan 2001 00:33:33 -0800 (PST) Received: from laptop.baldwin.cx (john@dhcp150.geekhouse.net [192.168.1.150]) by moby.geekhouse.net (8.11.0/8.9.3) with ESMTP id f0I8bXs67198; Thu, 18 Jan 2001 00:37:33 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20010118000721.M7240@fw.wintelcom.net> Date: Thu, 18 Jan 2001 00:33:30 -0800 (PST) From: John Baldwin To: Alfred Perlstein Subject: Re: HEADS-UP: await/asleep removal imminent Cc: current@FreeBSD.org, arch@FreeBSD.org, rjesup@wgate.com, Soren Schmidt Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 18-Jan-01 Alfred Perlstein wrote: > * Soren Schmidt [010118 00:03] wrote: >> It seems John Baldwin wrote: >> > > Anyhow, I have asked before to have you guys supply me with >> > > a kernel that has been compiled "the right way" and I'll test >> > > it out here just to make sure I dont do anything stupid.. >> > > >> > > Just a bare bones kernel, fxp & ata drivers will do nicely for the >> > > serverworks thingy, give me an URL I can get it from or put it >> > > on ftp.freebsd.dk/incoming and give my a ping when its there... >> > >> > http://www.FreeBSD.org/~jhb/kernel.sos >> > >> > It was compiled wtih INCLUDE_CONFIG_FILE, so you can grab the config file >> > from >> > that. >> >> Now this is "interesting", I booted on your kernel, and it has run >> through make world 4 times in a row... >> >> So I felt lucky, checked out a new fresh src/sys tree, and made a new >> kernel from the config file you used, reboot, starts test, and *HANG* >> >> Hmm!! >> >> Now I have tried a sys tree from today (0800 CET) two days ago and a >> week ago, and all 3 of them produces the lockup. >> >> So the only thing I can think of is that you guys have something in >> your src trees that cvs & I dont... >> >> Now what ? > > What are the compile flags you are using? I actually used this: CFLAGS ?= -O -pipe -mcpu=i686 -march=i686 COPTFLAGS ?= -O -pipe -mcpu=i686 -march=i686 All the diffs in sys/ on the box I built this kernel on are at http://www.FreeBSD.org/~jhb/patches/sys-mutex.patch. One possibly notable patch I had forgotten about is the fix to forward_signal() to lock with sched_lock. I've just committed that though. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 0:43: 1 2001 Delivered-To: freebsd-arch@freebsd.org Received: from freebsd.dk (freebsd.dk [212.242.42.178]) by hub.freebsd.org (Postfix) with ESMTP id 489AF37B400; Thu, 18 Jan 2001 00:42:39 -0800 (PST) Received: (from sos@localhost) by freebsd.dk (8.9.3/8.9.1) id JAA12751; Thu, 18 Jan 2001 09:42:37 +0100 (CET) (envelope-from sos) From: Soren Schmidt Message-Id: <200101180842.JAA12751@freebsd.dk> Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: from John Baldwin at "Jan 18, 2001 00:33:30 am" To: jhb@FreeBSD.org (John Baldwin) Date: Thu, 18 Jan 2001 09:42:37 +0100 (CET) Cc: bright@wintelcom.net (Alfred Perlstein), current@FreeBSD.org, arch@FreeBSD.org, rjesup@wgate.com X-Mailer: ELM [version 2.4ME+ PL54 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG It seems John Baldwin wrote: > >> > >> So the only thing I can think of is that you guys have something in > >> your src trees that cvs & I dont... > >> > >> Now what ? > > > > What are the compile flags you are using? > > I actually used this: > > CFLAGS ?= -O -pipe -mcpu=i686 -march=i686 > COPTFLAGS ?= -O -pipe -mcpu=i686 -march=i686 > > All the diffs in sys/ on the box I built this kernel on are at > http://www.FreeBSD.org/~jhb/patches/sys-mutex.patch. One possibly notable > patch I had forgotten about is the fix to forward_signal() to lock with > sched_lock. I've just committed that though. OK, I have installed those patches and will try out a kernel build on that asap.... -Søren To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 1:11:19 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mass.osd.bsdi.com (unknown [216.240.45.41]) by hub.freebsd.org (Postfix) with ESMTP id 3A6C937B402 for ; Thu, 18 Jan 2001 01:11:01 -0800 (PST) Received: from mass.osd.bsdi.com (localhost [127.0.0.1]) by mass.osd.bsdi.com (8.11.1/8.11.1) with ESMTP id f0I9PGj01700; Thu, 18 Jan 2001 01:25:16 -0800 (PST) (envelope-from msmith@mass.osd.bsdi.com) Message-Id: <200101180925.f0I9PGj01700@mass.osd.bsdi.com> X-Mailer: exmh version 2.1.1 10/15/1999 To: Tony Finch Cc: arch@FreeBSD.ORG Subject: Re: dynamic vs static sysctls? In-reply-to: Your message of "Thu, 18 Jan 2001 06:26:44 GMT." <20010118062644.D30538@hand.dotat.at> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 18 Jan 2001 01:25:16 -0800 From: Mike Smith Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Mike Smith wrote: > > > >> In my work on a background version of fsck, I have used sysctl to > >> allow me to pass information into the kernel that I want to have > >> updated in the filesystem. > > > >I'm not convinced that sysctl is the "right" way to go about doing this, > >really. But I can't think of a better one. 8) > > Why not an ioctl on the disk device? You could arrange to pass in an > array of free blocks to reduce the number of syscalls. Because there's no linkage between the disk device and the filesystem. An ioctl on the mountpoint might make (a little) more sense. -- ... every activity meets with opposition, everyone who acts has his rivals and unfortunately opponents also. But not because people want to be opponents, rather because the tasks and relationships force people to take different points of view. [Dr. Fritz Todt] V I C T O R Y N O T V E N G E A N C E To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 1:57:31 2001 Delivered-To: freebsd-arch@freebsd.org Received: from freebsd.dk (freebsd.dk [212.242.42.178]) by hub.freebsd.org (Postfix) with ESMTP id 5FD4437B69E; Thu, 18 Jan 2001 01:57:10 -0800 (PST) Received: (from sos@localhost) by freebsd.dk (8.9.3/8.9.1) id KAA33489; Thu, 18 Jan 2001 10:57:08 +0100 (CET) (envelope-from sos) From: Soren Schmidt Message-Id: <200101180957.KAA33489@freebsd.dk> Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: <200101180842.JAA12751@freebsd.dk> from Soren Schmidt at "Jan 18, 2001 09:42:37 am" To: sos@freebsd.dk (Soren Schmidt) Date: Thu, 18 Jan 2001 10:57:08 +0100 (CET) Cc: jhb@FreeBSD.ORG (John Baldwin), bright@wintelcom.net (Alfred Perlstein), current@FreeBSD.ORG, arch@FreeBSD.ORG, rjesup@wgate.com X-Mailer: ELM [version 2.4ME+ PL54 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG It seems Soren Schmidt wrote: > > I actually used this: > > > > CFLAGS ?= -O -pipe -mcpu=i686 -march=i686 > > COPTFLAGS ?= -O -pipe -mcpu=i686 -march=i686 > > > > All the diffs in sys/ on the box I built this kernel on are at > > http://www.FreeBSD.org/~jhb/patches/sys-mutex.patch. One possibly notable > > patch I had forgotten about is the fix to forward_signal() to lock with > > sched_lock. I've just committed that though. > > OK, I have installed those patches and will try out a kernel build on > that asap.... Hmm, now we are getting somewhere, it ran through a make world with that patch installed! I'll test some more, maybe you should get it all committed it seems to make a hell of a difference.... -Søren To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 4: 6:22 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mobile.wemm.org (c1315225-a.plstn1.sfba.home.com [65.0.135.147]) by hub.freebsd.org (Postfix) with ESMTP id BD8FB37B401; Thu, 18 Jan 2001 04:05:58 -0800 (PST) Received: from netplex.com.au (localhost [127.0.0.1]) by mobile.wemm.org (8.11.1/8.11.1) with ESMTP id f0IC5rK13062; Thu, 18 Jan 2001 04:05:54 -0800 (PST) (envelope-from peter@netplex.com.au) Message-Id: <200101181205.f0IC5rK13062@mobile.wemm.org> X-Mailer: exmh version 2.2 06/23/2000 with nmh-1.0.4 To: Soren Schmidt Cc: jhb@FreeBSD.ORG (John Baldwin), bright@wintelcom.net (Alfred Perlstein), current@FreeBSD.ORG, arch@FreeBSD.ORG, rjesup@wgate.com Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: <200101180957.KAA33489@freebsd.dk> Content-Transfer-Encoding: 8bit Date: Thu, 18 Jan 2001 04:05:53 -0800 From: Peter Wemm Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Soren Schmidt wrote: > It seems Soren Schmidt wrote: > > > I actually used this: > > > > > > CFLAGS ?= -O -pipe -mcpu=i686 -march=i686 > > > COPTFLAGS ?= -O -pipe -mcpu=i686 -march=i686 > > > > > > All the diffs in sys/ on the box I built this kernel on are at > > > http://www.FreeBSD.org/~jhb/patches/sys-mutex.patch. One possibly notabl e > > > patch I had forgotten about is the fix to forward_signal() to lock with > > > sched_lock. I've just committed that though. > > > > OK, I have installed those patches and will try out a kernel build on > > that asap.... > > Hmm, now we are getting somewhere, it ran through a make world with > that patch installed! > > I'll test some more, maybe you should get it all committed it seems to > make a hell of a difference.... Hmm. with the mp_machdep.c fix committed, that leaves the only other significant difference being the re-enable of HLT when a cpu goes idle in i386/i386/machdep.c. The refcount.[ch] stuff is not relevant to this problem. The kern/subr_prf.c change doesn't *appear* to be a likely candidate, unless you are printing lots of console messages during the buildworld.. The kern/vfs_aio.c are not relevant as VFS_AIO is not in GENERIC. The rest are comments, mtx_assert()'s or DDB activation related. Soren, can you retest a buildworld with the currently committed kernel with no other changes? Let us see if the forward_signal() stuff is the culprit, and if not, try adding just the i386/i386/machdep.c patch to HLT the idle CPU. (if *that* makes a difference then we have got trouble!) Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 5: 9:52 2001 Delivered-To: freebsd-arch@freebsd.org Received: from freebsd.dk (freebsd.dk [212.242.42.178]) by hub.freebsd.org (Postfix) with ESMTP id 948E537B400; Thu, 18 Jan 2001 05:09:32 -0800 (PST) Received: (from sos@localhost) by freebsd.dk (8.9.3/8.9.1) id OAA84777; Thu, 18 Jan 2001 14:09:21 +0100 (CET) (envelope-from sos) From: Soren Schmidt Message-Id: <200101181309.OAA84777@freebsd.dk> Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: <200101181205.f0IC5rK13062@mobile.wemm.org> from Peter Wemm at "Jan 18, 2001 04:05:53 am" To: peter@netplex.com.au (Peter Wemm) Date: Thu, 18 Jan 2001 14:09:21 +0100 (CET) Cc: jhb@FreeBSD.ORG (John Baldwin), bright@wintelcom.net (Alfred Perlstein), current@FreeBSD.ORG, arch@FreeBSD.ORG, rjesup@wgate.com X-Mailer: ELM [version 2.4ME+ PL54 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG It seems Peter Wemm wrote: > > Hmm. with the mp_machdep.c fix committed, that leaves the only other > significant difference being the re-enable of HLT when a cpu goes idle > in i386/i386/machdep.c. That still lockups, tried a freshly checked out sys... > The refcount.[ch] stuff is not relevant to this problem. > > The kern/subr_prf.c change doesn't *appear* to be a likely candidate, > unless you are printing lots of console messages during the buildworld.. > > The kern/vfs_aio.c are not relevant as VFS_AIO is not in GENERIC. > > The rest are comments, mtx_assert()'s or DDB activation related. > > Soren, can you retest a buildworld with the currently committed kernel > with no other changes? Let us see if the forward_signal() stuff is the > culprit, and if not, try adding just the i386/i386/machdep.c patch to HLT > the idle CPU. (if *that* makes a difference then we have got trouble!) I'll try adding the forward_signal stuff see if that helps... -Søren To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 5:39:57 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mobile.wemm.org (c1315225-a.plstn1.sfba.home.com [65.0.135.147]) by hub.freebsd.org (Postfix) with ESMTP id 50FC937B401; Thu, 18 Jan 2001 05:39:34 -0800 (PST) Received: from netplex.com.au (localhost [127.0.0.1]) by mobile.wemm.org (8.11.1/8.11.1) with ESMTP id f0IDdUK14717; Thu, 18 Jan 2001 05:39:30 -0800 (PST) (envelope-from peter@netplex.com.au) Message-Id: <200101181339.f0IDdUK14717@mobile.wemm.org> X-Mailer: exmh version 2.2 06/23/2000 with nmh-1.0.4 To: Soren Schmidt Cc: jhb@FreeBSD.ORG (John Baldwin), bright@wintelcom.net (Alfred Perlstein), current@FreeBSD.ORG, arch@FreeBSD.ORG, rjesup@wgate.com Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: <200101181309.OAA84777@freebsd.dk> Content-Transfer-Encoding: 8bit Date: Thu, 18 Jan 2001 05:39:30 -0800 From: Peter Wemm Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Soren Schmidt wrote: > It seems Peter Wemm wrote: > > > > Hmm. with the mp_machdep.c fix committed, that leaves the only other > > significant difference being the re-enable of HLT when a cpu goes idle > > in i386/i386/machdep.c. > > That still lockups, tried a freshly checked out sys... > > > The refcount.[ch] stuff is not relevant to this problem. > > > > The kern/subr_prf.c change doesn't *appear* to be a likely candidate, > > unless you are printing lots of console messages during the buildworld.. > > > > The kern/vfs_aio.c are not relevant as VFS_AIO is not in GENERIC. > > > > The rest are comments, mtx_assert()'s or DDB activation related. > > > > Soren, can you retest a buildworld with the currently committed kernel > > with no other changes? Let us see if the forward_signal() stuff is the > > culprit, and if not, try adding just the i386/i386/machdep.c patch to HLT > > the idle CPU. (if *that* makes a difference then we have got trouble!) > > I'll try adding the forward_signal stuff see if that helps... But John committed that! it should be in the fresh checkout you tried above.... Of course, that is assuming you cvsup'ed very recently.. > -Søren > Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 6: 9: 9 2001 Delivered-To: freebsd-arch@freebsd.org Received: from freebsd.dk (freebsd.dk [212.242.42.178]) by hub.freebsd.org (Postfix) with ESMTP id 9210F37B401; Thu, 18 Jan 2001 06:08:49 -0800 (PST) Received: (from sos@localhost) by freebsd.dk (8.9.3/8.9.1) id PAA00170; Thu, 18 Jan 2001 15:08:42 +0100 (CET) (envelope-from sos) From: Soren Schmidt Message-Id: <200101181408.PAA00170@freebsd.dk> Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: <200101181339.f0IDdUK14717@mobile.wemm.org> from Peter Wemm at "Jan 18, 2001 05:39:30 am" To: peter@netplex.com.au (Peter Wemm) Date: Thu, 18 Jan 2001 15:08:42 +0100 (CET) Cc: jhb@FreeBSD.ORG (John Baldwin), bright@wintelcom.net (Alfred Perlstein), current@FreeBSD.ORG, arch@FreeBSD.ORG, rjesup@wgate.com X-Mailer: ELM [version 2.4ME+ PL54 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG It seems Peter Wemm wrote: > > I'll try adding the forward_signal stuff see if that helps... > > But John committed that! it should be in the fresh checkout you tried > above.... Of course, that is assuming you cvsup'ed very recently.. Sorry that was not what I meant, I meant this patch to machdep.c and so far it looks as thats the culprit it hasn't locked up yet at least, but its not finished a world yet.... Index: machdep.c =================================================================== RCS file: /home/ncvs/src/sys/i386/i386/machdep.c,v retrieving revision 1.429 diff -u -r1.429 machdep.c --- machdep.c 2001/01/16 09:10:33 1.429 +++ machdep.c 2001/01/18 13:44:48 @@ -1005,6 +1005,10 @@ void cpu_idle(void) { +#ifdef SMP + /* first candidate for an interrupt */ + lapic.tpr = 0; +#endif #ifndef SMP if (cpu_idle_hlt) { disable_intr(); -Søren To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 7:26:20 2001 Delivered-To: freebsd-arch@freebsd.org Received: from freebsd.dk (freebsd.dk [212.242.42.178]) by hub.freebsd.org (Postfix) with ESMTP id E62AF37B402; Thu, 18 Jan 2001 07:26:00 -0800 (PST) Received: (from sos@localhost) by freebsd.dk (8.9.3/8.9.1) id QAA21260; Thu, 18 Jan 2001 16:25:52 +0100 (CET) (envelope-from sos) From: Soren Schmidt Message-Id: <200101181525.QAA21260@freebsd.dk> Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: <200101181205.f0IC5rK13062@mobile.wemm.org> from Peter Wemm at "Jan 18, 2001 04:05:53 am" To: peter@netplex.com.au (Peter Wemm) Date: Thu, 18 Jan 2001 16:25:52 +0100 (CET) Cc: jhb@FreeBSD.ORG (John Baldwin), bright@wintelcom.net (Alfred Perlstein), current@FreeBSD.ORG, arch@FreeBSD.ORG, rjesup@wgate.com X-Mailer: ELM [version 2.4ME+ PL54 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG It seems Peter Wemm wrote: > > Soren, can you retest a buildworld with the currently committed kernel > with no other changes? Let us see if the forward_signal() stuff is the > culprit, and if not, try adding just the i386/i386/machdep.c patch to HLT > the idle CPU. (if *that* makes a difference then we have got trouble!) It seems that the HLT thing is exactly the patch that makes it work! So we have trouble, I said that all along .5 :) -Søren To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 9:58:53 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail.interware.hu (mail.interware.hu [195.70.32.130]) by hub.freebsd.org (Postfix) with ESMTP id 7D08137B699; Thu, 18 Jan 2001 09:58:30 -0800 (PST) Received: from timbuktu-50.budapest.interware.hu ([195.70.51.242] helo=elischer.org) by mail.interware.hu with esmtp (Exim 3.16 #1 (Debian)) id 14JJKB-0004q7-00; Thu, 18 Jan 2001 18:58:15 +0100 Message-ID: <3A671284.A8DA14C1@elischer.org> Date: Thu, 18 Jan 2001 07:57:56 -0800 From: Julian Elischer X-Mailer: Mozilla 4.7 [en] (X11; U; FreeBSD 5.0-CURRENT i386) X-Accept-Language: en, hu MIME-Version: 1.0 To: John Baldwin Cc: Bosko Milekic , jasone@FreeBSD.org, arch@FreeBSD.org Subject: Re: mtx_init: MTX_RECURSE and potential SMP troublespots References: Content-Type: text/plain; charset=iso-8859-15 Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG John Baldwin wrote: > > > Also, MTX_NORECURSE is only useful on spin mutexes. If you compile a kernel > with MUTEX_DEBUG and ichsmb, it panics during boot. All the MTX_NORECURSE > flags just need to die here. yeah, recursing should always be illegal. :-) -- __--_|\ Julian Elischer / \ julian@elischer.org ( OZ ) World tour 2000 ---> X_.---._/ from Perth, presently in: Budapest v To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 10:24:36 2001 Delivered-To: freebsd-arch@freebsd.org Received: from moby.geekhouse.net (moby.geekhouse.net [64.81.6.36]) by hub.freebsd.org (Postfix) with ESMTP id 063F737B400; Thu, 18 Jan 2001 10:24:19 -0800 (PST) Received: from laptop.baldwin.cx (john@dhcp150.geekhouse.net [192.168.1.150]) by moby.geekhouse.net (8.11.0/8.9.3) with ESMTP id f0IISKs68410; Thu, 18 Jan 2001 10:28:21 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <3A671284.A8DA14C1@elischer.org> Date: Thu, 18 Jan 2001 10:24:14 -0800 (PST) From: John Baldwin To: Julian Elischer Subject: Re: mtx_init: MTX_RECURSE and potential SMP troublespots Cc: arch@FreeBSD.org, jasone@FreeBSD.org, Bosko Milekic Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 18-Jan-01 Julian Elischer wrote: > John Baldwin wrote: >> > >> >> Also, MTX_NORECURSE is only useful on spin mutexes. If you compile a kernel >> with MUTEX_DEBUG and ichsmb, it panics during boot. All the MTX_NORECURSE >> flags just need to die here. > > yeah, recursing should always be illegal. >:-) It will be by default. MTX_RECURSE will be an opt-in flag to mtx_init() (if that makes any sense). -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 10:25:25 2001 Delivered-To: freebsd-arch@freebsd.org Received: from moby.geekhouse.net (moby.geekhouse.net [64.81.6.36]) by hub.freebsd.org (Postfix) with ESMTP id 4F04737B400; Thu, 18 Jan 2001 10:25:06 -0800 (PST) Received: from laptop.baldwin.cx (john@dhcp150.geekhouse.net [192.168.1.150]) by moby.geekhouse.net (8.11.0/8.9.3) with ESMTP id f0IISFs68402; Thu, 18 Jan 2001 10:28:15 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200101180957.KAA33489@freebsd.dk> Date: Thu, 18 Jan 2001 10:24:08 -0800 (PST) From: John Baldwin To: Soren Schmidt Subject: Re: HEADS-UP: await/asleep removal imminent Cc: rjesup@wgate.com, arch@FreeBSD.org, current@FreeBSD.org, (Alfred Perlstein) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 18-Jan-01 Soren Schmidt wrote: > It seems Soren Schmidt wrote: >> > I actually used this: >> > >> > CFLAGS ?= -O -pipe -mcpu=i686 -march=i686 >> > COPTFLAGS ?= -O -pipe -mcpu=i686 -march=i686 >> > >> > All the diffs in sys/ on the box I built this kernel on are at >> > http://www.FreeBSD.org/~jhb/patches/sys-mutex.patch. One possibly notable >> > patch I had forgotten about is the fix to forward_signal() to lock with >> > sched_lock. I've just committed that though. >> >> OK, I have installed those patches and will try out a kernel build on >> that asap.... > > Hmm, now we are getting somewhere, it ran through a make world with > that patch installed! > > I'll test some more, maybe you should get it all committed it seems to > make a hell of a difference.... The refcount.h stuff isn't used anywhere, and is still in a bikeshed mode. The rest of the patches are a bogus SMP HLT implementation that isnt' done yet, some proc lock fixes in svr4, some weird kernel printf patch I got from someone else at some point in time, and the forward_signal() patch I committed. > -Søren -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 10:25:58 2001 Delivered-To: freebsd-arch@freebsd.org Received: from moby.geekhouse.net (moby.geekhouse.net [64.81.6.36]) by hub.freebsd.org (Postfix) with ESMTP id 58FF137B401; Thu, 18 Jan 2001 10:25:35 -0800 (PST) Received: from laptop.baldwin.cx (john@dhcp150.geekhouse.net [192.168.1.150]) by moby.geekhouse.net (8.11.0/8.9.3) with ESMTP id f0IISFs68406; Thu, 18 Jan 2001 10:28:15 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200101181408.PAA00170@freebsd.dk> Date: Thu, 18 Jan 2001 10:24:09 -0800 (PST) From: John Baldwin To: Soren Schmidt Subject: Re: HEADS-UP: await/asleep removal imminent Cc: rjesup@wgate.com, arch@FreeBSD.org, current@FreeBSD.org, (Alfred Perlstein) , (Peter Wemm) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 18-Jan-01 Soren Schmidt wrote: > It seems Peter Wemm wrote: >> > I'll try adding the forward_signal stuff see if that helps... >> >> But John committed that! it should be in the fresh checkout you tried >> above.... Of course, that is assuming you cvsup'ed very recently.. > > Sorry that was not what I meant, I meant this patch to machdep.c > and so far it looks as thats the culprit it hasn't locked up yet > at least, but its not finished a world yet.... > > Index: machdep.c > =================================================================== > RCS file: /home/ncvs/src/sys/i386/i386/machdep.c,v > retrieving revision 1.429 > diff -u -r1.429 machdep.c > --- machdep.c 2001/01/16 09:10:33 1.429 > +++ machdep.c 2001/01/18 13:44:48 > @@ -1005,6 +1005,10 @@ > void > cpu_idle(void) > { > +#ifdef SMP > + /* first candidate for an interrupt */ > + lapic.tpr = 0; > +#endif > #ifndef SMP > if (cpu_idle_hlt) { > disable_intr(); Hmmmmmmmmmmmmmmmmmm. Since cpl is dead, this is supposed to be a nop, since the tpr is always at 0 now (AFAIK). I'll look at this today. Weird. Peter? > -Søren -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 10:31:36 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mail.interware.hu (mail.interware.hu [195.70.32.130]) by hub.freebsd.org (Postfix) with ESMTP id 6AAC237B401; Thu, 18 Jan 2001 10:31:15 -0800 (PST) Received: from marakesh-16.budapest.interware.hu ([195.70.50.144] helo=elischer.org) by mail.interware.hu with esmtp (Exim 3.16 #1 (Debian)) id 14JJq1-0008E5-00; Thu, 18 Jan 2001 19:31:09 +0100 Message-ID: <3A6732EF.C83945CA@elischer.org> Date: Thu, 18 Jan 2001 10:16:15 -0800 From: Julian Elischer X-Mailer: Mozilla 4.7 [en] (X11; U; FreeBSD 5.0-CURRENT i386) X-Accept-Language: en, hu MIME-Version: 1.0 To: Soren Schmidt Cc: Peter Wemm , John Baldwin , Alfred Perlstein , current@FreeBSD.ORG, arch@FreeBSD.ORG, rjesup@wgate.com Subject: Re: HEADS-UP: await/asleep removal imminent References: <200101181525.QAA21260@freebsd.dk> Content-Type: text/plain; charset=iso-8859-15 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Soren Schmidt wrote: > > It seems Peter Wemm wrote: > > > > Soren, can you retest a buildworld with the currently committed kernel > > with no other changes? Let us see if the forward_signal() stuff is the > > culprit, and if not, try adding just the i386/i386/machdep.c patch to HLT > > the idle CPU. (if *that* makes a difference then we have got trouble!) > > It seems that the HLT thing is exactly the patch that makes it work! > > So we have trouble, I said that all along .5 :) I don't know about your system but mine gets a LOT warmer when HALT is disabled. > > -Søren > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-current" in the body of the message -- __--_|\ Julian Elischer / \ julian@elischer.org ( OZ ) World tour 2000 ---> X_.---._/ from Perth, presently in: Budapest v To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 10:41:56 2001 Delivered-To: freebsd-arch@freebsd.org Received: from moby.geekhouse.net (moby.geekhouse.net [64.81.6.36]) by hub.freebsd.org (Postfix) with ESMTP id D145537B402; Thu, 18 Jan 2001 10:41:35 -0800 (PST) Received: from laptop.baldwin.cx (john@dhcp150.geekhouse.net [192.168.1.150]) by moby.geekhouse.net (8.11.0/8.9.3) with ESMTP id f0IIjZs68447; Thu, 18 Jan 2001 10:45:35 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <3A6732EF.C83945CA@elischer.org> Date: Thu, 18 Jan 2001 10:41:30 -0800 (PST) From: John Baldwin To: Julian Elischer Subject: Re: HEADS-UP: await/asleep removal imminent Cc: rjesup@wgate.com, arch@FreeBSD.org, current@FreeBSD.org, Alfred Perlstein , Peter Wemm , Soren Schmidt Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 18-Jan-01 Julian Elischer wrote: > Soren Schmidt wrote: >> >> It seems Peter Wemm wrote: >> > >> > Soren, can you retest a buildworld with the currently committed kernel >> > with no other changes? Let us see if the forward_signal() stuff is the >> > culprit, and if not, try adding just the i386/i386/machdep.c patch to HLT >> > the idle CPU. (if *that* makes a difference then we have got trouble!) >> >> It seems that the HLT thing is exactly the patch that makes it work! >> >> So we have trouble, I said that all along .5 :) > > I don't know about your system but mine gets a LOT warmer > when HALT is disabled. We don't IPI a hlt'd CPU in setrunqueue() right now, so once a CPU is HLT'd it stays idle until the I/O APIC sends it an interrupt to handle. I've watched top(1) during a buildworld with this enabled and have seen it with 8 processes in SRUN, but one CPU idle. :( Also, FWIW, the quad xeon doesn't use this patch and has run fine for several days now. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 16:13:32 2001 Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (placeholder-dcat-1076843399.broadbandoffice.net [64.47.83.135]) by hub.freebsd.org (Postfix) with ESMTP id C006637B401; Thu, 18 Jan 2001 16:13:14 -0800 (PST) Received: (from dillon@localhost) by earth.backplane.com (8.11.1/8.9.3) id f0J0DEb88324; Thu, 18 Jan 2001 16:13:14 -0800 (PST) (envelope-from dillon) Date: Thu, 18 Jan 2001 16:13:14 -0800 (PST) From: Matt Dillon Message-Id: <200101190013.f0J0DEb88324@earth.backplane.com> To: Mike Smith Cc: Tony Finch , arch@FreeBSD.ORG Subject: Re: dynamic vs static sysctls? References: <200101180925.f0I9PGj01700@mass.osd.bsdi.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :> Why not an ioctl on the disk device? You could arrange to pass in an :> array of free blocks to reduce the number of syscalls. : :Because there's no linkage between the disk device and the filesystem. :An ioctl on the mountpoint might make (a little) more sense. The sysctl scares me... it's a massively unportable idea. I'd much prefer running an ioctl through the filesystem. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 21:20: 5 2001 Delivered-To: freebsd-arch@freebsd.org Received: from homer.softweyr.com (bsdconspiracy.net [208.187.122.220]) by hub.freebsd.org (Postfix) with ESMTP id 2B6E437B404 for ; Thu, 18 Jan 2001 21:19:44 -0800 (PST) Received: from [127.0.0.1] (helo=softweyr.com ident=Fools trust ident!) by homer.softweyr.com with esmtp (Exim 3.16 #1) id 14JIyT-0000Ov-00; Thu, 18 Jan 2001 10:35:49 -0700 Message-ID: <3A672974.554E6981@softweyr.com> Date: Thu, 18 Jan 2001 10:35:48 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.75 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Peter Pentchev Cc: arch@FreeBSD.org Subject: Re: no newgroup/newgrp in FreeBSD? References: <20010116161703.H364@ringworld.oblivion.bg> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Peter Pentchev wrote: > > Heyas, > > Is there a reason there is no newgroup/newgrp(1) in FreeBSD, besides > the fact that the kernel can support a simultaneous set of groups? > Still, it would be nice for compatibility (and SUSv2 compliance) > to have some kind of newgrp/newgroup implementation, albeit even > a null one. Here ya go: # cd /usr/bin # ln true newgrp -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 21:20: 9 2001 Delivered-To: freebsd-arch@freebsd.org Received: from homer.softweyr.com (bsdconspiracy.net [208.187.122.220]) by hub.freebsd.org (Postfix) with ESMTP id B8E8137B400 for ; Thu, 18 Jan 2001 21:19:44 -0800 (PST) Received: from [127.0.0.1] (helo=softweyr.com ident=Fools trust ident!) by homer.softweyr.com with esmtp (Exim 3.16 #1) id 14JIwV-0000On-00; Thu, 18 Jan 2001 10:33:47 -0700 Message-ID: <3A6728FB.76E7C687@softweyr.com> Date: Thu, 18 Jan 2001 10:33:47 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.75 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Will Andrews Cc: Peter Pentchev , arch@FreeBSD.ORG Subject: Re: no newgroup/newgrp in FreeBSD? References: <20010116161703.H364@ringworld.oblivion.bg> <20010116162219.I364@ringworld.oblivion.bg> <20010116094133.B1858@puck.firepipe.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Will Andrews wrote: > > On Tue, Jan 16, 2001 at 04:22:19PM +0200, Peter Pentchev wrote: > > http://ringwraith.online.bg/~roam/devel/sysutils/newgrp-1.0.tar.gz > [..] > > Sorry for being ignorant, but what's the difference between this and the > pw(8) group operations? System V only holds one GID per process. The newgrp command changes the active GID to another group by starting a new shell with the GID specified on the newgrp command line. You have to be a member of the group or know the group password. Since BSD uses a list of group memberships, newgrp is not needed. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 23: 1:44 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mobile.wemm.org (c1315225-a.plstn1.sfba.home.com [65.0.135.147]) by hub.freebsd.org (Postfix) with ESMTP id D0CA237B401; Thu, 18 Jan 2001 23:01:24 -0800 (PST) Received: from netplex.com.au (localhost [127.0.0.1]) by mobile.wemm.org (8.11.1/8.11.1) with ESMTP id f0J71Bk09591; Thu, 18 Jan 2001 23:01:11 -0800 (PST) (envelope-from peter@netplex.com.au) Message-Id: <200101190701.f0J71Bk09591@mobile.wemm.org> X-Mailer: exmh version 2.2 06/23/2000 with nmh-1.0.4 To: Soren Schmidt Cc: jhb@FreeBSD.ORG (John Baldwin), bright@wintelcom.net (Alfred Perlstein), current@FreeBSD.ORG, arch@FreeBSD.ORG, rjesup@wgate.com Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: <200101181525.QAA21260@freebsd.dk> Content-Transfer-Encoding: 8bit Date: Thu, 18 Jan 2001 23:01:11 -0800 From: Peter Wemm Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Soren Schmidt wrote: > It seems Peter Wemm wrote: > > > > Soren, can you retest a buildworld with the currently committed kernel > > with no other changes? Let us see if the forward_signal() stuff is the > > culprit, and if not, try adding just the i386/i386/machdep.c patch to HLT > > the idle CPU. (if *that* makes a difference then we have got trouble!) > > It seems that the HLT thing is exactly the patch that makes it work! > > So we have trouble, I said that all along .5 :) So, the difference between -current working for you or not is this: +++ machdep.c 2001/01/18 13:44:48 @@ -1005,6 +1005,10 @@ void cpu_idle(void) { +#ifdef SMP + /* first candidate for an interrupt */ + lapic.tpr = 0; +#endif #ifndef SMP if (cpu_idle_hlt) { disable_intr(); ?? No other changes? This is bad news.. This means we have races somewhere, or some other badness. Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jan 18 23:35:52 2001 Delivered-To: freebsd-arch@freebsd.org Received: from freebsd.dk (freebsd.dk [212.242.42.178]) by hub.freebsd.org (Postfix) with ESMTP id B555937B401; Thu, 18 Jan 2001 23:35:32 -0800 (PST) Received: (from sos@localhost) by freebsd.dk (8.9.3/8.9.1) id IAA60706; Fri, 19 Jan 2001 08:35:15 +0100 (CET) (envelope-from sos) From: Soren Schmidt Message-Id: <200101190735.IAA60706@freebsd.dk> Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: <200101190701.f0J71Bk09591@mobile.wemm.org> from Peter Wemm at "Jan 18, 2001 11:01:11 pm" To: peter@netplex.com.au (Peter Wemm) Date: Fri, 19 Jan 2001 08:35:14 +0100 (CET) Cc: jhb@FreeBSD.ORG (John Baldwin), bright@wintelcom.net (Alfred Perlstein), current@FreeBSD.ORG, arch@FreeBSD.ORG, rjesup@wgate.com X-Mailer: ELM [version 2.4ME+ PL54 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG It seems Peter Wemm wrote: > Soren Schmidt wrote: > > It seems Peter Wemm wrote: > > > > > > Soren, can you retest a buildworld with the currently committed kernel > > > with no other changes? Let us see if the forward_signal() stuff is the > > > culprit, and if not, try adding just the i386/i386/machdep.c patch to HLT > > > the idle CPU. (if *that* makes a difference then we have got trouble!) > > > > It seems that the HLT thing is exactly the patch that makes it work! > > > > So we have trouble, I said that all along .5 :) > > So, the difference between -current working for you or not is this: [diff snipped] > ?? No other changes? I need the FULL change to machdep.c and apparently the chagne to subr_prf.c is needed too, at least it hasnt broke yet with that in place too, not that I can see why it matters though. The new diff to machdep.c that jhb made yesterday also causes trouble. > This is bad news.. This means we have races somewhere, or some other badness. That is what I've been harping about for months... What strikes me here as a very serious problem is that the SMPng developers has told me over and over that it works fine for them on -current, but when we get to the matter after months of frustration, it suddenly appears that they are not running the same -current as the rest of us. Now guys, excuse my language, but that *sucks* bigtime. However I'm still running tests here, and I'm offering to test every patch you can come up with to help locate the problem, just as always... -Søren To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 19 0:20:53 2001 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (fw2.aub.dk [195.24.1.195]) by hub.freebsd.org (Postfix) with ESMTP id 920CE37B401; Fri, 19 Jan 2001 00:20:35 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id f0J8KMl76034; Fri, 19 Jan 2001 09:20:22 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Matt Dillon Cc: Mike Smith , Tony Finch , arch@FreeBSD.ORG Subject: Re: dynamic vs static sysctls? In-Reply-To: Your message of "Thu, 18 Jan 2001 16:13:14 PST." <200101190013.f0J0DEb88324@earth.backplane.com> Date: Fri, 19 Jan 2001 09:20:22 +0100 Message-ID: <76032.979892422@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <200101190013.f0J0DEb88324@earth.backplane.com>, Matt Dillon writes: >:> Why not an ioctl on the disk device? You could arrange to pass in an >:> array of free blocks to reduce the number of syscalls. >: >:Because there's no linkage between the disk device and the filesystem. >:An ioctl on the mountpoint might make (a little) more sense. > > The sysctl scares me... it's a massively unportable idea. I'd much > prefer running an ioctl through the filesystem. An ioctl would end up on the device, which should know nothing about filesystems. If fcntl was extensible like ioctl, opening the rootdir of the filesystem and doing an fcntl would be the right way. Lacking that, a sysctl directly into the filesystem sounds like a pretty good solution to me. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 19 0:23:50 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mobile.wemm.org (c1315225-a.plstn1.sfba.home.com [65.0.135.147]) by hub.freebsd.org (Postfix) with ESMTP id 3AC7937B402; Fri, 19 Jan 2001 00:23:30 -0800 (PST) Received: from netplex.com.au (localhost [127.0.0.1]) by mobile.wemm.org (8.11.1/8.11.1) with ESMTP id f0J8N6k10282; Fri, 19 Jan 2001 00:23:06 -0800 (PST) (envelope-from peter@netplex.com.au) Message-Id: <200101190823.f0J8N6k10282@mobile.wemm.org> X-Mailer: exmh version 2.2 06/23/2000 with nmh-1.0.4 To: Soren Schmidt Cc: jhb@FreeBSD.ORG (John Baldwin), bright@wintelcom.net (Alfred Perlstein), current@FreeBSD.ORG, arch@FreeBSD.ORG, rjesup@wgate.com Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: <200101190735.IAA60706@freebsd.dk> Content-Transfer-Encoding: 8bit Date: Fri, 19 Jan 2001 00:23:05 -0800 From: Peter Wemm Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Soren Schmidt wrote: > It seems Peter Wemm wrote: > > This is bad news.. This means we have races somewhere, or some other badne ss. > > That is what I've been harping about for months... > > What strikes me here as a very serious problem is that the SMPng developers > has told me over and over that it works fine for them on -current, but > when we get to the matter after months of frustration, it suddenly appears > that they are not running the same -current as the rest of us. > > Now guys, excuse my language, but that *sucks* bigtime. I *am* runnning unmodified SMPng kernels and dont see this problem. And those that have changes certainly dont have John's patches in them. > -Søren Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 19 0:31:48 2001 Delivered-To: freebsd-arch@freebsd.org Received: from meow.osd.bsdi.com (meow.osd.bsdi.com [204.216.28.88]) by hub.freebsd.org (Postfix) with ESMTP id B324137B698; Fri, 19 Jan 2001 00:31:27 -0800 (PST) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by meow.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id f0J8RHL75590; Fri, 19 Jan 2001 00:27:17 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200101190735.IAA60706@freebsd.dk> Date: Fri, 19 Jan 2001 00:29:34 -0800 (PST) From: John Baldwin To: Soren Schmidt Subject: Re: HEADS-UP: await/asleep removal imminent Cc: rjesup@wgate.com, arch@FreeBSD.org, current@FreeBSD.org, (Alfred Perlstein) , (Peter Wemm) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 19-Jan-01 Soren Schmidt wrote: > It seems Peter Wemm wrote: >> Soren Schmidt wrote: >> > It seems Peter Wemm wrote: >> > > >> > > Soren, can you retest a buildworld with the currently committed kernel >> > > with no other changes? Let us see if the forward_signal() stuff is the >> > > culprit, and if not, try adding just the i386/i386/machdep.c patch to >> > > HLT >> > > the idle CPU. (if *that* makes a difference then we have got trouble!) >> > >> > It seems that the HLT thing is exactly the patch that makes it work! >> > >> > So we have trouble, I said that all along .5 :) >> >> So, the difference between -current working for you or not is this: > > [diff snipped] > >> ?? No other changes? > > I need the FULL change to machdep.c and apparently the chagne to subr_prf.c > is needed too, at least it hasnt broke yet with that in place too, not > that I can see why it matters though. > > The new diff to machdep.c that jhb made yesterday also causes trouble. > >> This is bad news.. This means we have races somewhere, or some other >> badness. That my other diff didn't work is more bad news. > That is what I've been harping about for months... > > What strikes me here as a very serious problem is that the SMPng developers > has told me over and over that it works fine for them on -current, but > when we get to the matter after months of frustration, it suddenly appears > that they are not running the same -current as the rest of us. This is _not_ true. My quad xeon test box runs a pure source tree, and has not had a single problem building many worlds and releases since the fix to atomic_store_rel_ptr(). Also, I rebuilt a fresh kernel from a freshly checked out tree without any patches today on mutex (the machine I built kernel.sos on) and that kernel has built worlds today without _any_ lockups. In fact, smp_hlt.patch is only something I've played with a bit for the last few weeks, and it doesn't really work anyways, as you can end up with CPU's staying in the HLT state for long periods of time even though there are runnable jobs, which is why I haven't bothered committing this. Can you post a dmesg of your box? -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 19 0:34:31 2001 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (fw2.aub.dk [195.24.1.195]) by hub.freebsd.org (Postfix) with ESMTP id 99B8937B404; Fri, 19 Jan 2001 00:34:12 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id f0J8YCl81453; Fri, 19 Jan 2001 09:34:12 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: John Baldwin Cc: Soren Schmidt , rjesup@wgate.com, arch@FreeBSD.ORG, current@FreeBSD.ORG, bright@wintelcom.net (Alfred Perlstein), peter@netplex.com.au (Peter Wemm) Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: Your message of "Fri, 19 Jan 2001 00:29:34 PST." Date: Fri, 19 Jan 2001 09:34:12 +0100 Message-ID: <81448.979893252@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message , John Baldwin writes: > >This is _not_ true. My quad xeon test box runs a pure source tree, and has not >had a single problem building many worlds and releases since the fix to >atomic_store_rel_ptr(). How many disks are active when you build world on that box ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 19 0:43:51 2001 Delivered-To: freebsd-arch@freebsd.org Received: from meow.osd.bsdi.com (meow.osd.bsdi.com [204.216.28.88]) by hub.freebsd.org (Postfix) with ESMTP id 4104537B402; Fri, 19 Jan 2001 00:43:30 -0800 (PST) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by meow.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id f0J8ctL75908; Fri, 19 Jan 2001 00:38:55 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <81448.979893252@critter> Date: Fri, 19 Jan 2001 00:41:12 -0800 (PST) From: John Baldwin To: Poul-Henning Kamp Subject: Re: HEADS-UP: await/asleep removal imminent Cc: (Peter Wemm) , (Alfred Perlstein) , current@FreeBSD.org, arch@FreeBSD.org, rjesup@wgate.com, Soren Schmidt Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 19-Jan-01 Poul-Henning Kamp wrote: > In message , John Baldwin writes: >> >>This is _not_ true. My quad xeon test box runs a pure source tree, and has >>not >>had a single problem building many worlds and releases since the fix to >>atomic_store_rel_ptr(). > > How many disks are active when you build world on that box ? Just one. I don't have any boxes actually with lots of disks in them. Details like this would be helpful though. I'll see if I can't find someone to rummage together a vinum stripe to build on or some such. However, after thinking some more, even in a preemptive kernel, Giant will protect against the *strategy() race you brought up, because we won't get a context switch in kernel mode that releases Giant until either we exit the kernel or we call tsleep(). The interrupt handler will block on Giant when its thread runs, so it won't run until the top half finishes and returns. Does that make sense? -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 19 0:46: 0 2001 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (fw2.aub.dk [195.24.1.195]) by hub.freebsd.org (Postfix) with ESMTP id AA72137B698; Fri, 19 Jan 2001 00:45:41 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id f0J8jfl82883; Fri, 19 Jan 2001 09:45:42 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: John Baldwin Cc: peter@netplex.com.au (Peter Wemm), bright@wintelcom.net (Alfred Perlstein), current@FreeBSD.org, arch@FreeBSD.org, rjesup@wgate.com, Soren Schmidt Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: Your message of "Fri, 19 Jan 2001 00:41:12 PST." Date: Fri, 19 Jan 2001 09:45:41 +0100 Message-ID: <82881.979893941@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message , John Baldwin writes: >> >> How many disks are active when you build world on that box ? > >Just one. [...] >Details like this would be helpful though. They have never been secret, and I voiced this issue early on. > However, after >thinking some more, even in a preemptive kernel, Giant will protect against the >*strategy() race you brought up, [...] Could be. Many disks are still a defining difference on sos's machine and practically everybody any other machine. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 19 0:52:13 2001 Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (placeholder-dcat-1076843399.broadbandoffice.net [64.47.83.135]) by hub.freebsd.org (Postfix) with ESMTP id EDE2B37B698; Fri, 19 Jan 2001 00:51:56 -0800 (PST) Received: (from dillon@localhost) by earth.backplane.com (8.11.1/8.9.3) id f0J8oO891717; Fri, 19 Jan 2001 00:50:24 -0800 (PST) (envelope-from dillon) Date: Fri, 19 Jan 2001 00:50:24 -0800 (PST) From: Matt Dillon Message-Id: <200101190850.f0J8oO891717@earth.backplane.com> To: Poul-Henning Kamp Cc: Mike Smith , Tony Finch , arch@FreeBSD.ORG Subject: Re: dynamic vs static sysctls? References: <76032.979892422@critter> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :An ioctl would end up on the device, which should know nothing about :filesystems. : :If fcntl was extensible like ioctl, opening the rootdir of the filesystem :and doing an fcntl would be the right way. : :Lacking that, a sysctl directly into the filesystem sounds like a :pretty good solution to me. : :-- :Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 :phk@FreeBSD.ORG | TCP/IP since RFC 956 fcntl would work... after all, the POSIX locking functions already do copyin/copyout using fcntl. It should give Kirk everything he needs. It would certainly be better then sysctl. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 19 1: 4: 9 2001 Delivered-To: freebsd-arch@freebsd.org Received: from freebsd.dk (freebsd.dk [212.242.42.178]) by hub.freebsd.org (Postfix) with ESMTP id 83AF137B699; Fri, 19 Jan 2001 01:03:49 -0800 (PST) Received: (from sos@localhost) by freebsd.dk (8.9.3/8.9.1) id KAA81774; Fri, 19 Jan 2001 10:03:47 +0100 (CET) (envelope-from sos) From: Soren Schmidt Message-Id: <200101190903.KAA81774@freebsd.dk> Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: <81448.979893252@critter> from Poul-Henning Kamp at "Jan 19, 2001 09:34:12 am" To: phk@critter.freebsd.dk (Poul-Henning Kamp) Date: Fri, 19 Jan 2001 10:03:47 +0100 (CET) Cc: jhb@FreeBSD.ORG (John Baldwin), rjesup@wgate.com, arch@FreeBSD.ORG, current@FreeBSD.ORG, bright@wintelcom.net (Alfred Perlstein), peter@netplex.com.au (Peter Wemm) X-Mailer: ELM [version 2.4ME+ PL54 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG It seems Poul-Henning Kamp wrote: > In message , John Baldwin writes: > > > >This is _not_ true. My quad xeon test box runs a pure source tree, and has not > >had a single problem building many worlds and releases since the fix to > >atomic_store_rel_ptr(). > > How many disks are active when you build world on that box ? And on how many controllers/interrupts ? -Søren To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 19 3:47:59 2001 Delivered-To: freebsd-arch@freebsd.org Received: from freebsd.dk (freebsd.dk [212.242.42.178]) by hub.freebsd.org (Postfix) with ESMTP id 1F02E37B400; Fri, 19 Jan 2001 03:47:35 -0800 (PST) Received: (from sos@localhost) by freebsd.dk (8.9.3/8.9.1) id MAA20267; Fri, 19 Jan 2001 12:47:32 +0100 (CET) (envelope-from sos) From: Soren Schmidt Message-Id: <200101191147.MAA20267@freebsd.dk> Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: from John Baldwin at "Jan 19, 2001 00:29:34 am" To: jhb@FreeBSD.org (John Baldwin) Date: Fri, 19 Jan 2001 12:47:32 +0100 (CET) Cc: rjesup@wgate.com, arch@FreeBSD.org, current@FreeBSD.org, bright@wintelcom.net, peter@netplex.com.au X-Mailer: ELM [version 2.4ME+ PL54 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG It seems John Baldwin wrote: > > Can you post a dmesg of your box? Sure, attached the dmesg from the system that is worst affected: Copyright (c) 1992-2001 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 5.0-CURRENT #6: Thu Jan 18 21:39:51 CET 2001 sos@test.freebsd.dk:/u1/src/sys/compile/TEST Timecounter "i8254" frequency 1193182 Hz CPU: Pentium III/Pentium III Xeon/Celeron (910.36-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x683 Stepping = 3 Features=0x383fbff real memory = 536850432 (524268K bytes) avail memory = 518602752 (506448K bytes) Changing APIC ID for IO APIC #1 from 3 to 1 in MP table Changing APIC ID for IO APIC #1 from 3 to 1 on chip Programming 16 pins in IOAPIC #0 IOAPIC #0 intpin 2 -> irq 0 Programming 16 pins in IOAPIC #1 IOAPIC #1 intpin 0 -> irq 5 IOAPIC #1 intpin 1 -> irq 6 IOAPIC #1 intpin 4 -> irq 2 IOAPIC #1 intpin 8 -> irq 11 IOAPIC #1 intpin 9 -> irq 12 IOAPIC #1 intpin 14 -> irq 10 FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 3, version: 0x00040011, at 0xfee00000 cpu1 (AP): apic id: 0, version: 0x00040011, at 0xfee00000 io0 (APIC): apic id: 2, version: 0x000f0011, at 0xfec00000 io1 (APIC): apic id: 1, version: 0x000f0011, at 0xfec01000 Preloaded elf kernel "kernel" at 0xc03f3000. Preloaded elf module "random.ko" at 0xc03f309c. Pentium Pro MTRR support enabled Using $PIR table, 11 entries at 0xc00f0cb0 npx0: on motherboard npx0: INT 16 interface pcib0: at pcibus 0 on motherboard pci0: on pcib0 fxp0: port 0xd800-0xd83f mem 0xfd800000-0xfd8fffff,0xfe000000-0xfe000fff irq 2 at device 2.0 on pci0 fxp0: Ethernet address 00:e0:18:00:c8:15 atapci0: port 0xb000-0xb0ff,0xb400-0xb403,0xb800-0xb807,0xd000-0xd003,0xd400-0xd407 irq 5 at device 3.0 on pci0 ata2: at 0xd400 on atapci0 ata3: at 0xb800 on atapci0 atapci1: port 0x9400-0x94ff,0x9800-0x9803,0xa000-0xa007,0xa400-0xa403,0xa800-0xa807 irq 6 at device 4.0 on pci0 ata4: at 0xa800 on atapci1 ata5: at 0xa000 on atapci1 pci0: at 7.0 (no driver attached) isab0: at device 15.0 on pci0 isa0: on isab0 atapci2: port 0x8800-0x880f at device 15.1 on pci0 ata0: at 0x1f0 irq 14 on atapci2 ata1: at 0x170 irq 15 on atapci2 ohci0: mem 0xfb000000-0xfb000fff irq 10 at device 15.2 on pci0 usb0: OHCI version 1.0, legacy support usb0: on ohci0 usb0: USB revision 1.0 uhub0: (unknown) OHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 4 ports with 4 removable, self powered pcib1: at pcibus 1 on motherboard pci1: on pcib1 pci1: at 5.0 (no driver attached) pci1: at 5.1 (no driver attached) atkbdc0: at port 0x60,0x64 on isa0 atkbd0: flags 0x1 irq 1 on atkbdc0 ppc0: at port 0x378-0x37f irq 7 on isa0 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode plip0: on ppbus0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 sio0: type 16550A sio1 at port 0x2f8-0x2ff irq 3 on isa0 sio1: type 16550A vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 unknown: can't assign resources unknown: can't assign resources unknown: can't assign resources unknown: can't assign resources APIC_IO: Testing 8254 interrupt delivery APIC_IO: Broken MP table detected: 8254 is not connected to IOAPIC #0 intpin 2 APIC_IO: routing 8254 via 8259 and IOAPIC #0 intpin 0 ad0: 8063MB [16383/16/63] at ata0-master UDMA33 ar0: 146617MB [18691/255/63] subdisks: ad4: 73308MB [148945/16/63] at ata2-master UDMA100 ad6: 73308MB [148945/16/63] at ata3-master UDMA100 ar1: 146617MB [18691/255/63] subdisks: ad10: 73308MB [148945/16/63] at ata5-master UDMA100 ad8: 73308MB [148945/16/63] at ata4-master UDMA100 acd0: DVD-RAM at ata0-slave using PIO4 acd1: CD-RW at ata1-slave using PIO4 Mounting root from ufs:/dev/ad0a SMP: AP CPU #1 Launched! -Søren To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 19 6: 5:55 2001 Delivered-To: freebsd-arch@freebsd.org Received: from vega.dmnshq.net (vega.dmnshq.net [194.19.34.94]) by hub.freebsd.org (Postfix) with ESMTP id AAF8737B400; Fri, 19 Jan 2001 06:05:36 -0800 (PST) Received: (from eivind@localhost) by vega.dmnshq.net (8.11.1/8.9.3) id f0JE4wY25435; Fri, 19 Jan 2001 15:04:58 +0100 (CET) (envelope-from eivind) Date: Fri, 19 Jan 2001 15:04:57 +0100 From: Eivind Eklund To: Peter Jeremy Cc: Brian Somers , Sergey Babkin , freebsd-arch@FreeBSD.ORG Subject: Re: cvs commit: src/sys/gnu/i386/isa dgb.c dgm.c dgmreg.h dgreg.h Message-ID: <20010119150457.A18464@thinksec.com> References: <200101052146.f05LkDi49430@hak.lan.Awfulhak.org> <20010111064126.A10214@pc0640.alcatel.com.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010111064126.A10214@pc0640.alcatel.com.au>; from peter.jeremy@alcatel.com.au on Thu, Jan 11, 2001 at 06:41:26AM +1100 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Thu, Jan 11, 2001 at 06:41:26AM +1100, Peter Jeremy wrote: > On Fri, Jan 05, 2001 at 09:46:13PM +0000, Brian Somers wrote: > >I have the a driver that will replace dgb and dgm and support a bunch > >of other digiboards. Now all I need is the time to add back the > >{E,}ISA probe routines. > > There are come significant differences between the [E]ISA aand PCI > DigiBoards: > - The [E]ISA cards use I/O ports whilst the PCI boards use > memory-mapped registers. > - The PCI cards have a flat 4MB window. The [E]ISA cards access > on-board memory via a 8KB or 32KB window. > > This means that the code does a lot of checking to see if the card > is PCI or not, resulting in expressions like: > ((IS_PCI(board_type)) ? *mem[reg] : inb(base + reg)) > > The Linux driver (maintained by Digi) uses (the equivalent of) virtual > functions for all card accesses. > > Both these approaches add run-time overheads and can adversely affect > code legibility. The technique I've used for similar cases is to do the access through macros, putting the main source in a separate file, and having one "main source" that use the macros, and is included in two separate files that define separate macros. Ie, something like dgb_pci.c: #define GETDATA(mem, base, reg) (*mem[reg]) #include "dbg_main.c" dgb_eisa.c: #define GETDATA(mem, base, reg) inb(base + reg) #include "dgb_main.c" ... with dgb_main.c just using GETDATA() for accesses. (This is just based on the above problem description, not any knowledge of the DigiBoard. And GETDATA is probably not the right name.) Eivind. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 19 9:43:39 2001 Delivered-To: freebsd-arch@freebsd.org Received: from smtp02.primenet.com (smtp02.primenet.com [206.165.6.132]) by hub.freebsd.org (Postfix) with ESMTP id 23EEA37B402 for ; Fri, 19 Jan 2001 09:43:21 -0800 (PST) Received: (from daemon@localhost) by smtp02.primenet.com (8.9.3/8.9.3) id KAA08010; Fri, 19 Jan 2001 10:37:46 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp02.primenet.com, id smtpdAAAlcaiIp; Fri Jan 19 10:37:36 2001 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id KAA10770; Fri, 19 Jan 2001 10:43:06 -0700 (MST) From: Terry Lambert Message-Id: <200101191743.KAA10770@usr08.primenet.com> Subject: Re: no newgroup/newgrp in FreeBSD? To: wes@softweyr.com (Wes Peters) Date: Fri, 19 Jan 2001 17:42:55 +0000 (GMT) Cc: will@physics.purdue.edu (Will Andrews), roam@orbitel.bg (Peter Pentchev), arch@FreeBSD.ORG In-Reply-To: <3A6728FB.76E7C687@softweyr.com> from "Wes Peters" at Jan 18, 2001 10:33:47 AM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > Sorry for being ignorant, but what's the difference between this and the > > pw(8) group operations? > > System V only holds one GID per process. The newgrp command changes the > active GID to another group by starting a new shell with the GID specified > on the newgrp command line. You have to be a member of the group or know > the group password. Since BSD uses a list of group memberships, newgrp > is not needed. Actually, it could be useful. In SVR4, the most powerful effect is that newgrp permits you to obtain membership in a group without explicit configuration, so long as there is communication of the password to you. Specifically, it allows adjunct group membership, through knowledge of a password. The BSD method grants membership in groups explicitly. As an example, consider membership in group "wheel". To be able to "su", you must have two things: membership in group "wheel", and the "root" password. With newgrp, assuming that that you were to permit it, by assigning a group password to "wheel", instead of disallowing a "wheel" password (which would give you the explicit group membership, equivalent to what is there today), a person would need to know the "wheel" password and the "root" password instead (newgrp to "wheel", then su to "root"). An alternate formulation of this would be to have multiple group membership be ``provisional''; that is, you are a member of the explicit group associated with your passwd file entry, and a provisional member of the groups lin the group file which list you as a member. For those groups in the group file for which there is a disallowed password, membership would be granted immediately, and no newgrp-based attempt would be permitted (that's what we have today, effectively). For those groups with an explicit password, group membership would be open to all, but session transient, and require the use of a password. You could go further, and formulate it as a negative, where in order to relinquish membership, you have to supply a group password; this would be useful for exclusion groups. In any case, it's useful to think of newgrp on BSD, if only to set the "default" group to one of the groups of which a user is a member, other than their primary group (as set in the user's passwd file entry). This would have real effect in BSD, limited, of course, by the SGID bit on the directory in which a create was taking place. Again, one could consider a password: this would let you limit some operations by group membership or non-membership (as today), and add the ability to limit others based on whether or not primary membership is a particular group. Obviously, any of this could be used with password protection and group ownership and permissions on directories and files to effectively password-protect filesystem objects. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 19 9:48: 2 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 39B8637B402; Fri, 19 Jan 2001 09:47:42 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id EAA32665; Sat, 20 Jan 2001 04:47:35 +1100 Date: Sat, 20 Jan 2001 04:48:03 +1100 (EST) From: Bruce Evans X-Sender: bde@besplex.bde.org To: John Baldwin Cc: Poul-Henning Kamp , peter@netplex.com.au, bright@wintelcom.net, current@FreeBSD.ORG, arch@FreeBSD.ORG, rjesup@wgate.com, Soren Schmidt Subject: Re: HEADS-UP: await/asleep removal imminent In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, 19 Jan 2001, John Baldwin wrote: > rummage together a vinum stripe to build on or some such. However, after > thinking some more, even in a preemptive kernel, Giant will protect against the > *strategy() race you brought up, because we won't get a context switch in > kernel mode that releases Giant until either we exit the kernel or we call > tsleep(). The interrupt handler will block on Giant when its thread runs, so > it won't run until the top half finishes and returns. Does that make sense? This would be a bug in the preemptive kernel. To work as well as old kernels, everything that was not run because of a lock must be run when the lock is released. This corresponds to splx() running all software interrupt handlers that became runnable. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 19 10:53:15 2001 Delivered-To: freebsd-arch@freebsd.org Received: from meow.osd.bsdi.com (meow.osd.bsdi.com [204.216.28.88]) by hub.freebsd.org (Postfix) with ESMTP id 92B1F37B402; Fri, 19 Jan 2001 10:52:56 -0800 (PST) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by meow.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id f0JIn4L85589; Fri, 19 Jan 2001 10:49:05 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Fri, 19 Jan 2001 10:51:27 -0800 (PST) From: John Baldwin To: Bruce Evans Subject: Re: HEADS-UP: await/asleep removal imminent Cc: Soren Schmidt , rjesup@wgate.com, arch@FreeBSD.org, current@FreeBSD.org, bright@wintelcom.net, peter@netplex.com.au, Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 19-Jan-01 Bruce Evans wrote: > On Fri, 19 Jan 2001, John Baldwin wrote: > >> rummage together a vinum stripe to build on or some such. However, after >> thinking some more, even in a preemptive kernel, Giant will protect against >> the >> *strategy() race you brought up, because we won't get a context switch in >> kernel mode that releases Giant until either we exit the kernel or we call >> tsleep(). The interrupt handler will block on Giant when its thread runs, >> so >> it won't run until the top half finishes and returns. Does that make sense? > > This would be a bug in the preemptive kernel. To work as well as old > kernels, everything that was not run because of a lock must be run when > the lock is released. This corresponds to splx() running all software > interrupt handlers that became runnable. I don't think we need to emulate every possible race condition of the old kernel. :) Eventually, what you ask _will_ be done because when we release a mutex, we make a process blocked on it runnable again, and if the new process has higher prioriry than the currnet priority. When you combine this with priority propagation, it will actually all work the way you want. It is just a more generic way of doing it since now any mtx_exit() of a sleep lock can effect this change, whereas before only splx() could effect this change. > Bruce -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 19 12: 8:34 2001 Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id C225837B400 for ; Fri, 19 Jan 2001 12:08:10 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id f0JK80l98604; Fri, 19 Jan 2001 21:08:05 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Kirk McKusick Cc: Alfred Perlstein , arch@FreeBSD.ORG Subject: Re: dynamic vs static sysctls? In-Reply-To: Your message of "Tue, 16 Jan 2001 15:06:18 PST." <200101162306.PAA23554@beastie.mckusick.com> Date: Fri, 19 Jan 2001 21:08:00 +0100 Message-ID: <98602.979934880@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <200101162306.PAA23554@beastie.mckusick.com>, Kirk McKusick writes: > > >I want > >to do one call to sysctlnametomib (a new, but obviously trivial > >function) to return the numeric mib, and then use that mib on all > >the subsequent sysctl calls. > > As new-sysctl "designer" I think this is a good idea. It's a trivial > thing to implement, so are there any warm and able bodies who want > to try their hands at it and send the patch to me for review ? > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > >Here is my proposal (along with a test program for it). Looks good, commit it :-) Poul-Henning > > Kirk > >=-=-=-=-=-=-=-=-= > >#include >#include > >int sysctlnametomib __P((char *name, int *oidp, size_t *sizep)); > >/* > * This function uses a presently undocumented interface to the kernel > * to walk the tree and get the type so it can print the value. > * This interface is under work and consideration, and should probably > * be killed with a big axe by the first person who can find the time. > * (be aware though, that the proper interface isn't as obvious as it > * may seem, there are various conflicting requirements. > */ >int >sysctlnametomib(char *name, int *oidp, size_t *sizep) >{ > int oid[2]; > int error; > > oid[0] = 0; > oid[1] = 3; > > *sizep *= sizeof (int); > error = sysctl(oid, 2, oidp, sizep, name, strlen(name)); > *sizep /= sizeof (int); > if (error < 0) > return (error); > return (0); >} > >/* > * Test program for sysctlnametomib. > * > * Usage: sysctlnametomib name > */ >main(ac, av) > int ac; > char *av[]; >{ > int i, mib[CTL_MAXNAME]; > size_t size; > > if (ac < 2) { > printf("Usage: %s name\n", av[0]); > exit(1); > } > size = CTL_MAXNAME; > if (sysctlnametomib(av[1], mib, &size) < 0) { > perror("sysctlnametomib"); > exit(1); > } > for (i = 0; i < size; i++) > printf("mib[%d] = %d\n", i, mib[i]); > exit(0); >} > -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 19 12:57:40 2001 Delivered-To: freebsd-arch@freebsd.org Received: from salmon.maths.tcd.ie (salmon.maths.tcd.ie [134.226.81.11]) by hub.freebsd.org (Postfix) with SMTP id 3313637B400 for ; Fri, 19 Jan 2001 12:57:23 -0800 (PST) Received: from gosset.maths.tcd.ie by salmon.maths.tcd.ie with SMTP id ; 19 Jan 2001 20:57:22 +0000 (GMT) To: Matt Dillon Cc: Poul-Henning Kamp , arch@FreeBSD.ORG Subject: Re: dynamic vs static sysctls? In-Reply-To: Your message of "Fri, 19 Jan 2001 00:50:24 PST." <200101190850.f0J8oO891717@earth.backplane.com> Date: Fri, 19 Jan 2001 20:57:21 +0000 From: Ian Dowse Message-ID: <200101192057.aa79706@salmon.maths.tcd.ie> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <200101190850.f0J8oO891717@earth.backplane.com>, Matt Dillon writes: >:Lacking that, a sysctl directly into the filesystem sounds like a >:pretty good solution to me. > fcntl would work... after all, the POSIX locking functions already do > copyin/copyout using fcntl. It should give Kirk everything he needs. > It would certainly be better then sysctl. > There's also mount(2). NFS export lists are passed to the filesystem via a special mount call this way. It's hardly an interface designed to be extensible, but it does go straight to the filesystem from a syscall. Ian To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 19 13: 2:43 2001 Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (placeholder-dcat-1076843399.broadbandoffice.net [64.47.83.135]) by hub.freebsd.org (Postfix) with ESMTP id D403F37B402 for ; Fri, 19 Jan 2001 13:02:25 -0800 (PST) Received: (from dillon@localhost) by earth.backplane.com (8.11.1/8.9.3) id f0JL2F698129; Fri, 19 Jan 2001 13:02:15 -0800 (PST) (envelope-from dillon) Date: Fri, 19 Jan 2001 13:02:15 -0800 (PST) From: Matt Dillon Message-Id: <200101192102.f0JL2F698129@earth.backplane.com> To: Ian Dowse Cc: Poul-Henning Kamp , arch@FreeBSD.ORG Subject: Re: dynamic vs static sysctls? References: <200101192057.aa79706@salmon.maths.tcd.ie> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG : :There's also mount(2). NFS export lists are passed to the filesystem :via a special mount call this way. It's hardly an interface designed :to be extensible, but it does go straight to the filesystem from :a syscall. : :Ian Yup. Though mount is black magic. I'd avoid messing with its interface, the locking issues will drive you nuts. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 19 13:57:59 2001 Delivered-To: freebsd-arch@freebsd.org Received: from ringworld.nanolink.com (ringworld.nanolink.com [195.24.48.189]) by hub.freebsd.org (Postfix) with SMTP id 3C0BE37B401 for ; Fri, 19 Jan 2001 13:57:40 -0800 (PST) Received: (qmail 50059 invoked by uid 1000); 19 Jan 2001 21:56:16 -0000 Date: Fri, 19 Jan 2001 23:56:16 +0200 From: Peter Pentchev To: Terry Lambert Cc: Wes Peters , Will Andrews , arch@FreeBSD.ORG Subject: Re: no newgroup/newgrp in FreeBSD? Message-ID: <20010119235616.A49279@ringworld.oblivion.bg> Mail-Followup-To: Terry Lambert , Wes Peters , Will Andrews , arch@FreeBSD.ORG References: <3A6728FB.76E7C687@softweyr.com> <200101191743.KAA10770@usr08.primenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200101191743.KAA10770@usr08.primenet.com>; from tlambert@primenet.com on Fri, Jan 19, 2001 at 05:42:55PM +0000 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, Jan 19, 2001 at 05:42:55PM +0000, Terry Lambert wrote: > > > Sorry for being ignorant, but what's the difference between this and the > > > pw(8) group operations? > > > > System V only holds one GID per process. The newgrp command changes the > > active GID to another group by starting a new shell with the GID specified > > on the newgrp command line. You have to be a member of the group or know > > the group password. Since BSD uses a list of group memberships, newgrp > > is not needed. > > Actually, it could be useful. > > In SVR4, the most powerful effect is that newgrp permits you > to obtain membership in a group without explicit configuration, > so long as there is communication of the password to you. > > Specifically, it allows adjunct group membership, through > knowledge of a password. The BSD method grants membership > in groups explicitly. > > As an example, consider membership in group "wheel". To > be able to "su", you must have two things: membership in > group "wheel", and the "root" password. [snip] Hmm.. good points. The 1.0.1 version of my newgrp(1) implementation, at http://ringwraith.online.bg/~roam/devel/sysutils/newgrp-1.0.1.tar.gz allows the users to change to a group they are not members of, if that group is password-protected. It also restricts access to groups users *are* members of, if those are password-protected. Both those items are controlled by compile-time defines. G'luck, Peter -- I've heard that this sentence is a rumor. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 19 16:33:57 2001 Delivered-To: freebsd-arch@freebsd.org Received: from net2.gendyn.com (nat2.gendyn.com [204.60.171.12]) by hub.freebsd.org (Postfix) with ESMTP id 2C85C37B401; Fri, 19 Jan 2001 16:33:34 -0800 (PST) Received: from [153.11.109.12] (helo=fatboy.clc.gdeb.com) by net2.gendyn.com with esmtp (Exim 2.12 #1) id 14Jly3-000Fqc-00; Fri, 19 Jan 2001 19:33:19 -0500 Received: from vigrid.com (localhost [127.0.0.1]) by fatboy.clc.gdeb.com (8.11.0/8.9.3) with ESMTP id f0K0c0O92335; Fri, 19 Jan 2001 19:38:01 -0500 (EST) (envelope-from eischen@vigrid.com) Message-ID: <3A68DDE8.7F8D3C51@vigrid.com> Date: Fri, 19 Jan 2001 19:38:00 -0500 From: Daniel Eischen X-Mailer: Mozilla 4.76 [en] (X11; U; FreeBSD 4.1.1-STABLE i386) X-Accept-Language: en MIME-Version: 1.0 To: arch@freebsd.org Cc: current@freebsd.org Subject: Request For Review: libc/libc_r changes to allow -lc_r Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG [ Followups to -arch only please ] I've got some changes to libc and libc_r that I'd like reviewed. These changes eliminate the _THREAD_SAFE macro and allow a libc and libc_r that can be linked together via -lc_r. This also means that -pthread and -D_THREAD_SAFE can be deprecated. Note that libc_r could be now be renamed libpthread, but I would like to reserve that for KSE Project development where we might want to have both libraries around while libpthread (using KSEs) is being developed. The diffs are posted at: http://people.freebsd.org/~deischen/libc.diffs http://people.freebsd.org/~deischen/libc_r.diffs The libc diffs include src/include, src/lib/libc, as well as src/usr.sbin/pppctl (our only threaded app) changes, whereas the libc_r diffs are just those changes to src/lib/libc_r. I've tested these changes with ACE, but still have other testing that I'll be doing over the weekend. The patches have also survived a buildworld, but the installworld will have to wait for this weekend. If you try these patches make sure you save a copy of libc.so.5 just in case... And BTW, many thanks to John Polstra for explaining some of the issues involved with linking :-) Overview of the changes ----------------------- For those not familiar with our current libc_r, it is currently built to include a thread-safe libc as well as the POSIX threads routines. On the other hand, libc is built to be non-thread safe. This differs from other OSs and from what POSIX mandates and means that we require non-standard hacks like the linker option -pthread (which links to libc_r and prevents linking to libc). System calls that need to be wrapped (HIDDEN_SYSCALLS) are now defined in libc/Makefile instead of libc_r/Makefile. This means that libc now contains: _thread_sys_foo - actual syscall _foo - weak definition to _thread_sys_foo foo - weak definition to _thread_sys_foo I've changed all the instances of foo() to _foo() in libc for those hidden system calls. Anyone modifying or adding to libc will have to be careful to use the same conventions. In order to define the prototypes for _foo(), we introduce namespace.h and un-namespace.h. BDE suggested this as a way to define the prototypes for the hidden system calls. Use them as follows: #include "namespace.h" #include /* Other standard includes... */ #include "un-namespace.h" /* * DB contains the member 'close' which is defined to * _close if included before un-namespace.h */ #include /* Other local includes... */ int somelibcfunc(int fd, void *buf, size_t len) { ... ret = (int)_read(fd, buf, len); ... return (ret); } Including namespace.h will define those hidden system calls used within libc from foo to _foo; un-namespace.h will undefine them. A few others are defined as well, such as _pthread_mutex_lock and friends. This allows libc_r to provide replacement functions for them if it is also linked in. One note about lib/libc/stdio/mktemp.c. This file really wants to use namespace.h but can't because it is used in building part of the C compiler (or something like that; the error has long since scrolled off my window). There are basically only two changes to libc_r. One is to implement the pthread_foo() functions as _pthread_foo() and make pthread_foo a weak definition to _pthread_foo. This allows an application to have it's own pthread_foo(). I noticed Solaris does this to libpthread as well as to libc. The other change to libc_r is to eliminate references to the global _thread_run and replace them with a function call to get the currently running thread (which will be needed for the KSE project). Yes, I could do something like errno, but that wouldn't encourage getting the running thread once and caching it. Comments welcome, -- Dan Eischen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 19 20:51:59 2001 Delivered-To: freebsd-arch@freebsd.org Received: from VL-MS-MR001.sc1.videotron.ca (relais.videotron.ca [24.201.245.36]) by hub.freebsd.org (Postfix) with ESMTP id A55CA37B6A5; Fri, 19 Jan 2001 20:51:33 -0800 (PST) Received: from jehovah ([24.201.144.31]) by VL-MS-MR001.sc1.videotron.ca (Netscape Messaging Server 4.15) with SMTP id G7G2TX00.TU9; Fri, 19 Jan 2001 23:51:33 -0500 Message-ID: <000701c0829c$e0378d60$1f90c918@jehovah> From: "Bosko Milekic" To: Cc: Subject: wakeup_one() return value Date: Fri, 19 Jan 2001 23:53:05 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2919.6700 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Hi, Here's a diff: http://people.freebsd.org/~bmilekic/code/wakeupone.diff that makes wakeup_one() return an integer specifying whether a process was actually taken off the sleep queue, or not. The reason for the proposed change as to do with SMP and potential race conditions in certain situations. For example, wakeup_one() is sometimes (and quite effectively) used in this scenario: thread 1: --------- mtx_enter(lock_A, foo); do stuff; slp_cnt++; if (msleep(foobar, lock_A, ..., some_time_period) == EWOULDBLOCK) slp_cnt--; do more stuff; mtx_exit(lock_A, foo); thread 2: --------- mtx_enter(lock_A, foo); do stuff; if (slp_cnt >= 0) { wakeup_one(foobar); slp_cnt--; } mtx_exit(lock_A, foo); The problem is that under some conditions, there is a race and slp_cnt can get decremented twice. For example, consider slp_cnt == 0, and thread 1 comes in and increments it to 1. It goes to sleep but wakes up and grabs sched_lock (it hasn't yet regrabbed lock_A). At this time, thread 2 has lock_A and is spinning on sched_lock in wakeup_one(). But, thread 1 has timed out, and removes itself from the sleep queue; it returns EWOULDBLOCK, so slp_cnt goes down to 0. sched_lock is dropped, the thread blocks waiting for thread 2 to let go of lock_A so it can "do more stuff." Meanwhile, thread 2 gets sched_lock and does a wakeup_one() (effectively a no-op) and decrements slp_cnt again. There's an underflow. The patch will allow this, for thread 2: mtx_enter(lock_A, foo); do stuff; if (slp_cnt >= 0) { if (wakeup_one(foobar)) slp_cnt--; } mtx_exit(lock_A, foo); Please note that there should be no issue with wakeup(), because wakeup() will take away all processes matching `foobar' from the sleepqueue, so nobody implements a slp_cnt for uses of wakeup(). Any objections? If this goes in, we should probably do similar to the cond-var interface function that wakes up one sleeper. Re: diff; I cast (void)wakeup_one() to all the wakeup_one()s for now, which means we should go ahead and review them when/if this goes in. I'll take care of the sys/mbuf.h case, but I think that at least softupdates should be checked too (I saw some counters in there). If someone doesn't like the casts, let me know. Thanks, Bosko. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jan 19 21:25:21 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mobile.wemm.org (c1315225-a.plstn1.sfba.home.com [65.0.135.147]) by hub.freebsd.org (Postfix) with ESMTP id 63E6537B69B for ; Fri, 19 Jan 2001 21:25:04 -0800 (PST) Received: from netplex.com.au (localhost [127.0.0.1]) by mobile.wemm.org (8.11.1/8.11.1) with ESMTP id f0K5P4k31887 for ; Fri, 19 Jan 2001 21:25:04 -0800 (PST) (envelope-from peter@netplex.com.au) Message-Id: <200101200525.f0K5P4k31887@mobile.wemm.org> X-Mailer: exmh version 2.2 06/23/2000 with nmh-1.0.4 To: arch@FreeBSD.ORG Reply-To: arch@FreeBSD.ORG Subject: Re: Request For Review: libc/libc_r changes to allow -lc_r In-Reply-To: <3A68DDE8.7F8D3C51@vigrid.com> Date: Fri, 19 Jan 2001 21:25:04 -0800 From: Peter Wemm Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Daniel Eischen wrote: > [ Followups to -arch only please ] > > I've got some changes to libc and libc_r that I'd like reviewed. > These changes eliminate the _THREAD_SAFE macro and allow a libc > and libc_r that can be linked together via -lc_r. This also means > that -pthread and -D_THREAD_SAFE can be deprecated. Note that > libc_r could be now be renamed libpthread, but I would like to > reserve that for KSE Project development where we might want to > have both libraries around while libpthread (using KSEs) is being > developed. > > The diffs are posted at: > > http://people.freebsd.org/~deischen/libc.diffs > http://people.freebsd.org/~deischen/libc_r.diffs ... > Comments welcome, Wow! This is seriously overdue and desperately needed! Great stuff! Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 20 1:57:54 2001 Delivered-To: freebsd-arch@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 1283C37B400 for ; Sat, 20 Jan 2001 01:57:35 -0800 (PST) Received: (qmail 18166 invoked by uid 1142); 20 Jan 2001 09:57:28 -0000 Date: 20 Jan 2001 01:57:28 -0800 Date: Sat, 20 Jan 2001 01:57:20 -0800 From: Jason Evans To: Bosko Milekic Cc: freebsd-arch@freebsd.org Subject: Re: wakeup_one() return value Message-ID: <20010120015720.K69199@canonware.com> References: <000701c0829c$e0378d60$1f90c918@jehovah> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <000701c0829c$e0378d60$1f90c918@jehovah>; from bmilekic@technokratis.com on Fri, Jan 19, 2001 at 11:53:05PM -0500 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, Jan 19, 2001 at 11:53:05PM -0500, Bosko Milekic wrote: > Here's a diff: http://people.freebsd.org/~bmilekic/code/wakeupone.diff > that makes wakeup_one() return an integer specifying whether a process was > actually taken off the sleep queue, or not. The reason for the proposed > change as to do with SMP and potential race conditions in certain situations. > For example, wakeup_one() is sometimes (and quite effectively) used in this > scenario: > > thread 1: > --------- > > mtx_enter(lock_A, foo); > do stuff; > slp_cnt++; > if (msleep(foobar, lock_A, ..., some_time_period) == EWOULDBLOCK) > slp_cnt--; > do more stuff; > mtx_exit(lock_A, foo); > > > thread 2: > --------- > > mtx_enter(lock_A, foo); > do stuff; > if (slp_cnt >= 0) { > wakeup_one(foobar); > slp_cnt--; > } > mtx_exit(lock_A, foo); > > > The problem is that under some conditions, there is a race and slp_cnt can > get decremented twice. For example, consider slp_cnt == 0, and thread 1 comes > in and increments it to 1. It goes to sleep but wakes up and grabs sched_lock > (it hasn't yet regrabbed lock_A). At this time, thread 2 has lock_A and is > spinning on sched_lock in wakeup_one(). But, thread 1 has timed out, and > removes itself from the sleep queue; it returns EWOULDBLOCK, so slp_cnt goes ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > down to 0. sched_lock is dropped, the thread blocks waiting for thread 2 to ^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > let go of lock_A so it can "do more stuff." Meanwhile, thread 2 gets ^^^^^^^^^^^^^^^^ > sched_lock and does a wakeup_one() (effectively a no-op) and decrements > slp_cnt again. There's an underflow. I don't quite follow this paragraph, but I think that part of it rests on the assumption that slp_cnt is decremented before lock_A is reacquired, which is impossible. The ordering that you describe in the underscored text is not possible, since msleep() returns with lock_A held. It is possible for msleep() to return EWOULDBLOCK _after_ thread 2 has released lock_A though, so the race condition you describe is still possible. Your example can be "fixed" as follows: thread 1: --------- mtx_enter(lock_A, foo); do stuff; slp_cnt++; msleep(foobar, lock_A, ..., some_time_period); slp_cnt--; do more stuff; mtx_exit(lock_A, foo); thread 2: --------- mtx_enter(lock_A, foo); do stuff; if (slp_cnt >= 0) wakeup_one(foobar); mtx_exit(lock_A, foo); Now, slp_cnt will always be correct. However, if the example is actually more complex than what you present and there are multiple threads that can potentially call wakeup_one(), it is possible to call wakeup_one() more times than there are sleeping threads. If the example is even more complex, perhaps that means that too many threads were woken (i.e. it's a bad thing to have woken two threads instead of just one). This new problem can be "fixed" by introducing another variable that keeps track of the number of times wakeup_one() is called. I won't go there now though. =) > Any objections? If this goes in, we should probably do similar to the > cond-var interface function that wakes up one sleeper. I object to this change (even if you do come up with an example that would benefit from the change =) ). The example tries to use slp_cnt to pass information between two threads in an unsafe way. Perhaps you oversimplified the example; there is no reason I can see for even using slp_cnt. The following works fine: thread 1: --------- mtx_enter(lock_A, foo); do stuff; msleep(foobar, lock_A, ..., some_time_period); do more stuff; mtx_exit(lock_A, foo); thread 2: --------- mtx_enter(lock_A, foo); do stuff; wakeup_one(foobar); mtx_exit(lock_A, foo); If thread 1 isn't sleeping, the call to wakeup_one() has no effect, which is fine. Basically, all my rambling can be summarized thus: 1) The example code is broken (as you point out), but it may fail to highlight the point you were trying to make. 2) The methodology behind any similar example that does highlight a race condition would be demonstrating a flawed programming idiom, not a weakness of wakeup_one(). Thread synchronization rests on the foundation of mutexes and condition variables (msleep() is an equivalent). You are suggesting a fundamental change to one of those synchronization primitives. That should be a pretty strong indicator by itself that the change isn't needed. =) Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 20 10:28:34 2001 Delivered-To: freebsd-arch@freebsd.org Received: from VL-MS-MR002.sc1.videotron.ca (relais.videotron.ca [24.201.245.36]) by hub.freebsd.org (Postfix) with ESMTP id 50E3437B400 for ; Sat, 20 Jan 2001 10:28:13 -0800 (PST) Received: from jehovah ([24.201.144.31]) by VL-MS-MR002.sc1.videotron.ca (Netscape Messaging Server 4.15) with SMTP id G7H4MR02.59T; Sat, 20 Jan 2001 13:28:03 -0500 Message-ID: <007c01c0830e$f1c84a70$1f90c918@jehovah> From: "Bosko Milekic" To: "Jason Evans" Cc: References: <000701c0829c$e0378d60$1f90c918@jehovah> <20010120015720.K69199@canonware.com> Subject: Re: wakeup_one() return value Date: Sat, 20 Jan 2001 13:29:37 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.00.2919.6700 X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Jason Evans wrote: > On Fri, Jan 19, 2001 at 11:53:05PM -0500, Bosko Milekic wrote: > > Here's a diff: http://people.freebsd.org/~bmilekic/code/wakeupone.diff > > that makes wakeup_one() return an integer specifying whether a process was > > actually taken off the sleep queue, or not. The reason for the proposed > > change as to do with SMP and potential race conditions in certain situations. > > For example, wakeup_one() is sometimes (and quite effectively) used in this > > scenario: > > > > thread 1: > > --------- > > > > mtx_enter(lock_A, foo); > > do stuff; > > slp_cnt++; > > if (msleep(foobar, lock_A, ..., some_time_period) == EWOULDBLOCK) > > slp_cnt--; > > do more stuff; > > mtx_exit(lock_A, foo); > > > > > > thread 2: > > --------- > > > > mtx_enter(lock_A, foo); > > do stuff; > > if (slp_cnt >= 0) { > > wakeup_one(foobar); > > slp_cnt--; > > } > > mtx_exit(lock_A, foo); > > > > > > The problem is that under some conditions, there is a race and slp_cnt can > > get decremented twice. For example, consider slp_cnt == 0, and thread 1 comes > > in and increments it to 1. It goes to sleep but wakes up and grabs sched_lock > > (it hasn't yet regrabbed lock_A). At this time, thread 2 has lock_A and is > > spinning on sched_lock in wakeup_one(). But, thread 1 has timed out, and > > removes itself from the sleep queue; it returns EWOULDBLOCK, so slp_cnt goes > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > down to 0. sched_lock is dropped, the thread blocks waiting for thread 2 to > ^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > let go of lock_A so it can "do more stuff." Meanwhile, thread 2 gets > ^^^^^^^^^^^^^^^^ > > sched_lock and does a wakeup_one() (effectively a no-op) and decrements > > slp_cnt again. There's an underflow. > > I don't quite follow this paragraph, but I think that part of it rests on > the assumption that slp_cnt is decremented before lock_A is reacquired, > which is impossible. The ordering that you describe in the underscored > text is not possible, since msleep() returns with lock_A held. It is > possible for msleep() to return EWOULDBLOCK _after_ thread 2 has released > lock_A though, so the race condition you describe is still possible. Your That's exactly what I described. See "thread 1 blocks waiting for thread 2 to let go of lock_A so [that it can return EWOULDBLOCK finally] and do more stuff." Perhaps it was a tad misleading. > example can be "fixed" as follows: > > thread 1: > --------- > > mtx_enter(lock_A, foo); > do stuff; > slp_cnt++; > msleep(foobar, lock_A, ..., some_time_period); > slp_cnt--; > do more stuff; > mtx_exit(lock_A, foo); > > > thread 2: > --------- > > mtx_enter(lock_A, foo); > do stuff; > if (slp_cnt >= 0) > wakeup_one(foobar); > mtx_exit(lock_A, foo); > > Now, slp_cnt will always be correct. However, if the example is actually > more complex than what you present and there are multiple threads that can > potentially call wakeup_one(), it is possible to call wakeup_one() more > times than there are sleeping threads. If the example is even more > complex, perhaps that means that too many threads were woken (i.e. it's a > bad thing to have woken two threads instead of just one). > > This new problem can be "fixed" by introducing another variable that keeps > track of the number of times wakeup_one() is called. I won't go there now > though. =) > > > Any objections? If this goes in, we should probably do similar to the > > cond-var interface function that wakes up one sleeper. > > I object to this change (even if you do come up with an example that would > benefit from the change =) ). The example tries to use slp_cnt to pass > information between two threads in an unsafe way. Perhaps you > oversimplified the example; there is no reason I can see for even using > slp_cnt. The following works fine: > > thread 1: > --------- > > mtx_enter(lock_A, foo); > do stuff; > msleep(foobar, lock_A, ..., some_time_period); > do more stuff; > mtx_exit(lock_A, foo); > > > thread 2: > --------- > > mtx_enter(lock_A, foo); > do stuff; > wakeup_one(foobar); > mtx_exit(lock_A, foo); > > If thread 1 isn't sleeping, the call to wakeup_one() has no effect, which > is fine. > > Basically, all my rambling can be summarized thus: > > 1) The example code is broken (as you point out), but it may fail to > highlight the point you were trying to make. > > 2) The methodology behind any similar example that does highlight a race > condition would be demonstrating a flawed programming idiom, not a > weakness of wakeup_one(). > > Thread synchronization rests on the foundation of mutexes and condition > variables (msleep() is an equivalent). You are suggesting a fundamental > change to one of those synchronization primitives. That should be a pretty > strong indicator by itself that the change isn't needed. =) > > Jason Okay, this is much better. Your first solution is better, because: * msleep is called with A held. * wakeup_one only happens when A is held. so wakeup_one() may be called multiple times if, during the period between thread1 dropping sched_lock and thread2 doing it's stuff and releasing A, some thread3 jumps in with perfect timing and gets A before thread1 does (this is probably highly unlikely). But even if it's called multiple times, as long as slp_cnt is only decremented once, there's no problem. As for your last solution, I would prefer calling wakeup_one() [and traversing the sleep queue with sched_lock held for no good reason] only if need be. Conclusion: I'll scrap the diff and use the method described in your first solution (i.e. sleeping thread does all tampering with the slp_cnt). Note: I think there used to be a reason why I didn't do it this way before, I just can't remember it right now. Maybe it just slipped through. Sorry about that and thanks for the pointer. :-) Cheers, Bosko. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 20 11:21:20 2001 Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id A9D1737B400 for ; Sat, 20 Jan 2001 11:21:00 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id UAA61600; Sat, 20 Jan 2001 20:20:59 +0100 (CET) (envelope-from des@ofug.org) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: arch@freebsd.org Subject: First vm_zone patch From: Dag-Erling Smorgrav Date: 20 Jan 2001 20:20:58 +0100 Message-ID: Lines: 14 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG http://people.freebsd.org/~des/software/vm_zone-20010120.diff Currently, zalloci() and zfreei() always lock the zone, while zalloc() and zfree() only do so in SMP kernels. This patch removes zalloci() and zfreei() and makes zalloc() and zfree() always lock the zone. It also cleans up vm_zone.[ch] a little. I've verified that a patched kernel builds, but not that it runs. The next step is to replace the simple_lock with a mutex, and add a top-level mutex to protect zlist. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 20 12:42:23 2001 Delivered-To: freebsd-arch@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id DAB5837B400 for ; Sat, 20 Jan 2001 12:42:06 -0800 (PST) Received: (qmail 32775 invoked by uid 1142); 20 Jan 2001 20:42:06 -0000 Date: 20 Jan 2001 12:42:06 -0800 Date: Sat, 20 Jan 2001 12:42:00 -0800 From: Jason Evans To: Dag-Erling Smorgrav Cc: arch@freebsd.org Subject: Re: First vm_zone patch Message-ID: <20010120124200.O69199@canonware.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from des@ofug.org on Sat, Jan 20, 2001 at 08:20:58PM +0100 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sat, Jan 20, 2001 at 08:20:58PM +0100, Dag-Erling Smorgrav wrote: > The next step is to replace the simple_lock with a mutex, and add a > top-level mutex to protect zlist. FYI, I'm currently testing a patch that completely removes simplelocks. Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 20 13:32:17 2001 Delivered-To: freebsd-arch@freebsd.org Received: from gw.nectar.com (gw.nectar.com [208.42.49.153]) by hub.freebsd.org (Postfix) with ESMTP id CB42437B402 for ; Sat, 20 Jan 2001 13:31:59 -0800 (PST) Received: from hamlet.nectar.com (hamlet.nectar.com [10.0.1.102]) by gw.nectar.com (Postfix) with ESMTP id 97266193EE; Sat, 20 Jan 2001 15:31:58 -0600 (CST) Received: (from nectar@localhost) by hamlet.nectar.com (8.11.1/8.9.3) id f0KLVwc88206; Sat, 20 Jan 2001 15:31:58 -0600 (CST) (envelope-from nectar@spawn.nectar.com) Date: Sat, 20 Jan 2001 15:31:58 -0600 From: "Jacques A. Vidrine" To: Daniel Eischen Cc: arch@freebsd.org Subject: Re: Request For Review: libc/libc_r changes to allow -lc_r Message-ID: <20010120153158.A88123@hamlet.nectar.com> References: <3A68DDE8.7F8D3C51@vigrid.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <3A68DDE8.7F8D3C51@vigrid.com>; from eischen@vigrid.com on Fri, Jan 19, 2001 at 07:38:00PM -0500 X-Url: http://www.nectar.com/ Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, Jan 19, 2001 at 07:38:00PM -0500, Daniel Eischen wrote: [snip] > Overview of the changes > ----------------------- Thanks for the hard work and the great overview! I have one objection: [snip] > _thread_sys_foo - actual syscall > _foo - weak definition to _thread_sys_foo > foo - weak definition to _thread_sys_foo > > I've changed all the instances of foo() to _foo() in libc for > those hidden system calls. Anyone modifying or adding to libc > will have to be careful to use the same conventions. Please, no. Kill `un-namespace' and let us continue to use the correct name for `foo'. Adding underscores in front of lotsa common calls hurt my eyes and hinders porting between different libc implementations (e.g. our `old' one, other *BSDs). By the way, should it be __thread_sys_foo and __foo? Two underscores? ISTR some rule about using a single leading underscore for file scope (e.g. macros) and two for global scope. Finally, I hope this will lead us into introducing all non- Standard C (or at least non-POSIX) function identifiers in the same fashion, so as to clean up our namespace. For example, err(3). Thanks again & Cheers, -- Jacques Vidrine / n@nectar.com / jvidrine@verio.net / nectar@FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 20 13:49:54 2001 Delivered-To: freebsd-arch@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 77B1837B400 for ; Sat, 20 Jan 2001 13:49:36 -0800 (PST) Received: (qmail 34239 invoked by uid 1142); 20 Jan 2001 21:49:35 -0000 Date: 20 Jan 2001 13:49:35 -0800 Date: Sat, 20 Jan 2001 13:49:26 -0800 From: Jason Evans To: "Jacques A. Vidrine" Cc: Daniel Eischen , arch@freebsd.org Subject: Re: Request For Review: libc/libc_r changes to allow -lc_r Message-ID: <20010120134926.P69199@canonware.com> References: <3A68DDE8.7F8D3C51@vigrid.com> <20010120153158.A88123@hamlet.nectar.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20010120153158.A88123@hamlet.nectar.com>; from n@nectar.com on Sat, Jan 20, 2001 at 03:31:58PM -0600 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sat, Jan 20, 2001 at 03:31:58PM -0600, Jacques A. Vidrine wrote: > On Fri, Jan 19, 2001 at 07:38:00PM -0500, Daniel Eischen wrote: > I have one objection: > > [snip] > > _thread_sys_foo - actual syscall > > _foo - weak definition to _thread_sys_foo > > foo - weak definition to _thread_sys_foo > > > > I've changed all the instances of foo() to _foo() in libc for > > those hidden system calls. Anyone modifying or adding to libc > > will have to be careful to use the same conventions. > > Please, no. Kill `un-namespace' and let us continue to use the > correct name for `foo'. Adding underscores in front of lotsa common > calls hurt my eyes and hinders porting between different libc > implementations (e.g. our `old' one, other *BSDs). Do you have any alternative suggestions? We have been doing this for a number of system calls for almost a year in order to make thread cancellation work, and I can't think of any way to make linking against libc _and_ libpthread work correctly. Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 20 14:27:12 2001 Delivered-To: freebsd-arch@freebsd.org Received: from pcnet1.pcnet.com (pcnet1.pcnet.com [204.213.232.3]) by hub.freebsd.org (Postfix) with ESMTP id EA58D37B400 for ; Sat, 20 Jan 2001 14:26:54 -0800 (PST) Received: (from eischen@localhost) by pcnet1.pcnet.com (8.8.7/PCNet) id RAA09540; Sat, 20 Jan 2001 17:26:27 -0500 (EST) Date: Sat, 20 Jan 2001 17:26:26 -0500 (EST) From: Daniel Eischen To: "Jacques A. Vidrine" Cc: arch@freebsd.org Subject: Re: Request For Review: libc/libc_r changes to allow -lc_r In-Reply-To: <20010120153158.A88123@hamlet.nectar.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sat, 20 Jan 2001, Jacques A. Vidrine wrote: > On Fri, Jan 19, 2001 at 07:38:00PM -0500, Daniel Eischen wrote: > [snip] > > Overview of the changes > > ----------------------- > > Thanks for the hard work and the great overview! > > I have one objection: > > [snip] > > _thread_sys_foo - actual syscall > > _foo - weak definition to _thread_sys_foo > > foo - weak definition to _thread_sys_foo > > > > I've changed all the instances of foo() to _foo() in libc for > > those hidden system calls. Anyone modifying or adding to libc > > will have to be careful to use the same conventions. > > Please, no. Kill `un-namespace' and let us continue to use the > correct name for `foo'. Adding underscores in front of lotsa common > calls hurt my eyes and hinders porting between different libc > implementations (e.g. our `old' one, other *BSDs). Sorry, I don't think that's going to work, especially for sigaction, flock, and kevent. These cannot be #define'd to there underscore(_) equivalents because it would also redefine struct sigaction, struct flock, and struct kevent. The tree has already been populated with lotsa _foo calls; this just finishes the job. > By the way, should it be __thread_sys_foo and __foo? Two underscores? > ISTR some rule about using a single leading underscore for file scope > (e.g. macros) and two for global scope. I don't recall that, but anything for file scope that isn't a macro can be static and not use the underscores. Macros are usually upper case anyways. > Finally, I hope this will lead us into introducing all non- Standard C > (or at least non-POSIX) function identifiers in the same fashion, so > as to clean up our namespace. For example, err(3). I think that's a good idea, and am willing to do the work if given list of the functions that need to be changed. -- Dan Eischen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 20 14:30:17 2001 Delivered-To: freebsd-arch@freebsd.org Received: from pcnet1.pcnet.com (pcnet1.pcnet.com [204.213.232.3]) by hub.freebsd.org (Postfix) with ESMTP id E8D7637B404 for ; Sat, 20 Jan 2001 14:29:59 -0800 (PST) Received: (from eischen@localhost) by pcnet1.pcnet.com (8.8.7/PCNet) id RAA09756; Sat, 20 Jan 2001 17:29:34 -0500 (EST) Date: Sat, 20 Jan 2001 17:29:34 -0500 (EST) From: Daniel Eischen To: Jason Evans Cc: "Jacques A. Vidrine" , arch@freebsd.org Subject: Re: Request For Review: libc/libc_r changes to allow -lc_r In-Reply-To: <20010120134926.P69199@canonware.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 20 Jan 2001, Jason Evans wrote: > On Sat, Jan 20, 2001 at 03:31:58PM -0600, Jacques A. Vidrine wrote: > > On Fri, Jan 19, 2001 at 07:38:00PM -0500, Daniel Eischen wrote: > > I have one objection: > > > > [snip] > > > _thread_sys_foo - actual syscall > > > _foo - weak definition to _thread_sys_foo > > > foo - weak definition to _thread_sys_foo > > > > > > I've changed all the instances of foo() to _foo() in libc for > > > those hidden system calls. Anyone modifying or adding to libc > > > will have to be careful to use the same conventions. > > > > Please, no. Kill `un-namespace' and let us continue to use the > > correct name for `foo'. Adding underscores in front of lotsa common > > calls hurt my eyes and hinders porting between different libc > > implementations (e.g. our `old' one, other *BSDs). > > Do you have any alternative suggestions? We have been doing this for a > number of system calls for almost a year in order to make thread > cancellation work, and I can't think of any way to make linking against > libc _and_ libpthread work correctly. I agree. We've spent a not-so-trivial amount of time making these changes, and I'd hate for us to have to go back to the drawing board. -- Dan Eischen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 20 15:57:47 2001 Delivered-To: freebsd-arch@freebsd.org Received: from vilnya.demon.co.uk (vilnya.demon.co.uk [158.152.19.238]) by hub.freebsd.org (Postfix) with ESMTP id E666A37B400; Sat, 20 Jan 2001 15:57:27 -0800 (PST) Received: from darkstar (darkstar.rings [10.2.4.16]) by vilnya.demon.co.uk (Postfix) with SMTP id 80552D9B8; Sat, 20 Jan 2001 23:57:24 +0000 (GMT) Message-ID: <003401c0833c$39ac3f40$1004020a@darkstar> From: "Cameron Grant" To: , Subject: newpcm/kobj MFC Date: Sat, 20 Jan 2001 23:53:45 -0000 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG this is a followup to my proposal some weeks ago which recieved no relevant commentary. unless vetoed for good reason, i intend to MFC kobj support into 4.x and update newpcm to use it in the next few days. this will not affect any other subsystems apart from kernel .m file expansion which will be changed to determine whether its source file is for kobj or device ops and invoke the relevant script. this change will change the newpcm dki again so modules maintained outside the tree will require updating, but will allow many improvements including the latency improvements now present in -current, and the addition of several new drivers, along with future developments. without this change, newpcm in -stable goes into maintenence mode as i have neither time or motivation to to development in two seperate trees. silence will be interpreted as assent. -cg To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 20 16: 3:20 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mobile.wemm.org (c1315225-a.plstn1.sfba.home.com [65.0.135.147]) by hub.freebsd.org (Postfix) with ESMTP id 09C4137B401 for ; Sat, 20 Jan 2001 16:03:03 -0800 (PST) Received: from netplex.com.au (localhost [127.0.0.1]) by mobile.wemm.org (8.11.1/8.11.1) with ESMTP id f0L02rk43575; Sat, 20 Jan 2001 16:02:54 -0800 (PST) (envelope-from peter@netplex.com.au) Message-Id: <200101210002.f0L02rk43575@mobile.wemm.org> X-Mailer: exmh version 2.2 06/23/2000 with nmh-1.0.4 To: "Jacques A. Vidrine" Cc: Daniel Eischen , arch@FreeBSD.ORG Subject: Re: Request For Review: libc/libc_r changes to allow -lc_r In-Reply-To: <20010120153158.A88123@hamlet.nectar.com> Date: Sat, 20 Jan 2001 16:02:53 -0800 From: Peter Wemm Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG "Jacques A. Vidrine" wrote: > On Fri, Jan 19, 2001 at 07:38:00PM -0500, Daniel Eischen wrote: > [snip] > > Overview of the changes > > ----------------------- > > Thanks for the hard work and the great overview! > > I have one objection: > > [snip] > > _thread_sys_foo - actual syscall > > _foo - weak definition to _thread_sys_foo > > foo - weak definition to _thread_sys_foo > > > > I've changed all the instances of foo() to _foo() in libc for > > those hidden system calls. Anyone modifying or adding to libc > > will have to be careful to use the same conventions. > > Please, no. Kill `un-namespace' and let us continue to use the > correct name for `foo'. Adding underscores in front of lotsa common > calls hurt my eyes and hinders porting between different libc > implementations (e.g. our `old' one, other *BSDs). Hmm. I went through the diff looking for reasons why we needed the un-namespace.h stuff and to use the _ prefix, but now I am not sure. I suspect we probably should add the _ characters and leave the #defines active. Unless I missed something.. On the subject of namespace etc, I wish this went further. Consider malloc(). We provide a strong 'malloc' symbol. I think it would be better if we provided _malloc(), _free(), _calloc() and a weak malloc->_malloc pointer etc. This way if the user process provides an alternative malloc, then libc and everything else should use the one single malloc. There is nothing worse than the internal libc strdup() calling the internal malloc() and passing a pointer back for you to free with *your* free(). > Finally, I hope this will lead us into introducing all non- Standard C > (or at least non-POSIX) function identifiers in the same fashion, so > as to clean up our namespace. For example, err(3). Yes! Something like what bind did with the resolver namespace? ie: if you want to use the res_foo() functions, you must #include in order to get the prototypes and the #define res_foo __res_foo into scope. This stops res_foo() in libc conflicting with your res_foo() in your program. err() is another prime example as you point out. Things like sendfile() would have been good if they could have been handled this way too, but since that was prototyped in a common header ( if I recall correctly), it would not have made much difference for this one. > Thanks again & Cheers, > -- > Jacques Vidrine / n@nectar.com / jvidrine@verio.net / nectar@FreeBSD.org Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 20 16: 3:36 2001 Delivered-To: freebsd-arch@freebsd.org Received: from moby.geekhouse.net (moby.geekhouse.net [64.81.6.36]) by hub.freebsd.org (Postfix) with ESMTP id CFFCB37B400 for ; Sat, 20 Jan 2001 16:03:19 -0800 (PST) Received: from laptop.baldwin.cx (john@dhcp150.geekhouse.net [192.168.1.150]) by moby.geekhouse.net (8.11.0/8.9.3) with ESMTP id f0L07Ws76180; Sat, 20 Jan 2001 16:07:32 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <003401c0833c$39ac3f40$1004020a@darkstar> Date: Sat, 20 Jan 2001 16:03:11 -0800 (PST) From: John Baldwin To: Cameron Grant Subject: RE: newpcm/kobj MFC Cc: arch@FreeBSD.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 20-Jan-01 Cameron Grant wrote: > silence will be interpreted as assent. > > -cg -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 20 16:24: 5 2001 Delivered-To: freebsd-arch@freebsd.org Received: from mobile.wemm.org (c1315225-a.plstn1.sfba.home.com [65.0.135.147]) by hub.freebsd.org (Postfix) with ESMTP id 270C937B400; Sat, 20 Jan 2001 16:23:48 -0800 (PST) Received: from netplex.com.au (localhost [127.0.0.1]) by mobile.wemm.org (8.11.1/8.11.1) with ESMTP id f0L0Nkk43785; Sat, 20 Jan 2001 16:23:46 -0800 (PST) (envelope-from peter@netplex.com.au) Message-Id: <200101210023.f0L0Nkk43785@mobile.wemm.org> X-Mailer: exmh version 2.2 06/23/2000 with nmh-1.0.4 To: "Cameron Grant" Cc: arch@FreeBSD.ORG, stable@FreeBSD.ORG Subject: Re: newpcm/kobj MFC In-Reply-To: <003401c0833c$39ac3f40$1004020a@darkstar> Date: Sat, 20 Jan 2001 16:23:46 -0800 From: Peter Wemm Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG "Cameron Grant" wrote: > this is a followup to my proposal some weeks ago which recieved no relevant > commentary. > > unless vetoed for good reason, i intend to MFC kobj support into 4.x and > update newpcm to use it in the next few days. Please do. Since it doesn't affect anything else, there is no harm at all in this. Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 20 18:58:35 2001 Delivered-To: freebsd-arch@freebsd.org Received: from pcnet1.pcnet.com (pcnet1.pcnet.com [204.213.232.3]) by hub.freebsd.org (Postfix) with ESMTP id 6352037B402 for ; Sat, 20 Jan 2001 18:58:17 -0800 (PST) Received: (from eischen@localhost) by pcnet1.pcnet.com (8.8.7/PCNet) id VAA11038; Sat, 20 Jan 2001 21:57:49 -0500 (EST) Date: Sat, 20 Jan 2001 21:57:49 -0500 (EST) From: Daniel Eischen To: Peter Wemm Cc: "Jacques A. Vidrine" , arch@FreeBSD.ORG Subject: Re: Request For Review: libc/libc_r changes to allow -lc_r In-Reply-To: <200101210002.f0L02rk43575@mobile.wemm.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sat, 20 Jan 2001, Peter Wemm wrote: > "Jacques A. Vidrine" wrote: > > On Fri, Jan 19, 2001 at 07:38:00PM -0500, Daniel Eischen wrote: > > > _thread_sys_foo - actual syscall > > > _foo - weak definition to _thread_sys_foo > > > foo - weak definition to _thread_sys_foo > > > > > > I've changed all the instances of foo() to _foo() in libc for > > > those hidden system calls. Anyone modifying or adding to libc > > > will have to be careful to use the same conventions. > > > > Please, no. Kill `un-namespace' and let us continue to use the > > correct name for `foo'. Adding underscores in front of lotsa common > > calls hurt my eyes and hinders porting between different libc > > implementations (e.g. our `old' one, other *BSDs). > > Hmm. I went through the diff looking for reasons why we needed the > un-namespace.h stuff and to use the _ prefix, but now I am not sure. I > suspect we probably should add the _ characters and leave the #defines > active. Unless I missed something.. At one point, I was convinced that I also needed #pragma weak _foo (which would have gone in un-namespace.h) in order to force weak _references_ to _foo and to make it work correctly in static libraries. This was the problem with the non MT-safe libgcc.a being included in libstdc++. If libstdc++ was linked into a threaded program, it prevented libgcc_r from being pulled in because libstdc++ already included the symbols that were needed (from the non MT-safe libgcc). This was solved by using a libgcc that used weak references to the pthread routines, so that if the threads library was linked in, the pthread_foo references would be non-null and libgcc would then know to use them to make it MT-safe. JDP convinced me that we didn't need to use weak references and I pulled these pragmas out of un-namespace.h. Once thing that I have just added to un-namespace.h are the prototypes to _sigaction, _kevent, and _flock which can't be defined. I do something like this in un-namespace.h: #ifdef _SIGNAL_H int _sigaction(int, const struct sigaction *, struct sigaction *); #endif As for leaving un-namespace.h active, I'd prefer not to and explicity use the _ characters. Some things can't be defined and we need to explicity use their underscored versions. > On the subject of namespace etc, I wish this went further. Consider > malloc(). We provide a strong 'malloc' symbol. I think it would be better > if we provided _malloc(), _free(), _calloc() and a weak malloc->_malloc > pointer etc. This way if the user process provides an alternative malloc, > then libc and everything else should use the one single malloc. This was one of the things I was thinking about too. > There is nothing worse than the internal libc strdup() calling the internal > malloc() and passing a pointer back for you to free with *your* free(). > > > Finally, I hope this will lead us into introducing all non- Standard C > > (or at least non-POSIX) function identifiers in the same fashion, so > > as to clean up our namespace. For example, err(3). > > Yes! Something like what bind did with the resolver namespace? ie: if you > want to use the res_foo() functions, you must #include in order > to get the prototypes and the #define res_foo __res_foo into scope. This > stops res_foo() in libc conflicting with your res_foo() in your program. > err() is another prime example as you point out. Things like sendfile() > would have been good if they could have been handled this way too, but > since that was prototyped in a common header ( if I recall > correctly), it would not have made much difference for this one. I'm all for making other changes, but I'd first like to get these changes committed before trying to come up with an all-encompassing patch set that everyone can agreee on ;-) -- Dan Eischen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 20 19:33:43 2001 Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id B645837B400 for ; Sat, 20 Jan 2001 19:33:26 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id EAA63404; Sun, 21 Jan 2001 04:33:22 +0100 (CET) (envelope-from des@ofug.org) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Jason Evans Cc: arch@freebsd.org Subject: Re: First vm_zone patch References: <20010120124200.O69199@canonware.com> From: Dag-Erling Smorgrav Date: 21 Jan 2001 04:33:21 +0100 In-Reply-To: Jason Evans's message of "20 Jan 2001 12:42:06 -0800, Sat, 20 Jan 2001 12:42:00 -0800" Message-ID: Lines: 11 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Jason Evans writes: > On Sat, Jan 20, 2001 at 08:20:58PM +0100, Dag-Erling Smorgrav wrote: > > The next step is to replace the simple_lock with a mutex, and add a > > top-level mutex to protect zlist. > FYI, I'm currently testing a patch that completely removes simplelocks. Does that mean I should give up reworking vm_zone? DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 20 19:47: 6 2001 Delivered-To: freebsd-arch@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 9C93737B402 for ; Sat, 20 Jan 2001 19:46:49 -0800 (PST) Received: (qmail 42174 invoked by uid 1142); 21 Jan 2001 03:46:48 -0000 Date: 20 Jan 2001 19:46:48 -0800 Date: Sat, 20 Jan 2001 19:46:41 -0800 From: Jason Evans To: Dag-Erling Smorgrav Cc: arch@freebsd.org Subject: Re: First vm_zone patch Message-ID: <20010120194641.Q69199@canonware.com> References: <20010120124200.O69199@canonware.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from des@ofug.org on Sun, Jan 21, 2001 at 04:33:21AM +0100 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sun, Jan 21, 2001 at 04:33:21AM +0100, Dag-Erling Smorgrav wrote: > Jason Evans writes: > > On Sat, Jan 20, 2001 at 08:20:58PM +0100, Dag-Erling Smorgrav wrote: > > > The next step is to replace the simple_lock with a mutex, and add a > > > top-level mutex to protect zlist. > > FYI, I'm currently testing a patch that completely removes simplelocks. > > Does that mean I should give up reworking vm_zone? No, I just wanted to let you know that I have some patches that touch the same code. My only changes are to convert the simplelocks to mutexes, so your work is still of additional value. Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 20 20: 7:47 2001 Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 4D01337B401 for ; Sat, 20 Jan 2001 20:07:29 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id FAA63520; Sun, 21 Jan 2001 05:07:25 +0100 (CET) (envelope-from des@ofug.org) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Jason Evans Cc: arch@FreeBSD.ORG Subject: Re: First vm_zone patch References: <20010120124200.O69199@canonware.com> <20010120194641.Q69199@canonware.com> From: Dag-Erling Smorgrav Date: 21 Jan 2001 05:07:24 +0100 In-Reply-To: Jason Evans's message of "20 Jan 2001 19:46:48 -0800, Sat, 20 Jan 2001 19:46:41 -0800" Message-ID: Lines: 10 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Jason Evans writes: > No, I just wanted to let you know that I have some patches that touch the > same code. My only changes are to convert the simplelocks to mutexes, so > your work is still of additional value. Are those patches available on freefall? DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jan 20 20:10:17 2001 Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id D247C37B401 for ; Sat, 20 Jan 2001 20:09:59 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id FAA63533; Sun, 21 Jan 2001 05:09:56 +0100 (CET) (envelope-from des@ofug.org) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Jason Evans Cc: arch@FreeBSD.ORG Subject: Re: First vm_zone patch References: <20010120124200.O69199@canonware.com> <20010120194641.Q69199@canonware.com> From: Dag-Erling Smorgrav Date: 21 Jan 2001 05:09:55 +0100 In-Reply-To: Dag-Erling Smorgrav's message of "21 Jan 2001 05:07:24 +0100" Message-ID: Lines: 15 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Dag-Erling Smorgrav writes: > Jason Evans writes: > > No, I just wanted to let you know that I have some patches that touch the > > same code. My only changes are to convert the simplelocks to mutexes, so > > your work is still of additional value. > Are those patches available on freefall? Stupid question - I found them within a minute of asking. Your vm_zone.c patch is a subset of the changes I had in mind, I can handle that much of a conflict if you commit your patch in the not-to-far future. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message