From owner-freebsd-arch@FreeBSD.ORG Sun May 23 19:58:33 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1694F16A4CE for ; Sun, 23 May 2004 19:58:33 -0700 (PDT) Received: from smtp2.server.rpi.edu (smtp2.server.rpi.edu [128.113.2.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id A573B43D2F for ; Sun, 23 May 2004 19:58:32 -0700 (PDT) (envelope-from drosih@rpi.edu) Received: from [128.113.24.47] (gilead.netel.rpi.edu [128.113.24.47]) by smtp2.server.rpi.edu (8.12.8/8.12.8) with ESMTP id i4O2wOIX026775 for ; Sun, 23 May 2004 22:58:24 -0400 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: Date: Sun, 23 May 2004 22:58:23 -0400 To: arch@FreeBSD.ORG From: Garance A Drosihn Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Scanned-By: CanIt (www . canit . ca) Subject: Change to "kludge option processing" in /bin/ps X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2004 02:58:33 -0000 After staring at the kludge-option processing in `ps' for awhile, I found enough edge-cases where it simply did not work "right" (for my definition of "right", at keast...), so I rewrote it. As part of that, I also removed the ancient "BACKWARD_COMPATIBILITY" compile-time option, where: ps -options arg1 arg2 (with no '-' on "arg1" and "arg2") was treated as: ps -options -N arg1 -M arg2 I did this because I have often been puzzled/annoyed when I type: ps 12 to get process 12, and then realize I wanted it shown in a different format so I type: ps -u 12 and I get a completely different list of processes, or I type: ps 12 34 and I only see process 12, or I type ps 12 34 56 and get the error message: ps: 56: No such file or directory This BACKWARD_COMPATIBILITY is not documented in the usage() or the man page, so I'd like to replace it. So, I changed `ps' to check for any additional arguments after processing all '-'-options, and if those start with a digit then `ps' will try to use the entire argument as a pid or pidlist. If an extra argument does not start with a digit, then `ps' just prints an error and exits. I want to do more testing on this before committing, but I thought that all this was enough of a change that I should also give people a chance to scream before I commit it. Also, I'm not sure if I might have clobbered some subtle aspect of the kludge processing. If anyone wants to try the update, it is available at: http://people.freebsd.org/~gad/ps-kludge.diff [disclaimer: at the moment it's only had about 15 minutes of testing, but I *think* this is about the way I want it to work] Assuming there aren't any major objections to these ideas, I plan to do some more testing on this and commit it next weekend. -- Garance Alistair Drosehn = gad@gilead.netel.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu From owner-freebsd-arch@FreeBSD.ORG Mon May 24 02:09:00 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 40C9116A4CE; Mon, 24 May 2004 02:09:00 -0700 (PDT) Received: from n33.kp.t-systems-sfr.com (n33.kp.t-systems-sfr.com [129.247.16.33]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3BF1543D2F; Mon, 24 May 2004 02:08:59 -0700 (PDT) (envelope-from harti@freebsd.org) Received: from n81.sp.op.dlr.de (n81g.sp.op.dlr.de [129.247.163.1]) i4O98Fj87866; Mon, 24 May 2004 11:08:16 +0200 Received: from zeus.nt.op.dlr.de (zeus.nt.op.dlr.de [129.247.173.3]) by n81.sp.op.dlr.de (AIX5.1/8.11.6p2/8.11.0) with ESMTP id i4O98FK151224; Mon, 24 May 2004 11:08:15 +0200 Received: from localhost (brandt@localhost) by zeus.nt.op.dlr.de (8.11.7+Sun/8.9.1) with ESMTP id i4O98DM11557; Mon, 24 May 2004 11:08:13 +0200 (MET DST) X-Authentication-Warning: zeus.nt.op.dlr.de: brandt owned process doing -bs Date: Mon, 24 May 2004 11:08:13 +0200 (MET DST) From: Harti Brandt X-X-Sender: brandt@zeus To: Robert Watson In-Reply-To: Message-ID: References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org Subject: Re: Network Stack Locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Harti Brandt List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2004 09:09:00 -0000 On Thu, 20 May 2004, Robert Watson wrote: RW>- ATM -- Harti? :-) Sure. At least netnatm, netgraph/atm and the various drivers. At one point I want to get rid of netatm, so I don't want to put effort into netatm (just keep it working and compliling). netgraph/atm should be clean to the point as netgraph in general is clean (with regards to locking). The drivers (en, [pfh]atm) also are locked, but I'll probably do another iteration when I get my working environment back (I'm currently moving from Berlin to Munich). RW>- Network device drivers -- some have locking, some have correct locking, RW> some have potential interactions with other pieces of the system (such RW> as the USB stack). Note that for a driver to work correctly with a RW> Giant-free system, it must be safe to invoke ifp->if_start() without RW> holding Giant, and for if_start() to be aware that it cannot RW> acquire Giant without generating a lock order issue. It's OK for RW> if_input() to be called with Giant, although undesirable generally. RW> Some drivers also have locking that is commented out by default due to RW> use of recursive locks, but I'm not sure this is necessarily sufficient RW> problem not to just turn on the locking. Is there anybody working on the interaction between the network drivers and the module loader (race condition between the interrupt handler and xxx_detach())? harti From owner-freebsd-arch@FreeBSD.ORG Sun May 23 21:08:12 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 815B916A4CE; Sun, 23 May 2004 21:08:12 -0700 (PDT) Received: from smtp2.server.rpi.edu (smtp2.server.rpi.edu [128.113.2.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1032543D1D; Sun, 23 May 2004 21:08:12 -0700 (PDT) (envelope-from drosih@rpi.edu) Received: from [128.113.24.47] (gilead.netel.rpi.edu [128.113.24.47]) by smtp2.server.rpi.edu (8.12.8/8.12.8) with ESMTP id i4O47ZIX008279; Mon, 24 May 2004 00:07:35 -0400 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: Date: Mon, 24 May 2004 00:07:34 -0400 To: freebsd-ports@freebsd.org From: Garance A Drosihn Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Scanned-By: CanIt (www . canit . ca) X-Mailman-Approved-At: Mon, 24 May 2004 04:41:58 -0700 Subject: Third "RFC" on on pkg-data ideas for ports X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2004 04:08:12 -0000 [this is BCC'ed to -hackers and -arch just so everyone has a chance to see it, but I expect the bulk of the discussion should take place on the freebsd-ports mailing list] Well, Darren and I have done more work on my "pkg-data" ideas, but we're also getting closer to the time where Darren will be busy with his own full-time job, at which point the progress on this will be much slower. So, I'd like to show some of what we've been working on, make a third proposal, and see if this one is interesting enough for us to pursue. If not, then I'll probably just update my web pages with my thoughts so far, and then put this whole idea on a back-burner. [and if you thought progress was slow before, imagine how slow it will be when moved to a back-burner!] In the last go-round, someone pointed out that it could be helpful just to have a better idea of what the ports-collection really *is*. So we took some time to write a script which goes through a ports collection and gathers some statistics what files exist (on a per-port basis), and how much room they take up. I'll post some results of that script as a follow-up to this message. (that reply will only go to freebsd-ports...). So, hopefully that information will be of some interest even if we never do anything with the pkg-data ideas. Someone else (whose name I also forget) said something which focused my attention a bit more on patch-files per se, and how they really aren't the same as the other files I'm trying to collapse into pkg-data. Also, I haven't gotten quite as far along with figuring out what to do with pkg-descr files, so (in the interests of time), I think I'll "leave those alone" for this proposal. We've worked on some other ideas too, but those aren't far enough along yet. So I'll just write them up as "future work" (when I update the web pages...). The third proposal is basically: a) move most "standard" files into a new pkg-data file, as described in previous proposals, except for pkg-descr and "patch" files. b) create a new directory at the root directory of the ports collection. That directory would be called "Patches", and inside would be a directory for each category. Inside each Patches/category directory would be a single-file for each port in that category, where that single-file would have all the "ports-collection patches" for the matching port. c) [minor] in the pkg-data section for distinfo, I'd like to change the format for each file from, eg: MD5 (bash-2.05b.tar.gz) = 5238251b4926d778dfe162f6ce729733 SIZE (bash-2.05b.tar.gz) = 1956216 to 5238251b4926d778dfe162f6ce729733 1956216 bash-2.05b.tar.gz So it collapses most standard files into the pkg-data file, and collapses the patch-related files for a given port into files such as: ports/Patches/shells/patches-bash2. This will not result in as dramatic a drop in inodes, but it has the nice side-effect that Patches are separated from all the other files. Thus, end-users could 'cvsup refuse' the patches for categories that they do not care about, and it would not break operations which work on the entire ports collection (such as `make index'). Our current transform script doesn't do part 'c' yet, but I thought it would be interesting to note the result of 'b': (63) du -sk pd-new/ports pd-new/ports/Patches 190944 pd-new/ports 28414 pd-new/ports/Patches 162530 == "ports without the Patches" And to compare the present ports collection to a transformed ports collection, the result would look like: 1K-blocks Inodes Used Used 238742 79154 pd-orig/ports 190944 49321 pd-new/ports 20% 37% = reduction So, should we pursue any of this? -- Garance Alistair Drosehn = gad@gilead.netel.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu From owner-freebsd-arch@FreeBSD.ORG Mon May 24 07:39:05 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 14E0116A4CE for ; Mon, 24 May 2004 07:39:05 -0700 (PDT) Received: from mail6.speakeasy.net (mail6.speakeasy.net [216.254.0.206]) by mx1.FreeBSD.org (Postfix) with ESMTP id D8F1343D2F for ; Mon, 24 May 2004 07:39:04 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Received: (qmail 6035 invoked from network); 24 May 2004 14:37:53 -0000 Received: from dsl027-160-063.atl1.dsl.speakeasy.net (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) encrypted SMTP for ; 24 May 2004 14:37:53 -0000 Received: from 10.50.40.205 (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.11/8.12.11) with ESMTP id i4OEbgBP098253; Mon, 24 May 2004 10:37:42 -0400 (EDT) (envelope-from jhb@FreeBSD.org) From: John Baldwin To: freebsd-arch@FreeBSD.org Date: Mon, 24 May 2004 10:38:19 -0400 User-Agent: KMail/1.6 References: In-Reply-To: MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200405241038.19589.jhb@FreeBSD.org> X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on server.baldwin.cx cc: arch@FreeBSD.org cc: mtm@FreeBSD.org cc: Garance A Drosihn cc: Julian Elischer Subject: Re: atomic reference counting primatives. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2004 14:39:05 -0000 On Friday 21 May 2004 08:44 pm, Garance A Drosihn wrote: > At 1:56 PM -0700 5/20/04, Julian Elischer wrote: > >This has been raised before but I have come across uses for > >it again and again so I'm raising it again. JHB once posted > >some atomic reference counting primitives. (Do you still have > >them John?) Alfred once said he had some somewhere too, and > >others have commented on this before, but we still don't seem > >to have any. > > Btw, does this thread have anything to do with the present > buuldworld-breakage for sparc64? I notice the compile-time > errors are something like: No. > /usr/src/lib/libthr/thread/thr_cancel.c: In function `testcancel': > /usr/src/lib/libthr/thread/thr_cancel.c:123: warning: passing > arg 1 of `atomic_cmpset_int' from incompatible pointer type > > My guess is that this is related to Mike's change to "Make libthr > async-signal-safe without costly signal masking. [...etc...]". > > This breakage underlines one reason that it would be mighty > convenient to have some "official" set of primitives. It is > one thing if a developer has to roll-their-own solution for > i386, but somewhat more challenging if that solution has to > work across a half-dozen different hardware platforms. atomic_cmpset() is an "official" primitive. The problem is that Mike is using an enum and assuming that all enum's are ints which is not necessarily true. The code should perhaps use an int with #define's instead to guarantee that the variable is an int and not a short, char, or long. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve" = http://www.FreeBSD.org From owner-freebsd-arch@FreeBSD.ORG Mon May 24 07:39:05 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3DF2916A4CE for ; Mon, 24 May 2004 07:39:05 -0700 (PDT) Received: from mail6.speakeasy.net (mail6.speakeasy.net [216.254.0.206]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2306643D41 for ; Mon, 24 May 2004 07:39:05 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Received: (qmail 6035 invoked from network); 24 May 2004 14:37:53 -0000 Received: from dsl027-160-063.atl1.dsl.speakeasy.net (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) encrypted SMTP for ; 24 May 2004 14:37:53 -0000 Received: from 10.50.40.205 (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.11/8.12.11) with ESMTP id i4OEbgBP098253; Mon, 24 May 2004 10:37:42 -0400 (EDT) (envelope-from jhb@FreeBSD.org) From: John Baldwin To: freebsd-arch@FreeBSD.org Date: Mon, 24 May 2004 10:38:19 -0400 User-Agent: KMail/1.6 References: In-Reply-To: MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200405241038.19589.jhb@FreeBSD.org> X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on server.baldwin.cx cc: arch@FreeBSD.org cc: mtm@FreeBSD.org cc: Garance A Drosihn cc: Julian Elischer Subject: Re: atomic reference counting primatives. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2004 14:39:05 -0000 On Friday 21 May 2004 08:44 pm, Garance A Drosihn wrote: > At 1:56 PM -0700 5/20/04, Julian Elischer wrote: > >This has been raised before but I have come across uses for > >it again and again so I'm raising it again. JHB once posted > >some atomic reference counting primitives. (Do you still have > >them John?) Alfred once said he had some somewhere too, and > >others have commented on this before, but we still don't seem > >to have any. > > Btw, does this thread have anything to do with the present > buuldworld-breakage for sparc64? I notice the compile-time > errors are something like: No. > /usr/src/lib/libthr/thread/thr_cancel.c: In function `testcancel': > /usr/src/lib/libthr/thread/thr_cancel.c:123: warning: passing > arg 1 of `atomic_cmpset_int' from incompatible pointer type > > My guess is that this is related to Mike's change to "Make libthr > async-signal-safe without costly signal masking. [...etc...]". > > This breakage underlines one reason that it would be mighty > convenient to have some "official" set of primitives. It is > one thing if a developer has to roll-their-own solution for > i386, but somewhat more challenging if that solution has to > work across a half-dozen different hardware platforms. atomic_cmpset() is an "official" primitive. The problem is that Mike is using an enum and assuming that all enum's are ints which is not necessarily true. The code should perhaps use an int with #define's instead to guarantee that the variable is an int and not a short, char, or long. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve" = http://www.FreeBSD.org From owner-freebsd-arch@FreeBSD.ORG Mon May 24 07:50:49 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3EE5F16A4CE; Mon, 24 May 2004 07:50:49 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id E89D843D4C; Mon, 24 May 2004 07:50:48 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.10/8.12.1) with ESMTP id i4OEo68Q014526; Mon, 24 May 2004 10:50:06 -0400 (EDT) Date: Mon, 24 May 2004 10:50:06 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: John Baldwin In-Reply-To: <200405241038.19589.jhb@FreeBSD.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org cc: mtm@freebsd.org cc: Garance A Drosihn cc: Julian Elischer cc: freebsd-arch@freebsd.org Subject: Re: atomic reference counting primatives. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2004 14:50:49 -0000 On Mon, 24 May 2004, John Baldwin wrote: > On Friday 21 May 2004 08:44 pm, Garance A Drosihn wrote: > > At 1:56 PM -0700 5/20/04, Julian Elischer wrote: > > >This has been raised before but I have come across uses for > > >it again and again so I'm raising it again. JHB once posted > > >some atomic reference counting primitives. (Do you still have > > >them John?) Alfred once said he had some somewhere too, and > > >others have commented on this before, but we still don't seem > > >to have any. > > > > Btw, does this thread have anything to do with the present > > buuldworld-breakage for sparc64? I notice the compile-time > > errors are something like: > > No. > > > /usr/src/lib/libthr/thread/thr_cancel.c: In function `testcancel': > > /usr/src/lib/libthr/thread/thr_cancel.c:123: warning: passing > > arg 1 of `atomic_cmpset_int' from incompatible pointer type > > > > My guess is that this is related to Mike's change to "Make libthr > > async-signal-safe without costly signal masking. [...etc...]". > > > > This breakage underlines one reason that it would be mighty > > convenient to have some "official" set of primitives. It is > > one thing if a developer has to roll-their-own solution for > > i386, but somewhat more challenging if that solution has to > > work across a half-dozen different hardware platforms. > > atomic_cmpset() is an "official" primitive. The problem is that Mike is using > an enum and assuming that all enum's are ints which is not necessarily true. > The code should perhaps use an int with #define's instead to guarantee that > the variable is an int and not a short, char, or long. You can't use atomic_cmpset() in userland on 386, so if it is being used in libthr, the machine must be checked to make sure it will work, otherwise should fall back to something else... -- Dan Eischen From owner-freebsd-arch@FreeBSD.ORG Mon May 24 07:50:49 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3EE5F16A4CE; Mon, 24 May 2004 07:50:49 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id E89D843D4C; Mon, 24 May 2004 07:50:48 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.10/8.12.1) with ESMTP id i4OEo68Q014526; Mon, 24 May 2004 10:50:06 -0400 (EDT) Date: Mon, 24 May 2004 10:50:06 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: John Baldwin In-Reply-To: <200405241038.19589.jhb@FreeBSD.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org cc: mtm@freebsd.org cc: Garance A Drosihn cc: Julian Elischer cc: freebsd-arch@freebsd.org Subject: Re: atomic reference counting primatives. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2004 14:50:49 -0000 On Mon, 24 May 2004, John Baldwin wrote: > On Friday 21 May 2004 08:44 pm, Garance A Drosihn wrote: > > At 1:56 PM -0700 5/20/04, Julian Elischer wrote: > > >This has been raised before but I have come across uses for > > >it again and again so I'm raising it again. JHB once posted > > >some atomic reference counting primitives. (Do you still have > > >them John?) Alfred once said he had some somewhere too, and > > >others have commented on this before, but we still don't seem > > >to have any. > > > > Btw, does this thread have anything to do with the present > > buuldworld-breakage for sparc64? I notice the compile-time > > errors are something like: > > No. > > > /usr/src/lib/libthr/thread/thr_cancel.c: In function `testcancel': > > /usr/src/lib/libthr/thread/thr_cancel.c:123: warning: passing > > arg 1 of `atomic_cmpset_int' from incompatible pointer type > > > > My guess is that this is related to Mike's change to "Make libthr > > async-signal-safe without costly signal masking. [...etc...]". > > > > This breakage underlines one reason that it would be mighty > > convenient to have some "official" set of primitives. It is > > one thing if a developer has to roll-their-own solution for > > i386, but somewhat more challenging if that solution has to > > work across a half-dozen different hardware platforms. > > atomic_cmpset() is an "official" primitive. The problem is that Mike is using > an enum and assuming that all enum's are ints which is not necessarily true. > The code should perhaps use an int with #define's instead to guarantee that > the variable is an int and not a short, char, or long. You can't use atomic_cmpset() in userland on 386, so if it is being used in libthr, the machine must be checked to make sure it will work, otherwise should fall back to something else... -- Dan Eischen From owner-freebsd-arch@FreeBSD.ORG Mon May 24 08:34:13 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2AB9B16A4CE for ; Mon, 24 May 2004 08:34:13 -0700 (PDT) Received: from darkness.comp.waw.pl (darkness.comp.waw.pl [195.117.238.236]) by mx1.FreeBSD.org (Postfix) with ESMTP id 61E8C43D2D for ; Mon, 24 May 2004 08:34:12 -0700 (PDT) (envelope-from pjd@darkness.comp.waw.pl) Received: by darkness.comp.waw.pl (Postfix, from userid 1009) id 840D1ACAFB; Mon, 24 May 2004 13:27:08 +0200 (CEST) Date: Mon, 24 May 2004 13:27:08 +0200 From: Pawel Jakub Dawidek To: freebsd-arch@freebsd.org Message-ID: <20040524112708.GT845@darkness.comp.waw.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Cvw9JPw5vn6N7Cjk" Content-Disposition: inline User-Agent: Mutt/1.4.2i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 5.2.1-RC2 i386 Subject: '-h'/'-H' functionality. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2004 15:34:13 -0000 --Cvw9JPw5vn6N7Cjk Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello. Code for printing disk sizes in human-readable form exists in du(1) and df(1). It is just copied. I want put this functionality to diskinfo(8), but I don't want to copy it again, so maybe it should be moved to libc or somewhere? --=20 Pawel Jakub Dawidek http://www.FreeBSD.org pjd@FreeBSD.org http://garage.freebsd.pl FreeBSD committer Am I Evil? Yes, I Am! --Cvw9JPw5vn6N7Cjk Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (FreeBSD) iD8DBQFAsdwMForvXbEpPzQRAh6AAKCL+ECOgWIKvyS5D7iPFJYPU2xCLQCbBHPo 0f/Bx3BSgi4II+pyoWUqJ04= =vIzq -----END PGP SIGNATURE----- --Cvw9JPw5vn6N7Cjk-- From owner-freebsd-arch@FreeBSD.ORG Mon May 24 08:34:14 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6ACB116A4CE; Mon, 24 May 2004 08:34:14 -0700 (PDT) Received: from arginine.spc.org (arginine.spc.org [195.206.69.236]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2407043D45; Mon, 24 May 2004 08:34:14 -0700 (PDT) (envelope-from bms@spc.org) Received: from localhost (localhost [127.0.0.1]) by arginine.spc.org (Postfix) with ESMTP id 1587C65211; Mon, 24 May 2004 16:33:48 +0100 (BST) Received: from arginine.spc.org ([127.0.0.1]) by localhost (arginine.spc.org [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 55644-04-9; Mon, 24 May 2004 16:33:47 +0100 (BST) Received: from empiric.dek.spc.org (82-147-17-88.dsl.uk.rapidplay.com [82.147.17.88]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by arginine.spc.org (Postfix) with ESMTP id 9902A651F7; Mon, 24 May 2004 16:33:47 +0100 (BST) Received: by empiric.dek.spc.org (Postfix, from userid 1001) id E41AA6137; Mon, 24 May 2004 16:33:46 +0100 (BST) Date: Mon, 24 May 2004 16:33:46 +0100 From: Bruce M Simpson To: Harti Brandt Message-ID: <20040524153346.GI90465@empiric.dek.spc.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: cc: arch@freebsd.org cc: Robert Watson Subject: Re: Network Stack Locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2004 15:34:14 -0000 On Mon, May 24, 2004 at 11:08:13AM +0200, Harti Brandt wrote: > RW>- ATM -- Harti? :-) > > Sure. At least netnatm, netgraph/atm and the various drivers. At one > point I want to get rid of netatm, so I don't want to put effort into > netatm (just keep it working and compliling). I second this. HATM has had its time, NATM is the way to go; it's much more flexible. Regards, BMS From owner-freebsd-arch@FreeBSD.ORG Mon May 24 08:58:00 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 520C716A4CE; Mon, 24 May 2004 08:58:00 -0700 (PDT) Received: from harmony.village.org (rover.village.org [168.103.84.182]) by mx1.FreeBSD.org (Postfix) with ESMTP id C5A9C43D48; Mon, 24 May 2004 08:57:55 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.11/8.12.11) with ESMTP id i4OFthgp085133; Mon, 24 May 2004 09:55:43 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Mon, 24 May 2004 09:55:54 -0600 (MDT) Message-Id: <20040524.095554.03050687.imp@bsdimp.com> To: pjd@freebsd.org From: "M. Warner Losh" In-Reply-To: <20040524112708.GT845@darkness.comp.waw.pl> References: <20040524112708.GT845@darkness.comp.waw.pl> X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: freebsd-arch@freebsd.org Subject: Re: '-h'/'-H' functionality. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2004 15:58:00 -0000 In message: <20040524112708.GT845@darkness.comp.waw.pl> Pawel Jakub Dawidek writes: : Code for printing disk sizes in human-readable form exists in du(1) and : df(1). It is just copied. I want put this functionality to diskinfo(8), : but I don't want to copy it again, so maybe it should be moved to libc : or somewhere? NetBSD has a common set of human readable number routines in libc last time I looked into this. Warner From owner-freebsd-arch@FreeBSD.ORG Mon May 24 09:31:55 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 830BF16A4CE; Mon, 24 May 2004 09:31:55 -0700 (PDT) Received: from srv01.sparkit.no (srv01.sparkit.no [193.69.116.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id DA80643D2D; Mon, 24 May 2004 09:31:54 -0700 (PDT) (envelope-from eivind@FreeBSD.org) Received: from ws.nada ([193.69.114.88]) by srv01.sparkit.no (8.12.11/8.12.11) with ESMTP id i4OGUsiH019896; Mon, 24 May 2004 18:30:54 +0200 (CEST) (envelope-from eivind@FreeBSD.org) Received: from ws.nada (localhost [127.0.0.1]) by ws.nada (8.12.9/8.12.10) with ESMTP id i4OGS4RI008956; Mon, 24 May 2004 16:28:04 GMT (envelope-from eivind@ws.nada) Received: (from eivind@localhost) by ws.nada (8.12.9/8.12.10/Submit) id i4OGS3gE008955; Mon, 24 May 2004 16:28:03 GMT (envelope-from eivind) Date: Mon, 24 May 2004 16:28:02 +0000 From: Eivind Eklund To: Robert Watson Message-ID: <20040524162802.GB2476@FreeBSD.org> References: <200405210103.i4L13QWT068012@apollo.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.4i cc: arch@FreeBSD.org Subject: Re: Network Stack Locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2004 16:31:55 -0000 On Fri, May 21, 2004 at 01:23:51PM -0400, Robert Watson wrote: > The other concern I have is whether the message queues get deep or not: > many of the benefits of message queues come when the queues allow > coallescing of context switches to process multiple packets. If you're > paying a context switch per packet passing through the stack each time you > cross a boundary, there's a non-trivial operational cost to that. I don't know what Matt has done here, but at least with the design we used for G2 (a private DFly-like project that John Dyson, I, and a few other people I don't know if want to be anonymous or not ran), this should not an issue. We used thread context passing with an API that contained putmsg_and_terminate() and message ports that automatically could spawn new handler threads. Effectively, a message-related context switch turned into "assemble everything I care about in a small package, reset the stack pointer, and go". The expectation was that this should end up with less overhead than function calls, as we could drop the call frames for "higher levels in the chain". We never got to the point where we could measure if it worked out that way in practice, though. Eivind. From owner-freebsd-arch@FreeBSD.ORG Mon May 24 10:28:00 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6935016A4CE for ; Mon, 24 May 2004 10:28:00 -0700 (PDT) Received: from mail3.speakeasy.net (mail3.speakeasy.net [216.254.0.203]) by mx1.FreeBSD.org (Postfix) with ESMTP id 35CEA43D2D for ; Mon, 24 May 2004 10:28:00 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Received: (qmail 17520 invoked from network); 24 May 2004 17:27:49 -0000 Received: from dsl027-160-063.atl1.dsl.speakeasy.net (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) encrypted SMTP for ; 24 May 2004 17:27:49 -0000 Received: from 10.50.40.205 (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.11/8.12.11) with ESMTP id i4OHRYFt099249; Mon, 24 May 2004 13:27:35 -0400 (EDT) (envelope-from jhb@FreeBSD.org) From: John Baldwin To: freebsd-arch@FreeBSD.org Date: Mon, 24 May 2004 13:28:05 -0400 User-Agent: KMail/1.6 References: <20040524112708.GT845@darkness.comp.waw.pl> In-Reply-To: <20040524112708.GT845@darkness.comp.waw.pl> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200405241328.05456.jhb@FreeBSD.org> X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on server.baldwin.cx cc: Pawel Jakub Dawidek Subject: Re: '-h'/'-H' functionality. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2004 17:28:00 -0000 On Monday 24 May 2004 07:27 am, Pawel Jakub Dawidek wrote: > Hello. > > Code for printing disk sizes in human-readable form exists in du(1) and > df(1). It is just copied. I want put this functionality to diskinfo(8), > but I don't want to copy it again, so maybe it should be moved to libc > or somewhere? Perhaps libutil to avoid bloating libc further? -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve" = http://www.FreeBSD.org From owner-freebsd-arch@FreeBSD.ORG Mon May 24 11:58:47 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0E4BF16A4CE for ; Mon, 24 May 2004 11:58:46 -0700 (PDT) Received: from mail1.speakeasy.net (mail1.speakeasy.net [216.254.0.201]) by mx1.FreeBSD.org (Postfix) with ESMTP id B67F043D1F for ; Mon, 24 May 2004 11:58:46 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Received: (qmail 26249 invoked from network); 24 May 2004 18:58:33 -0000 Received: from dsl027-160-063.atl1.dsl.speakeasy.net (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) encrypted SMTP for ; 24 May 2004 18:58:33 -0000 Received: from 10.50.40.205 (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.11/8.12.11) with ESMTP id i4OIwSE3099802; Mon, 24 May 2004 14:58:28 -0400 (EDT) (envelope-from jhb@FreeBSD.org) From: John Baldwin To: Daniel Eischen Date: Mon, 24 May 2004 14:59:04 -0400 User-Agent: KMail/1.6 References: In-Reply-To: MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200405241459.04503.jhb@FreeBSD.org> X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on server.baldwin.cx cc: arch@FreeBSD.org cc: mtm@FreeBSD.org cc: Garance A Drosihn cc: Julian Elischer cc: freebsd-arch@FreeBSD.org Subject: Re: atomic reference counting primatives. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2004 18:58:47 -0000 On Monday 24 May 2004 10:50 am, Daniel Eischen wrote: > On Mon, 24 May 2004, John Baldwin wrote: > > atomic_cmpset() is an "official" primitive. The problem is that Mike is > > using an enum and assuming that all enum's are ints which is not > > necessarily true. The code should perhaps use an int with #define's > > instead to guarantee that the variable is an int and not a short, char, > > or long. > > You can't use atomic_cmpset() in userland on 386, so > if it is being used in libthr, the machine must be > checked to make sure it will work, otherwise should > fall back to something else... I'd be fine with it being a compile option to be honest. We already don't support 80386's out of the box since they need a custom kernel. I'd rather not pessimize world + dog for the 80386. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve" = http://www.FreeBSD.org From owner-freebsd-arch@FreeBSD.ORG Mon May 24 11:58:47 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 116E916A4CF for ; Mon, 24 May 2004 11:58:46 -0700 (PDT) Received: from mail1.speakeasy.net (mail1.speakeasy.net [216.254.0.201]) by mx1.FreeBSD.org (Postfix) with ESMTP id CF68043D2D for ; Mon, 24 May 2004 11:58:46 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Received: (qmail 26249 invoked from network); 24 May 2004 18:58:33 -0000 Received: from dsl027-160-063.atl1.dsl.speakeasy.net (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) encrypted SMTP for ; 24 May 2004 18:58:33 -0000 Received: from 10.50.40.205 (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.11/8.12.11) with ESMTP id i4OIwSE3099802; Mon, 24 May 2004 14:58:28 -0400 (EDT) (envelope-from jhb@FreeBSD.org) From: John Baldwin To: Daniel Eischen Date: Mon, 24 May 2004 14:59:04 -0400 User-Agent: KMail/1.6 References: In-Reply-To: MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200405241459.04503.jhb@FreeBSD.org> X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on server.baldwin.cx cc: arch@FreeBSD.org cc: mtm@FreeBSD.org cc: Garance A Drosihn cc: Julian Elischer cc: freebsd-arch@FreeBSD.org Subject: Re: atomic reference counting primatives. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2004 18:58:47 -0000 On Monday 24 May 2004 10:50 am, Daniel Eischen wrote: > On Mon, 24 May 2004, John Baldwin wrote: > > atomic_cmpset() is an "official" primitive. The problem is that Mike is > > using an enum and assuming that all enum's are ints which is not > > necessarily true. The code should perhaps use an int with #define's > > instead to guarantee that the variable is an int and not a short, char, > > or long. > > You can't use atomic_cmpset() in userland on 386, so > if it is being used in libthr, the machine must be > checked to make sure it will work, otherwise should > fall back to something else... I'd be fine with it being a compile option to be honest. We already don't support 80386's out of the box since they need a custom kernel. I'd rather not pessimize world + dog for the 80386. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve" = http://www.FreeBSD.org From owner-freebsd-arch@FreeBSD.ORG Mon May 24 12:18:56 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1EAF816A4CE; Mon, 24 May 2004 12:18:56 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id AA10943D45; Mon, 24 May 2004 12:18:55 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.11/8.12.11) with ESMTP id i4OJIAYw049267; Mon, 24 May 2004 15:18:10 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i4OJI9Bs049264; Mon, 24 May 2004 15:18:09 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Mon, 24 May 2004 15:18:09 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Eivind Eklund In-Reply-To: <20040524162802.GB2476@FreeBSD.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@FreeBSD.org Subject: Re: Network Stack Locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2004 19:18:56 -0000 On Mon, 24 May 2004, Eivind Eklund wrote: > On Fri, May 21, 2004 at 01:23:51PM -0400, Robert Watson wrote: > > The other concern I have is whether the message queues get deep or not: > > many of the benefits of message queues come when the queues allow > > coallescing of context switches to process multiple packets. If you're > > paying a context switch per packet passing through the stack each time you > > cross a boundary, there's a non-trivial operational cost to that. > > I don't know what Matt has done here, but at least with the design we > used for G2 (a private DFly-like project that John Dyson, I, and a few > other people I don't know if want to be anonymous or not ran), this > should not an issue. We used thread context passing with an API that > contained putmsg_and_terminate() and message ports that automatically > could spawn new handler threads. Effectively, a message-related context > switch turned into "assemble everything I care about in a small package, > reset the stack pointer, and go". The expectation was that this should > end up with less overhead than function calls, as we could drop the call > frames for "higher levels in the chain". We never got to the point > where we could measure if it worked out that way in practice, though. Sounds a lot like a lot of the Mach IPC optimizations, including their use of continuations during IPC to avoid a full context switch. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Senior Research Scientist, McAfee Research From owner-freebsd-arch@FreeBSD.ORG Mon May 24 13:25:59 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 08ADE16A4CE; Mon, 24 May 2004 13:25:59 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 87E1243D3F; Mon, 24 May 2004 13:25:58 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.10/8.12.1) with ESMTP id i4OKPq8Q014789; Mon, 24 May 2004 16:25:52 -0400 (EDT) Date: Mon, 24 May 2004 16:25:52 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: John Baldwin In-Reply-To: <200405241459.04503.jhb@FreeBSD.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@FreeBSD.org cc: mtm@FreeBSD.org cc: Garance A Drosihn cc: Julian Elischer cc: freebsd-arch@FreeBSD.org Subject: Re: atomic reference counting primatives. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2004 20:25:59 -0000 On Mon, 24 May 2004, John Baldwin wrote: > On Monday 24 May 2004 10:50 am, Daniel Eischen wrote: > > On Mon, 24 May 2004, John Baldwin wrote: > > > atomic_cmpset() is an "official" primitive. The problem is that Mike is > > > using an enum and assuming that all enum's are ints which is not > > > necessarily true. The code should perhaps use an int with #define's > > > instead to guarantee that the variable is an int and not a short, char, > > > or long. > > > > You can't use atomic_cmpset() in userland on 386, so > > if it is being used in libthr, the machine must be > > checked to make sure it will work, otherwise should > > fall back to something else... > > I'd be fine with it being a compile option to be honest. We already don't > support 80386's out of the box since they need a custom kernel. I'd rather > not pessimize world + dog for the 80386. I'm not too concerned about whether it is a compile or run-time option, just that the code doesn't rely on having that operation in order to work. Unless we just throw up our hands and say libfoo isn't supported on 386... -- Dan Eischen From owner-freebsd-arch@FreeBSD.ORG Mon May 24 13:25:59 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 08ADE16A4CE; Mon, 24 May 2004 13:25:59 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 87E1243D3F; Mon, 24 May 2004 13:25:58 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.10/8.12.1) with ESMTP id i4OKPq8Q014789; Mon, 24 May 2004 16:25:52 -0400 (EDT) Date: Mon, 24 May 2004 16:25:52 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: John Baldwin In-Reply-To: <200405241459.04503.jhb@FreeBSD.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@FreeBSD.org cc: mtm@FreeBSD.org cc: Garance A Drosihn cc: Julian Elischer cc: freebsd-arch@FreeBSD.org Subject: Re: atomic reference counting primatives. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 May 2004 20:25:59 -0000 On Mon, 24 May 2004, John Baldwin wrote: > On Monday 24 May 2004 10:50 am, Daniel Eischen wrote: > > On Mon, 24 May 2004, John Baldwin wrote: > > > atomic_cmpset() is an "official" primitive. The problem is that Mike is > > > using an enum and assuming that all enum's are ints which is not > > > necessarily true. The code should perhaps use an int with #define's > > > instead to guarantee that the variable is an int and not a short, char, > > > or long. > > > > You can't use atomic_cmpset() in userland on 386, so > > if it is being used in libthr, the machine must be > > checked to make sure it will work, otherwise should > > fall back to something else... > > I'd be fine with it being a compile option to be honest. We already don't > support 80386's out of the box since they need a custom kernel. I'd rather > not pessimize world + dog for the 80386. I'm not too concerned about whether it is a compile or run-time option, just that the code doesn't rely on having that operation in order to work. Unless we just throw up our hands and say libfoo isn't supported on 386... -- Dan Eischen From owner-freebsd-arch@FreeBSD.ORG Mon May 24 20:39:40 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C77FB16A4CE; Mon, 24 May 2004 20:39:40 -0700 (PDT) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7A50643D39; Mon, 24 May 2004 20:39:40 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) i4P3dL7Z090506; Mon, 24 May 2004 20:39:21 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.9p2/8.12.9/Submit) id i4P3dLBX090505; Mon, 24 May 2004 20:39:21 -0700 (PDT) (envelope-from dillon) Date: Mon, 24 May 2004 20:39:21 -0700 (PDT) From: Matthew Dillon Message-Id: <200405250339.i4P3dLBX090505@apollo.backplane.com> To: Robert Watson References: cc: arch@FreeBSD.org cc: Eivind Eklund Subject: Re: Network Stack Locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 May 2004 03:39:41 -0000 :On Mon, 24 May 2004, Eivind Eklund wrote: : :> On Fri, May 21, 2004 at 01:23:51PM -0400, Robert Watson wrote: :> > The other concern I have is whether the message queues get deep or not: :> > many of the benefits of message queues come when the queues allow :> > coallescing of context switches to process multiple packets. If you're :> > paying a context switch per packet passing through the stack each time you :> > cross a boundary, there's a non-trivial operational cost to that. :> :> I don't know what Matt has done here, but at least with the design we :> used for G2 (a private DFly-like project that John Dyson, I, and a few :> other people I don't know if want to be anonymous or not ran), this :> should not an issue. We used thread context passing with an API that :> contained putmsg_and_terminate() and message ports that automatically :> could spawn new handler threads. Effectively, a message-related context :> switch turned into "assemble everything I care about in a small package, :> reset the stack pointer, and go". The expectation was that this should :> end up with less overhead than function calls, as we could drop the call :> frames for "higher levels in the chain". We never got to the point :> where we could measure if it worked out that way in practice, though. : :Sounds a lot like a lot of the Mach IPC optimizations, including their use :of continuations during IPC to avoid a full context switch. : :Robert N M Watson FreeBSD Core Team, TrustedBSD Projects :robert@fledge.watson.org Senior Research Scientist, McAfee Research Well, I like the performance aspects of a continuation mechanism, but I really dislike the memory overhead. Even a minimal stack is expensive when you multiply it by potentially hundreds of thousands of 'blocking' entities such as PCBs.. say, a TCP output stream. Because of this the overhead and cache pollution generated by the continuation mechanism increases as system load increases rather then decreases. Deep message queues aren't necessarily a problem and, in fact, having one or two dozen messages backed up in a protocol thread's message port is actually good because the thread can then process all the messages in a tight loop (cpu and cache locality of reference). If designed properly, this directly mitigates the cost of a thread switch as system load increases. So message queueing has the opposite effect... per-unit handling overhead *decreases* as system load increases. (Also, DragonFly's thread scheduler is a much lighter weight mechanism then what you have in FBsd-4 or FBsd-5). e.g.: lets say you have a context switch overhead of 1uS and a message processing overhead of 100ns. light load: 100 messages/sec: 1.1uS/message medium load: 1000 messages/sec, average 10 messages in queue at context switch: 10*100ns+1uS = 2uS/10 = 200ns/msg heavy load: 10000 msgs/sec, average 100 msgs in queue: 100*100ns+1uS = 11uS/100= 110ns/msg The reason a deep message queue is not a problem vs other mechanisms is simple... a message represents a unit of work. The work must be done regardless, and on the cpu it was told to be done on, no matter whether you use a message or a continuation or some other mechanism. In otherwords, a deep message queue is actually an effect of the problem, not a cause of that problem. Solving the problem (if it actually is a problem) does not involve dealing with the deep message queue, it involves dealing with the set of circumstances that are causing that deep message queue to occur. Now, certainly end-to-end latency is an issue. But when one is talking about context switching one is talking about nanoseconds and microseconds. Turn-around latency just isn't an issue most of the time, and in those extremely rare cases where it might be one does the turn-around in the driver interrupt anyway. -Matt Matthew Dillon From owner-freebsd-arch@FreeBSD.ORG Mon May 24 20:51:25 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8DEFC16A4CE; Mon, 24 May 2004 20:51:25 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 19DFE43D31; Mon, 24 May 2004 20:51:25 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.11/8.12.11) with ESMTP id i4P3nCN6075871; Mon, 24 May 2004 23:49:12 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i4P3nCfT075868; Mon, 24 May 2004 23:49:12 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Mon, 24 May 2004 23:49:11 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Matthew Dillon In-Reply-To: <200405250339.i4P3dLBX090505@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@FreeBSD.org cc: Eivind Eklund Subject: Re: Network Stack Locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 May 2004 03:51:25 -0000 On Mon, 24 May 2004, Matthew Dillon wrote: > Deep message queues aren't necessarily a problem and, in fact, having > one or two dozen messages backed up in a protocol thread's message > port is actually good because the thread can then process all the > messages in a tight loop (cpu and cache locality of reference). If > designed properly, this directly mitigates the cost of a thread switch > as system load increases. So message queueing has the opposite effect... > per-unit handling overhead *decreases* as system load increases. Actually, this was the specific point I was making also :-). The question I was asking was about the depth of the message queues between protocol stack layers in actual measurements -- are you observing substantial coallescing between layers as a result of the queues at this point? I'm looking for emperical evidence that the coallescing does make up for the extra context switches of the model in practice... Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Senior Research Scientist, McAfee Research From owner-freebsd-arch@FreeBSD.ORG Mon May 24 23:55:42 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 302C516A4CE for ; Mon, 24 May 2004 23:55:42 -0700 (PDT) Received: from cs.rice.edu (cs.rice.edu [128.42.1.30]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0E12343D3F for ; Mon, 24 May 2004 23:55:42 -0700 (PDT) (envelope-from alc@cs.rice.edu) Received: from localhost (calypso.cs.rice.edu [128.42.1.127]) by cs.rice.edu (Postfix) with ESMTP id 6F6BB4AA05; Tue, 25 May 2004 01:54:11 -0500 (CDT) Received: from cs.rice.edu ([128.42.1.30]) by localhost (calypso.cs.rice.edu [128.42.1.127]) (amavisd-new, port 10024) with LMTP id 02786-01-87; Tue, 25 May 2004 01:54:11 -0500 (CDT) Received: by cs.rice.edu (Postfix, from userid 19572) id 1A73E4A9F2; Tue, 25 May 2004 01:54:11 -0500 (CDT) Date: Tue, 25 May 2004 01:54:10 -0500 From: Alan Cox To: dillon@backplane.com, arch@freebsd.org Message-ID: <20040525065410.GA23877@cs.rice.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.3.28i X-Virus-Scanned: by amavis-20030616-p7 at cs.rice.edu Subject: Re: Network Stack Locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 May 2004 06:55:42 -0000 >:Sounds a lot like a lot of the Mach IPC optimizations, including their use >:of continuations during IPC to avoid a full context switch. >: >:Robert N M Watson FreeBSD Core Team, TrustedBSD Projects >:robert@fledge.watson.org Senior Research Scientist, McAfee Research > > Well, I like the performance aspects of a continuation mechanism, but > I really dislike the memory overhead. Even a minimal stack is > expensive when you multiply it by potentially hundreds of thousands > of 'blocking' entities such as PCBs.. say, a TCP output stream. > Because of this the overhead and cache pollution generated by the > continuation mechanism increases as system load increases rather > then decreases. When the explicit continuation mechanism was used, the thread's stack was freed when the thread blocked and a new stack allocated when the thread was restarted. Here is a URL: http://citeseer.ist.psu.edu/draves91using.html. Notice the mention of space reduction in the abstract. It's worth reading. Alan From owner-freebsd-arch@FreeBSD.ORG Tue May 25 11:21:07 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0218A16A4CE; Tue, 25 May 2004 11:21:07 -0700 (PDT) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id AC8AE43D48; Tue, 25 May 2004 11:21:04 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) i4PIKn7Z093872; Tue, 25 May 2004 11:20:49 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.9p2/8.12.9/Submit) id i4PIKnil093871; Tue, 25 May 2004 11:20:49 -0700 (PDT) (envelope-from dillon) Date: Tue, 25 May 2004 11:20:49 -0700 (PDT) From: Matthew Dillon Message-Id: <200405251820.i4PIKnil093871@apollo.backplane.com> To: Robert Watson References: cc: arch@FreeBSD.org cc: Eivind Eklund Subject: Re: Network Stack Locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 May 2004 18:21:07 -0000 : : :On Mon, 24 May 2004, Matthew Dillon wrote: : :> Deep message queues aren't necessarily a problem and, in fact, having :> one or two dozen messages backed up in a protocol thread's message :> port is actually good because the thread can then process all the :> messages in a tight loop (cpu and cache locality of reference). If :> designed properly, this directly mitigates the cost of a thread switch :> as system load increases. So message queueing has the opposite effect... :> per-unit handling overhead *decreases* as system load increases. : :Actually, this was the specific point I was making also :-). The question :I was asking was about the depth of the message queues between protocol :stack layers in actual measurements -- are you observing substantial :coallescing between layers as a result of the queues at this point? I'm :looking for emperical evidence that the coallescing does make up for the :extra context switches of the model in practice... : :Robert N M Watson FreeBSD Core Team, TrustedBSD Projects :robert@fledge.watson.org Senior Research Scientist, McAfee Research I won't be able to test that for a while. So far most of the tests we've conducted have been simply TCP and UDP streams (e.g. rcp, NFS mounts, etc). I'll keep it in mind when we get to that point. It should also be noted that since today's processors are so damn fast, actually maxing out the cpu before maxing out the PCI bus(es), at least on a standard workstation, is difficult. -Matt Matthew Dillon From owner-freebsd-arch@FreeBSD.ORG Tue May 25 11:30:02 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C3DF916A4CE for ; Tue, 25 May 2004 11:30:02 -0700 (PDT) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 76F2F43D39 for ; Tue, 25 May 2004 11:30:00 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) i4PITj7Z093922; Tue, 25 May 2004 11:29:45 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.9p2/8.12.9/Submit) id i4PITjbp093921; Tue, 25 May 2004 11:29:45 -0700 (PDT) (envelope-from dillon) Date: Tue, 25 May 2004 11:29:45 -0700 (PDT) From: Matthew Dillon Message-Id: <200405251829.i4PITjbp093921@apollo.backplane.com> To: Alan Cox References: <20040525065410.GA23877@cs.rice.edu> cc: arch@freebsd.org Subject: Re: Network Stack Locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 May 2004 18:30:02 -0000 :>:Sounds a lot like a lot of the Mach IPC optimizations, including their use :>:of continuations during IPC to avoid a full context switch. :>: :>:Robert N M Watson FreeBSD Core Team, TrustedBSD Projects :>:robert@fledge.watson.org Senior Research Scientist, McAfee Research :> :> Well, I like the performance aspects of a continuation mechanism, but :> I really dislike the memory overhead. Even a minimal stack is :> expensive when you multiply it by potentially hundreds of thousands :> of 'blocking' entities such as PCBs.. say, a TCP output stream. :> Because of this the overhead and cache pollution generated by the :> continuation mechanism increases as system load increases rather :> then decreases. : :When the explicit continuation mechanism was used, the thread's stack :was freed when the thread blocked and a new stack allocated when the :thread was restarted. Here is a URL: :http://citeseer.ist.psu.edu/draves91using.html. Notice the mention of :space reduction in the abstract. It's worth reading. : :Alan Ah, now I understand. The continuation is an exit from the procedure. This sounds very similar to an interrupt mechanism I designed about a decade ago. Instead of saving and restoring the interrupt context for the interrupt thread, the thread switch was special-cased to basically create a context (by pushing a procedure call on the stack) on switch-in and to throw it away on switch-out (which was always at the end of the interrupt routine). I extended the same mechanism down into 'userland' by creating a 'waitforever()' system call which basically did nothing but wait for and dispatch signal vectors (most of the programs were event oriented and had no main loop). The nice thing about this was that no context had to be saved while the program was sitting in waitforever(), or restored when the program returned from a signal handler. This more then doubled scheduler performance (which on a 10MHz 68000 was important). In many ways, the continuation mechanism and the message queue mechanism appear to be nearly identical. If an explicit exit from a procedure is required to optimize the stack with the continuation mechanism, then that isn't much different then moving the message to an event queue and returning to the message processing loop. Neither case allows you to save stack context or to save the current procedural stacking level, and both mechanisms allow you to reuse your current stack to handle multiple messages/continuations. -Matt Matthew Dillon From owner-freebsd-arch@FreeBSD.ORG Tue May 25 11:32:20 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E123E16A4CE; Tue, 25 May 2004 11:32:20 -0700 (PDT) Received: from xorpc.icir.org (xorpc.icir.org [192.150.187.68]) by mx1.FreeBSD.org (Postfix) with ESMTP id A689343D48; Tue, 25 May 2004 11:32:20 -0700 (PDT) (envelope-from rizzo@icir.org) Received: from xorpc.icir.org (localhost [127.0.0.1]) by xorpc.icir.org (8.12.9p1/8.12.8) with ESMTP id i4PIW8gd074604; Tue, 25 May 2004 11:32:08 -0700 (PDT) (envelope-from rizzo@xorpc.icir.org) Received: (from rizzo@localhost) by xorpc.icir.org (8.12.9p1/8.12.3/Submit) id i4PIW8Cq074603; Tue, 25 May 2004 11:32:08 -0700 (PDT) (envelope-from rizzo) Date: Tue, 25 May 2004 11:32:08 -0700 From: Luigi Rizzo To: Matthew Dillon Message-ID: <20040525113208.A74130@xorpc.icir.org> References: <200405251820.i4PIKnil093871@apollo.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <200405251820.i4PIKnil093871@apollo.backplane.com>; from dillon@apollo.backplane.com on Tue, May 25, 2004 at 11:20:49AM -0700 cc: arch@freebsd.org cc: Robert Watson Subject: Re: Network Stack Locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 May 2004 18:32:21 -0000 On Tue, May 25, 2004 at 11:20:49AM -0700, Matthew Dillon wrote: ... > :Actually, this was the specific point I was making also :-). The question > :I was asking was about the depth of the message queues between protocol > :stack layers in actual measurements -- are you observing substantial > :coallescing between layers as a result of the queues at this point? I'm ... > It should also be noted that since today's processors are so damn fast, > actually maxing out the cpu before maxing out the PCI bus(es), at least > on a standard workstation, is difficult. that's where the soekris comes handy :) (yes i understand that much of the issues here are related to SMP, but there lot of evidence that a non-smp 5.x kernel is significantly slower than the equivalent 4.x code, even on heavy network i/o loads, so there is certainly something to clean up there too). cheers luigi From owner-freebsd-arch@FreeBSD.ORG Tue May 25 11:42:41 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4AFA516A4CF; Tue, 25 May 2004 11:42:41 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id BCE7043D1F; Tue, 25 May 2004 11:42:40 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.11/8.12.11) with ESMTP id i4PIfowP093428; Tue, 25 May 2004 14:41:50 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i4PIfods093425; Tue, 25 May 2004 14:41:50 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Tue, 25 May 2004 14:41:50 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Luigi Rizzo In-Reply-To: <20040525113208.A74130@xorpc.icir.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org Subject: Re: Network Stack Locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 May 2004 18:42:41 -0000 On Tue, 25 May 2004, Luigi Rizzo wrote: > On Tue, May 25, 2004 at 11:20:49AM -0700, Matthew Dillon wrote: > ... > > :Actually, this was the specific point I was making also :-). The question > > :I was asking was about the depth of the message queues between protocol > > :stack layers in actual measurements -- are you observing substantial > > :coallescing between layers as a result of the queues at this point? I'm > ... > > It should also be noted that since today's processors are so damn fast, > > actually maxing out the cpu before maxing out the PCI bus(es), at least > > on a standard workstation, is difficult. > > that's where the soekris comes handy :) (yes i understand that much of > the issues here are related to SMP, but there lot of evidence that a > non-smp 5.x kernel is significantly slower than the equivalent 4.x code, > even on heavy network i/o loads, so there is certainly something to > clean up there too). Happily, I'm well endowed with old and crufty hardware when it comes to UP. That said, it's hard to find a nice middle ground with SMP -- even relatively slow boxes can typically can max out 32-bit PCI, so you need to go with 64-bit if you want to not have that be the bottleneck for gig-E. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Senior Research Scientist, McAfee Research From owner-freebsd-arch@FreeBSD.ORG Tue May 25 14:17:59 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 20A7C16A4CE for ; Tue, 25 May 2004 14:17:59 -0700 (PDT) Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by mx1.FreeBSD.org (Postfix) with ESMTP id E2B8043D4C for ; Tue, 25 May 2004 14:17:57 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([24.7.73.28]) by comcast.net (rwcrmhc13) with ESMTP id <200405252117430150070bode>; Tue, 25 May 2004 21:17:44 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id OAA43664; Tue, 25 May 2004 14:17:41 -0700 (PDT) Date: Tue, 25 May 2004 14:17:40 -0700 (PDT) From: Julian Elischer To: Matthew Dillon In-Reply-To: <200405251829.i4PITjbp093921@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org cc: Alan Cox Subject: Re: Network Stack Locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 May 2004 21:17:59 -0000 On Tue, 25 May 2004, Matthew Dillon wrote: > > :>:Sounds a lot like a lot of the Mach IPC optimizations, including their use > :>:of continuations during IPC to avoid a full context switch. > :>: > :>:Robert N M Watson FreeBSD Core Team, TrustedBSD Projects > :>:robert@fledge.watson.org Senior Research Scientist, McAfee Research > :> > :> Well, I like the performance aspects of a continuation mechanism, but > :> I really dislike the memory overhead. Even a minimal stack is > :> expensive when you multiply it by potentially hundreds of thousands > :> of 'blocking' entities such as PCBs.. say, a TCP output stream. > :> Because of this the overhead and cache pollution generated by the > :> continuation mechanism increases as system load increases rather > :> then decreases. > : > :When the explicit continuation mechanism was used, the thread's stack > :was freed when the thread blocked and a new stack allocated when the > :thread was restarted. Here is a URL: > :http://citeseer.ist.psu.edu/draves91using.html. Notice the mention of > :space reduction in the abstract. It's worth reading. > : > :Alan > > Ah, now I understand. The continuation is an exit from the procedure. > > This sounds very similar to an interrupt mechanism I designed about > a decade ago. Instead of saving and restoring the interrupt > context for the interrupt thread, the thread switch was special-cased > to basically create a context (by pushing a procedure call on the stack) > on switch-in and to throw it away on switch-out (which was always at > the end of the interrupt routine). I extended the same mechanism down > into 'userland' by creating a 'waitforever()' system call which basically > did nothing but wait for and dispatch signal vectors (most of the programs > were event oriented and had no main loop). The nice thing about this was > that no context had to be saved while the program was sitting in > waitforever(), or restored when the program returned from a signal handler. > This more then doubled scheduler performance (which on a 10MHz 68000 was > important). > > In many ways, the continuation mechanism and the message queue mechanism > appear to be nearly identical. If an explicit exit from a procedure > is required to optimize the stack with the continuation mechanism, then > that isn't much different then moving the message to an event queue > and returning to the message processing loop. Neither case allows > you to save stack context or to save the current procedural stacking > level, and both mechanisms allow you to reuse your current stack to > handle multiple messages/continuations. This is how netgraph works. (particularly in 5.x) messages are queued for a node.. they can have 3 types 1/ mbuf data 2/ control message 3/ procedure execution request. the 3rd type is for such things as: timeouts.. the timeout request simply adds a request to the queue that the node execute it's timeout function. The queue entry specifies whether it needs to have a reader or writer lock on the node. By default (but not written in stone), mbuf data looks for a reader, and the others request a writer lock, so data could pass through the same node at the same time on different cpus but a control message woudl not be able to be done at the same time as anything else. If the node lock can be had at the time that the item is to be queued, and there is nothing else in the queue, then the context of the caller can be used to process the item imediatly. Otherwise whoever has the lock should process all waiting items when they finish. > > -Matt > Matthew Dillon > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Tue May 25 18:21:15 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7E69E16A4CE for ; Tue, 25 May 2004 18:21:15 -0700 (PDT) Received: from mail-in-01.arcor-online.net (mail-in-01.arcor-online.net [151.189.21.41]) by mx1.FreeBSD.org (Postfix) with ESMTP id B44DF43D39 for ; Tue, 25 May 2004 18:21:14 -0700 (PDT) (envelope-from mailnull@mips.inka.de) Received: from kemoauc.mips.inka.de (dsl-082-083-167-055.arcor-ip.net [82.83.167.55]) by mail-in-01.arcor-online.net (Postfix) with ESMTP id 69EF8BFD762 for ; Wed, 26 May 2004 03:21:07 +0200 (CEST) Received: from kemoauc.mips.inka.de (localhost [127.0.0.1]) by kemoauc.mips.inka.de (8.12.11/8.12.10) with ESMTP id i4Q1L6Zm069998 for ; Wed, 26 May 2004 03:21:06 +0200 (CEST) (envelope-from mailnull@kemoauc.mips.inka.de) Received: (from mailnull@localhost) by kemoauc.mips.inka.de (8.12.11/8.12.11/Submit) id i4Q1L6OC069997 for freebsd-arch@freebsd.org; Wed, 26 May 2004 03:21:06 +0200 (CEST) (envelope-from mailnull) From: naddy@mips.inka.de (Christian Weisgerber) Date: Wed, 26 May 2004 01:21:05 +0000 (UTC) Message-ID: References: <20040524112708.GT845@darkness.comp.waw.pl> Originator: naddy@mips.inka.de (Christian Weisgerber) To: freebsd-arch@freebsd.org Subject: Re: '-h'/'-H' functionality. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 May 2004 01:21:15 -0000 Pawel Jakub Dawidek wrote: > Code for printing disk sizes in human-readable form exists in du(1) and > df(1). It is just copied. I want put this functionality to diskinfo(8), > but I don't want to copy it again, so maybe it should be moved to libc > or somewhere? OpenBSD put fmt_scaled(3) and scan_scaled(3) into libutil. -- Christian "naddy" Weisgerber naddy@mips.inka.de From owner-freebsd-arch@FreeBSD.ORG Wed May 26 04:42:56 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6763016A4CE for ; Wed, 26 May 2004 04:42:56 -0700 (PDT) Received: from darkness.comp.waw.pl (darkness.comp.waw.pl [195.117.238.236]) by mx1.FreeBSD.org (Postfix) with ESMTP id 15B1B43D48 for ; Wed, 26 May 2004 04:42:55 -0700 (PDT) (envelope-from pjd@darkness.comp.waw.pl) Received: by darkness.comp.waw.pl (Postfix, from userid 1009) id A9F65ACAE6; Wed, 26 May 2004 13:42:01 +0200 (CEST) Date: Wed, 26 May 2004 13:42:01 +0200 From: Pawel Jakub Dawidek To: Christian Weisgerber Message-ID: <20040526114201.GI845@darkness.comp.waw.pl> References: <20040524112708.GT845@darkness.comp.waw.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="qSA16V2Xa97CV7Fv" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 5.2.1-RC2 i386 cc: freebsd-arch@freebsd.org Subject: Re: '-h'/'-H' functionality. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 May 2004 11:42:56 -0000 --qSA16V2Xa97CV7Fv Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, May 26, 2004 at 01:21:05AM +0000, Christian Weisgerber wrote: +> > Code for printing disk sizes in human-readable form exists in du(1) and +> > df(1). It is just copied. I want put this functionality to diskinfo(8), +> > but I don't want to copy it again, so maybe it should be moved to libc +> > or somewhere? +>=20 +> OpenBSD put fmt_scaled(3) and scan_scaled(3) into libutil. I already committed NetBSD's humanize_number(3). --=20 Pawel Jakub Dawidek http://www.FreeBSD.org pjd@FreeBSD.org http://garage.freebsd.pl FreeBSD committer Am I Evil? Yes, I Am! --qSA16V2Xa97CV7Fv Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (FreeBSD) iD8DBQFAtIKJForvXbEpPzQRAlcrAKDfy3etk6qP2jrQjzOTReQ8j07/3QCfZUAI tsgeJeO6iLUMReBh9rWWao8= =J0YY -----END PGP SIGNATURE----- --qSA16V2Xa97CV7Fv-- From owner-freebsd-arch@FreeBSD.ORG Wed May 26 14:27:29 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BD6A116A4CE for ; Wed, 26 May 2004 14:27:29 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3FF5A43D41 for ; Wed, 26 May 2004 14:27:29 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.11/8.12.11) with ESMTP id i4QLQVU4026244; Wed, 26 May 2004 17:26:31 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i4QLQVff026241; Wed, 26 May 2004 17:26:31 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Wed, 26 May 2004 17:26:31 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Matthew Dillon In-Reply-To: <200405251829.i4PITjbp093921@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org cc: Alan Cox Subject: Re: Network Stack Locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 May 2004 21:27:29 -0000 On Tue, 25 May 2004, Matthew Dillon wrote: > This sounds very similar to an interrupt mechanism I designed about > a decade ago. Instead of saving and restoring the interrupt > context for the interrupt thread, the thread switch was special-cased > to basically create a context (by pushing a procedure call on the stack) > on switch-in and to throw it away on switch-out (which was always at > the end of the interrupt routine). I extended the same mechanism down > into 'userland' by creating a 'waitforever()' system call which basically > did nothing but wait for and dispatch signal vectors (most of the programs > were event oriented and had no main loop). The nice thing about this was > that no context had to be saved while the program was sitting in > waitforever(), or restored when the program returned from a signal handler. > This more then doubled scheduler performance (which on a 10MHz 68000 was > important). > > In many ways, the continuation mechanism and the message queue mechanism > appear to be nearly identical. If an explicit exit from a procedure > is required to optimize the stack with the continuation mechanism, then > that isn't much different then moving the message to an event queue > and returning to the message processing loop. Neither case allows > you to save stack context or to save the current procedural stacking > level, and both mechanisms allow you to reuse your current stack to > handle multiple messages/continuations. I'm not sure if you've spent much time looking at the Mach kernel and literature, but a lot of the concepts in AmigaOS and DFBSD are highly parallel to concepts in Mach (and hence in many derived systems). The original Mach project at CMU (from about 1985-1993) still has a web page: http://www-2.cs.cmu.edu/afs/cs/project/mach/public/www/mach.html I don't remember if there's specifically a paper on continuations there, but it should be discussed in their developer guides, threading and parallelism bits, and documentation of their scheduler. There's also been follow-up work on Mach in a great many environments, including work at OSF, University of Utah (FLUX, etc), by the GNU folks on Hurd, NeXT, Apple, et al. There's also been follow-up work at CMU on real time scheduling in Mach. The reason I tend to raise aspects of Darwin when responding to your e-mails regarding DFBSD is that, while the details are often pretty different, the general approach reveals many parallels. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Senior Research Scientist, McAfee Research From owner-freebsd-arch@FreeBSD.ORG Wed May 26 14:30:42 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 264D916A4CE for ; Wed, 26 May 2004 14:30:42 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id B81E543D31 for ; Wed, 26 May 2004 14:30:41 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.11/8.12.11) with ESMTP id i4QLStZI026288; Wed, 26 May 2004 17:28:55 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i4QLSnc6026284; Wed, 26 May 2004 17:28:55 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Wed, 26 May 2004 17:28:49 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Julian Elischer In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org cc: Alan Cox Subject: Re: Network Stack Locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 May 2004 21:30:42 -0000 On Tue, 25 May 2004, Julian Elischer wrote: > This is how netgraph works. (particularly in 5.x) It's also worth pointing out, for those looking at various approaches, that NetBSD is currently discussing continuation primitives on the tech-kern mailing list (I seem to have gotten added to the CC line lately, which I don't object to :-). You can also argue that things like bio's, PCB's, et al, are implementations of the continuation philosophy, just lacking a abstracted primitive. They maintain state to be picked up by a context but discarding the stack. Callouts and callbacks are used to perform scheduling, however, rather than the scheduler explicitly. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Senior Research Scientist, McAfee Research From owner-freebsd-arch@FreeBSD.ORG Wed May 26 21:26:18 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5AC6B16A4CE for ; Wed, 26 May 2004 21:26:18 -0700 (PDT) Received: from beastie.mckusick.com (dsl081-247-227.sfo1.dsl.speakeasy.net [64.81.247.227]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0113543D54 for ; Wed, 26 May 2004 21:26:18 -0700 (PDT) (envelope-from mckusick@mckusick.com) Received: from beastie.mckusick.com (localhost [127.0.0.1]) by beastie.mckusick.com (8.12.8/8.12.9) with ESMTP id i4R4OwU9024429 for ; Wed, 26 May 2004 21:24:58 -0700 (PDT) (envelope-from mckusick@beastie.mckusick.com) Message-Id: <200405270424.i4R4OwU9024429@beastie.mckusick.com> To: arch@freebsd.org X-URL: http://WWW.McKusick.COM/ Date: Wed, 26 May 2004 21:24:58 -0700 From: Kirk McKusick Subject: The Design and Implementation of the FreeBSD Operating System X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Kirk McKusick List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 May 2004 04:26:18 -0000 My forthcoming book, ``The Design and Implementation of the FreeBSD Operating System'' can now be pre-ordered from Amazon.com, see http://tinyurl.com/2bklt. Details on its coverage can be found at http://www.mckusick.com/FreeBSDbook.html. Kirk McKusick From owner-freebsd-arch@FreeBSD.ORG Thu May 27 01:18:05 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4D19216A4CE; Thu, 27 May 2004 01:18:05 -0700 (PDT) Received: from smtp.omnis.com (smtp.omnis.com [216.239.128.26]) by mx1.FreeBSD.org (Postfix) with ESMTP id 15EC143D31; Thu, 27 May 2004 01:18:05 -0700 (PDT) (envelope-from wes@softweyr.com) Received: from softweyr.homeunix.net (24-161-166-146.san.rr.com [24.161.166.146]) by smtp-relay.omnis.com (Postfix) with ESMTP id 52338881BD7; Thu, 27 May 2004 01:17:42 -0700 (PDT) From: Wes Peters Organization: Softweyr.COM To: freebsd-arch@freebsd.org Date: Thu, 27 May 2004 01:17:42 -0700 User-Agent: KMail/1.6.1 References: <20040524112708.GT845@darkness.comp.waw.pl> <20040526114201.GI845@darkness.comp.waw.pl> In-Reply-To: <20040526114201.GI845@darkness.comp.waw.pl> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200405270117.42657.wes@softweyr.com> cc: Pawel Jakub Dawidek cc: Christian Weisgerber Subject: Re: '-h'/'-H' functionality. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 May 2004 08:18:05 -0000 On Wednesday 26 May 2004 04:42, Pawel Jakub Dawidek wrote: > On Wed, May 26, 2004 at 01:21:05AM +0000, Christian Weisgerber wrote: > +> > Code for printing disk sizes in human-readable form exists in du(1) > and +> > df(1). It is just copied. I want put this functionality to > diskinfo(8), +> > but I don't want to copy it again, so maybe it should > be moved to libc +> > or somewhere? > +> > +> OpenBSD put fmt_scaled(3) and scan_scaled(3) into libutil. > > I already committed NetBSD's humanize_number(3). Roxxor! Don't you just love it when things go right, especially when they go right at someone else's expense? ;^) -- Where am I, and what am I doing in this handbasket? Wes Peters wes@softweyr.com From owner-freebsd-arch@FreeBSD.ORG Thu May 27 03:52:23 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8CE3F16A4CE; Thu, 27 May 2004 03:52:23 -0700 (PDT) Received: from mx.nsu.ru (mx.nsu.ru [212.192.164.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id B97D343D31; Thu, 27 May 2004 03:52:22 -0700 (PDT) (envelope-from danfe@regency.nsu.ru) Received: from regency.nsu.ru ([193.124.210.26]) by mx.nsu.ru with esmtp (Exim 4.30) id 1BTIUM-00070T-8o; Thu, 27 May 2004 17:51:54 +0700 Received: from regency.nsu.ru (localhost [127.0.0.1]) by regency.nsu.ru (8.12.10/8.12.10) with ESMTP id i4RAhwAT052479; Thu, 27 May 2004 17:43:58 +0700 (NOVST) (envelope-from danfe@regency.nsu.ru) Received: (from danfe@localhost) by regency.nsu.ru (8.12.10/8.12.10/Submit) id i4RAgq0w051959; Thu, 27 May 2004 17:42:52 +0700 (NOVST) (envelope-from danfe) Date: Thu, 27 May 2004 17:42:52 +0700 From: Alexey Dokuchaev To: Wes Peters Message-ID: <20040527104252.GA39780@regency.nsu.ru> References: <20040524112708.GT845@darkness.comp.waw.pl> <20040526114201.GI845@darkness.comp.waw.pl> <200405270117.42657.wes@softweyr.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200405270117.42657.wes@softweyr.com> User-Agent: Mutt/1.4.2.1i cc: Pawel Jakub Dawidek cc: Christian Weisgerber cc: freebsd-arch@freebsd.org Subject: Re: '-h'/'-H' functionality. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 May 2004 10:52:23 -0000 On Thu, May 27, 2004 at 01:17:42AM -0700, Wes Peters wrote: > On Wednesday 26 May 2004 04:42, Pawel Jakub Dawidek wrote: > > On Wed, May 26, 2004 at 01:21:05AM +0000, Christian Weisgerber wrote: > > +> > Code for printing disk sizes in human-readable form exists in du(1) > > and +> > df(1). It is just copied. I want put this functionality to > > diskinfo(8), +> > but I don't want to copy it again, so maybe it should > > be moved to libc +> > or somewhere? > > +> > > +> OpenBSD put fmt_scaled(3) and scan_scaled(3) into libutil. > > > > I already committed NetBSD's humanize_number(3). > > Roxxor! Don't you just love it when things go right, especially when they > go right at someone else's expense? ;^) Yeah, especially considering that NetBSD's implemenation IMHO is better. ./danfe From owner-freebsd-arch@FreeBSD.ORG Wed May 26 10:21:57 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8236B16A4CF for ; Wed, 26 May 2004 10:21:57 -0700 (PDT) Received: from smtp105.mail.sc5.yahoo.com (smtp105.mail.sc5.yahoo.com [66.163.169.225]) by mx1.FreeBSD.org (Postfix) with SMTP id 35E8C43D1D for ; Wed, 26 May 2004 10:21:57 -0700 (PDT) (envelope-from jvr_78@yahoo.com.ar) Received: from unknown (HELO 192) (jvr?78@200.69.53.82 with login) by smtp105.mail.sc5.yahoo.com with SMTP; 26 May 2004 17:21:18 -0000 Message-ID: <001801c44345$e41fa4e0$0900a8c0@168.0.1.sion> From: "Javier" To: Date: Wed, 26 May 2004 14:21:30 -0300 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1409 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1409 X-Mailman-Approved-At: Thu, 27 May 2004 04:37:59 -0700 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.1 Subject: Platforms of FreeBSD X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 May 2004 17:21:57 -0000 Hi How do you know the platforms of FreeBSD for me? Regards Javier Ramirez From owner-freebsd-arch@FreeBSD.ORG Fri May 28 07:13:47 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E7D9416A4DB for ; Fri, 28 May 2004 07:13:47 -0700 (PDT) Received: from hanoi.cronyx.ru (hanoi.cronyx.ru [144.206.181.53]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7E48743D1D for ; Fri, 28 May 2004 07:13:46 -0700 (PDT) (envelope-from rik@cronyx.ru) Received: (from root@localhost) by hanoi.cronyx.ru id i4SEB6fc046588 for arch@FreeBSD.org.checked; (8.12.8/vak/2.1) Fri, 28 May 2004 18:11:06 +0400 (MSD) (envelope-from rik@cronyx.ru) Received: from cronyx.ru (hi.cronyx.ru [144.206.181.94]) by hanoi.cronyx.ru with ESMTP id i4SE96P0046461; (8.12.8/vak/2.1) Fri, 28 May 2004 18:09:07 +0400 (MSD) (envelope-from rik@cronyx.ru) Message-ID: <40B74840.5090600@cronyx.ru> Date: Fri, 28 May 2004 18:10:08 +0400 From: Roman Kurakin User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6b) Gecko/20031208 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Robert Watson References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit cc: arch@FreeBSD.org Subject: Re: Network Stack Locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 May 2004 14:13:48 -0000 Robert Watson wrote: >SPPP Roman Kurakin >Userspace build Roman Kurakin > > //depot/users/rik/netperf/... I don't have it on the web, but I'll put it soon. Best regards, Roman Kurakin From owner-freebsd-arch@FreeBSD.ORG Fri May 28 07:18:59 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CD81816A4CE for ; Fri, 28 May 2004 07:18:59 -0700 (PDT) Received: from hanoi.cronyx.ru (hanoi.cronyx.ru [144.206.181.53]) by mx1.FreeBSD.org (Postfix) with ESMTP id 10AE743D45 for ; Fri, 28 May 2004 07:18:59 -0700 (PDT) (envelope-from rik@cronyx.ru) Received: (from root@localhost) by hanoi.cronyx.ru id i4SEEl6O046708 for arch@freebsd.org.checked; (8.12.8/vak/2.1) Fri, 28 May 2004 18:14:47 +0400 (MSD) (envelope-from rik@cronyx.ru) Received: from cronyx.ru (hi.cronyx.ru [144.206.181.94]) by hanoi.cronyx.ru with ESMTP id i4SEBxP0046616; (8.12.8/vak/2.1) Fri, 28 May 2004 18:11:59 +0400 (MSD) (envelope-from rik@cronyx.ru) Message-ID: <40B748ED.6080804@cronyx.ru> Date: Fri, 28 May 2004 18:13:01 +0400 From: Roman Kurakin User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.6b) Gecko/20031208 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Kirk McKusick References: <200405270424.i4R4OwU9024429@beastie.mckusick.com> In-Reply-To: <200405270424.i4R4OwU9024429@beastie.mckusick.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: arch@freebsd.org Subject: Re: The Design and Implementation of the FreeBSD Operating System X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 May 2004 14:18:59 -0000 Kirk McKusick wrote: >My forthcoming book, ``The Design and Implementation of the FreeBSD >Operating System'' can now be pre-ordered from Amazon.com, see >http://tinyurl.com/2bklt. Details on its coverage can be found >at http://www.mckusick.com/FreeBSDbook.html. > > Any discounts for commiters? rik > Kirk McKusick >_______________________________________________ >freebsd-arch@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-arch >To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > > > > From owner-freebsd-arch@FreeBSD.ORG Fri May 28 08:52:24 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5CFEF16A4CE for ; Fri, 28 May 2004 08:52:24 -0700 (PDT) Received: from mailhost.stack.nl (vaak.stack.nl [131.155.140.140]) by mx1.FreeBSD.org (Postfix) with ESMTP id C2AA943D3F for ; Fri, 28 May 2004 08:52:23 -0700 (PDT) (envelope-from jilles@stack.nl) Received: from turtle.stack.nl (turtle.stack.nl [IPv6:2001:610:1108:5010::132]) by mailhost.stack.nl (Postfix) with ESMTP id 7BC881F189; Fri, 28 May 2004 17:51:40 +0200 (CEST) Received: by turtle.stack.nl (Postfix, from userid 1677) id 4A3781CD8A; Fri, 28 May 2004 17:51:40 +0200 (CEST) Date: Fri, 28 May 2004 17:51:40 +0200 From: Jilles Tjoelker To: Garance A Drosihn Message-ID: <20040528155139.GB85017@stack.nl> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD 5.2.1-RELEASE-p8 i386 User-Agent: Mutt/1.5.6i cc: arch@FreeBSD.ORG Subject: Re: Change to "kludge option processing" in /bin/ps X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 May 2004 15:52:24 -0000 On Sun, May 23, 2004 at 10:58:23PM -0400, Garance A Drosihn wrote: > After staring at the kludge-option processing in `ps' for awhile, I > found enough edge-cases where it simply did not work "right" (for my > definition of "right", at keast...), so I rewrote it. As part of that, > I also removed the ancient "BACKWARD_COMPATIBILITY" compile-time > option, where: > ps -options arg1 arg2 > (with no '-' on "arg1" and "arg2") was treated as: > ps -options -N arg1 -M arg2 > I did this because I have often been puzzled/annoyed when I type: > ps 12 > to get process 12, and then realize I wanted it shown in a > different format so I type: > ps -u 12 What you should use is 'ps u12' :) That also works on old (pre-Reno) BSD ps, and it's shorter too. It doesn't work on Solaris /usr/ucb/ps, but my ps shell function takes care of that (for me). > and I get a completely different list of processes, or I type: > ps 12 34 > and I only see process 12, or I type > ps 12 34 56 > and get the error message: > ps: 56: No such file or directory > This BACKWARD_COMPATIBILITY is not documented in the usage() > or the man page, so I'd like to replace it. Note that netstat(1) has similar backward compatibility, and gdb -k uses the same order (namelist then memory). You're right that this is rather confusing. > So, I changed `ps' to check for any additional arguments after > processing all '-'-options, and if those start with a digit then > `ps' will try to use the entire argument as a pid or pidlist. > If an extra argument does not start with a digit, then `ps' just > prints an error and exits. > I want to do more testing on this before committing, but I thought > that all this was enough of a change that I should also give people > a chance to scream before I commit it. Also, I'm not sure if I > might have clobbered some subtle aspect of the kludge processing. > If anyone wants to try the update, it is available at: > http://people.freebsd.org/~gad/ps-kludge.diff > [disclaimer: at the moment it's only had about 15 minutes of > testing, but I *think* this is about the way I want it to work] > Assuming there aren't any major objections to these ideas, I plan > to do some more testing on this and commit it next weekend. Unfortunately, I can't try it out because I don't have access to a very recent -CURRENT. What's the deeper purpose of the `optlist' argument for `kludge_oldps_options' if it's always set to PS_ARGS? I have some doubts on whether `ps t' instead of `ps T' should still be supported, since `ps t p0' doesn't work because of it. You could only change 't' to 'T' when there are no more arguments. Am I right that `ps U0' is equivalent to `ps -U0' with the patch? (This wasn't really much of an issue before `-U' accepted uids instead of names.) What happens when you do `ps 1,2,3,4'? And `ps uww0,1'? Or even `ps "v1 $$"'? All this complexity makes me think of not using getopt(3), as Albert Cahalan suggested :) -- Jilles Tjoelker From owner-freebsd-arch@FreeBSD.ORG Fri May 28 09:40:49 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 63C4716A4CE for ; Fri, 28 May 2004 09:40:49 -0700 (PDT) Received: from hourri.hittite.isp.9tel.net (hourri.hittite.isp.9tel.net [62.62.156.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8220E43D1D for ; Fri, 28 May 2004 09:40:46 -0700 (PDT) (envelope-from cyrille.lefevre@laposte.net) Received: from pc2k (unknown [80.119.167.76]) by hourri.hittite.isp.9tel.net (Postfix) with SMTP id 0AF4B1575FB; Fri, 28 May 2004 19:18:54 +0200 (CEST) Message-ID: <007601c444d2$6f90ad70$7890a8c0@dyndns.org> From: "Cyrille Lefevre" To: "Roman Kurakin" , "Kirk McKusick" References: <200405270424.i4R4OwU9024429@beastie.mckusick.com> <40B748ED.6080804@cronyx.ru> Date: Fri, 28 May 2004 18:40:07 +0200 Organization: ACME MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1409 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1409 cc: arch@freebsd.org Subject: Re: The Design and Implementation of the FreeBSD Operating System X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 May 2004 16:40:49 -0000 "Roman Kurakin" wrote: > Kirk McKusick wrote: > > >My forthcoming book, ``The Design and Implementation of the FreeBSD > >Operating System'' can now be pre-ordered from Amazon.com, see > >http://tinyurl.com/2bklt. Details on its coverage can be found > >at http://www.mckusick.com/FreeBSDbook.html. > > > > > Any discounts for commiters? don't forget contributors ;^) Cyrille Lefevre. -- home: mailto:cyrille.lefevre@laposte.net From owner-freebsd-arch@FreeBSD.ORG Fri May 28 10:25:47 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5FBFA16A4CE for ; Fri, 28 May 2004 10:25:47 -0700 (PDT) Received: from smtp2.server.rpi.edu (smtp2.server.rpi.edu [128.113.2.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0596143D53 for ; Fri, 28 May 2004 10:25:47 -0700 (PDT) (envelope-from drosih@rpi.edu) Received: from [128.113.24.47] (gilead.netel.rpi.edu [128.113.24.47]) by smtp2.server.rpi.edu (8.12.8/8.12.8) with ESMTP id i4SHOXIX019310; Fri, 28 May 2004 13:24:33 -0400 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: <20040528155139.GB85017@stack.nl> References: <20040528155139.GB85017@stack.nl> Date: Fri, 28 May 2004 13:24:31 -0400 To: Jilles Tjoelker From: Garance A Drosihn Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Scanned-By: CanIt (www . canit . ca) cc: arch@FreeBSD.ORG Subject: Re: Change to "kludge option processing" in /bin/ps X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 May 2004 17:25:47 -0000 At 5:51 PM +0200 5/28/04, Jilles Tjoelker wrote: >On Sun, May 23, 2004, Garance A Drosihn wrote: > >> If anyone wants to try the update, it is available at: >> > > http://people.freebsd.org/~gad/ps-kludge.diff >> > > Assuming there aren't any major objections to these ideas, I plan > > to do some more testing on this and commit it next weekend. > >What's the deeper purpose of the `optlist' argument for >`kludge_oldps_options' if it's always set to PS_ARGS? a) After a future update, it may be set to something else. b) An earlier version of this update called the kludge- processing for all remaining arguments, and used NULL for optlist to distinguish between argument #1 and "all the rest". After a little testing I realized that that idea was simply not going to work the way I wanted it to, so I got rid of that code but left the parameter. >I have some doubts on whether `ps t' instead of `ps T' should >still be supported, since `ps t p0' doesn't work because of it. >You could only change 't' to 'T' when there are no more >arguments. Hmm, that is an interesting point. I'll check into it. I should note that I'm not going to knock myself out trying to create ideal processing for the historical '-'-less options. However I do think I might be able to improve on the 't'-kludge a little bit more. >Am I right that `ps U0' is equivalent to `ps -U0' with the patch? >(This wasn't really much of an issue before `-U' accepted uids >instead of names.) Yes, after the patch `ps U0' == `ps -U0' Before the patch, `ps U0' == ps: No user named 'p0' >What happens when you do `ps 1,2,3,4'? And `ps uww0,1'? Or even >`ps "v1 $$"'? `ps 1,2,3,4' == `ps -p 1,2,3,4' `ps uww0,1' == `ps -uww -p0,1' BUT `ps u ww0,1' == an error, since kludge processing is only done on argument #1 `ps v1 10' == `ps -p1 -p10' `ps "v1 10"' == an error, for reasons unrelated to this update >All this complexity makes me think of not using getopt(3), as >Albert Cahalan suggested :) Eh. I don't think it's all that bad, not yet at least. As long as we limit "kludge" (the "-"-less option) processing to the first argument, I think we can use getopt() without too much trouble. -- Garance Alistair Drosehn = gad@gilead.netel.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu From owner-freebsd-arch@FreeBSD.ORG Fri May 28 20:52:36 2004 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DEFED16A4CE for ; Fri, 28 May 2004 20:52:36 -0700 (PDT) Received: from beastie.mckusick.com (dsl081-247-227.sfo1.dsl.speakeasy.net [64.81.247.227]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0F00943D41 for ; Fri, 28 May 2004 20:52:35 -0700 (PDT) (envelope-from mckusick@mckusick.com) Received: from beastie.mckusick.com (localhost [127.0.0.1]) by beastie.mckusick.com (8.12.8/8.12.9) with ESMTP id i4SKtOU9028071; Fri, 28 May 2004 13:55:24 -0700 (PDT) (envelope-from mckusick@beastie.mckusick.com) Message-Id: <200405282055.i4SKtOU9028071@beastie.mckusick.com> To: Roman Kurakin In-Reply-To: Your message of "Fri, 28 May 2004 18:13:01 +0400." <40B748ED.6080804@cronyx.ru> Date: Fri, 28 May 2004 13:55:24 -0700 From: Kirk McKusick cc: arch@freebsd.org Subject: Re: The Design and Implementation of the FreeBSD Operating System X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 May 2004 03:52:37 -0000 > Date: Fri, 28 May 2004 18:13:01 +0400 > From: Roman Kurakin > To: Kirk McKusick > CC: arch@freebsd.org > Subject: Re: The Design and Implementation of the FreeBSD Operating System > X-ASK-Info: Our key was found in the mail > > Kirk McKusick wrote: > > >My forthcoming book, ``The Design and Implementation of the FreeBSD > >Operating System'' can now be pre-ordered from Amazon.com, see > >http://tinyurl.com/2bklt. Details on its coverage can be found > >at http://www.mckusick.com/FreeBSDbook.html. > > > > Kirk McKusick > > > Any discounts for commiters? > > rik I have not been successful in getting discounts in the past as this book is put out through the textbook side of Addison-Wesley. Since they have a lock on the student market - that is students generally have to buy the books for their classes - the books just sell for list price. I can get a 10% discount if I buy 50 books, but by the time I pay to ship them the savings is lost over paying list price to Amazon.com and getting free shipping. So far at least even Amazon.com is offering no discount and they have a bit more leverage than I do :-) Anyway, if I do find out about a better deal, I will let you (and the other committers) know. Kirk McKusick