From owner-freebsd-arch@FreeBSD.ORG Mon Jul 28 04:00:39 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EB22B37B401; Mon, 28 Jul 2003 04:00:38 -0700 (PDT) Received: from whale.sunbay.crimea.ua (whale.sunbay.crimea.ua [212.110.138.65]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1647943F3F; Mon, 28 Jul 2003 04:00:28 -0700 (PDT) (envelope-from ru@sunbay.com) Received: from whale.sunbay.crimea.ua (ru@localhost [127.0.0.1]) h6SB080U070509 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 28 Jul 2003 14:00:08 +0300 (EEST) (envelope-from ru@sunbay.com) Received: (from ru@localhost) by whale.sunbay.crimea.ua (8.12.9/8.12.8/Submit) id h6SB08kV070504; Mon, 28 Jul 2003 14:00:08 +0300 (EEST) (envelope-from ru) Date: Mon, 28 Jul 2003 14:00:08 +0300 From: Ruslan Ermilov To: arch@FreeBSD.org Message-ID: <20030728110008.GA63471@sunbay.com> References: <20030728084254.GA51172@rot13.obsecurity.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="8GpibOaaTibBMecb" Content-Disposition: inline In-Reply-To: <20030728084254.GA51172@rot13.obsecurity.org> User-Agent: Mutt/1.5.4i cc: Kris Kennaway Subject: Conditional evaluation in make(1) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jul 2003 11:00:39 -0000 --8GpibOaaTibBMecb Content-Type: multipart/mixed; boundary="nFreZHaLTZJo0R7j" Content-Disposition: inline --nFreZHaLTZJo0R7j Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Gang, > Script started on Mon Jul 28 11:02:58 2003 > goshik# cd /usr/ports ; make index ; cd /usr/local/etc; exit >=20 > Generating INDEX - please wait.."/usr/ports/print/pips2200/../pips800/Mak= efile", line 60: Malformed conditional (${PRTYPE} =3D=3D 780cs) > "/usr/ports/print/pips2200/../pips800/Makefile", line 63: Malformed condi= tional (${PRTYPE} =3D=3D 820ug) > "/usr/ports/print/pips2200/../pips800/Makefile", line 1: Malformed condit= ional (${PRTYPE} =3D=3D 750_2000) > "/usr/ports/print/pips2200/../pips800/Makefile", line 1: Need an operator > "/usr/ports/print/pips2200/../pips800/Makefile", line 5: Malformed condit= ional (${PRTYPE} =3D=3D 780cs) > "/usr/ports/print/pips2200/../pips800/Makefile", line 7: Malformed condit= ional (${PRTYPE} =3D=3D 820ug) > "/usr/ports/print/pips2200/../pips800/Makefile", line 305: if-less endif > "/usr/ports/print/pips2200/../pips800/Makefile", line 305: Need an operat= or > make: fatal errors encountered -- cannot continue > make_index: icemc-0.2.4: no entry for /usr/ports/x11-toolkits/qt31 > [...] > make_index: bugzilla-2.16.3_1: no entry for /usr/ports/www/p5-Template-To= olkit > Warning: Duplicate INDEX entry: *** Error code 1 > Warning: Duplicate INDEX entry: > Done. > exit >=20 > Script done on Mon Jul 28 11:13:07 2003 I've just fixed ports/print/pips800/Makefile that was broken. Its string conditional operators were broken, and just sitting hidden behind another bug in make(1) that was recently fixed. As explained in section 4.3 of the "PMake -- A Tutorial" (PSD:12-39) book, : The arithmetic and string operators may only be used to test : the value of a variable. The lefthand side must contain the : variable expansion, while the righthand side contains either : a string, enclosed in double-quotes, or a number. The stan- ^^^^^^^^^^^^^^^^^^^^^^^^^ : dard C numeric conventions (except for specifying an octal : number) apply to both sides. E.g. :=20 : #if $(OS) =3D=3D 4.3 :=20 : #if $(MACHINE) =3D=3D "sun3" :=20 : #if $(LOAD_ADDR) < 0xc000 :=20 : are all valid conditionals. Since "1.3.2" doesn't represent a valid number, the test condition against it should be written like this (with RHS enclosed in quotes): .if ${PORTVERSION} =3D=3D "1.3.2" But not like this: .if ${PORTVERSION} =3D=3D 1.3.2 which currently results in a "malformed conditional" complaint =66rom make(1), and in my opinion, for a good reason. The reason why I think conditionals of the form .if ${VAR} =3D=3D (number)(non-number) (without double quotes) should be complained about by make(1) is to avoid hiding user bugs of the form: .if ${VAR} =3D=3D 0z where "z" was put by mistake. Note that .if ${VAR} =3D=3D (all-non-number) conditionals with an unquoted RHS are still perfectly supported, and result in a string comparison being performed. The -dc option to make(1) can be used to debug conditional evalulation. I can easily "fix" our make code to restore the support for broken string comparisons with the attached patch, so it's rather a question of do we want to continue supporting this bug or not. Cheers, --=20 Ruslan Ermilov Sysadmin and DBA, ru@sunbay.com Sunbay Software Ltd, ru@FreeBSD.org FreeBSD committer --nFreZHaLTZJo0R7j Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=p Index: cond.c =================================================================== RCS file: /home/ncvs/src/usr.bin/make/cond.c,v retrieving revision 1.26 diff -u -r1.26 cond.c --- cond.c 4 Jul 2003 13:33:48 -0000 1.26 +++ cond.c 28 Jul 2003 10:47:29 -0000 @@ -688,7 +688,8 @@ } } else { char *c = CondCvtArg(rhs, &right); - if (c == rhs) + if (*c != '\0' && !isspace((unsigned char) *c) && + *c != ')') goto do_string_compare; if (rhs == condExpr) { /* --nFreZHaLTZJo0R7j-- --8GpibOaaTibBMecb Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (FreeBSD) iD8DBQE/JQI4Ukv4P6juNwoRAm6eAJ4xSF6SnNuzdKLONajdUDlVn0g13wCcD9Fk 4lJlZVSyfJb1txr+vE0SHR8= =5VtI -----END PGP SIGNATURE----- --8GpibOaaTibBMecb-- From owner-freebsd-arch@FreeBSD.ORG Mon Jul 28 14:02:30 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3E46737B401 for ; Mon, 28 Jul 2003 14:02:30 -0700 (PDT) Received: from ns2.gnf.org (ns2.gnf.org [63.196.132.68]) by mx1.FreeBSD.org (Postfix) with ESMTP id 42E7443F75 for ; Mon, 28 Jul 2003 14:02:27 -0700 (PDT) (envelope-from gtetlow@gnf.org) Received: from EXCHCLUSTER01.lj.gnf.org (exch01.lj.gnf.org [172.25.10.19]) by ns2.gnf.org (8.12.8p1/8.12.8) with ESMTP id h6SL2Moq091028 for ; Mon, 28 Jul 2003 14:02:22 -0700 (PDT) (envelope-from gtetlow@gnf.org) Received: from roark.gnf.org ([172.25.24.15]) by EXCHCLUSTER01.lj.gnf.org with Microsoft SMTPSVC(5.0.2195.5329); Mon, 28 Jul 2003 14:02:24 -0700 Received: from roark.gnf.org (localhost [127.0.0.1]) by roark.gnf.org (8.12.9/8.12.9) with ESMTP id h6SL2Oi2002746; Mon, 28 Jul 2003 14:02:24 -0700 (PDT) (envelope-from gtetlow@gnf.org) Received: (from gtetlow@localhost) by roark.gnf.org (8.12.9/8.12.9/Submit) id h6SL2OO5002745; Mon, 28 Jul 2003 14:02:24 -0700 (PDT) Date: Mon, 28 Jul 2003 14:02:23 -0700 From: Gordon Tetlow To: Garance A Drosihn Message-ID: <20030728210223.GZ12996@roark.gnf.org> References: <20030725230839.GR12996@roark.gnf.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="F3hO7rXQyZUbKHFQ" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4i X-Habeas-SWE-1: winter into spring X-Habeas-SWE-2: brightly anticipated X-Habeas-SWE-3: like Habeas SWE (tm) X-Habeas-SWE-4: Copyright 2002 Habeas (tm) X-Habeas-SWE-5: Sender Warranted Email (SWE) (tm). The sender of this X-Habeas-SWE-6: email in exchange for a license for this Habeas X-Habeas-SWE-7: warrant mark warrants that this is a Habeas Compliant X-Habeas-SWE-8: Message (HCM) and not spam. Please report use of this X-Habeas-SWE-9: mark in spam to . X-OriginalArrivalTime: 28 Jul 2003 21:02:24.0895 (UTC) FILETIME=[8B9100F0:01C3554B] cc: arch@freebsd.org Subject: Re: Patch to crunchgen to fix errors using make -P X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jul 2003 21:02:30 -0000 --F3hO7rXQyZUbKHFQ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jul 25, 2003 at 08:11:01PM -0400, Garance A Drosihn wrote: > At 4:08 PM -0700 7/25/03, Gordon Tetlow wrote: > >I have a quick patch that I'd like to commit, but I was > >wondering if anyone had any objections or reason to keep > >the MAKEFLAGS environment variable. >=20 > Following up on what you figured out, I realized that we > could also fix this by changing the line in >=20 > /usr/src/rescue/rescue/Makefile >=20 > so it's: >=20 > MAKEFLAGS=3D MAKEOBJDIRPREFIX=3D${CRUNCHOBJS} crunchgen -q -m ... >=20 > A quick test showed that this allowed me to add all the debug > flags I wanted to add, without having to change crunchgen. Good point, but I suppose the real question is whether we want to fix crunchgen in the general sense of just this one instance. I'm not opposed to either solution. If I don't hear anything, I'll commit the Makefile solution and possibly add a comment to crunchgen warning of the potential issues from the environment that crunchgen inherits. -gordon --F3hO7rXQyZUbKHFQ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (FreeBSD) iD8DBQE/JY9fRu2t9DV9ZfsRAgI5AKCfRREbO29/xOJOok7hXYc30e1kfQCg0+wU xtX1bh6wma2PR5GfjFWrJDY= =FFqr -----END PGP SIGNATURE----- --F3hO7rXQyZUbKHFQ-- From owner-freebsd-arch@FreeBSD.ORG Mon Jul 28 15:36:02 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 099A837B401 for ; Mon, 28 Jul 2003 15:36:02 -0700 (PDT) Received: from smtp1.server.rpi.edu (smtp1.server.rpi.edu [128.113.2.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4435743FBD for ; Mon, 28 Jul 2003 15:36:01 -0700 (PDT) (envelope-from drosih@rpi.edu) Received: from [128.113.24.47] (gilead.netel.rpi.edu [128.113.24.47]) by smtp1.server.rpi.edu (8.12.9/8.12.9) with ESMTP id h6SMZx9t001312; Mon, 28 Jul 2003 18:35:59 -0400 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: <20030728210223.GZ12996@roark.gnf.org> References: <20030725230839.GR12996@roark.gnf.org> <20030728210223.GZ12996@roark.gnf.org> Date: Mon, 28 Jul 2003 18:35:58 -0400 To: Gordon Tetlow From: Garance A Drosihn Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Scanned-By: CanIt (www . canit . ca) cc: arch@freebsd.org Subject: Re: Patch to crunchgen to fix errors using make -P X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jul 2003 22:36:02 -0000 At 2:02 PM -0700 7/28/03, Gordon Tetlow wrote: >On Fri, Jul 25, 2003, Garance A Drosihn wrote: > > >> Following up on what you figured out, I realized that we >> could also fix this by changing the line in >> >> /usr/src/rescue/rescue/Makefile >> >> so it's: >> >> MAKEFLAGS= MAKEOBJDIRPREFIX=${CRUNCHOBJS} crunchgen -q -m ... >> >> A quick test showed that this allowed me to add all the debug >> flags I wanted to add, without having to change crunchgen. > >Good point, but I suppose the real question is whether we want to >fix crunchgen in the general sense -- or just this one instance. >I'm not opposed to either solution. I am playing around with some make options that I *would* want to pass through crunchgen (if I can get them to work right). By having only the change in the rescue/Makefile, it would be easy to get that to happen when I want it to. I don't feel too strongly about it, but this just "feels" more flexible. >If I don't hear anything, I'll commit the Makefile solution and >possibly add a comment to crunchgen warning of the potential >issues from the environment that crunchgen inherits. It would probably be a good idea to put a short paragraph in the 'CAVEATS' section of the man page. Someday maybe we could change crunchgen so it knew what output lines it could ignore, but there's no need to worry about that right now. -- Garance Alistair Drosehn = gad@gilead.netel.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu From owner-freebsd-arch@FreeBSD.ORG Tue Jul 29 13:43:47 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0312D37B401; Tue, 29 Jul 2003 13:43:47 -0700 (PDT) Received: from www.ambrisko.com (adsl-64-174-51-42.dsl.snfc21.pacbell.net [64.174.51.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id 44D5F43F3F; Tue, 29 Jul 2003 13:43:46 -0700 (PDT) (envelope-from ambrisko@www.ambrisko.com) Received: from www.ambrisko.com (localhost [127.0.0.1]) by www.ambrisko.com (8.12.8p1/8.12.8) with ESMTP id h6TKhBO7078196; Tue, 29 Jul 2003 13:43:11 -0700 (PDT) (envelope-from ambrisko@www.ambrisko.com) Received: (from ambrisko@localhost) by www.ambrisko.com (8.12.8p1/8.12.8/Submit) id h6TKhBIh078195; Tue, 29 Jul 2003 13:43:11 -0700 (PDT) (envelope-from ambrisko) From: Doug Ambrisko Message-Id: <200307292043.h6TKhBIh078195@www.ambrisko.com> In-Reply-To: To: Julian Elischer Date: Tue, 29 Jul 2003 13:43:11 -0700 (PDT) X-Mailer: ELM [version 2.4ME+ PL94b (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII cc: arch@freebsd.org cc: Sean Kelly Subject: Re: Remove "options HW_WDOG"? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Jul 2003 20:43:47 -0000 Julian Elischer writes: | this code WAS used in the interjet. | We had modules that linked in and just needed somewhere to hook into.. | the hardware watchdog was held off by our software, but we needed to add | code to the core-dump routines to routinely call the watchdog hold-off | or we could never get a coredump because the watchdog would always go | off before the dump was completed. | | I doubt it is used by anyone any more but It's good that you asked.. | I did notice some people were working on the watchdog support for the | chipsets that have a watchdog in them so I guess they wil have all their | own entrypoints. Most sane watchdogs let you disable them. The ones I've implemented allow this. I then added code to the panic/debugger code to disable consmute and disable the watchdog. I know the Whistle one didn't let you do this so we had to tickle it during panics etc. Doug A. From owner-freebsd-arch@FreeBSD.ORG Tue Jul 29 14:06:27 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E943237B401 for ; Tue, 29 Jul 2003 14:06:27 -0700 (PDT) Received: from area51.slashnet.org (area51.slashnet.org [209.150.101.138]) by mx1.FreeBSD.org (Postfix) with ESMTP id 577B143FA3 for ; Tue, 29 Jul 2003 14:06:27 -0700 (PDT) (envelope-from smkelly@freebsd.org) Received: from edgemaster.zombie.org (ip68-13-71-251.om.om.cox.net [68.13.71.251]) by area51.slashnet.org (Postfix) with ESMTP id 6623849F2A; Tue, 29 Jul 2003 17:06:23 -0400 (EDT) Received: by edgemaster.zombie.org (Postfix, from userid 1001) id D7D6C3983A; Tue, 29 Jul 2003 16:06:24 -0500 (CDT) Date: Tue, 29 Jul 2003 16:06:24 -0500 From: Sean Kelly To: Doug Ambrisko Message-ID: <20030729210624.GA45102@edgemaster.zombie.org> References: <200307292043.h6TKhBIh078195@www.ambrisko.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200307292043.h6TKhBIh078195@www.ambrisko.com> User-Agent: Mutt/1.5.4i cc: arch@freebsd.org cc: Julian Elischer Subject: Re: Remove "options HW_WDOG"? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Jul 2003 21:06:28 -0000 First, I apologize for the original crosspost between -current and -arch. On Tue, Jul 29, 2003 at 01:43:11PM -0700, Doug Ambrisko wrote: > Julian Elischer writes: > | this code WAS used in the interjet. > | We had modules that linked in and just needed somewhere to hook into.. > | the hardware watchdog was held off by our software, but we needed to add > | code to the core-dump routines to routinely call the watchdog hold-off > | or we could never get a coredump because the watchdog would always go > | off before the dump was completed. > | > | I doubt it is used by anyone any more but It's good that you asked.. > | I did notice some people were working on the watchdog support for the > | chipsets that have a watchdog in them so I guess they wil have all their > | own entrypoints. My basic point is that nothing in the 5.x tree uses the HW_WDOG knob anymore. As Bruce Evans kindly pointed out, the only use of HW_WDOG is in FreeBSD 4.x in i386/isa/wd.c, pc98/pc98/wd.c, and kern/kern_shutdown.c to "tickle" the watchdog in the middle of a crashdump. The 'wd' driver has since died and gone on to a better place, and kern_shutdown.c doesn't use HW_WDOG to call a 'tickle' function anymore. Why have an option if nothing in our tree actually uses it beyond defining a tickle function type (watchdog_tickle_fn) which is never used? If somebody is actually porting a watchdog patch from 4.x to 5.x, they have enough code to change that needing to implement their own tickle option and function should be no big deal. Is there really a reason to keep this around? Can't it be purged for a better (and complete) implementation by hardware watchdog developers? > Most sane watchdogs let you disable them. The ones I've implemented > allow this. I then added code to the panic/debugger code to disable > consmute and disable the watchdog. The old HW_WDOG code also assumed that tickling was necessary during a crashdump, though it only worked for ATA disks. There was no framework for traits such as whether a watchdog can be disabled and reenabled. I just see the current HW_WDOG bits that are still hanging around as debris that needs cleaned up since it is no longer working or used by anything in the source tree. -- Sean Kelly | PGP KeyID: D2E5E296 smkelly@FreeBSD.org | http://www.sean-kelly.org/ From owner-freebsd-arch@FreeBSD.ORG Tue Jul 29 16:04:23 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 826E537B401; Tue, 29 Jul 2003 16:04:23 -0700 (PDT) Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by mx1.FreeBSD.org (Postfix) with ESMTP id 15D5243F3F; Tue, 29 Jul 2003 16:04:23 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([12.233.125.100]) by attbi.com (rwcrmhc13) with ESMTP id <2003072923042201500f14j7e>; Tue, 29 Jul 2003 23:04:22 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id QAA16424; Tue, 29 Jul 2003 16:04:17 -0700 (PDT) Date: Tue, 29 Jul 2003 16:04:15 -0700 (PDT) From: Julian Elischer To: Sean Kelly In-Reply-To: <20030729210624.GA45102@edgemaster.zombie.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org Subject: Re: Remove "options HW_WDOG"? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Jul 2003 23:04:23 -0000 On Tue, 29 Jul 2003, Sean Kelly wrote: > First, I apologize for the original crosspost between -current and -arch. > > > I just see the current HW_WDOG bits that are still hanging around as debris > that needs cleaned up since it is no longer working or used by > anything in the source tree. hey, we are agreeing with you.. :-) From owner-freebsd-arch@FreeBSD.ORG Wed Jul 30 13:51:28 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1D77A37B404 for ; Wed, 30 Jul 2003 13:51:28 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id E559143FB1 for ; Wed, 30 Jul 2003 13:51:25 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.9/8.12.9) with ESMTP id h6UKoSai053027 for ; Wed, 30 Jul 2003 16:50:28 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)h6UKoSS1053024 for ; Wed, 30 Jul 2003 16:50:28 -0400 (EDT) Date: Wed, 30 Jul 2003 16:50:28 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: arch@FreeBSD.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: Breaking out kern_mac.c into multiple files X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jul 2003 20:51:28 -0000 As the scope of the MAC Framework has grown, so has kern_mac.c. It's reached the point where breaking it into separate files would make it a lot easier to read, by virtue of logically grouping its exposed functions, APIs, etc. Similarly scoped extension frameworks, such as NetGraph and GEOM, have opted to go into sys/$framework, with files named similarly. My leaning was to do something similar -- add sys/mac, and then have mac_framework.c, mac_net.c, mac_sysvipc.c, etc. I probably won't get to this for a bit because I want to avoid introducing large numbers of conflicts for our outstanding changes, but I was going to poll for general interest in placement, naming, etc. Some of the other choices would be to keep it in kern/, but rename (similar to the System V IPC bits, VFS bits, et al). Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Network Associates Laboratories From owner-freebsd-arch@FreeBSD.ORG Wed Jul 30 14:04:42 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6958337B405 for ; Wed, 30 Jul 2003 14:04:42 -0700 (PDT) Received: from mail.speakeasy.net (mail9.speakeasy.net [216.254.0.209]) by mx1.FreeBSD.org (Postfix) with ESMTP id EB67643F3F for ; Wed, 30 Jul 2003 14:04:40 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Received: (qmail 25144 invoked from network); 30 Jul 2003 21:04:40 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender )encrypted SMTP for ; 30 Jul 2003 21:04:40 -0000 Received: from laptop.baldwin.cx (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.9/8.12.9) with ESMTP id h6UL4cGI068928; Wed, 30 Jul 2003 17:04:38 -0400 (EDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.4 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Wed, 30 Jul 2003 17:04:59 -0400 (EDT) From: John Baldwin To: Robert Watson cc: arch@FreeBSD.org Subject: RE: Breaking out kern_mac.c into multiple files X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jul 2003 21:04:42 -0000 On 30-Jul-2003 Robert Watson wrote: > > As the scope of the MAC Framework has grown, so has kern_mac.c. It's > reached the point where breaking it into separate files would make it a > lot easier to read, by virtue of logically grouping its exposed functions, > APIs, etc. Similarly scoped extension frameworks, such as NetGraph and > GEOM, have opted to go into sys/$framework, with files named similarly. > My leaning was to do something similar -- add sys/mac, and then have > mac_framework.c, mac_net.c, mac_sysvipc.c, etc. I probably won't get to > this for a bit because I want to avoid introducing large numbers of > conflicts for our outstanding changes, but I was going to poll for general > interest in placement, naming, etc. Some of the other choices would be to > keep it in kern/, but rename (similar to the System V IPC bits, VFS bits, > et al). sys/security/mac please. We already have a sys/security directory, let's use it. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ From owner-freebsd-arch@FreeBSD.ORG Wed Jul 30 14:08:47 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A014737B401 for ; Wed, 30 Jul 2003 14:08:47 -0700 (PDT) Received: from mail.speakeasy.net (mail14.speakeasy.net [216.254.0.214]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1119643FA3 for ; Wed, 30 Jul 2003 14:08:47 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Received: (qmail 19227 invoked from network); 30 Jul 2003 21:08:46 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender )encrypted SMTP for ; 30 Jul 2003 21:08:46 -0000 Received: from laptop.baldwin.cx (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.9/8.12.9) with ESMTP id h6UL8iGI068948 for ; Wed, 30 Jul 2003 17:08:45 -0400 (EDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.4 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 Date: Wed, 30 Jul 2003 17:09:05 -0400 (EDT) From: John Baldwin To: arch@FreeBSD.org Subject: Make long ddb not suck X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jul 2003 21:08:47 -0000 I have a patch that adds a simple paging facility to ddb at the db_printf() level using a one-shot callback mechanism. It includes a simple paging callback that rearms itself based on the users input (space does another page, enter another line). I've used this facility to replace the hand-rolled paging in 'ps', 'show pci', and 'show ktr'. The patch is at http://www.FreeBSD.org/~jhb/patches/ddb.patch Comments? Also, I notice that we have a 'show threads' command commented out from the original Mach sources. I think we should change 'ps' back to just showing simple process info (and hopefully back to 80 cols) and only print thread info for 'show threads'. Maybe show threads should take a PID as the argument? Thus, one would have: db> ps 1 blah blah sleeping on "foo" 2 blah blah threaded db> show threads 2 0 blah blah sleeping on "bar" 1 blah blah running on cpuX etc. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ From owner-freebsd-arch@FreeBSD.ORG Wed Jul 30 14:33:06 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8A79E37B408; Wed, 30 Jul 2003 14:33:06 -0700 (PDT) Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by mx1.FreeBSD.org (Postfix) with ESMTP id A7E7843FBD; Wed, 30 Jul 2003 14:33:05 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([12.233.125.100]) by attbi.com (rwcrmhc13) with ESMTP id <2003073021330401500c7a8re>; Wed, 30 Jul 2003 21:33:04 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id OAA25060; Wed, 30 Jul 2003 14:33:03 -0700 (PDT) Date: Wed, 30 Jul 2003 14:33:01 -0700 (PDT) From: Julian Elischer To: John Baldwin In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@FreeBSD.org Subject: Re: Make long ddb not suck X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jul 2003 21:33:06 -0000 On Wed, 30 Jul 2003, John Baldwin wrote: > I have a patch that adds a simple paging facility to ddb at the > db_printf() level using a one-shot callback mechanism. It includes > a simple paging callback that rearms itself based on the users > input (space does another page, enter another line). I've used this > facility to replace the hand-rolled paging in 'ps', 'show pci', and > 'show ktr'. The patch is at http://www.FreeBSD.org/~jhb/patches/ddb.patch > > Comments? > > Also, I notice that we have a 'show threads' command commented out from > the original Mach sources. I think we should change 'ps' back to just > showing simple process info (and hopefully back to 80 cols) and only > print thread info for 'show threads'. Maybe show threads should take a > PID as the argument? we do have "show thread (addr)" that shows the stacktrace. "show threads {pid}" would be good to identify the address of the thread to examine.. > > Thus, one would have: > > db> ps > 1 blah blah sleeping on "foo" > 2 blah blah threaded > > db> show threads 2 > 0 blah blah sleeping on "bar" > 1 blah blah running on cpuX > currently ps shows the threads by default.. either way is ok I guess. > etc. > > -- > > John Baldwin <>< http://www.FreeBSD.org/~jhb/ > "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Wed Jul 30 14:38:37 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D964B37B401; Wed, 30 Jul 2003 14:38:37 -0700 (PDT) Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by mx1.FreeBSD.org (Postfix) with ESMTP id B250143F75; Wed, 30 Jul 2003 14:38:36 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([12.233.125.100]) by attbi.com (rwcrmhc13) with ESMTP id <2003073021383601500o5lg2e>; Wed, 30 Jul 2003 21:38:36 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id OAA25117; Wed, 30 Jul 2003 14:38:34 -0700 (PDT) Date: Wed, 30 Jul 2003 14:38:31 -0700 (PDT) From: Julian Elischer To: John Baldwin In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@FreeBSD.org Subject: Re: Make long ddb not suck X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jul 2003 21:38:38 -0000 On Wed, 30 Jul 2003, Julian Elischer wrote: > > > On Wed, 30 Jul 2003, John Baldwin wrote: > > > I have a patch that adds a simple paging facility to ddb at the > > db_printf() level using a one-shot callback mechanism. It includes > > a simple paging callback that rearms itself based on the users > > input (space does another page, enter another line). I've used this > > facility to replace the hand-rolled paging in 'ps', 'show pci', and > > 'show ktr'. The patch is at http://www.FreeBSD.org/~jhb/patches/ddb.patch > > > > Comments? p.s. I saw your changes in p4.. I liked them. > > > > Also, I notice that we have a 'show threads' command commented out from > > the original Mach sources. I think we should change 'ps' back to just > > showing simple process info (and hopefully back to 80 cols) and only > > print thread info for 'show threads'. Maybe show threads should take a > > PID as the argument? > > > we do have "show thread (addr)" > that shows the stacktrace. > > "show threads {pid}" > would be good to identify the address > of the thread to examine.. > > > > > Thus, one would have: > > > > db> ps > > 1 blah blah sleeping on "foo" > > 2 blah blah threaded > > > > db> show threads 2 > > 0 blah blah sleeping on "bar" > > 1 blah blah running on cpuX > > > > currently ps shows the threads by default.. > either way is ok I guess. > > > etc. > > > > -- > > > > John Baldwin <>< http://www.FreeBSD.org/~jhb/ > > "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ > > _______________________________________________ > > freebsd-arch@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > > > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Wed Jul 30 14:43:30 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6BD0937B401; Wed, 30 Jul 2003 14:43:30 -0700 (PDT) Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by mx1.FreeBSD.org (Postfix) with ESMTP id DC71443F93; Wed, 30 Jul 2003 14:43:29 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([12.233.125.100]) by attbi.com (rwcrmhc13) with ESMTP id <2003073021432801500o58ple>; Wed, 30 Jul 2003 21:43:28 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id OAA25157; Wed, 30 Jul 2003 14:43:27 -0700 (PDT) Date: Wed, 30 Jul 2003 14:43:25 -0700 (PDT) From: Julian Elischer To: John Baldwin In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@FreeBSD.org Subject: Re: Make long ddb not suck X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jul 2003 21:43:30 -0000 answering to myself again.... The main use I have for 'ps' in ddb is when I get a 'hang' and I can get into ddb I use it to try idientify what everything is sleeping on.. if we make ps only show processes then I'd have to do: 'ps' "oh there are 7 KDE/Gnome threaded processess." get paper.. write down pids show threads pid xxx1 show threads pid xxx2 show threads pid xxx3 show threads pid xxx4 show threads pid xxx5 show threads pid xxx6 show threads pid xxx7 to see what everything is waiting on. which is a pain in the neck (particularly when hunting around the machine room looking for a pen and paper) maybe a switch or another command that shows all the wait channels? On Wed, 30 Jul 2003, Julian Elischer wrote: > > > On Wed, 30 Jul 2003, John Baldwin wrote: > > > I have a patch that adds a simple paging facility to ddb at the > > db_printf() level using a one-shot callback mechanism. It includes > > a simple paging callback that rearms itself based on the users > > input (space does another page, enter another line). I've used this > > facility to replace the hand-rolled paging in 'ps', 'show pci', and > > 'show ktr'. The patch is at http://www.FreeBSD.org/~jhb/patches/ddb.patch > > > > Comments? > > > > Also, I notice that we have a 'show threads' command commented out from > > the original Mach sources. I think we should change 'ps' back to just > > showing simple process info (and hopefully back to 80 cols) and only > > print thread info for 'show threads'. Maybe show threads should take a > > PID as the argument? > > > we do have "show thread (addr)" > that shows the stacktrace. > > "show threads {pid}" > would be good to identify the address > of the thread to examine.. > > > > > Thus, one would have: > > > > db> ps > > 1 blah blah sleeping on "foo" > > 2 blah blah threaded > > > > db> show threads 2 > > 0 blah blah sleeping on "bar" > > 1 blah blah running on cpuX > > > > currently ps shows the threads by default.. > either way is ok I guess. > > > etc. > > > > -- > > > > John Baldwin <>< http://www.FreeBSD.org/~jhb/ > > "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ > > _______________________________________________ > > freebsd-arch@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > > > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Wed Jul 30 16:52:24 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5EC4F37B401; Wed, 30 Jul 2003 16:52:24 -0700 (PDT) Received: from mail.cyberonic.com (mail.cyberonic.com [4.17.179.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 573BD43F85; Wed, 30 Jul 2003 16:52:23 -0700 (PDT) (envelope-from jmg@hydrogen.funkthat.com) Received: from hydrogen.funkthat.com (node-40244c0a.sfo.onnet.us.uu.net [64.36.76.10]) by mail.cyberonic.com (8.12.8/8.12.5) with ESMTP id h6V0Op0n009119; Wed, 30 Jul 2003 20:24:51 -0400 Received: (from jmg@localhost) by hydrogen.funkthat.com (8.12.9/8.11.6) id h6UNqKT7059224; Wed, 30 Jul 2003 16:52:20 -0700 (PDT) (envelope-from jmg) Date: Wed, 30 Jul 2003 16:52:20 -0700 From: John-Mark Gurney To: Robert Watson Message-ID: <20030730235220.GH10708@funkthat.com> Mail-Followup-To: Robert Watson , arch@freebsd.org References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i X-Operating-System: FreeBSD 4.2-RELEASE i386 X-PGP-Fingerprint: B7 EC EF F8 AE ED A7 31 96 7A 22 B3 D8 56 36 F4 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html cc: arch@freebsd.org Subject: Re: Breaking out kern_mac.c into multiple files X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: John-Mark Gurney List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jul 2003 23:52:24 -0000 Robert Watson wrote this message on Wed, Jul 30, 2003 at 16:50 -0400: > As the scope of the MAC Framework has grown, so has kern_mac.c. It's > reached the point where breaking it into separate files would make it a > lot easier to read, by virtue of logically grouping its exposed functions, > APIs, etc. Similarly scoped extension frameworks, such as NetGraph and > GEOM, have opted to go into sys/$framework, with files named similarly. > My leaning was to do something similar -- add sys/mac, and then have > mac_framework.c, mac_net.c, mac_sysvipc.c, etc. I probably won't get to > this for a bit because I want to avoid introducing large numbers of > conflicts for our outstanding changes, but I was going to poll for general > interest in placement, naming, etc. Some of the other choices would be to > keep it in kern/, but rename (similar to the System V IPC bits, VFS bits, > et al). Can the MAC framework be loaded as a module? If so, then sys/(security/)?/mac is my vote. If it's going to be an intregal part of the system that will be standard, then kern/ is open. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Wed Jul 30 17:05:07 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E67CB37B404 for ; Wed, 30 Jul 2003 17:05:07 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2510343FD7 for ; Wed, 30 Jul 2003 17:05:07 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.9/8.12.9) with ESMTP id h6V047ai082801; Wed, 30 Jul 2003 20:04:07 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)h6V047b7082798; Wed, 30 Jul 2003 20:04:07 -0400 (EDT) Date: Wed, 30 Jul 2003 20:04:07 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: John-Mark Gurney In-Reply-To: <20030730235220.GH10708@funkthat.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org Subject: Re: Breaking out kern_mac.c into multiple files X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2003 00:05:08 -0000 On Wed, 30 Jul 2003, John-Mark Gurney wrote: > Can the MAC framework be loaded as a module? If so, then > sys/(security/)?/mac is my vote. If it's going to be an intregal part > of the system that will be standard, then kern/ is open. Nope -- the MAC Framework permits you to plug in modules, but it cannot (itself) be made into a module. Either you got it, or you don't :-). Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Network Associates Laboratories From owner-freebsd-arch@FreeBSD.ORG Wed Jul 30 17:23:07 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 317A737B401 for ; Wed, 30 Jul 2003 17:23:07 -0700 (PDT) Received: from area51.slashnet.org (area51.slashnet.org [209.150.101.138]) by mx1.FreeBSD.org (Postfix) with ESMTP id A767D43F3F for ; Wed, 30 Jul 2003 17:23:06 -0700 (PDT) (envelope-from smkelly@FreeBSD.org) Received: from edgemaster.zombie.org (ip68-13-71-251.om.om.cox.net [68.13.71.251]) by area51.slashnet.org (Postfix) with ESMTP id 6F38A49F33 for ; Wed, 30 Jul 2003 20:23:03 -0400 (EDT) Received: by edgemaster.zombie.org (Postfix, from userid 1001) id ED0EE3983D; Wed, 30 Jul 2003 19:23:04 -0500 (CDT) Date: Wed, 30 Jul 2003 19:23:04 -0500 From: Sean Kelly To: arch@freebsd.org Message-ID: <20030731002304.GA5131@edgemaster.zombie.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.4i Subject: A working watchdog API X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2003 00:23:07 -0000 After some thought and my thread over the HW_WDOG option, I have come up with a proposal for a new watchdog API and am looking for comments and ideas from people who actually use and implement hardware watchdog drivers. First up, unless somebody says otherwise to me, I plan to commit a patch shortly to remove the HW_WDOG option from 5.x as discussed in my previous thread. You can review the patch here if you wish: http://www.sean-kelly.org/hw_wdog.diff Now that HW_WDOG is no longer functional and going away, it seems to me that a better interface needs to exist for watchdogs. In my opinion, this interface needs to be more robust, long-lasting, and require less coding for watchdog driver developers. Therefore, I propose a new watchdog type. Something along the lines of: typedef struct _watchdog_t { char name[16]; int flags; int (*enable)(int seconds); int (*disable)(); int (*tickle)(); /* Add more to me! */ } watchdog_t; Possible 'flags' could include CAN_DISABLE, to differentiate between watchdogs that can and can not be disabled. If a watchdog can't be disabled, then it will be necessary to tickle it during activities such as a crashdump instead of disabling it instead. I'm not sure you'd ever want to disable it in the first place anyway... If we wanted to support multiple watchdogs in a single system, then a simple linked list could be made of all the filled out 'watchdog_t's. The only semi-complex part would be determining the lowest necessary 'tickle' frequency to keep them all happy. Functions such as watchdog_enable(), watchdog_disable(), watchdog_tickle() would provide a standardized interface across the entire kernel source tree for maintaining watchdog state. For example: int watchdog_tickle() { /* This could be a loop through many watchdogs */ if (watchdog->tickle != NULL) (*watchdog->tickle)(); } For systems with no hardware watchdog, it would be possible to use my software watchdog code. That code could be easily wrapped up inside this API, thus putting all the watchdog family under a nice little umbrella. Comments? Ideas? Questions? Death threats? -- Sean Kelly | PGP KeyID: D2E5E296 smkelly@FreeBSD.org | http://www.sean-kelly.org/ From owner-freebsd-arch@FreeBSD.ORG Wed Jul 30 22:49:05 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3A4C137B401; Wed, 30 Jul 2003 22:49:05 -0700 (PDT) Received: from mail.cyberonic.com (mail.cyberonic.com [4.17.179.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5E5CB43F75; Wed, 30 Jul 2003 22:49:04 -0700 (PDT) (envelope-from jmg@hydrogen.funkthat.com) Received: from hydrogen.funkthat.com (node-40244c0a.sfo.onnet.us.uu.net [64.36.76.10]) by mail.cyberonic.com (8.12.8/8.12.5) with ESMTP id h6V6Lb0n032408; Thu, 31 Jul 2003 02:21:38 -0400 Received: (from jmg@localhost) by hydrogen.funkthat.com (8.12.9/8.11.6) id h6V5n22M063760; Wed, 30 Jul 2003 22:49:02 -0700 (PDT) (envelope-from jmg) Date: Wed, 30 Jul 2003 22:49:02 -0700 From: John-Mark Gurney To: Sean Kelly Message-ID: <20030731054902.GJ10708@funkthat.com> Mail-Followup-To: Sean Kelly , arch@freebsd.org References: <20030731002304.GA5131@edgemaster.zombie.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030731002304.GA5131@edgemaster.zombie.org> User-Agent: Mutt/1.4.1i X-Operating-System: FreeBSD 4.2-RELEASE i386 X-PGP-Fingerprint: B7 EC EF F8 AE ED A7 31 96 7A 22 B3 D8 56 36 F4 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html cc: arch@freebsd.org Subject: Re: A working watchdog API X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: John-Mark Gurney List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2003 05:49:05 -0000 Sean Kelly wrote this message on Wed, Jul 30, 2003 at 19:23 -0500: > watchdog_tickle() Why not pet the dog to keep it happy. If you tickle it, it might bite your hand off! (A friend of mine told me that the man page for Solaris talks about petting the dog to keep it from rebooting the box, etc.) -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Wed Jul 30 23:14:19 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B3EF937B401; Wed, 30 Jul 2003 23:14:19 -0700 (PDT) Received: from phk.freebsd.dk (phk.freebsd.dk [212.242.86.175]) by mx1.FreeBSD.org (Postfix) with ESMTP id B7EAB43FB1; Wed, 30 Jul 2003 23:14:18 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by phk.freebsd.dk (8.12.8/8.12.8) with ESMTP id h6V6EDV3001774; Thu, 31 Jul 2003 06:14:13 GMT (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h6V6EC5H007896; Thu, 31 Jul 2003 08:14:12 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: John-Mark Gurney From: "Poul-Henning Kamp" In-Reply-To: Your message of "Wed, 30 Jul 2003 22:49:02 PDT." <20030731054902.GJ10708@funkthat.com> Date: Thu, 31 Jul 2003 08:14:12 +0200 Message-ID: <7895.1059632052@critter.freebsd.dk> cc: arch@freebsd.org cc: Sean Kelly Subject: Re: A working watchdog API X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2003 06:14:20 -0000 In message <20030731054902.GJ10708@funkthat.com>, John-Mark Gurney writes: >Sean Kelly wrote this message on Wed, Jul 30, 2003 at 19:23 -0500: >> watchdog_tickle() > >Why not pet the dog to keep it happy. If you tickle it, it might bite >your hand off! > >(A friend of mine told me that the man page for Solaris talks about >petting the dog to keep it from rebooting the box, etc.) We're way ahead of you: $ grep IOC /sys/sys/watchdog.h #define WDIOCPATPAT _IOW('W', 42, u_int) * program calls WDIOCPATPAT again before the timer expires -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Thu Jul 31 01:43:46 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2B0E837B401; Thu, 31 Jul 2003 01:43:45 -0700 (PDT) Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7E92143F85; Thu, 31 Jul 2003 01:43:44 -0700 (PDT) (envelope-from bde@zeta.org.au) Received: from gamplex.bde.org (katana.zip.com.au [61.8.7.246]) by mailman.zeta.org.au (8.9.3p2/8.8.7) with ESMTP id SAA11237; Thu, 31 Jul 2003 18:43:41 +1000 Date: Thu, 31 Jul 2003 18:43:40 +1000 (EST) From: Bruce Evans X-X-Sender: bde@gamplex.bde.org To: John Baldwin In-Reply-To: Message-ID: <20030731182956.T5869@gamplex.bde.org> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org Subject: Re: Make long ddb not suck X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2003 08:43:47 -0000 > I have a patch that adds a simple paging facility to ddb at the > db_printf() level using a one-shot callback mechanism. It includes > a simple paging callback that rearms itself based on the users > input (space does another page, enter another line). I've used this > facility to replace the hand-rolled paging in 'ps', 'show pci', and > 'show ktr'. The patch is at http://www.FreeBSD.org/~jhb/patches/ddb.patch > > Comments? I like it. It also needs to have some idea of line lengths, so that 20-line pages with too-long lines don't scroll off 25-line terminals. > Also, I notice that we have a 'show threads' command commented out from > the original Mach sources. I think we should change 'ps' back to just > showing simple process info (and hopefully back to 80 cols) and only > print thread info for 'show threads'. Maybe show threads should take a > PID as the argument? I would like almost anything to make it readable again. It now mostly fits in 80 columns on i386's but is still unreadable because columns don't line up. It will never fit in 80 colums on arches with 64-bit pointers. Bruce From owner-freebsd-arch@FreeBSD.ORG Thu Jul 31 04:13:46 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4505237B401; Thu, 31 Jul 2003 04:13:46 -0700 (PDT) Received: from critter.freebsd.dk (esplanaden.cybercity.dk [212.242.40.114]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5D69D43FA3; Thu, 31 Jul 2003 04:13:45 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h6VBDh3D002923; Thu, 31 Jul 2003 13:13:43 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: Sean Kelly From: "Poul-Henning Kamp" In-Reply-To: Your message of "Wed, 30 Jul 2003 19:23:04 CDT." <20030731002304.GA5131@edgemaster.zombie.org> Date: Thu, 31 Jul 2003 13:13:43 +0200 Message-ID: <2922.1059650023@critter.freebsd.dk> cc: arch@freebsd.org Subject: Re: A working watchdog API X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2003 11:13:46 -0000 In message <20030731002304.GA5131@edgemaster.zombie.org>, Sean Kelly writes: >Comments? Ideas? Questions? Death threats? Looks good to me. I tacitly assume that (something like) the interface will remain for userland components to "take over" the patting of the watchdog. One thing we may want to consider is to write a configurable userland watchdogd process which will examine various vital statistics and pat the watchdog if everything is fine. Amongst the things I could imagine watchdogd looking at: Precense of carrier on network interfaces A certain minimum amount of trafic on an interface Ping'ability of a given IP# Being able to connect to a given tcp port. A given process still running. Being able to create, write, fsync, close and remove a given file (to see that a disk is not hung). etc. Any takes ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Thu Jul 31 09:31:16 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 28E3237B404 for ; Thu, 31 Jul 2003 09:31:16 -0700 (PDT) Received: from mail.speakeasy.net (mail16.speakeasy.net [216.254.0.216]) by mx1.FreeBSD.org (Postfix) with ESMTP id 879DB43FA3 for ; Thu, 31 Jul 2003 09:31:15 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Received: (qmail 10938 invoked from network); 31 Jul 2003 16:31:15 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender )encrypted SMTP for ; 31 Jul 2003 16:31:15 -0000 Received: from laptop.baldwin.cx (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.9/8.12.9) with ESMTP id h6VGVDGI071580; Thu, 31 Jul 2003 12:31:13 -0400 (EDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.4 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Thu, 31 Jul 2003 12:31:33 -0400 (EDT) From: John Baldwin To: Julian Elischer cc: arch@FreeBSD.org Subject: Re: Make long ddb not suck X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2003 16:31:17 -0000 On 30-Jul-2003 Julian Elischer wrote: > > > On Wed, 30 Jul 2003, John Baldwin wrote: > >> I have a patch that adds a simple paging facility to ddb at the >> db_printf() level using a one-shot callback mechanism. It includes >> a simple paging callback that rearms itself based on the users >> input (space does another page, enter another line). I've used this >> facility to replace the hand-rolled paging in 'ps', 'show pci', and >> 'show ktr'. The patch is at http://www.FreeBSD.org/~jhb/patches/ddb.patch >> >> Comments? >> >> Also, I notice that we have a 'show threads' command commented out from >> the original Mach sources. I think we should change 'ps' back to just >> showing simple process info (and hopefully back to 80 cols) and only >> print thread info for 'show threads'. Maybe show threads should take a >> PID as the argument? > > > we do have "show thread (addr)" > that shows the stacktrace. > > "show threads {pid}" > would be good to identify the address > of the thread to examine.. > >> >> Thus, one would have: >> >> db> ps >> 1 blah blah sleeping on "foo" >> 2 blah blah threaded >> >> db> show threads 2 >> 0 blah blah sleeping on "bar" >> 1 blah blah running on cpuX >> > > currently ps shows the threads by default.. > either way is ok I guess. ps is hardly readable anymore because it is so cluttered now. I would like ps to go back to something more like it was prior to KSE and then use a separate show threads when one needs info about threads within a process. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ From owner-freebsd-arch@FreeBSD.ORG Thu Jul 31 09:31:22 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4DE1937B43E for ; Thu, 31 Jul 2003 09:31:22 -0700 (PDT) Received: from mail.speakeasy.net (mail12.speakeasy.net [216.254.0.212]) by mx1.FreeBSD.org (Postfix) with ESMTP id B5BC643F75 for ; Thu, 31 Jul 2003 09:31:21 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Received: (qmail 27780 invoked from network); 31 Jul 2003 16:31:21 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender )encrypted SMTP for ; 31 Jul 2003 16:31:21 -0000 Received: from laptop.baldwin.cx (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.9/8.12.9) with ESMTP id h6VGVJGI071586; Thu, 31 Jul 2003 12:31:19 -0400 (EDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.4 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20030731182956.T5869@gamplex.bde.org> Date: Thu, 31 Jul 2003 12:31:39 -0400 (EDT) From: John Baldwin To: Bruce Evans cc: arch@freebsd.org Subject: Re: Make long ddb not suck X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2003 16:31:23 -0000 On 31-Jul-2003 Bruce Evans wrote: >> I have a patch that adds a simple paging facility to ddb at the >> db_printf() level using a one-shot callback mechanism. It includes >> a simple paging callback that rearms itself based on the users >> input (space does another page, enter another line). I've used this >> facility to replace the hand-rolled paging in 'ps', 'show pci', and >> 'show ktr'. The patch is at http://www.FreeBSD.org/~jhb/patches/ddb.patch >> >> Comments? > > I like it. It also needs to have some idea of line lengths, so that > 20-line pages with too-long lines don't scroll off 25-line terminals. That's harder. :) That might can be added in the future however. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ From owner-freebsd-arch@FreeBSD.ORG Thu Jul 31 11:26:36 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2B5FA37B401; Thu, 31 Jul 2003 11:26:36 -0700 (PDT) Received: from rwcrmhc12.comcast.net (rwcrmhc12.comcast.net [216.148.227.85]) by mx1.FreeBSD.org (Postfix) with ESMTP id C1B5B43F3F; Thu, 31 Jul 2003 11:26:34 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([12.233.125.100]) by attbi.com (rwcrmhc12) with ESMTP id <2003073118262801400aoddte>; Thu, 31 Jul 2003 18:26:28 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id LAA33114; Thu, 31 Jul 2003 11:26:24 -0700 (PDT) Date: Thu, 31 Jul 2003 11:26:23 -0700 (PDT) From: Julian Elischer To: John Baldwin In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@FreeBSD.org Subject: Re: Make long ddb not suck X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2003 18:26:36 -0000 On Thu, 31 Jul 2003, John Baldwin wrote: > > On 30-Jul-2003 Julian Elischer wrote: > > > > > > On Wed, 30 Jul 2003, John Baldwin wrote: > > > >> I have a patch that adds a simple paging facility to ddb at the > >> db_printf() level using a one-shot callback mechanism. It includes > >> a simple paging callback that rearms itself based on the users > >> input (space does another page, enter another line). I've used this > >> facility to replace the hand-rolled paging in 'ps', 'show pci', and > >> 'show ktr'. The patch is at http://www.FreeBSD.org/~jhb/patches/ddb.patch > >> > >> Comments? > >> > >> Also, I notice that we have a 'show threads' command commented out from > >> the original Mach sources. I think we should change 'ps' back to just > >> showing simple process info (and hopefully back to 80 cols) and only > >> print thread info for 'show threads'. Maybe show threads should take a > >> PID as the argument? > > > > > > we do have "show thread (addr)" > > that shows the stacktrace. > > > > "show threads {pid}" > > would be good to identify the address > > of the thread to examine.. > > > >> > >> Thus, one would have: > >> > >> db> ps > >> 1 blah blah sleeping on "foo" > >> 2 blah blah threaded > >> > >> db> show threads 2 > >> 0 blah blah sleeping on "bar" > >> 1 blah blah running on cpuX > >> > > > > currently ps shows the threads by default.. > > either way is ok I guess. > > ps is hardly readable anymore because it is so cluttered now. I would > like ps to go back to something more like it was prior to KSE and > then use a separate show threads when one needs info about threads > within a process. one thing it shows now is that there can be many "inhibitors set on a thread/process at a time.. e.g. it can be: suspended and sleeping and swapped out. presently I enumerate them with [SWP][SLP][SUSP] but having a single field S S S L I L W U C W P P S K T wmesg ---------------------- bla - - 1 - - 1 1 1 - - iowait 1 1 - - - sbwait - - - 1 - Giant - - - - 1 int2 or even, in a more compact form: SSSLI LWUCW PPSKT wmesg ---------------------- bla --S-- ZXS-- iowait ZX--- sbwait ---W- Giant ----I int2 These would take a fixed size and would show the combinatorial possibilities.. > > -- > > John Baldwin <>< http://www.FreeBSD.org/~jhb/ > "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ > From owner-freebsd-arch@FreeBSD.ORG Thu Jul 31 11:31:15 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7EED437B401; Thu, 31 Jul 2003 11:31:15 -0700 (PDT) Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by mx1.FreeBSD.org (Postfix) with ESMTP id DBF0C43FBD; Thu, 31 Jul 2003 11:31:14 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([12.233.125.100]) by attbi.com (rwcrmhc13) with ESMTP id <2003073118311401500o5pu6e>; Thu, 31 Jul 2003 18:31:14 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id LAA33151; Thu, 31 Jul 2003 11:31:06 -0700 (PDT) Date: Thu, 31 Jul 2003 11:31:05 -0700 (PDT) From: Julian Elischer To: John Baldwin In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@FreeBSD.org Subject: Re: Make long ddb not suck X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2003 18:31:15 -0000 On Thu, 31 Jul 2003, Julian Elischer wrote: > > one thing it shows now is that there can be many "inhibitors set on a > thread/process at a time.. > e.g. it can be: > suspended and sleeping and swapped out. > > presently I enumerate them with [SWP][SLP][SUSP] > > but having a single field > > S S S L I > L W U C W > P P S K T wmesg > ---------------------- > bla - - 1 - - > 1 1 1 - - iowait > 1 1 - - - sbwait > - - - 1 - Giant > - - - - 1 int2 > > > or even, in a more compact form: > SSSLI > LWUCW > PPSKT wmesg > ---------------------- > bla --S-- > ZXS-- iowait > ZX--- sbwait > ---W- Giant oops ---L- Giant > ----I int2 > > > > These would take a fixed size > and would show the combinatorial possibilities.. > From owner-freebsd-arch@FreeBSD.ORG Thu Jul 31 14:19:36 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7034937B401 for ; Thu, 31 Jul 2003 14:19:36 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7AAF543FBD for ; Thu, 31 Jul 2003 14:19:35 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h6VLJWmE000748 for ; Thu, 31 Jul 2003 23:19:33 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: arch@freebsd.org From: Poul-Henning Kamp Date: Thu, 31 Jul 2003 23:19:31 +0200 Message-ID: <747.1059686371@critter.freebsd.dk> Subject: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2003 21:19:36 -0000 I am in the process of changing the on-disk layout policy used by the swap_pager. Currently we have a compiled in max number of swap devices, and the virtual "swapdevice" is striped over these. People who do not adjust NSWAPDEV to match their configuration spend a lot of RAM on holding the radix bitmap which is 3/4 empty: critter phk> sysctl kern.malloc | grep -i swap SWAP 2 753K 753K 2 64 critter phk> pstat -s Device 1K-blocks Used Avail Capacity /dev/ad0s1b 1321312 0 1321312 0% This is because the radix bitmap gets allocated for the full stripe width, but only one quarter of it is actually used. Another thing is that striping does not belong in the swap_pager in the first place, we have CCD and similar pieces of code for that. The new layout will allocate the devices sequentially in the "virtual swapdevice" and allocation will (for now) be round robin between the devices. Later I would like to implement load based device selection (ie: pick the device with the lowest recent average access time). If disk striping is necessary for the disks used for swap-space, do it with ccd(4) and call swapon on that. This change will also remove the compiled in limit on the number of swapdevices, and may also increase the upper size limit for each swap device by a factor four. Casualties of this will the ability to get swap information out of kvm images. If this, contrary to current evidence is very important to us, we can reimplement it later. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Thu Jul 31 16:09:18 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8AFA937B401 for ; Thu, 31 Jul 2003 16:09:18 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id BFF1843F75 for ; Thu, 31 Jul 2003 16:09:17 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.9/8.12.9) with ESMTP id h6VN8Hai086895; Thu, 31 Jul 2003 19:08:17 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)h6VN8HRC086892; Thu, 31 Jul 2003 19:08:17 -0400 (EDT) Date: Thu, 31 Jul 2003 19:08:17 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Poul-Henning Kamp In-Reply-To: <747.1059686371@critter.freebsd.dk> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2003 23:09:18 -0000 On Thu, 31 Jul 2003, Poul-Henning Kamp wrote: > I am in the process of changing the on-disk layout policy used by the > swap_pager. Do you anticipate any performance changes as a result of this change? > Casualties of this will the ability to get swap information out of kvm > images. If this, contrary to current evidence is very important to us, > we can reimplement it later. I.e., the ability to do pstat -s -M /var/crash/foo -N /boot/kernel/kernel Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Network Associates Laboratories From owner-freebsd-arch@FreeBSD.ORG Fri Aug 1 00:06:22 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1973537B401; Fri, 1 Aug 2003 00:06:22 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id E3DB943FA3; Fri, 1 Aug 2003 00:06:20 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h7176ImE005894; Fri, 1 Aug 2003 09:06:19 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: Robert Watson From: "Poul-Henning Kamp" In-Reply-To: Your message of "Thu, 31 Jul 2003 19:08:17 EDT." Date: Fri, 01 Aug 2003 09:06:18 +0200 Message-ID: <5893.1059721578@critter.freebsd.dk> cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2003 07:06:22 -0000 In message , Robe rt Watson writes: > >On Thu, 31 Jul 2003, Poul-Henning Kamp wrote: > >> I am in the process of changing the on-disk layout policy used by the >> swap_pager. > >Do you anticipate any performance changes as a result of this change? Anticipate ? No. Be surprised by improvements ? No. The striping code limits the I/O size for paging activity to a small size. By laying out each disk sequentially in the "swap device" we can increase that size to what our I/O system can lift. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Fri Aug 1 01:23:46 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 24E6937B401 for ; Fri, 1 Aug 2003 01:23:46 -0700 (PDT) Received: from HAL9000.homeunix.com (ip114.bella-vista.sfo.interquest.net [66.199.86.114]) by mx1.FreeBSD.org (Postfix) with ESMTP id 49EF843FDD for ; Fri, 1 Aug 2003 01:23:44 -0700 (PDT) (envelope-from das@FreeBSD.ORG) Received: from HAL9000.homeunix.com (localhost [127.0.0.1]) by HAL9000.homeunix.com (8.12.9/8.12.9) with ESMTP id h718NcBG075867; Fri, 1 Aug 2003 01:23:38 -0700 (PDT) (envelope-from das@FreeBSD.ORG) Received: (from das@localhost) by HAL9000.homeunix.com (8.12.9/8.12.9/Submit) id h718Nava075866; Fri, 1 Aug 2003 01:23:36 -0700 (PDT) (envelope-from das@FreeBSD.ORG) Date: Fri, 1 Aug 2003 01:23:36 -0700 From: David Schultz To: Poul-Henning Kamp Message-ID: <20030801082336.GA75619@HAL9000.homeunix.com> Mail-Followup-To: Poul-Henning Kamp , arch@freebsd.org References: <747.1059686371@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <747.1059686371@critter.freebsd.dk> cc: arch@FreeBSD.ORG Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2003 08:23:46 -0000 On Thu, Jul 31, 2003, Poul-Henning Kamp wrote: > People who do not adjust NSWAPDEV to match their configuration spend > a lot of RAM on holding the radix bitmap which is 3/4 empty: > > critter phk> sysctl kern.malloc | grep -i swap > SWAP 2 753K 753K 2 64 > critter phk> pstat -s > Device 1K-blocks Used Avail Capacity > /dev/ad0s1b 1321312 0 1321312 0% > > This is because the radix bitmap gets allocated for the full > stripe width, but only one quarter of it is actually used. Static striping has many annoying limitations, but the memory footprint isn't so significant. If people cared enough, we could save a factor of 2 by fixing the bugs that were introduced when daddr_t became 64 bits, and a factor of 8 by tracking swap space in PAGE_SIZEd quantities instead of DEV_BSIZEd quantities. By comparison, more clever striping strategies probably have a higher memory footprint, but have better flexibility and performance. But since you just ripped out the debugging stuff for the radix tree bitmap, I'm guessing that your plan is to replace the bitmap with something else within the next few days anyway... > Another thing is that striping does not belong in the swap_pager in > the first place, we have CCD and similar pieces of code for that. > > > The new layout will allocate the devices sequentially in the "virtual > swapdevice" and allocation will (for now) be round robin between > the devices. Later I would like to implement load based device > selection (ie: pick the device with the lowest recent average access > time). > > If disk striping is necessary for the disks used for swap-space, > do it with ccd(4) and call swapon on that. Can you please describe how the interface between ccd(4) and the swap subsystem will work? I like this idea, but it isn't immediately obvious how you plan to implement striping based on load and dynamic addition and removal of swap space. From owner-freebsd-arch@FreeBSD.ORG Fri Aug 1 01:28:45 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 06DC137B401; Fri, 1 Aug 2003 01:28:45 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id D98C343F93; Fri, 1 Aug 2003 01:28:43 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h718SfmE006613; Fri, 1 Aug 2003 10:28:42 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: David Schultz From: "Poul-Henning Kamp" In-Reply-To: Your message of "Fri, 01 Aug 2003 01:23:36 PDT." <20030801082336.GA75619@HAL9000.homeunix.com> Date: Fri, 01 Aug 2003 10:28:41 +0200 Message-ID: <6612.1059726521@critter.freebsd.dk> cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2003 08:28:45 -0000 In message <20030801082336.GA75619@HAL9000.homeunix.com>, David Schultz writes: >But since you just ripped out the debugging stuff for the radix >tree bitmap, I'm guessing that your plan is to replace the bitmap >with something else within the next few days anyway... I actually do not plan on replacing the radix code in this round. I want to see how it performs before I vote it out. I don't currently see any resource managers in the kernel which obviously would do a better job. >Can you please describe how the interface between ccd(4) and the >swap subsystem will work? I like this idea, but it isn't >immediately obvious how you plan to implement striping based on >load and dynamic addition and removal of swap space. ccdconfig -c ccd0 128 0 /dev/da0b /dev/da1b /dev/da2b swapon /dev/ccd0 -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Fri Aug 1 01:36:36 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6AEB637B401 for ; Fri, 1 Aug 2003 01:36:36 -0700 (PDT) Received: from skywalker.creative.net.au (skywalker.creative.net.au [203.56.168.1]) by mx1.FreeBSD.org (Postfix) with SMTP id B1CA543FBD for ; Fri, 1 Aug 2003 01:36:34 -0700 (PDT) (envelope-from freebsd@skywalker.creative.net.au) Received: (qmail 64488 invoked by uid 1008); 1 Aug 2003 08:36:28 -0000 Date: Fri, 1 Aug 2003 16:36:28 +0800 From: Adrian Chadd To: Poul-Henning Kamp Message-ID: <20030801083628.GB56492@skywalker.creative.net.au> References: <20030801082336.GA75619@HAL9000.homeunix.com> <6612.1059726521@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <6612.1059726521@critter.freebsd.dk> User-Agent: Mutt/1.4i cc: arch@freebsd.org cc: David Schultz Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2003 08:36:36 -0000 On Fri, Aug 01, 2003, Poul-Henning Kamp wrote: > >Can you please describe how the interface between ccd(4) and the > >swap subsystem will work? I like this idea, but it isn't > >immediately obvious how you plan to implement striping based on > >load and dynamic addition and removal of swap space. > > ccdconfig -c ccd0 128 0 /dev/da0b /dev/da1b /dev/da2b > swapon /dev/ccd0 .. and if I want to add another swap device, will that work? Adrian -- Adrian Chadd learning is bad it just makes the people around you dumber (angryskul == alfred@irc) :( From owner-freebsd-arch@FreeBSD.ORG Fri Aug 1 01:44:24 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B7A2537B401 for ; Fri, 1 Aug 2003 01:44:24 -0700 (PDT) Received: from HAL9000.homeunix.com (ip114.bella-vista.sfo.interquest.net [66.199.86.114]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4950743FE9 for ; Fri, 1 Aug 2003 01:44:23 -0700 (PDT) (envelope-from das@freebsd.org) Received: from HAL9000.homeunix.com (localhost [127.0.0.1]) by HAL9000.homeunix.com (8.12.9/8.12.9) with ESMTP id h718iHBG076002; Fri, 1 Aug 2003 01:44:18 -0700 (PDT) (envelope-from das@freebsd.org) Received: (from das@localhost) by HAL9000.homeunix.com (8.12.9/8.12.9/Submit) id h718iF5j076001; Fri, 1 Aug 2003 01:44:15 -0700 (PDT) (envelope-from das@freebsd.org) Date: Fri, 1 Aug 2003 01:44:15 -0700 From: David Schultz To: Poul-Henning Kamp Message-ID: <20030801084415.GA75930@HAL9000.homeunix.com> Mail-Followup-To: Poul-Henning Kamp , arch@freebsd.org References: <20030801082336.GA75619@HAL9000.homeunix.com> <6612.1059726521@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <6612.1059726521@critter.freebsd.dk> cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2003 08:44:25 -0000 On Fri, Aug 01, 2003, Poul-Henning Kamp wrote: > In message <20030801082336.GA75619@HAL9000.homeunix.com>, David Schultz writes: > > >But since you just ripped out the debugging stuff for the radix > >tree bitmap, I'm guessing that your plan is to replace the bitmap > >with something else within the next few days anyway... > > I actually do not plan on replacing the radix code in this round. > I want to see how it performs before I vote it out. I don't currently > see any resource managers in the kernel which obviously would do > a better job. So why has the debugging support for the radix tree bitmaps been removed, then? > >Can you please describe how the interface between ccd(4) and the > >swap subsystem will work? I like this idea, but it isn't > >immediately obvious how you plan to implement striping based on > >load and dynamic addition and removal of swap space. > > ccdconfig -c ccd0 128 0 /dev/da0b /dev/da1b /dev/da2b > swapon /dev/ccd0 That's what I thought, but the swap subsystem just sees one big virtual disk, and I now add /dev/da3b, how will the new space be interleaved with the existing space? How is it possible to support anything other than fixed interleave in this model? From owner-freebsd-arch@FreeBSD.ORG Fri Aug 1 01:47:52 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C493137B405 for ; Fri, 1 Aug 2003 01:47:52 -0700 (PDT) Received: from cirb503493.alcatel.com.au (c211-28-27-130.belrs2.nsw.optusnet.com.au [211.28.27.130]) by mx1.FreeBSD.org (Postfix) with ESMTP id 983C943F93 for ; Fri, 1 Aug 2003 01:47:50 -0700 (PDT) (envelope-from PeterJeremy@optushome.com.au) Received: from cirb503493.alcatel.com.au (localhost.alcatel.com.au [127.0.0.1])h718lhgh044801; Fri, 1 Aug 2003 18:47:43 +1000 (EST) (envelope-from jeremyp@cirb503493.alcatel.com.au) Received: (from jeremyp@localhost) by cirb503493.alcatel.com.au (8.12.8/8.12.8/Submit) id h718lcCg044800; Fri, 1 Aug 2003 18:47:38 +1000 (EST) Date: Fri, 1 Aug 2003 18:47:38 +1000 From: Peter Jeremy To: Poul-Henning Kamp Message-ID: <20030801084737.GA44755@cirb503493.alcatel.com.au> References: <747.1059686371@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <747.1059686371@critter.freebsd.dk> User-Agent: Mutt/1.4.1i cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2003 08:47:53 -0000 On Thu, Jul 31, 2003 at 11:19:31PM +0200, Poul-Henning Kamp wrote: >Another thing is that striping does not belong in the swap_pager in >the first place, we have CCD and similar pieces of code for that. Can (does) the swap striping code gain any performance advantage by knowing more about the paging system I/O behaviour than a general purpose striping tool would? >The new layout will allocate the devices sequentially in the "virtual >swapdevice" and allocation will (for now) be round robin between >the devices. Later I would like to implement load based device >selection (ie: pick the device with the lowest recent average access >time). My gut feeling is that the paging allocation algorithm has room for improvement[1]. I'm not sure that allocating based on disk busyness is necessarily the right direction. AFAIK, FreeBSD frees the backing swap page if the page in memory is modified. This means that a backing page is written once and read probably a fairly small number of times. The allocation algorithm needs to optimise performance across both the write and subsequent read(s). Measuring disk busyness over a short period will tend to optimise the write allocation but have little impact on the reads. Using a longer period runs the risk that paging spikes can saturate a disk in the short term. >If disk striping is necessary for the disks used for swap-space, >do it with ccd(4) and call swapon on that. Is the intent to have the paging subsystem spread page allocations across the available space or preferentially allocate from one end? In the former case, I don't believe striping has any advantages over concatenation. (Though I believe the latter may improve paging performance in the case where only a small part of the swap space is in use - which is probably the most common case). Peter [1] Based on using X, Mozilla and OpenOffice simultaneously on a system with only 128MB RAM. From owner-freebsd-arch@FreeBSD.ORG Fri Aug 1 02:03:23 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1ED6F37B401 for ; Fri, 1 Aug 2003 02:03:23 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 12E4543F93 for ; Fri, 1 Aug 2003 02:03:22 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h7193JmE006956; Fri, 1 Aug 2003 11:03:20 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: Peter Jeremy From: "Poul-Henning Kamp" In-Reply-To: Your message of "Fri, 01 Aug 2003 18:47:38 +1000." <20030801084737.GA44755@cirb503493.alcatel.com.au> Date: Fri, 01 Aug 2003 11:03:19 +0200 Message-ID: <6955.1059728599@critter.freebsd.dk> cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2003 09:03:23 -0000 In message <20030801084737.GA44755@cirb503493.alcatel.com.au>, Peter Jeremy wri tes: >On Thu, Jul 31, 2003 at 11:19:31PM +0200, Poul-Henning Kamp wrote: >>Another thing is that striping does not belong in the swap_pager in >>the first place, we have CCD and similar pieces of code for that. > >Can (does) the swap striping code gain any performance advantage by >knowing more about the paging system I/O behaviour than a general >purpose striping tool would? no. >>The new layout will allocate the devices sequentially in the "virtual >>swapdevice" and allocation will (for now) be round robin between >>the devices. Later I would like to implement load based device >>selection (ie: pick the device with the lowest recent average access >>time). > >My gut feeling is that the paging allocation algorithm has room for >improvement[1]. I'm not sure that allocating based on disk busyness >is necessarily the right direction. Neither am I, I'm not even sure how relevant it is to optimize this in the first place, so that's why I plan to do the simple "round robin" as a first cut. >AFAIK, FreeBSD frees the backing swap page if the page in memory is >modified. This means that a backing page is written once and read >probably a fairly small number of times. The allocation algorithm >needs to optimise performance across both the write and subsequent >read(s). Measuring disk busyness over a short period will tend to >optimise the write allocation but have little impact on the reads. >Using a longer period runs the risk that paging spikes can saturate >a disk in the short term. The thing you overlook is that often when things gets paged out, the system is short on memory and therefore more likely to not do anything productive, whereas when things gets paged in, there are a better chance of some other process being able to use the CPU time productively. If we did predictive pageouts like some of the "serious" mainfram OS's this would be less true. >>If disk striping is necessary for the disks used for swap-space, >>do it with ccd(4) and call swapon on that. > >Is the intent to have the paging subsystem spread page allocations >across the available space or preferentially allocate from one end? >In the former case, I don't believe striping has any advantages over >concatenation. (Though I believe the latter may improve paging >performance in the case where only a small part of the swap space is >in use - which is probably the most common case). Pulling the internal striping out of the swapper will allow people to actually experiment with this... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Fri Aug 1 02:18:41 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 27EB537B401; Fri, 1 Aug 2003 02:18:41 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2FFED43F3F; Fri, 1 Aug 2003 02:18:40 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h719IdmE007148; Fri, 1 Aug 2003 11:18:39 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: Adrian Chadd From: "Poul-Henning Kamp" In-Reply-To: Your message of "Fri, 01 Aug 2003 16:36:28 +0800." <20030801083628.GB56492@skywalker.creative.net.au> Date: Fri, 01 Aug 2003 11:18:39 +0200 Message-ID: <7147.1059729519@critter.freebsd.dk> cc: arch@freebsd.org cc: David Schultz Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2003 09:18:41 -0000 In message <20030801083628.GB56492@skywalker.creative.net.au>, Adrian Chadd wri tes: >On Fri, Aug 01, 2003, Poul-Henning Kamp wrote: > >> >Can you please describe how the interface between ccd(4) and the >> >swap subsystem will work? I like this idea, but it isn't >> >immediately obvious how you plan to implement striping based on >> >load and dynamic addition and removal of swap space. >> >> ccdconfig -c ccd0 128 0 /dev/da0b /dev/da1b /dev/da2b >> swapon /dev/ccd0 > >.. and if I want to add another swap device, will that work? Yes, but it will of course not be striped with the rest, it will be laid out sequentially. And as a side effect you will not run into the NSWAP=4 limitation any more. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Fri Aug 1 02:22:07 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F394C37B401 for ; Fri, 1 Aug 2003 02:22:06 -0700 (PDT) Received: from mail.broadpark.no (mail.broadpark.no [217.13.4.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id F207843F75 for ; Fri, 1 Aug 2003 02:22:05 -0700 (PDT) (envelope-from des@des.no) Received: from smtp.des.no (37.80-203-228.nextgentel.com [80.203.228.37]) by mail.broadpark.no (Postfix) with ESMTP id 8982B78B84; Fri, 1 Aug 2003 11:22:04 +0200 (MEST) Received: by smtp.des.no (Pony Express, from userid 666) id 53B0D962E6; Fri, 1 Aug 2003 11:22:04 +0200 (CEST) Received: from dwp.des.no (dwp.des.no [10.0.0.4]) by smtp.des.no (Pony Express) with ESMTP id B8BEC960FA; Fri, 1 Aug 2003 11:22:00 +0200 (CEST) Received: by dwp.des.no (Postfix, from userid 2602) id 88C62B824; Fri, 1 Aug 2003 11:22:00 +0200 (CEST) To: "Poul-Henning Kamp" References: <6955.1059728599@critter.freebsd.dk> From: des@des.no (Dag-Erling =?iso-8859-1?q?Sm=F8rgrav?=) Date: Fri, 01 Aug 2003 11:22:00 +0200 In-Reply-To: <6955.1059728599@critter.freebsd.dk> (Poul-Henning Kamp's message of "Fri, 01 Aug 2003 11:03:19 +0200") Message-ID: User-Agent: Gnus/5.090024 (Oort Gnus v0.24) Emacs/21.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, hits=-3.0 required=8.0 tests=EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT,REFERENCES, REPLY_WITH_QUOTES,USER_AGENT_GNUS_UA version=2.55 X-Spam-Level: X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp) cc: Peter Jeremy cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2003 09:22:07 -0000 "Poul-Henning Kamp" writes: > The thing you overlook is that often when things gets paged out, the > system is short on memory and therefore more likely to not do anything > productive, whereas when things gets paged in, there are a better chance > of some other process being able to use the CPU time productively. > If we did predictive pageouts like some of the "serious" mainfram OS's > this would be less true. How hard would it be to get the kernel to write the pages "most likely to be swapped out" to swap in the idle loop, to save time if / when they actually need to be swapped out later? I thought we already did this to some extent (ref. FAQ 16.1), but apparently I was wrong? DES --=20 Dag-Erling Sm=F8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Fri Aug 1 03:02:07 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5FC6E37B401 for ; Fri, 1 Aug 2003 03:02:07 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7223243FBD for ; Fri, 1 Aug 2003 03:02:06 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h71A20mE007380; Fri, 1 Aug 2003 12:02:00 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: des@des.no (Dag-Erling =?iso-8859-1?q?Sm=F8rgrav?=) From: "Poul-Henning Kamp" In-Reply-To: Your message of "Fri, 01 Aug 2003 11:22:00 +0200." Date: Fri, 01 Aug 2003 12:02:00 +0200 Message-ID: <7379.1059732120@critter.freebsd.dk> cc: Peter Jeremy cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2003 10:02:07 -0000 In message , Dag-Erling =?iso-8859-1?q?Sm=F8rgrav?= writes: >"Poul-Henning Kamp" writes: >> The thing you overlook is that often when things gets paged out, the >> system is short on memory and therefore more likely to not do anything >> productive, whereas when things gets paged in, there are a better chance >> of some other process being able to use the CPU time productively. >> If we did predictive pageouts like some of the "serious" mainfram OS's >> this would be less true. > >How hard would it be to get the kernel to write the pages "most likely >to be swapped out" to swap in the idle loop, to save time if / when >they actually need to be swapped out later? I don't know :-) Quite frankly, given the sizes of RAM we see these days, I think that paging optimizations may be largely a thing of the past. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Fri Aug 1 07:41:18 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4EFA337B401 for ; Fri, 1 Aug 2003 07:41:18 -0700 (PDT) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6975343F85 for ; Fri, 1 Aug 2003 07:41:17 -0700 (PDT) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.12.9/8.12.9) id h71Ef9DI049878; Fri, 1 Aug 2003 09:41:09 -0500 (CDT) (envelope-from dan) Date: Fri, 1 Aug 2003 09:41:09 -0500 From: Dan Nelson To: Poul-Henning Kamp Message-ID: <20030801144109.GG13080@dan.emsphone.com> References: <7379.1059732120@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <7379.1059732120@critter.freebsd.dk> X-OS: FreeBSD 5.1-CURRENT X-message-flag: Outlook Error User-Agent: Mutt/1.5.4i cc: arch@freebsd.org cc: Dag-Erling Smorgrav cc: Peter Jeremy Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2003 14:41:18 -0000 In the last episode (Aug 01), Poul-Henning Kamp said: > In message , Dag-Erling =?iso-8859-1?q?Sm=F8rgrav?= > writes: > >"Poul-Henning Kamp" writes: > >> The thing you overlook is that often when things gets paged out, > >> the system is short on memory and therefore more likely to not do > >> anything productive, whereas when things gets paged in, there are > >> a better chance of some other process being able to use the CPU > >> time productively. If we did predictive pageouts like some of the > >> "serious" mainfram OS's this would be less true. > > > >How hard would it be to get the kernel to write the pages "most > >likely to be swapped out" to swap in the idle loop, to save time if > >/ when they actually need to be swapped out later? > > I don't know :-) > > Quite frankly, given the sizes of RAM we see these days, I think that > paging optimizations may be largely a thing of the past. RAM is like disk space; if it's there, users will consume it. If you have 8GB of RAM, someone will write a program that needs 2gb of RAM, then someone else will decide to run that program in 5 vtys (speaking from experience here). Predictive swapping would be neat. -- Dan Nelson dnelson@allantgroup.com From owner-freebsd-arch@FreeBSD.ORG Fri Aug 1 08:15:00 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D21A337B401 for ; Fri, 1 Aug 2003 08:15:00 -0700 (PDT) Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0408543F3F for ; Fri, 1 Aug 2003 08:15:00 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.9/8.12.3) with ESMTP id h71FEuFL080884; Fri, 1 Aug 2003 09:14:56 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Fri, 01 Aug 2003 09:13:17 -0600 (MDT) Message-Id: <20030801.091317.09080634.imp@bsdimp.com> To: phk@phk.freebsd.dk From: "M. Warner Losh" In-Reply-To: <7379.1059732120@critter.freebsd.dk> References: <7379.1059732120@critter.freebsd.dk> X-Mailer: Mew version 2.1 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: arch@freebsd.org cc: des@des.no cc: PeterJeremy@optushome.com.au Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2003 15:15:01 -0000 In message: <7379.1059732120@critter.freebsd.dk> "Poul-Henning Kamp" writes: : Quite frankly, given the sizes of RAM we see these days, I think that : paging optimizations may be largely a thing of the past. FreeBSD is being deployed still on machines with 32M or 64M of RAM. However, these systems tend not to have swap at all, so only the preemptive text page reclaimation stuff would be relevant to that. Warner From owner-freebsd-arch@FreeBSD.ORG Fri Aug 1 09:51:53 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9FF7137B401; Fri, 1 Aug 2003 09:51:53 -0700 (PDT) Received: from HAL9000.homeunix.com (ip114.bella-vista.sfo.interquest.net [66.199.86.114]) by mx1.FreeBSD.org (Postfix) with ESMTP id E275A43FBD; Fri, 1 Aug 2003 09:51:52 -0700 (PDT) (envelope-from das@freebsd.org) Received: from HAL9000.homeunix.com (localhost [127.0.0.1]) by HAL9000.homeunix.com (8.12.9/8.12.9) with ESMTP id h71GpoBG078178; Fri, 1 Aug 2003 09:51:50 -0700 (PDT) (envelope-from das@freebsd.org) Received: (from das@localhost) by HAL9000.homeunix.com (8.12.9/8.12.9/Submit) id h71GpovP078177; Fri, 1 Aug 2003 09:51:50 -0700 (PDT) (envelope-from das@freebsd.org) Date: Fri, 1 Aug 2003 09:51:50 -0700 From: David Schultz To: Poul-Henning Kamp Message-ID: <20030801165150.GA78091@HAL9000.homeunix.com> Mail-Followup-To: Poul-Henning Kamp , Adrian Chadd , arch@freebsd.org References: <20030801083628.GB56492@skywalker.creative.net.au> <7147.1059729519@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <7147.1059729519@critter.freebsd.dk> cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2003 16:51:53 -0000 On Fri, Aug 01, 2003, Poul-Henning Kamp wrote: > In message <20030801083628.GB56492@skywalker.creative.net.au>, Adrian Chadd wri > tes: > >On Fri, Aug 01, 2003, Poul-Henning Kamp wrote: > > > >> >Can you please describe how the interface between ccd(4) and the > >> >swap subsystem will work? I like this idea, but it isn't > >> >immediately obvious how you plan to implement striping based on > >> >load and dynamic addition and removal of swap space. > >> > >> ccdconfig -c ccd0 128 0 /dev/da0b /dev/da1b /dev/da2b > >> swapon /dev/ccd0 > > > >.. and if I want to add another swap device, will that work? > > Yes, but it will of course not be striped with the rest, it will > be laid out sequentially. > > And as a side effect you will not run into the NSWAP=4 limitation > any more. Aah, okay. That's both a small improvement and a small regression in one. The point of the static striping was to allow you to add swap devices and have them be automatically interleaved, so of course you can ``fix'' its limitations by removing it completely. I was kinda hoping you had a more flexible (albeit complex) plan in mind that would allow swap space to be striped on the fly based on disk load, free space on each disk, etc. From owner-freebsd-arch@FreeBSD.ORG Fri Aug 1 10:00:52 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6203E37B404; Fri, 1 Aug 2003 10:00:52 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 367AB43F3F; Fri, 1 Aug 2003 10:00:51 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h71H0mbS001187; Fri, 1 Aug 2003 19:00:49 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: David Schultz From: "Poul-Henning Kamp" In-Reply-To: Your message of "Fri, 01 Aug 2003 09:51:50 PDT." <20030801165150.GA78091@HAL9000.homeunix.com> Date: Fri, 01 Aug 2003 19:00:48 +0200 Message-ID: <1186.1059757248@critter.freebsd.dk> cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2003 17:00:52 -0000 In message <20030801165150.GA78091@HAL9000.homeunix.com>, David Schultz writes: >Aah, okay. That's both a small improvement and a small regression >in one. The point of the static striping was to allow you to add >swap devices and have them be automatically interleaved, so of >course you can ``fix'' its limitations by removing it completely. Well, I think it will for all practically purposes be better to not stripe (using ccd) but use the round-robin allocation on the individual components. Doing that, adding another swap device gets it "interleaved" with the rest just like before. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Fri Aug 1 17:28:16 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2B1D837B401 for ; Fri, 1 Aug 2003 17:28:16 -0700 (PDT) Received: from franky.speednet.com.au (franky.speednet.com.au [203.57.65.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 933DF43FB1 for ; Fri, 1 Aug 2003 17:28:14 -0700 (PDT) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h720SBDs084828; Sat, 2 Aug 2003 10:28:12 +1000 (EST) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h720SAHT039544; Sat, 2 Aug 2003 10:28:11 +1000 (EST) (envelope-from andyf@speednet.com.au) Date: Sat, 2 Aug 2003 10:28:10 +1000 (EST) From: Andy Farkas X-X-Sender: andyf@hewey.af.speednet.com.au To: Poul-Henning Kamp In-Reply-To: <747.1059686371@critter.freebsd.dk> Message-ID: <20030802100150.H39348-100000@hewey.af.speednet.com.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 00:28:16 -0000 Poul-Henning Kamp wrote: > People who do not adjust NSWAPDEV to match their configuration spend > a lot of RAM on holding the radix bitmap which is 3/4 empty: > > critter phk> sysctl kern.malloc | grep -i swap > SWAP 2 753K 753K 2 64 > critter phk> pstat -s > Device 1K-blocks Used Avail Capacity > /dev/ad0s1b 1321312 0 1321312 0% > > This is because the radix bitmap gets allocated for the full > stripe width, but only one quarter of it is actually used. > > Another thing is that striping does not belong in the swap_pager in > the first place, we have CCD and similar pieces of code for that. I wasn't going to say anything (you can delete now) but the more I think about it, the more I think "if it aint broke, dont fix it". For a start, what do those kern.malloc numbers mean? Mine are different: # sysctl kern.malloc | grep -i swap SWAP 2 73K 137K 12 64,65536 # pstat -s Device 1K-blocks Used Avail Capacity Type /dev/ad0s1b 65536 144 65392 0% Interleaved /dev/md0 65104 0 65104 0% Interleaved /dev/md1 65104 0 65104 0% Interleaved /dev/md2 130208 0 130208 0% Interleaved Total 325952 144 325808 0% I don't understand what the concern about using "a lot of RAM" is when you yourself say that RAM is cheap. If its affecting KVA or something, then maybe NSWAPDEV should be added into GENERIC along with an explanation that tuning it saves memory. If the user^Wadmin is aware of the "problem" then she can adjust the value when she customises her kernel. Maybe it should be mentioned somewhere (sysinstall or swapctl.8) that the *size* of swap also affect the memory footprint of the kernel. The general rule is to have swap 2x RAM - do people really make huge swapfiles that never get used? My box has 512M RAM, my swapfile size is 64M and hardly used. I don't do crashdumps. Sure, developers will want crashdumps...but thats different. So I guess Im saying you could "fix" this issue but just documenting what effects swapfile metrics have on your RAM. -- :{ andyf@speednet.com.au Andy Farkas System Administrator Speednet Communications http://www.speednet.com.au/ From owner-freebsd-arch@FreeBSD.ORG Fri Aug 1 23:06:12 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B5C8C37B401 for ; Fri, 1 Aug 2003 23:06:12 -0700 (PDT) Received: from relay.pair.com (relay.pair.com [209.68.1.20]) by mx1.FreeBSD.org (Postfix) with SMTP id E411E43FBF for ; Fri, 1 Aug 2003 23:06:11 -0700 (PDT) (envelope-from silby@silby.com) Received: (qmail 8639 invoked from network); 2 Aug 2003 06:06:10 -0000 Received: from niwun.pair.com (HELO localhost) (209.68.2.70) by relay.pair.com with SMTP; 2 Aug 2003 06:06:10 -0000 X-pair-Authenticated: 209.68.2.70 Date: Sat, 2 Aug 2003 01:04:09 -0500 (CDT) From: Mike Silbersack To: arch@freebsd.org Message-ID: <20030801235716.T2165@odysseus.silby.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: Another 4.x ABI question; uidinfo X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 06:06:13 -0000 Ok, so I took another at the uidinfo ui_ref field being only a u_short in 4.x. As I recall, the reason this was not changed to a u_int was binary compatibility... however, as I look around the source tree, it appears that the only places uidinfos are used are within kern/, and then generally only by things touching procs. Thirdly, it appears that the refcount is only modified within three functions in kern_resource.c, and that ui_ref is the last member in that structure. So, is there _really_ a problem in promoting ui_ref to an int from a short? As far as I can tell, any network or sound driver should be completely insulated from such a change; do we have any other class of binary modules to worry about? Thanks, Mike "Silby" Silbersack From owner-freebsd-arch@FreeBSD.ORG Sat Aug 2 00:30:09 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B9BA837B401 for ; Sat, 2 Aug 2003 00:30:09 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5140D43F3F for ; Sat, 2 Aug 2003 00:30:08 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h727U4bS008036; Sat, 2 Aug 2003 09:30:05 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: Andy Farkas From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sat, 02 Aug 2003 10:28:10 +1000." <20030802100150.H39348-100000@hewey.af.speednet.com.au> Date: Sat, 02 Aug 2003 09:30:04 +0200 Message-ID: <8035.1059809404@critter.freebsd.dk> cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 07:30:10 -0000 In message <20030802100150.H39348-100000@hewey.af.speednet.com.au>, Andy Farkas writes: >I wasn't going to say anything (you can delete now) but the more I think >about it, the more I think "if it aint broke, dont fix it". It is broken, it contains a bogo-vnode and it wastes RAM (it may be cheap but you shouldn't use 4 times the necessary RAM). -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sat Aug 2 00:46:01 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0C9D937B401 for ; Sat, 2 Aug 2003 00:46:01 -0700 (PDT) Received: from franky.speednet.com.au (franky.speednet.com.au [203.57.65.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 806AE43F75 for ; Sat, 2 Aug 2003 00:45:59 -0700 (PDT) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h727juDs095472; Sat, 2 Aug 2003 17:45:57 +1000 (EST) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [172.22.2.17])h727jtHT040568; Sat, 2 Aug 2003 17:45:56 +1000 (EST) (envelope-from andyf@speednet.com.au) Date: Sat, 2 Aug 2003 17:45:55 +1000 (EST) From: Andy Farkas X-X-Sender: andyf@hewey.af.speednet.com.au To: Poul-Henning Kamp In-Reply-To: <8035.1059809404@critter.freebsd.dk> Message-ID: <20030802174138.G39348-100000@hewey.af.speednet.com.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 07:46:01 -0000 Poul-Henning Kamp wrote: > It is broken, it contains a bogo-vnode and it wastes RAM (it may be cheap > but you shouldn't use 4 times the necessary RAM). I don't know what a bogo-vnode is or how it affects the system in general. Is RAM wasted if I have only one swap disk and set NSWAPDEV=1 ?? -- :{ andyf@speednet.com.au Andy Farkas System Administrator Speednet Communications http://www.speednet.com.au/ From owner-freebsd-arch@FreeBSD.ORG Sat Aug 2 02:47:10 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9479C37B401 for ; Sat, 2 Aug 2003 02:47:10 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9F20E43FBF for ; Sat, 2 Aug 2003 02:47:09 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h729l6bS009263; Sat, 2 Aug 2003 11:47:07 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: Andy Farkas From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sat, 02 Aug 2003 17:45:55 +1000." <20030802174138.G39348-100000@hewey.af.speednet.com.au> Date: Sat, 02 Aug 2003 11:47:06 +0200 Message-ID: <9262.1059817626@critter.freebsd.dk> cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 09:47:10 -0000 In message <20030802174138.G39348-100000@hewey.af.speednet.com.au>, Andy Farkas writes: >Is RAM wasted if I have only one swap disk and set NSWAPDEV=1 ?? no. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sat Aug 2 02:57:58 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6CA1337B401 for ; Sat, 2 Aug 2003 02:57:58 -0700 (PDT) Received: from franky.speednet.com.au (franky.speednet.com.au [203.57.65.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id DA0A943F85 for ; Sat, 2 Aug 2003 02:57:56 -0700 (PDT) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h729vsDs098521; Sat, 2 Aug 2003 19:57:54 +1000 (EST) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h729vrHT040935; Sat, 2 Aug 2003 19:57:53 +1000 (EST) (envelope-from andyf@speednet.com.au) Date: Sat, 2 Aug 2003 19:57:52 +1000 (EST) From: Andy Farkas X-X-Sender: andyf@hewey.af.speednet.com.au To: Poul-Henning Kamp In-Reply-To: <9262.1059817626@critter.freebsd.dk> Message-ID: <20030802195504.O40924-100000@hewey.af.speednet.com.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 09:57:58 -0000 Poul-Henning Kamp wrote: > >Is RAM wasted if I have only one swap disk and set NSWAPDEV=1 ?? > > no. You're not making it easy for me to understand. If GENERIC has NSWAPDEV=1 then there is no problem, you dont have to fsck around with code that has been working for years, and everyone is happy. -- :{ andyf@speednet.com.au Andy Farkas System Administrator Speednet Communications http://www.speednet.com.au/ From owner-freebsd-arch@FreeBSD.ORG Sat Aug 2 04:33:38 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6FD4A37B401 for ; Sat, 2 Aug 2003 04:33:38 -0700 (PDT) Received: from freebie.xs4all.nl (freebie.xs4all.nl [213.84.32.253]) by mx1.FreeBSD.org (Postfix) with ESMTP id 193CD43F93 for ; Sat, 2 Aug 2003 04:33:37 -0700 (PDT) (envelope-from wkb@freebie.xs4all.nl) Received: from freebie.xs4all.nl (localhost [127.0.0.1]) by freebie.xs4all.nl (8.12.9/8.12.9) with ESMTP id h72BXZCE096360; Sat, 2 Aug 2003 13:33:35 +0200 (CEST) (envelope-from wkb@freebie.xs4all.nl) Received: (from wkb@localhost) by freebie.xs4all.nl (8.12.9/8.12.9/Submit) id h72BXYiW096359; Sat, 2 Aug 2003 13:33:34 +0200 (CEST) Date: Sat, 2 Aug 2003 13:33:34 +0200 From: Wilko Bulte To: Andy Farkas Message-ID: <20030802113334.GA96271@freebie.xs4all.nl> References: <9262.1059817626@critter.freebsd.dk> <20030802195504.O40924-100000@hewey.af.speednet.com.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030802195504.O40924-100000@hewey.af.speednet.com.au> User-Agent: Mutt/1.4.1i X-OS: FreeBSD 4.8-STABLE X-PGP: finger wilko@freebsd.org cc: arch@FreeBSD.ORG cc: Poul-Henning Kamp Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 11:33:38 -0000 On Sat, Aug 02, 2003 at 07:57:52PM +1000, Andy Farkas wrote: > Poul-Henning Kamp wrote: > > > >Is RAM wasted if I have only one swap disk and set NSWAPDEV=1 ?? > > > > no. > > You're not making it easy for me to understand. > > If GENERIC has NSWAPDEV=1 then there is no problem, you dont have to fsck > around with code that has been working for years, and everyone is happy. And you point is? There is constant fsck-ing around as you call it with code that worked for years. If not, we would still be using V6 on PDP.. -- | / o / /_ _ wilko@FreeBSD.org |/|/ / / /( (_) Bulte From owner-freebsd-arch@FreeBSD.ORG Sat Aug 2 04:43:50 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6E8B237B401 for ; Sat, 2 Aug 2003 04:43:50 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6D9CA43F93 for ; Sat, 2 Aug 2003 04:43:49 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h72BhkbS010007; Sat, 2 Aug 2003 13:43:46 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: Wilko Bulte From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sat, 02 Aug 2003 13:33:34 +0200." <20030802113334.GA96271@freebie.xs4all.nl> Date: Sat, 02 Aug 2003 13:43:46 +0200 Message-ID: <10006.1059824626@critter.freebsd.dk> cc: arch@FreeBSD.ORG cc: Andy Farkas Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 11:43:50 -0000 In message <20030802113334.GA96271@freebie.xs4all.nl>, Wilko Bulte writes: >On Sat, Aug 02, 2003 at 07:57:52PM +1000, Andy Farkas wrote: >> Poul-Henning Kamp wrote: >> >> > >Is RAM wasted if I have only one swap disk and set NSWAPDEV=1 ?? >> > >> > no. >> >> You're not making it easy for me to understand. >> >> If GENERIC has NSWAPDEV=1 then there is no problem, you dont have to fsck >> around with code that has been working for years, and everyone is happy. > >And you point is? There is constant fsck-ing around as you call it with >code that worked for years. If not, we would still be using V6 on PDP.. (I'm not quite sure why I have not seen Andys original email, but I'll reply to this one). Andy, I can suggest 200 more worthwhile ways you could contribute to FreeBSD rather than spend time telling me what to do or not do about code you clearly do not seem to understand yourself. In a situation like this, it would probably be a good idea to take a clue from the lack of people walking in your parade, and maybe use that to trigger an re-evaluation of your agitation. Setting NSWAPDEV=1 is not an option, because that means that administrators which find themselves in a pinch can not add another swap-space to ride off a storm. And yes, Wilko is spot on too. If you want stagnation, go run a commercial unix. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sat Aug 2 05:46:29 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AFB2C37B401 for ; Sat, 2 Aug 2003 05:46:26 -0700 (PDT) Received: from franky.speednet.com.au (franky.speednet.com.au [203.57.65.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3DA7B43F85 for ; Sat, 2 Aug 2003 05:46:25 -0700 (PDT) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h72CkMDs002426; Sat, 2 Aug 2003 22:46:23 +1000 (EST) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [172.22.2.17])h72CkLHT041281; Sat, 2 Aug 2003 22:46:21 +1000 (EST) (envelope-from andyf@speednet.com.au) Date: Sat, 2 Aug 2003 22:46:20 +1000 (EST) From: Andy Farkas X-X-Sender: andyf@hewey.af.speednet.com.au To: Poul-Henning Kamp In-Reply-To: <10006.1059824626@critter.freebsd.dk> Message-ID: <20030802221832.S41132-100000@hewey.af.speednet.com.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 12:46:29 -0000 Poul-Henning Kamp wrote: > Setting NSWAPDEV=1 is not an option, because that means that > administrators which find themselves in a pinch can not add another > swap-space to ride off a storm. You still are not explaining why the current code is wrong. You say "bogo-vnode". Please explain what that means. What does "in a pinch" mean?? What do those numbers in kern.malloc mean?? I've had only one experience in the five years ive been running FreeBSD in my production environment when swapping has had an affect on normal the production environment - a mail server that only had 32M RAM and wasnt tuned properly, one time decided to spawn a dozen or so children each time the client connected and got kicked off because we have a 5 minute redial-policy... anyways, the box had so many processes running (sendmails) that the physical disk couldn't keep up with demand. I sat at the console for an hour watching the disk-busy LED full on for an hour while trying to recover from what was happening. I eventually pushed the 'reset' button and hoped for the best. Please, I know you are are God and are way above me, but try and be gentle to us lowlife scum that populate userland..... -- :{ andyf@speednet.com.au Andy Farkas System Administrator Speednet Communications http://www.speednet.com.au/ From owner-freebsd-arch@FreeBSD.ORG Sat Aug 2 06:00:26 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 47E8837B401 for ; Sat, 2 Aug 2003 06:00:26 -0700 (PDT) Received: from franky.speednet.com.au (franky.speednet.com.au [203.57.65.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 03C5743FAF for ; Sat, 2 Aug 2003 06:00:24 -0700 (PDT) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h72D0MDs002674 for ; Sat, 2 Aug 2003 23:00:23 +1000 (EST) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [172.22.2.17])h72D0LHT041318 for ; Sat, 2 Aug 2003 23:00:21 +1000 (EST) (envelope-from andyf@speednet.com.au) Date: Sat, 2 Aug 2003 23:00:21 +1000 (EST) From: Andy Farkas X-X-Sender: andyf@hewey.af.speednet.com.au To: arch@freebsd.org In-Reply-To: <10006.1059824626@critter.freebsd.dk> Message-ID: <20030802225738.I41132-100000@hewey.af.speednet.com.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 13:00:26 -0000 > In a situation like this, it would probably be a good idea to take a > clue from the lack of people walking in your parade, and maybe use > that to trigger an re-evaluation of your agitation. Can we put this in the '-o' fortune file please? -- :{ andyf@speednet.com.au Andy Farkas System Administrator Speednet Communications http://www.speednet.com.au/ From owner-freebsd-arch@FreeBSD.ORG Sat Aug 2 06:15:11 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D24D137B401 for ; Sat, 2 Aug 2003 06:15:11 -0700 (PDT) Received: from franky.speednet.com.au (franky.speednet.com.au [203.57.65.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8B96C43FBD for ; Sat, 2 Aug 2003 06:15:10 -0700 (PDT) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h72DF4Ds003006; Sat, 2 Aug 2003 23:15:04 +1000 (EST) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [172.22.2.1])h72DF4HT041358; Sat, 2 Aug 2003 23:15:04 +1000 (EST) (envelope-from andyf@speednet.com.au) Date: Sat, 2 Aug 2003 23:15:03 +1000 (EST) From: Andy Farkas X-X-Sender: andyf@hewey.af.speednet.com.au To: Wilko Bulte In-Reply-To: <20030802113334.GA96271@freebie.xs4all.nl> Message-ID: <20030802231356.V41132-100000@hewey.af.speednet.com.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 13:15:12 -0000 Wilko Bulte wrote: > > If GENERIC has NSWAPDEV=1 then there is no problem, you dont have to fsck > > around with code that has been working for years, and everyone is happy. > > And you point is? There is constant fsck-ing around as you call it with > code that worked for years. If not, we would still be using V6 on PDP.. My point is that the code works fine. Whats wrong with it? -- :{ andyf@speednet.com.au Andy Farkas System Administrator Speednet Communications http://www.speednet.com.au/ From owner-freebsd-arch@FreeBSD.ORG Sat Aug 2 06:46:49 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0643A37B401 for ; Sat, 2 Aug 2003 06:46:49 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 220F443F3F for ; Sat, 2 Aug 2003 06:46:48 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h72DkjbS010604; Sat, 2 Aug 2003 15:46:45 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: Andy Farkas From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sat, 02 Aug 2003 22:46:20 +1000." <20030802221832.S41132-100000@hewey.af.speednet.com.au> Date: Sat, 02 Aug 2003 15:46:45 +0200 Message-ID: <10603.1059832005@critter.freebsd.dk> cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 13:46:49 -0000 In message <20030802221832.S41132-100000@hewey.af.speednet.com.au>, Andy Farkas writes: >Poul-Henning Kamp wrote: > >> Setting NSWAPDEV=1 is not an option, because that means that >> administrators which find themselves in a pinch can not add another >> swap-space to ride off a storm. > >You still are not explaining why the current code is wrong. You say >"bogo-vnode". Please explain what that means. A bogos vnode. >What does "in a pinch" mean?? That for instance you are running out of swap, if NSWAPDEV==1 you cannot add another swapdev because the one and only slot is occupied (and the device is filled too). >What do those numbers in kern.malloc mean?? That the corresponding amount of RAM is unchangably occupied by a table which is 3/4 empty and therefore nothing else can use that RAM productively. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sat Aug 2 07:20:42 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C994837B401 for ; Sat, 2 Aug 2003 07:20:42 -0700 (PDT) Received: from franky.speednet.com.au (franky.speednet.com.au [203.57.65.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4F32843F75 for ; Sat, 2 Aug 2003 07:20:41 -0700 (PDT) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h72EKcDs004245; Sun, 3 Aug 2003 00:20:39 +1000 (EST) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [172.22.2.1])h72EKbHT041542; Sun, 3 Aug 2003 00:20:37 +1000 (EST) (envelope-from andyf@speednet.com.au) Date: Sun, 3 Aug 2003 00:20:37 +1000 (EST) From: Andy Farkas X-X-Sender: andyf@hewey.af.speednet.com.au To: Poul-Henning Kamp In-Reply-To: <10603.1059832005@critter.freebsd.dk> Message-ID: <20030802235543.N41132-100000@hewey.af.speednet.com.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 14:20:43 -0000 Poul-Henning Kamp wrote: > >You say > >"bogo-vnode". Please explain what that means. > > A bogos vnode. More would've been nice. > >What does "in a pinch" mean?? > > That for instance you are running out of swap, if NSWAPDEV==1 you cannot > add another swapdev because the one and only slot is occupied (and the > device is filled too). "running out of swap" is a situation I was *not* in, yet my system was totally unresponsive because it was continuously swapping. The physical disk could not keep up with page-in demand. How am I supposed to magically "add" another [physical] disk while the box is spasticly trying to swap around a 200M plus working-set into 32M physical RAM. When you're box starts swapping (in *my* experience) your box is fsck'd. All I'm saying is that whatever you do with the code (and I don't give a fsck what happens to it) it aint gonna fix, or change, anything. You are wasting your time. And no, i'm not telling you what or how to code. > >What do those numbers in kern.malloc mean?? > > That the corresponding amount of RAM is unchangably occupied by a table > which is 3/4 empty and therefore nothing else can use that RAM productively. # sysctl kern.malloc | grep -i swap SWAP 2 73K 137K 12 64,65536 Again, more explaining would be good. "corresponding amount of RAM" [you must mean the 73K and 173K values] "unchangably occupied" [my values change as I add and remove swap devices] "by a table which is 3/4 empty" [fixed by specifying NSWAPDEV=1] Please, bash me more...I still dont understand why you are trying to fix unbroken code... yes, I feel very alone in my crusade... -- :{ andyf@speednet.com.au Andy Farkas System Administrator Speednet Communications http://www.speednet.com.au/ From owner-freebsd-arch@FreeBSD.ORG Sat Aug 2 07:39:39 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 395A337B401 for ; Sat, 2 Aug 2003 07:39:39 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3D2DC43FB1 for ; Sat, 2 Aug 2003 07:39:38 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h72EdabS010873; Sat, 2 Aug 2003 16:39:37 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: Andy Farkas From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sun, 03 Aug 2003 00:20:37 +1000." <20030802235543.N41132-100000@hewey.af.speednet.com.au> Date: Sat, 02 Aug 2003 16:39:36 +0200 Message-ID: <10872.1059835176@critter.freebsd.dk> cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 14:39:39 -0000 In message <20030802235543.N41132-100000@hewey.af.speednet.com.au>, Andy Farkas writes: >All I'm saying is that whatever you do with the code (and I don't give a >fsck what happens to it) it aint gonna fix, or change, anything. You are >wasting your time. And no, i'm not telling you what or how to code. Well, if I didn't think it would change anything, I surely wouldn't be doing it. Andy, you're on your own from here, I have better ways to use my time. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Sat Aug 2 07:41:29 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7F66137B401 for ; Sat, 2 Aug 2003 07:41:29 -0700 (PDT) Received: from franky.speednet.com.au (franky.speednet.com.au [203.57.65.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 37B5043FA3 for ; Sat, 2 Aug 2003 07:41:28 -0700 (PDT) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h72EfQDs004636; Sun, 3 Aug 2003 00:41:26 +1000 (EST) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [172.22.2.17])h72EfPHT041607; Sun, 3 Aug 2003 00:41:26 +1000 (EST) (envelope-from andyf@speednet.com.au) Date: Sun, 3 Aug 2003 00:41:25 +1000 (EST) From: Andy Farkas X-X-Sender: andyf@hewey.af.speednet.com.au To: Poul-Henning Kamp In-Reply-To: <10872.1059835176@critter.freebsd.dk> Message-ID: <20030803004026.X41132-100000@hewey.af.speednet.com.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 14:41:29 -0000 Poul-Henning Kamp wrote: > Well, if I didn't think it would change anything, I surely wouldn't > be doing it. > > Andy, you're on your own from here, I have better ways to use my time. Rebuttal accepted. Thanks for explaining, not. -- :{ andyf@speednet.com.au Andy Farkas System Administrator Speednet Communications http://www.speednet.com.au/ From owner-freebsd-arch@FreeBSD.ORG Sat Aug 2 07:46:35 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 86A8C37B401 for ; Sat, 2 Aug 2003 07:46:35 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id C68D843F93 for ; Sat, 2 Aug 2003 07:46:34 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.9/8.12.9) with ESMTP id h72EjUai001082; Sat, 2 Aug 2003 10:45:30 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)h72EjTnn001079; Sat, 2 Aug 2003 10:45:29 -0400 (EDT) Date: Sat, 2 Aug 2003 10:45:29 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Mike Silbersack In-Reply-To: <20030801235716.T2165@odysseus.silby.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org Subject: Re: Another 4.x ABI question; uidinfo X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 14:46:35 -0000 On Sat, 2 Aug 2003, Mike Silbersack wrote: > Ok, so I took another at the uidinfo ui_ref field being only a u_short > in 4.x. As I recall, the reason this was not changed to a u_int was > binary compatibility... however, as I look around the source tree, it > appears that the only places uidinfos are used are within kern/, and > then generally only by things touching procs. Thirdly, it appears that > the refcount is only modified within three functions in kern_resource.c, > and that ui_ref is the last member in that structure. > > So, is there _really_ a problem in promoting ui_ref to an int from a > short? As far as I can tell, any network or sound driver should be > completely insulated from such a change; do we have any other class of > binary modules to worry about? I'm guessing it would be fine, then -- if nothing is accessing it using libkvm, it shouldn't be a problem. Changing the last field in the structure won't change dereferences from other compiled kernel modules -- the only thing I think you'd need to worry about are reference count changes in other modules (possibly, but unlikely), or statically allocated uidinfo storage (unlikely). Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Network Associates Laboratories From owner-freebsd-arch@FreeBSD.ORG Sat Aug 2 07:57:39 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A44A737B401 for ; Sat, 2 Aug 2003 07:57:39 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id E69D243F3F for ; Sat, 2 Aug 2003 07:57:38 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.9/8.12.9) with ESMTP id h72EuYai002722; Sat, 2 Aug 2003 10:56:34 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)h72EuXoF002719; Sat, 2 Aug 2003 10:56:33 -0400 (EDT) Date: Sat, 2 Aug 2003 10:56:33 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Poul-Henning Kamp In-Reply-To: <8035.1059809404@critter.freebsd.dk> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org cc: Andy Farkas Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 14:57:40 -0000 On Sat, 2 Aug 2003, Poul-Henning Kamp wrote: > In message <20030802100150.H39348-100000@hewey.af.speednet.com.au>, Andy Farkas > writes: > > >I wasn't going to say anything (you can delete now) but the more I think > >about it, the more I think "if it aint broke, dont fix it". > > It is broken, it contains a bogo-vnode and it wastes RAM (it may be > cheap but you shouldn't use 4 times the necessary RAM). For me, the biggest benefit of this change is the removal of the bogo-vnode; I'd just like to make sure we don't see an observable performance hit in interesting cases. To clarify the notion of bogo-vnodes, for those less familiar with it, there are a few vnodes that are randomly pulled out of hats for some arbitrary things. They break some of the normal working assumptions of vnodes -- for example, almost every vnode in the system has a non-NULL v_mount pointer, and is associated with a mountpoint. The vnodes I know of that don't meet this property are the swap striping vnode, and the vnodes "pulled out of a hat" to refer to devices when you don't yet have access to the device file system. This occurs in bdevvp(), which converts a dev_t into a dummy vnode pointing at the device. Under normal circumstances, non-dead vnodes always have a file system they are associated with. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Network Associates Laboratories From owner-freebsd-arch@FreeBSD.ORG Sat Aug 2 08:13:49 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A76A537B401 for ; Sat, 2 Aug 2003 08:13:49 -0700 (PDT) Received: from mail.yadt.co.uk (yadt.demon.co.uk [158.152.4.134]) by mx1.FreeBSD.org (Postfix) with SMTP id BEE8C43F75 for ; Sat, 2 Aug 2003 08:13:46 -0700 (PDT) (envelope-from davidt@yadt.co.uk) Received: (qmail 59200 invoked from network); 2 Aug 2003 15:13:43 -0000 Received: from unknown (HELO mail.gattaca.yadt.co.uk) (@10.0.0.2) by yadt.demon.co.uk with SMTP; 2 Aug 2003 15:13:43 -0000 Received: (qmail 60789 invoked by uid 1000); 2 Aug 2003 15:13:43 -0000 Date: Sat, 2 Aug 2003 16:13:42 +0100 From: David Taylor To: Andy Farkas Message-ID: <20030802151342.GA9426@gattaca.yadt.co.uk> Mail-Followup-To: Andy Farkas , arch@freebsd.org References: <10872.1059835176@critter.freebsd.dk> <20030803004026.X41132-100000@hewey.af.speednet.com.au> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline In-Reply-To: <20030803004026.X41132-100000@hewey.af.speednet.com.au> User-Agent: Mutt/1.4.1i cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 15:13:49 -0000 On Sun, 03 Aug 2003, Andy Farkas wrote: > Poul-Henning Kamp wrote: > > > Well, if I didn't think it would change anything, I surely wouldn't > > be doing it. > > > > Andy, you're on your own from here, I have better ways to use my time. > > Rebuttal accepted. Thanks for explaining, not. If all the committers had to explain every commit to everyone who decided that they couldn't see any particular benefit in the change, they would spend their entire lives arguing with people on mailing lists about different ways of doing things. The change makes the code cleaner by removing the bogus vnode which has now been explained by rwatson. It also saves RAM without limiting you to one swap device. You have come up with 0 reasons why making the change would actually be a bad thing, other than "I can't see any problems in the code I don't understand[1] so don't change it." Why, exactly, should anyone waste their time listening to your (or my) vague opinions of this change? Also, in a pinch, I'm sure you could do something with mdconfig and create a swap file on your disk, and use that as emergency swap-space, rather than having to fit a new physical disk. [1] You've spent the last 5 posts asking for explainations of the code, so I assume, like me, you don't particularly understand the internals of the code. -- David Taylor davidt@yadt.co.uk "The future just ain't what it used to be" From owner-freebsd-arch@FreeBSD.ORG Sat Aug 2 08:28:55 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 37E2037B401 for ; Sat, 2 Aug 2003 08:28:55 -0700 (PDT) Received: from franky.speednet.com.au (franky.speednet.com.au [203.57.65.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id C2C4E43F75 for ; Sat, 2 Aug 2003 08:28:53 -0700 (PDT) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h72FSnDs005516; Sun, 3 Aug 2003 01:28:49 +1000 (EST) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [172.22.2.17])h72FSmHT041775; Sun, 3 Aug 2003 01:28:48 +1000 (EST) (envelope-from andyf@speednet.com.au) Date: Sun, 3 Aug 2003 01:28:47 +1000 (EST) From: Andy Farkas X-X-Sender: andyf@hewey.af.speednet.com.au To: David Taylor In-Reply-To: <20030802151342.GA9426@gattaca.yadt.co.uk> Message-ID: <20030803012738.N41683-100000@hewey.af.speednet.com.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 15:28:55 -0000 David Taylor wrote: > Also, in a pinch, I'm sure you could do something with mdconfig and create > a swap file on your disk, and use that as emergency swap-space, rather > than having to fit a new physical disk. No. Try it sometime. > [1] You've spent the last 5 posts asking for explainations of the code, so > I assume, like me, you don't particularly understand the internals of the > code. Correct. -- :{ andyf@speednet.com.au Andy Farkas System Administrator Speednet Communications http://www.speednet.com.au/ From owner-freebsd-arch@FreeBSD.ORG Sat Aug 2 09:15:52 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 73F9937B401 for ; Sat, 2 Aug 2003 09:15:52 -0700 (PDT) Received: from franky.speednet.com.au (franky.speednet.com.au [203.57.65.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2AC5F43FCB for ; Sat, 2 Aug 2003 09:15:51 -0700 (PDT) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h72GFnDs006487 for ; Sun, 3 Aug 2003 02:15:49 +1000 (EST) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h72GFlHT041887 for ; Sun, 3 Aug 2003 02:15:49 +1000 (EST) (envelope-from andyf@speednet.com.au) Date: Sun, 3 Aug 2003 02:15:47 +1000 (EST) From: Andy Farkas X-X-Sender: andyf@hewey.af.speednet.com.au To: arch@freebsd.org In-Reply-To: <20030802100150.H39348-100000@hewey.af.speednet.com.au> Message-ID: <20030803020222.W41683-100000@hewey.af.speednet.com.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 16:15:52 -0000 One last stupid question... > Poul-Henning Kamp wrote: > > Another thing is that striping does not belong in the swap_pager in > > the first place, we have CCD and similar pieces of code for that. So when NSWAPDEV is > 1 the CCD module is auto-loaded? How much RAM does the CCD module use? -- :{ andyf@speednet.com.au Andy Farkas System Administrator Speednet Communications http://www.speednet.com.au/ From owner-freebsd-arch@FreeBSD.ORG Sat Aug 2 09:58:47 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A831037B401 for ; Sat, 2 Aug 2003 09:58:47 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id DDDB943FBF for ; Sat, 2 Aug 2003 09:58:46 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.9/8.12.9) with ESMTP id h72Gvfai018858; Sat, 2 Aug 2003 12:57:41 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)h72GvfCK018855; Sat, 2 Aug 2003 12:57:41 -0400 (EDT) Date: Sat, 2 Aug 2003 12:57:41 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Andy Farkas In-Reply-To: <20030803020222.W41683-100000@hewey.af.speednet.com.au> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: arch@freebsd.org Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 16:58:48 -0000 On Sun, 3 Aug 2003, Andy Farkas wrote: > One last stupid question... > > > Poul-Henning Kamp wrote: > > > Another thing is that striping does not belong in the swap_pager in > > > the first place, we have CCD and similar pieces of code for that. > > So when NSWAPDEV is > 1 the CCD module is auto-loaded? > > How much RAM does the CCD module use? I don't have much expertise in this area of the code, but my reading of the changes proposed by Poul-Henning is that they don't require the use of CCD to support multiple swap devices, that simply has to do with the strategy by which bits of swap is targetted at the devices connected to the swap subsystem. I.e., swapon /dev/ad0s1b followed by swapon /dev/ad1s1b doesn't require you to use CCD. You only have to use CCD if you want to reproduce the striping effect. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Network Associates Laboratories From owner-freebsd-arch@FreeBSD.ORG Sat Aug 2 10:14:01 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EFD8337B401 for ; Sat, 2 Aug 2003 10:14:01 -0700 (PDT) Received: from whale.sunbay.crimea.ua (whale.sunbay.crimea.ua [212.110.138.65]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8080943F75 for ; Sat, 2 Aug 2003 10:13:58 -0700 (PDT) (envelope-from ru@sunbay.com) Received: from whale.sunbay.crimea.ua (ru@localhost [127.0.0.1]) h72HDs0U038219 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 2 Aug 2003 20:13:54 +0300 (EEST) (envelope-from ru@sunbay.com) Received: (from ru@localhost) by whale.sunbay.crimea.ua (8.12.9/8.12.8/Submit) id h72HDrMZ038214; Sat, 2 Aug 2003 20:13:53 +0300 (EEST) (envelope-from ru) Date: Sat, 2 Aug 2003 20:13:52 +0300 From: Ruslan Ermilov To: Faried Nawaz Message-ID: <20030802171352.GA36129@sunbay.com> References: <20030730212049.GI33188@sunbay.com> <20030730162320.A66578@FreeBSD.org> <20030730212744.GJ33188@sunbay.com> <20030730163705.A68092@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="dDRMvlgZJXvWKvBx" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.4i cc: arch@FreeBSD.org Subject: Re: make -U X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 17:14:02 -0000 --dDRMvlgZJXvWKvBx Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable [ Moved to -arch ] > From the message in freebsd-hackers which first introduced > this patch: >=20 > - Date: Tue, 29 Jul 2003 09:09:17 -0700 > - From: Faried Nawaz > - Subject: patch to add make -U >=20 > While working around a port issue (ports/55013), I discovered > that make couldn't unset variables using make -U. I've written > a small patch that adds -U functionality, but I haven't tested > it extensively. >=20 > http://web.nilpotent.org/tmp/make.diff.bz2 (~ 3KB unpacked) > against yesterday's -CURRENT code. >=20 The patch looks sane. > A simple Makefile I used to test it: >=20 > -- cut here -- > FOO =3D bar >=20 > .ifdef FOO > SAY =3D y > .else > SAY =3D n > .endif >=20 > all: > echo $(SAY) > -- cut here -- >=20 > Try "make -U FOO". >=20 > Personally I think this is a reasonable option to implement. > An undefined variable is not the same as a variable which is > defined to be a null string. >=20 I want that everyone understands the effect of the patch: the -U FOO option causes the FOO variable to always be reported by make(1) as undefined, not only in this makefile, but even in sub-make processes, and affects all makefiles wishing to set it -- it just doesn't let them do this, globally. So the actual meaning of this option is "prevent variable FOO =66rom being set". (The implementation is somewhat different; it causes make(1) to lie that variable being is, but the net effect is the same.) Given the above, do we really want this option in our make? Cheers, --=20 Ruslan Ermilov Sysadmin and DBA, ru@sunbay.com Sunbay Software Ltd, ru@FreeBSD.org FreeBSD committer --dDRMvlgZJXvWKvBx Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (FreeBSD) iD8DBQE/K/FQUkv4P6juNwoRAo38AJ4wsIqLvP9XYPa5EM93/fZYeio+zwCgiITa oxuryPizIgJFkhdQIIUgbmU= =K3iD -----END PGP SIGNATURE----- --dDRMvlgZJXvWKvBx-- From owner-freebsd-arch@FreeBSD.ORG Sat Aug 2 11:59:14 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BDB4137B401 for ; Sat, 2 Aug 2003 11:59:14 -0700 (PDT) Received: from 141.com (mail.141.com [65.168.139.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id D127B43F75 for ; Sat, 2 Aug 2003 11:59:13 -0700 (PDT) (envelope-from arlankfo@141.com) Date: Sat, 2 Aug 2003 13:02:58 -0600 Message-Id: <200308021302.AA158662852@141.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii From: "Andrew Lankford" To: Wilko Bulte , Andy Farkas , X-Mailer: X-Declude-Sender: arlankfo@141.com [127.0.0.1] X-Note: This E-mail was scanned for spam. Subject: Re: headsup: swap_pager.c X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: arlankfo@141.com List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 18:59:15 -0000 >And you point is? There is constant fsck-ing around as you >call it with code that worked for years. If not, we would >still be using V6 on PDP.. > Fie on this Farkas fracas! For shame! :) Andrew Lankford From owner-freebsd-arch@FreeBSD.ORG Sat Aug 2 12:15:59 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8A12037B401 for ; Sat, 2 Aug 2003 12:15:59 -0700 (PDT) Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id AF95943F93 for ; Sat, 2 Aug 2003 12:15:57 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.9/8.12.3) with ESMTP id h72JFtFL095155 for ; Sat, 2 Aug 2003 13:15:55 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Sat, 02 Aug 2003 13:15:29 -0600 (MDT) Message-Id: <20030802.131529.105115862.imp@bsdimp.com> To: arch@freebsd.org From: "M. Warner Losh" X-Mailer: Mew version 2.1 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Multipart/Mixed; boundary="--Next_Part(Sat_Aug__2_13:15:29_2003_848)--" Content-Transfer-Encoding: 7bit Subject: Fw: New make modifiers: :C///W, :tW, and :[] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: bsd-api-discuss@metzdowd.com List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Aug 2003 19:15:59 -0000 ----Next_Part(Sat_Aug__2_13:15:29_2003_848)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Comments? If so, please forward them to the bsd-api-discuss@metzdowd.com list. Thanks. Warner ----Next_Part(Sat_Aug__2_13:15:29_2003_848)-- Content-Type: Message/Rfc822 Content-Transfer-Encoding: 7bit Content-Disposition: inline Delivery-Date: Sat, 02 Aug 2003 13:08:59 -0600 Received: from rover.village.org (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.9/8.12.3) with ESMTP id h72J8pFL095103 for ; Sat, 2 Aug 2003 13:08:51 -0600 (MDT) owner-bsd-api-discuss+imp=harmony.village.org@metzdowd.com) Received: from red.metdow.com (72.muf123.nycm.n54ny05r18.dsl.att.net [12.103.123.72]) by rover.village.org (8.12.8/8.12.8) with ESMTP id h72J8m7L039645 for ; Sat, 2 Aug 2003 13:08:51 -0600 (MDT) owner-bsd-api-discuss+imp=harmony.village.org@metzdowd.com) Received: by red.metdow.com (Postfix, from userid 1002) id E6080182F5D; Sat, 2 Aug 2003 19:08:19 +0000 (UTC) X-Original-To: bsd-api-discuss@metzdowd.com Delivered-To: bsd-api-discuss@metzdowd.com Received: from snark.piermont.com (snark.piermont.com [166.84.151.72]) by red.metdow.com (Postfix) with ESMTP id E8B00182F59 for ; Sat, 2 Aug 2003 19:08:18 +0000 (UTC) Received: by snark.piermont.com (Postfix, from userid 1000) id D0CBAD97C5; Sat, 2 Aug 2003 15:08:17 -0400 (EDT) X-Original-To: bsd-api-discuss@metzdowd.com Received: from citadel.cequrux.com (citadel.cequrux.com [192.96.22.18]) by red.metdow.com (Postfix) with ESMTP id 127B0182F59 for ; Thu, 31 Jul 2003 17:59:53 +0000 (UTC) Received: (from nobody@localhost) by citadel.cequrux.com (8.8.8/8.6.9) id TAA02853 for ; Thu, 31 Jul 2003 19:59:41 +0200 (SAST) Received: by citadel.cequrux.com via recvmail id 2819; Thu, 31 Jul 2003 19:58:58 +0200 (SAST) Date: Thu, 31 Jul 2003 19:58:56 +0200 From: Alan Barrett To: bsd-api-discuss@metzdowd.com Subject: New make modifiers: :C///W, :tW, and :[] Message-ID: <20030731175401.GE2078@apb.cequrux.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-bsd-api-discuss@metzdowd.com Precedence: bulk X-Spam-Status: No, hits=0.0 required=5.0 tests=none version=2.55 X-Spam-Level: X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp) I propose adding a new :[] modifier to make(1) in NetBSD, and possibly also adding some other modifiers. ":[]" will allow you to extract a single word from a space-separated list, like this: CC= /usr/bin/gcc -Wall .if exists(${CC:[1]}) ... See http://mail-index.netbsd.org/tech-userlevel/2003/07/31/0000.html for a post to NetBSD's tech-userlevel list where I discusssed the problem and several proposed solutions. See other nearby messages linked from http://mail-index.netbsd.org/tech-userlevel/2003/07/ for responses. Briefly, the proposals are :tW Set a flag that causes subsequent modifiers to treat the value as a single entity instead of as a space-separated list. Why ":tW"? We already have :tu and :tl to convert to upper or lower case, so starting with "t" is natural. "tW" is supposed to suggest "convert to one big word". :tw Undo the effect of :tW. "tw" is supposed to suggest "convert to small words". :C///W A "one big word" variant of the existing :C/// regular expression substitution modifier. Unlike the usual :C/// (which works separately on each space-separated word), this would work on the entire string all at once, so it would be able to match teh spaces between words. :S///W A "one big word" variant of the existing :S/// simple string substitution operator. :[N] Select the Nth word, where N is a positive integer. :[1] selects the first word. :[-N] Select the Nth word from the end. :[-1] selects the last word. :[0] Select the entire string, but treat it as one big word. (Equivalent to :tw.) :[*] Select the entire string, treating it as a sequence of space-separated words. (Undoes the effects of :[0]; equivalent to :tw.) There seems to be support for adding :[] to NetBSD's make(1). :tW/:tw and :C///W are not really necessary if you have :[0] and :[*], but they might nevertheless be useful enough to add. Would other BSDs be likely to pick up these features, or at least not use the same modifier characters with incompatible meanings? --apb (Alan Barrett) --------------------------------------------------------------------- The BSD APIs Discussion Mailing List To unsubscribe: send "unsubscribe bsd-api-discuss" to majordomo@metzdowd.com ----Next_Part(Sat_Aug__2_13:15:29_2003_848)---- From owner-freebsd-arch@FreeBSD.ORG Sat Aug 2 19:21:48 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9009137B401 for ; Sat, 2 Aug 2003 19:21:48 -0700 (PDT) Received: from smtp2.server.rpi.edu (smtp2.server.rpi.edu [128.113.2.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id D1E0A43F75 for ; Sat, 2 Aug 2003 19:21:47 -0700 (PDT) (envelope-from drosih@rpi.edu) Received: from [128.113.24.47] (gilead.netel.rpi.edu [128.113.24.47]) by smtp2.server.rpi.edu (8.12.9/8.12.9) with ESMTP id h732LkCx013303; Sat, 2 Aug 2003 22:21:46 -0400 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: <20030802.131529.105115862.imp@bsdimp.com> References: <20030802.131529.105115862.imp@bsdimp.com> Date: Sat, 2 Aug 2003 22:21:45 -0400 To: bsd-api-discuss@metzdowd.com, arch@freebsd.org From: Garance A Drosihn Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Scanned-By: CanIt (www . canit . ca) Subject: Re: Fw: New make modifiers: :C///W, :tW, and :[] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Aug 2003 02:21:48 -0000 At 1:15 PM -0600 8/2/03, M. Warner Losh wrote: >Comments? If so, please forward them to the >bsd-api-discuss@metzdowd.com list. Thanks. > >Warner > >Date: Thu, 31 Jul 2003 19:58:56 +0200 >From: Alan Barrett >To: bsd-api-discuss@metzdowd.com >Subject: New make modifiers: :C///W, :tW, and :[] > >I propose adding a new :[] modifier to make(1) in NetBSD, >and possibly also adding some other modifiers. Just a note. In my reply on that list, I made the following offer: If other freebsd developers do not object to the *idea* of these changes, I would be willing to look at installing similar changes to FreeBSD, after NetBSD has worked with them for a little while. So, speak up if the proposal strikes you as a bad idea. (also note that I'm perfectly happy for someone else to do this work on freebsd's make, if anyone wants to) -- Garance Alistair Drosehn = gad@gilead.netel.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu