From owner-freebsd-arch Sun Feb 9 0:31:15 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A4DDE37B401 for ; Sun, 9 Feb 2003 00:31:14 -0800 (PST) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6D73C43FB1 for ; Sun, 9 Feb 2003 00:31:13 -0800 (PST) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.6/8.12.6) with ESMTP id h198V3qV025687; Sun, 9 Feb 2003 09:31:07 +0100 (CET) (envelope-from phk@phk.freebsd.dk) To: Alfred Perlstein Cc: Peter Wemm , Jake Burkholder , Jeff Roberson , arch@FreeBSD.ORG Subject: Re: New kernel allocation API From: phk@phk.freebsd.dk In-Reply-To: Your message of "Sat, 08 Feb 2003 23:57:06 PST." <20030209075706.GO88781@elvis.mu.org> Date: Sun, 09 Feb 2003 09:31:03 +0100 Message-ID: <25686.1044779463@critter.freebsd.dk> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20030209075706.GO88781@elvis.mu.org>, Alfred Perlstein writes: >I may see if I can convert some paths in the kernel to use zones. :) I have a number of places where zones can be used instead of malloc, but since none of the places I have tried previously have resulted in measurable performance differences it has not been high on my agenda. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 9 0:53:24 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 28F8A37B401 for ; Sun, 9 Feb 2003 00:53:23 -0800 (PST) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id D1D4743F93 for ; Sun, 9 Feb 2003 00:53:22 -0800 (PST) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id 9D7B3AE28A; Sun, 9 Feb 2003 00:53:17 -0800 (PST) Date: Sun, 9 Feb 2003 00:53:17 -0800 From: Alfred Perlstein To: phk@phk.freebsd.dk Cc: Peter Wemm , Jake Burkholder , Jeff Roberson , arch@FreeBSD.ORG Subject: Re: New kernel allocation API Message-ID: <20030209085317.GP88781@elvis.mu.org> References: <20030209075706.GO88781@elvis.mu.org> <25686.1044779463@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <25686.1044779463@critter.freebsd.dk> User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * phk@phk.freebsd.dk [030209 00:31] wrote: > In message <20030209075706.GO88781@elvis.mu.org>, Alfred Perlstein writes: > > >I may see if I can convert some paths in the kernel to use zones. :) > > I have a number of places where zones can be used instead of malloc, > but since none of the places I have tried previously have resulted > in measurable performance differences it has not been high on my > agenda. Well, once converted to zones they could also be modded to take advantage of the init/fini processing which _should_ make things faster. Also don't forget even if you don't see a perf improvement you'll see siginifigant space savings if your size is not a power of two or multiple of page size. -- -Alfred Perlstein [alfred@freebsd.org] 'Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom.' To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 9 0:58:25 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0709737B401 for ; Sun, 9 Feb 2003 00:58:24 -0800 (PST) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 332C443F93 for ; Sun, 9 Feb 2003 00:58:23 -0800 (PST) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.6/8.12.6) with ESMTP id h198wIqV025849; Sun, 9 Feb 2003 09:58:18 +0100 (CET) (envelope-from phk@phk.freebsd.dk) To: Alfred Perlstein Cc: Peter Wemm , Jake Burkholder , Jeff Roberson , arch@FreeBSD.ORG Subject: Re: New kernel allocation API From: phk@phk.freebsd.dk In-Reply-To: Your message of "Sun, 09 Feb 2003 00:53:17 PST." <20030209085317.GP88781@elvis.mu.org> Date: Sun, 09 Feb 2003 09:58:18 +0100 Message-ID: <25848.1044781098@critter.freebsd.dk> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20030209085317.GP88781@elvis.mu.org>, Alfred Perlstein writes: >* phk@phk.freebsd.dk [030209 00:31] wrote: >> In message <20030209075706.GO88781@elvis.mu.org>, Alfred Perlstein writes: >> >> >I may see if I can convert some paths in the kernel to use zones. :) >> >> I have a number of places where zones can be used instead of malloc, >> but since none of the places I have tried previously have resulted >> in measurable performance differences it has not been high on my >> agenda. > >Well, once converted to zones they could also be modded to take >advantage of the init/fini processing which _should_ make things >faster. Also don't forget even if you don't see a perf improvement >you'll see siginifigant space savings if your size is not a power of >two or multiple of page size. I'm aware of all this, but I guess most of the objects I deal with don't really have enough trafic to come out measurable, probably because they don't contain mutexes. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 9 3:35:36 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CA15637B401 for ; Sun, 9 Feb 2003 03:35:32 -0800 (PST) Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2BA7A43F93 for ; Sun, 9 Feb 2003 03:35:32 -0800 (PST) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id h19BZ5Z83461; Sun, 9 Feb 2003 06:35:05 -0500 (EST) (envelope-from jroberson@chesapeake.net) Date: Sun, 9 Feb 2003 06:35:05 -0500 (EST) From: Jeff Roberson To: phk@phk.freebsd.dk Cc: Alfred Perlstein , Peter Wemm , Jake Burkholder , Subject: Re: New kernel allocation API In-Reply-To: <25848.1044781098@critter.freebsd.dk> Message-ID: <20030209063342.W77658-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, 9 Feb 2003 phk@phk.freebsd.dk wrote: > In message <20030209085317.GP88781@elvis.mu.org>, Alfred Perlstein writes: > >* phk@phk.freebsd.dk [030209 00:31] wrote: > >> In message <20030209075706.GO88781@elvis.mu.org>, Alfred Perlstein writes: > >> > >> >I may see if I can convert some paths in the kernel to use zones. :) > >> > >> I have a number of places where zones can be used instead of malloc, > >> but since none of the places I have tried previously have resulted > >> in measurable performance differences it has not been high on my > >> agenda. > > > >Well, once converted to zones they could also be modded to take > >advantage of the init/fini processing which _should_ make things > >faster. Also don't forget even if you don't see a perf improvement > >you'll see siginifigant space savings if your size is not a power of > >two or multiple of page size. > > I'm aware of all this, but I guess most of the objects I deal with > don't really have enough trafic to come out measurable, probably > because they don't contain mutexes. > I did pipes once when I originally did UMA. The perf was measurable. It had carefully crafted init/fini ctor/dtor though. Also, creds could possibly benefit. UMA mbufs would rule too. :-) Cheers, Jeff To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 9 6:17:53 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8136837B40D for ; Sun, 9 Feb 2003 06:17:51 -0800 (PST) Received: from angelica.unixdaemons.com (angelica.unixdaemons.com [209.148.64.135]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9226343FE1 for ; Sun, 9 Feb 2003 06:17:48 -0800 (PST) (envelope-from hiten@angelica.unixdaemons.com) Received: from angelica.unixdaemons.com (hiten@localhost.unixdaemons.com [127.0.0.1]) by angelica.unixdaemons.com (8.12.7/8.12.1) with ESMTP id h19EHcOm030316; Sun, 9 Feb 2003 09:17:38 -0500 (EST) Received: (from hiten@localhost) by angelica.unixdaemons.com (8.12.7/8.12.1/Submit) id h19EHcD5030315; Sun, 9 Feb 2003 09:17:38 -0500 (EST) (envelope-from hiten) Date: Sun, 9 Feb 2003 09:17:38 -0500 From: Hiten Pandya To: Jeff Roberson Cc: arch@FreeBSD.ORG Subject: Re: New kernel allocation API Message-ID: <20030209141738.GA29856@unixdaemons.com> References: <25848.1044781098@critter.freebsd.dk> <20030209063342.W77658-100000@mail.chesapeake.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030209063342.W77658-100000@mail.chesapeake.net> User-Agent: Mutt/1.4i X-Operating-System: FreeBSD i386 X-Public-Key: http://www.pittgoth.com/~hiten/pubkey.asc X-URL: http://www.unixdaemons.com/~hiten X-PGP: http://pgp.mit.edu:11371/pks/lookup?search=Hiten+Pandya&op=index Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, Feb 09, 2003 at 06:35:05AM -0500, Jeff Roberson wrote the words in effect of: > On Sun, 9 Feb 2003 phk@phk.freebsd.dk wrote: > > > In message <20030209085317.GP88781@elvis.mu.org>, Alfred Perlstein writes: > > >* phk@phk.freebsd.dk [030209 00:31] wrote: > > >> In message <20030209075706.GO88781@elvis.mu.org>, Alfred Perlstein writes: > > >> > > >> >I may see if I can convert some paths in the kernel to use zones. :) > > >> > > >> I have a number of places where zones can be used instead of malloc, > > >> but since none of the places I have tried previously have resulted > > >> in measurable performance differences it has not been high on my > > >> agenda. > > > > > >Well, once converted to zones they could also be modded to take > > >advantage of the init/fini processing which _should_ make things > > >faster. Also don't forget even if you don't see a perf improvement > > >you'll see siginifigant space savings if your size is not a power of > > >two or multiple of page size. > > > > I'm aware of all this, but I guess most of the objects I deal with > > don't really have enough trafic to come out measurable, probably > > because they don't contain mutexes. > > > > I did pipes once when I originally did UMA. The perf was measurable. It > had carefully crafted init/fini ctor/dtor though. Also, creds could > possibly benefit. > > UMA mbufs would rule too. :-) [ CC list trimmed ] Would it help with filesystems? -- Hiten Pandya (hiten@unixdaemons.com, hiten@uk.FreeBSD.org) http://www.unixdaemons.com/~hiten/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 9 11:14:25 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DAE8837B401 for ; Sun, 9 Feb 2003 11:14:23 -0800 (PST) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id D633443FFD for ; Sun, 9 Feb 2003 11:14:21 -0800 (PST) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.6/8.12.6) with ESMTP id h19J3OqV031119 for ; Sun, 9 Feb 2003 20:03:24 +0100 (CET) (envelope-from phk@phk.freebsd.dk) To: arch@freebsd.org Subject: Our lemming-syncer caught in the act. From: Poul-Henning Kamp Date: Sun, 09 Feb 2003 20:03:24 +0100 Message-ID: <31118.1044817404@critter.freebsd.dk> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I have suspected our syncer of being subobtimal for some time, based simply on my perception of the disk-light on my laptop and the dynamics of the "dirty" counter in systat. I played with the new GEOM I/O statistics stuff and guess what: I caught it in the act: http://phk.freebsd.dk/misc/disk.png Green is the number of unfinished requests. Most of the red "impulse" represents one request finishing after as many milliseconds as it is tall. The remainder of the impulses cover more than one request, the height is then the average of the time it has taken to service them. An image is worth a thousand words, but in this case it only says three words: "Man, that sucks!". Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 9 11:16:31 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 34ABE37B401 for ; Sun, 9 Feb 2003 11:16:30 -0800 (PST) Received: from canning.wemm.org (canning.wemm.org [192.203.228.65]) by mx1.FreeBSD.org (Postfix) with ESMTP id C606E43F75 for ; Sun, 9 Feb 2003 11:16:29 -0800 (PST) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by canning.wemm.org (Postfix) with ESMTP id 88B942A8B4; Sun, 9 Feb 2003 11:16:29 -0800 (PST) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: phk@phk.freebsd.dk Cc: Alfred Perlstein , Jake Burkholder , Jeff Roberson , arch@FreeBSD.ORG Subject: Re: New kernel allocation API In-Reply-To: <25686.1044779463@critter.freebsd.dk> Date: Sun, 09 Feb 2003 11:16:29 -0800 From: Peter Wemm Message-Id: <20030209191629.88B942A8B4@canning.wemm.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG phk@phk.freebsd.dk wrote: > In message <20030209075706.GO88781@elvis.mu.org>, Alfred Perlstein writes: > > >I may see if I can convert some paths in the kernel to use zones. :) > > I have a number of places where zones can be used instead of malloc, > but since none of the places I have tried previously have resulted > in measurable performance differences it has not been high on my > agenda. I'd be interested to know what it did to KVA usage on i386. I would expect to find a more efficient KVA utilization (and memory utilization) due to better packing of objects instead of the power-of-two roundup that malloc does. Cheers, -Peter -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 9 11:25:46 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 25E1637B41A for ; Sun, 9 Feb 2003 11:25:44 -0800 (PST) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 068BD43F3F for ; Sun, 9 Feb 2003 11:25:43 -0800 (PST) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.6/8.12.6) with ESMTP id h19JPWqV031324; Sun, 9 Feb 2003 20:25:32 +0100 (CET) (envelope-from phk@phk.freebsd.dk) To: Peter Wemm Cc: Alfred Perlstein , Jake Burkholder , Jeff Roberson , arch@FreeBSD.ORG Subject: Re: New kernel allocation API From: phk@phk.freebsd.dk In-Reply-To: Your message of "Sun, 09 Feb 2003 11:16:29 PST." <20030209191629.88B942A8B4@canning.wemm.org> Date: Sun, 09 Feb 2003 20:25:32 +0100 Message-ID: <31323.1044818732@critter.freebsd.dk> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20030209191629.88B942A8B4@canning.wemm.org>, Peter Wemm writes: >phk@phk.freebsd.dk wrote: >> In message <20030209075706.GO88781@elvis.mu.org>, Alfred Perlstein writes: >> >> >I may see if I can convert some paths in the kernel to use zones. :) >> >> I have a number of places where zones can be used instead of malloc, >> but since none of the places I have tried previously have resulted >> in measurable performance differences it has not been high on my >> agenda. > >I'd be interested to know what it did to KVA usage on i386. I would expect >to find a more efficient KVA utilization (and memory utilization) due to >better packing of objects instead of the power-of-two roundup that malloc >does. Well, none if it has very large memory footprints, we have like 10-20k of dev_t's and while GEOM sucks up more than that, it is a lot of different sizes so it actually ends up wasting space. struct bio may be a candidate, but we're usually in the low hundreds of those at any time so there may be 30k to save or something. It did do wonders FFS inodes and dinodes, but those come in numbers approaching 100k on my system. sysctl kern.malloc shows where the gains are to be had. Unfortunately, the vfs-cache stores variable length strings... bang# sysctl kern.malloc | sort +0.19 -n | tail -10 MSDOSFS mount 1 256K 256K 1 kobj 80 320K 320K 80 4096 NFS hash 1 512K 512K 1 UFS ihash 1 512K 512K 1 ISOFS mount 1 512K 512K 1 inodedep 144 530K 2645K 299243 128,256 devbuf 455 1004K 1043K 918 16,32,64,128,256,512,1024,2048,4096,16384 GEOM 6081 1533K 1533K 6327 16,32,64,128,256,512,1024,4096 SWAP 2 3281K 3281K 2 64 vfscache 90101 6668K 7708K 1246645 64,128,256 -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 9 15:29:17 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4BA7037B405 for ; Sun, 9 Feb 2003 15:29:15 -0800 (PST) Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 71B0A43F85 for ; Sun, 9 Feb 2003 15:29:14 -0800 (PST) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id h19NTDj24866 for ; Sun, 9 Feb 2003 18:29:14 -0500 (EST) (envelope-from jroberson@chesapeake.net) Date: Sun, 9 Feb 2003 18:29:13 -0500 (EST) From: Jeff Roberson To: arch@freebsd.org Subject: cvs commit: src/sys/kern vfs_bio.c Message-ID: <20030209181740.Q1025-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Modified files: sys/kern vfs_bio.c Log: - Lock down the buffer cache's infrastructure code. This includes locks on buf lists, synchronization variables, and atomic ops for the counters. This change does not remove giant from any code although some pushdown may be possible. - In vfs_bio_awrite() don't access buf fields without the buf lock. I realize I should have brought this up for review before commiting. I'm sending this mail out to give people a chance to comment on my strategy. Here's the scoop. There are many counters for active and free resources in the buf cache that are used to determine behavior at certain points. These, in most cases, are influenced by a min/max value that are setup at boot time and do not change. For these counters I used atomic ops to update them but not locks. The results of reading these atomicly updated values stale will simply be slightly oversteping some boundaries such as maxufmallocspace. At the msleep/wakeup synchronization points the synchronization variables that sometimes contain state have locks to protect them. These locks conveniently protect against sleep/wakeup races as well. I need to add some synchronization to bufwait() etc. I think I may just use one global lock for now since each buf does not need a mutex. Other than that, I put a lock on the buf queues. This is pretty straightforward. To lock the bufs themselves I examined how extensively the buf lock is used. There are only a few ways to get to a buf: getblk(), bread(), etc. These return a locked buf. gbincore(), incore(). These return bufs with unknown lockstates. vnode's dirty/clean buf lists. These return bufs with unknown lockstates. So I have gone through the tree looking for places where we use the unlocked access methods and tried to make these safe. So far I've mostly just cleaned up unlocked accesses in syncer. I've done this by protecting a new flags field in buf with the vnode interlock. This is very convenient for the dirty/clean list cases. In the end the buf lock will protect the majority of the bufs. This is turning into a surprisingly small amount of effort in the individual filesystems. I'm very pleased at this. Comments are, of course, welcome. Thanks, Jeff To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 9 16: 4:44 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5091337B401 for ; Sun, 9 Feb 2003 16:04:42 -0800 (PST) Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.80]) by mx1.FreeBSD.org (Postfix) with ESMTP id D093843FA3 for ; Sun, 9 Feb 2003 16:04:40 -0800 (PST) (envelope-from grog@lemis.com) Received: by wantadilla.lemis.com (Postfix, from userid 1004) id 188C251978; Mon, 10 Feb 2003 10:34:39 +1030 (CST) Date: Mon, 10 Feb 2003 10:34:39 +1030 From: Greg 'groggy' Lehey To: Poul-Henning Kamp Cc: arch@freebsd.org Subject: Re: Our lemming-syncer caught in the act. Message-ID: <20030210000439.GG60203@wantadilla.lemis.com> References: <31118.1044817404@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="2E/hm+v6kSLEYT3h" Content-Disposition: inline In-Reply-To: <31118.1044817404@critter.freebsd.dk> User-Agent: Mutt/1.4i Organization: The FreeBSD Project Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.FreeBSD.org/ X-PGP-Fingerprint: 9A1B 8202 BCCE B846 F92F 09AC 22E6 F290 507A 4223 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --2E/hm+v6kSLEYT3h Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Sunday, 9 February 2003 at 20:03:24 +0100, Poul-Henning Kamp wrote: > > I have suspected our syncer of being subobtimal for some time, based > simply on my perception of the disk-light on my laptop and the > dynamics of the "dirty" counter in systat. > > I played with the new GEOM I/O statistics stuff and guess what: I > caught it in the act: > > http://phk.freebsd.dk/misc/disk.png > > Green is the number of unfinished requests. > > Most of the red "impulse" represents one request finishing after > as many milliseconds as it is tall. The remainder of the impulses > cover more than one request, the height is then the average of the > time it has taken to service them. > > An image is worth a thousand words, but in this case it only > says three words: "Man, that sucks!". Nope, it's one of the prettiest graphs I've seen in a long time. Greg -- See complete headers for address and phone numbers --2E/hm+v6kSLEYT3h Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.0 (FreeBSD) iD8DBQE+RuyWIubykFB6QiMRAlhlAKCw8KJW6qTbTSI4GpVXcM4s67ODhQCeMx2a n4sCUUYSZLW0Xh1hA3EcgFY= =uXuQ -----END PGP SIGNATURE----- --2E/hm+v6kSLEYT3h-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Feb 9 17:12:11 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BCF0137B401 for ; Sun, 9 Feb 2003 17:12:09 -0800 (PST) Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id D2C1A43F93 for ; Sun, 9 Feb 2003 17:12:06 -0800 (PST) (envelope-from des@ofug.org) Received: by flood.ping.uio.no (Postfix, from userid 2602) id 922E1536E; Mon, 10 Feb 2003 02:12:04 +0100 (CET) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Poul-Henning Kamp Cc: arch@freebsd.org Subject: Re: Our lemming-syncer caught in the act. From: Dag-Erling Smorgrav Date: Mon, 10 Feb 2003 02:12:03 +0100 In-Reply-To: <31118.1044817404@critter.freebsd.dk> (Poul-Henning Kamp's message of "Sun, 09 Feb 2003 20:03:24 +0100") Message-ID: User-Agent: Gnus/5.090014 (Oort Gnus v0.14) Emacs/21.2 (i386--freebsd) References: <31118.1044817404@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Poul-Henning Kamp writes: > An image is worth a thousand words, but in this case it only > says three words: "Man, that sucks!". Yes, it's really amazing that you draw conclusions about the quality of our syncer (and expect us to instantly agree with them) based on such a shitty graph. I see no information at all about how it was created, what the units are for the axes (and the green curve is most likely in a different unit than the red bars), your bars are averages which means there's no information about how many requests were satisfied at each point in time, you don't say anything about what you call a request and how you determine when it has been satisfied, etc., etc. For all I know the green curve could show requests at the UFS level and the red bars could show the time it takes before they hit the disk, in which case the graph simply shows that softupdates is enabled on the partition you did the testing on. If you're trying to say what I think you're trying to say - i.e. "some requests take a long time to be satisfied" - you should plot a histogram where each bin corresponds to a time range and the height of each bar corresponds to the number of requests whose age was in that range when they were satisfied. And you should make very very sure that your definitions of "initiated" and "satisfied" were consistent (i.e. at the exact same level in the I/O stack). DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 1: 6:50 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 886E237B401 for ; Mon, 10 Feb 2003 01:06:49 -0800 (PST) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5A05D43FEA for ; Mon, 10 Feb 2003 01:05:28 -0800 (PST) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.6/8.12.6) with ESMTP id h1A95QqV037083; Mon, 10 Feb 2003 10:05:27 +0100 (CET) (envelope-from phk@phk.freebsd.dk) To: Dag-Erling Smorgrav Cc: arch@freebsd.org Subject: Re: Our lemming-syncer caught in the act. From: phk@phk.freebsd.dk In-Reply-To: Your message of "Mon, 10 Feb 2003 02:12:03 +0100." Date: Mon, 10 Feb 2003 10:05:26 +0100 Message-ID: <37082.1044867926@critter.freebsd.dk> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message , Dag-Erling Smorgrav writes: >Poul-Henning Kamp writes: >> An image is worth a thousand words, but in this case it only >> says three words: "Man, that sucks!". > >Yes, it's really amazing that you draw conclusions about the quality >of our syncer (and expect us to instantly agree with them) based on >such a shitty graph. I see no information at all about how it was >created, what the units are for the axes (and the green curve is most >likely in a different unit than the red bars), your bars are averages >which means there's no information about how many requests were >satisfied at each point in time, you don't say anything about what you >call a request and how you determine when it has been satisfied, etc., >etc. You must obviously have overlooked the majority of my email: > Green is the number of unfinished requests. > > Most of the red "impulse" represents one request finishing after > as many milliseconds as it is tall. The remainder of the impulses > cover more than one request, the height is then the average of the > time it has taken to service them. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 1:14:40 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2E96137B401 for ; Mon, 10 Feb 2003 01:14:39 -0800 (PST) Received: from HAL9000.homeunix.com (12-233-57-224.client.attbi.com [12.233.57.224]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7F58B43FE0 for ; Mon, 10 Feb 2003 01:13:18 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Received: from HAL9000.homeunix.com (localhost [127.0.0.1]) by HAL9000.homeunix.com (8.12.6/8.12.5) with ESMTP id h1A9DHoH005418; Mon, 10 Feb 2003 01:13:17 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Received: (from das@localhost) by HAL9000.homeunix.com (8.12.6/8.12.5/Submit) id h1A9DHKE005417; Mon, 10 Feb 2003 01:13:17 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Date: Mon, 10 Feb 2003 01:13:17 -0800 From: David Schultz To: Poul-Henning Kamp Cc: arch@FreeBSD.ORG Subject: Re: Our lemming-syncer caught in the act. Message-ID: <20030210091317.GD5165@HAL9000.homeunix.com> Mail-Followup-To: Poul-Henning Kamp , arch@FreeBSD.ORG References: <31118.1044817404@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <31118.1044817404@critter.freebsd.dk> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Thus spake Poul-Henning Kamp : > I have suspected our syncer of being subobtimal for some time, based > simply on my perception of the disk-light on my laptop and the > dynamics of the "dirty" counter in systat. > > I played with the new GEOM I/O statistics stuff and guess what: I > caught it in the act: > > http://phk.freebsd.dk/misc/disk.png > > Green is the number of unfinished requests. > > Most of the red "impulse" represents one request finishing after > as many milliseconds as it is tall. The remainder of the impulses > cover more than one request, the height is then the average of the > time it has taken to service them. > > An image is worth a thousand words, but in this case it only > says three words: "Man, that sucks!". When a large file times out, a significant amount of I/O can be generated. This is still far better than the old syncer that flushed everything every 30 seconds. The reasons for this behavior are explained in src/sys/ufs/ffs/README. After reading that, do you still think it makes sense to try to do better? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 1:24: 1 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7B09637B401 for ; Mon, 10 Feb 2003 01:24:00 -0800 (PST) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2E1A243FE3 for ; Mon, 10 Feb 2003 01:23:17 -0800 (PST) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.6/8.12.6) with ESMTP id h1A9NFqV037474; Mon, 10 Feb 2003 10:23:16 +0100 (CET) (envelope-from phk@phk.freebsd.dk) To: David Schultz Cc: arch@FreeBSD.ORG Subject: Re: Our lemming-syncer caught in the act. From: phk@phk.freebsd.dk In-Reply-To: Your message of "Mon, 10 Feb 2003 01:13:17 PST." <20030210091317.GD5165@HAL9000.homeunix.com> Date: Mon, 10 Feb 2003 10:23:15 +0100 Message-ID: <37473.1044868995@critter.freebsd.dk> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20030210091317.GD5165@HAL9000.homeunix.com>, David Schultz writes: >When a large file times out, a significant amount of I/O can be >generated. This is still far better than the old syncer that >flushed everything every 30 seconds. The reasons for this >behavior are explained in src/sys/ufs/ffs/README. After reading >that, do you still think it makes sense to try to do better? Yes, it makes a lot of sense. There is no point in batching up writes to the point of showing 200 requests off at once then wait 30 seconds, then do it again etc etc. We can and need to do better than that. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 1:50:44 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7EE5A37B401 for ; Mon, 10 Feb 2003 01:50:42 -0800 (PST) Received: from HAL9000.homeunix.com (12-233-57-224.client.attbi.com [12.233.57.224]) by mx1.FreeBSD.org (Postfix) with ESMTP id B1C1343F75 for ; Mon, 10 Feb 2003 01:50:41 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Received: from HAL9000.homeunix.com (localhost [127.0.0.1]) by HAL9000.homeunix.com (8.12.6/8.12.5) with ESMTP id h1A9oeoH005612; Mon, 10 Feb 2003 01:50:40 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Received: (from das@localhost) by HAL9000.homeunix.com (8.12.6/8.12.5/Submit) id h1A9oeIH005611; Mon, 10 Feb 2003 01:50:40 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Date: Mon, 10 Feb 2003 01:50:40 -0800 From: David Schultz To: phk@phk.freebsd.dk Cc: arch@FreeBSD.ORG Subject: Re: Our lemming-syncer caught in the act. Message-ID: <20030210095040.GB5515@HAL9000.homeunix.com> Mail-Followup-To: phk@phk.freebsd.dk, arch@FreeBSD.ORG References: <20030210091317.GD5165@HAL9000.homeunix.com> <37473.1044868995@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <37473.1044868995@critter.freebsd.dk> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Thus spake phk@phk.freebsd.dk : > In message <20030210091317.GD5165@HAL9000.homeunix.com>, David Schultz writes: > > >When a large file times out, a significant amount of I/O can be > >generated. This is still far better than the old syncer that > >flushed everything every 30 seconds. The reasons for this > >behavior are explained in src/sys/ufs/ffs/README. After reading > >that, do you still think it makes sense to try to do better? > > Yes, it makes a lot of sense. There is no point in batching up > writes to the point of showing 200 requests off at once then > wait 30 seconds, then do it again etc etc. > > We can and need to do better than that. It sounds like you're describing the old syncer. The new one still shoves groups of buffers out at once, but on a file-by-file basis rather than globally every 30 seconds. (Maybe that's what you meant.) To fix the problem correctly, you would probably need to track the timeouts on each individual buffer and pay the extra management overhead. Alternatively, perhaps it would be possible to ``pace'' files that have large numbers of dirty buffers associated with them. You could extend softdep_fsync() by adding an additional argument specifying the maximum number of buffers to write, so that when a large file times out, it can be staggered over several runs of the syncer (which is activated every second.) Kirk would be a good person to talk to, I think. By the way, can you see what effect lowering the following sysctls has? John Dyson reported that the default timeouts are well into diminishing returns for improving performance, and a few months ago I actually saw a small but measurable performance *increase* with the values below: kern.filedelay: 7 kern.dirdelay: 6 kern.metadelay: 5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 2: 1:48 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 80AAD37B401 for ; Mon, 10 Feb 2003 02:01:47 -0800 (PST) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id A141943F3F for ; Mon, 10 Feb 2003 02:01:46 -0800 (PST) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.6/8.12.6) with ESMTP id h1AA1jqV037780; Mon, 10 Feb 2003 11:01:45 +0100 (CET) (envelope-from phk@phk.freebsd.dk) To: David Schultz Cc: arch@FreeBSD.ORG Subject: Re: Our lemming-syncer caught in the act. From: phk@phk.freebsd.dk In-Reply-To: Your message of "Mon, 10 Feb 2003 01:50:40 PST." <20030210095040.GB5515@HAL9000.homeunix.com> Date: Mon, 10 Feb 2003 11:01:45 +0100 Message-ID: <37779.1044871305@critter.freebsd.dk> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20030210095040.GB5515@HAL9000.homeunix.com>, David Schultz writes: >Thus spake phk@phk.freebsd.dk : >> Yes, it makes a lot of sense. There is no point in batching up >> writes to the point of showing 200 requests off at once then >> wait 30 seconds, then do it again etc etc. >> >> We can and need to do better than that. > >It sounds like you're describing the old syncer. The new one >still shoves groups of buffers out at once, but on a file-by-file >basis rather than globally every 30 seconds. The plot is a short segment of a trace I made on a -current box yesterday and it shows actual performance of our current syncer(s). -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 3:38:50 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4CC2337B401 for ; Mon, 10 Feb 2003 03:38:49 -0800 (PST) Received: from unox.bitpit.net (t-23-57.athome.tue.nl [131.155.227.57]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8938843FA3 for ; Mon, 10 Feb 2003 03:38:45 -0800 (PST) (envelope-from marijn@bitpit.net) Received: from hoop.bitpit.net (unknown [192.168.1.2]) by unox.bitpit.net (Postfix) with ESMTP id 24B434B9 for ; Mon, 10 Feb 2003 12:38:44 +0100 (CET) Received: from hoop.bitpit.net (localhost [127.0.0.1]) by hoop.bitpit.net (8.12.6/8.12.6) with ESMTP id h1ABdTf2048014 for ; Mon, 10 Feb 2003 12:39:29 +0100 (CET) (envelope-from marijn@hoop.bitpit.net) Received: (from marijn@localhost) by hoop.bitpit.net (8.12.6/8.12.6/Submit) id h1ABdSrV048013 for arch@freebsd.org; Mon, 10 Feb 2003 12:39:28 +0100 (CET) Date: Mon, 10 Feb 2003 12:39:28 +0100 From: Marijn Meijles To: arch@freebsd.org Subject: Re: New scheduler problem Message-ID: <20030210113928.GA10552@hoop.bitpit.net> References: <20030206094040.GA2000@kevad.internal> <20030207211000.P72073-100000@mail.chesapeake.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030207211000.P72073-100000@mail.chesapeake.net> User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Jeff Roberson wrote: > On Thu, 6 Feb 2003, Vallo Kallaste wrote: > > > On Wed, Feb 05, 2003 at 09:12:36PM +0100, Marijn Meijles > > wrote: > > > > > I just tried the new scheduler (yesterdays cvsup), but it gives horrible > > > interactve performance in some situations. Especially with mixed interactive > > > and non-interactive applications like mozilla. This is perfectly > > > > Running two seti@home processes (2 CPU's) at nice 19 kills > > interactive peformance entirely. Worse than 300baud modem on the > > console, mouse will not work at all. SCHED_4BSD is fine. > > -- > > I botched it last weekend. I will fix it this weekend. I'll let you know > when it's ready to go again. I appreciate the exposure. With strict rescheduling turned on it runs a lot better. It is about as responsive as the 4BSD scheduler. -- Marijn@bitpit.net --- This line says This line says To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 3:38:58 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C009F37B401; Mon, 10 Feb 2003 03:38:57 -0800 (PST) Received: from melusine.cuivre.fr.eu.org (melusine.cuivre.fr.eu.org [62.212.105.185]) by mx1.FreeBSD.org (Postfix) with ESMTP id D58A843F85; Mon, 10 Feb 2003 03:38:56 -0800 (PST) (envelope-from thomas@FreeBSD.ORG) Received: by melusine.cuivre.fr.eu.org (Postfix, from userid 1000) id DB4212C3D2; Mon, 10 Feb 2003 12:38:55 +0100 (CET) Date: Mon, 10 Feb 2003 12:38:55 +0100 From: Thomas Quinot To: phk@freebsd.org Cc: arch@freebsd.org Subject: Re: GEOM and CDROM media (was: CFR: Volume labels in FFS) Message-ID: <20030210113855.GA90800@melusine.cuivre.fr.eu.org> Reply-To: Thomas Quinot References: <20030124222718.GN53114@roark.gnf.org> <4614.1043447912@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <4614.1043447912@critter.freebsd.dk> User-Agent: Mutt/1.4i X-message-flag: WARNING! Using Outlook can damage your computer. Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Le 2003-01-24, phk@freebsd.org crivait : > ATAPI CD's do neat tricks and present each track as a (cloned) device > so you can do things like /dev/acd0t05 etc. I think this is the way to go. This scheme has the advandatage of providing a neat mapping of device names <-> actual tracks on the CD. All the presentation magic (multisession, audio) can be implemented as supplementary layers on top of that. Thomas. -- Thomas.Quinot@Cuivre.FR.EU.ORG To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 3:47:50 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 00F6637B401 for ; Mon, 10 Feb 2003 03:47:50 -0800 (PST) Received: from axl.seasidesoftware.co.za (axl.seasidesoftware.co.za [196.31.7.201]) by mx1.FreeBSD.org (Postfix) with ESMTP id 789FE43F3F for ; Mon, 10 Feb 2003 03:47:47 -0800 (PST) (envelope-from sheldonh@starjuice.net) Received: from sheldonh by axl.seasidesoftware.co.za with local (Exim 4.12) id 18iCPU-00039h-00; Mon, 10 Feb 2003 13:47:40 +0200 Date: Mon, 10 Feb 2003 13:47:40 +0200 From: Sheldon Hearn To: phk@phk.freebsd.dk Cc: arch@freebsd.org Subject: Re: Indiscriminately installing all .h files in /usr/include/* Message-ID: <20030210114740.GG5355@starjuice.net> Mail-Followup-To: phk@phk.freebsd.dk, arch@freebsd.org References: <200302081325.h18DPmrB010982@repoman.freebsd.org> <18902.1044711101@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <18902.1044711101@critter.freebsd.dk> User-Agent: Mutt/1.5.3i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On (2003/02/08 14:31), phk@phk.freebsd.dk wrote: > Not being the kind of guy who casually walks through /usr/include > once in a while, I only now just discovered that "make installincludes" > indiscriminately installs all .h files under /usr/include. > > I understand the desirablity of simple makefile magic for this task, > but couldn't we agree on some sort of magic-marker to put in files > we want or don't want installed in /usr/include ? Could you give some specific examples of files you wouldn't want installed by this target? Ciao, Sheldon. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 3:49:33 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 454A337B401 for ; Mon, 10 Feb 2003 03:49:32 -0800 (PST) Received: from melusine.cuivre.fr.eu.org (melusine.cuivre.fr.eu.org [62.212.105.185]) by mx1.FreeBSD.org (Postfix) with ESMTP id A779243F3F for ; Mon, 10 Feb 2003 03:49:31 -0800 (PST) (envelope-from thomas@freebsd.org) Received: by melusine.cuivre.fr.eu.org (Postfix, from userid 1000) id 7AD9F2C3D3; Mon, 10 Feb 2003 12:49:30 +0100 (CET) Date: Mon, 10 Feb 2003 12:49:30 +0100 From: Thomas Quinot To: arch@freebsd.org Subject: syslog.conf syntax change (multiple program/host specifications) Message-ID: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4i X-message-flag: WARNING! Using Outlook can damage your computer. Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I'm about to commit a change to syslogd to allow multiple hosts or programs to be named in program or host specifications, eg: !foo,bar *.* /var/log/only_foo_or_bar.log !-foo,bar *.* /var/log/all_except_foo_or_bar.log Any objections to this change? A patch is available for review at http://www.cuivre.fr.eu.org/~thomas/syslogd.c.diff. Thomas. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 4: 1:33 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0B04B37B401 for ; Mon, 10 Feb 2003 04:01:32 -0800 (PST) Received: from bluejay.mail.pas.earthlink.net (bluejay.mail.pas.earthlink.net [207.217.120.218]) by mx1.FreeBSD.org (Postfix) with ESMTP id 759BA43F93 for ; Mon, 10 Feb 2003 04:01:31 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0003.cvx22-bradley.dialup.earthlink.net ([209.179.198.3] helo=mindspring.com) by bluejay.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18iCcq-0007Kx-00; Mon, 10 Feb 2003 04:01:29 -0800 Message-ID: <3E479440.D89E90F5@mindspring.com> Date: Mon, 10 Feb 2003 04:00:00 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: phk@phk.freebsd.dk Cc: David Schultz , arch@FreeBSD.ORG Subject: Re: Our lemming-syncer caught in the act. References: <37473.1044868995@critter.freebsd.dk> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4fd7664590b0d84904e7a5b02e442166fa8438e0f32a48e08350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG phk@phk.freebsd.dk wrote: > In message <20030210091317.GD5165@HAL9000.homeunix.com>, David Schultz writes: > >When a large file times out, a significant amount of I/O can be > >generated. This is still far better than the old syncer that > >flushed everything every 30 seconds. The reasons for this > >behavior are explained in src/sys/ufs/ffs/README. After reading > >that, do you still think it makes sense to try to do better? > > Yes, it makes a lot of sense. There is no point in batching up > writes to the point of showing 200 requests off at once then > wait 30 seconds, then do it again etc etc. > > We can and need to do better than that. Are there any statistics on how many requests are prevented by soft updates? Maybe you are really talking about a value that would be best expressed as a ratio? It seems to me that this would be a necessary part of any changes made on the basis of instrumentation that might decrease the interval, but increase the absolute number of requests, as a result. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 4: 6:11 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4CA0A37B401 for ; Mon, 10 Feb 2003 04:06:10 -0800 (PST) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5FD2243F3F for ; Mon, 10 Feb 2003 04:06:09 -0800 (PST) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.6/8.12.6) with ESMTP id h1AC65qV038937; Mon, 10 Feb 2003 13:06:07 +0100 (CET) (envelope-from phk@phk.freebsd.dk) To: Sheldon Hearn Cc: arch@freebsd.org Subject: Re: Indiscriminately installing all .h files in /usr/include/* From: phk@phk.freebsd.dk In-Reply-To: Your message of "Mon, 10 Feb 2003 13:47:40 +0200." <20030210114740.GG5355@starjuice.net> Date: Mon, 10 Feb 2003 13:06:05 +0100 Message-ID: <38936.1044878765@critter.freebsd.dk> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20030210114740.GG5355@starjuice.net>, Sheldon Hearn writes: >On (2003/02/08 14:31), phk@phk.freebsd.dk wrote: > >> Not being the kind of guy who casually walks through /usr/include >> once in a while, I only now just discovered that "make installincludes" >> indiscriminately installs all .h files under /usr/include. >> >> I understand the desirablity of simple makefile magic for this task, >> but couldn't we agree on some sort of magic-marker to put in files >> we want or don't want installed in /usr/include ? > >Could you give some specific examples of files you wouldn't want >installed by this target? geom/geom.h for starters. Other examples: cam/cam_extend.h cam/cam_periph.h cam/cam_queue.h etc etc -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 10:35:19 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CC96737B401 for ; Mon, 10 Feb 2003 10:35:17 -0800 (PST) Received: from canning.wemm.org (canning.wemm.org [192.203.228.65]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6F8FC43FA3 for ; Mon, 10 Feb 2003 10:35:17 -0800 (PST) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by canning.wemm.org (Postfix) with ESMTP id 5F0052A8BB; Mon, 10 Feb 2003 10:35:17 -0800 (PST) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: phk@phk.freebsd.dk Cc: Sheldon Hearn , arch@freebsd.org Subject: Re: Indiscriminately installing all .h files in /usr/include/* In-Reply-To: <38936.1044878765@critter.freebsd.dk> Date: Mon, 10 Feb 2003 10:35:17 -0800 From: Peter Wemm Message-Id: <20030210183517.5F0052A8BB@canning.wemm.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG phk@phk.freebsd.dk wrote: > In message <20030210114740.GG5355@starjuice.net>, Sheldon Hearn writes: > >On (2003/02/08 14:31), phk@phk.freebsd.dk wrote: > > > >> Not being the kind of guy who casually walks through /usr/include > >> once in a while, I only now just discovered that "make installincludes" > >> indiscriminately installs all .h files under /usr/include. > >> > >> I understand the desirablity of simple makefile magic for this task, > >> but couldn't we agree on some sort of magic-marker to put in files > >> we want or don't want installed in /usr/include ? > > > >Could you give some specific examples of files you wouldn't want > >installed by this target? > > geom/geom.h for starters. > > Other examples: > cam/cam_extend.h > cam/cam_periph.h > cam/cam_queue.h I've been wondering if we need an explicit list of files to install for a while. We've got lots of junk in /usr/include/dev/* that really shouldn't be there. The only stuff that should be there are things to define sysctl, ioctl etc user interfaces. Things like hardware register definitions and random driver internals definitions do not really belong there. /usr/include/usb/* is a good example. We do similar bad things for the file systems. Cheers, -Peter -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 10:49:36 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 46D8337B401 for ; Mon, 10 Feb 2003 10:49:35 -0800 (PST) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id C0C1143FAF for ; Mon, 10 Feb 2003 10:49:34 -0800 (PST) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id 7E2C2AE25C; Mon, 10 Feb 2003 10:49:34 -0800 (PST) Date: Mon, 10 Feb 2003 10:49:34 -0800 From: Alfred Perlstein To: Peter Wemm Cc: phk@phk.freebsd.dk, Sheldon Hearn , arch@freebsd.org Subject: Re: Indiscriminately installing all .h files in /usr/include/* Message-ID: <20030210184934.GW88781@elvis.mu.org> References: <38936.1044878765@critter.freebsd.dk> <20030210183517.5F0052A8BB@canning.wemm.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030210183517.5F0052A8BB@canning.wemm.org> User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Peter Wemm [030210 10:35] wrote: > phk@phk.freebsd.dk wrote: > > In message <20030210114740.GG5355@starjuice.net>, Sheldon Hearn writes: > > >On (2003/02/08 14:31), phk@phk.freebsd.dk wrote: > > > > > >> Not being the kind of guy who casually walks through /usr/include > > >> once in a while, I only now just discovered that "make installincludes" > > >> indiscriminately installs all .h files under /usr/include. > > >> > > >> I understand the desirablity of simple makefile magic for this task, > > >> but couldn't we agree on some sort of magic-marker to put in files > > >> we want or don't want installed in /usr/include ? > > > > > >Could you give some specific examples of files you wouldn't want > > >installed by this target? > > > > geom/geom.h for starters. > > > > Other examples: > > cam/cam_extend.h > > cam/cam_periph.h > > cam/cam_queue.h > > I've been wondering if we need an explicit list of files to install for > a while. We've got lots of junk in /usr/include/dev/* that really shouldn't > be there. The only stuff that should be there are things to define > sysctl, ioctl etc user interfaces. Things like hardware register definitions > and random driver internals definitions do not really belong there. > /usr/include/usb/* is a good example. We do similar bad things for the > file systems. If there was a way to do it via cpp defines inside the header that would be nice. -- -Alfred Perlstein [alfred@freebsd.org] 'Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom.' To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 12:45:32 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 30C7937B401 for ; Mon, 10 Feb 2003 12:45:31 -0800 (PST) Received: from brev.stud.ntnu.no (brev.stud.ntnu.no [129.241.56.70]) by mx1.FreeBSD.org (Postfix) with ESMTP id ED82943FBD for ; Mon, 10 Feb 2003 12:45:27 -0800 (PST) (envelope-from morten@rodal.no) Received: from localhost (localhost [127.0.0.1]) by brev.stud.ntnu.no (Postfix) with ESMTP id 90B9713EE35; Mon, 10 Feb 2003 21:45:26 +0100 (CET) Received: from slurp.rodal.no (m200h.studby.ntnu.no [129.241.135.200]) by brev.stud.ntnu.no (Postfix) with ESMTP id A9D5713EE27; Mon, 10 Feb 2003 21:45:25 +0100 (CET) Received: (from morten@localhost) by slurp.rodal.no (8.12.6/8.12.6/Submit) id h1AKjOCt018340; Mon, 10 Feb 2003 21:45:24 +0100 (CET) (envelope-from morten) Date: Mon, 10 Feb 2003 21:45:24 +0100 From: Morten Rodal To: phk@phk.freebsd.dk Cc: David Schultz , arch@FreeBSD.ORG Subject: Re: Our lemming-syncer caught in the act. Message-ID: <20030210204523.GF12240@slurp.rodal.no> References: <20030210091317.GD5165@HAL9000.homeunix.com> <37473.1044868995@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <37473.1044868995@critter.freebsd.dk> X-Virus-Scanned: by AMaViS perl-11 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, Feb 10, 2003 at 10:23:15AM +0100, phk@phk.freebsd.dk wrote: > In message <20030210091317.GD5165@HAL9000.homeunix.com>, David Schultz writes: > > >When a large file times out, a significant amount of I/O can be > >generated. This is still far better than the old syncer that > >flushed everything every 30 seconds. The reasons for this > >behavior are explained in src/sys/ufs/ffs/README. After reading > >that, do you still think it makes sense to try to do better? > > Yes, it makes a lot of sense. There is no point in batching up > writes to the point of showing 200 requests off at once then > wait 30 seconds, then do it again etc etc. > > We can and need to do better than that. > I am very sorry to pitch in on your discussion here, but I feel that this needs a little light. It should probably have been posted to -current as an own thread, but I felt that since you guys where already discussing the syncer I'd tell my tale. I upgraded my workstation to 5.0-RELEASE shortly after it became available, and there is one thing that keeps bugging me. When I delete a file that is large, say >600MB, the system performance and interactivity will drop remarkebly. I mean, after a little while (probably when the/syncer kicks in), the mouse starts to lag in X. I can type several letters before they appear, and xmms starts to skip while playing audio. Using top with the -S flag I can see syncer eating a lot of CPU and as soon as the syncer is done doing whatever it does everything goes back to normal. After I became aware of this syncer thingy I found out that whenever I hear skips in the music it is because the syncer uses CPU (according to top). Of couse this could be that I'm just using a shitty computer (it's an old dual p2-300mhz), but I never experienced this with 4.X series. As for the installation it's pretty standard. Softupdates on all mount points except root, no sysctl overrides or anything of that kind. This is, at least yet, the only thing that is really bugging me about 5.0. A fix for this would be really appreciated. -- Morten Rodal To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 12:54:44 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 05E4137B401 for ; Mon, 10 Feb 2003 12:54:44 -0800 (PST) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8536F43FCB for ; Mon, 10 Feb 2003 12:54:43 -0800 (PST) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id 68650AE2BE; Mon, 10 Feb 2003 12:54:43 -0800 (PST) Date: Mon, 10 Feb 2003 12:54:43 -0800 From: Alfred Perlstein To: Morten Rodal Cc: phk@phk.freebsd.dk, David Schultz , arch@FreeBSD.ORG Subject: Re: Our lemming-syncer caught in the act. Message-ID: <20030210205443.GA88781@elvis.mu.org> References: <20030210091317.GD5165@HAL9000.homeunix.com> <37473.1044868995@critter.freebsd.dk> <20030210204523.GF12240@slurp.rodal.no> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030210204523.GF12240@slurp.rodal.no> User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Morten Rodal [030210 12:45] wrote: > > I upgraded my workstation to 5.0-RELEASE shortly after it became > available, and there is one thing that keeps bugging me. When I > delete a file that is large, say >600MB, the system performance and > interactivity will drop remarkebly. This happens on my machine as well, a complete shot in the dark fix that I would try is to find where syncer loops and purposefully have it drop and pick up Giant between batches of work. -Alfred To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 14: 9:17 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A02D037B401 for ; Mon, 10 Feb 2003 14:09:15 -0800 (PST) Received: from canning.wemm.org (canning.wemm.org [192.203.228.65]) by mx1.FreeBSD.org (Postfix) with ESMTP id E7E904400B for ; Mon, 10 Feb 2003 14:09:05 -0800 (PST) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by canning.wemm.org (Postfix) with ESMTP id D5CA32A8C2; Mon, 10 Feb 2003 14:09:05 -0800 (PST) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: Alfred Perlstein Cc: phk@phk.freebsd.dk, arch@freebsd.org Subject: Re: Indiscriminately installing all .h files in /usr/include/* In-Reply-To: <20030210184934.GW88781@elvis.mu.org> Date: Mon, 10 Feb 2003 14:09:05 -0800 From: Peter Wemm Message-Id: <20030210220905.D5CA32A8C2@canning.wemm.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Alfred Perlstein wrote: > * Peter Wemm [030210 10:35] wrote: > > phk@phk.freebsd.dk wrote: > > > In message <20030210114740.GG5355@starjuice.net>, Sheldon Hearn writes: > > > >On (2003/02/08 14:31), phk@phk.freebsd.dk wrote: > > > > > > > >> Not being the kind of guy who casually walks through /usr/include > > > >> once in a while, I only now just discovered that "make installincludes " > > > >> indiscriminately installs all .h files under /usr/include. > > > >> > > > >> I understand the desirablity of simple makefile magic for this task, > > > >> but couldn't we agree on some sort of magic-marker to put in files > > > >> we want or don't want installed in /usr/include ? > > > > > > > >Could you give some specific examples of files you wouldn't want > > > >installed by this target? > > > > > > geom/geom.h for starters. > > > > > > Other examples: > > > cam/cam_extend.h > > > cam/cam_periph.h > > > cam/cam_queue.h > > > > I've been wondering if we need an explicit list of files to install for > > a while. We've got lots of junk in /usr/include/dev/* that really shouldn' t > > be there. The only stuff that should be there are things to define > > sysctl, ioctl etc user interfaces. Things like hardware register definitio ns > > and random driver internals definitions do not really belong there. > > /usr/include/usb/* is a good example. We do similar bad things for the > > file systems. > > If there was a way to do it via cpp defines inside the header that would > be nice. Something like this perhaps? ^/* EXPORT_HEADER */$ Cheers, -Peter -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 14:15: 3 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 221FB37B40F for ; Mon, 10 Feb 2003 14:15:02 -0800 (PST) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id B96A743F75 for ; Mon, 10 Feb 2003 14:15:01 -0800 (PST) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id 98CFDAE2C1; Mon, 10 Feb 2003 14:15:01 -0800 (PST) Date: Mon, 10 Feb 2003 14:15:01 -0800 From: Alfred Perlstein To: Peter Wemm Cc: phk@phk.freebsd.dk, arch@freebsd.org Subject: Re: Indiscriminately installing all .h files in /usr/include/* Message-ID: <20030210221501.GF88781@elvis.mu.org> References: <20030210184934.GW88781@elvis.mu.org> <20030210220905.D5CA32A8C2@canning.wemm.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030210220905.D5CA32A8C2@canning.wemm.org> User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Peter Wemm [030210 14:09] wrote: > > > > If there was a way to do it via cpp defines inside the header that would > > be nice. > > Something like this perhaps? > ^/* EXPORT_HEADER */$ Yes, something like that. I know of one company that brackets the headers with $COMPANY_PRIVATE for stuff that shouldn't be visible in /usr/include. if you wind up with an empty file it isn't even installed. -- -Alfred Perlstein [alfred@freebsd.org] 'Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom.' To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 14:25:50 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6490F37B405 for ; Mon, 10 Feb 2003 14:25:48 -0800 (PST) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1C78543FAF for ; Mon, 10 Feb 2003 14:25:47 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.12.6/8.12.6) with ESMTP id h1AMPkSJ023703; Mon, 10 Feb 2003 14:25:46 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.6/8.12.6/Submit) id h1AMPkTE023700; Mon, 10 Feb 2003 14:25:46 -0800 (PST) Date: Mon, 10 Feb 2003 14:25:46 -0800 (PST) From: Matthew Dillon Message-Id: <200302102225.h1AMPkTE023700@apollo.backplane.com> To: Alfred Perlstein Cc: Morten Rodal , phk@phk.freebsd.dk, David Schultz , arch@FreeBSD.ORG Subject: Re: Our lemming-syncer caught in the act. References: <20030210091317.GD5165@HAL9000.homeunix.com> <37473.1044868995@critter.freebsd.dk> <20030210204523.GF12240@slurp.rodal.no> <20030210205443.GA88781@elvis.mu.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :* Morten Rodal [030210 12:45] wrote: :> :> I upgraded my workstation to 5.0-RELEASE shortly after it became :> available, and there is one thing that keeps bugging me. When I :> delete a file that is large, say >600MB, the system performance and :> interactivity will drop remarkebly. : :This happens on my machine as well, a complete shot in the dark fix :that I would try is to find where syncer loops and purposefully have :it drop and pick up Giant between batches of work. : :-Alfred Try reproducing the problem with kernel profiling turned on. The syncer is not likely to be the cause of the large file removal issue, since that involves only bitmaps. The cause is more likely to be softupdate's bitmap dependancies. Try turning off softupdates and see if that improves performance. Softupdates creates some rather nasty buffer cache issues due to the number of buffers it removes from management. Also, dependancies prevent the buffer cache from being able to flush a buffer. The buffer cache was not really designed to deal with flushes which don't undirty a buffer or large numbers of buffers being temporarily removed from management. The syncer is more likely to be the cause of performance issues during large file data operations. Because the syncer holds onto the vnode lock you can get into a situation where lookup()s cause a directory locking chain to reach the root directory which has the effect of stalling just about everything. Either Net or OpenBSD solved this problem by doing partial fsyncs instead of full fsyncs to avoid holding onto the vnode lock for insanely long periods of time. Another solution would be to use the namei cache to lock path names from create/remove/rename ops so the directory vnodes do not have to be locked for the duration of a path lookup. If the directory vnode is not exclusively locked then a race-to-root will not happen due to some random file vnode being locked for a long period of time. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 14:26: 5 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DADFF37B401 for ; Mon, 10 Feb 2003 14:26:04 -0800 (PST) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id A32B543F85 for ; Mon, 10 Feb 2003 14:26:03 -0800 (PST) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.6/8.12.6) with ESMTP id h1AMPxqV043625; Mon, 10 Feb 2003 23:26:00 +0100 (CET) (envelope-from phk@phk.freebsd.dk) To: Alfred Perlstein Cc: Peter Wemm , arch@freebsd.org Subject: Re: Indiscriminately installing all .h files in /usr/include/* From: phk@phk.freebsd.dk In-Reply-To: Your message of "Mon, 10 Feb 2003 14:15:01 PST." <20030210221501.GF88781@elvis.mu.org> Date: Mon, 10 Feb 2003 23:25:59 +0100 Message-ID: <43624.1044915959@critter.freebsd.dk> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20030210221501.GF88781@elvis.mu.org>, Alfred Perlstein writes: >* Peter Wemm [030210 14:09] wrote: >> > >> > If there was a way to do it via cpp defines inside the header that would >> > be nice. >> >> Something like this perhaps? >> ^/* EXPORT_HEADER */$ > >Yes, something like that. I know of one company that brackets the >headers with $COMPANY_PRIVATE for stuff that shouldn't be visible in >/usr/include. if you wind up with an empty file it isn't even installed. I would prefer labeling the stuff that should go into /usr/include, it shouldn't be the default to export stuff unless you ask for it. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 14:34:51 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A3A3937B401 for ; Mon, 10 Feb 2003 14:34:50 -0800 (PST) Received: from ebb.errno.com (ebb.errno.com [66.127.85.87]) by mx1.FreeBSD.org (Postfix) with ESMTP id 400A343FAF for ; Mon, 10 Feb 2003 14:34:45 -0800 (PST) (envelope-from sam@errno.com) Received: from melange (melange.errno.com [66.127.85.82]) (authenticated bits=0) by ebb.errno.com (8.12.5/8.12.1) with ESMTP id h1AMYfnN093202 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=NO); Mon, 10 Feb 2003 14:34:44 -0800 (PST)?g (envelope-from sam@errno.com) X-Authentication-Warning: ebb.errno.com: Host melange.errno.com [66.127.85.82] claimed to be melange Message-ID: <143501c2d154$9c3c70e0$52557f42@errno.com> From: "Sam Leffler" To: "Alfred Perlstein" , Cc: "Peter Wemm" , References: <43624.1044915959@critter.freebsd.dk> Subject: Re: Indiscriminately installing all .h files in /usr/include/* Date: Mon, 10 Feb 2003 14:34:11 -0800 Organization: Errno Consulting MIME-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > In message <20030210221501.GF88781@elvis.mu.org>, Alfred Perlstein writes: > >* Peter Wemm [030210 14:09] wrote: > >> > > >> > If there was a way to do it via cpp defines inside the header that would > >> > be nice. > >> > >> Something like this perhaps? > >> ^/* EXPORT_HEADER */$ > > > >Yes, something like that. I know of one company that brackets the > >headers with $COMPANY_PRIVATE for stuff that shouldn't be visible in > >/usr/include. if you wind up with an empty file it isn't even installed. > > I would prefer labeling the stuff that should go into /usr/include, > it shouldn't be the default to export stuff unless you ask for it. Sticking this stuff inside include files means modifying imported files. I'd actually prefer a mechanism like this to lurk in the Makefiles as you can fully encapsulate requirements like the /usr/include/ is different than the source directory. Sam To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 14:39: 9 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 21AB337B407 for ; Mon, 10 Feb 2003 14:39:07 -0800 (PST) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 96F5243FE1 for ; Mon, 10 Feb 2003 14:39:04 -0800 (PST) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id 4B415AE162; Mon, 10 Feb 2003 14:39:04 -0800 (PST) Date: Mon, 10 Feb 2003 14:39:04 -0800 From: Alfred Perlstein To: Matthew Dillon Cc: Morten Rodal , phk@phk.freebsd.dk, David Schultz , arch@FreeBSD.ORG Subject: Re: Our lemming-syncer caught in the act. Message-ID: <20030210223904.GG88781@elvis.mu.org> References: <20030210091317.GD5165@HAL9000.homeunix.com> <37473.1044868995@critter.freebsd.dk> <20030210204523.GF12240@slurp.rodal.no> <20030210205443.GA88781@elvis.mu.org> <200302102225.h1AMPkTE023700@apollo.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200302102225.h1AMPkTE023700@apollo.backplane.com> User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Matthew Dillon [030210 14:25] wrote: > > Try reproducing the problem with kernel profiling turned on. The > syncer is not likely to be the cause of the large file removal issue, > since that involves only bitmaps. The cause is more likely to be > softupdate's bitmap dependancies. Try turning off softupdates and see > if that improves performance. Softupdates creates some rather nasty > buffer cache issues due to the number of buffers it removes from > management. Also, dependancies prevent the buffer cache from being > able to flush a buffer. The buffer cache was not really designed to > deal with flushes which don't undirty a buffer or large numbers of > buffers being temporarily removed from management. > > The syncer is more likely to be the cause of performance issues > during large file data operations. Because the syncer holds onto the > vnode lock you can get into a situation where lookup()s cause a directory > locking chain to reach the root directory which has the effect of stalling > just about everything. Either Net or OpenBSD solved this problem by > doing partial fsyncs instead of full fsyncs to avoid holding onto the > vnode lock for insanely long periods of time. Another solution would > be to use the namei cache to lock path names from create/remove/rename > ops so the directory vnodes do not have to be locked for the duration > of a path lookup. If the directory vnode is not exclusively locked then > a race-to-root will not happen due to some random file vnode being locked > for a long period of time. There are several good points in your mail, especially about performing partial fsyncs, however this doesn't really explain why 4.x is "ok" versus 5.x having issues. The only explanation I can think of is that the syncer runs with giant for a long time but under 4.x only needs splbio() for short periods. I don't have a 5.x box handy here, but if someone could run this patch and let me know what it does I would appreciate it. cd /usr/src/sys/kern && patch < path_to_this_file cvs server: Diffing . Index: vfs_subr.c =================================================================== RCS file: /home/ncvs/src/sys/kern/vfs_subr.c,v retrieving revision 1.427 diff -u -r1.427 vfs_subr.c --- vfs_subr.c 21 Jan 2003 08:55:55 -0000 1.427 +++ vfs_subr.c 10 Feb 2003 22:38:31 -0000 @@ -1725,6 +1725,11 @@ VOP_UNLOCK(vp, 0, td); vn_finished_write(mp); } +#ifdef other_premption_did_not_work + /* XXX: pre-emption point to avoid Giant hogging. */ + mtx_unlock(&Giant); + mtx_lock(&Giant); +#endif s = splbio(); mtx_lock(&sync_mtx); if (LIST_FIRST(slp) == vp) { @@ -1775,6 +1780,9 @@ if (rushjob > 0) { rushjob -= 1; mtx_unlock(&sync_mtx); + /* XXX: pre-emption point to avoid Giant hogging. */ + mtx_unlock(&Giant); + mtx_lock(&Giant); continue; } mtx_unlock(&sync_mtx); -- -Alfred Perlstein [alfred@freebsd.org] 'Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom.' To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 14:40:21 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2962A37B401 for ; Mon, 10 Feb 2003 14:40:20 -0800 (PST) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6B57D43F93 for ; Mon, 10 Feb 2003 14:40:16 -0800 (PST) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id 476B9AE165; Mon, 10 Feb 2003 14:40:16 -0800 (PST) Date: Mon, 10 Feb 2003 14:40:16 -0800 From: Alfred Perlstein To: Sam Leffler Cc: phk@phk.freebsd.dk, Peter Wemm , arch@FreeBSD.ORG Subject: Re: Indiscriminately installing all .h files in /usr/include/* Message-ID: <20030210224016.GH88781@elvis.mu.org> References: <43624.1044915959@critter.freebsd.dk> <143501c2d154$9c3c70e0$52557f42@errno.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <143501c2d154$9c3c70e0$52557f42@errno.com> User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG * Sam Leffler [030210 14:34] wrote: > > > > I would prefer labeling the stuff that should go into /usr/include, > > it shouldn't be the default to export stuff unless you ask for it. > > Sticking this stuff inside include files means modifying imported files. > I'd actually prefer a mechanism like this to lurk in the Makefiles as you > can fully encapsulate requirements like the /usr/include/ is different > than the source directory. That makes sense. -Alfred To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 14:44:11 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8DE8937B406 for ; Mon, 10 Feb 2003 14:44:09 -0800 (PST) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9524743FFB for ; Mon, 10 Feb 2003 14:43:34 -0800 (PST) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.6/8.12.6) with ESMTP id h1AMhSqV043867; Mon, 10 Feb 2003 23:43:28 +0100 (CET) (envelope-from phk@phk.freebsd.dk) To: Alfred Perlstein Cc: Sam Leffler , Peter Wemm , arch@FreeBSD.ORG Subject: Re: Indiscriminately installing all .h files in /usr/include/* From: phk@phk.freebsd.dk In-Reply-To: Your message of "Mon, 10 Feb 2003 14:40:16 PST." <20030210224016.GH88781@elvis.mu.org> Date: Mon, 10 Feb 2003 23:43:28 +0100 Message-ID: <43866.1044917008@critter.freebsd.dk> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20030210224016.GH88781@elvis.mu.org>, Alfred Perlstein writes: >* Sam Leffler [030210 14:34] wrote: >> > >> > I would prefer labeling the stuff that should go into /usr/include, >> > it shouldn't be the default to export stuff unless you ask for it. >> >> Sticking this stuff inside include files means modifying imported files. >> I'd actually prefer a mechanism like this to lurk in the Makefiles as you >> can fully encapsulate requirements like the /usr/include/ is different >> than the source directory. > >That makes sense. I can live with that too, as long as the default is "don't install". -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 14:46:19 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0999937B401 for ; Mon, 10 Feb 2003 14:46:18 -0800 (PST) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id B080F43FCB for ; Mon, 10 Feb 2003 14:46:16 -0800 (PST) (envelope-from arr@watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.6/8.12.5) with ESMTP id h1AMk4P3064085; Mon, 10 Feb 2003 17:46:04 -0500 (EST) (envelope-from arr@watson.org) Received: from localhost (arr@localhost) by fledge.watson.org (8.12.6/8.12.6/Submit) with SMTP id h1AMk3KA064082; Mon, 10 Feb 2003 17:46:03 -0500 (EST) X-Authentication-Warning: fledge.watson.org: arr owned process doing -bs Date: Mon, 10 Feb 2003 17:46:03 -0500 (EST) From: "Andrew R. Reiter" To: Sam Leffler Cc: Alfred Perlstein , phk@phk.freebsd.dk, Peter Wemm , arch@FreeBSD.ORG Subject: Re: Indiscriminately installing all .h files in /usr/include/* In-Reply-To: <143501c2d154$9c3c70e0$52557f42@errno.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, 10 Feb 2003, Sam Leffler wrote: :> In message <20030210221501.GF88781@elvis.mu.org>, Alfred Perlstein writes: :> >* Peter Wemm [030210 14:09] wrote: :> >> > :> >> > If there was a way to do it via cpp defines inside the header that :would :> >> > be nice. :> >> :> >> Something like this perhaps? :> >> ^/* EXPORT_HEADER */$ :> > :> >Yes, something like that. I know of one company that brackets the :> >headers with $COMPANY_PRIVATE for stuff that shouldn't be visible in :> >/usr/include. if you wind up with an empty file it isn't even installed. :> :> I would prefer labeling the stuff that should go into /usr/include, :> it shouldn't be the default to export stuff unless you ask for it. : :Sticking this stuff inside include files means modifying imported files. :I'd actually prefer a mechanism like this to lurk in the Makefiles as you :can fully encapsulate requirements like the /usr/include/ is different :than the source directory. I hate adding more files to complicate matters, but it would nice to have some sort of template that could be used to generate where files ended up. Sort of like fsdef (file system definition) file does with fsgen. Not sure if it's exactly what's wanted for here, but it's just what came to mind when this "issue" came up. Cheers, Andrew -- Andrew R. Reiter arr@watson.org arr@FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 14:54:15 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1D30E37B401 for ; Mon, 10 Feb 2003 14:54:14 -0800 (PST) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6222A43F3F for ; Mon, 10 Feb 2003 14:54:07 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.12.6/8.12.6) with ESMTP id h1AMs7SJ023932; Mon, 10 Feb 2003 14:54:07 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.6/8.12.6/Submit) id h1AMs7ra023931; Mon, 10 Feb 2003 14:54:07 -0800 (PST) Date: Mon, 10 Feb 2003 14:54:07 -0800 (PST) From: Matthew Dillon Message-Id: <200302102254.h1AMs7ra023931@apollo.backplane.com> To: Alfred Perlstein Cc: Morten Rodal , phk@phk.freebsd.dk, David Schultz , arch@FreeBSD.ORG Subject: Re: Our lemming-syncer caught in the act. References: <20030210091317.GD5165@HAL9000.homeunix.com> <37473.1044868995@critter.freebsd.dk> <20030210204523.GF12240@slurp.rodal.no> <20030210205443.GA88781@elvis.mu.org> <200302102225.h1AMPkTE023700@apollo.backplane.com> <20030210223904.GG88781@elvis.mu.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :There are several good points in your mail, especially about performing :partial fsyncs, however this doesn't really explain why 4.x is "ok" versus :5.x having issues. The only explanation I can think of is that the syncer :runs with giant for a long time but under 4.x only needs splbio() for short :periods. The syncer tends to block quite a bit under normal operation, so it seems unlikely that Giant would be an issue unless the syncer is looping without getting any work accomplished. *THIS* is possible, since we've already had to hack the syncer considerably to prevent it from looping on unflushable softupdates buffers. There could be new cases that have not been addressed. If I recall I have made one change in the last few months which might be worth looking at. dillon 2002/12/28 13:03:42 PST Modified files: sys/vm vm_object.c vm_pager.h vnode_pager.c Log: Allow the VM object flushing code to cluster. When the filesystem syncer comes along and flushes a file which has been mmap()'d SHARED/RW, with dirty pages, it was flushing the underlying VM object asynchronously, resulting in thousands of 8K writes. With this change the VM Object flushing code will cluster dirty pages in 64K blocks. Note that until the low memory deadlock issue is reviewed, it is not safe to allow the pageout daemon to use this feature. Forced pageouts still use fs block size'd ops for the moment. MFC after: 3 days Revision Changes Path 1.250 +10 -3 src/sys/vm/vm_object.c 1.37 +4 -2 src/sys/vm/vm_pager.h 1.165 +8 -2 src/sys/vm/vnode_pager.c But, as you can see, it is in -stable as well. I doubt the above could be the cause. Profiling the problem in action might give us some more clues. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 15: 3:42 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DB7D437B401 for ; Mon, 10 Feb 2003 15:03:40 -0800 (PST) Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by mx1.FreeBSD.org (Postfix) with ESMTP id 64F7943FEA for ; Mon, 10 Feb 2003 15:03:40 -0800 (PST) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by rwcrmhc52.attbi.com (rwcrmhc52) with ESMTP id <2003021023033905200jv1p1e>; Mon, 10 Feb 2003 23:03:39 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id PAA51114; Mon, 10 Feb 2003 15:03:38 -0800 (PST) Date: Mon, 10 Feb 2003 15:03:37 -0800 (PST) From: Julian Elischer To: phk@phk.freebsd.dk Cc: Alfred Perlstein , Sam Leffler , Peter Wemm , arch@FreeBSD.ORG Subject: Re: Indiscriminately installing all .h files in /usr/include/* In-Reply-To: <43866.1044917008@critter.freebsd.dk> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, 10 Feb 2003 phk@phk.freebsd.dk wrote: > In message <20030210224016.GH88781@elvis.mu.org>, Alfred Perlstein writes: > >* Sam Leffler [030210 14:34] wrote: > >> > > >> > I would prefer labeling the stuff that should go into /usr/include, > >> > it shouldn't be the default to export stuff unless you ask for it. > >> > >> Sticking this stuff inside include files means modifying imported files. > >> I'd actually prefer a mechanism like this to lurk in the Makefiles as you > >> can fully encapsulate requirements like the /usr/include/ is different > >> than the source directory. > > > >That makes sense. > > I can live with that too, as long as the default is "don't install". How about "Don't install if the variable INSTALLABLE_INCLUDES is defined and the file is not listed." That would give us a transition from here to there.. untouched Makefile would act as they do now.. > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 17:41:14 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DA98A37B405 for ; Mon, 10 Feb 2003 17:41:12 -0800 (PST) Received: from heron.mail.pas.earthlink.net (heron.mail.pas.earthlink.net [207.217.120.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id DA4AC43FCB for ; Mon, 10 Feb 2003 17:41:11 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0213.cvx22-bradley.dialup.earthlink.net ([209.179.198.213] helo=mindspring.com) by heron.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18iPPp-0001NP-00; Mon, 10 Feb 2003 17:40:54 -0800 Message-ID: <3E485453.E0CFD785@mindspring.com> Date: Mon, 10 Feb 2003 17:39:31 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Peter Wemm Cc: phk@phk.freebsd.dk, Sheldon Hearn , arch@freebsd.org Subject: Re: Indiscriminately installing all .h files in /usr/include/* References: <20030210183517.5F0052A8BB@canning.wemm.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a481dbf857cd71f230932e4df00d097ddf350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Peter Wemm wrote: > I've been wondering if we need an explicit list of files to install for > a while. We've got lots of junk in /usr/include/dev/* that really shouldn't > be there. The only stuff that should be there are things to define > sysctl, ioctl etc user interfaces. Things like hardware register definitions > and random driver internals definitions do not really belong there. > /usr/include/usb/* is a good example. We do similar bad things for the > file systems. Isn't this already handled by not placing the header files into an area where header files are normally installed from and/or not naming them as install targets in the makefile? For example, if_fxpreg.h lives in /sys/pci, and is not installed into the /usr/include hierarchy, anywhere. There are other examples in the /usr/src/usr.bin and other utility trees, so this isn't a kernel-only thing. Isn't this really about programmers putting header files in the wrong place in the source tree, rather than the wrong place in the source tree's Makefile doing what it's been told to do? -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 19:14:42 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E5B5337B401 for ; Mon, 10 Feb 2003 19:14:39 -0800 (PST) Received: from web13404.mail.yahoo.com (web13404.mail.yahoo.com [216.136.175.62]) by mx1.FreeBSD.org (Postfix) with SMTP id 2BEDC43FBD for ; Mon, 10 Feb 2003 19:14:39 -0800 (PST) (envelope-from giffunip@yahoo.com) Message-ID: <20030211031438.42114.qmail@web13404.mail.yahoo.com> Received: from [200.24.79.141] by web13404.mail.yahoo.com via HTTP; Tue, 11 Feb 2003 04:14:38 CET Date: Tue, 11 Feb 2003 04:14:38 +0100 (CET) From: "=?iso-8859-1?q?Pedro=20F.=20Giffuni?=" Subject: FFS performance improvements? To: freebsd-fs@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG (sorry for the crossposting, replies should probably go to -fs) Hi; I heard a complaint on the FreeBSD lists about journalling being faster than softupdates. While I'm not extremely interested in comparisons with other fs's that I can't use, I do recall it is possible to enhance FFS speed with the tricks used in CFFS. JIC ppl forgot, and since it is a good reference, the classic document is here: http://www.pdos.lcs.mit.edu/pubs.html (under Storage Management) Of the two tricks explained there, explicit grouping gave the most significant performance improvements, without causing problematic side effects. I've been "bugging" the only two persons I know that might have some patches for this with the following results: __________ I considered doing the co-location game in UFS2, but in the end decided that the performance gain was not sufficient to take the effort to do it. So, I put my effort into other things. Kirk McKusick __________ C-FFS required changes to OpenBSD so that the buffer cache would cache raw disk blocks and would consult those cached raw disk blocks before going to the disk device. I do have some old patches and I'll try to dig them up. It was my first kernel project for BSD unix and is definitely nowhere close to production quality. (probaby lots of mistakes related to concurrency and resource management) http://pdos.lcs.mit.edu/~csapuntz/cffs.tar.gz -Costa __________ So, my questions are... 1) Although porting C-FFS is probably not the way to go, do we need to modify the VM to fulfill the requirements, or perhaps this is something GEOM can do? (sorry I don't have clear exactly what GEOM does, but it would seem somewhat related to caching raw disks) 2) Since I'm not really into kernel hacking, if I do find the patches for FFS is someone interested in playing with them in exchange for improvements of "just" between 10-50% with small files ?? ;). cheers, Pedro. ______________________________________________________________________ Yahoo! Cellulari: loghi, suonerie, picture message per il tuo telefonino http://it.yahoo.com/mail_it/foot/?http://it.mobile.yahoo.com/index2002.html To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 20:17:23 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 32ECB37B401 for ; Mon, 10 Feb 2003 20:17:22 -0800 (PST) Received: from canning.wemm.org (canning.wemm.org [192.203.228.65]) by mx1.FreeBSD.org (Postfix) with ESMTP id D3BD843FAF for ; Mon, 10 Feb 2003 20:17:21 -0800 (PST) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by canning.wemm.org (Postfix) with ESMTP id B3AEE2A8B4; Mon, 10 Feb 2003 20:17:21 -0800 (PST) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: "Sam Leffler" Cc: "Alfred Perlstein" , phk@phk.freebsd.dk, arch@FreeBSD.ORG Subject: Re: Indiscriminately installing all .h files in /usr/include/* In-Reply-To: <143501c2d154$9c3c70e0$52557f42@errno.com> Date: Mon, 10 Feb 2003 20:17:21 -0800 From: Peter Wemm Message-Id: <20030211041721.B3AEE2A8B4@canning.wemm.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "Sam Leffler" wrote: > > In message <20030210221501.GF88781@elvis.mu.org>, Alfred Perlstein writes: > > >* Peter Wemm [030210 14:09] wrote: > > >> > > > >> > If there was a way to do it via cpp defines inside the header that > would > > >> > be nice. > > >> > > >> Something like this perhaps? > > >> ^/* EXPORT_HEADER */$ > > > > > >Yes, something like that. I know of one company that brackets the > > >headers with $COMPANY_PRIVATE for stuff that shouldn't be visible in > > >/usr/include. if you wind up with an empty file it isn't even installed. > > > > I would prefer labeling the stuff that should go into /usr/include, > > it shouldn't be the default to export stuff unless you ask for it. > > Sticking this stuff inside include files means modifying imported files. > I'd actually prefer a mechanism like this to lurk in the Makefiles as you > can fully encapsulate requirements like the /usr/include/ is different > than the source directory. I'd prefer an explicit list in the Makefiles rather than tagging. Under no circimstances should we be munging these on the way to /usr/include though since we still can have symlinks pointing to the sys tree... In other words, I'm fine with the kernel parts of /usr/include being a *subset* of whats in the kernel tree. But not different to it. Part of the reason why its done like it is right now is that nobody really got around to figuring out what was needed and doing the Makefile lists and testing that it didn't break stuff. Cheers, -Peter -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 23:26:42 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 71E7C37B401 for ; Mon, 10 Feb 2003 23:26:41 -0800 (PST) Received: from axl.seasidesoftware.co.za (axl.seasidesoftware.co.za [196.31.7.201]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3F37243FB1 for ; Mon, 10 Feb 2003 23:26:39 -0800 (PST) (envelope-from sheldonh@starjuice.net) Received: from sheldonh by axl.seasidesoftware.co.za with local (Exim 4.12) id 18iUoC-0005Xz-00; Tue, 11 Feb 2003 09:26:24 +0200 Date: Tue, 11 Feb 2003 09:26:24 +0200 From: Sheldon Hearn To: Terry Lambert Cc: Peter Wemm , phk@phk.freebsd.dk, arch@freebsd.org Subject: Re: Indiscriminately installing all .h files in /usr/include/* Message-ID: <20030211072624.GC19612@starjuice.net> Mail-Followup-To: Terry Lambert , Peter Wemm , phk@phk.freebsd.dk, arch@freebsd.org References: <20030210183517.5F0052A8BB@canning.wemm.org> <3E485453.E0CFD785@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3E485453.E0CFD785@mindspring.com> User-Agent: Mutt/1.5.3i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On (2003/02/10 17:39), Terry Lambert wrote: > Isn't this really about programmers putting header files in the > wrong place in the source tree, rather than the wrong place in > the source tree's Makefile doing what it's been told to do? That's why I asked for examples; the geom and cam examples that phk gave in response look like headers that are simply misplaced in the src tree. Ciao, Sheldon. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Feb 10 23:30:14 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6437C37B401 for ; Mon, 10 Feb 2003 23:30:13 -0800 (PST) Received: from dhcp01.pn.xcllnt.net (209-128-86-226.BAYAREA.NET [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6BB5A43F75 for ; Mon, 10 Feb 2003 23:30:12 -0800 (PST) (envelope-from marcel@dhcp01.pn.xcllnt.net) Received: from dhcp01.pn.xcllnt.net (localhost [127.0.0.1]) by dhcp01.pn.xcllnt.net (8.12.6/8.12.6) with ESMTP id h1B7UCRq000619; Mon, 10 Feb 2003 23:30:12 -0800 (PST) (envelope-from marcel@dhcp01.pn.xcllnt.net) Received: (from marcel@localhost) by dhcp01.pn.xcllnt.net (8.12.6/8.12.6/Submit) id h1B7UB3M000618; Mon, 10 Feb 2003 23:30:11 -0800 (PST) Date: Mon, 10 Feb 2003 23:30:11 -0800 From: Marcel Moolenaar To: Peter Wemm Cc: arch@FreeBSD.ORG Subject: Re: Indiscriminately installing all .h files in /usr/include/* Message-ID: <20030211073011.GA576@dhcp01.pn.xcllnt.net> References: <143501c2d154$9c3c70e0$52557f42@errno.com> <20030211041721.B3AEE2A8B4@canning.wemm.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030211041721.B3AEE2A8B4@canning.wemm.org> User-Agent: Mutt/1.5.3i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, Feb 10, 2003 at 08:17:21PM -0800, Peter Wemm wrote: > > Part of the reason why its done like it is right now is that nobody really > got around to figuring out what was needed and doing the Makefile lists > and testing that it didn't break stuff. Testing is really the hard part. It would be nice to have a way to have the whole ports collection built with a particular change before the change is committed. Ie, use the ports collection for regression testing. -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 11 0:41:28 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D2A5A37B406 for ; Tue, 11 Feb 2003 00:41:27 -0800 (PST) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id E718D43F75 for ; Tue, 11 Feb 2003 00:41:26 -0800 (PST) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.6/8.12.6) with ESMTP id h1B8fJqV048504; Tue, 11 Feb 2003 09:41:20 +0100 (CET) (envelope-from phk@phk.freebsd.dk) To: Sheldon Hearn Cc: Terry Lambert , Peter Wemm , arch@freebsd.org Subject: Re: Indiscriminately installing all .h files in /usr/include/* From: phk@phk.freebsd.dk In-Reply-To: Your message of "Tue, 11 Feb 2003 09:26:24 +0200." <20030211072624.GC19612@starjuice.net> Date: Tue, 11 Feb 2003 09:41:19 +0100 Message-ID: <48503.1044952879@critter.freebsd.dk> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20030211072624.GC19612@starjuice.net>, Sheldon Hearn writes: >On (2003/02/10 17:39), Terry Lambert wrote: > >> Isn't this really about programmers putting header files in the >> wrong place in the source tree, rather than the wrong place in >> the source tree's Makefile doing what it's been told to do? > >That's why I asked for examples; the geom and cam examples that phk >gave in response look like headers that are simply misplaced in the src >tree. "misplaced" as "shouldn't be put there" or as "shouldn't be put _there_" ? I find it very logical that sys/geom/geom.h and sys/geom/geom_stat.h are located where they are, but I want a way to specify that only the latter should be installed in /usr/include/geom I don't have that currently. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 11 3:28:10 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F3B0637B401 for ; Tue, 11 Feb 2003 03:28:08 -0800 (PST) Received: from axl.seasidesoftware.co.za (axl.seasidesoftware.co.za [196.31.7.201]) by mx1.FreeBSD.org (Postfix) with ESMTP id 277D243F3F for ; Tue, 11 Feb 2003 03:28:07 -0800 (PST) (envelope-from sheldonh@starjuice.net) Received: from sheldonh by axl.seasidesoftware.co.za with local (Exim 4.12) id 18iYZw-0007rD-00; Tue, 11 Feb 2003 13:27:56 +0200 Date: Tue, 11 Feb 2003 13:27:56 +0200 From: Sheldon Hearn To: phk@phk.freebsd.dk Cc: Terry Lambert , Peter Wemm , arch@freebsd.org Subject: Re: Indiscriminately installing all .h files in /usr/include/* Message-ID: <20030211112756.GA29826@starjuice.net> Mail-Followup-To: phk@phk.freebsd.dk, Terry Lambert , Peter Wemm , arch@freebsd.org References: <20030211072624.GC19612@starjuice.net> <48503.1044952879@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <48503.1044952879@critter.freebsd.dk> User-Agent: Mutt/1.5.3i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On (2003/02/11 09:41), phk@phk.freebsd.dk wrote: > >That's why I asked for examples; the geom and cam examples that phk > >gave in response look like headers that are simply misplaced in the src > >tree. > > "misplaced" as "shouldn't be put there" or as "shouldn't be put _there_" ? > > I find it very logical that sys/geom/geom.h and sys/geom/geom_stat.h are > located where they are, but I want a way to specify that only the latter > should be installed in /usr/include/geom Okay, that makes sense. I'll back off, because I think the make infrastructure should be fixed, but am having trouble figuring out how we determine the value of INCS that we feed into bsd.incs.mk, so I can't propose what I'd consider "a real fix", and there's nothing quite so annoying as an objection without an alternative suggestion. Ciao, Sheldon. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 11 3:57:30 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EDAB437B401 for ; Tue, 11 Feb 2003 03:57:28 -0800 (PST) Received: from flaske.stud.ntnu.no (flaske.stud.ntnu.no [129.241.56.72]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0844543FB1 for ; Tue, 11 Feb 2003 03:57:28 -0800 (PST) (envelope-from morten@rodal.no) Received: from localhost (localhost [127.0.0.1]) by flaske.stud.ntnu.no (Postfix) with ESMTP id A41A8FF423; Tue, 11 Feb 2003 12:57:26 +0100 (CET) Received: from slurp.rodal.no (m200h.studby.ntnu.no [129.241.135.200]) by flaske.stud.ntnu.no (Postfix) with ESMTP id EC402FF3CB; Tue, 11 Feb 2003 12:57:25 +0100 (CET) Received: (from morten@localhost) by slurp.rodal.no (8.12.6/8.12.6/Submit) id h1BBvMqP000757; Tue, 11 Feb 2003 12:57:22 +0100 (CET) (envelope-from morten) Date: Tue, 11 Feb 2003 12:57:22 +0100 From: Morten Rodal To: Alfred Perlstein Cc: Matthew Dillon , arch@FreeBSD.ORG Subject: Re: Our lemming-syncer caught in the act. Message-ID: <20030211115722.GA716@slurp.rodal.no> References: <20030210091317.GD5165@HAL9000.homeunix.com> <37473.1044868995@critter.freebsd.dk> <20030210204523.GF12240@slurp.rodal.no> <20030210205443.GA88781@elvis.mu.org> <200302102225.h1AMPkTE023700@apollo.backplane.com> <20030210223904.GG88781@elvis.mu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030210223904.GG88781@elvis.mu.org> X-Virus-Scanned: by AMaViS perl-11 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, Feb 10, 2003 at 02:39:04PM -0800, Alfred Perlstein wrote: > * Matthew Dillon [030210 14:25] wrote: > > > > Try reproducing the problem with kernel profiling turned on. The > > syncer is not likely to be the cause of the large file removal issue, > > since that involves only bitmaps. The cause is more likely to be > > softupdate's bitmap dependancies. Try turning off softupdates and see > > if that improves performance. Softupdates creates some rather nasty > > buffer cache issues due to the number of buffers it removes from > > management. Also, dependancies prevent the buffer cache from being > > able to flush a buffer. The buffer cache was not really designed to > > deal with flushes which don't undirty a buffer or large numbers of > > buffers being temporarily removed from management. > > Profiling sounds like a good idea, but I have never even done profiling on a regular program so I think I'll wait a little before I try something like that on the kernel. Unfortunatly I will not be able to test the performance without softupdates right now, and I forgot to look at that when I tested Alfred's patch. > I don't have a 5.x box handy here, but if someone could run this patch > and let me know what it does I would appreciate it. > > cd /usr/src/sys/kern && patch < path_to_this_file > [snip patch] The patch did not solve the problem. At first I only tried with the last giant pre-emptive, but found out the problem were still there. I then enabled the ifdef (to be presise I removed it), but it did not seem to make any difference at all. -- Morten Rodal To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 11 5: 5:36 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D637E37B401; Tue, 11 Feb 2003 05:05:34 -0800 (PST) Received: from mail.tcoip.com.br (erato.tco.net.br [200.220.254.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 83F2C43FA3; Tue, 11 Feb 2003 05:05:32 -0800 (PST) (envelope-from dcs@tcoip.com.br) Received: from tcoip.com.br ([10.0.2.6]) by mail.tcoip.com.br (8.11.6/8.11.6) with ESMTP id h1BD5Re14805; Tue, 11 Feb 2003 11:05:27 -0200 Message-ID: <3E48F516.7060900@tcoip.com.br> Date: Tue, 11 Feb 2003 11:05:26 -0200 From: "Daniel C. Sobral" User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.3a) Gecko/20030206 X-Accept-Language: en-us, en, pt-br, ja MIME-Version: 1.0 To: Thomas Quinot Cc: arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> In-Reply-To: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Thomas Quinot wrote: > I'm about to commit a change to syslogd to allow multiple hosts or > programs to be named in program or host specifications, eg: > > !foo,bar > *.* /var/log/only_foo_or_bar.log > > !-foo,bar > *.* /var/log/all_except_foo_or_bar.log > > Any objections to this change? A patch is available for > review at http://www.cuivre.fr.eu.org/~thomas/syslogd.c.diff. Extend that to programs too, please. No sense in keeping hosts and programs features out of sync. -- Daniel C. Sobral (8-DCS) Gerencia de Operacoes Divisao de Comunicacao de Dados Coordenacao de Seguranca TCO Fones: 55-61-313-7654/Cel: 55-61-9618-0904 E-mail: Daniel.Capo@tco.net.br Daniel.Sobral@tcoip.com.br dcs@tcoip.com.br Outros: dcs@newsguy.com dcs@freebsd.org capo@notorious.bsdconspiracy.net I think ... therefore I am confused. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 11 7: 4:50 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 876AF37B401; Tue, 11 Feb 2003 07:04:49 -0800 (PST) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id 532E043F3F; Tue, 11 Feb 2003 07:04:46 -0800 (PST) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.12.6/8.12.6) id h1BF4jUq059047; Tue, 11 Feb 2003 09:04:45 -0600 (CST) (envelope-from dan) Date: Tue, 11 Feb 2003 09:04:45 -0600 From: Dan Nelson To: "Daniel C. Sobral" Cc: Thomas Quinot , arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) Message-ID: <20030211150445.GW5356@dan.emsphone.com> References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <3E48F516.7060900@tcoip.com.br> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3E48F516.7060900@tcoip.com.br> X-OS: FreeBSD 5.0-CURRENT X-message-flag: Outlook Error User-Agent: Mutt/1.5.3i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In the last episode (Feb 11), Daniel C. Sobral said: > Thomas Quinot wrote: > > I'm about to commit a change to syslogd to allow multiple hosts or > > programs to be named in program or host specifications, eg: > > > > !foo,bar > > *.* /var/log/only_foo_or_bar.log > > > > !-foo,bar > > *.* /var/log/all_except_foo_or_bar.log > > > > Any objections to this change? A patch is available for review at > > http://www.cuivre.fr.eu.org/~thomas/syslogd.c.diff. > > Extend that to programs too, please. No sense in keeping hosts and > programs features out of sync. Yay -- Dan Nelson dnelson@allantgroup.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 11 7:27: 7 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 851BC37B401; Tue, 11 Feb 2003 07:27:06 -0800 (PST) Received: from melusine.cuivre.fr.eu.org (melusine.cuivre.fr.eu.org [62.212.105.185]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9731843FAF; Tue, 11 Feb 2003 07:27:05 -0800 (PST) (envelope-from thomas@FreeBSD.ORG) Received: by melusine.cuivre.fr.eu.org (Postfix, from userid 1000) id 30D8E2C3D2; Tue, 11 Feb 2003 16:27:03 +0100 (CET) Date: Tue, 11 Feb 2003 16:27:03 +0100 From: Thomas Quinot To: "Daniel C. Sobral" Cc: Thomas Quinot , arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) Message-ID: <20030211152703.GA92759@melusine.cuivre.fr.eu.org> Reply-To: Thomas Quinot References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <3E48F516.7060900@tcoip.com.br> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <3E48F516.7060900@tcoip.com.br> User-Agent: Mutt/1.4i X-message-flag: WARNING! Using Outlook can damage your computer. Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Le 2003-02-11, Daniel C. Sobral crivait : > Subject: Re: syslog.conf syntax change (multiple program/host specifications) ^^^^^^^^^^^^ > >I'm about to commit a change to syslogd to allow multiple > > hosts or programs to be named in program or host specifications, eg: ^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ > Extend that to programs too, please. No sense in keeping hosts and > programs features out of sync. I am not sure I understand what you mean. The example I gave was for programs, but as mentioned in my original mail, the change would be identically effective for hosts. Using the same code for hosts and programs was precisely the point of my previous change syslogd.c, in preparation for this one (cf. rev. 1.111). Thomas. -- Thomas.Quinot@Cuivre.FR.EU.ORG To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 11 13:26:45 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B0D0937B401 for ; Tue, 11 Feb 2003 13:26:44 -0800 (PST) Received: from c001.snv.cp.net (h005.c001.snv.cp.net [209.228.32.119]) by mx1.FreeBSD.org (Postfix) with SMTP id B5C7643FA3 for ; Tue, 11 Feb 2003 13:26:43 -0800 (PST) (envelope-from kutulu@kutulu.org) Received: (cpmta 26344 invoked from network); 11 Feb 2003 13:26:42 -0800 Received: from 64.212.128.3 (HELO kutulu) by smtp.register-admin.com (209.228.32.119) with SMTP; 11 Feb 2003 13:26:42 -0800 X-Sent: 11 Feb 2003 21:26:42 GMT Message-ID: <00c101c2d214$45cff270$29330f0a@lcapps.educate.com> From: "Kutulu" To: Subject: SCHED_ULE and ps? Date: Tue, 11 Feb 2003 16:26:41 -0500 Organization: KutuluWare Software Services MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG (Sending this to arch@ cuz UPDATING said to) I just rebuilt my world/kernel with the new ULE scheduler and noticed this happening: root 36 0.0 0.0 0 12 ?? DL 31Dec69 0:00.86 (vnlru) root 37 0.0 0.0 0 12 ?? DL 31Dec69 0:08.40 (syncer) root 128 0.0 0.1 220 52 ?? Ss Mon01PM 0:00.00 adjkerntz -i root 214 0.0 0.4 1164 244 ?? Ss 6:02PM 0:00.45 /sbin/dhclient xl0 root 268 0.0 0.3 644 208 ?? Ss 6:02PM 0:07.38 /sbin/natd -config /usr/local/etc/natd.conf -n xl0 All of the processes in parens (I assume these are internal or system-level processes) show a timestamp of 0, as does /sbin/init itself. All other processes are showing the correct time, or something that looks reasonable. The only item I changed in my kernel config was to change the scheduler. I don't remember seeing this behavior with my previous kernel, which was built last week using the old scheduler, but I only ran it for a short time. I also checked the available 4.6 and 4.7 machines and they don't exhibit this behavior either. I did a complete buildworld and buildkernel, /bin/ps says it was built yesterday around the same time as the kernel, so I don't think there's a world/kernel sync issue here. I'm putting the 4BSD scheduler back to see if this helps, but has anyone else seen this issue? --Mike Edenfield To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 11 14:36:53 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 520FE37B401; Tue, 11 Feb 2003 14:36:51 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5ADA143FA3; Tue, 11 Feb 2003 14:36:46 -0800 (PST) (envelope-from wes@softweyr.com) Received: from salty.rapid.stbernard.com (corp-2.ipinc.com [199.245.188.2]) by smtp-relay.omnis.com (Postfix) with ESMTP id 5C1BA43948; Tue, 11 Feb 2003 14:30:35 -0800 (PST) From: Wesley Peters Organization: Softweyr To: Thomas Quinot , arch@freebsd.org Subject: Re: syslog.conf syntax change (multiple program/host specifications) Date: Tue, 11 Feb 2003 14:30:03 -0800 User-Agent: KMail/1.5 References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> In-Reply-To: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200302111430.03156.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Monday 10 February 2003 03:49, Thomas Quinot wrote: > I'm about to commit a change to syslogd to allow multiple hosts or > programs to be named in program or host specifications, eg: > > !foo,bar > *.* /var/log/only_foo_or_bar.log > > !-foo,bar > *.* /var/log/all_except_foo_or_bar.log > > Any objections to this change? A patch is available for > review at http://www.cuivre.fr.eu.org/~thomas/syslogd.c.diff. Excellent. I have another simple change to syslog I want to cast before the audience for consideration. I have an embedded platform that needs to not run the log partition out of disk space. We've encountered problems at a few very busy customer sites where something will babble into syslog faster than newsyslog can keep up. I've made the following change to syslogd and syslog.conf accomodate our needs: ... auth.info;authpriv.info /var/log/auth.log mail.info /var/log/maillog G 4M lpr.info /var/log/lpd-errs 256K local3.* /var/log/foo B 1K ftp.* /var/log/ftp B 1M cron.* /var/log/cron ... Each line, after the filename, can specify a logfile size limit and an optional disposition. The size limit should be self explanatory, size scalars of K through E plus B for blocks are supported. The optional disposition can be 'M' for Max file size, the existing logfile is renamed with a .bak extension and a new one created; 'B' or 'G' for bzip2 or gzip the file and create a new one, or 'R' to simply remove and recreate the file. The implied operation is 'M' if none is specified, as in the lpr.info line above. This mod has been running on my 4.7R workstation for a month now, and on a (4.4 + security fixes) appliance in our test lab as well. We've noticed no problems. Attached is a diff against 4.7R. If anyone at all is interested in this I'll provide a patch against -CURRENT as soon as I get my -CURRENT machine up at home. Anyone who thinks this is dumb or unneeded should let me know sooner rather than later. On a side note, this is the first of what we hope will be many small and hopefully larger transfers of functionality from ${DAY_JOB} to FreeBSD. 'tis a good place I've landed, aye. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 11 15:10:34 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B72C837B401 for ; Tue, 11 Feb 2003 15:10:32 -0800 (PST) Received: from khavrinen.lcs.mit.edu (khavrinen.lcs.mit.edu [18.24.4.193]) by mx1.FreeBSD.org (Postfix) with ESMTP id E167543FA3 for ; Tue, 11 Feb 2003 15:10:31 -0800 (PST) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: from khavrinen.lcs.mit.edu (localhost [IPv6:::1]) by khavrinen.lcs.mit.edu (8.12.6/8.12.6) with ESMTP id h1BNAUbs019098 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=OK); Tue, 11 Feb 2003 18:10:30 -0500 (EST) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: (from wollman@localhost) by khavrinen.lcs.mit.edu (8.12.6/8.12.6/Submit) id h1BNAUBS019097; Tue, 11 Feb 2003 18:10:30 -0500 (EST) (envelope-from wollman) Date: Tue, 11 Feb 2003 18:10:30 -0500 (EST) From: Garrett Wollman Message-Id: <200302112310.h1BNAUBS019097@khavrinen.lcs.mit.edu> To: wes@softweyr.com Subject: Re: syslog.conf syntax change (multiple program/host specifications) In-Reply-To: <200302111430.03156.wes@softweyr.com> References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> Organization: MIT Laboratory for Computer Science Cc: arch@FreeBSD.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In article <200302111430.03156.wes@softweyr.com> you write: >Excellent. I have another simple change to syslog I want to cast before the >audience for consideration. I have had near the bottom of the CFT pile for some time now a project to do the following (which your example is just crying out for): 1) Integrate syslogd and newsyslog. 2) Replace syslog.conf and newsyslog.conf with a new configuration file that contains complete descriptions of log sources and destinations. As a part of (2) I have been thinking at very low priority about how to represent the new configuration file, and I've come to the conclusion that XML is probably the best answer. Although XML syntax is vile, it is also unambiguous and relatively easy to parse (particularly given that we now have an XML lexer library in the base system). It also has the benefit that it can be easily transformed into a human-readable description of log configuration, which might make some tasks easier. Before starting on this I wanted to take up a somewhat easier task, like inetd, which is also crying for relief from its kluge-encrusted historical syntax. -GAWollman -- Garrett A. Wollman | [G]enes make enzymes, and enzymes control the rates of wollman@lcs.mit.edu | chemical processes. Genes do not make ``novelty- Opinions not those of| seeking'' or any other complex and overt behavior. MIT, LCS, CRS, or NSA| - Stephen Jay Gould (1941-2002) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 11 15:48:57 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E3A3A37B401; Tue, 11 Feb 2003 15:48:46 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id C2A7443FA3; Tue, 11 Feb 2003 15:48:43 -0800 (PST) (envelope-from wes@softweyr.com) Received: from salty.rapid.stbernard.com (corp-2.ipinc.com [199.245.188.2]) by smtp-relay.omnis.com (Postfix) with ESMTP id 2DEEC434BC; Tue, 11 Feb 2003 15:48:41 -0800 (PST) From: Wesley Peters Organization: Softweyr To: Thomas Quinot , arch@freebsd.org Subject: Re: syslog.conf syntax change (multiple program/host specifications) Date: Tue, 11 Feb 2003 15:48:41 -0800 User-Agent: KMail/1.5 References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302111430.03156.wes@softweyr.com> In-Reply-To: <200302111430.03156.wes@softweyr.com> MIME-Version: 1.0 Content-Type: Multipart/Mixed; boundary="Boundary-00=_ZvYS+aFCBwL+8hZ" Message-Id: <200302111548.41114.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --Boundary-00=_ZvYS+aFCBwL+8hZ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline On Tuesday 11 February 2003 14:30, Wesley Peters wrote: > > This mod has been running on my 4.7R workstation for a month now, and on > a (4.4 + security fixes) appliance in our test lab as well. We've > noticed no problems. Attached is a diff against 4.7R. > Duh. Here's the diff: -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ --Boundary-00=_ZvYS+aFCBwL+8hZ Content-Type: text/x-diff; charset="iso-8859-1"; name="syslogd.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="syslogd.diff" --- syslogd.c.orig Tue Jan 14 10:31:10 2003 +++ syslogd.c Tue Feb 11 15:46:42 2003 @@ -144,6 +144,20 @@ #define ISKERNEL 0x010 /* kernel generated message */ /* + * File I/O and permission flags. + */ +#define OPEN_FLAGS O_WRONLY|O_APPEND +#define BACKUP_FLAGS O_WRONLY|O_APPEND|O_CREAT|O_TRUNC +#define REOPEN_FLAGS O_WRONLY|O_APPEND|O_CREAT|O_EXCL + +/* + * Action types for file size limiting. Not all are implemented at this + * time. + */ +enum f_limit { LIM_NONE = '0', LIM_BZIP = 'B', LIM_GZIP = 'G', + LIM_MAX = 'M', LIM_RM = 'R' }; + +/* * This structure represents the files that will have log * copies printed. */ @@ -167,7 +181,14 @@ struct addrinfo *f_addr; } f_forw; /* forwarding address */ + struct { char f_fname[MAXPATHLEN]; + off_t f_fsize; + uid_t f_uid; + gid_t f_gid; + mode_t f_mode; + enum f_limit f_flimit; + } f_logfile; struct { char f_pname[MAXPATHLEN]; pid_t f_pid; @@ -253,6 +274,19 @@ "FORW", "USERS", "WALL", "PIPE" }; +/* types of compressors we support for compressing rolled logfiles */ + +struct zipper { + char *binpath; + char *extension; +} zipper[] = { + { "/usr/bin/bzip2", ".bz2" }, + { "/usr/bin/gzip", ".gz" } +}; + +#define BZIPPER 0 +#define GZIPPER 1 + struct filed *Files; struct filed consfile; @@ -287,7 +321,7 @@ volatile sig_atomic_t MarkSet, WantDie; int allowaddr __P((char *)); -void cfline __P((const char *, struct filed *, const char *, const char *)); +void cfline __P((char *, struct filed *, const char *, const char *)); const char *cvthname __P((struct sockaddr *)); void deadq_enter __P((pid_t, const char *)); int deadq_remove __P((pid_t)); @@ -411,8 +445,9 @@ endservent(); consfile.f_type = F_CONSOLE; - (void)strlcpy(consfile.f_un.f_fname, ctty + sizeof _PATH_DEV - 1, - sizeof(consfile.f_un.f_fname)); + (void)strlcpy(consfile.f_un.f_logfile.f_fname, + ctty + sizeof _PATH_DEV - 1, + sizeof(consfile.f_un.f_logfile.f_fname)); (void)strlcpy(bootfile, getbootfile(), sizeof(bootfile)); (void)signal(SIGTERM, dodie); (void)signal(SIGINT, Debug ? dodie : SIG_IGN); @@ -894,7 +929,8 @@ sizeof(f->f_prevhost)); if (msglen < MAXSVLINE) { f->f_prevlen = msglen; - (void)strlcpy(f->f_prevline, msg, sizeof(f->f_prevline)); + (void)strlcpy(f->f_prevline, msg, + sizeof(f->f_prevline)); fprintlog(f, flags, (char *)NULL); } else { f->f_prevline[0] = 0; @@ -906,6 +942,148 @@ (void)sigsetmask(omask); } +/* + * Reopen the logfile described by f. Assume the logfile no longer exists; + * the caller is supposed to have removed it before calling us. + */ +static void +reopen(struct filed *f) +{ + short oldfd = f->f_file; + + f->f_file = open(f->f_un.f_logfile.f_fname, REOPEN_FLAGS, + f->f_un.f_logfile.f_mode); + if (f->f_file < 0) { + f->f_type = F_UNUSED; + logerror(f->f_un.f_logfile.f_fname); + } + fchown(f->f_file, f->f_un.f_logfile.f_uid, f->f_un.f_logfile.f_gid); + dprintf("reopen(\"%s\", %o, %d:%d) fd was %d is %d\n", + f->f_un.f_logfile.f_fname, + f->f_un.f_logfile.f_mode, + f->f_un.f_logfile.f_uid, + f->f_un.f_logfile.f_gid, + oldfd, f->f_file); +} + +static void +backup(struct filed *f) +{ + char backup[MAXPATHLEN]; + + strlcpy(backup, f->f_un.f_logfile.f_fname, sizeof(backup)); + strlcat(backup, ".old", sizeof(backup)); + dprintf("backup(\"%s\") -> \"%s\"\n", f->f_un.f_logfile.f_fname, + backup); + rename(f->f_un.f_logfile.f_fname, backup); +} + +/* + * Compress the logfile f. Call this routine after closing the file, so the + * buffers have been flushed, but before unlinking the file. This function + * re-opens the log file for input in the parent process, before forking, so + * once this routine has returned to the caller it is safe to unlink the + * file. + */ +static void +compress(struct filed *f, struct zipper compressor) +{ + pid_t pid; + char backup[MAXPATHLEN]; + int in, out; + + strlcpy(backup, f->f_un.f_logfile.f_fname, sizeof(backup)); + strlcat(backup, compressor.extension, sizeof(backup)); + + dprintf("compress(\"%s\")\n", f->f_un.f_logfile.f_fname, backup); + + in = open(f->f_un.f_logfile.f_fname, O_RDONLY); + if (in < 0) { + logerror("reopening log file for input"); + return; + } + dprintf("compress: reopened %s for input on %d\n", + f->f_un.f_logfile.f_fname, in); + out = open(backup, BACKUP_FLAGS, f->f_un.f_logfile.f_mode); + if (out < 0) { + logerror("opening compressed logfile for output"); + close(in); + return; + } + fchown(out, f->f_un.f_logfile.f_uid, f->f_un.f_logfile.f_gid); + dprintf("compress: opened %s %o %d:%d for output on %d\n", + f->f_un.f_logfile.f_fname, + f->f_un.f_logfile.f_mode, + f->f_un.f_logfile.f_uid, + f->f_un.f_logfile.f_gid, + in); + + if ((pid = fork()) < 0) { + logerror("bzip2 fork"); + return; + } else if (pid == 0) { + dprintf("compress: %sing now...\n", compressor.binpath); + + dup2(in, STDIN_FILENO); + dup2(out, STDOUT_FILENO); + execl(compressor.binpath, "-f", NULL); + err(1, compressor.binpath); + } else { + /* Only in the parent process here, let bzip2 crank up. */ + dprintf("compress: Letting %s crank up...\n", + compressor.binpath); + usleep(10); + } + dprintf("compress: removing %s\n", f->f_un.f_logfile.f_fname); + close(in); + close(out); + unlink(f->f_un.f_logfile.f_fname); +} + +static void +roll(struct filed *f) +{ + struct stat s; + + if (f->f_un.f_logfile.f_flimit == LIM_NONE) + return; + + dprintf("roll(\"%s\")\n", f->f_un.f_logfile.f_fname); + + if (fstat(f->f_file, &s) == -1) { + logerror("fstat"); + return; + } + dprintf("roll: size %lld max %lld\n", s.st_size, + f->f_un.f_logfile.f_fsize); + if (s.st_size < f->f_un.f_logfile.f_fsize) + return; + + /* This logfile has grown too big, roll it per our instructions. */ + f->f_un.f_logfile.f_uid = s.st_uid; + f->f_un.f_logfile.f_gid = s.st_gid; + f->f_un.f_logfile.f_mode = s.st_mode; + close(f->f_file); + switch (f->f_un.f_logfile.f_flimit) { + case LIM_RM: + dprintf("roll: remove %s\n", f->f_un.f_logfile.f_fname); + unlink(f->f_un.f_logfile.f_fname); + break; + case LIM_MAX: + backup(f); + break; + case LIM_BZIP: + compress(f, zipper[BZIPPER]); + break; + case LIM_GZIP: + compress(f, zipper[GZIPPER]); + break; + case LIM_NONE: /* Can't happen, but keep compiler from whining. */ + break; + } + reopen(f); +} + void fprintlog(f, flags, msg) struct filed *f; @@ -1060,7 +1238,7 @@ break; case F_FILE: - dprintf(" %s\n", f->f_un.f_fname); + dprintf(" %s\n", f->f_un.f_logfile.f_fname); v->iov_base = "\n"; v->iov_len = 1; if (writev(f->f_file, iov, 7) < 0) { @@ -1068,9 +1246,11 @@ (void)close(f->f_file); f->f_type = F_UNUSED; errno = e; - logerror(f->f_un.f_fname); + logerror(f->f_un.f_logfile.f_fname); } else if (flags & SYNC_FILE) (void)fsync(f->f_file); + if (f->f_un.f_logfile.f_flimit != LIM_NONE) + roll(f); break; case F_PIPE: @@ -1105,12 +1285,12 @@ /* FALLTHROUGH */ case F_TTY: - dprintf(" %s%s\n", _PATH_DEV, f->f_un.f_fname); + dprintf(" %s%s\n", _PATH_DEV, f->f_un.f_logfile.f_fname); v->iov_base = "\r\n"; v->iov_len = 2; errno = 0; /* ttymsg() only sometimes returns an errno */ - if ((msgret = ttymsg(iov, 7, f->f_un.f_fname, 10))) { + if ((msgret = ttymsg(iov, 7, f->f_un.f_logfile.f_fname, 10))) { f->f_type = F_UNUSED; logerror(msgret); } @@ -1492,12 +1672,15 @@ printf("%s: ", TypeNames[f->f_type]); switch (f->f_type) { case F_FILE: - printf("%s", f->f_un.f_fname); + printf("%s", f->f_un.f_logfile.f_fname); + if (f->f_un.f_logfile.f_flimit != LIM_NONE) + printf(" %c %lld", f->f_un.f_logfile.f_flimit, + f->f_un.f_logfile.f_fsize); break; case F_CONSOLE: case F_TTY: - printf("%s%s", _PATH_DEV, f->f_un.f_fname); + printf("%s%s", _PATH_DEV, f->f_un.f_logfile.f_fname); break; case F_FORW: @@ -1509,7 +1692,9 @@ break; case F_USERS: - for (i = 0; i < MAXUNAMES && *f->f_un.f_uname[i]; i++) + for (i = 0; + i < MAXUNAMES && *f->f_un.f_uname[i]; + i++) printf("%s, ", f->f_un.f_uname[i]); break; } @@ -1538,13 +1723,13 @@ */ void cfline(line, f, prog, host) - const char *line; + char *line; struct filed *f; const char *prog; const char *host; { struct addrinfo hints, *res; - int error, i, pri; + int error, i, pri, scale; const char *p, *q; char *bp; char buf[MAXLINE], ebuf[100]; @@ -1701,7 +1886,44 @@ break; case '/': - if ((f->f_file = open(p, O_WRONLY|O_APPEND, 0)) < 0) { + /* + * find end of filename, terminate, leave q pointing at file + * options. + */ + f->f_un.f_logfile.f_flimit = LIM_NONE; + if ((q = strpbrk(p, "\t ")) != NULL) { + *q++ = '\0'; + while (*q == '\t' || *q == ' ') + q++; + /* determine filesize limiting action, if specified */ + switch (*q) { + case LIM_BZIP: case LIM_GZIP: + case LIM_MAX: case LIM_RM: + f->f_un.f_logfile.f_flimit = *q++; + break; + case '0'...'9': + /* implied action is LIM_MAX */ + f->f_un.f_logfile.f_flimit = LIM_MAX; + break; + } + /* file size is being limited, get the size */ + if (f->f_un.f_logfile.f_flimit != LIM_NONE) { + f->f_un.f_logfile.f_fsize = strtol(q, &bp, 0); + scale = 0; + switch (toupper(*bp)) { + case 'E': scale += 10; /* Exabyte */ + case 'P': scale += 10; /* Petabyte */ + case 'T': scale += 10; /* get the idea... */ + case 'G': scale += 10; + case 'M': scale += 10; + case 'K': scale += 5; + case 'B': scale += 5; /* blocks */ + } + f->f_un.f_logfile.f_fsize <<= scale; + } + } + + if ((f->f_file = open(p, OPEN_FLAGS, 0)) < 0) { f->f_type = F_UNUSED; logerror(p); break; @@ -1711,17 +1933,20 @@ f->f_type = F_CONSOLE; else f->f_type = F_TTY; - (void)strlcpy(f->f_un.f_fname, p + sizeof(_PATH_DEV) - 1, - sizeof(f->f_un.f_fname)); + (void)strlcpy(f->f_un.f_logfile.f_fname, + p + sizeof(_PATH_DEV) - 1, + sizeof(f->f_un.f_logfile.f_fname)); } else { - (void)strlcpy(f->f_un.f_fname, p, sizeof(f->f_un.f_fname)); + (void)strlcpy(f->f_un.f_logfile.f_fname, p, + sizeof(f->f_un.f_logfile.f_fname)); f->f_type = F_FILE; } break; case '|': f->f_un.f_pipe.f_pid = 0; - (void)strlcpy(f->f_un.f_fname, p + 1, sizeof(f->f_un.f_fname)); + (void)strlcpy(f->f_un.f_pipe.f_pname, p + 1, + sizeof(f->f_un.f_pipe.f_pname)); f->f_type = F_PIPE; break; --Boundary-00=_ZvYS+aFCBwL+8hZ-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 11 20:42:13 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F384B37B401 for ; Tue, 11 Feb 2003 20:42:12 -0800 (PST) Received: from c001.snv.cp.net (h000.c001.snv.cp.net [209.228.32.114]) by mx1.FreeBSD.org (Postfix) with SMTP id 24E5043FBD for ; Tue, 11 Feb 2003 20:42:12 -0800 (PST) (envelope-from kutulu@kutulu.org) Received: (cpmta 18384 invoked from network); 11 Feb 2003 20:42:11 -0800 Received: from 68.32.161.3 (HELO KutuluWare) by smtp.register-admin.com (209.228.32.114) with SMTP; 11 Feb 2003 20:42:11 -0800 X-Sent: 12 Feb 2003 04:42:11 GMT Message-ID: <002401c2d251$08de5e50$0245a8c0@KutuluWare> From: "Kutulu" To: Subject: Re: SCHED_ULE & ps Date: Tue, 11 Feb 2003 23:41:39 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Ok, I was wrong, my odd ps behavior is not related to SCHED_ULE. It's still appearing on a kernel with SCHED_4BSD built in. Sorry for the false alarm. --Mike To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 11 21:24:35 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CC95B37B401 for ; Tue, 11 Feb 2003 21:24:33 -0800 (PST) Received: from obsecurity.dyndns.org (adsl-67-119-52-61.dsl.lsan03.pacbell.net [67.119.52.61]) by mx1.FreeBSD.org (Postfix) with ESMTP id F2A6A43F85 for ; Tue, 11 Feb 2003 21:24:32 -0800 (PST) (envelope-from kris@obsecurity.org) Received: from rot13.obsecurity.org (rot13.obsecurity.org [10.0.0.5]) by obsecurity.dyndns.org (Postfix) with ESMTP id 9FB7267B88; Tue, 11 Feb 2003 21:24:32 -0800 (PST) Received: by rot13.obsecurity.org (Postfix, from userid 1000) id 83EF3100A; Tue, 11 Feb 2003 21:24:32 -0800 (PST) Date: Tue, 11 Feb 2003 21:24:32 -0800 From: Kris Kennaway To: Kutulu Cc: freebsd-arch@FreeBSD.ORG Subject: Re: SCHED_ULE & ps Message-ID: <20030212052432.GB31883@rot13.obsecurity.org> References: <002401c2d251$08de5e50$0245a8c0@KutuluWare> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="U+BazGySraz5kW0T" Content-Disposition: inline In-Reply-To: <002401c2d251$08de5e50$0245a8c0@KutuluWare> User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --U+BazGySraz5kW0T Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tue, Feb 11, 2003 at 11:41:39PM -0500, Kutulu wrote: > Ok, I was wrong, my odd ps behavior is not related to SCHED_ULE. It's still > appearing on a kernel with SCHED_4BSD built in. Sorry for the false alarm. Yes, this has been reported a number of times recently. Kris --U+BazGySraz5kW0T Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (FreeBSD) iD8DBQE+SdqPWry0BWjoQKURAsheAKD4n185bBAqvoV5JC7kS/qFYSW+dgCdEDKA Md5X+A2pD/SndmOgcJr/1lo= =eHNC -----END PGP SIGNATURE----- --U+BazGySraz5kW0T-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 11 22:40:48 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 395CB37B401 for ; Tue, 11 Feb 2003 22:40:47 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id 931DC43F3F for ; Tue, 11 Feb 2003 22:40:46 -0800 (PST) (envelope-from wes@softweyr.com) Received: from softweyr.homeunix.net (66-75-151-22.san.rr.com [66.75.151.22]) by smtp-relay.omnis.com (Postfix) with ESMTP id 6269B43581; Tue, 11 Feb 2003 22:32:37 -0800 (PST) From: Wes Peters Organization: Softweyr To: Garrett Wollman Subject: Re: syslog.conf syntax change (multiple program/host specifications) Date: Wed, 12 Feb 2003 06:32:36 +0000 User-Agent: KMail/1.5 Cc: arch@FreeBSD.org References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302112310.h1BNAUBS019097@khavrinen.lcs.mit.edu> In-Reply-To: <200302112310.h1BNAUBS019097@khavrinen.lcs.mit.edu> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200302120632.36583.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tuesday 11 February 2003 23:10, Garrett Wollman wrote: > In article <200302111430.03156.wes@softweyr.com> you write: > >Excellent. I have another simple change to syslog I want to cast before > > the audience for consideration. > > I have had near the bottom of the CFT pile for some time now a project > to do the following (which your example is just crying out for): > > 1) Integrate syslogd and newsyslog. > 2) Replace syslog.conf and newsyslog.conf with a new configuration file > that contains complete descriptions of log sources and destinations. > > As a part of (2) I have been thinking at very low priority about how > to represent the new configuration file, and I've come to the > conclusion that XML is probably the best answer. There was a time I would have agreed with you, but that time has passed. SGML-ish languages have such tortured syntax and are such an abomination on the eyes they should never be exposed to human eyes. I'd much prefer to develop a simple yacc-able grammar and develope the configuration parser that way, or use a key=value configuration, so human beings can understand the configuration with a program to help them. XML is the ultimate Least Common Denominator; it's the best tool for every job, except you can come up with a better tool for any job. -- Where am I, and what am I doing in this handbasket? Wes Peters wes@softweyr.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 11 22:50:24 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D356B37B401 for ; Tue, 11 Feb 2003 22:50:23 -0800 (PST) Received: from w250.z064001178.sjc-ca.dsl.cnc.net (adsl-66.218.45.239.dslextreme.com [66.218.45.239]) by mx1.FreeBSD.org (Postfix) with SMTP id 4702243F75 for ; Tue, 11 Feb 2003 22:50:20 -0800 (PST) (envelope-from jos@catnook.com) Received: (qmail 80082 invoked by uid 1000); 12 Feb 2003 06:50:38 -0000 Date: Tue, 11 Feb 2003 22:50:16 -0800 From: Jos Backus To: arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) Message-ID: <20030212065038.GA99658@lizzy.catnook.com> Reply-To: jos@catnook.com Mail-Followup-To: arch@FreeBSD.ORG References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302112310.h1BNAUBS019097@khavrinen.lcs.mit.edu> <200302120632.36583.wes@softweyr.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200302120632.36583.wes@softweyr.com> User-Agent: Mutt/1.5.3i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Feb 12, 2003 at 06:32:36AM +0000, Wes Peters wrote: > XML is the ultimate Least Common Denominator; it's the best tool for every > job, except you can come up with a better tool for any job. What about YAML? http://www.yaml.org/ -- Jos Backus _/ _/_/_/ Sunnyvale, CA _/ _/ _/ _/ _/_/_/ _/ _/ _/ _/ jos at catnook.com _/_/ _/_/_/ require 'std/disclaimer' To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 11 23:11: 0 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 74BA237B401 for ; Tue, 11 Feb 2003 23:10:58 -0800 (PST) Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.80]) by mx1.FreeBSD.org (Postfix) with ESMTP id AC9AF43F93 for ; Tue, 11 Feb 2003 23:10:56 -0800 (PST) (envelope-from grog@lemis.com) Received: by wantadilla.lemis.com (Postfix, from userid 1004) id A24FB51971; Wed, 12 Feb 2003 17:40:54 +1030 (CST) Date: Wed, 12 Feb 2003 17:40:54 +1030 From: Greg 'groggy' Lehey To: Wes Peters Cc: Garrett Wollman , arch@FreeBSD.org Subject: Re: syslog.conf syntax change (multiple program/host specifications) Message-ID: <20030212071054.GA32298@wantadilla.lemis.com> References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302112310.h1BNAUBS019097@khavrinen.lcs.mit.edu> <200302120632.36583.wes@softweyr.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="qMm9M+Fa2AknHoGS" Content-Disposition: inline In-Reply-To: <200302120632.36583.wes@softweyr.com> User-Agent: Mutt/1.4i Organization: The FreeBSD Project Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.FreeBSD.org/ X-PGP-Fingerprint: 9A1B 8202 BCCE B846 F92F 09AC 22E6 F290 507A 4223 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --qMm9M+Fa2AknHoGS Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Wednesday, 12 February 2003 at 6:32:36 +0000, Wes Peters wrote: > On Tuesday 11 February 2003 23:10, Garrett Wollman wrote: >> In article <200302111430.03156.wes@softweyr.com> you write: >>> Excellent. I have another simple change to syslog I want to cast before >>> the audience for consideration. >> >> I have had near the bottom of the CFT pile for some time now a project >> to do the following (which your example is just crying out for): >> >> 1) Integrate syslogd and newsyslog. >> 2) Replace syslog.conf and newsyslog.conf with a new configuration file >> that contains complete descriptions of log sources and destinations. >> >> As a part of (2) I have been thinking at very low priority about how >> to represent the new configuration file, and I've come to the >> conclusion that XML is probably the best answer. > > There was a time I would have agreed with you, but that time has > passed. SGML-ish languages have such tortured syntax and are such > an abomination on the eyes they should never be exposed to human > eyes. Amen. Greg -- See complete headers for address and phone numbers --qMm9M+Fa2AknHoGS Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.0 (FreeBSD) iD8DBQE+SfN+IubykFB6QiMRArl4AJ4jjRKkaiisMRZPTmBEUnQuFRyG0gCgtEYD 02zxEsNUqr3vOanlulUYYYU= =cRLD -----END PGP SIGNATURE----- --qMm9M+Fa2AknHoGS-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Feb 11 23:34:23 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 97FFA37B401 for ; Tue, 11 Feb 2003 23:34:22 -0800 (PST) Received: from mail0.rawbw.com (mail0.rawbw.com [198.144.192.41]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1202343FAF for ; Tue, 11 Feb 2003 23:34:22 -0800 (PST) (envelope-from jdd@greatschools.net) Received: from cube (m198-158.dsl.rawbw.com [198.144.198.158]) by mail0.rawbw.com (8.11.3/8.11.3) with ESMTP id h1C7Xv054379; Tue, 11 Feb 2003 23:33:58 -0800 (PST) (envelope-from jdd@greatschools.net) Date: Tue, 11 Feb 2003 23:36:15 -0800 (PST) From: John David Duncan X-X-Sender: jdd@cube To: Garrett Wollman Cc: wes@softweyr.com, arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) In-Reply-To: <200302112310.h1BNAUBS019097@khavrinen.lcs.mit.edu> Message-ID: References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302112310.h1BNAUBS019097@khavrinen.lcs.mit.edu> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > > 1) Integrate syslogd and newsyslog. > 2) Replace syslog.conf and newsyslog.conf with a new configuration file > that contains complete descriptions of log sources and destinations. Then what would happen to newsyslog? People use it for rotating other logs outside of syslogd. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 0: 6:35 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5917337B406 for ; Wed, 12 Feb 2003 00:06:32 -0800 (PST) Received: from kkman.com.tw (TC210-202-84-32.adsl.pl.apol.com.tw [210.202.84.32]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9928643F3F for ; Wed, 12 Feb 2003 00:06:29 -0800 (PST) (envelope-from junny@kkman.com.tw) From: junny@kkman.com.tw Subject: =?ISO-8859-1?B?UmU6p9qkXaxPs+EuLi4=?= Reply-To: junny@kkman.com.tw Date: 12 Feb 2003 16:06:30 +0800 MIME-Version: 1.0 Content-Type: text/html Content-Transfer-Encoding: 8bit Message-Id: <20030212080629.9928643F3F@mx1.FreeBSD.org> To: undisclosed-recipients: ; Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG ~֥~M

                                                  ~֥~M!!

             AH|A??

          pGٰg!! ~Ѹ`έn~]iNOA

        

                                        

  

To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 0: 7:18 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4AF7737B401 for ; Wed, 12 Feb 2003 00:07:17 -0800 (PST) Received: from halt-in.cisco.com (halt-in.cisco.com [171.70.144.185]) by mx1.FreeBSD.org (Postfix) with ESMTP id C73AC43FDF for ; Wed, 12 Feb 2003 00:07:15 -0800 (PST) (envelope-from molter@tin.it) Received: from tin.it (144.254.74.60) by halt-in.cisco.com with ESMTP; 12 Feb 2003 00:06:58 -0800 Received: from cisco.com (localhost [127.0.0.1]) by ams-msg-core-1.cisco.com (8.12.2/8.12.6) with ESMTP id h1C85ccV005900 for ; Wed, 12 Feb 2003 09:05:38 +0100 (MET) Received: from www.example.org (dhcp-nic-val-26-108.cisco.com [64.103.26.108]) by cisco.com (8.8.8+Sun/8.8.8) with SMTP id JAA02746 for ; Wed, 12 Feb 2003 09:07:13 +0100 (MET) Received: (qmail 4227 invoked by uid 1000); 12 Feb 2003 08:07:10 -0000 Date: Wed, 12 Feb 2003 09:07:10 +0100 From: Marco Molteni To: arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) Message-ID: <20030212080710.GA4212@cobweb.example.org> Mail-Followup-To: arch@FreeBSD.ORG References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302112310.h1BNAUBS019097@khavrinen.lcs.mit.edu> <200302120632.36583.wes@softweyr.com> <20030212065038.GA99658@lizzy.catnook.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030212065038.GA99658@lizzy.catnook.com> User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Jos Backus wrote [2003-02-11]: > On Wed, Feb 12, 2003 at 06:32:36AM +0000, Wes Peters wrote: > > XML is the ultimate Least Common Denominator; it's the best tool for every > > job, except you can come up with a better tool for any job. > > What about YAML? > > http://www.yaml.org/ I love YAML. On a side note, I am finishing my mods to syslogd to optionally log the date and time according to the ISO 8601 standard. For example: 2003-01-21 21:32:54+0100 cobweb newsyslog[3198]: logfile turned over Marco To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 0:25:10 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 66F8C37B401; Wed, 12 Feb 2003 00:25:03 -0800 (PST) Received: from is1.mh.itc.u-tokyo.ac.jp (is1.mh.itc.u-tokyo.ac.jp [133.11.205.11]) by mx1.FreeBSD.org (Postfix) with ESMTP id D88AE43FAF; Wed, 12 Feb 2003 00:25:00 -0800 (PST) (envelope-from simokawa@sat.t.u-tokyo.ac.jp) Received: from is1.mh.itc.u-tokyo.ac.jp (is1.mh.itc.u-tokyo.ac.jp [127.0.0.1]) by is1.mh.itc.u-tokyo.ac.jp (Postfix) with ESMTP id F124E21809C; Wed, 12 Feb 2003 17:24:57 +0900 (JST) Received: from mailhosting.itc.u-tokyo.ac.jp (IDENT:mirapoint@mailhosting.itc.u-tokyo.ac.jp [133.11.205.3]) by is1.mh.itc.u-tokyo.ac.jp (8.11.3/8.11.3) with ESMTP id h1C8Ovc15112; Wed, 12 Feb 2003 17:24:57 +0900 Received: from ett.sat.t.u-tokyo.ac.jp (ett.sat.t.u-tokyo.ac.jp [133.11.135.3]) by mailhosting.itc.u-tokyo.ac.jp (Mirapoint Messaging Server MOS 2.9.3.2) with ESMTP id AHV47997; Wed, 12 Feb 2003 17:24:54 +0900 (JST) Date: Wed, 12 Feb 2003 17:24:54 +0900 Message-ID: From: Hidetoshi Shimokawa To: freebsd-arch@freebsd.org Cc: "Glenn Gombert" , julian@elischer.org, ikob@freebsd.org Subject: You may want to have FireWire.... In-Reply-To: <20030209193737.D91611DEAB@www.fastmail.fm> User-Agent: Wanderlust/2.11.0 (Wonderwall) REMI/1.14.3 (Matsudai) FLIM/1.14.3 (=?ISO-8859-1?Q?Unebigory=F2mae?=) APEL/10.3 MULE XEmacs/21.4 (patch 8) (Honest Recruiter) (i386--freebsd) X-Face: OE([KxWyJI0r[R~S/>7ia}SJ)i%a,$-9%7{*yihQk|]gl}2p#"oXmX/fT}Bn7: #j7i14gu$jgR\S*&C3R/pJX List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Do you think this useful? /\ Hidetoshi Shimokawa \/ simokawa@sat.t.u-tokyo.ac.jp PGP public key: http://www.sat.t.u-tokyo.ac.jp/~simokawa/pgp.html -------------------------------- * What's dcons and fwchat for? I wrote this driver(dcons(4)) and utility(fwchat(8)) for a replacement of the serial console. This framework exploits physical access faculty of the IEEE1394 OHCI chip(fwochi(4)). This means that you cannot access this console until the OHCI is initialized on target system but once it's initialized, there is no need of OS support to do transactions on the FireWire bus until next bus reset occurs. This enables us to use this console as DDB/GDB port too. By analogy with pty(4), dcons(4) acts like a slave device and fwchat(8) acts like a master device, but they are on different machines and interact via FireWire. Please note dcons(4) is device(FireWire)-independent by itself. (And almost plathome independent) dcons(4) can be loaded as a modules but I recommend you to link it to kernel statically for several reasons especially when used with gdb. On -current, thanks to multiple console device support(conscontrol(8)), we can use syscons(4) and dcons(4) as console and DDB port at the same time. On -stable, because the only one console device is allowed, we have to hijack the console port from syscons(4) to use dcons(4) as the console. (Define FORCE_CONSOLE to 1 in dcons.c for this.) Even it is not used as console, it can still be used as GDB(not DDB) port. I have tested them on i386 -current mainly. But I expect that they should work on -stable and other architecture too. I'm not sure whether fwochi(4) driver is working on other plathomes like sparc64 and ia64. Even it doesn't work, it shouldn't be so difficult to make it work because we only need initialization step to work on the target system at least. My VAIO C1 has no serial port, no ethernet port nor SCSI bus but a i.Link(FireWire) port. Now it has two virtual serial port by dcons(4), a virtual ethernet port by if_fwe(4) and native SBP(SCSI) bus by sbp(4) :-) * Open problem: buffer allocation fwchat(8) needs to know the physical address of the buffer on the target, this is the most pain part for users. If we can allocated the buffer at some fixed physical address, it's very useful. (4 bytes buffer should be enough to hold a pointer to the real buffer.) I need some help from someone familiar with bootstrap process on each architecture! But I still don't want dcons(4) to be architecture dependent too much. there are several scenarios when the initialization routine is called. (1) called from cninit(): dcons(4) is statically compiled into kernel and its (console) priority is higher than any other devices. In this case, we can not use (contig)malloc yet. Currently I allocated static buf for this case. (2) called in SI_SUB_DRIVERS stage ( i) statically linked but low priority we can use contigmalloc'ed and static buf. ( ii) loaded as module in loader. we can use contigmalloc'ed and static buf. (iii) loaded as module after boot. we have to use contigmalloc because static buf might be fragmented in physical address space. I'm not sure how many developpers and administrators use serial console extensively and how many of them think this framework is useful. Any comments and suggestions are welcome! The current development code can be find under: http://people.freebsd.org/~simokawa/firewire with name "dcons-XXX.tar.gz". Hidetoshi Simokawa , FWCHAT(8) FreeBSD System Manager's Manual FWCHAT(8) NAME fwchat - Proxy between dcons over FireWire and TCP/IP SYNOPSIS fwchat [-brv] [-p base port] -t target eui64 -a address --------- ------------ ------- DESCRIPTION The fwchat utility is designed to provide a way for users to access dcons(4) (dumb console device) on a target system. The fwchat interacts with dcons over FireWire and interact with a user over TCP/IP. The fwchat and dcons(4) communicate using 2 port, one for console port and the other for remote gdb port. The user is supposed to access fwchat using telnet and/or gdb. The fwchat listens on local- host:base port for console port and localhost:(base port + 1) for gdb - - port. By analogy with pty(4) device, the dcons(4) acts as a slave device and fwchat acts as a master device with telnetd(8). -b Translate Ctrl-C to ALT BREAK(CR + ~ + Ctrl-B) on gdb port. - -r Disable adhoc workaround for telnet and forward raw data on con- sole port. It might be useful when you access fwchat using other than telnet. -v Verbose debug output. Multiple '-v' increase verbosity. -p base port --------- Specify base port. Currently, default is 5555. - -t target eui64 ------------ Specify the 64bit extended unique identifier of the target. -a address ------- Specify the physical address of dcons buffer. See dcons(4) for details. EXAMPLE To run fwchat, you have to specify the eui64 of the target and the pyhsi- cal address of the dcons buffer. You can obtain EUI64 by running fwcontorl(4) without options. The first EUI64 is of the host running fwcontrol and others on the bus follow. # fwcontrol 2 devices (info len=2) - node EUI64 status 1 7766554433221100 0 0 0011223344556677 1 The EUI64 doesn't change unless you chage the hardware as the ethernet address. Getting the physical address of the dcons buffer is somewhat a pain. There are several ways to obtain it but only one method is explained here. See dcons(4) for the other methods. The method explained here needs kernel image including symbol table but can be done on host side without help of the target. # sysctl hw.firewire.fwmem.eui64 hi=0x00112233 - # sysctl hw.firewire.fwmem.eui64 lo=0x44556677 - # gdb -k kernel.debug /dev/fwmem0 (kgdb) p/x dcons paddr - $1 = 0x123450 Now we can run the fwchat. # fwchat -vb -p 12345 -t 0x00112233445566677 -a 0x123450 Currently, fwchat doesn't become a daemon. Open another window and run: % telnet localhost:12345 You'll get console output of the target and login prompt if a getty is running on dcons. You can break to DDB with ALT BREAK (CR + ~ + Ctrl-B) - if DDB and ALT BREAK TO DEBUGGER is enabled in the target kernel. - - - Using gdb port is almost the same as remote gdb over serial line except using TCP/IP instead of /dev/cu*. See "On-line Kernel Debuggind Using Remote GDB" section of The FreeBSD Developpers Handbook. % gdb -k kernel.debug (kgdb) target remote :12346 Once gdb is attached and you specified '-b' option to fwchat, typing "Ctrl-C" on gdb causes break to debugger. FILES /dev/fwmem0 SEE ALSO ddb(4), firewire(4), fwohci(4), fwcontrol(8,) gdb(1), telnet(1) AUTHORS Hidetoshi Shimokawa BUGS This utility is still under development and it could have serious secu- rity problem. FreeBSD 4.7 February 11, 2003 FreeBSD 4.7 DCONS(4) FreeBSD Kernel Interfaces Manual DCONS(4) NAME dcons - dumb console device driver SYNOPSIS device dcons options DDB options ALT BREAK TO DEBUGGER - - - device fwohci device firewire DESCRIPTION The dcons device is the simple console device which just reads from and writes to an allocated buffer for input and output respectivly. It is no use by itself and it is supposed that the buffer is accessed via a bus like FireWire(4) for interaction. The buffer consists of 4 channels. There are 2 ports, one for console tty and other is GDB ports then each port has a input channel and a out- put channel. The physical address of the buffer is sometimes neccesary to acess the buffer. You can get the address by sysctl(8) or dmesg(8) EXAMPLE If you want to run getty(8) on dcons, insert following line into /etc/ttys(5) and send a HUP signal to init(8) using kill(1). dcons "/usr/libexec/getty std.9600" vt100 on secure You can use either of the following commands to obtain physical address of the buffer. % sysctl kern.dcons.paddr kern.dcons.paddr: 4732704 % dmesg | grep dcons: | tail -1 dcons: virtual 0xc0483720 physical 0x483720 quad 0x120dc8 In this example, the buffer is located at 4732704 in decimal and 0x483720 in hex. Once fwochi(4) device is initialized to allow physical access, the buffer can be accessed from another host via FireWire bus using fwchat(8) appli- cation. See fwchat(8) for more details. FILES /dev/dcons /dev/dconsctl /etc/ttys SEE ALSO ddb(4), firewire(4), fwohci(4), fwchat(8), fwcontrol(8), ttys(5) AUTHORS Hidetoshi Shimokawa BUGS This driver is still under development. FreeBSD 4.7 February 11, 2003 FreeBSD 4.7 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 0:56:58 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0E7C137B401; Wed, 12 Feb 2003 00:56:57 -0800 (PST) Received: from axl.seasidesoftware.co.za (axl.seasidesoftware.co.za [196.31.7.201]) by mx1.FreeBSD.org (Postfix) with ESMTP id BE36643F93; Wed, 12 Feb 2003 00:56:54 -0800 (PST) (envelope-from sheldonh@starjuice.net) Received: from sheldonh by axl.seasidesoftware.co.za with local (Exim 4.12) id 18ishC-000APq-00; Wed, 12 Feb 2003 10:56:46 +0200 Date: Wed, 12 Feb 2003 10:56:46 +0200 From: Sheldon Hearn To: Wesley Peters Cc: Thomas Quinot , arch@freebsd.org Subject: Re: syslog.conf syntax change (multiple program/host specifications) Message-ID: <20030212085646.GE39728@starjuice.net> Mail-Followup-To: Wesley Peters , Thomas Quinot , arch@freebsd.org References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302111430.03156.wes@softweyr.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200302111430.03156.wes@softweyr.com> User-Agent: Mutt/1.5.3i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On (2003/02/11 14:30), Wesley Peters wrote: > Excellent. I have another simple change to syslog I want to cast before the > audience for consideration. It's about that time of year again when someone points out that syslogd(8) is just begging for an alternative configuration file, syslog.xml. :-) And then it'll be time for someone to point out that such notions are nothing more than fancy because we don't have XML support in the base syst... Oh dear. Ciao, Sheldon. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 1:59:32 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1F9C737B401; Wed, 12 Feb 2003 01:59:31 -0800 (PST) Received: from crf-consulting.co.uk (pc-80-194-99-19-hy.blueyonder.co.uk [80.194.99.19]) by mx1.FreeBSD.org (Postfix) with ESMTP id 76A9243F75; Wed, 12 Feb 2003 01:59:29 -0800 (PST) (envelope-from nik@crf-consulting.co.uk) Received: from clan.nothing-going-on.org (clan.nothing-going-on.org [192.168.1.20]) by crf-consulting.co.uk (8.12.3/8.12.3) with ESMTP id h1C9xRaw069068; Wed, 12 Feb 2003 09:59:27 GMT (envelope-from nik@catkin) Received: from clan.nothing-going-on.org (localhost [127.0.0.1]) by clan.nothing-going-on.org (8.12.3/8.12.3) with ESMTP id h1C9xRm0083449; Wed, 12 Feb 2003 09:59:27 GMT (envelope-from nik@clan.nothing-going-on.org) Received: (from nik@localhost) by clan.nothing-going-on.org (8.12.3/8.12.3/Submit) id h1C9xPV8083448; Wed, 12 Feb 2003 09:59:25 GMT Date: Wed, 12 Feb 2003 09:59:25 +0000 From: Nik Clayton To: Sheldon Hearn Cc: Wesley Peters , Thomas Quinot , arch@freebsd.org Subject: Re: syslog.conf syntax change (multiple program/host specifications) Message-ID: <20030212095925.GA83388@clan.nothing-going-on.org> References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302111430.03156.wes@softweyr.com> <20030212085646.GE39728@starjuice.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="jRHKVT23PllUwdXP" Content-Disposition: inline In-Reply-To: <20030212085646.GE39728@starjuice.net> User-Agent: Mutt/1.4i Organization: FreeBSD Project Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --jRHKVT23PllUwdXP Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Feb 12, 2003 at 10:56:46AM +0200, Sheldon Hearn wrote: > It's about that time of year again when someone points out that > syslogd(8) is just begging for an alternative configuration file, > syslog.xml. :-) >=20 > And then it'll be time for someone to point out that such notions are > nothing more than fancy because we don't have XML support in the base > syst... >=20 > Oh dear. GEOM. N --=20 FreeBSD: The Power to Serve http://www.freebsd.org/ (__) FreeBSD Documentation Project http://www.freebsd.org/docproj/ \\\'',) \/ \= ^ --- 15B8 3FFC DDB4 34B0 AA5F 94B7 93A8 0764 2C37 E375 --- .\._/= _) --jRHKVT23PllUwdXP Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (FreeBSD) iD8DBQE+Shr8k6gHZCw343URAih8AJ4uokmq8oFvTYL4mXyxN1vyjSYzYQCeNnsd BJvTjugI7gulyTrF1ACPQCE= =JF2J -----END PGP SIGNATURE----- --jRHKVT23PllUwdXP-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 2:38:21 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B042A37B401; Wed, 12 Feb 2003 02:38:20 -0800 (PST) Received: from axl.seasidesoftware.co.za (axl.seasidesoftware.co.za [196.31.7.201]) by mx1.FreeBSD.org (Postfix) with ESMTP id BDA7043FA3; Wed, 12 Feb 2003 02:38:18 -0800 (PST) (envelope-from sheldonh@starjuice.net) Received: from sheldonh by axl.seasidesoftware.co.za with local (Exim 4.12) id 18iuHP-000B4m-00; Wed, 12 Feb 2003 12:38:15 +0200 Date: Wed, 12 Feb 2003 12:38:15 +0200 From: Sheldon Hearn To: Nik Clayton Cc: Wesley Peters , Thomas Quinot , arch@freebsd.org Subject: Re: syslog.conf syntax change (multiple program/host specifications) Message-ID: <20030212103815.GH39728@starjuice.net> Mail-Followup-To: Nik Clayton , Wesley Peters , Thomas Quinot , arch@freebsd.org References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302111430.03156.wes@softweyr.com> <20030212085646.GE39728@starjuice.net> <20030212095925.GA83388@clan.nothing-going-on.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030212095925.GA83388@clan.nothing-going-on.org> User-Agent: Mutt/1.5.3i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On (2003/02/12 09:59), Nik Clayton wrote: > > It's about that time of year again when someone points out that > > syslogd(8) is just begging for an alternative configuration file, > > syslog.xml. :-) > > > > And then it'll be time for someone to point out that such notions are > > nothing more than fancy because we don't have XML support in the base > > syst... > > > > Oh dear. > > GEOM. Surely you don't mean to suggest that libbsdxml could be used to provide configuration files that are flexible, extensible, readable and machine-parsable? ;-) Ciao, Sheldon. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 4:16:43 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DFBD337B401; Wed, 12 Feb 2003 04:16:41 -0800 (PST) Received: from crf-consulting.co.uk (pc-80-194-99-19-hy.blueyonder.co.uk [80.194.99.19]) by mx1.FreeBSD.org (Postfix) with ESMTP id 056D743F85; Wed, 12 Feb 2003 04:16:40 -0800 (PST) (envelope-from nik@crf-consulting.co.uk) Received: from clan.nothing-going-on.org (clan.nothing-going-on.org [192.168.1.20]) by crf-consulting.co.uk (8.12.3/8.12.3) with ESMTP id h1CCGcaw070029; Wed, 12 Feb 2003 12:16:39 GMT (envelope-from nik@catkin) Received: from clan.nothing-going-on.org (localhost [127.0.0.1]) by clan.nothing-going-on.org (8.12.3/8.12.3) with ESMTP id h1CCGcm0083858; Wed, 12 Feb 2003 12:16:38 GMT (envelope-from nik@clan.nothing-going-on.org) Received: (from nik@localhost) by clan.nothing-going-on.org (8.12.3/8.12.3/Submit) id h1CCGcqQ083857; Wed, 12 Feb 2003 12:16:38 GMT Date: Wed, 12 Feb 2003 12:16:38 +0000 From: Nik Clayton To: Nik Clayton , Wesley Peters , Thomas Quinot , arch@freebsd.org Subject: Re: syslog.conf syntax change (multiple program/host specifications) Message-ID: <20030212121638.GD83388@clan.nothing-going-on.org> References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302111430.03156.wes@softweyr.com> <20030212085646.GE39728@starjuice.net> <20030212095925.GA83388@clan.nothing-going-on.org> <20030212103815.GH39728@starjuice.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Q0rSlbzrZN6k9QnT" Content-Disposition: inline In-Reply-To: <20030212103815.GH39728@starjuice.net> User-Agent: Mutt/1.4i Organization: FreeBSD Project Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --Q0rSlbzrZN6k9QnT Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Feb 12, 2003 at 12:38:15PM +0200, Sheldon Hearn wrote: > Surely you don't mean to suggest that libbsdxml could be used to provide > configuration files that are flexible, extensible, readable and > machine-parsable? ;-) Who, me? For a very early snapshot of something I was doing (checks dates) good God, over a year ago, take a look at: http://people.freebsd.org/~nik/xml-servers/ Good for a laugh if nothing else. N --=20 FreeBSD: The Power to Serve http://www.freebsd.org/ (__) FreeBSD Documentation Project http://www.freebsd.org/docproj/ \\\'',) \/ \= ^ --- 15B8 3FFC DDB4 34B0 AA5F 94B7 93A8 0764 2C37 E375 --- .\._/= _) --Q0rSlbzrZN6k9QnT Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (FreeBSD) iD8DBQE+Sjslk6gHZCw343URAkjxAJ9buUAqwinfkvKYOfBYMk6lcQwQlgCfWzDR QH3MOO80mxQitKICIDV1HA8= =hHj6 -----END PGP SIGNATURE----- --Q0rSlbzrZN6k9QnT-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 5:53:25 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6CA3537B401 for ; Wed, 12 Feb 2003 05:53:24 -0800 (PST) Received: from khavrinen.lcs.mit.edu (khavrinen.lcs.mit.edu [18.24.4.193]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2930043F85 for ; Wed, 12 Feb 2003 05:53:23 -0800 (PST) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: from khavrinen.lcs.mit.edu (localhost [IPv6:::1]) by khavrinen.lcs.mit.edu (8.12.6/8.12.6) with ESMTP id h1CDrLbs024927 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=OK); Wed, 12 Feb 2003 08:53:22 -0500 (EST) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: (from wollman@localhost) by khavrinen.lcs.mit.edu (8.12.6/8.12.6/Submit) id h1CDrLMg024924; Wed, 12 Feb 2003 08:53:21 -0500 (EST) (envelope-from wollman) Date: Wed, 12 Feb 2003 08:53:21 -0500 (EST) From: Garrett Wollman Message-Id: <200302121353.h1CDrLMg024924@khavrinen.lcs.mit.edu> To: John David Duncan Cc: arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) In-Reply-To: References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302112310.h1BNAUBS019097@khavrinen.lcs.mit.edu> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG < said: > Then what would happen to newsyslog? > People use it for rotating other logs outside of syslogd. /var/run/food.pid HUP -GAWollman To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 6:11:31 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 05A1C37B401 for ; Wed, 12 Feb 2003 06:11:30 -0800 (PST) Received: from khavrinen.lcs.mit.edu (khavrinen.lcs.mit.edu [18.24.4.193]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3B19643F93 for ; Wed, 12 Feb 2003 06:11:29 -0800 (PST) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: from khavrinen.lcs.mit.edu (localhost [IPv6:::1]) by khavrinen.lcs.mit.edu (8.12.6/8.12.6) with ESMTP id h1CEBSbs025075 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=OK); Wed, 12 Feb 2003 09:11:28 -0500 (EST) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: (from wollman@localhost) by khavrinen.lcs.mit.edu (8.12.6/8.12.6/Submit) id h1CEBRSe025071; Wed, 12 Feb 2003 09:11:27 -0500 (EST) (envelope-from wollman) Date: Wed, 12 Feb 2003 09:11:27 -0500 (EST) From: Garrett Wollman Message-Id: <200302121411.h1CEBRSe025071@khavrinen.lcs.mit.edu> To: Wes Peters Cc: arch@FreeBSD.org Subject: Re: syslog.conf syntax change (multiple program/host specifications) In-Reply-To: <200302120632.36583.wes@softweyr.com> References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302112310.h1BNAUBS019097@khavrinen.lcs.mit.edu> <200302120632.36583.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG < said: > I'd much prefer to develop a simple yacc-able grammar and > develope the configuration parser that way, or use a key=value > configuration, so human beings can understand the configuration > with a program to help them. I don't think either of these is truly a useful answer, particularly for something like syslog which is already straining for a more expressive syntax. There are just too many little languages in the system already, each one subtly different in syntax, such that it's nearly impossible to do anything programmatic with them that isn't built in to the program that they configure. I'd prefer S-expressions, myself, but in terms of general accessibility and availability of automated processing tools, XML wins over Lisp. (However much I dislike XML, I find myself writing more and more of it as time goes by, and I've spent a good deal of my time becoming familiar with its restrictions. Some of the languages built on top of XML, like XSLT, are much worse [says the man who will some day release his XSLT package for doing Web photo galleries].) -GAWollman To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 7:18: 9 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BFE0637B401 for ; Wed, 12 Feb 2003 07:18:08 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3011A43FA3 for ; Wed, 12 Feb 2003 07:18:08 -0800 (PST) (envelope-from wes@softweyr.com) Received: from softweyr.homeunix.net (66-75-151-22.san.rr.com [66.75.151.22]) by smtp-relay.omnis.com (Postfix) with ESMTP id 6160442E39; Wed, 12 Feb 2003 07:18:05 -0800 (PST) From: Wes Peters Organization: Softweyr To: jos@catnook.com, arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) Date: Wed, 12 Feb 2003 15:18:03 +0000 User-Agent: KMail/1.5 References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302120632.36583.wes@softweyr.com> <20030212065038.GA99658@lizzy.catnook.com> In-Reply-To: <20030212065038.GA99658@lizzy.catnook.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200302121518.03235.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wednesday 12 February 2003 06:50, Jos Backus wrote: > On Wed, Feb 12, 2003 at 06:32:36AM +0000, Wes Peters wrote: > > XML is the ultimate Least Common Denominator; it's the best tool for > > every job, except you can come up with a better tool for any job. > > What about YAML? > > http://www.yaml.org/ Looks good, except there is no C (or C++) callable library yet. Sort of. Sigh. -- Where am I, and what am I doing in this handbasket? Wes Peters wes@softweyr.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 7:21:36 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 23DB837B405 for ; Wed, 12 Feb 2003 07:21:35 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9692D43F93 for ; Wed, 12 Feb 2003 07:21:34 -0800 (PST) (envelope-from wes@softweyr.com) Received: from softweyr.homeunix.net (66-75-151-22.san.rr.com [66.75.151.22]) by smtp-relay.omnis.com (Postfix) with ESMTP id CCA3542EF4; Wed, 12 Feb 2003 07:21:33 -0800 (PST) From: Wes Peters Organization: Softweyr To: Garrett Wollman Subject: Re: syslog.conf syntax change (multiple program/host specifications) Date: Wed, 12 Feb 2003 15:21:33 +0000 User-Agent: KMail/1.5 Cc: arch@FreeBSD.org References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302120632.36583.wes@softweyr.com> <200302121411.h1CEBRSe025071@khavrinen.lcs.mit.edu> In-Reply-To: <200302121411.h1CEBRSe025071@khavrinen.lcs.mit.edu> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200302121521.33506.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wednesday 12 February 2003 14:11, Garrett Wollman wrote: > < said: > > I'd much prefer to develop a simple yacc-able grammar and > > develope the configuration parser that way, or use a key=value > > configuration, so human beings can understand the configuration > > with a program to help them. > > I don't think either of these is truly a useful answer, particularly > for something like syslog which is already straining for a more > expressive syntax. There are just too many little languages in the > system already, each one subtly different in syntax, such that it's > nearly impossible to do anything programmatic with them that isn't > built in to the program that they configure. So you're preferring the software over the human operator. In my world view, that is completely whacked. It works fine if you're planning on providing a program to present the configuration to the user in a GUI or some such interface, but not at all if you expect the user to edit the configuration file as the normal mode of configuring the program. Is THAT answer useful enough? -- Where am I, and what am I doing in this handbasket? Wes Peters wes@softweyr.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 7:22:19 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 25A3637B401 for ; Wed, 12 Feb 2003 07:22:18 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id A961943FBF for ; Wed, 12 Feb 2003 07:22:17 -0800 (PST) (envelope-from wes@softweyr.com) Received: from softweyr.homeunix.net (66-75-151-22.san.rr.com [66.75.151.22]) by smtp-relay.omnis.com (Postfix) with ESMTP id D06D842FF0; Wed, 12 Feb 2003 07:22:16 -0800 (PST) From: Wes Peters Organization: Softweyr To: John David Duncan , Garrett Wollman Subject: Re: syslog.conf syntax change (multiple program/host specifications) Date: Wed, 12 Feb 2003 15:22:16 +0000 User-Agent: KMail/1.5 Cc: arch@FreeBSD.ORG References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302112310.h1BNAUBS019097@khavrinen.lcs.mit.edu> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200302121522.16726.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wednesday 12 February 2003 07:36, John David Duncan wrote: > > 1) Integrate syslogd and newsyslog. > > 2) Replace syslog.conf and newsyslog.conf with a new configuration file > > that contains complete descriptions of log sources and destinations. > > Then what would happen to newsyslog? > People use it for rotating other logs outside of syslogd. A) Leave it around for people that still use it thusly, or B) Move it to ports! ;^) -- Where am I, and what am I doing in this handbasket? Wes Peters wes@softweyr.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 7:34:53 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5D13337B401; Wed, 12 Feb 2003 07:34:52 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id B243843FDD; Wed, 12 Feb 2003 07:34:51 -0800 (PST) (envelope-from wes@softweyr.com) Received: from softweyr.homeunix.net (66-75-151-22.san.rr.com [66.75.151.22]) by smtp-relay.omnis.com (Postfix) with ESMTP id 4FA7C4387D; Wed, 12 Feb 2003 07:25:03 -0800 (PST) From: Wes Peters Organization: Softweyr To: Sheldon Hearn , Nik Clayton Subject: Re: syslog.conf syntax change (multiple program/host specifications) Date: Wed, 12 Feb 2003 15:25:03 +0000 User-Agent: KMail/1.5 Cc: Thomas Quinot , arch@freebsd.org References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <20030212095925.GA83388@clan.nothing-going-on.org> <20030212103815.GH39728@starjuice.net> In-Reply-To: <20030212103815.GH39728@starjuice.net> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200302121525.03156.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wednesday 12 February 2003 10:38, Sheldon Hearn wrote: > On (2003/02/12 09:59), Nik Clayton wrote: > > > It's about that time of year again when someone points out that > > > syslogd(8) is just begging for an alternative configuration file, > > > syslog.xml. :-) > > > > > > And then it'll be time for someone to point out that such notions are > > > nothing more than fancy because we don't have XML support in the base > > > syst... > > > > > > Oh dear. > > > > GEOM. > > Surely you don't mean to suggest that libbsdxml could be used to provide > configuration files that are flexible, extensible, readable and > machine-parsable? ;-) Well, pick 2. Or maybe 3. No, 2 at best. XML is extensible (sort of, but you have to re-write the parsing code to do it) and machine- parsable. -- Where am I, and what am I doing in this handbasket? Wes Peters wes@softweyr.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 8:15:47 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0E1D037B401 for ; Wed, 12 Feb 2003 08:15:46 -0800 (PST) Received: from khavrinen.lcs.mit.edu (khavrinen.lcs.mit.edu [18.24.4.193]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2018E43F3F for ; Wed, 12 Feb 2003 08:15:43 -0800 (PST) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: from khavrinen.lcs.mit.edu (localhost [IPv6:::1]) by khavrinen.lcs.mit.edu (8.12.6/8.12.6) with ESMTP id h1CGFdbs025694 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=OK); Wed, 12 Feb 2003 11:15:39 -0500 (EST) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: (from wollman@localhost) by khavrinen.lcs.mit.edu (8.12.6/8.12.6/Submit) id h1CGFdGG025691; Wed, 12 Feb 2003 11:15:39 -0500 (EST) (envelope-from wollman) Date: Wed, 12 Feb 2003 11:15:39 -0500 (EST) From: Garrett Wollman Message-Id: <200302121615.h1CGFdGG025691@khavrinen.lcs.mit.edu> To: Wes Peters Cc: arch@FreeBSD.org Subject: Re: syslog.conf syntax change (multiple program/host specifications) In-Reply-To: <200302121521.33506.wes@softweyr.com> References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302120632.36583.wes@softweyr.com> <200302121411.h1CEBRSe025071@khavrinen.lcs.mit.edu> <200302121521.33506.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG < said: > So you're preferring the software over the human operator. I would not necessarily jump to that conclusion. What I want to eschew is a proliferation of lots of little languages, each one subtly different, such that users are forced to learn all of them. We already have: foocap fstab group inetd.conf files manpath.config master.passwd named.conf and dhcpd.conf newsyslog.conf nsswitch.conf ntp.conf pam.d/* passwd pccard.conf ppp.conf {rc,periodic}.conf shells and ftpusers ssh{,d}_config syslog.conf ttys Every single one of these has a different syntax that the admin must learn in addition to the relevant semantics, and which any sort of front-end or configuration-analysis tool must be able to interpret, in order to do anything useful with the programs they control. The benefit of something like XML (or Lisp, for that matter) is that, while still providing for functionally significant differences, the *lexical* structure is identical across many functions -- and this makes it much easier to use other tools (like structured editors) to maintain and document the files. Yes, XML syntax is pretty hateful -- but because it is lexically regular, my editor can do much of the work for me. It can do even more of the work when the XML is coupled with a DTD so that it understands more of the structure. -GAWollman To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 9: 3: 2 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B7C7037B401; Wed, 12 Feb 2003 09:03:01 -0800 (PST) Received: from Daffy.timing.com (daffy.timing.com [206.168.13.218]) by mx1.FreeBSD.org (Postfix) with ESMTP id 68FC443F3F; Wed, 12 Feb 2003 09:03:00 -0800 (PST) (envelope-from ben@timing.com) Received: from piglet.timing.com (piglet.timing.com [206.168.13.178]) by Daffy.timing.com (8.11.3/8.11.3) with ESMTP id h1CH2sH10447; Wed, 12 Feb 2003 10:02:54 -0700 (MST) (envelope-from ben@timing.com) Received: from piglet.timing.com (localhost.timing.com [127.0.0.1]) by piglet.timing.com (8.12.6/8.12.6) with ESMTP id h1CH2sXl055059; Wed, 12 Feb 2003 10:02:54 -0700 (MST) (envelope-from ben@piglet.timing.com) Received: (from ben@localhost) by piglet.timing.com (8.12.6/8.12.6/Submit) id h1CH2aaj055056; Wed, 12 Feb 2003 10:02:36 -0700 (MST) From: Ben Mesander MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15946.32300.248327.613178@piglet.timing.com> Date: Wed, 12 Feb 2003 10:02:36 -0700 To: Nik Clayton Cc: Wesley Peters , Thomas Quinot , arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) In-Reply-To: <20030212121638.GD83388@clan.nothing-going-on.org> References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302111430.03156.wes@softweyr.com> <20030212085646.GE39728@starjuice.net> <20030212095925.GA83388@clan.nothing-going-on.org> <20030212103815.GH39728@starjuice.net> <20030212121638.GD83388@clan.nothing-going-on.org> X-Mailer: VM 7.00 under Emacs 21.2.95.2 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Nik Clayton writes: > For a very early snapshot of something I was doing (checks dates) good > God, over a year ago, take a look at: > > http://people.freebsd.org/~nik/xml-servers/ > > Good for a laugh if nothing else. I wrote a cluster management tool for a Linux distribution that used this approach; the XSLT files generated all configuration files for hosts in the cluster from a master set of XML files (which was generated from a GUI; not created by hand; I don't like editing XML by hand). It also dealt with filling in any unique information for each host. The benefits of this approach seemed to increase as the number of hosts you were generating from a single set of config files increased. It would have been a fairly painful way of setting up one machine vs. just editing a few configuration files, but it was a pretty nice way to generate configs for 128 nearly-identical machines. --Ben To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 9:40:19 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E991D37B401 for ; Wed, 12 Feb 2003 09:40:17 -0800 (PST) Received: from heron.mail.pas.earthlink.net (heron.mail.pas.earthlink.net [207.217.120.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id 607E643F3F for ; Wed, 12 Feb 2003 09:40:17 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0017.cvx22-bradley.dialup.earthlink.net ([209.179.198.17] helo=mindspring.com) by heron.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18j0rl-0002VJ-00; Wed, 12 Feb 2003 09:40:14 -0800 Message-ID: <3E4A869B.EABBC83D@mindspring.com> Date: Wed, 12 Feb 2003 09:38:35 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Wes Peters Cc: Garrett Wollman , arch@FreeBSD.org Subject: Re: syslog.conf syntax change (multiple program/host specifications) References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302120632.36583.wes@softweyr.com> <200302121411.h1CEBRSe025071@khavrinen.lcs.mit.edu> <200302121521.33506.wes@softweyr.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a451de94fcc991560d71476593d5832bce350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Wes Peters wrote: > So you're preferring the software over the human operator. In my world > view, that is completely whacked. It works fine if you're planning on > providing a program to present the configuration to the user in a GUI or > some such interface, but not at all if you expect the user to edit the > configuration file as the normal mode of configuring the program. > > Is THAT answer useful enough? In the fine tradition of Open Source, you build a machine-readable configuration file for the program, and then you leave the tool as something for someone else to write. And no one ever write it. But in *theory*, your replacement for the old code is just as good as the old code. Heck, it's even better! ... In terms of rolling newsyslog in: please don't. As someone else pointed out, there are programs that don't use syslog that do use newsyslog (e.g. Samba, #1 answer, on the board...). The other issue is that "revisionist history" should be a feature of a newsyslog program; that is, if you set size limits and an old version count limit, and you start the program up, it should "take care of" old files. Not something you'd want in syslog. Finally, it's useful to replace syslog with a program that does the same thing *and* remember program state. Just because some people don't follow the machine readable logging draft RFC, doesn't mean that it can't be useful to other people (e.g. for external SNMP instrumentation). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 10:26:53 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7188837B401 for ; Wed, 12 Feb 2003 10:26:51 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id BEADD43F75 for ; Wed, 12 Feb 2003 10:26:50 -0800 (PST) (envelope-from wes@softweyr.com) Received: from softweyr.homeunix.net (66-75-151-22.san.rr.com [66.75.151.22]) by smtp-relay.omnis.com (Postfix) with ESMTP id 0DD05437C9; Wed, 12 Feb 2003 10:25:59 -0800 (PST) From: Wes Peters Organization: Softweyr To: Terry Lambert Subject: Re: syslog.conf syntax change (multiple program/host specifications) Date: Wed, 12 Feb 2003 18:25:57 +0000 User-Agent: KMail/1.5 Cc: Garrett Wollman , arch@FreeBSD.org References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302121521.33506.wes@softweyr.com> <3E4A869B.EABBC83D@mindspring.com> In-Reply-To: <3E4A869B.EABBC83D@mindspring.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200302121825.57798.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wednesday 12 February 2003 17:38, Terry Lambert wrote: > > In the fine tradition of Open Source, you build a machine-readable > configuration file for the program, and then you leave the tool as > something for someone else to write. And no one ever write it. > > But in *theory*, your replacement for the old code is just as good > as the old code. Heck, it's even better! Exactly. So the users get stuck editing XML files, which are even worse than win ini files. Gack. Of course this doesn't offend the "purists," they think everyone in the world should know XML anyhow. Personally I'd rather have my doctor studying how to effectively kill this recurring infection in my tear ducts than learning XML. > In terms of rolling newsyslog in: please don't. As someone else > pointed out, there are programs that don't use syslog that do use > newsyslog (e.g. Samba, #1 answer, on the board...). I didn't propose to REPLACE newsyslog, just to add some simple file size limits to syslogd. Go back and read the original message, which had to do with syslog racing along filling up filesystems faster than newsyslog can save the system. And we run newsyslog every 5 minutes, too. Log files are important, but not at the expense of keeping the system running. I imagine this *simple change* might be useful to others who run on flash or ram disks, and offer it for their consideration. > The other issue is that "revisionist history" should be a feature > of a newsyslog program; that is, if you set size limits and an old > version count limit, and you start the program up, it should "take > care of" old files. Not something you'd want in syslog. Nope, not at all. > Finally, it's useful to replace syslog with a program that does > the same thing *and* remember program state. Just because some > people don't follow the machine readable logging draft RFC, doesn't > mean that it can't be useful to other people (e.g. for external > SNMP instrumentation). So you're going to contribute a nice daemon any day now, right? -- Where am I, and what am I doing in this handbasket? Wes Peters wes@softweyr.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 10:28:56 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6077B37B401 for ; Wed, 12 Feb 2003 10:28:55 -0800 (PST) Received: from melusine.cuivre.fr.eu.org (melusine.cuivre.fr.eu.org [62.212.105.185]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5FA3043FA3 for ; Wed, 12 Feb 2003 10:28:54 -0800 (PST) (envelope-from thomas@cuivre.fr.eu.org) Received: by melusine.cuivre.fr.eu.org (Postfix, from userid 1000) id C1DFA2C3D2; Wed, 12 Feb 2003 19:28:52 +0100 (CET) Date: Wed, 12 Feb 2003 19:28:52 +0100 From: Thomas Quinot To: Garrett Wollman Cc: wes@softweyr.com, arch@FreeBSD.org Subject: Re: syslog.conf syntax change (multiple program/host specifications) Message-ID: <20030212182852.GA94317@melusine.cuivre.fr.eu.org> Reply-To: Thomas Quinot References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302112310.h1BNAUBS019097@khavrinen.lcs.mit.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <200302112310.h1BNAUBS019097@khavrinen.lcs.mit.edu> User-Agent: Mutt/1.4i X-message-flag: WARNING! Using Outlook can damage your computer. Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Le 2003-02-12, Garrett Wollman crivait : > Before starting on this I wanted to take up a somewhat easier task, > like inetd, which is also crying for relief from its kluge-encrusted > historical syntax. I am generally uneasy with such proposals, which would make FreeBSD incompatible with estblished practice of other *nix systems. I always found AIX's /etc/filesystems and RedHat's /etc/xinetd.* annoying, because their syntax is gratuitously different from time-honored standards. The benefit of such changes is often unclear; configuration files should be easy to read and modify for a human user. Whether or not they are easy to parse is irrelevant once the parser is written. :) Thomas. -- Thomas.Quinot@Cuivre.FR.EU.ORG To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 12:19:19 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4C3DB37B401 for ; Wed, 12 Feb 2003 12:19:18 -0800 (PST) Received: from khavrinen.lcs.mit.edu (khavrinen.lcs.mit.edu [18.24.4.193]) by mx1.FreeBSD.org (Postfix) with ESMTP id 88B3443FAF for ; Wed, 12 Feb 2003 12:19:17 -0800 (PST) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: from khavrinen.lcs.mit.edu (localhost [IPv6:::1]) by khavrinen.lcs.mit.edu (8.12.6/8.12.6) with ESMTP id h1CKIxbs026985 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=OK); Wed, 12 Feb 2003 15:18:59 -0500 (EST) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: (from wollman@localhost) by khavrinen.lcs.mit.edu (8.12.6/8.12.6/Submit) id h1CKIxjT026982; Wed, 12 Feb 2003 15:18:59 -0500 (EST) (envelope-from wollman) Date: Wed, 12 Feb 2003 15:18:59 -0500 (EST) From: Garrett Wollman Message-Id: <200302122018.h1CKIxjT026982@khavrinen.lcs.mit.edu> To: Thomas Quinot Cc: arch@FreeBSD.org Subject: Re: syslog.conf syntax change (multiple program/host specifications) In-Reply-To: <20030212182852.GA94317@melusine.cuivre.fr.eu.org> References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302112310.h1BNAUBS019097@khavrinen.lcs.mit.edu> <20030212182852.GA94317@melusine.cuivre.fr.eu.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG < said: > The benefit of such changes is often unclear; configuration files should > be easy to read and modify for a human user. Whether or not they are > easy to parse is irrelevant once the parser is written. :) The benefit of such changes is that, when new features are necessary (e.g., MAC labels for servers started by inetd), it is not necessary to create yet another kluge to shoehorn the new feature into the old, inflexible syntax. AIX actually had sort-of-the-right-idea, in that most of the AIX-specific configuration files are lexically identical, vary very little in syntax, and are easily extensible. Unfortunately, there is no great community of "stanza" format processing tools like there are for XML or even (gack) Windows .INI files. Don't get me wrong: I'm not putting forward XML because I like it; I am however noting that none of the comparably-capable alternatives have anything like the broad support that XML does. -GAWollman To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 12:38:29 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B304E37B401 for ; Wed, 12 Feb 2003 12:38:26 -0800 (PST) Received: from thunderer.cnchost.com (thunderer.concentric.net [207.155.252.72]) by mx1.FreeBSD.org (Postfix) with ESMTP id EC85A43FB1 for ; Wed, 12 Feb 2003 12:38:25 -0800 (PST) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (adsl-209-204-185-216.sonic.net [209.204.185.216]) by thunderer.cnchost.com id PAA00115; Wed, 12 Feb 2003 15:38:21 -0500 (EST) [ConcentricHost SMTP Relay 1.15] Message-ID: <200302122038.PAA00115@thunderer.cnchost.com> To: Garrett Wollman Cc: Wes Peters , arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) In-reply-to: Your message of "Wed, 12 Feb 2003 11:15:39 EST." <200302121615.h1CGFdGG025691@khavrinen.lcs.mit.edu> Date: Wed, 12 Feb 2003 12:38:20 -0800 From: Bakul Shah Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > eschew is a proliferation of lots of little languages, each one subtly > different, such that users are forced to learn all of them. We > already have: [list deleted] > Every single one of these has a different syntax that the admin must > learn in addition to the relevant semantics, and which any sort of > front-end or configuration-analysis tool must be able to interpret, in > order to do anything useful with the programs they control. The > benefit of something like XML (or Lisp, for that matter) is that, You have my vote for a Lisp syntax (s-experessions). S-exprs are quite readable and lightweight -- you can't go lighter than s-exprs and still have a universal structured data representation. And you can parse them with a couple of hundered lines of C code (without having to worry about any buffer overflows or limits except the system imposed ones). > Yes, XML syntax is pretty hateful -- but because it is lexically > regular, my editor can do much of the work for me. It can do even > more of the work when the XML is coupled with a DTD so that it > understands more of the structure. In my view XML is to data representation what Roman numerals are to math. You _can_ do math in Roman numerals but people didn't do much for many centuries. The "syntax" must have overwhelmed them:-) A notation should encourage, not stifle, expression of ideas. At any rate XML is just another data representation language de jour. Likely, it will replaced by another one (and another) and all the lemmings will go over _that_ cliff. s-exprs been around for over 40 years without "evolving" significantly. Rather than repeat all the arguments I'll point you to what knowledgeable people like Erik Naggum have to say about XML (search comp.lang.lisp on groups.google.com). Naggum used to be a strong proponent of SGML until he "saw the light"! A couple of recent thread on XML on comp.lang.lisp are worth browsing. Another view is to not get tied too closely to the actual representation but standardize on a procedural interface. Then one can choose different representations depending on the context (e.g. when you have 10 accounts, use of a btree database for /etc/passwd is foolish; when you have 10000 accounts, use of a linear list is foolish -- but client programs of /etc/passwd should *not* need to be recompiled if the representation is changed). Yet another but related view is to extend the known interface of files and directories to configuration as they are the only truly first class objects in unix (in terms of programs that understand them). Pretty much every config file you mentioned but particularly files with simple and nested parameters naturally fit this paradigm. For instance, /etc/rc.conf would be a directory, files within it name specified parameters and their contents if any are their values. It is natural to "union mount" /etc/defaults/rc.conf _under_ it. Similarly, a kernel config can be a directory. Subsystems can be subdirectories and so on. Specific derived kernel confs can be union mounted on atop of that. But I guess that way lies plan9. -- bakul PS: The following Alan Perlis epigrams seem apropos. Fools ignore complexity. Pragmatists suffer it. Some can avoid it. Geniuses remove it. It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures. Simplicity does not precede complexity, but follows it. Functions delay binding; data structures induce binding. Moral: Structure data late in the programming process. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 13:12:54 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 795CC37B401 for ; Wed, 12 Feb 2003 13:12:53 -0800 (PST) Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id 73B2F43F85 for ; Wed, 12 Feb 2003 13:12:52 -0800 (PST) (envelope-from des@ofug.org) Received: by flood.ping.uio.no (Postfix, from userid 2602) id 37CF1536E; Wed, 12 Feb 2003 22:12:50 +0100 (CET) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Bakul Shah Cc: Garrett Wollman , Wes Peters , arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) From: Dag-Erling Smorgrav Date: Wed, 12 Feb 2003 22:12:49 +0100 In-Reply-To: <200302122038.PAA00115@thunderer.cnchost.com> (Bakul Shah's message of "Wed, 12 Feb 2003 12:38:20 -0800") Message-ID: User-Agent: Gnus/5.090014 (Oort Gnus v0.14) Emacs/21.2 (i386--freebsd) References: <200302122038.PAA00115@thunderer.cnchost.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Bakul Shah writes: > Rather than repeat all the arguments I'll point you to what > knowledgeable people like Erik Naggum have to say about XML > (search comp.lang.lisp on groups.google.com). Naggum used to > be a strong proponent of SGML until he "saw the light"! A > couple of recent thread on XML on comp.lang.lisp are worth > browsing. Erik Naggum may be knowledgeable, but he's not what I would call well-balanced. I wouldn't rely too much on his opinion about anything if I were you. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 13:52:10 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0D70237B401; Wed, 12 Feb 2003 13:51:50 -0800 (PST) Received: from freebsd.org (p50867E3A.dip.t-dialin.net [80.134.126.58]) by mx1.FreeBSD.org (Postfix) with SMTP id 8985943FE5; Wed, 12 Feb 2003 13:51:33 -0800 (PST) (envelope-from james1k3@netscape.net) X-Priority: 1 From: To: "majordomo@freebsd.org" Subject: [private and confindential] Date: Thu, 13 Feb 2003 22:53:20 +0000 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_01BC2B74.89D1CCC0" Message-Id: <20030212215133.8985943FE5@mx1.FreeBSD.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This is a multi-part message in MIME format. ------=_NextPart_000_01BC2B74.89D1CCC0 Content-type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable AMSTERDAM, THE NETHERLANDS. CONFIDENTIAL BUSINESS PROPOSAL. You may be surprised to receive this letter from me since you do not know me personally. I am Carol Makali, the only child of James Makali,the most popular black farmer in Zimbabwe who was murdered in the land dispute in my country. I got your contact through network online hence decided to write you. Before the death of my father, he had taken me to Johannesburg to deposit the sum of US8.6 million (Eight million, Six Hundred thousand United States dollars), in one of the private security company, as he foresaw the looming danger in Zimbabwe this money was deposited in a box as gem stones to avoid much demurrage from security company. This amount was meant for the purchase of new machines and chemicals for the Farms and establishment of new farms in Swaziland. This land problem came when Zimbabwean President Mr. Robert Mugabe introduced a new Land Act Reform wholly affecting the rich white farmers and some few black farmers, and this resulted to the killing and mob action by Zimbabwean war veterans and some lunatics in the society. In fact a lot of people were killed because of this Land reform Act for which my father was one of the victims. It is against this background that, I and my mother fled Zimbabwe for fear of our lives and are currently staying in the Netherlands where we are seeking political asylum and moreso have decided to transfer my fathers money to a more reliable foreign account. since the law of Netherlands prohibits a refugee (asylum seeker) to open any bank account or to be involved in any financial transaction throughout the territorial zone of Netherlands, As the only child of my father, I am saddled with the responsibility of seeking a genuine foreign account where this money could be transferred without the knowledge of my government who are bent on taking everything we have got. The South African government seems to be playing along with them. I am faced with the dilemma of moving this amount of money out of South Africa for fear of going through the same experience in future, both countries have similar political history. As a businesslady,I am seeking for a partner who I have to entrust my future and that of my family in his hands, I must let you know that this transaction is risk free. If you accept to assist me and my mother, all I want you to do for me, is to make an arrangements with the security company to clear the consignment(funds) from their afiliate office here in the Netherlands as i have already given directives for the consignment to be brought to the Netherlands from South Africa.But before then all modalities will have to be put in place like change of ownership to the consignment and more importantly this money I intend to use for investment. I have two options for you. Firstly you can choose to have certain percentage of the money for nominating your account for this transaction. Or you can go into partnership with me for the proper profitable investment of the money in your country. Whichever the option you want, feel free to notify me. I have also mapped out 5% of this money for all kinds of expenses incurred in the process of this transaction. If you do not prefer a partnership I am willing to give you 20% of the money while the remaining 75% will be for my investment in your country. Contact me with the above telephone number or my E-mail address while I implore you to maintain the absolute secrecy required in this transaction. Thanks, GOD BLESS YOU Yours Faithfully, Carol Makali. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 15:20:51 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 17B0537B401 for ; Wed, 12 Feb 2003 15:20:49 -0800 (PST) Received: from rodney.cnchost.com (rodney.concentric.net [207.155.252.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5870643F3F for ; Wed, 12 Feb 2003 15:20:48 -0800 (PST) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (adsl-209-204-185-216.sonic.net [209.204.185.216]) by rodney.cnchost.com id SAA25721; Wed, 12 Feb 2003 18:20:39 -0500 (EST) [ConcentricHost SMTP Relay 1.15] Message-ID: <200302122320.SAA25721@rodney.cnchost.com> To: Dag-Erling Smorgrav Cc: Garrett Wollman , Wes Peters , arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) In-reply-to: Your message of "Wed, 12 Feb 2003 22:12:49 +0100." Date: Wed, 12 Feb 2003 15:20:37 -0800 From: Bakul Shah Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > > Rather than repeat all the arguments I'll point you to what > > knowledgeable people like Erik Naggum have to say about XML > > (search comp.lang.lisp on groups.google.com). Naggum used to > > be a strong proponent of SGML until he "saw the light"! A > > couple of recent thread on XML on comp.lang.lisp are worth > > browsing. > > Erik Naggum may be knowledgeable, but he's not what I would call > well-balanced. I wouldn't rely too much on his opinion about anything > if I were you. Are you impugning his sanity? I wonder what your fellow Norwegian would have to say about that:-) And people who spend more than 8 hours a day on FreeBSD are well-balanced? [No, don't answer that!] What I suggested was to look at the arguments Naggum & others have made (and, implicitly, to make up your own mind). Naggum can be quite vitriolic and unreasonable sometimes but pay attention to his technical posts and he actually makes a lot of sense. But not just him. Read the threads. A number of people who have used XML feel the same way about it and their arguments are worth listening to before believing any XML promises. For a while I considered using gnucash. With its help I converted my Quicken data file into gnucash'es XML form. About 6400 transactions (needing 550K in Quicken's QIF format) now need 7.5M of XML. But it is not just the fat. See for yourself just how ugly it is! Here is QIF of one transaction: D11/29' 1 U-36.90 T-36.90 PWhole Foods Market LGroceries ^ Here is XML for the same transaction. 2913d199bca3e6c43b85228899213a4b 2001-11-29 00:00:00 -0800 2002-04-07 16:08:20 -0700 277267000 Whole Foods Market 328f86287462647246e7ce838024d992 n 3690/100 3690/100 ddc3123d65cd99a12efe4820e16c0c71 981232fbcaf55580985ce6e20f6734b6 n -3690/100 -3690/100 588f87861f4538b62734a3b7301bacef Human readable? Riiiight. Can they have done better? Probably. But they didn't. When I saw the above, Frank Zappa's Valley Girl lyrics immedieately came to mind: It's like GRODY... GRODY TO THE MAX I'm sure It's like really nauseating Like BARF OUT GAG ME WITH A SPOON GROSS I am SURE TOTALLY... (alas, it loses its impact in print -- you have to listen to the song) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 15:35:22 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C2C5637B401 for ; Wed, 12 Feb 2003 15:35:21 -0800 (PST) Received: from ns1.xcllnt.net (209-128-86-226.BAYAREA.NET [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 62CA843FB1 for ; Wed, 12 Feb 2003 15:34:49 -0800 (PST) (envelope-from marcel@xcllnt.net) Received: from athlon.pn.xcllnt.net (athlon.pn.xcllnt.net [192.168.4.3]) by ns1.xcllnt.net (8.12.6/8.12.6) with ESMTP id h1CNYm1o024326; Wed, 12 Feb 2003 15:34:49 -0800 (PST) (envelope-from marcel@piii.pn.xcllnt.net) Received: from athlon.pn.xcllnt.net (localhost [127.0.0.1]) by athlon.pn.xcllnt.net (8.12.6/8.12.6) with ESMTP id h1CNYmo5017937; Wed, 12 Feb 2003 15:34:48 -0800 (PST) (envelope-from marcel@athlon.pn.xcllnt.net) Received: (from marcel@localhost) by athlon.pn.xcllnt.net (8.12.6/8.12.6/Submit) id h1CNYmLF017936; Wed, 12 Feb 2003 15:34:48 -0800 (PST) (envelope-from marcel) Date: Wed, 12 Feb 2003 15:34:48 -0800 From: Marcel Moolenaar To: Wes Peters Cc: Garrett Wollman , arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) Message-ID: <20030212233448.GA12939@athlon.pn.xcllnt.net> References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302120632.36583.wes@softweyr.com> <200302121411.h1CEBRSe025071@khavrinen.lcs.mit.edu> <200302121521.33506.wes@softweyr.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200302121521.33506.wes@softweyr.com> User-Agent: Mutt/1.5.3i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Feb 12, 2003 at 03:21:33PM +0000, Wes Peters wrote: > > > > I don't think either of these is truly a useful answer, particularly > > for something like syslog which is already straining for a more > > expressive syntax. There are just too many little languages in the > > system already, each one subtly different in syntax, such that it's > > nearly impossible to do anything programmatic with them that isn't > > built in to the program that they configure. > > So you're preferring the software over the human operator. In my world > view, that is completely whacked. Not in mine. Human operators don't share the same preferences and/or opinions. One cannot ever create something that is liked by all human operators. The inherent contradiction makes it a very difficult (dare I say impossible) group to please. I therefore prefer something that people can use as a building block, which automaticly gives preference to software and/or automation... -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 16:18:52 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 26E2637B401; Wed, 12 Feb 2003 16:18:51 -0800 (PST) Received: from melusine.cuivre.fr.eu.org (melusine.cuivre.fr.eu.org [62.212.105.185]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5B75043F85; Wed, 12 Feb 2003 16:18:50 -0800 (PST) (envelope-from thomas@FreeBSD.ORG) Received: by melusine.cuivre.fr.eu.org (Postfix, from userid 1000) id A33C82C3D1; Thu, 13 Feb 2003 01:18:45 +0100 (CET) Date: Thu, 13 Feb 2003 01:18:45 +0100 From: Thomas Quinot To: Wesley Peters Cc: Thomas Quinot , arch@freebsd.org Subject: Re: syslog.conf syntax change (multiple program/host specifications) Message-ID: <20030213001845.GA85036@melusine.cuivre.fr.eu.org> Reply-To: Thomas Quinot References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302111430.03156.wes@softweyr.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <200302111430.03156.wes@softweyr.com> User-Agent: Mutt/1.4i X-message-flag: WARNING! Using Outlook can damage your computer. Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Le 2003-02-11, Wesley Peters crivait : > mail.info /var/log/maillog G 4M > lpr.info /var/log/lpd-errs 256K > local3.* /var/log/foo B 1K > ftp.* /var/log/ftp B 1M I think this can be achieved without modifying syslogd. You can achieve the same effect by using an appropriate program as the log destination (think Apache's rotatelogs). Eg: local3.* |/usr/local/sbin/loglimit /var/log/foo B 1K Where loglimit would copy stdin to the named file, rotating it when appropriate. Thomas. -- Thomas.Quinot@Cuivre.FR.EU.ORG To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 18:16:31 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A75D937B405 for ; Wed, 12 Feb 2003 18:16:30 -0800 (PST) Received: from puffin.mail.pas.earthlink.net (puffin.mail.pas.earthlink.net [207.217.120.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id C6A9643FD7 for ; Wed, 12 Feb 2003 18:16:29 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0418.cvx22-bradley.dialup.earthlink.net ([209.179.199.163] helo=mindspring.com) by puffin.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18j8vK-0001ZA-00; Wed, 12 Feb 2003 18:16:26 -0800 Message-ID: <3E4AFFA3.99E9F2A6@mindspring.com> Date: Wed, 12 Feb 2003 18:14:59 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Wes Peters Cc: Garrett Wollman , arch@FreeBSD.org Subject: Re: syslog.conf syntax change (multiple program/host specifications) References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302121521.33506.wes@softweyr.com> <3E4A869B.EABBC83D@mindspring.com> <200302121825.57798.wes@softweyr.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a49aed1a85b83e6ed17549d2c0b4055102350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Wes Peters wrote: > > Finally, it's useful to replace syslog with a program that does > > the same thing *and* remember program state. Just because some > > people don't follow the machine readable logging draft RFC, doesn't > > mean that it can't be useful to other people (e.g. for external > > SNMP instrumentation). > > So you're going to contribute a nice daemon any day now, right? I'll contribute a machine readable logging RFC and patches to one or two programs, if you want... 8-). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 18:23:11 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2BFD137B401; Wed, 12 Feb 2003 18:23:10 -0800 (PST) Received: from www.fastmail.fm (www.fastmail.fm [66.111.4.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7984C43FCB; Wed, 12 Feb 2003 18:23:09 -0800 (PST) (envelope-from freebsd@fastmail.fm) Received: from www.fastmail.fm (localhost [127.0.0.1]) by localhost.localdomain (Postfix) with ESMTP id 6597B2259C; Wed, 12 Feb 2003 21:23:05 -0500 (EST) Received: from 127.0.0.1 ([127.0.0.1] helo=www.fastmail.fm) by fastmail.fm with SMTP; Wed, 12 Feb 2003 21:23:05 -0500 Received: by www.fastmail.fm (Postfix, from userid 99) id 610D02496D; Wed, 12 Feb 2003 21:23:05 -0500 (EST) Content-Disposition: inline Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="ISO-8859-1" MIME-Version: 1.0 X-Mailer: MIME::Lite 1.2 (F2.71; T1.001; A1.51; B2.12; Q2.03) From: "Glenn Gombert" To: "Hidetoshi Shimokawa" , freebsd-arch@freebsd.org Date: Wed, 12 Feb 2003 21:23:05 -0500 X-Epoch: 1045102985 X-Sasl-enc: D4KAqWyBoboa+Gk7Q5D6Wg Cc: julian@elischer.org, ikob@freebsd.org Subject: Re: You may want to have FireWire.... References: In-Reply-To: Message-Id: <20030213022305.610D02496D@www.fastmail.fm> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hello Hidetoshi, Yes this looks quite interesting! I will try and download and build your code this weekend. I have two boxes with the Firewire interfaces in them that you helped me get running sometime ago. But the gdb debugging was not complete at that time so I didn't stopped experimenting with it. I will keep you posted on my progress.. Thanks again, Glenn G On Wed, 12 Feb 2003 17:24:54 +0900, "Hidetoshi Shimokawa" said: > Do you think this useful? > > /\ Hidetoshi Shimokawa > \/ simokawa@sat.t.u-tokyo.ac.jp -- Glenn Gombert freebsd@fastmail.fm "Never trust any operating system you don't have the source code for" -- http://fastmail.fm - A fast, anti-spam email service. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 18:40:51 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9ECF337B401 for ; Wed, 12 Feb 2003 18:40:50 -0800 (PST) Received: from puffin.mail.pas.earthlink.net (puffin.mail.pas.earthlink.net [207.217.120.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2104E43FAF for ; Wed, 12 Feb 2003 18:40:50 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0418.cvx22-bradley.dialup.earthlink.net ([209.179.199.163] helo=mindspring.com) by puffin.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18j9Ip-0004Xi-00; Wed, 12 Feb 2003 18:40:44 -0800 Message-ID: <3E4B055A.641E4B6A@mindspring.com> Date: Wed, 12 Feb 2003 18:39:22 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Bakul Shah Cc: Garrett Wollman , Wes Peters , arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) References: <200302122038.PAA00115@thunderer.cnchost.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4eba57c6774183d52bce7f42df6f7ad58548b785378294e88350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Bakul Shah wrote: > In my view XML is to data representation what Roman numerals > are to math. *This* belongs in the fortunes database. > Rather than repeat all the arguments I'll point you to what > knowledgeable people like Erik Naggum have to say about XML > (search comp.lang.lisp on groups.google.com). Naggum used to > be a strong proponent of SGML until he "saw the light"! A > couple of recent thread on XML on comp.lang.lisp are worth > browsing. Erik is hardly unbiased. Back "in the old days" of the early 1990's, when we were first discussing the formulation of the SGML standard on Usenet, he lost some bitter battles with the main SGML guys about representation in DTD's and a number of other points. He may or may not be the "forgive and forget" type of guy. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 19:40:23 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4F35037B401 for ; Wed, 12 Feb 2003 19:40:21 -0800 (PST) Received: from smtp1.rpi.edu (smtp1.server.rpi.edu [128.113.2.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 54E3243F93 for ; Wed, 12 Feb 2003 19:40:20 -0800 (PST) (envelope-from drosih@rpi.edu) Received: from [128.113.24.47] (gilead.netel.rpi.edu [128.113.24.47]) by smtp1.rpi.edu (8.12.7/8.12.7) with ESMTP id h1D3eI7J021823; Wed, 12 Feb 2003 22:40:19 -0500 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: <200302121615.h1CGFdGG025691@khavrinen.lcs.mit.edu> References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302120632.36583.wes@softweyr.com> <200302121411.h1CEBRSe025071@khavrinen.lcs.mit.edu> <200302121521.33506.wes@softweyr.com> <200302121615.h1CGFdGG025691@khavrinen.lcs.mit.edu> Date: Wed, 12 Feb 2003 22:40:17 -0500 To: Garrett Wollman , Wes Peters From: Garance A Drosihn Subject: Re: syslog.conf syntax change (multiple program/host specifications) Cc: arch@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-RPI-Spam-Score: -0.8 () IN_REP_TO,REFERENCES,SIGNATURE_SHORT_DENSE,SPAM_PHRASE_00_01 X-Scanned-By: MIMEDefang 2.28 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG At 11:15 AM -0500 2/12/03, Garrett Wollman wrote: >< said: > >> So you're preferring the software over the human operator. > >I would not necessarily jump to that conclusion. What I want to >eschew is a proliferation of lots of little languages, each one >subtly different, such that users are forced to learn all of them. >We already have: > > foocap Besides my favorite /etc/printcap file, there's also /etc/hosts.lpd and /etc/hosts.equiv which are referenced by lpd. Those two are of particular interest to me right now, as I want to add some options which would be set based on what host is sending the job to lpd. I either have to add a third file, or I have to add some options to entries in /etc/hosts.lpd. > fstab > [...etc...] > >Every single one of these has a different syntax that the admin >must learn in addition to the relevant semantics, and which any >sort of front-end or configuration-analysis tool must be able >to interpret, in order to do anything useful with the programs >they control. The benefit of something like XML (or Lisp, for >that matter) is that, while still providing for functionally >significant differences, the *lexical* structure is identical >across many functions -- and this makes it much easier to use >other tools (like structured editors) to maintain and document >the files. I bounce back and forth on XML. I can see that it's useful for some things, but I don't think it is appropriate for config files that a user is going to type in. And I certainly don't want to *require* XML config files, because some of our (RPI) config files are automatically generated and we assume the same format of file across a number of unix platforms. I think that trying to XML-ize config files is something that will take some time and effort to do right, and I suspect we would be better off if that was put off for 6.0. I don't think we want to drive our users nuts by constantly changing the format of these files during a stable-branch, and I doubt we'll have the perfect file format in time for 5.1-release. Wes's original proposal is a simple extension to the current syntax, and I think we could decide on what we think about that as a near-term change, without tieing it to the discussion of what to do about XML-izing all config files. As to Wes's patch, I like the new feature but I don't think the config file should be quite so cryptic. I guess I would like to think about it some more. -- Garance Alistair Drosehn = gad@gilead.netel.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 20:17:13 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1715B37B401 for ; Wed, 12 Feb 2003 20:17:12 -0800 (PST) Received: from out001.verizon.net (out001pub.verizon.net [206.46.170.140]) by mx1.FreeBSD.org (Postfix) with ESMTP id BA1A943FB1 for ; Wed, 12 Feb 2003 20:17:10 -0800 (PST) (envelope-from mtm@identd.net) Received: from kokeb.ambesa.net ([151.200.10.217]) by out001.verizon.net (InterMail vM.5.01.05.20 201-253-122-126-120-20021101) with ESMTP id <20030213041709.XOQ23484.out001.verizon.net@kokeb.ambesa.net>; Wed, 12 Feb 2003 22:17:09 -0600 Date: Wed, 12 Feb 2003 23:17:08 -0500 From: Mike Makonnen To: Garance A Drosihn Cc: wollman@lcs.mit.edu, wes@softweyr.com, arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) In-Reply-To: References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302120632.36583.wes@softweyr.com> <200302121411.h1CEBRSe025071@khavrinen.lcs.mit.edu> <200302121521.33506.wes@softweyr.com> <200302121615.h1CGFdGG025691@khavrinen.lcs.mit.edu> X-Mailer: Sylpheed version 0.8.10 (GTK+ 1.2.10; i386-portbld-freebsd5.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Authentication-Info: Submitted using SMTP AUTH at out001.verizon.net from [151.200.10.217] at Wed, 12 Feb 2003 22:17:08 -0600 Message-Id: <20030213041709.XOQ23484.out001.verizon.net@kokeb.ambesa.net> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 12 Feb 2003 22:40:17 -0500 Garance A Drosihn wrote: > I bounce back and forth on XML. I can see that it's useful for > some things, but I don't think it is appropriate for config > files that a user is going to type in. And I certainly don't > want to *require* XML config files, because some of our (RPI) > config files are automatically generated and we assume the > same format of file across a number of unix platforms. Correct me if I'm wrong, but XML was never meant for humans to parse. "Human-readable" doesn't mean the user is supposed to go in and edit it. It means that if you should ever hit a worst case scenario, a human can always go in with a text editor and manipulate the data directly, not necessarily that it would be easy to do so. The primary goal of XML is to allow non-intelligent machines/software to understand the structure of a piece of data. > > I think that trying to XML-ize config files is something that > will take some time and effort to do right, and I suspect we > would be better off if that was put off for 6.0. I don't think > we want to drive our users nuts by constantly changing the > format of these files during a stable-branch, and I doubt we'll > have the perfect file format in time for 5.1-release. There's no way in hell we or any other BSD is going exclusively to XML for any particular configuration file. Some Linux distro might try it, but it would never work. The current file layouts are too entrenched. What would work; however, is XML as an intermediate data layout. The XML would sit between some program the user can use to make choices and the underlying text file. Take rc.conf for example. You could put all that data in XML format and some gui would let the user choose whatever he/she wanted and that would then be piped through an XSLT script that renders it in the current key=value layout. The data would be kept in defaults/rc.conf.xml and defaults/rc.conf would just become another generated file. Cheers. -- Mike Makonnen | GPG-KEY: http://www.identd.net/~mtm/mtm.asc mtm@identd.net | Fingerprint: D228 1A6F C64E 120A A1C9 A3AA DAE1 E2AF DBCC 68B9 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 20:46:57 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A11B437B401; Wed, 12 Feb 2003 20:46:56 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1415D43F93; Wed, 12 Feb 2003 20:46:52 -0800 (PST) (envelope-from wes@softweyr.com) Received: from softweyr.homeunix.net (66-75-151-22.san.rr.com [66.75.151.22]) by smtp-relay.omnis.com (Postfix) with ESMTP id 854E242F44; Wed, 12 Feb 2003 20:46:52 -0800 (PST) From: Wes Peters Organization: Softweyr To: Thomas Quinot Subject: Re: syslog.conf syntax change (multiple program/host specifications) Date: Thu, 13 Feb 2003 04:46:50 +0000 User-Agent: KMail/1.5 Cc: Thomas Quinot , arch@freebsd.org References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302111430.03156.wes@softweyr.com> <20030213001845.GA85036@melusine.cuivre.fr.eu.org> In-Reply-To: <20030213001845.GA85036@melusine.cuivre.fr.eu.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Message-Id: <200302130446.50943.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thursday 13 February 2003 00:18, Thomas Quinot wrote: > Le 2003-02-11, Wesley Peters =E9crivait : > > mail.info /var/log/maillog G 4M > > lpr.info /var/log/lpd-errs 256K > > local3.* /var/log/foo B 1K > > ftp.* /var/log/ftp B 1M > > I think this can be achieved without modifying syslogd. You can achieve > the same effect by using an appropriate program as the log destination > (think Apache's rotatelogs). > > Eg: > local3.* |/usr/local/sbin/loglimit /var/log/foo B 1K > > Where loglimit would copy stdin to the named file, rotating it when > appropriate. Now that's an innovative solution to the problem. It has the over- head of an additional process for every log file you want to limit, but it is certainly worthy of consideration. Thank you. =2D-=20 Where am I, and what am I doing in this handbasket? Wes Peters wes@softweyr.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 20:48: 7 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 61C8237B401 for ; Wed, 12 Feb 2003 20:48:06 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5EFFA43FA3 for ; Wed, 12 Feb 2003 20:48:05 -0800 (PST) (envelope-from wes@softweyr.com) Received: from softweyr.homeunix.net (66-75-151-22.san.rr.com [66.75.151.22]) by smtp-relay.omnis.com (Postfix) with ESMTP id F2171431E0; Wed, 12 Feb 2003 20:48:05 -0800 (PST) From: Wes Peters Organization: Softweyr To: Terry Lambert Subject: Re: syslog.conf syntax change (multiple program/host specifications) Date: Thu, 13 Feb 2003 04:48:04 +0000 User-Agent: KMail/1.5 Cc: Garrett Wollman , arch@FreeBSD.org References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302121825.57798.wes@softweyr.com> <3E4AFFA3.99E9F2A6@mindspring.com> In-Reply-To: <3E4AFFA3.99E9F2A6@mindspring.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200302130448.04457.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thursday 13 February 2003 02:14, Terry Lambert wrote: > Wes Peters wrote: > > > Finally, it's useful to replace syslog with a program that does > > > the same thing *and* remember program state. Just because some > > > people don't follow the machine readable logging draft RFC, doesn't > > > mean that it can't be useful to other people (e.g. for external > > > SNMP instrumentation). > > > > So you're going to contribute a nice daemon any day now, right? > > I'll contribute a machine readable logging RFC and patches to one > or two programs, if you want... 8-). That'd be a good start. An accepted RFC means there has to be two implementations, right? Oh, wait, we threw that baby out and kept the bathwater years ago. -- Where am I, and what am I doing in this handbasket? Wes Peters wes@softweyr.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 20:49:48 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 82CF937B401 for ; Wed, 12 Feb 2003 20:49:47 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id E74AE43FB1 for ; Wed, 12 Feb 2003 20:49:46 -0800 (PST) (envelope-from wes@softweyr.com) Received: from softweyr.homeunix.net (66-75-151-22.san.rr.com [66.75.151.22]) by smtp-relay.omnis.com (Postfix) with ESMTP id 28CB443059; Wed, 12 Feb 2003 20:49:47 -0800 (PST) From: Wes Peters Organization: Softweyr To: Terry Lambert , Bakul Shah Subject: Re: syslog.conf syntax change (multiple program/host specifications) Date: Thu, 13 Feb 2003 04:49:45 +0000 User-Agent: KMail/1.5 Cc: Garrett Wollman , arch@FreeBSD.ORG References: <200302122038.PAA00115@thunderer.cnchost.com> <3E4B055A.641E4B6A@mindspring.com> In-Reply-To: <3E4B055A.641E4B6A@mindspring.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200302130449.45547.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thursday 13 February 2003 02:39, Terry Lambert wrote: > Bakul Shah wrote: > > In my view XML is to data representation what Roman numerals > > are to math. > > *This* belongs in the fortunes database. > > > Rather than repeat all the arguments I'll point you to what > > knowledgeable people like Erik Naggum have to say about XML > > (search comp.lang.lisp on groups.google.com). Naggum used to > > be a strong proponent of SGML until he "saw the light"! A > > couple of recent thread on XML on comp.lang.lisp are worth > > browsing. > > Erik is hardly unbiased. Back "in the old days" of the early > 1990's, when we were first discussing the formulation of the > SGML standard on Usenet, he lost some bitter battles with the > main SGML guys about representation in DTD's and a number of > other points. He may or may not be the "forgive and forget" > type of guy. Uh, formulation of the SGML standard in the early 90s? I bought my first copy of the ISO SGML standard before I left Logicon in 1987. I might still have it around here somewhere... -- Where am I, and what am I doing in this handbasket? Wes Peters wes@softweyr.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 21: 4:29 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A19AC37B401 for ; Wed, 12 Feb 2003 21:04:27 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2F64343F3F for ; Wed, 12 Feb 2003 21:04:27 -0800 (PST) (envelope-from wes@softweyr.com) Received: from softweyr.homeunix.net (66-75-151-22.san.rr.com [66.75.151.22]) by smtp-relay.omnis.com (Postfix) with ESMTP id 02D214340F; Wed, 12 Feb 2003 20:59:39 -0800 (PST) From: Wes Peters Organization: Softweyr To: Garance A Drosihn , Garrett Wollman Subject: Re: syslog.conf syntax change (multiple program/host specifications) Date: Thu, 13 Feb 2003 04:59:38 +0000 User-Agent: KMail/1.5 Cc: arch@FreeBSD.ORG References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302121615.h1CGFdGG025691@khavrinen.lcs.mit.edu> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200302130459.38238.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thursday 13 February 2003 03:40, Garance A Drosihn wrote: > > I bounce back and forth on XML. I can see that it's useful for > some things, but I don't think it is appropriate for config > files that a user is going to type in. My point exactly. Garrett's point about all those programs having differing syntax in their configuration files completely ignores the fact that all those programs do different things; the differing syntax is a feature of the differences of the program. > And I certainly don't > want to *require* XML config files, because some of our (RPI) > config files are automatically generated and we assume the > same format of file across a number of unix platforms. Oh, you can just XML-ize all those systems too. XML is the future, you know. Actually, XML would be perfect for a metalanguage for communicating configuration information between disparate systems because you could then use the generally good XML tools to generate the local systems flavor of configuration file from the XML. Just don't make poor human beings read the crap. > Wes's original proposal is a simple extension to the current > syntax, and I think we could decide on what we think about that > as a near-term change, without tieing it to the discussion of > what to do about XML-izing all config files. > > As to Wes's patch, I like the new feature but I don't think the > config file should be quite so cryptic. I guess I would like > to think about it some more. Constructive criticism. The feature as implemented now was done as a minimal change to effect the outcome we wanted, which is to keep syslog from filling our /var ramdisk and causing problems. I'm very open to suggestions of a more intuitive syntax for such a simple modification, but I'm not trying to duplicate the complete functionality of newsyslog. I *would* be amenable to having syslogd start a newsyslog whenever a file size limit is reached; that might be quite useful. Since newsyslog can be passed the name of the file to act upon on its command line, this would be a useful extension. I'll hack that in at work tomorrow and as soon as I get my -CURRENT box back up to date I'll produce a patch. The file size limiting might still be useful for memory-limited applications; as you can see from the patch the changes are quite small. -- Where am I, and what am I doing in this handbasket? Wes Peters wes@softweyr.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 22:35:24 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AF65237B408; Wed, 12 Feb 2003 22:35:02 -0800 (PST) Received: from globalpromoter.com (97.35.35.65.cfl.rr.com [65.35.35.97]) by mx1.FreeBSD.org (Postfix) with SMTP id A97D9440DF; Wed, 12 Feb 2003 22:33:58 -0800 (PST) (envelope-from levi@globalpromoter.com) Message-ID: <001711e2cb06$bbc57055$52272768@ssujvro.vuy> From: To: Webmaster@FreeBSD.ORG Subject: GlobalPromoter.com 1st Page listings on Google Date: Thu, 13 Feb 2003 02:11:00 +0500 MIME-Version: 1.0 Content-Type: text/html; charset="iso-8859-1" X-Priority: 3 X-Mailer: QUALCOMM Windows Eudora Version 5.1 Importance: Normal Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG globalpromoter

Hi,

My name is Levi Lewis and I'm with GlobalPromoter.com,
a search engine optimization firm specializing in guaranteed
first page listings in the major search engines. I would like
to speak with someone from your company regarding possible
partnership opportunities.

GlobalPromoter is an industry leader in the website optimization
and search engine submission business and has partnerships with
search engines including Overture, Inktomi, LookSmart and Findwhat
just to name a few.

We guarantee first page listings in the major search engines on most
of our packages and guarantee those listings within 14 days of
placing your order. If you would like to view some sample before
and after ranking reports of our clients please contact us.

We also offer our partners generous commissions even if they never
use our service.

In addition to our paid services we also offer several FREE and Unique
online tools (link at the bottom of page) that your users would find quite powerful. If you have any
"do-it-yourself marketers" I think they'll be quite impressed with the
analysis and recommendation tools we offer. Here are just a few of
the free tools.

Keyword Suggestion Tool:
The user enters a keyword and receives every variance of that keyword
and the number of times it's been searched on so far this month on the
Overture network of search engines.

Site Readiness Tool:
The user enters a url and keyword phrase and gets a detailed report
with recommendations to improve their ranking on the major search
engines.

Spider Your Competition Tool:
User can enter a url and extract the contents of your competitions
pages.

Plus there are many more. And we would be more than happy to put
the free tools on your site at no charge if we come to an agreement.

Please contact us with any questions and or comments you might
have. We look forward to hearing from you.

Best Regards,
Levi Lewis
levi@globalpromoter.com

http://www.GlobalPromoter.com
"Guaranteed first page placement among the major search engines"
(321)459-5591

Our free unique online tools :     http://globalpromoter.com/free_search_engine_promotion_tools.cfm

Signup for a free account:     http://globalpromoter.com/directory/signup_form.cfm



 

To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 22:51:53 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BC4C837B401 for ; Wed, 12 Feb 2003 22:51:52 -0800 (PST) Received: from puffin.mail.pas.earthlink.net (puffin.mail.pas.earthlink.net [207.217.120.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3AEC543FAF for ; Wed, 12 Feb 2003 22:51:52 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0060.cvx21-bradley.dialup.earthlink.net ([209.179.192.60] helo=mindspring.com) by puffin.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18jDDT-0007f1-00; Wed, 12 Feb 2003 22:51:28 -0800 Message-ID: <3E4B401D.B470841A@mindspring.com> Date: Wed, 12 Feb 2003 22:50:05 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Wes Peters Cc: Bakul Shah , Garrett Wollman , arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) References: <200302122038.PAA00115@thunderer.cnchost.com> <3E4B055A.641E4B6A@mindspring.com> <200302130449.45547.wes@softweyr.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4b33fe5c147bdbdc3dab883d4e4712d1b667c3043c0873f7e350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Wes Peters wrote: > > Erik is hardly unbiased. Back "in the old days" of the early > > 1990's, when we were first discussing the formulation of the > > SGML standard on Usenet, he lost some bitter battles with the > > main SGML guys about representation in DTD's and a number of > > other points. He may or may not be the "forgive and forget" > > type of guy. > > Uh, formulation of the SGML standard in the early 90s? I bought > my first copy of the ISO SGML standard before I left Logicon in > 1987. I might still have it around here somewhere... Early 90's, late 80's... after 20 years, it's "in the old days". 8-) 8-). I was working at Century at the time, and participated in the discussion as "terry@cs.werber.edu". -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Feb 12 23:24:52 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 500C137B401 for ; Wed, 12 Feb 2003 23:24:51 -0800 (PST) Received: from heron.mail.pas.earthlink.net (heron.mail.pas.earthlink.net [207.217.120.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id BEB1F43FB1 for ; Wed, 12 Feb 2003 23:24:50 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0060.cvx21-bradley.dialup.earthlink.net ([209.179.192.60] helo=mindspring.com) by heron.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18jDjj-0005nr-00; Wed, 12 Feb 2003 23:24:48 -0800 Message-ID: <3E4B47ED.895FB3CA@mindspring.com> Date: Wed, 12 Feb 2003 23:23:25 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Wes Peters Cc: Garrett Wollman , arch@FreeBSD.org Subject: Re: syslog.conf syntax change (multiple program/host specifications) References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302121825.57798.wes@softweyr.com> <3E4AFFA3.99E9F2A6@mindspring.com> <200302130448.04457.wes@softweyr.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a443fc89003e1ae8eb0f6cb8e11f2ebd20387f7b89c61deb1d350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Wes Peters wrote: > > > So you're going to contribute a nice daemon any day now, right? > > > > I'll contribute a machine readable logging RFC and patches to one > > or two programs, if you want... 8-). > > That'd be a good start. An accepted RFC means there has to be two > implementations, right? Oh, wait, we threw that baby out and kept > the bathwater years ago. Hold on, cowboy. I have an update of a draft RFC, not an already passed standards track document. If you want, I can bang it into shape. The original stuff was by someone else, who wanted to change the face of logging. That's just not going to happen, and they were kind of mistaken to try and get away from syslog, and they didn't provide real state information (both of which I don't do). Needless to say, the first one by the other author didn't pass the IETF, so it's a hard row to hoe. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 13 1:44:31 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 07B2337B401 for ; Thu, 13 Feb 2003 01:44:31 -0800 (PST) Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6861843F75 for ; Thu, 13 Feb 2003 01:44:30 -0800 (PST) (envelope-from des@ofug.org) Received: by flood.ping.uio.no (Postfix, from userid 2602) id 047E6536E; Thu, 13 Feb 2003 10:44:27 +0100 (CET) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Bakul Shah Cc: Garrett Wollman , Wes Peters , arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) From: Dag-Erling Smorgrav Date: Thu, 13 Feb 2003 10:44:27 +0100 In-Reply-To: <200302122320.SAA25721@rodney.cnchost.com> (Bakul Shah's message of "Wed, 12 Feb 2003 15:20:37 -0800") Message-ID: User-Agent: Gnus/5.090014 (Oort Gnus v0.14) Emacs/21.2 (i386--freebsd) References: <200302122320.SAA25721@rodney.cnchost.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Bakul Shah writes: > Here is XML for the same transaction. Don't blame XML for developers' failure to use it properly. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 13 3:23: 4 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 54B6D37B401 for ; Thu, 13 Feb 2003 03:23:03 -0800 (PST) Received: from mailout03.sul.t-online.com (mailout03.sul.t-online.com [194.25.134.81]) by mx1.FreeBSD.org (Postfix) with ESMTP id C395043FA3 for ; Thu, 13 Feb 2003 03:22:59 -0800 (PST) (envelope-from Alexander@Leidinger.net) Received: from fwd06.sul.t-online.de by mailout03.sul.t-online.com with smtp id 18jHRl-00011V-07; Thu, 13 Feb 2003 12:22:29 +0100 Received: from Andro-Beta.Leidinger.net (520065502893-0001@[217.229.216.98]) by fmrl06.sul.t-online.com with esmtp id 18jHRb-1Y7gy8C; Thu, 13 Feb 2003 12:22:19 +0100 Received: from Magelan.Leidinger.net (Magelan [192.168.1.1]) by Andro-Beta.Leidinger.net (8.12.6/8.12.6) with ESMTP id h1DBMI8Z016474; Thu, 13 Feb 2003 12:22:18 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from Magelan.Leidinger.net (netchild@localhost [127.0.0.1]) by Magelan.Leidinger.net (8.12.6/8.12.6) with SMTP id h1DBMIQE000814; Thu, 13 Feb 2003 12:22:18 +0100 (CET) (envelope-from Alexander@Leidinger.net) Date: Thu, 13 Feb 2003 12:22:18 +0100 From: Alexander Leidinger To: Mike Makonnen Cc: drosih@rpi.edu, wollman@lcs.mit.edu, wes@softweyr.com, arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) Message-Id: <20030213122218.3d72c6ba.Alexander@Leidinger.net> In-Reply-To: <20030213041709.XOQ23484.out001.verizon.net@kokeb.ambesa.net> References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302120632.36583.wes@softweyr.com> <200302121411.h1CEBRSe025071@khavrinen.lcs.mit.edu> <200302121521.33506.wes@softweyr.com> <200302121615.h1CGFdGG025691@khavrinen.lcs.mit.edu> <20030213041709.XOQ23484.out001.verizon.net@kokeb.ambesa.net> X-Mailer: Sylpheed version 0.8.9claws (GTK+ 1.2.10; i386-portbld-freebsd5.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Sender: 520065502893-0001@t-dialin.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 12 Feb 2003 23:17:08 -0500 Mike Makonnen wrote: > never work. The current file layouts are too entrenched. What would work; > however, is XML as an intermediate data layout. The XML would sit between some > program the user can use to make choices and the underlying text file. Take > rc.conf for example. You could put all that data in XML format and some gui > would let the user choose whatever he/she wanted and that would then be piped > through an XSLT script that renders it in the current key=value layout. The data > would be kept in defaults/rc.conf.xml and defaults/rc.conf would just become > another generated file. How do you want to solve the problem of: - T1: a human edited /etc/rc.conf without updating the .xml - T2: someone else (or the same human) uses the GUI to change something --> the hand edited change in rc.conf is lost If you have a parser for rc.conf -> .xml (note: I also have some special r.conf's with "if, else, ...") then you can directly use this parser in the GUI instead of translating it into xml. OT-Question: should we "export" existing parsers into share/ so interested people just have to look into share/parsers/ instead of searching it in /usr/src? Bye, Alexander. -- Secret hacker rule #11: hackers read manuals. http://www.Leidinger.net Alexander @ Leidinger.net GPG fingerprint = C518 BC70 E67F 143F BE91 3365 79E2 9C60 B006 3FE7 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 13 3:30: 4 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A2EA137B401 for ; Thu, 13 Feb 2003 03:30:02 -0800 (PST) Received: from melusine.cuivre.fr.eu.org (melusine.cuivre.fr.eu.org [62.212.105.185]) by mx1.FreeBSD.org (Postfix) with ESMTP id CAA8343F3F for ; Thu, 13 Feb 2003 03:30:01 -0800 (PST) (envelope-from thomas@freebsd.org) Received: by melusine.cuivre.fr.eu.org (Postfix, from userid 1000) id 889352C3D1; Thu, 13 Feb 2003 12:30:00 +0100 (CET) Date: Thu, 13 Feb 2003 12:30:00 +0100 From: Thomas Quinot To: Garrett Wollman Cc: arch@FreeBSD.org Subject: Re: syslog.conf syntax change (multiple program/host specifications) Message-ID: <20030213113000.GD55974@melusine.cuivre.fr.eu.org> Reply-To: Thomas Quinot References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302112310.h1BNAUBS019097@khavrinen.lcs.mit.edu> <20030212182852.GA94317@melusine.cuivre.fr.eu.org> <200302122018.h1CKIxjT026982@khavrinen.lcs.mit.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <200302122018.h1CKIxjT026982@khavrinen.lcs.mit.edu> User-Agent: Mutt/1.4i X-message-flag: WARNING! Using Outlook can damage your computer. Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Le 2003-02-12, Garrett Wollman crivait : > The benefit of such changes is that, when new features are necessary > (e.g., MAC labels for servers started by inetd), it is not necessary > to create yet another kluge to shoehorn the new feature into the old, > inflexible syntax. Indeed, there is a balance to be found between integration of new features and compatibility with legacy syntax. The current state of syslog.conf IMO demonstrates that sometimes this balance can be found by evolution of the existing format, and not necessarily by a complete change of syntax. > AIX actually had sort-of-the-right-idea, in that most of the > AIX-specific configuration files are lexically identical, vary very > little in syntax, and are easily extensible. Unfortunately, there is I am more than willing to buy this argument for vendor-specific configuration files, but my point concerns files that have had similar layouts on many variants of *nix for years. > not putting forward XML because I like it; I am however noting that > none of the comparably-capable alternatives have anything like the > broad support that XML does. Sure, and as several suggested XML might be an interesting option as an intermediate representation of configuration information in the context of automated generation of config files. Thomas. -- Thomas.Quinot@Cuivre.FR.EU.ORG To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 13 6:47:53 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 375F637B401 for ; Thu, 13 Feb 2003 06:47:52 -0800 (PST) Received: from murdoch.servitor.co.uk (murdoch.servitor.co.uk [217.151.99.80]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9D05E43FA3 for ; Thu, 13 Feb 2003 06:47:50 -0800 (PST) (envelope-from paul@iconoplex.co.uk) Received: from mmu-firewall.mmu.ac.uk ([149.170.101.200] helo=miter96pq2w1fz) by murdoch.servitor.co.uk with smtp (Exim 3.33 #3) id 18jI71-000Fnz-00 for arch@FreeBSD.ORG; Thu, 13 Feb 2003 12:05:07 +0000 From: "Paul Robinson" To: Subject: RE: syslog.conf syntax change (multiple program/host specifications) Date: Thu, 13 Feb 2003 12:05:06 -0000 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) In-Reply-To: <20030213122218.3d72c6ba.Alexander@Leidinger.net> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Importance: Normal Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Alexander wrote: > How do you want to solve the problem of: > - T1: a human edited /etc/rc.conf without updating the .xml > - T2: someone else (or the same human) uses the GUI to change something > --> the hand edited change in rc.conf is lost Oh, that's easy. What you do to keep two views of the same information consistent, is keep all the data in an SQL database, and then just software that pulls the information out into the two different formats, and pushes the data back in on changes! So, what we do is put all the config information into mySQL and insist it ships as standard on every box! :-) On a more serious note, the only way this system would work completely is if there were some trigger mechanism after editing the file. I suppose something like the new aliases stuff under sendmail would have to be implemented. At that point, I think a lot of users would ask "what's the point?" XML is great for lots of things. It means my web server can interact with a customer database on an AS/400 with about half a day's work. It means that documents that should be in a standard non-proprietary format, like music scores for example, are indeed put into standard non-proprietary formats. For these things, and others, XML quite frankly, rocks. Unfortunately, it sucks for configuration files on Unix machines. It would be great for configs on Windows machines or Mac OS, because nobody ever manually plays with the files. I hate to play the pedant on this one, but as a last point, this might serve as a good reminder as to why XML is a bad idea in the framework of FreeBSD: http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/introdu ction-archguide.html The first, sixth and seventh points seem to apply here. I'm not going to start a whole new flamewar as to whether these guidelines should be upheld or not, but they do make sense and particularly relevant in my mind to this discussion. Just my ?0.02 worth. I'll go away and shut up now. -- Paul Robinson To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 13 7:59:42 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E3C2B37B401 for ; Thu, 13 Feb 2003 07:59:40 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6894943FAF for ; Thu, 13 Feb 2003 07:59:40 -0800 (PST) (envelope-from wes@softweyr.com) Received: from softweyr.homeunix.net (66-75-151-22.san.rr.com [66.75.151.22]) by smtp-relay.omnis.com (Postfix) with ESMTP id F0EBC43855; Thu, 13 Feb 2003 07:58:21 -0800 (PST) From: Wes Peters Organization: Softweyr To: "Paul Robinson" , Subject: Re: syslog.conf syntax change (multiple program/host specifications) Date: Thu, 13 Feb 2003 15:58:20 +0000 User-Agent: KMail/1.5 References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200302131558.20627.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thursday 13 February 2003 12:05, Paul Robinson wrote: > Alexander wrote: > > How do you want to solve the problem of: > > - T1: a human edited /etc/rc.conf without updating the .xml > > - T2: someone else (or the same human) uses the GUI to change > > something --> the hand edited change in rc.conf is lost > > Oh, that's easy. What you do to keep two views of the same information > consistent, is keep all the data in an SQL database, and then just > software that pulls the information out into the two different formats, > and pushes the data back in on changes! So, what we do is put all the > config information into mySQL and insist it ships as standard on every > box! :-) You'd be absolutely astonished at the number of embedded BSD systems that have come to exactly this conclusion, only they were all smart enough to choose PostgreSQL instead. -- Where am I, and what am I doing in this handbasket? Wes Peters wes@softweyr.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 13 8: 3: 6 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4284137B401 for ; Thu, 13 Feb 2003 08:03:05 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id C327043FB1 for ; Thu, 13 Feb 2003 08:03:04 -0800 (PST) (envelope-from wes@softweyr.com) Received: from softweyr.homeunix.net (66-75-151-22.san.rr.com [66.75.151.22]) by smtp-relay.omnis.com (Postfix) with ESMTP id E6FDB4340F; Thu, 13 Feb 2003 08:02:57 -0800 (PST) From: Wes Peters Organization: Softweyr To: Terry Lambert Subject: Re: syslog.conf syntax change (multiple program/host specifications) Date: Thu, 13 Feb 2003 16:02:56 +0000 User-Agent: KMail/1.5 Cc: arch@FreeBSD.ORG References: <200302122038.PAA00115@thunderer.cnchost.com> <200302130449.45547.wes@softweyr.com> <3E4B401D.B470841A@mindspring.com> In-Reply-To: <3E4B401D.B470841A@mindspring.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200302131602.56899.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thursday 13 February 2003 06:50, Terry Lambert wrote: > Wes Peters wrote: > > > > Uh, formulation of the SGML standard in the early 90s? I bought > > my first copy of the ISO SGML standard before I left Logicon in > > 1987. I might still have it around here somewhere... > > Early 90's, late 80's... after 20 years, it's "in the old days". The 80's weren't such a blur for me. Moving all over the country every year or two helps keep dates and times sorted out in your mind. > 8-) 8-). I was working at Century at the time, and participated > in the discussion as "terry@cs.werber.edu". An email address that shall live on in infamy. ;^) A Patchy BSD; too bad the web server guys swiped your project name before you hit upon it. -- Where am I, and what am I doing in this handbasket? Wes Peters wes@softweyr.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 13 8:17:16 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B6E2D37B401 for ; Thu, 13 Feb 2003 08:17:15 -0800 (PST) Received: from heron.mail.pas.earthlink.net (heron.mail.pas.earthlink.net [207.217.120.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3675143FBD for ; Thu, 13 Feb 2003 08:17:15 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0083.cvx21-bradley.dialup.earthlink.net ([209.179.192.83] helo=mindspring.com) by heron.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18jM2t-0005c3-00; Thu, 13 Feb 2003 08:17:08 -0800 Message-ID: <3E4BC495.E3A2FB66@mindspring.com> Date: Thu, 13 Feb 2003 08:15:17 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Paul Robinson Cc: arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4b6cd8c3bd68b133761b428bb5c8b4b24548b785378294e88350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Paul Robinson wrote: > On a more serious note, the only way this system would work completely is if > there were some trigger mechanism after editing the file. kqueue. But there are hard/soft service dependencies that arise out of file dependencies, and there's no notification for those, so you pretty much end up screwed. You *really* do not want to go around shooting "bind" in the head, like the "Life Magazine" cover shot of the Khymer Rouge; if you end up with a ton of dependencies, then you end up with a huge accumulated delay, any time there's a state change. It's no good. Cached Data Considered Harmful (and all that). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 13 8:22:29 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3301637B401 for ; Thu, 13 Feb 2003 08:22:28 -0800 (PST) Received: from heron.mail.pas.earthlink.net (heron.mail.pas.earthlink.net [207.217.120.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id A3B4C43FBF for ; Thu, 13 Feb 2003 08:22:27 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0083.cvx21-bradley.dialup.earthlink.net ([209.179.192.83] helo=mindspring.com) by heron.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18jM7w-0006Y2-00; Thu, 13 Feb 2003 08:22:21 -0800 Message-ID: <3E4BC5D2.3BD12EF2@mindspring.com> Date: Thu, 13 Feb 2003 08:20:34 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Wes Peters Cc: Paul Robinson , arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) References: <200302131558.20627.wes@softweyr.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4b6cd8c3bd68b13371091c3c4edb94d593ca473d225a0f487350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Wes Peters wrote: > On Thursday 13 February 2003 12:05, Paul Robinson wrote: > > Oh, that's easy. What you do to keep two views of the same information > > consistent, is keep all the data in an SQL database, and then just > > software that pulls the information out into the two different formats, > > and pushes the data back in on changes! So, what we do is put all the > > config information into mySQL and insist it ships as standard on every > > box! :-) > > You'd be absolutely astonished at the number of embedded BSD systems that > have come to exactly this conclusion, only they were all smart enough to > choose PostgreSQL instead. Or gdbm (with appropriate hazmat suits) or something they rolled themselves, etc.. It's a common problem that the OS vendors are unwilling to address, such that it never has to be addressed again. --- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 13 8:25:26 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7BA8537B401 for ; Thu, 13 Feb 2003 08:25:24 -0800 (PST) Received: from Daffy.timing.com (daffy.timing.com [206.168.13.218]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0551B43FDD for ; Thu, 13 Feb 2003 08:25:20 -0800 (PST) (envelope-from ben@timing.com) Received: from piglet.timing.com (piglet.timing.com [206.168.13.178]) by Daffy.timing.com (8.11.3/8.11.3) with ESMTP id h1DGPHH31643; Thu, 13 Feb 2003 09:25:17 -0700 (MST) (envelope-from ben@timing.com) Received: from piglet.timing.com (localhost.timing.com [127.0.0.1]) by piglet.timing.com (8.12.6/8.12.6) with ESMTP id h1DGPHXl059183; Thu, 13 Feb 2003 09:25:17 -0700 (MST) (envelope-from ben@piglet.timing.com) Received: (from ben@localhost) by piglet.timing.com (8.12.6/8.12.6/Submit) id h1DGPGgd059180; Thu, 13 Feb 2003 09:25:16 -0700 (MST) From: Ben Mesander MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15947.50924.514481.652584@piglet.timing.com> Date: Thu, 13 Feb 2003 09:25:16 -0700 To: Mike Makonnen Cc: Garance A Drosihn , wollman@lcs.mit.edu, wes@softweyr.com, arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) In-Reply-To: <20030213041709.XOQ23484.out001.verizon.net@kokeb.ambesa.net> References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302120632.36583.wes@softweyr.com> <200302121411.h1CEBRSe025071@khavrinen.lcs.mit.edu> <200302121521.33506.wes@softweyr.com> <200302121615.h1CGFdGG025691@khavrinen.lcs.mit.edu> <20030213041709.XOQ23484.out001.verizon.net@kokeb.ambesa.net> X-Mailer: VM 7.00 under Emacs 21.2.95.2 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Mike Makonnen writes: > There's no way in hell we or any other BSD is going exclusively to XML for any > particular configuration file. Some Linux distro might try it, but it would > never work. The current file layouts are too entrenched. What would work; > however, is XML as an intermediate data layout. The XML would sit between some > program the user can use to make choices and the underlying text file. Take > rc.conf for example. You could put all that data in XML format and some gui > would let the user choose whatever he/she wanted and that would then be piped > through an XSLT script that renders it in the current key=value layout. The data > would be kept in defaults/rc.conf.xml and defaults/rc.conf would just become > another generated file. This is similar to the approach AIX takes; you can use tools like SMIT to update entries in the ODM (the OO database which stores the system cfg). The traditional UNIX ASCII config files are generated from the ODM. But you can also choose to be a traditionalist, and edit the config files directly, and ignore SMIT and the ODM. Then it's like managing any other UNIX system. Additionally, there was a tool to sync the ODM with what the current state of all the ASCII files is. So you can actually switch back and forth between the two system administration models. --Ben To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 13 8:37:34 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D72E837B401 for ; Thu, 13 Feb 2003 08:37:33 -0800 (PST) Received: from khavrinen.lcs.mit.edu (khavrinen.lcs.mit.edu [18.24.4.193]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0E02F43FB1 for ; Thu, 13 Feb 2003 08:37:33 -0800 (PST) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: from khavrinen.lcs.mit.edu (localhost [IPv6:::1]) by khavrinen.lcs.mit.edu (8.12.6/8.12.6) with ESMTP id h1DGbVbs036830 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=OK); Thu, 13 Feb 2003 11:37:32 -0500 (EST) (envelope-from wollman@khavrinen.lcs.mit.edu) Received: (from wollman@localhost) by khavrinen.lcs.mit.edu (8.12.6/8.12.6/Submit) id h1DGbV92036827; Thu, 13 Feb 2003 11:37:31 -0500 (EST) (envelope-from wollman) Date: Thu, 13 Feb 2003 11:37:31 -0500 (EST) From: Garrett Wollman Message-Id: <200302131637.h1DGbV92036827@khavrinen.lcs.mit.edu> To: Wes Peters Cc: arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) In-Reply-To: <200302130459.38238.wes@softweyr.com> References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302121615.h1CGFdGG025691@khavrinen.lcs.mit.edu> <200302130459.38238.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG < said: > My point exactly. Garrett's point about all those programs having > differing syntax in their configuration files completely ignores > the fact that all those programs do different things; the differing > syntax is a feature of the differences of the program. No, the differing syntax is a consequence of the fact that the programs were all written by different people, none of whom could be bothered to create an extensible format and an appropriate set of reusable library routines to parse it. These differences in syntax are completely artificial, and unnecessary. (I think there's even a whole chapter about this in /The UNIX-Hater's Handbook/; the people who wrote that book were mostly coming from the Lisp Machine community where this is not true.) -GAWollman To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 13 8:47: 2 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4EB6637B401 for ; Thu, 13 Feb 2003 08:47:01 -0800 (PST) Received: from murdoch.servitor.co.uk (murdoch.servitor.co.uk [217.151.99.80]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4B3A543FAF for ; Thu, 13 Feb 2003 08:47:00 -0800 (PST) (envelope-from paul@iconoplex.co.uk) Received: from mmu-firewall.mmu.ac.uk ([149.170.101.200] helo=miter96pq2w1fz) by murdoch.servitor.co.uk with smtp (Exim 3.33 #3) id 18jMVh-0000Gb-00; Thu, 13 Feb 2003 16:46:53 +0000 From: "Paul Robinson" To: "Terry Lambert" Cc: Subject: RE: syslog.conf syntax change (multiple program/host specifications) Date: Thu, 13 Feb 2003 16:46:52 -0000 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) In-Reply-To: <3E4BC495.E3A2FB66@mindspring.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Importance: Normal Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Terry Lambert wrote: > You *really* do not want to go around shooting "bind" in the > head, like the "Life Magazine" cover shot of the Khymer Rouge; > if you end up with a ton of dependencies, then you end up with > a huge accumulated delay, any time there's a state change. It's > no good. Cached Data Considered Harmful (and all that). Agreed. Although there is another approach which gets rid of the dependancies, gives the flexibility people are talking about here, but requires a re-write of almost all Unix software (hey, PAM wasn't that hard was it?): You have an XML file describing where the configuration data lives. That means you have a file that describes to (for example) BIND, "traditional zone file lives in /etc/namedb" or equally "zone info held in this SQL database, in this format" or "take every third byte out of this jpeg and use that to construct zone info". This was an idea I had some time back but never implemented because it got scary, and quite frankly it's too big an issue for one guy to handle on his own. The result though, would be cool. What you have is a standard way of configuring daemons throughout the system, in that the XML represents traditional Unix config files, and that in itself becomes a standard. However, if I want my POP3 software to authenticate using PAM which is pulling info out of SQL, I can do so by changing the XML file, not the software itself. Equally, if somebody wants exactly what ships now with qpopper, they can do so. You implement the flexibility but without the problem of introducing non-standard config files. Of course, the draw here is that all processes then have to talk to a central daemon to do the data collection for it, that in turn understands the XML files, SQL, dbm, whatever else. Big project, big overheads, but still a nice idea. Anyway, it's unimplementable, so I'm rambling, but I thought I'd share in case somebody could adapt it to something realistic. -- Paul Robinson To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 13 9:16: 0 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 975DF37B401 for ; Thu, 13 Feb 2003 09:15:55 -0800 (PST) Received: from agamemnon.cnchost.com (agamemnon.cnchost.com [207.155.252.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4D0BA43F93 for ; Thu, 13 Feb 2003 09:15:53 -0800 (PST) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (adsl-209-204-185-216.sonic.net [209.204.185.216]) by agamemnon.cnchost.com id MAA21247; Thu, 13 Feb 2003 12:15:50 -0500 (EST) [ConcentricHost SMTP Relay 1.15] Message-ID: <200302131715.MAA21247@agamemnon.cnchost.com> To: Terry Lambert Cc: arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) In-reply-to: Your message of "Wed, 12 Feb 2003 18:39:22 PST." <3E4B055A.641E4B6A@mindspring.com> Date: Thu, 13 Feb 2003 09:15:49 -0800 From: Bakul Shah Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > > In my view XML is to data representation what Roman numerals > > are to math. > > *This* belongs in the fortunes database. :-> > > Rather than repeat all the arguments I'll point you to what > > knowledgeable people like Erik Naggum have to say about XML > > (search comp.lang.lisp on groups.google.com). Naggum used to > > be a strong proponent of SGML until he "saw the light"! A > > couple of recent thread on XML on comp.lang.lisp are worth > > browsing. > > Erik is hardly unbiased. Back "in the old days" of the early > 1990's, when we were first discussing the formulation of the > SGML standard on Usenet, he lost some bitter battles with the > main SGML guys about representation in DTD's and a number of > other points. He may or may not be the "forgive and forget" > type of guy. Well, of course he is biased; he spent over 6 years of his life doing SGML before bailing out! Though I doubt he holds a grudge because of some lost battles. The issue is the braindeadness of XML, not Naggum. Since no one likely wants to trawl through comp.lang.lisp, I'll quote a few things (with message-ids so that you can look them up if you want) while leaving out his colorful choice phrases. Please excuse the length of this message; I got carried away :-) In <3239150725773370@naggum.no> I do not consider myself "next generation" anything, but I have contributed to GNU Emacs just short of a decade, was a world-renown expert on SGML and related standards until I wrote a book about it and realized that 97.5% of SGML is braindamaged and I could not gloss over it, anymore. ... In <3239282082503601@naggum.no> he says: * Paolo Amoroso | Could you elaborate on the 2.5% of SGML that is not braindamaged? What | ideas are worth saving? Do such ideas also live somewhere else (e.g., in | XML)? To start from the end, the ideas survived unchanged in XML, neither improved nor weakened. The first core idea is hierarchical structuring of information. Lisp has had this forever. The second is textual representation. Lisp has had this forever. The third is validatability of the structure. Lisp has no notion of this separate from the semantics of special operators or the usual evaluation rules. The fourth is an entity structure that allow documents to share components. Lisp does this by loading things into the image and the `require'/`provide' pair of deprecated operators. From there on it is all downhill for the SGML family. The really grave mistakes include the element-attribute dichotomy, the new syntax for every meta-level, such as the attribute-value syntax, the qouting conventions, character entities and references, document type definitions, and ultimately the sgml declaration, the primitive language for content models, the changing syntax according as features are supported or not, the redundant end-tag, and failure to use a simple escaping mechanism for characters that could be confused as markup with the character references as data adding additional complexity. The sheer complexity of parsing SGML and XML and presenting it to the caller lies mainly in the syntactic mess and the utter lack of an intelligent model for in-memory representation. (DOM is completely braindamaged with no redeeming qualities.) ... In <3239316488198320@naggum.no> he says: * Oleg | Who said proper nesting is intrinsically a good thing, silly rules made up | by mortals notwithstanding? It is not the proper nesting that is the problem. It is the infinitely stupid separation of start- and end-tags. If you wrote @bold{foo} instead of foo, you would not even /want/ to say @bold{foo @italic{bar} zot} if you expected to get foo in bold, zot in italic, and bar in both. The stupidity of foobarzot is not in nesting, it is in the moronic syntax. ... ... The otherwise pretty smart people who dreamt up generalized markup (which is itself an important abstraction that few people have made independently) made a very serious mistake in making "tags" independent constructs of the element. Yes, abstract elements have starts and ends, but it is a serious mistake to make both sides verbose and explicit. ... ... When you give people verbose, explicit edges of elements, they will think of them the wrong way. It is not proper nesting that they do not understand. It is the fact that they see a kind of different edges than the syntax wants to represent. ... ... In many important ways, SGML is its own worst enemy. It has managed to teach information structuring completely backwards. Instead of making the edges explicit but non-intrusive as in Common Lisp, the edges are much too visible and verbose with the stupid tags that very quickly consume more of the large volume of characters in SGML/XML documents than the contents. I am sure it was thought of as useful back when people needed to be converted, but once converted, it gets in the way more than anything else and leads people to make mistakes if they do not think very carefully about what they try to do. ... In <3239423919385905@naggum.no> he says: * Johan Kullstam | Yes, but this what is stupid. The rules of the language says you | *must* properly nest the tags. However, the syntax suggests that not | nesting is possible. Why else introduce the verbose closing tag | unless you could close something else than the last opened tag? Historically, the idea was that omitted start- and end-tags should be inferrable. This is a bad idea for a large number of very good reasons, and XML did away with them, which is the same kind of improvement that drowning at a depth of 10 feet is an improvement over drowning at a depth of 100 feet, but in a Stroustrupesque move, XML decided to keep the antiquated end-tags which had now survived their usefulness. The whole syntax was an improvement over its predecessors in 1970, but after it had been adopted, it should have been further improved. ... In <3241162747773498@naggum.no> he says: * synthespian | I don't understand your point of view, in the light of the fact that a | couple of weeks ago you said that SGML and XML were "braindead." Well, let me put it this way: The XML crowd believes that if you only add enough markup, everything humankind has ever dreamt of will suddenly emerge from the chaos. I believe that this is a very serious misunderstanding of both chaos and emergent properties and that more markup will, in fact, prevent them from emerging. XML-style markup effectively prohibits the multiple perspectives on the same information that makes it usable for multiple purposes. In this sense, the more you employ XML to achieve your goals, the more irrelevant the result will become. In <3242173964750774@naggum.no>: | Am I wrong in assuming that with XML you are roughly as safe as you would be | with a _documented_ binary format? It is a myth that XML is documented. You have no idea what the elements /actually/ mean (and have warped into over time) until you see the source code for the application. XML becomes /more/ application-dependent over time than binary formats because it provides a false security and an appearance that belies its true nature. XML /is/ a binary format, it is just that it is the kind of binary formats that line printers and raw text editors can use, too, and it is no less dependent on the exactness and precision that other binary formats require. At least when you have a binary format, you know that you need to have a version field and proper version management. People who think SGML or XML will save them tend to forget version management and rely on stupid human tendencies to believe that that which they can "read" must also be understandable to the machine. ... In <3242255304507299@naggum.no> * Tim Josling | XML is an encoding format, no more than that. You may find it illuminating to do a web search on my name and SGML. | It is a pretty good encoding format because it is relatively simple and | semi-human-readable, though verbose. Compare with the alteratives - ad hoc | binary formats or the IEEE's binary format monstrosity whose name I forget. As long as you actually believe that such are the alternatives, yes, XML is better than the completely braindamaged. However, if you start to think about the problem, XML starts to become an idiotic non-solution that only creates more problems than it solves. It has all the disadvantages of an ad hoc binary format, and none of the benefits -- namely compactness and version sensitivity. ... | You can't blame this on XML. I can, and I do. Languages come with philosophies from which they cannot be separated. The XML philosphy is stale, stupid, and counter-productive relative to its own stated goals, among which the most important was supposed to be the independence of data from the application, which is actually worse with XML than even /your/ "alternatives". To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 13 9:45:37 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4B09637B401; Thu, 13 Feb 2003 09:45:35 -0800 (PST) Received: from ns1.gnf.org (ns1.gnf.org [63.196.132.67]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7034B43F85; Thu, 13 Feb 2003 09:45:34 -0800 (PST) (envelope-from gtetlow@gnf.org) Received: from EXCHCLUSTER01.lj.gnf.org (exch01.lj.gnf.org [172.25.10.19]) by ns1.gnf.org (8.12.3/8.12.3) with ESMTP id h1DHjXMf095824; Thu, 13 Feb 2003 09:45:33 -0800 (PST) (envelope-from gtetlow@gnf.org) Received: from roark.gnf.org ([172.25.24.15]) by EXCHCLUSTER01.lj.gnf.org with Microsoft SMTPSVC(5.0.2195.5329); Thu, 13 Feb 2003 09:45:34 -0800 Received: from roark.gnf.org (localhost [127.0.0.1]) by roark.gnf.org (8.12.6/8.12.6) with ESMTP id h1DHjXGj027812; Thu, 13 Feb 2003 09:45:33 -0800 (PST) (envelope-from gtetlow@gnf.org) Received: (from gtetlow@localhost) by roark.gnf.org (8.12.6/8.12.6/Submit) id h1DHjV7k027811; Thu, 13 Feb 2003 09:45:31 -0800 (PST) Date: Thu, 13 Feb 2003 09:45:31 -0800 From: Gordon Tetlow To: Thomas Quinot Cc: Wesley Peters , arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) Message-ID: <20030213174531.GZ83215@roark.gnf.org> References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302111430.03156.wes@softweyr.com> <20030213001845.GA85036@melusine.cuivre.fr.eu.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="95n2dZZDVPvlnkUw" Content-Disposition: inline In-Reply-To: <20030213001845.GA85036@melusine.cuivre.fr.eu.org> User-Agent: Mutt/1.4i X-OriginalArrivalTime: 13 Feb 2003 17:45:34.0101 (UTC) FILETIME=[B5A01C50:01C2D387] Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --95n2dZZDVPvlnkUw Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Feb 13, 2003 at 01:18:45AM +0100, Thomas Quinot wrote: > Le 2003-02-11, Wesley Peters ?crivait : >=20 > > mail.info /var/log/maillog G 4M > > lpr.info /var/log/lpd-errs 256K > > local3.* /var/log/foo B 1K > > ftp.* /var/log/ftp B 1M >=20 > I think this can be achieved without modifying syslogd. You can achieve > the same effect by using an appropriate program as the log destination > (think Apache's rotatelogs). >=20 > Eg: > local3.* |/usr/local/sbin/loglimit /var/log/foo B 1K >=20 > Where loglimit would copy stdin to the named file, rotating it when > appropriate. I personally like this solution better. Sure you get the overhead of a couple more processes running, but it falls back on each tool doing one specific job. This keeps our syslog.conf consistent with others and it should be relatively easy to implement a log roller (or steal one). -gordon --95n2dZZDVPvlnkUw Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (FreeBSD) iD8DBQE+S9m7Ru2t9DV9ZfsRAleSAJ9F4M2qo84fQLGcry3k8h9viAg2gwCeKh5p xrw+qO7E+XXjPZ8OUQ8ZfDE= =qB8r -----END PGP SIGNATURE----- --95n2dZZDVPvlnkUw-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 13 10:43:13 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 11D2537B401; Thu, 13 Feb 2003 10:43:13 -0800 (PST) Received: from melusine.cuivre.fr.eu.org (melusine.cuivre.fr.eu.org [62.212.105.185]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0C6FC43FAF; Thu, 13 Feb 2003 10:43:12 -0800 (PST) (envelope-from thomas@FreeBSD.ORG) Received: by melusine.cuivre.fr.eu.org (Postfix, from userid 1000) id 89C5C2C3D1; Thu, 13 Feb 2003 19:43:09 +0100 (CET) Date: Thu, 13 Feb 2003 19:43:09 +0100 From: Thomas Quinot To: Gordon Tetlow Cc: Thomas Quinot , Wesley Peters , arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) Message-ID: <20030213184309.GA53098@melusine.cuivre.fr.eu.org> Reply-To: Thomas Quinot References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302111430.03156.wes@softweyr.com> <20030213001845.GA85036@melusine.cuivre.fr.eu.org> <20030213174531.GZ83215@roark.gnf.org> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20030213174531.GZ83215@roark.gnf.org> User-Agent: Mutt/1.4i X-message-flag: WARNING! Using Outlook can damage your computer. Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Le 2003-02-13, Gordon Tetlow crivait : > it should be relatively easy to implement a log roller (or steal one). Yes, much code for that could be lifted from newsyslog presumably. Thomas. -- Thomas.Quinot@Cuivre.FR.EU.ORG To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 13 12:18:17 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BFE5437B401 for ; Thu, 13 Feb 2003 12:18:16 -0800 (PST) Received: from w250.z064001178.sjc-ca.dsl.cnc.net (adsl-66.218.45.239.dslextreme.com [66.218.45.239]) by mx1.FreeBSD.org (Postfix) with SMTP id 85C7A43F75 for ; Thu, 13 Feb 2003 12:18:14 -0800 (PST) (envelope-from jos@catnook.com) Received: (qmail 37265 invoked by uid 1000); 13 Feb 2003 20:18:34 -0000 Date: Thu, 13 Feb 2003 12:18:12 -0800 From: Jos Backus To: arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) Message-ID: <20030213201834.GA34850@lizzy.catnook.com> Reply-To: jos@catnook.com Mail-Followup-To: arch@FreeBSD.ORG References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302111430.03156.wes@softweyr.com> <20030213001845.GA85036@melusine.cuivre.fr.eu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030213001845.GA85036@melusine.cuivre.fr.eu.org> User-Agent: Mutt/1.5.3i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, Feb 13, 2003 at 01:18:45AM +0100, Thomas Quinot wrote: > Eg: > local3.* |/usr/local/sbin/loglimit /var/log/foo B 1K > > Where loglimit would copy stdin to the named file, rotating it when > appropriate. This program already exists, it's named multilog. See /usr/ports/sysutils/daemontools. -- Jos Backus _/ _/_/_/ Sunnyvale, CA _/ _/ _/ _/ _/_/_/ _/ _/ _/ _/ jos at catnook.com _/_/ _/_/_/ require 'std/disclaimer' To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Feb 13 16:36:18 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 21C7737B43E for ; Thu, 13 Feb 2003 16:36:06 -0800 (PST) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4673843F75 for ; Thu, 13 Feb 2003 16:36:05 -0800 (PST) (envelope-from scottl@FreeBSD.org) Received: from freefall.freebsd.org (scottl@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.6/8.12.6) with ESMTP id h1E0a5NS071041 for ; Thu, 13 Feb 2003 16:36:05 -0800 (PST) (envelope-from scottl@freefall.freebsd.org) Received: (from scottl@localhost) by freefall.freebsd.org (8.12.6/8.12.6/Submit) id h1E0a5Vi071040 for arch@freebsd.org; Thu, 13 Feb 2003 16:36:05 -0800 (PST) Date: Thu, 13 Feb 2003 16:36:05 -0800 (PST) From: Scott Long Message-Id: <200302140036.h1E0a5Vi071040@freefall.freebsd.org> To: arch@freebsd.org Subject: 5-STABLE Roadmap Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG All, Thanks to the hard work of everyone, FreeBSD 5.0 became a reality and is working better than most even hoped. However, there is still a lot of work to be done before we can create the RELENG_5/5-STABLE branch and declare success. Below is a document that I have drafted with the input and review of the Release Engineering Team, the Technical Review Board, and the Core Team that defines what needs to be done in order to reach 5-STABLE. I'm happy to take further input into this, and I will also mark it up and make it available online. The Roadmap for 5-STABLE 1. Introduction and background After nearly three years of work, FreeBSD 5.0 was released in January of 2003. Features like the GEOM block layer, Mandatory Access Controls, ACPI, sparc64 and ia64 platform support, and UFS snapshots, background filesystem checks, and 64-bit inode sizes make it an exciting operating system for both desktop and production users. However, some important features are not complete. The foundations for fine-grained locking and preemption in the kernel exist, but much more work is left to be done. Work on Kernel Schedulable Entities, also known as Scheduler Activations, has been ongoing but needs a push to realize its benefit. Performance compared to FreeBSD 4.x has declined and must be restored and surpassed. This is somewhat similar to the situation that FreeBSD faced in the 3.x series. Work on 3-CURRENT trudged along seemingly forever, and finally a cry was made to 'just ship it' and clean up later. This decision resulted in the 3.0 and 3.1 releases being very unsatisfying for most, and it wasn't until 3.2 that the series was considered 'stable'. To make matters worse, the RELENG_3 branch was created along with the 3.0 release, and the HEAD branch was allowed to advance immediately towards 4-CURRENT. This resulted in a quick divergence between HEAD and RELENG_3, making maintenance of the RELENG_3 branch very difficult. FreeBSD 2.2.8 was left for quite a while as the last production-quality version of FreeBSD. Our intent is to avoid repeating that scenario with FreeBSD 5.x. Delaying the RELENG_5 branch until it is stable and production quality will ensure that it stays maintainable and provides a compelling reason to upgrade from 4.x, To do this, we must identify the current areas of weakness and set clear goals for resolving them. This document contains what we as the release engineering team feel are the milestones and issues that must be resolved for the RELENG_5 branch. It does not dictate every aspect of FreeBSD development, and we welcome further input. Nothing that follows is meant to be a sleight against any person or group, or to trivialize any work that has been done. There are some significant issues, though, that need decisive and unbiased action. 2. Major issues The state of SMPng and kernel lockdown is the biggest concern for 5.x. To date, few major systems have come out from under the kernel-wide mutex known as 'Giant'. The SMP status page at http://www.FreeBSD.org/smp provides a comprehensive breakdown of the overall SMPng status. Status specific to SMPng progress in deivce drivers can be found at at httP//www.FreeBSD.org/projects/busdma. In summary: - VM - the kmem_malloc(M_NOWAIT) path no longer needs Giant held. The kmem_malloc(M_WAITOK) path is in progress and is expected to be finished in the coming weeks. Other facets of the VM system, like the vfs interface, buffer/cache, etc, are largely untouched. - GEOM - The GEOM block layer was designed to run free of Giant, but at this time no block drivers can run without Giant. Additionally, it has the potential to suffer performance loss due to its upcall/downcall data paths happening in kernel threads. Lightweight context switches might help this. - Network - Work is in progress to lock the TCP and UDP portions of the stack. This also includes locking the routing tree, ARP code, and ifaddr and inet data structures. RawIP, IPv6, Appletalk, etc, have not been touched. Locking the socket layer is in progress but is largely untested. None of the hardware drivers have been locked. - VFS - Initial pre-cleanup started. - buffer/cache - Initial work complete. - Proc - Work on locking the proc structure was ongoing for a while but seems to have stalled. - CAM - No significant work has occurred on the CAM SCSI layer. - Newbus - some work has started on locking down the device_t structure. - Pipes - complete with the exception of VM-related optimizations. - File descriptors - complete. - Process accounting - jails, credentials, MAC labels, and scheduler are out from under Giant. - MAC Framework - complete - Timekeeping - complete - kernel encryption - crypto drivers and core crypto framework are Giant-free. KAME IPsec and FAST IPSec have not been locked. - Sound subsystem - complete - kernel preemption - preemption for interrupt threads is enabled. However, contention due to Giant covering much of the kernel and most of the device driver interrupt routines causes excessive context switches and might actually be hurting performance. Work is underway to explore ways to make preemption be conditional. Another issue with SMPng is interrupt latency. The overhead of doing a complete context switch to a kernel interrupt thread is high and shows noticeable latency. Work is ongoing to implement lazy context switching on all platforms. Fine grained locking of drivers will also help this, as will converting drivers to be as efficient as possible in their interrupt routines. Next, the state of KSE must resolved for RELENG_5. Work on it has slowed noticeably in the past 6 months but appears to be picking up again. There are a number of issues that must be addressed: - Signal delivery to threads is not defined. Signals are delivered to the process, but which thread actually receives it is random. - There is confusion over whether upcalls are generated on every system call or when a thread blocks. The former is highly undesirable and needs to be investigated. - The userland threading library, currently called libkse, is incomplete and has not been used for any significant threaded application. - KSE has the potential to uncover latent race conditions and create new ones. An audit needs to be performed to ensure that no obvious problems exist. According to the release schedule below, KSE kernel and userland components must be functionality complete by June 2003 in order to be included in the RELENG_5 branch. For security and stability reasons, if KSE cannot be finished in time then, by default, all KSE-specific syscalls should be modified to return ENOSYS and all other KSE-specific interfaces disabled. Deprecating KSE from RELENG_5 but keeping it in the HEAD branch will pose problems in porting bugfixes and features between the two branches, so every effort should be made to finish it on time. 3. Goals for 5-STABLE The goals for the RELENG_5 branch point are: - All subsystems and interfaces must be mature enough to be maintainable for improvements and bug fixes - equal or better stability from FreeBSD 4.8. - no functional regressions from 4.8. It is important to make sure that users do not avoid upgrading to 5.x because of lost functionality. - performance on par with FreeBSD 4.8 for most common operations. Both UP and SMP configurations should be evaluated. SMP has the potential to perform much better than 4.x, though for the purposes of creating the RELENG_5 branch, comparable performance between the two should be acceptable. It is unrealistic to expect that the SMPng project will be fully complete by RELENG_5, or that performance will be significantly better than 4.x. However, focusing on a subset of the outstanding tasks will give enough benefit for the branch to be viable and maintainable. To break it down: - ABI/API/Infrastructure stability - Enough infrastructure must be in place and stable to allow fixes from HEAD to easily and safely be merged into RELENG_5. Also, we must draw a line as to what subsystems are to be locked down when we go into 5-STABLE. - SMPng - VM - Most codepaths, others than the ones that interact with VFS, should be Giant-free for RELENG_5. - Network - Taking the network stack out from under Giant poses the risk of uncovering latent bugs and races. Locking it down but not removing Giant imposes further performance penalties. A decision on whether to continue with locking the network layers, and whether they should be free from Giant for RELENG_5 should be made no later than March 15. If the decision is made to allow the locking to go forward, the IPv4, UDP, and TCP layers should be free of Giant. IPv6 and the socket layers would be nice to have also, though it should be investigated whether they can be safely locked down in 5.x after the RELENG_5 branch. If the decision is to keep the network stack under Giant for the branch, then an investigation should be made to determine if the present locking work can be reverted and deferred to 6-CURRENT. Having a Giant-free path from the the TCP/IP layers to the hardware should be investigated as it could allow significant performance gains in the network benchmarks. If this can be achieved then the hardware interface layer needs to allow for drivers to incrementally become free of Giant. Locking down at least two Ethernet drivers would be highly desirable. If the semantics are too complex to have the stack free of Giant but not the hardware drivers, investigation should be done into making it configurable. Lesser-used network stacks like netatlk, netipx, etc, should not break while this work is going on. However, locking them is not a high priority. - GEOM - At least 2 block drivers should be locked in order to demonstrate that others can also be locked without changing the interface to GEOM. The ATA driver is a good candidate for this, though caution should be taken as it is also extremely high-profile and any problems with it will affect nearly all users of FreeBSD. - Lazy context switching - sparc64 is the only platform that performs lazy context switching when entering the kernel. The performance gains promised by this are significant enough to require that it be implemented for all other Tier 1 platforms. - KSE - The kernel side of KSE must be functionally complete and have undergone a security audit. libkse must be complete enough to demonstrate a real-world application running correctly on it using the standard POSIX Threads API. Examples would be apache 2.0, squid, and/or mozilla. A functional regression test suite is also a requirement for RELENG_5 and should test signal delivery, scheduling, performance, and process security/credentials for both KSE and non- KSE processes. KSE kernel and userland components must also reach the same level of functionality for all Tier-1 platforms in both UP and SMP configurations. The definition of 'Tier-1 platforms' can be found at http://www.freebsd.org/doc/en_US.ISO8859-1/articles/committers-guide/archs.html. - busdma interface and drivers - architectures like PAE/i386 and sparc64 which don't have a direct mapping between host memory address space and expansion bus address space require the elimination for vtophys() and friends. The busdma interface was created to handle exactly this problem, but many drivers do not use it yet. The busdma project at http://www.FreeBSD.org/projects/busdma/index.html tracks the progress of this and should be used to determine which drivers must be converted for RELENG_5 and which can be left behind. Also, there has been talk by several developers and the original author to give the busdma interface a minor overhaul. If this is to happen, it needs to happen before RELENG_5. Otherwise, differences between the old and new API will make driver maintenance difficult. - PCI resource allocation - PC2003 compliance requires that x86 systems no longer configure PCI devices from the system BIOS, leaving this task soley to the OS. FreeBSD must gain the ability to manage and allocate PCI memory resources on its own. Implementing this should take into account cardbus, PCI-HotPlug, and laptop dockstation requirements. This feature will become increasingly critical through the lifetime of RELENG_5, and therefore is a requirement for the RELENG_5 branch. - Performance - most performance gains hinge on the progress of SMPng Areas that should be concentrated on are: - Storage I/O - I/O performance suffers from two problems, too many expensive context switches, and too much work being done in interrupt threads. Specifically, it takes 3 context switches for most drivers to get from the hardware completion interrupt to unblocking the user process: one for the interrupt thread, one for the GEOM g_up thread, and one to get back to the user thread. Drivers that attempt to be efficient and quick in their interrupt handlers (as all should be) usually also schedule a taskqueue, which adds a context switch in between the interrupt thread and the g_up thread and brings the total up to 4. Two things need to be done to attack this: - make all drivers defer most of their processing out of their interrupt thread. Significant performance gains have been shown recently in the aac(4) driver by making its interrupt handler be 'INTR_MPSAFE' and moving all processing to a taskqueue. - investigate eliminating the taskqueue context switch by adding a callback to the g_up thread that allows a driver to do its interrupt processing there instead of in the taskqueue. - Network - Network drivers suffer from the interrupt latency previously mentioned as well as from the network stack being partially locked down but not free from Giant. Possible strategies for addressing this are described in the previous section. - Other locking - XXX ? - Benchmarks and performance testing - Having a source of reliable and useful benchmarks is essential to identifying performance problems and guarding against performance regressions. A 'performance team' that is made up of people and resources for formulating, developing, and executing benchmark tests should be put into place soon. Comparisons should be made against both FreeBSD 4.x and Linux 2.4.x. Tests to consider are: - the classic 'worldstone' - webstone - /usr/ports/www/webstone - Fstress - http://www.cs.duke.edu/ari/fstress - ApacheBench - /usr/ports/www/p5-ApacheBench - netperf - /usr/ports/benchmarks/netperf - Features: - ACPI - Intel's ACPI power management and device configuration subsystem has become an integral part of FreeBSD's x86 and ia64 device configuration model. However, many bugs exist in Intel's vendor code, our OS-specific code, and motherboard BIOSes, causing many ACPI-enabled systems to fail to boot, misdetect drivers, and/or have many other problems. Fixing these problems seems to be an uphill battle and is often times causing a poor first-impression of FreeBSD 5.0. Most x86 systems can function with ACPI disabled, and logic should be added to the bootloader and sysinstall to allow users to easily and intuitively turn it off. Turning off ACPI by default is prone to problems also as many newer systems rely on it to provide correct interrupt routing information. Also, a centralized resource should be created to track ACPI problems and solutions. Linux uses the same Intel vendor sources as FreeBSD, so we should investigate how they have handled some of the known problems. - NEWCARD/OLDCARD - The NEWCARD subsystem was made the default for 5.0. Unfortunately, it contains no support for non-Cardbus bridges and falls victim to interrupt routine problems on some laptops. The classic 16-bit bridge support, OLDCARD, still exists and can be compiled in, but this is highly inconvenient for users of older laptops. If OLDCARD cannot be completely deprecated for RELENG_5, then provisions must be made to allow users to easily install an OLDCARD-enabled kernel. Documentation should be written to help trasition users from OLDCARD to NEWCARD and from 'pccardd' to 'devd'. The power management and 'dumpcis' functionality of pccardc(1) needs to be brought forward to work with NEWCARD, along with the ability to load CIS quirk entries. Most of this functionality can be integrated into devd and devctl. - New scheduler framework - The new scheduler framework is in place, and users can select between the classic 44bsd scheduler and the new ULE scheduler. A scheduler that demonstrates processor affinity, HyperThreading and KSE awareness, and no regressions in performance or interactivity characteristics must be available for RELENG_5. - sparc64 local console - neither syscons nor vt work on sparc64, leaving it with only serial and 'fake' OFW console support. This is a major support hole for what is a Tier 1 platform. Whether syscons can be shoe-horned in or wscons be adopted from NetBSD is up for debate. However, sparc64 must have local console support for RELENG_5. Having this will also allow the XFree86 server to run, which is also a requirement for RELENG_5. - gcc/toolchain - gcc 3.3 might be available in time for RELENG_5 and might offer some attractive benefits, but also likely to introduce ABI incompatibility with prior gcc versions. ABI compatibility should be locked down for the RELENG_5 branch. There has also been a request to move /usr/include/g++ to /usr/include/g++-v3 to be more compliant with the stock behavior of gcc. This should be investigated for RELENG_5 also. - gdb - gdb from the base system should work for sparc64. It should also understand KSE thread semantics, assuming that KSE is included in the RELENG_5 branch. gdb 5.3 is available and there are reports that it should address the sparc64 issue. - disklabel(8) regressions - The biggest casualty of the introduction of GEOM appears to be the disklabel utility. The '-r' option gives unpredictable results in most cases now and should be removed or fixed. Work is planned for a new unified interface for modifying labels and slices, however this should not preclude disklabel from being fixed. - Documentation: - The manual pages, Handbook, and FAQ should be free from content specific to FreeBSD 4.x, i.e. all text should be equally applicable to FreeBSD 5.x. The installation section of the handbook needs the most work in this area. - The release documentation needs to be complete and accurate for all Tier 1 architectures. The hardware notes and installation guides need specific attention. - If FreeBSD 5.1 is not the branch point for RELENG_5 then the Early Adopters Guide needs to be updated. This document should then be removed just before the release closest to the RELENG_5 branch point. 4. Schedule If branching RELENG_5 at the 5.1 release is paramount, 5.1 will probably need to move out by at least 3 months. The schedule would be: - Jun 30, 2003 - KSE and SMPng feature freeze - Aug 4, 2003 - 5.1-BETA, general code freeze - Aug 18, 2003 - 5.1-RC1, RELENG_5 and RELENG_5_1 branched - Aug 25, 2003 - 5.1-RC2 - Sept 1, 2003 - 5.1-RELEASE Taking an incremental approach might be more beneficial. Releasing 5.1 in time for USENIX ATC 2003 will provide a wide audience for productive feedback and will keep FreeBSD visible. In this scenario, 5.1 should offer a significant improvement over 5.0 in terms of bug fixes and performance. Lockdowns and improvements to the storage subsystem and scheduler should be expected, the NEWCARD/OLDCARD issues should be addressed, and all known bugs and regressions from the 5.0 errata list should be fixed. KSE and other SMPng tasks that cannot finish in time for 5.1 should also not reduce the stability of the release. The schedule for this would be: - May 5, 2003 - 5.1-BETA, general code freeze - May 19, 2003 - 5.1-RC1, RELENG_5_1 branched - May 27, 2003 - 5.1-RC2 - Jun 2, 2003 - 5.1-RELEASE - Jun 30, 2003 - KSE and SMPng feature freeze - Sept 1, 2003 - 5.2-BETA, general code freeze - Sept 15, 2003 - 5.2-RC1, RELENG_5 and RELENG_5_2 branched - Sept 22, 2003 - 5.2-RC2 - Sept 29, 2003 - 5.2-RELEASE 5. Post RELENG_5 direction As with all -STABLE development streams, the focus should be bug fixes and incremental improvements. Just like normal, everything should be vetted through the HEAD branch first and committed to RELENG_5 with caution. As before, new device drivers, incremental features, etc, will be welcome in the branch once they have been proven in HEAD. Further SMPng lockdowns will be divided into two categories, driver and subsystem. The only subsystem that will be sufficiently locked down for RELENG_5 will be GEOM, so incrementally locking down device drivers under it is a worthy goal for the branch. Full subsystem lockdowns will have to be fully tested and proven in HEAD before consideration will be given to merging them into RELENG_5. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 14 0:49:18 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2682237B401 for ; Fri, 14 Feb 2003 00:49:17 -0800 (PST) Received: from day.anthologeek.net (day.anthologeek.net [213.91.4.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id 384F543F85 for ; Fri, 14 Feb 2003 00:49:16 -0800 (PST) (envelope-from sw@anthologeek.net) Received: by day.anthologeek.net (Postfix, from userid 1000) id 416DA171A2; Fri, 14 Feb 2003 09:46:35 +0100 (CET) Date: Fri, 14 Feb 2003 09:46:35 +0100 From: Sameh Ghane To: Hidetoshi Shimokawa Cc: freebsd-arch@freebsd.org Subject: Re: You may want to have FireWire.... Message-ID: <20030214084635.GB87002@anthologeek.net> References: <20030209193737.D91611DEAB@www.fastmail.fm> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4i X-PGP-Keys: 0x1289F00D: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Le (On) Wed, Feb 12, 2003 at 05:24:54PM +0900, Hidetoshi Shimokawa ecrivit (wrote): > Do you think this useful? I was waiting so much for something like this ! I finally found a laptop with a serial port built-in, but it's a really good idea as nowadays they're rare. > I wrote this driver(dcons(4)) and utility(fwchat(8)) for a replacement > of the serial console. This framework exploits physical access > faculty of the IEEE1394 OHCI chip(fwochi(4)). This means that you > cannot access this console until the OHCI is initialized on target > system but once it's initialized, there is no need of OS support to > do transactions on the FireWire bus until next bus reset occurs. This > enables us to use this console as DDB/GDB port too. By analogy with > pty(4), dcons(4) acts like a slave device and fwchat(8) acts like a > master device, but they are on different machines and interact via > FireWire. Please note dcons(4) is device(FireWire)-independent by > itself. (And almost plathome independent) Humm, this won't allow people to grab boot messages before a freeze/hang/panic at boot time then ? Thank you for this feature ! Cheers, -- Sameh To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 14 4:22:45 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2628A37B401 for ; Fri, 14 Feb 2003 04:22:44 -0800 (PST) Received: from is1.mh.itc.u-tokyo.ac.jp (is1.mh.itc.u-tokyo.ac.jp [133.11.205.11]) by mx1.FreeBSD.org (Postfix) with ESMTP id C402643FBD for ; Fri, 14 Feb 2003 04:22:42 -0800 (PST) (envelope-from simokawa@sat.t.u-tokyo.ac.jp) Received: from is1.mh.itc.u-tokyo.ac.jp (is1.mh.itc.u-tokyo.ac.jp [127.0.0.1]) by is1.mh.itc.u-tokyo.ac.jp (Postfix) with ESMTP id 99A162180DF for ; Fri, 14 Feb 2003 21:22:41 +0900 (JST) Received: from mailhosting.itc.u-tokyo.ac.jp (IDENT:mirapoint@mailhosting.itc.u-tokyo.ac.jp [133.11.205.3]) by is1.mh.itc.u-tokyo.ac.jp (8.11.3/8.11.3) with ESMTP id h1ECMfc17313; Fri, 14 Feb 2003 21:22:41 +0900 Received: from ett.sat.t.u-tokyo.ac.jp (ett.sat.t.u-tokyo.ac.jp [133.11.135.3]) by mailhosting.itc.u-tokyo.ac.jp (Mirapoint Messaging Server MOS 2.9.3.2) with ESMTP id AHV89793; Fri, 14 Feb 2003 21:22:40 +0900 (JST) Date: Fri, 14 Feb 2003 21:22:40 +0900 Message-ID: From: Hidetoshi Shimokawa To: Sameh Ghane Cc: freebsd-arch@freebsd.org Subject: Re: You may want to have FireWire.... In-Reply-To: <20030214084635.GB87002@anthologeek.net> References: <20030209193737.D91611DEAB@www.fastmail.fm> <20030214084635.GB87002@anthologeek.net> User-Agent: Wanderlust/2.11.0 (Wonderwall) REMI/1.14.3 (Matsudai) FLIM/1.14.3 (=?ISO-8859-1?Q?Unebigory=F2mae?=) APEL/10.3 MULE XEmacs/21.4 (patch 8) (Honest Recruiter) (i386--freebsd) X-Face: OE([KxWyJI0r[R~S/>7ia}SJ)i%a,$-9%7{*yihQk|]gl}2p#"oXmX/fT}Bn7: #j7i14gu$jgR\S*&C3R/pJX List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG At Fri, 14 Feb 2003 09:46:35 +0100, Sameh Ghane wrote: > Humm, this won't allow people to grab boot messages before a freeze/hang/panic > at boot time then ? It won't allow if the disaster happens before fwohci initilization step. I wish loader or BIOS would initialize the chip some day... /\ Hidetoshi Shimokawa \/ simokawa@sat.t.u-tokyo.ac.jp PGP public key: http://www.sat.t.u-tokyo.ac.jp/~simokawa/pgp.html To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 14 11: 4:15 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6F1D937B405; Fri, 14 Feb 2003 11:04:14 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id 89CF343FD7; Fri, 14 Feb 2003 11:04:08 -0800 (PST) (envelope-from wes@softweyr.com) Received: from salty.rapid.stbernard.com (corp-2.ipinc.com [199.245.188.2]) by smtp-relay.omnis.com (Postfix) with ESMTP id 6F717446C4; Fri, 14 Feb 2003 11:00:25 -0800 (PST) From: Wes Peters Organization: Softweyr.com To: Thomas Quinot , Gordon Tetlow Subject: Re: syslog.conf syntax change (multiple program/host specifications) Date: Fri, 14 Feb 2003 11:00:23 -0800 User-Agent: KMail/1.5 Cc: Thomas Quinot , arch@FreeBSD.ORG References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <20030213174531.GZ83215@roark.gnf.org> <20030213184309.GA53098@melusine.cuivre.fr.eu.org> In-Reply-To: <20030213184309.GA53098@melusine.cuivre.fr.eu.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Message-Id: <200302141100.23529.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thursday 13 February 2003 10:43, Thomas Quinot wrote: > Le 2003-02-13, Gordon Tetlow =E9crivait : > > it should be relatively easy to implement a log roller (or steal one). > > Yes, much code for that could be lifted from newsyslog presumably. To this end I've implemented another feature, 'N' for newsyslog. When the file size limit is reached, newsyslog is run with the=20 log filename as the only argument. The size limitation in syslog.conf and newsyslog.conf should agree or you won't get what you expect. Does this seem reasonable? Can anyone suggest a better syntax that is in keeping with the=20 current syslog.conf? I am *not* going to replace the syslog.conf=20 with a different format, only extend it to encompass this simple=20 functionality. On with the bikeshed... =2D-=20 "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 14 12:17:39 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 119B537B401; Fri, 14 Feb 2003 12:17:38 -0800 (PST) Received: from smtp4.server.rpi.edu (smtp4.server.rpi.edu [128.113.2.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2810E43F75; Fri, 14 Feb 2003 12:17:37 -0800 (PST) (envelope-from drosih@rpi.edu) Received: from [128.113.24.47] (gilead.netel.rpi.edu [128.113.24.47]) by smtp4.server.rpi.edu (8.12.7/8.12.7) with ESMTP id h1EKHZ3v029705; Fri, 14 Feb 2003 15:17:35 -0500 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: <200302141100.23529.wes@softweyr.com> References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <20030213174531.GZ83215@roark.gnf.org> <20030213184309.GA53098@melusine.cuivre.fr.eu.org> <200302141100.23529.wes@softweyr.com> Date: Fri, 14 Feb 2003 15:17:33 -0500 To: Wes Peters , Thomas Quinot , Gordon Tetlow From: Garance A Drosihn Subject: Re: syslog.conf syntax change (multiple program/host specifications) Cc: Thomas Quinot , arch@FreeBSD.ORG Content-Type: text/plain; charset="iso-8859-1" ; format="flowed" Content-Transfer-Encoding: quoted-printable X-RPI-Spam-Score: -1.6 () IN_REP_TO,QUOTED_EMAIL_TEXT,REFERENCES,SIGNATURE_SHORT_DENSE,SPAM_PHRASE_00_01 X-Scanned-By: MIMEDefang 2.28 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG At 11:00 AM -0800 2/14/03, Wes Peters wrote: >On Thursday 13 February 2003 10:43, Thomas Quinot wrote: >> Le 2003-02-13, Gordon Tetlow =E9crivait : >> > it should be relatively easy to implement a log roller (or steal one). >> >> Yes, much code for that could be lifted from newsyslog presumably. > >To this end I've implemented another feature, 'N' for newsyslog. >When the file size limit is reached, newsyslog is run with >the log filename as the only argument. The size limitation >in syslog.conf and newsyslog.conf should agree or you won't >get what you expect. > >Does this seem reasonable? > >Can anyone suggest a better syntax that is in keeping with the >current syslog.conf? I am *not* going to replace the syslog.conf >with a different format, only extend it to encompass this simple >functionality. I would not replace the historic syntax, but I would like to see something "more readable" for the new options. Instead of cool 1-letter codes, I think we could go with something a bit more descriptive. Something like 'max=3D256m', 'rotate=3Dnewsyslog' and 'rotate=3Drm'. Or I guess it should be something like 'rotate=3Dtrunc', so you don't have to deal with permissions on a newly-created log file. I'd also suggest that newsyslog be run with some (new?) option that says "rotate this file, even if you don't think it needs to be rotated". I'd expect newsyslog to know *how* to do the rotation in this case (bzip, gzip, or alternate file-naming conventions, and permissions on the newly-created log file). -- Garance Alistair Drosehn =3D gad@gilead.netel.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 14 12:22:50 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5649237B401 for ; Fri, 14 Feb 2003 12:22:49 -0800 (PST) Received: from Daffy.timing.com (daffy.timing.com [206.168.13.218]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9169A43FBD for ; Fri, 14 Feb 2003 12:22:48 -0800 (PST) (envelope-from ben@timing.com) Received: from piglet.timing.com (piglet.timing.com [206.168.13.178]) by Daffy.timing.com (8.11.3/8.11.3) with ESMTP id h1EKMmH47408; Fri, 14 Feb 2003 13:22:48 -0700 (MST) (envelope-from ben@timing.com) Received: from piglet.timing.com (localhost.timing.com [127.0.0.1]) by piglet.timing.com (8.12.6/8.12.6) with ESMTP id h1EKMlXl065460; Fri, 14 Feb 2003 13:22:47 -0700 (MST) (envelope-from ben@piglet.timing.com) Received: (from ben@localhost) by piglet.timing.com (8.12.6/8.12.6/Submit) id h1EKMlWb065457; Fri, 14 Feb 2003 13:22:47 -0700 (MST) From: Ben Mesander MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15949.20503.795005.19315@piglet.timing.com> Date: Fri, 14 Feb 2003 13:22:47 -0700 To: Wes Peters , arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) In-Reply-To: References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <20030213174531.GZ83215@roark.gnf.org> <20030213184309.GA53098@melusine.cuivre.fr.eu.org> <200302141100.23529.wes@softweyr.com> X-Mailer: VM 7.00 under Emacs 21.2.95.2 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Wes Peters writes: >Can anyone suggest a better syntax that is in keeping with the >current syslog.conf? I am *not* going to replace the syslog.conf >with a different format, only extend it to encompass this simple >functionality. I don't know if freebsd's syslogd insists on tabs and not spaces in its config file (most seem to), but if so, it'd be nice if it was equally happy treating both as whitespace. --Ben To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 14 14: 1:51 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AA22C37B401 for ; Fri, 14 Feb 2003 14:01:49 -0800 (PST) Received: from ns2.gnf.org (ns2.gnf.org [63.196.132.68]) by mx1.FreeBSD.org (Postfix) with ESMTP id E9E9343FB1 for ; Fri, 14 Feb 2003 14:01:46 -0800 (PST) (envelope-from gtetlow@gnf.org) Received: from EXCHCLUSTER01.lj.gnf.org (exch01.lj.gnf.org [172.25.10.19]) by ns2.gnf.org (8.12.3/8.12.3) with ESMTP id h1EM1itk019360 for ; Fri, 14 Feb 2003 14:01:44 -0800 (PST) (envelope-from gtetlow@gnf.org) Received: from roark.gnf.org ([172.25.24.15]) by EXCHCLUSTER01.lj.gnf.org with Microsoft SMTPSVC(5.0.2195.5329); Fri, 14 Feb 2003 14:01:46 -0800 Received: from roark.gnf.org (localhost [127.0.0.1]) by roark.gnf.org (8.12.6/8.12.6) with ESMTP id h1EM1kGj071946; Fri, 14 Feb 2003 14:01:46 -0800 (PST) (envelope-from gtetlow@gnf.org) Received: (from gtetlow@localhost) by roark.gnf.org (8.12.6/8.12.6/Submit) id h1EM1j92071945; Fri, 14 Feb 2003 14:01:45 -0800 (PST) Date: Fri, 14 Feb 2003 14:01:45 -0800 From: Gordon Tetlow To: Garance A Drosihn Cc: arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) Message-ID: <20030214220145.GM83215@roark.gnf.org> References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <20030213174531.GZ83215@roark.gnf.org> <20030213184309.GA53098@melusine.cuivre.fr.eu.org> <200302141100.23529.wes@softweyr.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Ws+C+3cFyVqcopd2" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4i X-OriginalArrivalTime: 14 Feb 2003 22:01:46.0662 (UTC) FILETIME=[AACC8460:01C2D474] Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --Ws+C+3cFyVqcopd2 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Feb 14, 2003 at 03:17:33PM -0500, Garance A Drosihn wrote: >=20 > I'd also suggest that newsyslog be run with some (new?) option > that says "rotate this file, even if you don't think it needs > to be rotated". I'd expect newsyslog to know *how* to do the > rotation in this case (bzip, gzip, or alternate file-naming > conventions, and permissions on the newly-created log file). =46rom newsyslog(8): -F Force newsyslog to trim the logs, even if the trim conditions have not been met. This option is useful for diagnosing system problems by providing you with fresh logs that contain only the problems. -gordon --Ws+C+3cFyVqcopd2 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (FreeBSD) iD8DBQE+TWdJRu2t9DV9ZfsRAiS3AJ0Z8HIOD3YbPyQsE9+Be/T0xU7WlgCeMQ+a qeExe6DCkTqybh6nUwK5pDA= =L3H1 -----END PGP SIGNATURE----- --Ws+C+3cFyVqcopd2-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 14 15:16:49 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C376737B401 for ; Fri, 14 Feb 2003 15:16:48 -0800 (PST) Received: from heron.mail.pas.earthlink.net (heron.mail.pas.earthlink.net [207.217.120.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2E4CB43F3F for ; Fri, 14 Feb 2003 15:16:48 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from dialup-209.247.143.154.dial1.sanjose1.level3.net ([209.247.143.154] helo=mindspring.com) by heron.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18jp4X-0001VL-00; Fri, 14 Feb 2003 15:16:46 -0800 Message-ID: <3E4D788A.19F3789@mindspring.com> Date: Fri, 14 Feb 2003 15:15:22 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Ben Mesander Cc: Wes Peters , arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/hostspecifications) References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <20030213174531.GZ83215@roark.gnf.org> <20030213184309.GA53098@melusine.cuivre.fr.eu.org> <200302141100.23529.wes@softweyr.com> <15949.20503.795005.19315@piglet.timing.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a42f662a1959d78bcfa68f58110b06891d350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Ben Mesander wrote: > I don't know if freebsd's syslogd insists on tabs and not spaces in > its config file (most seem to), but if so, it'd be nice if it was > equally happy treating both as whitespace. To let you put spaces in file names, without having to resort to quote processing (" or \). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 14 15:32:51 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0F31437B401 for ; Fri, 14 Feb 2003 15:32:49 -0800 (PST) Received: from heron.mail.pas.earthlink.net (heron.mail.pas.earthlink.net [207.217.120.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6F14143FA3 for ; Fri, 14 Feb 2003 15:32:48 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from dialup-209.247.143.154.dial1.sanjose1.level3.net ([209.247.143.154] helo=mindspring.com) by heron.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18jpJk-0003nH-00; Fri, 14 Feb 2003 15:32:29 -0800 Message-ID: <3E4D7C2B.DDFC9DBE@mindspring.com> Date: Fri, 14 Feb 2003 15:30:51 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Gordon Tetlow Cc: Garance A Drosihn , arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <20030213174531.GZ83215@roark.gnf.org> <20030213184309.GA53098@melusine.cuivre.fr.eu.org> <200302141100.23529.wes@softweyr.com> <20030214220145.GM83215@roark.gnf.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a416ffea56bbeca56be0f7df2afe0ba4cf667c3043c0873f7e350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Gordon Tetlow wrote: > From newsyslog(8): > > -F Force newsyslog to trim the logs, even if the trim conditions > have not been met. This option is useful for diagnosing system > problems by providing you with fresh logs that contain only the > problems. Not really useful. For example, we had a system where cron would occasionally die, because it was FreeBSD 3.x based, and there was a bug in the copy-on-write code in the FreeBSD VM for pages that were mapped COW and read-only. The problem showed up occasionally, when Vixie cron would modify the contents of the pwent struct, thus assuming that the values pointed to by the struct elements were static data areas, rather than pointers into a Berkeley db record that was mmap'ed with these attributes out of the passwd db file, instead. The result was that it would write a page of password db file entry over top of the cron tab, which was open at the time. So then cron would die. The reason this is interesting for a discussion of newsyslog is that newsyslog was run from cron. The upshot of this is that you'd get a box that seemed OK, but on which newsyslog would not be run for a very long time. This resulted in very large log files. The "fix", which wasn't one, was to regenerate the crontab from a template and stored data, shen you rebooted the system. Then you would reboot, the crontab would be regenerated, and cron would run normally again, running newsyslog. Only newsyslog is stupid. No matter what options you gave it, the first thing it would do is the moral equivalent of -F. So instead of a 60M Samba log file "/var/log/samba", you ended up with a "/var/log/samba.1" that was 60M, and a "/var/log/samba" that was empty. As you can see, this leaves /var full, because it did not "rewrite history", and break the log file up into pieces the size of the retained log, according to newsyslog's config, *as if newsyslog had run normally*. E.g., if I keep 40K per log for 5 versions, then: old new expected (rewrite) samba 60M 0K 0K samba.1 41K 60M 40K samba.2 40K 41K 40K samba.3 43K 40K 40K samba.4 47K 43K 40K samba.5 42K 47K 40K total: 60.3M 60.2M 120K The result was a machine that turns into a doorstop, and then stays a doorstop. I would really object to rolling this same thing into syslog, without fixing this first. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 14 17:25: 3 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E04BA37B401 for ; Fri, 14 Feb 2003 17:25:01 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id 785FC43FA3 for ; Fri, 14 Feb 2003 17:25:01 -0800 (PST) (envelope-from wes@softweyr.com) Received: from salty.rapid.stbernard.com (corp-2.ipinc.com [199.245.188.2]) by smtp-relay.omnis.com (Postfix) with ESMTP id BFC7642DA6; Fri, 14 Feb 2003 17:25:00 -0800 (PST) From: Wes Peters Organization: Softweyr.com To: Terry Lambert , Gordon Tetlow Subject: Re: syslog.conf syntax change (multiple program/host specifications) Date: Fri, 14 Feb 2003 17:25:00 -0800 User-Agent: KMail/1.5 Cc: Garance A Drosihn , arch@FreeBSD.ORG References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <20030214220145.GM83215@roark.gnf.org> <3E4D7C2B.DDFC9DBE@mindspring.com> In-Reply-To: <3E4D7C2B.DDFC9DBE@mindspring.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200302141725.00421.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Friday 14 February 2003 15:30, Terry Lambert wrote: > > Only newsyslog is stupid. > > No matter what options you gave it, the first thing it would do is > the moral equivalent of -F. > > So instead of a 60M Samba log file "/var/log/samba", you ended up > with a "/var/log/samba.1" that was 60M, and a "/var/log/samba" > that was empty. The real problem we're looking at is that ftruncate(2) doesn't take a third argument describing which end you want to truncate at. Such a facility would end this argument quickly, syslog could simply truncate the first (filesize - desired log size) bytes of the file and be done with it. Short of getting ufs2 to provide frag blocks at both ends of the file, this is about as good as it gets. I'll get my -current system up this weekend and have a patch to review in a few days. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 14 17:34: 5 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A9FF937B401; Fri, 14 Feb 2003 17:34:03 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id 19E5643FA3; Fri, 14 Feb 2003 17:34:03 -0800 (PST) (envelope-from wes@softweyr.com) Received: from salty.rapid.stbernard.com (corp-2.ipinc.com [199.245.188.2]) by smtp-relay.omnis.com (Postfix) with ESMTP id A897C433F5; Fri, 14 Feb 2003 17:33:29 -0800 (PST) From: Wes Peters Organization: Softweyr.com To: Garance A Drosihn , Thomas Quinot , Gordon Tetlow Subject: Re: syslog.conf syntax change (multiple program/host specifications) Date: Fri, 14 Feb 2003 17:33:29 -0800 User-Agent: KMail/1.5 Cc: Thomas Quinot , arch@FreeBSD.ORG References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302141100.23529.wes@softweyr.com> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200302141733.29304.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Friday 14 February 2003 12:17, Garance A Drosihn wrote: > > I would not replace the historic syntax, but I would like to see > something "more readable" for the new options. Instead of cool > 1-letter codes, I think we could go with something a bit more > descriptive. OK, I'm cool with this. > Something like 'max=256m', 'rotate=newsyslog' and 'rotate=rm'. > Or I guess it should be something like 'rotate=trunc', so you > don't have to deal with permissions on a newly-created log file. Keywords are easier to parse than name=value pairs in this case. How 'bout: lpr.info /var/log/lpd-errs 256K local3.* /var/log/foo new 1K ftp.* /var/log/ftp bzip 1M cron.* /var/log/cron rm > I'd also suggest that newsyslog be run with some (new?) option > that says "rotate this file, even if you don't think it needs > to be rotated". newsyslog doesn't really seen to have this feature; -F just truncates the file. I'll look into a proper "force" flag for newsyslog. > I'd expect newsyslog to know *how* to do the > rotation in this case (bzip, gzip, or alternate file-naming > conventions, and permissions on the newly-created log file). What should it do if told to newsyslog a file it is not configured to handle? -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 14 17:34:34 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AAD8D37B401 for ; Fri, 14 Feb 2003 17:34:33 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id E85B843F85 for ; Fri, 14 Feb 2003 17:34:32 -0800 (PST) (envelope-from wes@softweyr.com) Received: from salty.rapid.stbernard.com (corp-2.ipinc.com [199.245.188.2]) by smtp-relay.omnis.com (Postfix) with ESMTP id 5266442EE7; Fri, 14 Feb 2003 17:34:17 -0800 (PST) From: Wes Peters Organization: Softweyr.com To: Terry Lambert , Ben Mesander Subject: Re: syslog.conf syntax change (multiple program/hostspecifications) Date: Fri, 14 Feb 2003 17:34:17 -0800 User-Agent: KMail/1.5 Cc: arch@FreeBSD.ORG References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <15949.20503.795005.19315@piglet.timing.com> <3E4D788A.19F3789@mindspring.com> In-Reply-To: <3E4D788A.19F3789@mindspring.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200302141734.17039.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Friday 14 February 2003 15:15, Terry Lambert wrote: > Ben Mesander wrote: > > I don't know if freebsd's syslogd insists on tabs and not spaces in > > its config file (most seem to), but if so, it'd be nice if it was > > equally happy treating both as whitespace. > > To let you put spaces in file names, without having to resort to > quote processing (" or \). FreeBSD allows either but notes your syslog.conf files might be incompatible with other systems. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 14 19:44:51 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5364F37B401 for ; Fri, 14 Feb 2003 19:44:49 -0800 (PST) Received: from smtp1.server.rpi.edu (smtp1.server.rpi.edu [128.113.2.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6239A43FA3 for ; Fri, 14 Feb 2003 19:44:48 -0800 (PST) (envelope-from drosih@rpi.edu) Received: from [128.113.24.47] (gilead.netel.rpi.edu [128.113.24.47]) by smtp1.server.rpi.edu (8.12.7/8.12.7) with ESMTP id h1F3ikF4008768; Fri, 14 Feb 2003 22:44:46 -0500 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: <3E4D7C2B.DDFC9DBE@mindspring.com> References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <20030213174531.GZ83215@roark.gnf.org> <20030213184309.GA53098@melusine.cuivre.fr.eu.org> <200302141100.23529.wes@softweyr.com> <20030214220145.GM83215@roark.gnf.org> <3E4D7C2B.DDFC9DBE@mindspring.com> Date: Fri, 14 Feb 2003 22:44:45 -0500 To: Terry Lambert , Gordon Tetlow From: Garance A Drosihn Subject: Re: syslog.conf syntax change (multiple program/host specifications) Cc: arch@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-RPI-Spam-Score: -1.9 () IN_REP_TO,QUOTED_EMAIL_TEXT,REFERENCES,SIGNATURE_SHORT_DENSE,SPAM_PHRASE_01_02 X-Scanned-By: MIMEDefang 2.28 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG At 3:30 PM -0800 2/14/03, Terry Lambert wrote: >Gordon Tetlow wrote: >> From newsyslog(8): > > > > -F Force newsyslog to trim the logs, even if the trim > > conditions have not been met. This option is useful > > for diagnosing system problems by providing you with > > fresh logs that contain only the problems. > >Not really useful. > >For example, we had a system where cron would occasionally die, Note that what we are saying is that syslog would execute newsyslog, at the moment syslog notes the logfile has grown too large. Newsyslog would be told to "rotate this file now". So, it wouldn't matter if cron dies. And the log would be rotated right after the first syslog write which caused the file to pass the size limit. >Only newsyslog is stupid. > >No matter what options you gave it, the first thing it would do is >the moral equivalent of -F. > >So instead of a 60M Samba log file "/var/log/samba", you ended up >with a "/var/log/samba.1" that was 60M, and a "/var/log/samba" >that was empty. I still have some message squirrelled away where you describe this problem. I do agree this would be good to fix. It seems to me that would be separate from the Wes's syslog change. Ie, it's good to fix this whether or not Wes's change goes in, and it's good for Wes's change to go in whether or not this is fixed. >I would really object to rolling this same thing into syslog, >without fixing this first. Some people talked about (basically) moving all of newsyslog functions into syslog, but I don't think we are heading that way. We're now just talking about having syslog run newsyslog on a specific logfile. -- Garance Alistair Drosehn = gad@gilead.netel.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 14 19:55:56 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6CF4237B401 for ; Fri, 14 Feb 2003 19:55:54 -0800 (PST) Received: from puffin.mail.pas.earthlink.net (puffin.mail.pas.earthlink.net [207.217.120.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id A7A4043FB1 for ; Fri, 14 Feb 2003 19:55:53 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0018.cvx21-bradley.dialup.earthlink.net ([209.179.192.18] helo=mindspring.com) by puffin.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18jtQZ-0004cf-00; Fri, 14 Feb 2003 19:55:48 -0800 Message-ID: <3E4DB9F1.83B7746D@mindspring.com> Date: Fri, 14 Feb 2003 19:54:25 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Wes Peters Cc: Gordon Tetlow , Garance A Drosihn , arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <20030214220145.GM83215@roark.gnf.org> <3E4D7C2B.DDFC9DBE@mindspring.com> <200302141725.00421.wes@softweyr.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4713b82fa6aa363421933c9ba2b6fea3c3ca473d225a0f487350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Wes Peters wrote: > On Friday 14 February 2003 15:30, Terry Lambert wrote: > > Only newsyslog is stupid. > > > > No matter what options you gave it, the first thing it would do is > > the moral equivalent of -F. > > > > So instead of a 60M Samba log file "/var/log/samba", you ended up > > with a "/var/log/samba.1" that was 60M, and a "/var/log/samba" > > that was empty. > > The real problem we're looking at is that ftruncate(2) doesn't take a > third argument describing which end you want to truncate at. Such a > facility would end this argument quickly, syslog could simply truncate > the first (filesize - desired log size) bytes of the file and be done > with it. This would solve the immediate "file too large" problem, but it wouldn't maintain the history properly; you'd end up with 0, 40K, a 60M-40K gap in the history, and then realy old 40K x 3. Probably the right interface is "donate this block range to file X starting at block offset Q, zeroing the corresponding blocks in this file, sparsely, if you can". It's a twisty little problem, probably not worth a general API, unless you can know the block granularity on the underlying FS, at which sparse blocks existed, and constrain it to use that. BTW: the SVR4 fcntl() that's the moral equivalent of ftruncate supports exactly the syntax you want (e.g. free up space at the start of the file for 60M-120K, replacing with sparseness -- it can put sparse blocks in the middle of files). What might be more useful, in this particular case, since you want only the last 120K is a means of truncating deleting cytes off the front of a file. You'd probably want to allow any value, but in terms of avoiding rewriting, in practice, you probably want to constrain it to an even multiple of the FS block size. > Short of getting ufs2 to provide frag blocks at both ends of the file, > this is about as good as it gets. I'll get my -current system up this > weekend and have a patch to review in a few days. If you do this, just implement the SVR4 fcntl(); it basically takes an fcntl() lock structure, e.g.: fcntl( fd, F_FREESP, struct flock *) l_whence SEEK_SET/SEEK_CUR/SEEK_END l_start l_len FWIW, the manual allows restriction to the end of the file, but in practice, that's not what happens on SVR4 systems. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 14 19:58:17 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6419A37B401 for ; Fri, 14 Feb 2003 19:58:16 -0800 (PST) Received: from puffin.mail.pas.earthlink.net (puffin.mail.pas.earthlink.net [207.217.120.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id E019543FAF for ; Fri, 14 Feb 2003 19:58:15 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0018.cvx21-bradley.dialup.earthlink.net ([209.179.192.18] helo=mindspring.com) by puffin.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18jtSv-0004qi-00; Fri, 14 Feb 2003 19:58:14 -0800 Message-ID: <3E4DBA84.2A838792@mindspring.com> Date: Fri, 14 Feb 2003 19:56:52 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Garance A Drosihn Cc: Gordon Tetlow , arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/hostspecifications) References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <20030213174531.GZ83215@roark.gnf.org> <20030213184309.GA53098@melusine.cuivre.fr.eu.org> <200302141100.23529.wes@softweyr.com> <20030214220145.GM83215@roark.gnf.org> <3E4D7C2B.DDFC9DBE@mindspring.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4713b82fa6aa363426f7209b5f8817690548b785378294e88350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Garance A Drosihn wrote: > Some people talked about (basically) moving all of newsyslog > functions into syslog, but I don't think we are heading that > way. We're now just talking about having syslog run newsyslog > on a specific logfile. In that case, I withdraw the objection; my objection was to pulling the (broken, IMO) newsyslog code into syslog sources. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 14 20: 4:16 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B58D837B401; Fri, 14 Feb 2003 20:04:14 -0800 (PST) Received: from smtp2.server.rpi.edu (smtp2.server.rpi.edu [128.113.2.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id E535E43F93; Fri, 14 Feb 2003 20:04:13 -0800 (PST) (envelope-from drosih@rpi.edu) Received: from [128.113.24.47] (gilead.netel.rpi.edu [128.113.24.47]) by smtp2.server.rpi.edu (8.12.7/8.12.7) with ESMTP id h1F44CeY006563; Fri, 14 Feb 2003 23:04:12 -0500 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: <200302141733.29304.wes@softweyr.com> References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302141100.23529.wes@softweyr.com> <200302141733.29304.wes@softweyr.com> Date: Fri, 14 Feb 2003 23:04:11 -0500 To: Wes Peters , Thomas Quinot , Gordon Tetlow From: Garance A Drosihn Subject: Re: syslog.conf syntax change (multiple program/host specifications) Cc: Thomas Quinot , arch@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-RPI-Spam-Score: -1.9 () IN_REP_TO,QUOTED_EMAIL_TEXT,REFERENCES,SIGNATURE_SHORT_DENSE,SPAM_PHRASE_01_02 X-Scanned-By: MIMEDefang 2.28 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG At 5:33 PM -0800 2/14/03, Wes Peters wrote: >On Friday 14 February 2003 12:17, Garance A Drosihn wrote: > > Something like 'max=256m', 'rotate=newsyslog' and 'rotate=rm'. >> Or I guess it should be something like 'rotate=trunc', so you >> don't have to deal with permissions on a newly-created log file. > >Keywords are easier to parse than name=value pairs in this case. >How 'bout: > >lpr.info /var/log/lpd-errs 256K >local3.* /var/log/foo new 1K >ftp.* /var/log/ftp bzip 1M >cron.* /var/log/cron rm Well, how about keep the size specification as you have it, but have keywords which just happen to include '=' sign in them? 'atfull=newsyslog', 'atfull=bzip', 'atfull=rm'. Still use a simple strncmp() to do the check, but something that would be a bit more obvious to someone seeing it (in the syslog.conf file) for the first time. Also, that way it's less likely to conflict with any other cool ideas that some other OS adds to syslog.conf. I'm not thrilled with 'atfull'. Maybe 'rotate' or something. > > I'd also suggest that newsyslog be run with some (new?) option >> that says "rotate this file, even if you don't think it needs >> to be rotated". > >newsyslog doesn't really seen to have this feature; -F just >truncates the file. I'll look into a proper "force" flag for >newsyslog. Hmm. Well, I should do something more than just annoy you with requests of things to do. I have some other stuff to do this weekend (stuff for my real job), but I'll see if I can come up with some update for the newsyslog side of things. > > I'd expect newsyslog to know *how* to do the >> rotation in this case (bzip, gzip, or alternate file-naming >> conventions, and permissions on the newly-created log file). > >What should it do if told to newsyslog a file it is not >configured to handle? Some simple default action. "just rotate, keep 2 backups", or something. -- Garance Alistair Drosehn = gad@gilead.netel.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 14 22:16:20 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 09F2E37B401 for ; Fri, 14 Feb 2003 22:16:19 -0800 (PST) Received: from cirb503493.alcatel.com.au (c18609.belrs1.nsw.optusnet.com.au [210.49.80.204]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3FF5E43FA3 for ; Fri, 14 Feb 2003 22:16:17 -0800 (PST) (envelope-from peterjeremy@optushome.com.au) Received: from cirb503493.alcatel.com.au (localhost.alcatel.com.au [127.0.0.1]) by cirb503493.alcatel.com.au (8.12.5/8.12.5) with ESMTP id h1F6GELZ060438; Sat, 15 Feb 2003 17:16:15 +1100 (EST) (envelope-from jeremyp@cirb503493.alcatel.com.au) Received: (from jeremyp@localhost) by cirb503493.alcatel.com.au (8.12.6/8.12.5/Submit) id h1F6GC1Z060437; Sat, 15 Feb 2003 17:16:13 +1100 (EST) Date: Sat, 15 Feb 2003 17:16:12 +1100 From: Peter Jeremy To: Terry Lambert Cc: arch@FreeBSD.ORG Subject: Controlling spaseness (was Re: syslog.conf syntax change ...) Message-ID: <20030215061612.GA60369@cirb503493.alcatel.com.au> References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <20030214220145.GM83215@roark.gnf.org> <3E4D7C2B.DDFC9DBE@mindspring.com> <200302141725.00421.wes@softweyr.com> <3E4DB9F1.83B7746D@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3E4DB9F1.83B7746D@mindspring.com> User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, Feb 14, 2003 at 07:54:25PM -0800, Terry Lambert wrote: >BTW: the SVR4 fcntl() that's the moral equivalent of ftruncate >supports exactly the syntax you want (e.g. free up space at the >start of the file for 60M-120K, replacing with sparseness -- it >can put sparse blocks in the middle of files). ... >If you do this, just implement the SVR4 fcntl(); it basically takes >an fcntl() lock structure, e.g.: > > > fcntl( fd, F_FREESP, struct flock *) > > l_whence SEEK_SET/SEEK_CUR/SEEK_END > l_start > l_len This triggered a line of thought: One problem for archivers that access through the filesystem is that they are unable to differentiate between a block of NULs and a sparse block. The best they can do is guess. Dump(8) avoids the problem by reading the underlying device - but this means it has to re-implement UFS (at least the read-only bits). A reasonably simple solution to this would be an fcntl(2) option that returned a bitmap of allocated blocks: fcntl(fd, F_GETBITMAP, struct bitmap *); struct bitmap { size_t bm_len; /* size of bm_bitmap - I (O?) */ u_int32_t bm_blksiz; /* blocksize of bitmap - O */ off_t bm_filsiz; /* file size in bytes - O */ char bm_bitmap[]; /* bm_len bytes - O */ } bm_bitmap could be a pointer to the bitmap rather than using a variable length struct. bm_filsiz allows you to determine whether the final block is a fragment (and the size thereof). A logical extension is then to provide a F_SETBITMAP function that alters the block allocation to match the passed bitmap - either releasing blocks or filling in the holes as requested. This makes it a superset of ftruncate(). Both these functions seem fairly trivial to implement - at least for UFS. I'm not sure which other local filesystems support sparse files or whether NFS gives the client access to sparseness information. Peter To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 14 22:24:26 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 28D6337B401 for ; Fri, 14 Feb 2003 22:24:25 -0800 (PST) Received: from cirb503493.alcatel.com.au (c18609.belrs1.nsw.optusnet.com.au [210.49.80.204]) by mx1.FreeBSD.org (Postfix) with ESMTP id 10CA143FBF for ; Fri, 14 Feb 2003 22:24:24 -0800 (PST) (envelope-from peterjeremy@optushome.com.au) Received: from cirb503493.alcatel.com.au (localhost.alcatel.com.au [127.0.0.1]) by cirb503493.alcatel.com.au (8.12.5/8.12.5) with ESMTP id h1F6OGLZ060454; Sat, 15 Feb 2003 17:24:20 +1100 (EST) (envelope-from jeremyp@cirb503493.alcatel.com.au) Received: (from jeremyp@localhost) by cirb503493.alcatel.com.au (8.12.6/8.12.5/Submit) id h1F6OA7D060451; Sat, 15 Feb 2003 17:24:10 +1100 (EST) Date: Sat, 15 Feb 2003 17:24:10 +1100 From: Peter Jeremy To: Terry Lambert Cc: arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) Message-ID: <20030215062410.GB60369@cirb503493.alcatel.com.au> References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <20030213174531.GZ83215@roark.gnf.org> <20030213184309.GA53098@melusine.cuivre.fr.eu.org> <200302141100.23529.wes@softweyr.com> <20030214220145.GM83215@roark.gnf.org> <3E4D7C2B.DDFC9DBE@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3E4D7C2B.DDFC9DBE@mindspring.com> User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, Feb 14, 2003 at 03:30:51PM -0800, Terry Lambert wrote: >Only newsyslog is stupid. > >No matter what options you gave it, the first thing it would do is >the moral equivalent of -F. > >So instead of a 60M Samba log file "/var/log/samba", you ended up >with a "/var/log/samba.1" that was 60M, and a "/var/log/samba" >that was empty. I'm not sure this is "stupid" in all cases. Definitely, if you have the situation where newsyslog fails to run for an extended period, this is a problem. OTOH, if syslog is running normally and there is a massive burst of log activity (eg an attack) then you could lose older logs. This might make it easier for an attacker to destroy evidence of what they did - you know something happened because you have a pile of syslogs full of rubbish, but you don't know exactly what because the earliest syslogs have rotated out of existence. Peter To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Feb 14 22:48:17 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C09F237B401 for ; Fri, 14 Feb 2003 22:48:15 -0800 (PST) Received: from smtp4.server.rpi.edu (smtp4.server.rpi.edu [128.113.2.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id E6B2643F3F for ; Fri, 14 Feb 2003 22:48:14 -0800 (PST) (envelope-from drosih@rpi.edu) Received: from [128.113.24.47] (gilead.netel.rpi.edu [128.113.24.47]) by smtp4.server.rpi.edu (8.12.7/8.12.7) with ESMTP id h1F6mD3v032539; Sat, 15 Feb 2003 01:48:13 -0500 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: <200302141733.29304.wes@softweyr.com> References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302141100.23529.wes@softweyr.com> <200302141733.29304.wes@softweyr.com> Date: Sat, 15 Feb 2003 01:48:12 -0500 To: Wes Peters From: Garance A Drosihn Subject: Re: syslog.conf syntax change (multiple program/host specifications) Cc: arch@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-RPI-Spam-Score: -1.6 () IN_REP_TO,QUOTED_EMAIL_TEXT,REFERENCES,SIGNATURE_SHORT_DENSE,SPAM_PHRASE_00_01 X-Scanned-By: MIMEDefang 2.28 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG At 5:33 PM -0800 2/14/03, Wes Peters wrote: >Keywords are easier to parse than name=value pairs in this case. >How 'bout: > >lpr.info /var/log/lpd-errs 256K >local3.* /var/log/foo new 1K >ftp.* /var/log/ftp bzip 1M >cron.* /var/log/cron rm Hmm. If syslogd can call newsyslog, does it need bzip/gzip options? [I'm just wondering, I have no preference either way] > > I'd also suggest that newsyslog be run with some (new?) option >> that says "rotate this file, even if you don't think it needs >> to be rotated". > >newsyslog doesn't really seen to have this feature; -F just >truncates the file. I'll look into a proper "force" flag >for newsyslog. Based on a few minutes of testing, I think newsyslog will pretty much do the right thing if you call it as: newsyslog -Fr /var/log/somefilename The '-r' is just so newsyslog doesn't turn around and send a signal back to syslogd. I'm still tempted to add a '-R'. > > I'd expect newsyslog to know *how* to do the >> rotation in this case (bzip, gzip, or alternate file-naming >> conventions, and permissions on the newly-created log file). > >What should it do if told to newsyslog a file it is not >configured to handle? I would add some default rotate-action to newsyslog, which would be used if -R is specified and the file is not listed in the newsyslog.conf file. I think we'd then want some way for the user to change that default rotate-action. Probably by something like adding a line where the filename is . -- Garance Alistair Drosehn = gad@gilead.netel.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Feb 15 1: 5:34 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3663637B401 for ; Sat, 15 Feb 2003 01:05:32 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id BE32943FA3 for ; Sat, 15 Feb 2003 01:05:31 -0800 (PST) (envelope-from wes@softweyr.com) Received: from softweyr.homeunix.net (66-75-151-22.san.rr.com [66.75.151.22]) by smtp-relay.omnis.com (Postfix) with ESMTP id AE5D242F36; Sat, 15 Feb 2003 01:05:08 -0800 (PST) From: Wes Peters Organization: Softweyr To: Garance A Drosihn Subject: Re: syslog.conf syntax change (multiple program/host specifications) Date: Sat, 15 Feb 2003 09:05:08 +0000 User-Agent: KMail/1.5 Cc: arch@FreeBSD.ORG References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302141733.29304.wes@softweyr.com> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200302150905.08387.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Saturday 15 February 2003 06:48, Garance A Drosihn wrote: > At 5:33 PM -0800 2/14/03, Wes Peters wrote: > >Keywords are easier to parse than name=value pairs in this case. > >How 'bout: > > > >lpr.info /var/log/lpd-errs 256K > >local3.* /var/log/foo new 1K > >ftp.* /var/log/ftp bzip 1M > >cron.* /var/log/cron rm > > Hmm. If syslogd can call newsyslog, does it need bzip/gzip options? > [I'm just wondering, I have no preference either way] Yes. The intention of this is to benefit embedded systems, and some of them might want to shed newsyslog completely. We're not making that leap because we use newsyslog to rotate other files too, but I can see where others might want the feature. The code is tiny, it adds a few hundred bytes to the text. > > > I'd also suggest that newsyslog be run with some (new?) option > >> > >> that says "rotate this file, even if you don't think it needs > >> to be rotated". > > > >newsyslog doesn't really seen to have this feature; -F just > >truncates the file. I'll look into a proper "force" flag > >for newsyslog. > > Based on a few minutes of testing, I think newsyslog will pretty > much do the right thing if you call it as: > > newsyslog -Fr /var/log/somefilename > > The '-r' is just so newsyslog doesn't turn around and send a > signal back to syslogd. I'm still tempted to add a '-R'. I'll give that a try Tuesday, or maybe Monday if I get my current box running this weekend. > > > I'd expect newsyslog to know *how* to do the > >> > >> rotation in this case (bzip, gzip, or alternate file-naming > >> conventions, and permissions on the newly-created log file). > > > >What should it do if told to newsyslog a file it is not > >configured to handle? > > I would add some default rotate-action to newsyslog, which > would be used if -R is specified and the file is not listed > in the newsyslog.conf file. Sounds good to me. Are you going to look into that? I'll definitely want your changes to newsyslog to go along with my changes to syslog.conf. ;^) -- Where am I, and what am I doing in this handbasket? Wes Peters wes@softweyr.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Feb 15 1: 9:41 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D92C337B401 for ; Sat, 15 Feb 2003 01:09:39 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6635D43FB1 for ; Sat, 15 Feb 2003 01:09:39 -0800 (PST) (envelope-from wes@softweyr.com) Received: from softweyr.homeunix.net (66-75-151-22.san.rr.com [66.75.151.22]) by smtp-relay.omnis.com (Postfix) with ESMTP id 5A56A43028; Sat, 15 Feb 2003 01:09:38 -0800 (PST) From: Wes Peters Organization: Softweyr To: Garance A Drosihn , Terry Lambert , Gordon Tetlow Subject: Re: syslog.conf syntax change (multiple program/host specifications) Date: Sat, 15 Feb 2003 09:09:38 +0000 User-Agent: KMail/1.5 Cc: arch@FreeBSD.ORG References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <3E4D7C2B.DDFC9DBE@mindspring.com> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200302150909.38103.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Saturday 15 February 2003 03:44, Garance A Drosihn wrote: > At 3:30 PM -0800 2/14/03, Terry Lambert wrote: > >Gordon Tetlow wrote: > >> From newsyslog(8): > > > -F Force newsyslog to trim the logs, even if the trim > > > conditions have not been met. This option is useful > > > for diagnosing system problems by providing you with > > > fresh logs that contain only the problems. > > > >Not really useful. > > > >For example, we had a system where cron would occasionally die, > > Note that what we are saying is that syslog would execute > newsyslog, at the moment syslog notes the logfile has grown > too large. Newsyslog would be told to "rotate this file now". > So, it wouldn't matter if cron dies. And the log would be > rotated right after the first syslog write which caused the > file to pass the size limit. Yes, it's important to keep in mind the desired change that caused this discussion in the first place. ;^) > I still have some message squirrelled away where you describe > this problem. I do agree this would be good to fix. It seems > to me that would be separate from the Wes's syslog change. Ie, > it's good to fix this whether or not Wes's change goes in, and > it's good for Wes's change to go in whether or not this is fixed. Yes, there's a very loose coupling between the two. I suppose it would be best to fix that while we're dorking around with newsyslog, but the solutions are going after the same end via two different paths: the syslog.conf change prevents it from happening and the newsyslog change fixes it after it's happened and crashed things. > >I would really object to rolling this same thing into syslog, > >without fixing this first. > > Some people talked about (basically) moving all of newsyslog > functions into syslog, but I don't think we are heading that > way. We're now just talking about having syslog run newsyslog > on a specific logfile. Or some very simple rolling behavior. If you need the full function- ality of newsyslog, it awaits your log-rolling pleasure. If your needs (or capacity) are more limited, I'd like syslogd to handle that itself. -- Where am I, and what am I doing in this handbasket? Wes Peters wes@softweyr.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Feb 15 1:24:40 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AC78937B401 for ; Sat, 15 Feb 2003 01:24:37 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id 27F5443F3F for ; Sat, 15 Feb 2003 01:24:37 -0800 (PST) (envelope-from wes@softweyr.com) Received: from softweyr.homeunix.net (66-75-151-22.san.rr.com [66.75.151.22]) by smtp-relay.omnis.com (Postfix) with ESMTP id 2F45543491; Sat, 15 Feb 2003 01:18:10 -0800 (PST) From: Wes Peters Organization: Softweyr To: Terry Lambert Subject: Re: syslog.conf syntax change (multiple program/host specifications) Date: Sat, 15 Feb 2003 09:18:09 +0000 User-Agent: KMail/1.5 Cc: Gordon Tetlow , Garance A Drosihn , arch@FreeBSD.ORG References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302141725.00421.wes@softweyr.com> <3E4DB9F1.83B7746D@mindspring.com> In-Reply-To: <3E4DB9F1.83B7746D@mindspring.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200302150918.09807.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Saturday 15 February 2003 03:54, Terry Lambert wrote: > Wes Peters wrote: > > > > The real problem we're looking at is that ftruncate(2) doesn't take a > > third argument describing which end you want to truncate at. Such a > > facility would end this argument quickly, syslog could simply truncate > > the first (filesize - desired log size) bytes of the file and be done > > with it. > > This would solve the immediate "file too large" problem, but it > wouldn't maintain the history properly; you'd end up with 0, 40K, > a 60M-40K gap in the history, and then realy old 40K x 3. Probably > the right interface is "donate this block range to file X starting > at block offset Q, zeroing the corresponding blocks in this file, > sparsely, if you can". > > It's a twisty little problem, probably not worth a general API, > unless you can know the block granularity on the underlying FS, > at which sparse blocks existed, and constrain it to use that. I disagree, it's a problem I run into frequently and a general API would solve the entire class of problems once and for all. > BTW: the SVR4 fcntl() that's the moral equivalent of ftruncate > supports exactly the syntax you want (e.g. free up space at the > start of the file for 60M-120K, replacing with sparseness -- it > can put sparse blocks in the middle of files). Yes, I recall that fondly. It seems weird for BSD to have less functionality, especially in the filesystem. > What might be more useful, in this particular case, since you want > only the last 120K is a means of truncating deleting cytes off the > front of a file. You'd probably want to allow any value, but in > terms of avoiding rewriting, in practice, you probably want to > constrain it to an even multiple of the FS block size. Well, I certainly don't want any of that. I want to be able to lop the first 11 bytes of a 2^30 byte file and have exactly that data in the file the next time I access it. > > Short of getting ufs2 to provide frag blocks at both ends of the file, > > this is about as good as it gets. I'll get my -current system up this > > weekend and have a patch to review in a few days. > > If you do this, just implement the SVR4 fcntl(); it basically takes > an fcntl() lock structure, e.g.: > > > fcntl( fd, F_FREESP, struct flock *) > > l_whence SEEK_SET/SEEK_CUR/SEEK_END > l_start > l_len > > FWIW, the manual allows restriction to the end of the file, but in > practice, that's not what happens on SVR4 systems. Right, only I don't have anywhere near the filesystem-foo to implement such a change. I suppose it could be done relatively straightforward by allowing the original leading disk blocks to be marked unused and making an offset to the beginning of the file field in the inode, that counts bytes to skip into the first truly allocated block. It's easy to see why ufs doesn't have the functionality, but that doesn't keep me from wanting it... -- Where am I, and what am I doing in this handbasket? Wes Peters wes@softweyr.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Feb 15 3:53:53 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 52F2B37B401; Sat, 15 Feb 2003 03:53:52 -0800 (PST) Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id 062C543FA3; Sat, 15 Feb 2003 03:53:51 -0800 (PST) (envelope-from des@ofug.org) Received: by flood.ping.uio.no (Postfix, from userid 2602) id 1DD34536E; Sat, 15 Feb 2003 12:53:47 +0100 (CET) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Wes Peters Cc: Garance A Drosihn , Thomas Quinot , Gordon Tetlow , arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) From: Dag-Erling Smorgrav Date: Sat, 15 Feb 2003 12:53:47 +0100 In-Reply-To: <200302141733.29304.wes@softweyr.com> (Wes Peters's message of "Fri, 14 Feb 2003 17:33:29 -0800") Message-ID: User-Agent: Gnus/5.090014 (Oort Gnus v0.14) Emacs/21.2 (i386--freebsd) References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302141100.23529.wes@softweyr.com> <200302141733.29304.wes@softweyr.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Wes Peters writes: > newsyslog doesn't really seen to have this feature; -F just truncates > the file. I'll look into a proper "force" flag for newsyslog. No, it rotates the file. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Feb 15 3:56: 6 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0647A37B401; Sat, 15 Feb 2003 03:56:05 -0800 (PST) Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id 45C0143FAF; Sat, 15 Feb 2003 03:56:02 -0800 (PST) (envelope-from des@ofug.org) Received: by flood.ping.uio.no (Postfix, from userid 2602) id BB058536E; Sat, 15 Feb 2003 12:56:00 +0100 (CET) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Wes Peters Cc: Garance A Drosihn , Thomas Quinot , Gordon Tetlow , arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) From: Dag-Erling Smorgrav Date: Sat, 15 Feb 2003 12:55:59 +0100 In-Reply-To: (Dag-Erling Smorgrav's message of "Sat, 15 Feb 2003 12:53:47 +0100") Message-ID: User-Agent: Gnus/5.090014 (Oort Gnus v0.14) Emacs/21.2 (i386--freebsd) References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302141100.23529.wes@softweyr.com> <200302141733.29304.wes@softweyr.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Dag-Erling Smorgrav writes: > Wes Peters writes: > > newsyslog doesn't really seen to have this feature; -F just truncates > > the file. I'll look into a proper "force" flag for newsyslog. > No, it rotates the file. In case there's any doubt about this: root@des ~# tail /var/log/debug.log Feb 15 12:14:37 des su: in pam_set_item(): entering: PAM_AUTHTOK_PROMPT Feb 15 12:14:37 des su: in pam_set_item(): returning PAM_SUCCESS Feb 15 12:14:37 des su: in pam_set_item(): entering: PAM_OLDAUTHTOK_PROMPT Feb 15 12:14:37 des su: in pam_set_item(): returning PAM_SUCCESS Feb 15 12:14:37 des su: in pam_end(): returning PAM_SUCCESS Feb 15 12:23:01 des cvsupd[4949]: +3 root@dsa.thinksec.com [SNAP_16_1f/17.0] Feb 15 12:24:38 des cvsupd[4949]: =3 [2879Kin+257Kout] src-all/cvs Feb 15 12:24:38 des cvsupd[4949]: =3 [3306Kin+6Kout] ports-all/cvs Feb 15 12:24:38 des cvsupd[4949]: =3 [236Kin+14Kout] doc-all/cvs Feb 15 12:24:38 des cvsupd[4949]: -3 [6421Kin+276Kout] Finished successfully root@des ~# newsyslog -F /var/log/debug.log root@des ~# tail /var/log/debug.log Feb 15 12:54:35 des newsyslog[5029]: logfile turned over Feb 15 12:54:35 des syslogd: restart root@des ~# zcat /var/log/debug.log.0.gz | tail Feb 15 12:14:37 des su: in pam_set_item(): returning PAM_SUCCESS Feb 15 12:14:37 des su: in pam_set_item(): entering: PAM_OLDAUTHTOK_PROMPT Feb 15 12:14:37 des su: in pam_set_item(): returning PAM_SUCCESS Feb 15 12:14:37 des su: in pam_end(): returning PAM_SUCCESS Feb 15 12:23:01 des cvsupd[4949]: +3 root@dsa.thinksec.com [SNAP_16_1f/17.0] Feb 15 12:24:38 des cvsupd[4949]: =3 [2879Kin+257Kout] src-all/cvs Feb 15 12:24:38 des cvsupd[4949]: =3 [3306Kin+6Kout] ports-all/cvs Feb 15 12:24:38 des cvsupd[4949]: =3 [236Kin+14Kout] doc-all/cvs Feb 15 12:24:38 des cvsupd[4949]: -3 [6421Kin+276Kout] Finished successfully Feb 15 12:54:35 des newsyslog[5029]: logfile turned over DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Feb 15 9: 5:58 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3014D37B401; Sat, 15 Feb 2003 09:05:57 -0800 (PST) Received: from melusine.cuivre.fr.eu.org (melusine.cuivre.fr.eu.org [62.212.105.185]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6583A43F75; Sat, 15 Feb 2003 09:05:56 -0800 (PST) (envelope-from thomas@FreeBSD.ORG) Received: by melusine.cuivre.fr.eu.org (Postfix, from userid 1000) id 254E22C3D1; Sat, 15 Feb 2003 18:05:55 +0100 (CET) Date: Sat, 15 Feb 2003 18:05:55 +0100 From: Thomas Quinot To: Wes Peters Cc: Thomas Quinot , Gordon Tetlow , arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) Message-ID: <20030215170555.GB65740@melusine.cuivre.fr.eu.org> Reply-To: Thomas Quinot References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <20030213174531.GZ83215@roark.gnf.org> <20030213184309.GA53098@melusine.cuivre.fr.eu.org> <200302141100.23529.wes@softweyr.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <200302141100.23529.wes@softweyr.com> User-Agent: Mutt/1.4i X-message-flag: WARNING! Using Outlook can damage your computer. Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Le 2003-02-14, Wes Peters crivait : > To this end I've implemented another feature, 'N' for newsyslog. > When the file size limit is reached, newsyslog is run with the > log filename as the only argument. The size limitation in syslog.conf > and newsyslog.conf should agree or you won't get what you expect. Well, precisely for this reason it would seem even nicer to me to delegate the size limitation to newsyslog as well (perhaps rebuilding a tool similar to daemontool's multilog based on code shared with newsyslog). Thomas. -- Thomas.Quinot@Cuivre.FR.EU.ORG To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Feb 15 11:23:20 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CD99937B401; Sat, 15 Feb 2003 11:23:18 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5CD9F43F85; Sat, 15 Feb 2003 11:23:16 -0800 (PST) (envelope-from wes@softweyr.com) Received: from softweyr.homeunix.net (66-75-151-22.san.rr.com [66.75.151.22]) by smtp-relay.omnis.com (Postfix) with ESMTP id D1B5E431B9; Sat, 15 Feb 2003 11:23:15 -0800 (PST) From: Wes Peters Organization: Softweyr To: Thomas Quinot Subject: Re: syslog.conf syntax change (multiple program/host specifications) Date: Sat, 15 Feb 2003 19:23:14 +0000 User-Agent: KMail/1.5 Cc: Thomas Quinot , Gordon Tetlow , arch@FreeBSD.ORG References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302141100.23529.wes@softweyr.com> <20030215170555.GB65740@melusine.cuivre.fr.eu.org> In-Reply-To: <20030215170555.GB65740@melusine.cuivre.fr.eu.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Message-Id: <200302151923.14806.wes@softweyr.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Saturday 15 February 2003 17:05, Thomas Quinot wrote: > Le 2003-02-14, Wes Peters =E9crivait : > > To this end I've implemented another feature, 'N' for newsyslog. > > When the file size limit is reached, newsyslog is run with the > > log filename as the only argument. The size limitation in syslog.conf > > and newsyslog.conf should agree or you won't get what you expect. > > Well, precisely for this reason it would seem even nicer to me to > delegate the size limitation to newsyslog as well (perhaps rebuilding > a tool similar to daemontool's multilog based on code shared with > newsyslog). That's a better answer than incorporating multilog with all it's djb=20 licensing warts, but still costs another process for every log file you want to size-limit. Garance, did you get this one? Do you want to look at this? =2D-=20 Where am I, and what am I doing in this handbasket? Wes Peters wes@softweyr.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Feb 15 12:45:19 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3032D37B401 for ; Sat, 15 Feb 2003 12:45:18 -0800 (PST) Received: from cirb503493.alcatel.com.au (c18609.belrs1.nsw.optusnet.com.au [210.49.80.204]) by mx1.FreeBSD.org (Postfix) with ESMTP id ABF6D43F3F for ; Sat, 15 Feb 2003 12:45:16 -0800 (PST) (envelope-from peterjeremy@optushome.com.au) Received: from cirb503493.alcatel.com.au (localhost.alcatel.com.au [127.0.0.1]) by cirb503493.alcatel.com.au (8.12.5/8.12.5) with ESMTP id h1FKj7LZ061494; Sun, 16 Feb 2003 07:45:10 +1100 (EST) (envelope-from jeremyp@cirb503493.alcatel.com.au) Received: (from jeremyp@localhost) by cirb503493.alcatel.com.au (8.12.6/8.12.5/Submit) id h1FKj4QH061493; Sun, 16 Feb 2003 07:45:04 +1100 (EST) Date: Sun, 16 Feb 2003 07:45:04 +1100 From: Peter Jeremy To: Wes Peters Cc: arch@FreeBSD.ORG Subject: Re: syslog.conf syntax change (multiple program/host specifications) Message-ID: <20030215204503.GA56102@cirb503493.alcatel.com.au> References: <20030210114930.GB90800@melusine.cuivre.fr.eu.org> <200302141725.00421.wes@softweyr.com> <3E4DB9F1.83B7746D@mindspring.com> <200302150918.09807.wes@softweyr.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200302150918.09807.wes@softweyr.com> User-Agent: Mutt/1.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, Feb 15, 2003 at 09:18:09AM +0000, Wes Peters wrote: >On Saturday 15 February 2003 03:54, Terry Lambert wrote: >> What might be more useful, in this particular case, since you want >> only the last 120K is a means of truncating deleting cytes off the >> front of a file. You'd probably want to allow any value, but in >> terms of avoiding rewriting, in practice, you probably want to >> constrain it to an even multiple of the FS block size. > >Well, I certainly don't want any of that. I want to be able to lop the >first 11 bytes of a 2^30 byte file and have exactly that data in the file >the next time I access it. This is somewhat different to what Terry was proposing but I agree it would be immensely useful. I also fairly regularly run into the situation where I want an indefinitely sized FIFO and simulating such a beast is quite a nuisance without this. Definitely, for this sort of thing, you don't want to be constrained by the underlying filesystem blocksize. (One obvious situation is where you have a producer that generates important data and if the consumer stops or falls behind you can't afford to either block the producer or lose the data). >Right, only I don't have anywhere near the filesystem-foo to implement such >a change. I suppose it could be done relatively straightforward by >allowing the original leading disk blocks to be marked unused and making an >offset to the beginning of the file field in the inode, that counts bytes >to skip into the first truly allocated block. This doesn't sound too difficult - we can already free blocks from the end of a file so it shouldn't be too difficult to free blocks from the beginning of a file. Adding a start-of-file offset to I/O operations is almost a mechanical operation. The only hard part would be finding space in the inode for another off_t. The downside of this is that there would be an upper limit on the total number of bytes that can be written to the file (ie when you run out of triple-indirect blocks). You could avoid this by dropping unused blocks at the front and shifting the remaining blocks forwards in the inode. (Probably as groups of blocks to avoid the need to move block pointers within indirect blocks). This would also reduce the size of the offset from off_t to enough to represent an indirect block of bytes. Peter To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Feb 15 16:53:41 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D525337B401; Sat, 15 Feb 2003 16:53:38 -0800 (PST) Received: from hotmail.com (ts27-b193.Moscow.dial.rol.ru [212.46.245.193]) by mx1.FreeBSD.org (Postfix) with SMTP id C375943F85; Sat, 15 Feb 2003 16:53:27 -0800 (PST) (envelope-from ljlbr3@rambler.ru) Reply-To: ljlbr3@rambler.ru From: petrova olga Subject: #ԣԃݐ BTMYSVFOYF Date: Sun, 16 Feb 1992 03:51:48 +0300 MIME-Version: 1.0 Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: 7bit X-Priority: 1 X-MSMail-Priority: High X-Mailer: Microsoft Outlook Express 6.00.2800.1081 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1081 Message-Id: <20030216005327.C375943F85@mx1.FreeBSD.org> To: undisclosed-recipients: ; Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG #ԣԃݐ. Ҽ Ӝ ܟ ܐ#ȣ. Ȑ# ٝ. ------------------------------------ 50 Ԟ ٟ # - ƞ ܝ. http://www.oboi.h10.ru SRCMCSYNHHQREEWVFGGPJIJYWERYLNHFQRVPWU To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Feb 15 17:25:35 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 72CAB37B405 for ; Sat, 15 Feb 2003 17:25:33 -0800 (PST) Received: from mail26a.sbc-webhosting.com (mail26a.sbc-webhosting.com [216.173.237.36]) by mx1.FreeBSD.org (Postfix) with SMTP id 177D243FB1 for ; Sat, 15 Feb 2003 17:25:32 -0800 (PST) (envelope-from alc@imimic.com) Received: from www.imimic.com (64.143.12.21) by mail26a.sbc-webhosting.com (RS ver 1.0.63s) with SMTP id 099602; Sat, 15 Feb 2003 20:25:20 -0500 (EST) Message-ID: <3E4EE887.73508A59@imimic.com> Date: Sat, 15 Feb 2003 19:25:27 -0600 From: "Alan L. Cox" Organization: iMimic Networking, Inc. X-Mailer: Mozilla 4.8 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: en MIME-Version: 1.0 To: arch@freebsd.org Cc: trb@freebsd.org Subject: TRB resolution of M_* changes to the memory and mbuf allocators Content-Type: text/plain; charset=x-user-defined Content-Transfer-Encoding: 7bit X-Loop-Detect: 1 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG The TRB reviewed the change to the memory and mbuf allocators that removes the M_*WAIT* flags and replaces all occurrences in the source code with 0. Those in favor argued the change would: o eliminate typographic errors, o emphasize that these APIs have a default behavior (wait for operation to complete) with only exceptional cases marked by a flag argument, and o emphasize to developers that synchronization issues cannot be solved simply by making allocation requests non-blocking. Those opposed to the change argued: o this breaks source-code compatibility with prior releases of FreeBSD as well as other systems with the same APIs (e.g. netbsd, openbsd), o typographic errors of the sort described are better handled by defining names with unique non-zero values that can then be validated in the routines where they are passed as parameters, o the allocators have not historically had a default behaviour, o removing the explicit specification of whether to wait or not wait at each call site removes useful information from the source code, and o the synchronization issues intended to be addressed will not be solved in this way but instead require developer education. After careful review of arguments from both sides the TRB voted: In favor of change: 0 Opposed to change: 9 Abstain: 0 It is the TRB's decision that this change should be reverted; and instead the course of action outlined below should be followed. 1) Revert to the old API 2) Put enforcement into malloc/mbuf so that you have to specify a flag. 2a)with the default being NOWAIT/WAITOK if you don't for a transition period. 3) Put witness checks in malloc for "sleepability." 4) Make all the M_* flags unique to avoid possible misuse. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message