From owner-freebsd-arch@FreeBSD.ORG Tue Sep 16 16:53:34 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5437716A515 for ; Tue, 16 Sep 2003 16:53:32 -0700 (PDT) Received: from arginine.spc.org (arginine.spc.org [195.206.69.236]) by mx1.FreeBSD.org (Postfix) with ESMTP id 747244423E for ; Tue, 16 Sep 2003 16:27:27 -0700 (PDT) (envelope-from bms@spc.org) Received: from localhost (localhost [127.0.0.1]) by arginine.spc.org (Postfix) with ESMTP id C7D216530D for ; Wed, 17 Sep 2003 00:27:25 +0100 (BST) Received: from arginine.spc.org ([127.0.0.1]) by localhost (arginine.spc.org [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 07779-01 for ; Wed, 17 Sep 2003 00:27:25 +0100 (BST) Received: from saboteur.dek.spc.org (unknown [81.3.72.68]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by arginine.spc.org (Postfix) with ESMTP id 2D2B165285 for ; Wed, 17 Sep 2003 00:27:25 +0100 (BST) Received: by saboteur.dek.spc.org (Postfix, from userid 1001) id 41C9110; Wed, 17 Sep 2003 00:27:24 +0100 (BST) Date: Wed, 17 Sep 2003 00:27:24 +0100 From: Bruce M Simpson To: freebsd-arch@freebsd.org Message-ID: <20030916232724.GG3052@saboteur.dek.spc.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Organization: SPC Subject: devd limitations / automounting removable storage X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Sep 2003 23:53:36 -0000 Hi, I noticed that when I attach a USB key fob, I don't get a devd message to say e.g. 'da0 at umass-sim0 bus 0 target 0 lun 0'. Any objections to adding devd notifications to the CAM layer? It would allow us to automount such devices. Specifically I'm thinking that a program to change the running automounter map could be used. Of course, filesystems which could be removed at any time would have to use synchronous I/O completion and preferably a log-structured filesystem (see recent fsync/msdosfs bug and ohci problems). What do people think? Perhaps we could extrapolate this to when CDs are inserted into drives (one for Soren...) BMS From owner-freebsd-arch@FreeBSD.ORG Tue Sep 16 22:35:28 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D285816A4B3 for ; Tue, 16 Sep 2003 22:35:28 -0700 (PDT) Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0B62B43F85 for ; Tue, 16 Sep 2003 22:35:28 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.9/8.12.3) with ESMTP id h8H5ZQTX088675; Tue, 16 Sep 2003 23:35:27 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Tue, 16 Sep 2003 23:35:28 -0600 (MDT) Message-Id: <20030916.233528.64215542.imp@bsdimp.com> To: bms@spc.org From: "M. Warner Losh" In-Reply-To: <20030916232724.GG3052@saboteur.dek.spc.org> References: <20030916232724.GG3052@saboteur.dek.spc.org> X-Mailer: Mew version 2.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: freebsd-arch@freebsd.org Subject: Re: devd limitations / automounting removable storage X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Sep 2003 05:35:28 -0000 In message: <20030916232724.GG3052@saboteur.dek.spc.org> Bruce M Simpson writes: : Any objections to adding devd notifications to the CAM layer? It would : allow us to automount such devices. Specifically I'm thinking that a : program to change the running automounter map could be used. I'd make them different types of events. I'd also prototype it up to see how bad things get. In the long term, we want CAM to be newbus. However, in the long term we'd also like to move away from doing things to devices, and more towards doing things on the resources they provide to the system. When a network interface appears, we should use that event, not that a driver attached. The simple devd detach/attachment stuff we do now gets us a lot of bang for the buck, but there are limitations. Warner From owner-freebsd-arch@FreeBSD.ORG Tue Sep 16 23:51:29 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 515A916A4B3 for ; Tue, 16 Sep 2003 23:51:29 -0700 (PDT) Received: from ns1.xcllnt.net (209-128-86-226.BAYAREA.NET [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id D5CE643FBD for ; Tue, 16 Sep 2003 23:51:27 -0700 (PDT) (envelope-from marcel@xcllnt.net) Received: from athlon.pn.xcllnt.net (athlon.pn.xcllnt.net [192.168.4.3]) by ns1.xcllnt.net (8.12.9/8.12.9) with ESMTP id h8H6pRjR004527 for ; Tue, 16 Sep 2003 23:51:27 -0700 (PDT) (envelope-from marcel@piii.pn.xcllnt.net) Received: from athlon.pn.xcllnt.net (localhost [127.0.0.1]) by athlon.pn.xcllnt.net (8.12.9/8.12.9) with ESMTP id h8H6pRFd004385 for ; Tue, 16 Sep 2003 23:51:27 -0700 (PDT) (envelope-from marcel@athlon.pn.xcllnt.net) Received: (from marcel@localhost) by athlon.pn.xcllnt.net (8.12.9/8.12.9/Submit) id h8H6pRY3004384 for arch@FreeBSD.org; Tue, 16 Sep 2003 23:51:27 -0700 (PDT) (envelope-from marcel) Date: Tue, 16 Sep 2003 23:51:27 -0700 From: Marcel Moolenaar To: arch@FreeBSD.org Message-ID: <20030917065127.GB4261@athlon.pn.xcllnt.net> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="HcAYCG3uE/tztfnV" Content-Disposition: inline User-Agent: Mutt/1.5.4i Subject: make(1): adding sort modifiers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Sep 2003 06:51:29 -0000 --HcAYCG3uE/tztfnV Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Gang, Attached a patch that adds the functionality to make(1) to sort the words in a variable. With this functionality and a small change to bsd.subdir.mk (also attached), we automaticly have sorted subdirectory recursion, even though it's impossible or hard to do it in the makesfiles themselves. Is this too evil? -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net --HcAYCG3uE/tztfnV Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="make.diff" Index: var.c =================================================================== RCS file: /home/ncvs/src/usr.bin/make/var.c,v retrieving revision 1.42 diff -u -r1.42 var.c --- var.c 15 Jan 2003 22:36:15 -0000 1.42 +++ var.c 17 Sep 2003 06:30:11 -0000 @@ -606,7 +606,38 @@ return (str); } +static char * +VarSortWords(char *str, int (*cmp)(const void *, const void *)) +{ + Buffer buf; + char **av; + int ac, i; + + buf = Buf_Init(0); + av = brk_string(str, &ac, FALSE); + qsort((void*)(av + 1), ac - 1, sizeof(char*), cmp); + for (i = 1; i < ac; i++) { + Buf_AddBytes(buf, strlen(av[i]), (Byte *)av[i]); + Buf_AddByte(buf, (Byte)((i < ac - 1) ? ' ' : '\0')); + } + str = (char *)Buf_GetAll(buf, (int *)NULL); + Buf_Destroy(buf, FALSE); + return (str); +} + +static int +SortIncreasing(const void *l, const void *r) +{ + return (strcmp(*(char**)l, *(char**)r)); +} + +static int +SortDecreasing(const void *l, const void *r) +{ + return (-strcmp(*(char**)l, *(char**)r)); +} + /*- *----------------------------------------------------------------------- * VarGetPattern -- @@ -1448,6 +1479,22 @@ free(pattern.matches); break; } + case '<': + if (tstr[1] == endc || tstr[1] == ':') { + newStr = VarSortWords(str, SortIncreasing); + cp = tstr + 1; + termc = *cp; + break; + } + /* FALLTHROUGH */ + case '>': + if (tstr[1] == endc || tstr[1] == ':') { + newStr = VarSortWords(str, SortDecreasing); + cp = tstr + 1; + termc = *cp; + break; + } + /* FALLTHROUGH */ case 'Q': if (tstr[1] == endc || tstr[1] == ':') { newStr = VarQuote (str); --HcAYCG3uE/tztfnV Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="subdir.diff" Index: bsd.subdir.mk =================================================================== RCS file: /home/ncvs/src/share/mk/bsd.subdir.mk,v retrieving revision 1.44 diff -u -r1.44 bsd.subdir.mk --- bsd.subdir.mk 12 Jul 2002 15:09:35 -0000 1.44 +++ bsd.subdir.mk 17 Sep 2003 06:46:57 -0000 @@ -42,7 +42,7 @@ _SUBDIR: .USE .if defined(SUBDIR) && !empty(SUBDIR) && !defined(NO_SUBDIR) - @for entry in ${SUBDIR}; do \ + @for entry in ${SUBDIR:<}; do \ if test -d ${.CURDIR}/$${entry}.${MACHINE_ARCH}; then \ ${ECHODIR} "===> ${DIRPRFX}$${entry}.${MACHINE_ARCH}"; \ edir=$${entry}.${MACHINE_ARCH}; \ --HcAYCG3uE/tztfnV-- From owner-freebsd-arch@FreeBSD.ORG Wed Sep 17 00:01:16 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B086F16A4C0 for ; Wed, 17 Sep 2003 00:01:16 -0700 (PDT) Received: from phk.freebsd.dk (phk.freebsd.dk [212.242.86.175]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9ED4243FE0 for ; Wed, 17 Sep 2003 00:01:15 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by phk.freebsd.dk (8.12.8/8.12.8) with ESMTP id h8H71DV3069182; Wed, 17 Sep 2003 07:01:13 GMT (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h8H71C6f024226; Wed, 17 Sep 2003 09:01:12 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: Marcel Moolenaar From: "Poul-Henning Kamp" In-Reply-To: Your message of "Tue, 16 Sep 2003 23:51:27 PDT." <20030917065127.GB4261@athlon.pn.xcllnt.net> Date: Wed, 17 Sep 2003 09:01:12 +0200 Message-ID: <24225.1063782072@critter.freebsd.dk> cc: arch@freebsd.org Subject: Re: make(1): adding sort modifiers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Sep 2003 07:01:16 -0000 In message <20030917065127.GB4261@athlon.pn.xcllnt.net>, Marcel Moolenaar write s: > >--HcAYCG3uE/tztfnV >Content-Type: text/plain; charset=us-ascii >Content-Disposition: inline > >Gang, > >Attached a patch that adds the functionality to make(1) to sort >the words in a variable. With this functionality and a small >change to bsd.subdir.mk (also attached), we automaticly have >sorted subdirectory recursion, even though it's impossible or >hard to do it in the makesfiles themselves. Is this too evil? Do we actually want to do that ? I thought we had places where the ordering was explicit to express dependencies ? (Just to be clear: this is a concern for the bsd.subdir.mk part, not the make(1) part of the patch) -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Wed Sep 17 00:39:16 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B20EB16A4E2 for ; Wed, 17 Sep 2003 00:39:16 -0700 (PDT) Received: from mail.cyberonic.com (mail.cyberonic.com [4.17.179.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 76B8F43FBF for ; Wed, 17 Sep 2003 00:39:13 -0700 (PDT) (envelope-from jmg@hydrogen.funkthat.com) Received: from hydrogen.funkthat.com (node-40244c0a.sfo.onnet.us.uu.net [64.36.76.10]) by mail.cyberonic.com (8.12.8/8.12.5) with ESMTP id h8H7iabi031541; Wed, 17 Sep 2003 03:44:36 -0400 Received: (from jmg@localhost) by hydrogen.funkthat.com (8.12.9/8.11.6) id h8H7d0sC057496; Wed, 17 Sep 2003 00:39:00 -0700 (PDT) (envelope-from jmg) Date: Wed, 17 Sep 2003 00:39:00 -0700 From: John-Mark Gurney To: Marcel Moolenaar Message-ID: <20030917073900.GG39788@funkthat.com> Mail-Followup-To: Marcel Moolenaar , arch@freebsd.org References: <20030917065127.GB4261@athlon.pn.xcllnt.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030917065127.GB4261@athlon.pn.xcllnt.net> User-Agent: Mutt/1.4.1i X-Operating-System: FreeBSD 4.2-RELEASE i386 X-PGP-Fingerprint: B7 EC EF F8 AE ED A7 31 96 7A 22 B3 D8 56 36 F4 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html cc: arch@freebsd.org Subject: Re: make(1): adding sort modifiers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: John-Mark Gurney List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Sep 2003 07:39:16 -0000 Marcel Moolenaar wrote this message on Tue, Sep 16, 2003 at 23:51 -0700: > sorted subdirectory recursion, even though it's impossible or > hard to do it in the makesfiles themselves. Is this too evil? I always thought that sorted SUBDIR lines in the Makefile was for aiding the developer in seeing what is available, and where to put the new module. I don't know about you, but seeing a bunch of unsorted SUBDIRs wouldn't feel right. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Wed Sep 17 00:59:23 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8478216A4B3 for ; Wed, 17 Sep 2003 00:59:23 -0700 (PDT) Received: from ns1.xcllnt.net (209-128-86-226.BAYAREA.NET [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9D2E543FDF for ; Wed, 17 Sep 2003 00:59:22 -0700 (PDT) (envelope-from marcel@xcllnt.net) Received: from dhcp01.pn.xcllnt.net (dhcp01.pn.xcllnt.net [192.168.4.201]) by ns1.xcllnt.net (8.12.9/8.12.9) with ESMTP id h8H7xMjR004878; Wed, 17 Sep 2003 00:59:22 -0700 (PDT) (envelope-from marcel@piii.pn.xcllnt.net) Received: from dhcp01.pn.xcllnt.net (localhost [127.0.0.1]) by dhcp01.pn.xcllnt.net (8.12.9/8.12.9) with ESMTP id h8H7xMSG016036; Wed, 17 Sep 2003 00:59:22 -0700 (PDT) (envelope-from marcel@dhcp01.pn.xcllnt.net) Received: (from marcel@localhost) by dhcp01.pn.xcllnt.net (8.12.9/8.12.9/Submit) id h8H7xMUX016035; Wed, 17 Sep 2003 00:59:22 -0700 (PDT) (envelope-from marcel) Date: Wed, 17 Sep 2003 00:59:22 -0700 From: Marcel Moolenaar To: Poul-Henning Kamp Message-ID: <20030917075922.GA16024@dhcp01.pn.xcllnt.net> References: <20030917065127.GB4261@athlon.pn.xcllnt.net> <24225.1063782072@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <24225.1063782072@critter.freebsd.dk> User-Agent: Mutt/1.5.4i cc: arch@freebsd.org Subject: Re: make(1): adding sort modifiers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Sep 2003 07:59:23 -0000 On Wed, Sep 17, 2003 at 09:01:12AM +0200, Poul-Henning Kamp wrote: > In message <20030917065127.GB4261@athlon.pn.xcllnt.net>, Marcel Moolenaar writes: > > > >Attached a patch that adds the functionality to make(1) to sort > >the words in a variable. With this functionality and a small > >change to bsd.subdir.mk (also attached), we automaticly have > >sorted subdirectory recursion, even though it's impossible or > >hard to do it in the makesfiles themselves. Is this too evil? > > Do we actually want to do that ? I thought we had places > where the ordering was explicit to express dependencies ? /usr/src/lib has a problem, because we do encode dependencies in the ordering, but we shouldn't really do that anyway. Dependencies should be expressed by dependency rules. Something like: SUBDIR= a b c d a: b d: a -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net From owner-freebsd-arch@FreeBSD.ORG Wed Sep 17 01:01:28 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B955516A4B3 for ; Wed, 17 Sep 2003 01:01:28 -0700 (PDT) Received: from ns1.xcllnt.net (209-128-86-226.BAYAREA.NET [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id EA25743F85 for ; Wed, 17 Sep 2003 01:01:27 -0700 (PDT) (envelope-from marcel@xcllnt.net) Received: from dhcp01.pn.xcllnt.net (dhcp01.pn.xcllnt.net [192.168.4.201]) by ns1.xcllnt.net (8.12.9/8.12.9) with ESMTP id h8H81RjR004910 for ; Wed, 17 Sep 2003 01:01:27 -0700 (PDT) (envelope-from marcel@piii.pn.xcllnt.net) Received: from dhcp01.pn.xcllnt.net (localhost [127.0.0.1]) by dhcp01.pn.xcllnt.net (8.12.9/8.12.9) with ESMTP id h8H81RSG016065 for ; Wed, 17 Sep 2003 01:01:27 -0700 (PDT) (envelope-from marcel@dhcp01.pn.xcllnt.net) Received: (from marcel@localhost) by dhcp01.pn.xcllnt.net (8.12.9/8.12.9/Submit) id h8H81Rit016064 for arch@freebsd.org; Wed, 17 Sep 2003 01:01:27 -0700 (PDT) (envelope-from marcel) Date: Wed, 17 Sep 2003 01:01:27 -0700 From: Marcel Moolenaar To: arch@freebsd.org Message-ID: <20030917080127.GB16024@dhcp01.pn.xcllnt.net> References: <20030917065127.GB4261@athlon.pn.xcllnt.net> <20030917073900.GG39788@funkthat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030917073900.GG39788@funkthat.com> User-Agent: Mutt/1.5.4i Subject: Re: make(1): adding sort modifiers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Sep 2003 08:01:28 -0000 On Wed, Sep 17, 2003 at 12:39:00AM -0700, John-Mark Gurney wrote: > Marcel Moolenaar wrote this message on Tue, Sep 16, 2003 at 23:51 -0700: > > sorted subdirectory recursion, even though it's impossible or > > hard to do it in the makesfiles themselves. Is this too evil? > > I always thought that sorted SUBDIR lines in the Makefile was for > aiding the developer in seeing what is available, and where to put > the new module. Yes. What's your point? -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net From owner-freebsd-arch@FreeBSD.ORG Wed Sep 17 11:04:28 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9F8D816A4B3 for ; Wed, 17 Sep 2003 11:04:28 -0700 (PDT) Received: from mail.cyberonic.com (mail.cyberonic.com [4.17.179.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id E7CD243FB1 for ; Wed, 17 Sep 2003 11:04:24 -0700 (PDT) (envelope-from jmg@hydrogen.funkthat.com) Received: from hydrogen.funkthat.com (node-40244c0a.sfo.onnet.us.uu.net [64.36.76.10]) by mail.cyberonic.com (8.12.8/8.12.5) with ESMTP id h8HI9ubi032538; Wed, 17 Sep 2003 14:09:57 -0400 Received: (from jmg@localhost) by hydrogen.funkthat.com (8.12.9/8.11.6) id h8HI4L3D067235; Wed, 17 Sep 2003 11:04:21 -0700 (PDT) (envelope-from jmg) Date: Wed, 17 Sep 2003 11:04:21 -0700 From: John-Mark Gurney To: Marcel Moolenaar Message-ID: <20030917180421.GH39788@funkthat.com> Mail-Followup-To: Marcel Moolenaar , arch@freebsd.org References: <20030917065127.GB4261@athlon.pn.xcllnt.net> <20030917073900.GG39788@funkthat.com> <20030917080127.GB16024@dhcp01.pn.xcllnt.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030917080127.GB16024@dhcp01.pn.xcllnt.net> User-Agent: Mutt/1.4.1i X-Operating-System: FreeBSD 4.2-RELEASE i386 X-PGP-Fingerprint: B7 EC EF F8 AE ED A7 31 96 7A 22 B3 D8 56 36 F4 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html cc: arch@freebsd.org Subject: Re: make(1): adding sort modifiers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: John-Mark Gurney List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Sep 2003 18:04:28 -0000 Marcel Moolenaar wrote this message on Wed, Sep 17, 2003 at 01:01 -0700: > On Wed, Sep 17, 2003 at 12:39:00AM -0700, John-Mark Gurney wrote: > > Marcel Moolenaar wrote this message on Tue, Sep 16, 2003 at 23:51 -0700: > > > sorted subdirectory recursion, even though it's impossible or > > > hard to do it in the makesfiles themselves. Is this too evil? > > > > I always thought that sorted SUBDIR lines in the Makefile was for > > aiding the developer in seeing what is available, and where to put > > the new module. > > Yes. What's your point? The comment of impossible or hard made me think you were advocating dropping the requirement of sorted SUBDIR's since make now does that. But assuming from your response, that was not the case. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Wed Sep 17 11:36:03 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9B8A916A4B3 for ; Wed, 17 Sep 2003 11:36:03 -0700 (PDT) Received: from mail.speakeasy.net (mail10.speakeasy.net [216.254.0.210]) by mx1.FreeBSD.org (Postfix) with ESMTP id DD6E543FBD for ; Wed, 17 Sep 2003 11:36:02 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Received: (qmail 10941 invoked from network); 17 Sep 2003 18:36:02 -0000 Received: from unknown (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender )encrypted SMTP for ; 17 Sep 2003 18:36:02 -0000 Received: from laptop.baldwin.cx (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.9/8.12.9) with ESMTP id h8HIZx6Y083262; Wed, 17 Sep 2003 14:35:59 -0400 (EDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.5.4 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20030916.233528.64215542.imp@bsdimp.com> Date: Wed, 17 Sep 2003 14:36:01 -0400 (EDT) From: John Baldwin To: "M. Warner Losh" X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp) cc: bms@spc.org cc: freebsd-arch@freebsd.org Subject: Re: devd limitations / automounting removable storage X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Sep 2003 18:36:03 -0000 On 17-Sep-2003 M. Warner Losh wrote: > In message: <20030916232724.GG3052@saboteur.dek.spc.org> > Bruce M Simpson writes: >: Any objections to adding devd notifications to the CAM layer? It would >: allow us to automount such devices. Specifically I'm thinking that a >: program to change the running automounter map could be used. > > I'd make them different types of events. I'd also prototype it up to > see how bad things get. > > In the long term, we want CAM to be newbus. > > However, in the long term we'd also like to move away from doing > things to devices, and more towards doing things on the resources they > provide to the system. When a network interface appears, we should > use that event, not that a driver attached. > > The simple devd detach/attachment stuff we do now gets us a lot of > bang for the buck, but there are limitations. Maybe have GEOM send events when mountable entities show up? -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ From owner-freebsd-arch@FreeBSD.ORG Wed Sep 17 11:42:43 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E2BA616A4B3 for ; Wed, 17 Sep 2003 11:42:43 -0700 (PDT) Received: from ns1.xcllnt.net (209-128-86-226.BAYAREA.NET [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 906C543FAF for ; Wed, 17 Sep 2003 11:42:42 -0700 (PDT) (envelope-from marcel@xcllnt.net) Received: from dhcp01.pn.xcllnt.net (dhcp01.pn.xcllnt.net [192.168.4.201]) by ns1.xcllnt.net (8.12.9/8.12.9) with ESMTP id h8HIggjR008070 for ; Wed, 17 Sep 2003 11:42:42 -0700 (PDT) (envelope-from marcel@piii.pn.xcllnt.net) Received: from dhcp01.pn.xcllnt.net (localhost [127.0.0.1]) by dhcp01.pn.xcllnt.net (8.12.9/8.12.9) with ESMTP id h8HIggSG018198 for ; Wed, 17 Sep 2003 11:42:42 -0700 (PDT) (envelope-from marcel@dhcp01.pn.xcllnt.net) Received: (from marcel@localhost) by dhcp01.pn.xcllnt.net (8.12.9/8.12.9/Submit) id h8HIggqw018197 for arch@freebsd.org; Wed, 17 Sep 2003 11:42:42 -0700 (PDT) (envelope-from marcel) Date: Wed, 17 Sep 2003 11:42:41 -0700 From: Marcel Moolenaar To: arch@freebsd.org Message-ID: <20030917184241.GA18166@dhcp01.pn.xcllnt.net> References: <20030917065127.GB4261@athlon.pn.xcllnt.net> <20030917073900.GG39788@funkthat.com> <20030917080127.GB16024@dhcp01.pn.xcllnt.net> <20030917180421.GH39788@funkthat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030917180421.GH39788@funkthat.com> User-Agent: Mutt/1.5.4i Subject: Re: make(1): adding sort modifiers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Sep 2003 18:42:44 -0000 On Wed, Sep 17, 2003 at 11:04:21AM -0700, John-Mark Gurney wrote: > Marcel Moolenaar wrote this message on Wed, Sep 17, 2003 at 01:01 -0700: > > On Wed, Sep 17, 2003 at 12:39:00AM -0700, John-Mark Gurney wrote: > > > Marcel Moolenaar wrote this message on Tue, Sep 16, 2003 at 23:51 -0700: > > > > sorted subdirectory recursion, even though it's impossible or > > > > hard to do it in the makesfiles themselves. Is this too evil? > > > > > > I always thought that sorted SUBDIR lines in the Makefile was for > > > aiding the developer in seeing what is available, and where to put > > > the new module. > > > > Yes. What's your point? > > The comment of impossible or hard made me think you were advocating > dropping the requirement of sorted SUBDIR's since make now does that. > > But assuming from your response, that was not the case. Correct. We should keep lists sorted. It's when we need to combine multiple lists that we need a hand. Having multiple platforms and a busload of NO_way options is making it hard to get the final list sorted. If make(1) can give us a hand with that, then we can also keep an eye on maintainability and readability of our makefiles. I just wanted to know if that was a bad idea or not... -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net From owner-freebsd-arch@FreeBSD.ORG Wed Sep 17 12:58:05 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D3A0316A4B3; Wed, 17 Sep 2003 12:58:05 -0700 (PDT) Received: from phk.freebsd.dk (phk.freebsd.dk [212.242.86.175]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8545E43F3F; Wed, 17 Sep 2003 12:58:04 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by phk.freebsd.dk (8.12.8/8.12.8) with ESMTP id h8HJw0Q2080551; Wed, 17 Sep 2003 19:58:00 GMT (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h8HJvr6f030913; Wed, 17 Sep 2003 21:57:54 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: John Baldwin From: "Poul-Henning Kamp" In-Reply-To: Your message of "Wed, 17 Sep 2003 14:36:01 EDT." Date: Wed, 17 Sep 2003 21:57:53 +0200 Message-ID: <30912.1063828673@critter.freebsd.dk> cc: bms@spc.org cc: "M. Warner Losh" cc: freebsd-arch@freebsd.org Subject: Re: devd limitations / automounting removable storage X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Sep 2003 19:58:05 -0000 In message , John Baldwin writes: >Maybe have GEOM send events when mountable entities show up? There is only one question to figure out: Should it happen at the bottom or the top of GEOM ? At the bottom, your CF card would result in a single event on /dev/ad4, at the top you'd likely get two events, one for /dev/ad4 and one for /dev/ad4s1 (or whatever other gadgets geom autodiscovers). Apart from that, it's just a matter of telling me how to send the event... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Wed Sep 17 13:21:27 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BAF1C16A4C1; Wed, 17 Sep 2003 13:21:27 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6544F43FD7; Wed, 17 Sep 2003 13:21:26 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.9/8.12.9) with ESMTP id h8HKJwN6053531; Wed, 17 Sep 2003 16:19:58 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)h8HKJwex053528; Wed, 17 Sep 2003 16:19:58 -0400 (EDT) Date: Wed, 17 Sep 2003 16:19:57 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Poul-Henning Kamp In-Reply-To: <30912.1063828673@critter.freebsd.dk> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: bms@spc.org cc: "M. Warner Losh" cc: freebsd-arch@freebsd.org Subject: Re: devd limitations / automounting removable storage X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Sep 2003 20:21:27 -0000 On Wed, 17 Sep 2003, Poul-Henning Kamp wrote: > In message , John Baldwin writes: > > >Maybe have GEOM send events when mountable entities show up? > > There is only one question to figure out: Should it happen at the > bottom or the top of GEOM ? > > At the bottom, your CF card would result in a single event on /dev/ad4, > at the top you'd likely get two events, one for /dev/ad4 and one for > /dev/ad4s1 (or whatever other gadgets geom autodiscovers). > > Apart from that, it's just a matter of telling me how to send the > event... This is a very similar concern to the question I raised at BSDCon about distinguishing network interfaces at the newbus and network service levels. My temptation would be to assign a "layer" as well as a device name to devices when they are announced. I.e., Layer Device Meaning newbus xl0 A device named xl0 is now available devfs net/xl0 A device node named net/xl0 is now available ifnet xl0 A network interface named xl0 is now available newbus ad0 A device named ad0 is now available devfs ad0 A device node named ad0 is now available geom ad0 A storage device named ad0 is now available We might skip the devfs layer since it's probably implicit in the completion of ifnet_attach() and disk_create(), but it's worth thinking about. This would allow ifconfig commands to be issued once the network device is available, rather than once newbus has attached an xl0. Or for geom to announce ad0s1e even though newbus doesn't know about it. Or for devd to handle the introduction of synthetic interfaces such as vlan, tun, etc, which aren't visible to newbus. Or for the device driver names and interface names to differ (or for a non-one-one mapping, interface renames, etc). Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Network Associates Laboratories From owner-freebsd-arch@FreeBSD.ORG Wed Sep 17 14:03:06 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C764B16A4B3; Wed, 17 Sep 2003 14:03:06 -0700 (PDT) Received: from mail.cyberonic.com (mail.cyberonic.com [4.17.179.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8F99043F75; Wed, 17 Sep 2003 14:03:05 -0700 (PDT) (envelope-from jmg@hydrogen.funkthat.com) Received: from hydrogen.funkthat.com (node-40244c0a.sfo.onnet.us.uu.net [64.36.76.10]) by mail.cyberonic.com (8.12.8/8.12.5) with ESMTP id h8HL8Obi001784; Wed, 17 Sep 2003 17:08:24 -0400 Received: (from jmg@localhost) by hydrogen.funkthat.com (8.12.9/8.11.6) id h8HL2a5T078700; Wed, 17 Sep 2003 14:02:36 -0700 (PDT) (envelope-from jmg) Date: Wed, 17 Sep 2003 14:02:36 -0700 From: John-Mark Gurney To: Robert Watson Message-ID: <20030917210236.GB75714@funkthat.com> Mail-Followup-To: Robert Watson , Poul-Henning Kamp , bms@spc.org, "M. Warner Losh" , freebsd-arch@freebsd.org References: <30912.1063828673@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i X-Operating-System: FreeBSD 4.2-RELEASE i386 X-PGP-Fingerprint: B7 EC EF F8 AE ED A7 31 96 7A 22 B3 D8 56 36 F4 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html cc: bms@spc.org cc: Poul-Henning Kamp cc: "M. Warner Losh" cc: freebsd-arch@freebsd.org Subject: Re: devd limitations / automounting removable storage X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: John-Mark Gurney List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Sep 2003 21:03:06 -0000 Robert Watson wrote this message on Wed, Sep 17, 2003 at 16:19 -0400: > > Apart from that, it's just a matter of telling me how to send the > > event... > > This is a very similar concern to the question I raised at BSDCon about > distinguishing network interfaces at the newbus and network service > levels. My temptation would be to assign a "layer" as well as a device > name to devices when they are announced. I.e., > > Layer Device Meaning > newbus xl0 A device named xl0 is now available > devfs net/xl0 A device node named net/xl0 is now available > ifnet xl0 A network interface named xl0 is now available > newbus ad0 A device named ad0 is now available > devfs ad0 A device node named ad0 is now available > geom ad0 A storage device named ad0 is now available > > We might skip the devfs layer since it's probably implicit in the > completion of ifnet_attach() and disk_create(), but it's worth thinking > about. This would allow ifconfig commands to be issued once the network > device is available, rather than once newbus has attached an xl0. Or for > geom to announce ad0s1e even though newbus doesn't know about it. Or for > devd to handle the introduction of synthetic interfaces such as vlan, tun, > etc, which aren't visible to newbus. Or for the device driver names and > interface names to differ (or for a non-one-one mapping, interface > renames, etc). I was thinking about a more generic event posting mechanism, where modules can register to receive notifications when events came in. We should have each event contain: system (enum/int) subsystem? (enum/int) event (enum/int) device (char[64?]) additionaldata (void *, size_t combo) This means we can make it more extensible, and have a set of well defined system/subsystem/event triplets, while at the same time, letting future interfaces expand. The subsystem would be used for something like the network stack, which has multiple subsystems, like interface, routing, connections. The advantage is then we can reduce the number of /dev entries that convey the same information, but have different calling conventions. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Wed Sep 17 17:22:18 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 841F016A4B3; Wed, 17 Sep 2003 17:22:18 -0700 (PDT) Received: from mail.chesapeake.net (chesapeake.net [208.142.252.6]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0C2F043FDD; Wed, 17 Sep 2003 17:22:15 -0700 (PDT) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id h8I0Inm59677; Wed, 17 Sep 2003 20:18:50 -0400 (EDT) (envelope-from jroberson@chesapeake.net) Date: Wed, 17 Sep 2003 20:18:49 -0400 (EDT) From: Jeff Roberson To: John-Mark Gurney In-Reply-To: <20030917210236.GB75714@funkthat.com> Message-ID: <20030917201822.M55626-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: bms@spc.org cc: Poul-Henning Kamp cc: Robert Watson cc: "M. Warner Losh" cc: freebsd-arch@freebsd.org Subject: Re: devd limitations / automounting removable storage X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Sep 2003 00:22:18 -0000 On Wed, 17 Sep 2003, John-Mark Gurney wrote: > Robert Watson wrote this message on Wed, Sep 17, 2003 at 16:19 -0400: > > > Apart from that, it's just a matter of telling me how to send the > > > event... > > > > This is a very similar concern to the question I raised at BSDCon about > > distinguishing network interfaces at the newbus and network service > > levels. My temptation would be to assign a "layer" as well as a device > > name to devices when they are announced. I.e., > > > > Layer Device Meaning > > newbus xl0 A device named xl0 is now available > > devfs net/xl0 A device node named net/xl0 is now available > > ifnet xl0 A network interface named xl0 is now available > > newbus ad0 A device named ad0 is now available > > devfs ad0 A device node named ad0 is now available > > geom ad0 A storage device named ad0 is now available > > > > We might skip the devfs layer since it's probably implicit in the > > completion of ifnet_attach() and disk_create(), but it's worth thinking > > about. This would allow ifconfig commands to be issued once the network > > device is available, rather than once newbus has attached an xl0. Or for > > geom to announce ad0s1e even though newbus doesn't know about it. Or for > > devd to handle the introduction of synthetic interfaces such as vlan, tun, > > etc, which aren't visible to newbus. Or for the device driver names and > > interface names to differ (or for a non-one-one mapping, interface > > renames, etc). > > I was thinking about a more generic event posting mechanism, where > modules can register to receive notifications when events came in. > Please use kqueue. We should have 1 eventing mechanism in the kernel. > We should have each event contain: > system (enum/int) > subsystem? (enum/int) > event (enum/int) > device (char[64?]) > additionaldata (void *, size_t combo) > > This means we can make it more extensible, and have a set of well > defined system/subsystem/event triplets, while at the same time, letting > future interfaces expand. > > The subsystem would be used for something like the network stack, which > has multiple subsystems, like interface, routing, connections. > > The advantage is then we can reduce the number of /dev entries that > convey the same information, but have different calling conventions. > > -- > John-Mark Gurney Voice: +1 415 225 5579 > > "All that I will do, has been done, All that I have, has not." > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Wed Sep 17 17:36:12 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 46CBE16A4B3; Wed, 17 Sep 2003 17:36:12 -0700 (PDT) Received: from arginine.spc.org (arginine.spc.org [195.206.69.236]) by mx1.FreeBSD.org (Postfix) with ESMTP id EE69E43FE5; Wed, 17 Sep 2003 17:36:10 -0700 (PDT) (envelope-from bms@spc.org) Received: from localhost (localhost [127.0.0.1]) by arginine.spc.org (Postfix) with ESMTP id 81769653D7; Thu, 18 Sep 2003 01:36:09 +0100 (BST) Received: from arginine.spc.org ([127.0.0.1]) by localhost (arginine.spc.org [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 19534-02; Thu, 18 Sep 2003 01:36:08 +0100 (BST) Received: from saboteur.dek.spc.org (lardystuffer.demon.co.uk [212.228.40.202]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by arginine.spc.org (Postfix) with ESMTP id E548B653D0; Thu, 18 Sep 2003 01:36:01 +0100 (BST) Received: by saboteur.dek.spc.org (Postfix, from userid 1001) id 17BD339; Thu, 18 Sep 2003 01:35:56 +0100 (BST) Date: Thu, 18 Sep 2003 01:35:56 +0100 From: Bruce M Simpson To: Jeff Roberson Message-ID: <20030918003556.GA1025@saboteur.dek.spc.org> References: <20030917210236.GB75714@funkthat.com> <20030917201822.M55626-100000@mail.chesapeake.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030917201822.M55626-100000@mail.chesapeake.net> Organization: SPC cc: bms@spc.org cc: Poul-Henning Kamp cc: Robert Watson cc: freebsd-arch@freebsd.org cc: John-Mark Gurney cc: "M. Warner Losh" Subject: Re: devd limitations / automounting removable storage X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Sep 2003 00:36:12 -0000 On Wed, Sep 17, 2003 at 08:18:49PM -0400, Jeff Roberson wrote: > On Wed, 17 Sep 2003, John-Mark Gurney wrote: > > I was thinking about a more generic event posting mechanism, where > > modules can register to receive notifications when events came in. > > Please use kqueue. We should have 1 eventing mechanism in the kernel. Right now, the way devd/devctl works, it simply polls that device for changes. Interesting. Are you suggesting we ditch /dev/devctl and define event filters instead inside NEWBUS? Assuming kqueue can be made to play with SMP and that we can push Giant out of it this might not be such a bad idea. The routing daemon I've been working on (currently stalled due to other work) uses kqueue for all network and system event dispatch. I'm very happy with kqueue. BMS From owner-freebsd-arch@FreeBSD.ORG Wed Sep 17 18:46:37 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EE77416A4B3 for ; Wed, 17 Sep 2003 18:46:37 -0700 (PDT) Received: from lennier.cc.vt.edu (lennier.cc.vt.edu [198.82.162.213]) by mx1.FreeBSD.org (Postfix) with ESMTP id 122C443FE0 for ; Wed, 17 Sep 2003 18:46:37 -0700 (PDT) (envelope-from ssouhlal@vt.edu) Received: from dagger.cc.vt.edu (IDENT:mirapoint@evil-dagger [10.1.1.11]) by lennier.cc.vt.edu (8.12.8/8.12.8) with ESMTP id h8I1kWxK101761; Wed, 17 Sep 2003 21:46:32 -0400 (EDT) Received: from zZzZ (hc6524a8b.dhcp.vt.edu [198.82.74.139]) by dagger.cc.vt.edu (Mirapoint Messaging Server MOS 3.3.7-GR) with SMTP id BXO33854; Wed, 17 Sep 2003 21:46:30 -0400 (EDT) Date: Wed, 17 Sep 2003 21:46:30 -0400 From: Suleiman Souhlal To: Jeff Roberson Message-Id: <20030917214630.55da364b.ssouhlal@vt.edu> In-Reply-To: <20030917201822.M55626-100000@mail.chesapeake.net> References: <20030917210236.GB75714@funkthat.com> <20030917201822.M55626-100000@mail.chesapeake.net> X-Mailer: Sylpheed version 0.9.3claws (GTK+ 1.2.10; i386-portbld-freebsd5.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit cc: bms@spc.org cc: Poul-Henning Kamp cc: Robert Watson cc: freebsd-arch@freebsd.org cc: John-Mark Gurney cc: "M. Warner Losh" Subject: Re: devd limitations / automounting removable storage X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Sep 2003 01:46:38 -0000 On Wed, 17 Sep 2003 20:18:49 -0400 (EDT) Jeff Roberson wrote: > > Please use kqueue. We should have 1 eventing mechanism in the kernel. > I think that is not a bad idea, considering it should not be too difficult to add new event filters, and that it already works pretty well. Suleiman Souhlal From owner-freebsd-arch@FreeBSD.ORG Wed Sep 17 22:15:20 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4ADC916A4B3; Wed, 17 Sep 2003 22:15:20 -0700 (PDT) Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9F18743FBD; Wed, 17 Sep 2003 22:15:18 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.9/8.12.3) with ESMTP id h8I5F6TX099841; Wed, 17 Sep 2003 23:15:06 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Thu, 18 Sep 2003 06:13:25 -0600 (MDT) Message-Id: <20030918.061325.68161076.imp@bsdimp.com> To: gurney_j@efn.org From: "M. Warner Losh" In-Reply-To: <20030917210236.GB75714@funkthat.com> References: <30912.1063828673@critter.freebsd.dk> <20030917210236.GB75714@funkthat.com> X-Mailer: Mew version 2.1 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: bms@spc.org cc: phk@phk.freebsd.dk cc: rwatson@freebsd.org cc: freebsd-arch@freebsd.org Subject: Re: devd limitations / automounting removable storage X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Sep 2003 05:15:20 -0000 In message: <20030917210236.GB75714@funkthat.com> John-Mark Gurney writes: : system (enum/int) : subsystem? (enum/int) : event (enum/int) : device (char[64?]) : additionaldata (void *, size_t combo) I specifically made the protocol used extensible. It is an ASCII stream. Don't break that. :-) : This means we can make it more extensible, and have a set of well : defined system/subsystem/event triplets, while at the same time, letting : future interfaces expand. Already is perfectly extensible. Warner From owner-freebsd-arch@FreeBSD.ORG Wed Sep 17 22:18:59 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6B98C16A4B3; Wed, 17 Sep 2003 22:18:59 -0700 (PDT) Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 67EB643FB1; Wed, 17 Sep 2003 22:18:58 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.9/8.12.3) with ESMTP id h8I5ImTX099857; Wed, 17 Sep 2003 23:18:49 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Thu, 18 Sep 2003 06:17:07 -0600 (MDT) Message-Id: <20030918.061707.115654192.imp@bsdimp.com> To: bms@spc.org From: "M. Warner Losh" In-Reply-To: <20030918003556.GA1025@saboteur.dek.spc.org> References: <20030917210236.GB75714@funkthat.com> <20030917201822.M55626-100000@mail.chesapeake.net> <20030918003556.GA1025@saboteur.dek.spc.org> X-Mailer: Mew version 2.1 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: rwatson@freebsd.org cc: phk@phk.freebsd.dk cc: gurney_j@efn.org cc: freebsd-arch@freebsd.org Subject: Re: devd limitations / automounting removable storage X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Sep 2003 05:18:59 -0000 In message: <20030918003556.GA1025@saboteur.dek.spc.org> Bruce M Simpson writes: : On Wed, Sep 17, 2003 at 08:18:49PM -0400, Jeff Roberson wrote: : > On Wed, 17 Sep 2003, John-Mark Gurney wrote: : > > I was thinking about a more generic event posting mechanism, where : > > modules can register to receive notifications when events came in. : > : > Please use kqueue. We should have 1 eventing mechanism in the kernel. : : Right now, the way devd/devctl works, it simply polls that device for changes. No. devctl gets an event queued to its read channel. devd then reads it. That's different than polling for changes. : Interesting. Are you suggesting we ditch /dev/devctl and define event : filters instead inside NEWBUS? Assuming kqueue can be made to play with : SMP and that we can push Giant out of it this might not be such a bad idea. kqueue can report events. It can't transport arbitrary data, which is what is needed here. Warner From owner-freebsd-arch@FreeBSD.ORG Wed Sep 17 22:54:30 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 032E616A4B3; Wed, 17 Sep 2003 22:54:30 -0700 (PDT) Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 76BB943F3F; Wed, 17 Sep 2003 22:54:28 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.9/8.12.3) with ESMTP id h8I5sRTX000189; Wed, 17 Sep 2003 23:54:27 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Thu, 18 Sep 2003 06:52:59 -0600 (MDT) Message-Id: <20030918.065259.112623652.imp@bsdimp.com> To: rwatson@freebsd.org From: "M. Warner Losh" In-Reply-To: References: <30912.1063828673@critter.freebsd.dk> X-Mailer: Mew version 2.1 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: bms@spc.org cc: phk@phk.freebsd.dk cc: freebsd-arch@freebsd.org Subject: Re: devd limitations / automounting removable storage X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Sep 2003 05:54:30 -0000 In message: Robert Watson writes: : We might skip the devfs layer since it's probably implicit in the : completion of ifnet_attach() and disk_create(), but it's worth thinking : about. The thing about implicit stuff is that it move knowledge about what is created in the driver into userland. This isn't necessarily a good thing. For most of the examples talked about, this isn't an issue, but what about stranger hardware that creates many devices? : This would allow ifconfig commands to be issued once the network : device is available, rather than once newbus has attached an xl0. network interfaces are available for all drivers in the tree after attach finishes. However, interfaces that aren't an exact mirror of the hardware don't show up. tun0, for example. Also, network interfaces are in a different namespace than newbus device names. : Or for : geom to announce ad0s1e even though newbus doesn't know about it. Be careful here. You are confusing two different name spaces. dev_t is a third namespace in the kernel. sysctl is another one and of course file systems. You'd want to know that devfs events have happened in this case. GEOM likely doesn't need to get into the mix. And you can likely do that with a kqueue on the /dev directory. GEOM lives in the dev_t name space (right?) : Or for : devd to handle the introduction of synthetic interfaces such as vlan, tun, : etc, which aren't visible to newbus. Or for the device driver names and : interface names to differ (or for a non-one-one mapping, interface : renames, etc). Yes. devd started out life as a device_t only thing. However, it is better viewed as more generalized event dispatcher that currently only understand events from device_t entities. I was going to point out that there's not a connection between the device_t name and the dev_t name, except by convention. Even when there is a connection, it can be weird. Device foo0 might create /dev/foo0_error and /dev/foo0_data, or other such perverse things. Warner From owner-freebsd-arch@FreeBSD.ORG Wed Sep 17 23:17:17 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 64F4D16A4B3; Wed, 17 Sep 2003 23:17:17 -0700 (PDT) Received: from phk.freebsd.dk (phk.freebsd.dk [212.242.86.175]) by mx1.FreeBSD.org (Postfix) with ESMTP id 430DE43FDD; Wed, 17 Sep 2003 23:17:16 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by phk.freebsd.dk (8.12.8/8.12.8) with ESMTP id h8I6HBQ2090102; Thu, 18 Sep 2003 06:17:11 GMT (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h8I6H96f036393; Thu, 18 Sep 2003 08:17:10 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: "M. Warner Losh" From: "Poul-Henning Kamp" In-Reply-To: Your message of "Thu, 18 Sep 2003 06:52:59 MDT." <20030918.065259.112623652.imp@bsdimp.com> Date: Thu, 18 Sep 2003 08:17:09 +0200 Message-ID: <36392.1063865829@critter.freebsd.dk> cc: bms@spc.org cc: rwatson@freebsd.org cc: freebsd-arch@freebsd.org Subject: Re: devd limitations / automounting removable storage X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Sep 2003 06:17:17 -0000 In message <20030918.065259.112623652.imp@bsdimp.com>, "M. Warner Losh" writes: >You'd want to know that devfs events have happened in this case. GEOM >likely doesn't need to get into the mix. And you can likely do that >with a kqueue on the /dev directory. > >GEOM lives in the dev_t name space (right?) Generally speaking: yes. All "providers" in GEOM end up in /dev, so in that sense it is true. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Thu Sep 18 00:04:47 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 917EA16A4C0 for ; Thu, 18 Sep 2003 00:04:47 -0700 (PDT) Received: from mail.broadpark.no (mail.broadpark.no [217.13.4.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 87C2143FB1 for ; Thu, 18 Sep 2003 00:04:46 -0700 (PDT) (envelope-from des@des.no) Received: from smtp.des.no (37.80-203-228.nextgentel.com [80.203.228.37]) by mail.broadpark.no (Postfix) with ESMTP id 803A678E82; Thu, 18 Sep 2003 09:04:45 +0200 (MEST) Received: by smtp.des.no (Pony Express, from userid 666) id 4EEA8998CD; Thu, 18 Sep 2003 09:04:45 +0200 (CEST) Received: from dwp.des.no (dwp.des.no [10.0.0.4]) by smtp.des.no (Pony Express) with ESMTP id 67A8E998CB; Thu, 18 Sep 2003 09:04:41 +0200 (CEST) Received: by dwp.des.no (Postfix, from userid 2602) id 44FECB84A; Thu, 18 Sep 2003 09:04:41 +0200 (CEST) To: Marcel Moolenaar References: <20030917065127.GB4261@athlon.pn.xcllnt.net> <20030917073900.GG39788@funkthat.com> <20030917080127.GB16024@dhcp01.pn.xcllnt.net> <20030917180421.GH39788@funkthat.com> <20030917184241.GA18166@dhcp01.pn.xcllnt.net> From: des@des.no (Dag-Erling =?iso-8859-1?q?Sm=F8rgrav?=) Date: Thu, 18 Sep 2003 09:04:41 +0200 In-Reply-To: <20030917184241.GA18166@dhcp01.pn.xcllnt.net> (Marcel Moolenaar's message of "Wed, 17 Sep 2003 11:42:41 -0700") Message-ID: User-Agent: Gnus/5.090024 (Oort Gnus v0.24) Emacs/21.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, hits=-3.0 required=8.0 tests=EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT,REFERENCES, REPLY_WITH_QUOTES,USER_AGENT_GNUS_UA version=2.55 X-Spam-Level: X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp) cc: arch@freebsd.org Subject: Re: make(1): adding sort modifiers X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Sep 2003 07:04:47 -0000 Marcel Moolenaar writes: > Correct. We should keep lists sorted. It's when we need to combine > multiple lists that we need a hand. Having multiple platforms and > a busload of NO_way options is making it hard to get the final list > sorted. If make(1) can give us a hand with that, then we can also > keep an eye on maintainability and readability of our makefiles. Yep, the module build is my favorite bogeyman in this respect. I can never tell how far along it is because I can never remember which module is in which sequence (I believe there are three separate sequences on i386, each of wich is sorted). DES --=20 Dag-Erling Sm=F8rgrav - des@des.no From owner-freebsd-arch@FreeBSD.ORG Thu Sep 18 03:32:13 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9B55E16A4B3; Thu, 18 Sep 2003 03:32:13 -0700 (PDT) Received: from mail.qubesoft.com (gate.qubesoft.com [217.169.36.34]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3BE8943FBD; Thu, 18 Sep 2003 03:32:12 -0700 (PDT) (envelope-from dfr@nlsystems.com) Received: from bluebottle.qubesoft.com (bluebottle.qubesoft.com [192.168.1.2]) by mail.qubesoft.com (8.12.9/8.12.9) with ESMTP id h8IAVgSO007823; Thu, 18 Sep 2003 11:31:42 +0100 (BST) (envelope-from dfr@nlsystems.com) Received: from builder02.qubesoft.com (builder02.qubesoft.com [192.168.1.8]) h8IAVaTj015790; Thu, 18 Sep 2003 11:31:36 +0100 (BST) (envelope-from dfr@nlsystems.com) From: Doug Rabson To: Robert Watson In-Reply-To: References: Content-Type: text/plain Message-Id: <1063881095.12179.5.camel@builder02.qubesoft.com> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 18 Sep 2003 11:31:36 +0100 Content-Transfer-Encoding: 7bit cc: bms@spc.org cc: Poul-Henning Kamp cc: "M. Warner Losh" cc: freebsd-arch@freebsd.org Subject: Re: devd limitations / automounting removable storage X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Sep 2003 10:32:13 -0000 On Wed, 2003-09-17 at 21:19, Robert Watson wrote: > On Wed, 17 Sep 2003, Poul-Henning Kamp wrote: > > > In message , John Baldwin writes: > > > > >Maybe have GEOM send events when mountable entities show up? > > > > There is only one question to figure out: Should it happen at the > > bottom or the top of GEOM ? > > > > At the bottom, your CF card would result in a single event on /dev/ad4, > > at the top you'd likely get two events, one for /dev/ad4 and one for > > /dev/ad4s1 (or whatever other gadgets geom autodiscovers). > > > > Apart from that, it's just a matter of telling me how to send the > > event... > > This is a very similar concern to the question I raised at BSDCon about > distinguishing network interfaces at the newbus and network service > levels. My temptation would be to assign a "layer" as well as a device > name to devices when they are announced. I.e., > > Layer Device Meaning > newbus xl0 A device named xl0 is now available > devfs net/xl0 A device node named net/xl0 is now available > ifnet xl0 A network interface named xl0 is now available > newbus ad0 A device named ad0 is now available > devfs ad0 A device node named ad0 is now available > geom ad0 A storage device named ad0 is now available > > We might skip the devfs layer since it's probably implicit in the > completion of ifnet_attach() and disk_create(), but it's worth thinking > about. This would allow ifconfig commands to be issued once the network > device is available, rather than once newbus has attached an xl0. Or for > geom to announce ad0s1e even though newbus doesn't know about it. Or for > devd to handle the introduction of synthetic interfaces such as vlan, tun, > etc, which aren't visible to newbus. Or for the device driver names and > interface names to differ (or for a non-one-one mapping, interface > renames, etc). I've thought for a long time now that the right way to do this is to extend the newbus device tree much further down the hierarchy than it currently does. Currently the tree stops at the CAM/ATA controller. Both of those systems then use their own custom hand-crafted wheels to probe for and attach their attached drives. After finding the drives, we hand them over to yet another custom hand-crafted wheel (geom) to find the partitions. Surely the right thing would be to use the same wheel (newbus) for all the probing, driver auction, device attachment jobs in the kernel. That would seemlessly allow devd to receive device notification events for geom's leaf partitions in exactly the same way that it receives all other notification events. From owner-freebsd-arch@FreeBSD.ORG Thu Sep 18 04:44:04 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 92DCA16A4B3; Thu, 18 Sep 2003 04:44:04 -0700 (PDT) Received: from exc-1.cc.CyberCity.dk (esplanaden.cybercity.dk [212.242.40.114]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3A17043FCB; Thu, 18 Sep 2003 04:44:03 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk ([172.16.7.254]) by exc-1.cc.CyberCity.dk over TLS secured channel with Microsoft SMTPSVC(5.0.2195.6713); Thu, 18 Sep 2003 13:44:01 +0200 Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h8IBhjVT002560; Thu, 18 Sep 2003 13:43:50 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: Doug Rabson From: "Poul-Henning Kamp" In-Reply-To: Your message of "18 Sep 2003 11:31:36 BST." <1063881095.12179.5.camel@builder02.qubesoft.com> Date: Thu, 18 Sep 2003 13:43:45 +0200 Message-ID: <2559.1063885425@critter.freebsd.dk> X-OriginalArrivalTime: 18 Sep 2003 11:44:01.0779 (UTC) FILETIME=[27A23430:01C37DDA] cc: bms@spc.org cc: Robert Watson cc: "M. Warner Losh" cc: freebsd-arch@freebsd.org Subject: Re: devd limitations / automounting removable storage X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Sep 2003 11:44:04 -0000 In message <1063881095.12179.5.camel@builder02.qubesoft.com>, Doug Rabson write s: >I've thought for a long time now that the right way to do this is to >extend the newbus device tree much further down the hierarchy than it currently does. Currently the tree stops at the CAM/ATA controller. Both >of those systems then use their own custom hand-crafted wheels to probe >for and attach their attached drives. After finding the drives, we hand >them over to yet another custom hand-crafted wheel (geom) to find the >partitions. > >Surely the right thing would be to use the same wheel (newbus) for all >the probing, driver auction, device attachment jobs in the kernel. That >would seemlessly allow devd to receive device notification events for >geom's leaf partitions in exactly the same way that it receives all >other notification events. I'm sorry Doug, I don't belive in "one size fits all" because it invariably means that it fits nobody at all. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-arch@FreeBSD.ORG Thu Sep 18 07:40:26 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EA74116A4BF; Thu, 18 Sep 2003 07:40:26 -0700 (PDT) Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7615343FE5; Thu, 18 Sep 2003 07:40:24 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.9/8.12.3) with ESMTP id h8IEeNTX004265; Thu, 18 Sep 2003 08:40:23 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Thu, 18 Sep 2003 08:40:25 -0600 (MDT) Message-Id: <20030918.084025.71552256.imp@bsdimp.com> To: dfr@nlsystems.com From: "M. Warner Losh" In-Reply-To: <1063881095.12179.5.camel@builder02.qubesoft.com> References: <1063881095.12179.5.camel@builder02.qubesoft.com> X-Mailer: Mew version 2.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: bms@spc.org cc: phk@phk.freebsd.dk cc: rwatson@freebsd.org cc: freebsd-arch@freebsd.org Subject: Re: devd limitations / automounting removable storage X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Sep 2003 14:40:27 -0000 In message: <1063881095.12179.5.camel@builder02.qubesoft.com> Doug Rabson writes: : Surely the right thing would be to use the same wheel (newbus) for all : the probing, driver auction, device attachment jobs in the kernel. That : would seemlessly allow devd to receive device notification events for : geom's leaf partitions in exactly the same way that it receives all : other notification events. I tend to agree with the CAM/ata controller bit. Both were written before newbus was well integrated into the tree. It makes no sense that fdc has an fd newbus child but ata doesn't have an ad child. However, having said that, I think we do need additional event types because we're doing with different name spaces. But those events might be in devd rather than devctl given that we can get those events in userland in other ways if my investigations are OK. Warner From owner-freebsd-arch@FreeBSD.ORG Thu Sep 18 07:49:20 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 860DA16A4B3 for ; Thu, 18 Sep 2003 07:49:20 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id AF6F843FD7 for ; Thu, 18 Sep 2003 07:49:19 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.9/8.12.9) with ESMTP id h8IElmN6060955; Thu, 18 Sep 2003 10:47:48 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)h8IElmpq060952; Thu, 18 Sep 2003 10:47:48 -0400 (EDT) Date: Thu, 18 Sep 2003 10:47:48 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: "M. Warner Losh" In-Reply-To: <20030918.084025.71552256.imp@bsdimp.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: bms@spc.org cc: phk@phk.freebsd.dk cc: freebsd-arch@freebsd.org Subject: Re: devd limitations / automounting removable storage X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Sep 2003 14:49:20 -0000 On Thu, 18 Sep 2003, M. Warner Losh wrote: > In message: <1063881095.12179.5.camel@builder02.qubesoft.com> > Doug Rabson writes: > : Surely the right thing would be to use the same wheel (newbus) for all > : the probing, driver auction, device attachment jobs in the kernel. That > : would seemlessly allow devd to receive device notification events for > : geom's leaf partitions in exactly the same way that it receives all > : other notification events. > > I tend to agree with the CAM/ata controller bit. Both were written > before newbus was well integrated into the tree. It makes no sense that > fdc has an fd newbus child but ata doesn't have an ad child. > > However, having said that, I think we do need additional event types > because we're doing with different name spaces. But those events might > be in devd rather than devctl given that we can get those events in > userland in other ways if my investigations are OK. For ifnet events, we can use routing sockets. I don't know that we have GEOM events as yet. One reason to separately handle GEOM from devfs would be that GEOM "objects" tend to be storage devices or related notions, whereas devfs entries could be any number of things. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Network Associates Laboratories From owner-freebsd-arch@FreeBSD.ORG Thu Sep 18 11:15:08 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EBDA016A4B3; Thu, 18 Sep 2003 11:15:08 -0700 (PDT) Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1226143FDD; Thu, 18 Sep 2003 11:15:08 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.9/8.12.3) with ESMTP id h8IIF6TX006243; Thu, 18 Sep 2003 12:15:06 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Thu, 18 Sep 2003 12:15:07 -0600 (MDT) Message-Id: <20030918.121507.32721201.imp@bsdimp.com> To: rwatson@freebsd.org From: "M. Warner Losh" In-Reply-To: References: <20030918.084025.71552256.imp@bsdimp.com> X-Mailer: Mew version 2.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: bms@spc.org cc: phk@phk.freebsd.dk cc: freebsd-arch@freebsd.org Subject: Re: devd limitations / automounting removable storage X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Sep 2003 18:15:09 -0000 In message: Robert Watson writes: : For ifnet events, we can use routing sockets. I don't know that we have : GEOM events as yet. One reason to separately handle GEOM from devfs would : be that GEOM "objects" tend to be storage devices or related notions, : whereas devfs entries could be any number of things. While this is true, one can ask a /dev entry what kind of object it is. Since one can do that, one can construct filters that will only do things for storage objects. I worry about putting these new event streams at the wrong level and/or having too many of them making it hard to know what the appropriate level/event to do something at is. Warner From owner-freebsd-arch@FreeBSD.ORG Thu Sep 18 11:18:59 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4B4A816A4B3 for ; Thu, 18 Sep 2003 11:18:59 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 573A343FBD for ; Thu, 18 Sep 2003 11:18:58 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.9/8.12.9) with ESMTP id h8IIIsgL002174; Thu, 18 Sep 2003 14:18:54 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)h8IIIsoC002171; Thu, 18 Sep 2003 14:18:54 -0400 (EDT) Date: Thu, 18 Sep 2003 14:18:54 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: "M. Warner Losh" In-Reply-To: <20030918.121507.32721201.imp@bsdimp.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: bms@spc.org cc: phk@phk.freebsd.dk cc: freebsd-arch@freebsd.org Subject: Re: devd limitations / automounting removable storage X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Sep 2003 18:18:59 -0000 On Thu, 18 Sep 2003, M. Warner Losh wrote: > In message: > Robert Watson writes: > : For ifnet events, we can use routing sockets. I don't know that we have > : GEOM events as yet. One reason to separately handle GEOM from devfs would > : be that GEOM "objects" tend to be storage devices or related notions, > : whereas devfs entries could be any number of things. > > While this is true, one can ask a /dev entry what kind of object it is. > Since one can do that, one can construct filters that will only do > things for storage objects. Opening a device to ask it what it might be is generally a bad idea -- you can block other consumers from using the device (and related devices), cause a variety side-effects, etc. Also, I'm not clear that you can get a useful result using open/fstat/stat/ioctl to figure out what something is without apriori knowledge of device numbers, and even then the utility is limited. If you have a network layer announcement "Hey, this interface arrived", then there's no question that it's a network interface. > I worry about putting these new event streams at the wrong level and/or > having too many of them making it hard to know what the appropriate > level/event to do something at is. Agreed. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Network Associates Laboratories From owner-freebsd-arch@FreeBSD.ORG Thu Sep 18 11:23:44 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 59E6C16A4B3; Thu, 18 Sep 2003 11:23:44 -0700 (PDT) Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4D86743F85; Thu, 18 Sep 2003 11:23:41 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.9/8.12.3) with ESMTP id h8IINeTX006396; Thu, 18 Sep 2003 12:23:40 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Thu, 18 Sep 2003 12:23:42 -0600 (MDT) Message-Id: <20030918.122342.129340232.imp@bsdimp.com> To: rwatson@freebsd.org From: "M. Warner Losh" In-Reply-To: References: <20030918.121507.32721201.imp@bsdimp.com> X-Mailer: Mew version 2.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: bms@spc.org cc: phk@phk.freebsd.dk cc: freebsd-arch@freebsd.org Subject: Re: devd limitations / automounting removable storage X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Sep 2003 18:23:44 -0000 In message: Robert Watson writes: : : On Thu, 18 Sep 2003, M. Warner Losh wrote: : : > In message: : > Robert Watson writes: : > : For ifnet events, we can use routing sockets. I don't know that we have : > : GEOM events as yet. One reason to separately handle GEOM from devfs would : > : be that GEOM "objects" tend to be storage devices or related notions, : > : whereas devfs entries could be any number of things. : > : > While this is true, one can ask a /dev entry what kind of object it is. : > Since one can do that, one can construct filters that will only do : > things for storage objects. : : Opening a device to ask it what it might be is generally a bad idea -- you : can block other consumers from using the device (and related devices), : cause a variety side-effects, etc. Also, I'm not clear that you can get a : useful result using open/fstat/stat/ioctl to figure out what something is : without apriori knowledge of device numbers, and even then the utility is : limited. If you have a network layer announcement "Hey, this interface : arrived", then there's no question that it's a network interface. You don't need to open the device to know what it is in many cases. You know you have a potential mount point with a simple stat of the device. If you open the device, there's nothing more than you can learn from it than you can from stat, unless you go the "I'm going to try this non-destrictuve ioctl and hope for the best" approach. Tape drives are a big category of side effects from open. Few other devices have any meaningful side effects from opening them, but they do exist. Warner From owner-freebsd-arch@FreeBSD.ORG Thu Sep 18 11:50:03 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8AF2B16A4BF; Thu, 18 Sep 2003 11:50:03 -0700 (PDT) Received: from ebb.errno.com (ebb.errno.com [66.127.85.87]) by mx1.FreeBSD.org (Postfix) with ESMTP id C4A1F43FDF; Thu, 18 Sep 2003 11:50:02 -0700 (PDT) (envelope-from sam@errno.com) Received: from melange.errno.com (melange.errno.com [66.127.85.82]) (authenticated bits=0) by ebb.errno.com (8.12.9/8.12.9) with ESMTP id h8IInn17036640 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Thu, 18 Sep 2003 11:49:53 -0700 (PDT) (envelope-from sam@errno.com) Date: Thu, 18 Sep 2003 11:49:49 -0700 From: Sam Leffler To: "M. Warner Losh" , rwatson@freebsd.org Message-ID: <504726878.1063885789@melange.errno.com> In-Reply-To: <20030918.122342.129340232.imp@bsdimp.com> References: <20030918.121507.32721201.imp@bsdimp.com> <20030918.122342.129340232.imp@bsdimp.com> X-Mailer: Mulberry/3.0.3 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline cc: bms@spc.org cc: phk@phk.freebsd.dk cc: freebsd-arch@freebsd.org Subject: Re: devd limitations / automounting removable storage X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Sep 2003 18:50:03 -0000 > Few other > devices have any meaningful side effects from opening them, but they > do exist. Typically any removable media (floppies in particular). Sam From owner-freebsd-arch@FreeBSD.ORG Thu Sep 18 14:22:51 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DC6A616A4B3; Thu, 18 Sep 2003 14:22:51 -0700 (PDT) Received: from herring.nlsystems.com (mailgate.nlsystems.com [80.177.232.242]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4F91443F75; Thu, 18 Sep 2003 14:22:50 -0700 (PDT) (envelope-from dfr@nlsystems.com) Received: from [10.0.0.2] (herring.nlsystems.com [10.0.0.2]) by herring.nlsystems.com (8.12.9/8.12.8) with ESMTP id h8ILM8W2025271; Thu, 18 Sep 2003 22:22:09 +0100 (BST) (envelope-from dfr@nlsystems.com) From: Doug Rabson To: Poul-Henning Kamp In-Reply-To: <2559.1063885425@critter.freebsd.dk> References: <2559.1063885425@critter.freebsd.dk> Content-Type: text/plain Message-Id: <1063920128.18459.8.camel@herring.nlsystems.com> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 18 Sep 2003 21:22:08 +0000 Content-Transfer-Encoding: 7bit X-Spam-Status: No, hits=-4.9 required=5.0 tests=EMAIL_ATTRIBUTION,IN_REP_TO,QUOTED_EMAIL_TEXT,REFERENCES, REPLY_WITH_QUOTES,USER_AGENT_XIMIAN version=2.55 X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp) cc: bms@spc.org cc: Robert Watson cc: "M. Warner Losh" cc: freebsd-arch@FreeBSD.org Subject: Re: devd limitations / automounting removable storage X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Sep 2003 21:22:52 -0000 On Thu, 2003-09-18 at 11:43, Poul-Henning Kamp wrote: > In message <1063881095.12179.5.camel@builder02.qubesoft.com>, Doug Rabson write > s: > > >I've thought for a long time now that the right way to do this is to > >extend the newbus device tree much further down the hierarchy than it > currently does. Currently the tree stops at the CAM/ATA controller. Both > >of those systems then use their own custom hand-crafted wheels to probe > >for and attach their attached drives. After finding the drives, we hand > >them over to yet another custom hand-crafted wheel (geom) to find the > >partitions. > > > >Surely the right thing would be to use the same wheel (newbus) for all > >the probing, driver auction, device attachment jobs in the kernel. That > >would seemlessly allow devd to receive device notification events for > >geom's leaf partitions in exactly the same way that it receives all > >other notification events. > > I'm sorry Doug, I don't belive in "one size fits all" because it > invariably means that it fits nobody at all. Well in this case, its a size which seems to fit virtually everything else in the system pretty well. I remember what it was like before when every different kind of driver (pci, eisa, isa, whatever) was written in a completely different incompatible undocumented style and I happen to think that the new way works pretty well. I really doubt that the partition to driver matching system of geom or the device to driver matching system in ATA does anything very different from any other part of the device tree. From owner-freebsd-arch@FreeBSD.ORG Fri Sep 19 15:59:23 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B59FB16A4B3 for ; Fri, 19 Sep 2003 15:59:23 -0700 (PDT) Received: from ebb.errno.com (ebb.errno.com [66.127.85.87]) by mx1.FreeBSD.org (Postfix) with ESMTP id ED9AB43FDF for ; Fri, 19 Sep 2003 15:59:22 -0700 (PDT) (envelope-from sam@errno.com) Received: from melange.errno.com (melange.errno.com [66.127.85.82]) (authenticated bits=0) by ebb.errno.com (8.12.9/8.12.9) with ESMTP id h8JMxG0x011223 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO) for ; Fri, 19 Sep 2003 15:59:20 -0700 (PDT) (envelope-from sam@errno.com) Date: Fri, 19 Sep 2003 15:59:15 -0700 From: Sam Leffler To: freebsd-arch@freebsd.org Message-ID: <606095639.1063987155@melange.errno.com> X-Mailer: Mulberry/3.0.3 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline Subject: interrupt latency and driver locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Sep 2003 22:59:23 -0000 I enabled MUTEX_PROFILING on a fast machine I've got setup as a firewall/router. The machine had an fxp device on one side and an em on the other. I then ran a bunch of netperf tests through the machine. Unfortunately for the moment I'm stuck with 100baseT on the fxp side of the machine so my tests were all limited by the media. But one thing stuck out: max total count avg name 282 786 3 262 kern/kern_exec.c:853 (vm page queue mute 775 7450 33 225 kern/vfs_subr.c:3218 (mntvnode) 1119 2789 15 185 kern/kern_fork.c:218 (Giant) 222 2344 14 167 vm/uma_core.c:1186 (UMA lock) 460 2972 19 156 kern/kern_exit.c:282 (vm page queue mute 508 2415 19 127 kern/kern_exit.c:101 (Giant) 194 1801 16 112 kern/kern_exec.c:261 (Giant) 332 31113 277 112 dev/fxp/if_fxp.c:1798 (fxp0) 2435 31848 366 87 kern/kern_sysctl.c:1210 (Giant) 901 5306 66 80 vm/vm_map.c:1256 (Giant) 77 77 1 77 kern/uipc_syscalls.c:472 (Giant) 182 2901 39 74 vm/vm_kern.c:328 (system map) 507 841090 12508 67 dev/random/yarrow.c:172 (random reseed) 7005 68335 1136 60 kern/kern_intr.c:533 (Giant) 66 964 16 60 vm/vm_map.c:2241 (system map) 88 7471 139 53 dev/em/if_em.c:1531 (em0) 881 8032 191 42 i386/i386/trap.c:997 (Giant) 591 10568722 271157 38 net/netisr.c:215 (netisr lock) 38 292 8 36 vm/vm_fault.c:997 (Giant) Note that the fxp driver holds its driver lock for an average 112 us in one spot. This turns out to be in fxp_tick and the likely culprit is the call to mii_tick which is done with the lock held. Since there's only one lock in the driver this means interrupts are blocked during this time which is likely to do bad things. (The em driver comparison has a similar bottleneck though it's <1/2 as long.) I'm not sure if it's simple to move MII operations outside of the driver lock but it might be worth investigating. Sam From owner-freebsd-arch@FreeBSD.ORG Sat Sep 20 09:48:27 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 19D9316A4B3 for ; Sat, 20 Sep 2003 09:48:27 -0700 (PDT) Received: from blake.polstra.com (mail.polstra.com [206.213.73.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id B44BD43FEC for ; Sat, 20 Sep 2003 09:48:25 -0700 (PDT) (envelope-from jdp@polstra.com) Received: from strings.polstra.com (strings.polstra.com [206.213.73.20]) by blake.polstra.com (8.12.9/8.12.9) with ESMTP id h8KGmKIL015081; Sat, 20 Sep 2003 09:48:20 -0700 (PDT) (envelope-from jdp@polstra.com) Message-ID: X-Mailer: XFMail 1.5.4 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <606095639.1063987155@melange.errno.com> Date: Sat, 20 Sep 2003 09:48:20 -0700 (PDT) From: John Polstra To: Sam Leffler X-Bogosity: No, tests=bogofilter, spamicity=0.499998, version=0.14.5 cc: freebsd-arch@freebsd.org Subject: RE: interrupt latency and driver locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Sep 2003 16:48:27 -0000 On 19-Sep-2003 Sam Leffler wrote: > I enabled MUTEX_PROFILING on a fast machine I've got setup as a > firewall/router. The machine had an fxp device on one side and an em on > the other. I then ran a bunch of netperf tests through the machine. > Unfortunately for the moment I'm stuck with 100baseT on the fxp side of the > machine so my tests were all limited by the media. But one thing stuck out: [...] > Note that the fxp driver holds its driver lock for an average 112 us in one > spot. This turns out to be in fxp_tick and the likely culprit is the call > to mii_tick which is done with the lock held. Agreed. The MII calls usually have lots of DELAY() calls in them. > I'm not sure if it's simple to move MII operations outside of the driver > lock but it might be worth investigating. It seems like it ought to be possible to at least lock at a finer grain for those calls. The driver lock is held because the MII routines call back into the driver and use it to do the bit-twiddling necessary to shift data into and out of the PHY registers via the PHY's serial interface. That's normally pretty independent of anything else that's going on in the chip, so it could probably be done under a separate lock. Another idea would be to substitute something else for DELAY() calls of more than a few microseconds. I am not very up-to-date on -current these days, but as far as I know DELAY() is still a spin-wait. If the delay is for more than a few microseconds it would probably be better to switch to a different runnable thread and come back later. In fact, CPU speeds may have reached the point where that is always the right thing to do. Finally, the entire mii_tick() nonsense could be thrown out for many modern network adapters. Both the Broadcom and Intel gigabit chips are capable of interrupting on interesting link events, so there is no need at all to poll the PHY every second. That doesn't appear to be true of the fxp devices, though. They need to be polled. Still, you can avoid the expensive PHY reads and writes most of the time. For example, if you have received any packets on the interface in the past second, you can assume that link is good and never touch the PHY at all. John From owner-freebsd-arch@FreeBSD.ORG Sat Sep 20 10:09:19 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7029E16A4B3 for ; Sat, 20 Sep 2003 10:09:19 -0700 (PDT) Received: from xorpc.icir.org (xorpc.icir.org [192.150.187.68]) by mx1.FreeBSD.org (Postfix) with ESMTP id 828F643FFD for ; Sat, 20 Sep 2003 10:09:18 -0700 (PDT) (envelope-from rizzo@xorpc.icir.org) Received: from xorpc.icir.org (localhost [127.0.0.1]) by xorpc.icir.org (8.12.9p1/8.12.3) with ESMTP id h8KH9Bsd056568; Sat, 20 Sep 2003 10:09:11 -0700 (PDT) (envelope-from rizzo@xorpc.icir.org) Received: (from rizzo@localhost) by xorpc.icir.org (8.12.9p1/8.12.3/Submit) id h8KH9BCV056567; Sat, 20 Sep 2003 10:09:11 -0700 (PDT) (envelope-from rizzo) Date: Sat, 20 Sep 2003 10:09:11 -0700 From: Luigi Rizzo To: John Polstra Message-ID: <20030920100911.B55993@xorpc.icir.org> References: <606095639.1063987155@melange.errno.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: ; from jdp@polstra.com on Sat, Sep 20, 2003 at 09:48:20AM -0700 cc: Sam Leffler cc: freebsd-arch@freebsd.org Subject: Re: interrupt latency and driver locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Sep 2003 17:09:19 -0000 On Sat, Sep 20, 2003 at 09:48:20AM -0700, John Polstra wrote: ... > > Note that the fxp driver holds its driver lock for an average 112 us in one > > spot. This turns out to be in fxp_tick and the likely culprit is the call > > to mii_tick which is done with the lock held. > > Agreed. The MII calls usually have lots of DELAY() calls in them. > > > I'm not sure if it's simple to move MII operations outside of the driver > > lock but it might be worth investigating. > > It seems like it ought to be possible to at least lock at a finer > grain for those calls. The driver lock is held because the MII the main problem, as i see it, is that when there are PHY events you still need to do some expensive work while holding a lock that blocks interrupts, with very bad impact on the worst-case response of the system. One option in these cases could be to do the following: 1. disable the offending device while doing the expensive work (e.g. reconfigure the PHY); 2. acquire a different lock (lock2) on the device; 3. release the lock that prevents (other) interrupts to be delivered 4. do the thing, hoever long it takes 5. re-enable the device 6. release the lock2 this would work fine for PHY events because the interface is presumably not in a state to process traffic anyways, so disabling it should have no bad side effects; maybe other drivers (e.g. parallel ports, or devices where you have to do bit-banging to move data without too many timing constraints) could use a similar approach. cheers luigi > routines call back into the driver and use it to do the bit-twiddling > necessary to shift data into and out of the PHY registers via the > PHY's serial interface. That's normally pretty independent of > anything else that's going on in the chip, so it could probably be > done under a separate lock. > > Another idea would be to substitute something else for DELAY() calls > of more than a few microseconds. I am not very up-to-date on -current > these days, but as far as I know DELAY() is still a spin-wait. If > the delay is for more than a few microseconds it would probably be > better to switch to a different runnable thread and come back later. > In fact, CPU speeds may have reached the point where that is always > the right thing to do. > > Finally, the entire mii_tick() nonsense could be thrown out for many > modern network adapters. Both the Broadcom and Intel gigabit chips > are capable of interrupting on interesting link events, so there is no > need at all to poll the PHY every second. That doesn't appear to be > true of the fxp devices, though. They need to be polled. Still, you > can avoid the expensive PHY reads and writes most of the time. For > example, if you have received any packets on the interface in the past > second, you can assume that link is good and never touch the PHY at > all. > > John > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" From owner-freebsd-arch@FreeBSD.ORG Sat Sep 20 13:00:58 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0248616A4B3 for ; Sat, 20 Sep 2003 13:00:58 -0700 (PDT) Received: from blake.polstra.com (mail.polstra.com [206.213.73.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id F3C1C43FB1 for ; Sat, 20 Sep 2003 13:00:56 -0700 (PDT) (envelope-from jdp@polstra.com) Received: from strings.polstra.com (strings.polstra.com [206.213.73.20]) by blake.polstra.com (8.12.9/8.12.9) with ESMTP id h8KK0sIL015955; Sat, 20 Sep 2003 13:00:55 -0700 (PDT) (envelope-from jdp@polstra.com) Message-ID: X-Mailer: XFMail 1.5.4 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20030920100911.B55993@xorpc.icir.org> Date: Sat, 20 Sep 2003 13:00:54 -0700 (PDT) From: John Polstra To: Luigi Rizzo X-Bogosity: No, tests=bogofilter, spamicity=0.292749, version=0.14.5 cc: freebsd-arch@freebsd.org Subject: Re: interrupt latency and driver locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Sep 2003 20:00:58 -0000 On 20-Sep-2003 Luigi Rizzo wrote: > the main problem, as i see it, is that when there are PHY events you > still need to do some expensive work while holding a lock that > blocks interrupts, with very bad impact on the worst-case > response of the system. I agree that is a problem, but I don't think it is the main problem. In a running system, PHY events essentially never happen, so it doesn't matter much if they take a long time. In other words, the PHY really only needs attention when the link state changes, and for all practical purposes that never happens in a running system. What is killing us is the periodic polling of the PHY every second, only to find out that nothing has changed. John From owner-freebsd-arch@FreeBSD.ORG Sat Sep 20 14:12:00 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4285716A4B3 for ; Sat, 20 Sep 2003 14:12:00 -0700 (PDT) Received: from xorpc.icir.org (xorpc.icir.org [192.150.187.68]) by mx1.FreeBSD.org (Postfix) with ESMTP id A05D843FE3 for ; Sat, 20 Sep 2003 14:11:59 -0700 (PDT) (envelope-from rizzo@xorpc.icir.org) Received: from xorpc.icir.org (localhost [127.0.0.1]) by xorpc.icir.org (8.12.9p1/8.12.3) with ESMTP id h8KLBxsd004036; Sat, 20 Sep 2003 14:11:59 -0700 (PDT) (envelope-from rizzo@xorpc.icir.org) Received: (from rizzo@localhost) by xorpc.icir.org (8.12.9p1/8.12.3/Submit) id h8KLBwQq004035; Sat, 20 Sep 2003 14:11:58 -0700 (PDT) (envelope-from rizzo) Date: Sat, 20 Sep 2003 14:11:58 -0700 From: Luigi Rizzo To: John Polstra Message-ID: <20030920141158.B97439@xorpc.icir.org> References: <20030920100911.B55993@xorpc.icir.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: ; from jdp@polstra.com on Sat, Sep 20, 2003 at 01:00:54PM -0700 cc: freebsd-arch@freebsd.org Subject: Re: interrupt latency and driver locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Sep 2003 21:12:00 -0000 On Sat, Sep 20, 2003 at 01:00:54PM -0700, John Polstra wrote: > On 20-Sep-2003 Luigi Rizzo wrote: > > > the main problem, as i see it, is that when there are PHY events you > > still need to do some expensive work while holding a lock that > > blocks interrupts, with very bad impact on the worst-case > > response of the system. > > I agree that is a problem, but I don't think it is the main problem. what i mean is that i don't think the frequency of these events (once per second or lower) is a big issue, but it is the fact that when they hit they delay potentially urgent activities for possibly very long times. In fact, the 50..300us measured by Sam are not even the worst case; I am pretty sure that when you have an actual PHY change the delays involved can be much longer, e.g. if you do an 'ifconfig' involving a media change (or even the initial 'up'), the "em" driver does a busy wait of well above 1 second while in the interrupt to wait for the autoconfig to settle (with DEVICE_POLLING this hits you badly). Several other drivers have similar issues. This is why i advocate a design change so that non-real-time, time-consuming activities are run under a different lock that does not block other real-time activities (for other subsystems of course). cheers luigi From owner-freebsd-arch@FreeBSD.ORG Sat Sep 20 14:29:36 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D900E16A4B3 for ; Sat, 20 Sep 2003 14:29:36 -0700 (PDT) Received: from blake.polstra.com (mail.polstra.com [206.213.73.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id A094E43FB1 for ; Sat, 20 Sep 2003 14:29:35 -0700 (PDT) (envelope-from jdp@polstra.com) Received: from strings.polstra.com (strings.polstra.com [206.213.73.20]) by blake.polstra.com (8.12.9/8.12.9) with ESMTP id h8KLTZIL016435; Sat, 20 Sep 2003 14:29:35 -0700 (PDT) (envelope-from jdp@polstra.com) Message-ID: X-Mailer: XFMail 1.5.4 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20030920141158.B97439@xorpc.icir.org> Date: Sat, 20 Sep 2003 14:29:34 -0700 (PDT) From: John Polstra To: Luigi Rizzo X-Bogosity: No, tests=bogofilter, spamicity=0.500053, version=0.14.5 cc: freebsd-arch@freebsd.org Subject: Re: interrupt latency and driver locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Sep 2003 21:29:37 -0000 On 20-Sep-2003 Luigi Rizzo wrote: > > what i mean is that i don't think the frequency of these events (once per > second or lower) is a big issue, but it is the fact that > when they hit they delay potentially urgent activities for > possibly very long times. > In fact, the 50..300us measured by Sam are not even the worst case; > I am pretty sure that when you have an actual PHY change the > delays involved can be much longer, I think we are basically in agreement that this needs to be fixed and that it will involve a separate lock. I care less about actual link changes than I do about the polling, because in a typical stable setup there will be no link changes except at bootstrap and shutdown. > e.g. if you do an 'ifconfig' > involving a media change (or even the initial 'up'), the "em" driver > does a busy wait of well above 1 second while in the interrupt > to wait for the autoconfig to settle (with DEVICE_POLLING this > hits you badly). Several other drivers have similar issues. Yes, the em driver is especially nasty in this regard. I think maybe changing the #define of WAIT_FOR_AUTO_NEG_DEFAULT to 0 in "if_em.h" would fix it, but I haven't tried that. > This is why i advocate a design change so that non-real-time, > time-consuming activities are run under a different lock > that does not block other real-time activities (for other > subsystems of course). Yep, me too. John From owner-freebsd-arch@FreeBSD.ORG Sat Sep 20 20:48:36 2003 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A790416A4BF for ; Sat, 20 Sep 2003 20:48:36 -0700 (PDT) Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id B736543FBD for ; Sat, 20 Sep 2003 20:48:35 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.9p1/8.12.3) with ESMTP id h8L3mYGA019156; Sat, 20 Sep 2003 21:48:34 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Sat, 20 Sep 2003 21:48:35 -0600 (MDT) Message-Id: <20030920.214835.101559646.imp@bsdimp.com> To: jdp@polstra.com From: "M. Warner Losh" In-Reply-To: References: <20030920141158.B97439@xorpc.icir.org> X-Mailer: Mew version 2.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: freebsd-arch@freebsd.org Subject: Re: interrupt latency and driver locking X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Sep 2003 03:48:36 -0000 In message: John Polstra writes: : there will be no link changes except at bootstrap and shutdown. For server machines. For laptops, these events happen more often. However, for most laptops, the rate that they happen is typically on less than 1/hour. Still rare enough to not worry about optimizing it and your other points are good. I just wanted to make sure that it wasn't optimized to the point where disconnecting the cable from the laptop to move it accross the room, or another room doesn't cause huge problems. Warner