From owner-freebsd-arch Sun Dec 10 3:25:11 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 03:25:06 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 30C0537B400 for ; Sun, 10 Dec 2000 03:25:05 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id eBABP4L97288 for ; Sun, 10 Dec 2000 12:25:04 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: arch@freebsd.org Subject: inheriting the "nodump" flag ? From: Poul-Henning Kamp Date: Sun, 10 Dec 2000 12:25:04 +0100 Message-ID: <97286.976447504@critter> Sender: phk@critter.freebsd.dk Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I don't know how many of you know or even use the "nodump" file flag. It's utility is somewhat limited by the fact that it is not inherited from the parent directory. I like to mark my cvs tree as "nodump", there is no point in backing it up for me. Unfortunately, since the nodump flag is not inherited on creation I need to run "chflags -R nodump /home/ncvs" first every time I want to make a backup. I would like to propose that directories and files inherit the nodump flag if it is set on the directory they are created in. Comments ? protests ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 3:35:29 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 03:35:27 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mailout00.sul.t-online.com (mailout00.sul.t-online.com [194.25.134.16]) by hub.freebsd.org (Postfix) with ESMTP id 37E1937B400; Sun, 10 Dec 2000 03:35:26 -0800 (PST) Received: from fwd01.sul.t-online.com by mailout00.sul.t-online.com with smtp id 1454lI-0001mQ-07; Sun, 10 Dec 2000 12:35:24 +0100 Received: from neutron.cichlids.com (520050424122-0001@[62.157.56.159]) by fmrl01.sul.t-online.com with esmtp id 1454lA-0e4BpQC; Sun, 10 Dec 2000 12:35:16 +0100 Received: from cichlids.cichlids.com (cichlids.cichlids.com [192.168.0.10]) by neutron.cichlids.com (Postfix) with ESMTP id 043A2AB12; Sun, 10 Dec 2000 12:35:27 +0100 (CET) Received: by cichlids.cichlids.com (Postfix, from userid 1001) id 52AF314ACE; Sun, 10 Dec 2000 12:35:16 +0100 (CET) Date: Sun, 10 Dec 2000 12:35:16 +0100 To: Poul-Henning Kamp Cc: arch@FreeBSD.ORG Subject: Re: inheriting the "nodump" flag ? Message-ID: <20001210123516.A12435@cichlids.cichlids.com> References: <97286.976447504@critter> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <97286.976447504@critter>; from phk@FreeBSD.ORG on Sun, Dec 10, 2000 at 12:25:04PM +0100 X-PGP-Fingerprint: 44 28 CA 4C 46 5B D3 A8 A8 E3 BA F3 4E 60 7D 7F X-PGP-at: finger alex@big.endian.de X-Verwirrung: Dieser Header dient der allgemeinen Verwirrung. From: alex@big.endian.de (Alexander Langer) X-Sender: 520050424122-0001@t-dialin.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Thus spake Poul-Henning Kamp (phk@FreeBSD.ORG): > I would like to propose that directories and files inherit the > nodump flag if it is set on the directory they are created in. > Comments ? protests ? I think it is just a logical conclusion. To be honest, I can't even understand how files in a directory can be dumped if the directory itself is marked as nodump. Alex -- cat: /home/alex/.sig: No such file or directory To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 3:48:31 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 03:48:28 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mail.interware.hu (mail.interware.hu [195.70.32.130]) by hub.freebsd.org (Postfix) with ESMTP id 2714837B400; Sun, 10 Dec 2000 03:48:26 -0800 (PST) Received: from casablanca-30.budapest.interware.hu ([195.70.53.30] helo=elischer.org) by mail.interware.hu with esmtp (Exim 3.16 #1 (Debian)) id 1454xr-0006Y7-00; Sun, 10 Dec 2000 12:48:24 +0100 Sender: julian@FreeBSD.ORG Message-ID: <3A336D68.2B454829@elischer.org> Date: Sun, 10 Dec 2000 03:47:52 -0800 From: Julian Elischer X-Mailer: Mozilla 4.7 [en] (X11; U; FreeBSD 5.0-CURRENT i386) X-Accept-Language: en, hu MIME-Version: 1.0 To: Poul-Henning Kamp Cc: arch@freebsd.org Subject: Re: inheriting the "nodump" flag ? References: <97286.976447504@critter> Content-Type: text/plain; charset=iso-8859-15 Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Poul-Henning Kamp wrote: > > I don't know how many of you know or even use the "nodump" file flag. > > It's utility is somewhat limited by the fact that it is not > inherited from the parent directory. > > I like to mark my cvs tree as "nodump", there is no point in backing > it up for me. > > Unfortunately, since the nodump flag is not inherited on creation > I need to run "chflags -R nodump /home/ncvs" first every time I want > to make a backup. > > I would like to propose that directories and files inherit the > nodump flag if it is set on the directory they are created in. > > Comments ? protests ? I thought that it was supposed to stop the tree walker from walking down through the tree beyond that point if it is on a directory.. in other words, if you set it on a directory it is supposed to 'prune' the entire tree.. sounds like a problem with the tree walker rather than the kernel. You are supposed to be able to set it on a directory, do a backup, change your mind later, and unset it and have the next run recurse all the way down the tree. > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message -- __--_|\ Julian Elischer / \ julian@elischer.org ( OZ ) World tour 2000 ---> X_.---._/ presently in: Budapest v To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 3:53:47 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 03:53:41 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mail.interware.hu (mail.interware.hu [195.70.32.130]) by hub.freebsd.org (Postfix) with ESMTP id 519AC37B400; Sun, 10 Dec 2000 03:53:40 -0800 (PST) Received: from casablanca-30.budapest.interware.hu ([195.70.53.30] helo=elischer.org) by mail.interware.hu with esmtp (Exim 3.16 #1 (Debian)) id 14552w-0006gi-00; Sun, 10 Dec 2000 12:53:38 +0100 Sender: julian@FreeBSD.ORG Message-ID: <3A336EA3.9F1FE318@elischer.org> Date: Sun, 10 Dec 2000 03:53:07 -0800 From: Julian Elischer X-Mailer: Mozilla 4.7 [en] (X11; U; FreeBSD 5.0-CURRENT i386) X-Accept-Language: en, hu MIME-Version: 1.0 To: Poul-Henning Kamp , arch@freebsd.org Subject: Re: inheriting the "nodump" flag ? References: <97286.976447504@critter> <3A336D68.2B454829@elischer.org> Content-Type: text/plain; charset=iso-8859-15 Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Julian Elischer wrote: > > Poul-Henning Kamp wrote: > > > > I don't know how many of you know or even use the "nodump" file flag. > > > > It's utility is somewhat limited by the fact that it is not > > inherited from the parent directory. > > > > I like to mark my cvs tree as "nodump", there is no point in backing > > it up for me. > > > > Unfortunately, since the nodump flag is not inherited on creation > > I need to run "chflags -R nodump /home/ncvs" first every time I want > > to make a backup. > > > > I would like to propose that directories and files inherit the > > nodump flag if it is set on the directory they are created in. > > > > Comments ? protests ? > > I thought that it was supposed to stop the tree walker from walking > down through the tree beyond that point if it is on a directory.. > > in other words, if you set it on a directory it is supposed to > 'prune' the entire tree.. sounds like a problem with the > tree walker rather than the kernel. > > You are supposed to be able to set it on a directory, > do a backup, change your mind later, and unset it and have the next run > recurse all the way down the tree. I guess I'm saying 'no' also if you do then a file that is in teh directory before the flag is set is treated differently to one that is created afterwards, and one that is created in s subirectory that was created before the change is also treated differntly (because the subdir doesn't have the flag). I think the dumpper should be fixed to not dump files that are further along the file hierarchy than a directory that has the bit set.. > > > > > -- > > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > > phk@FreeBSD.ORG | TCP/IP since RFC 956 > > FreeBSD committer | BSD since 4.3-tahoe > > Never attribute to malice what can adequately be explained by incompetence. > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > with "unsubscribe freebsd-arch" in the body of the message > > -- > __--_|\ Julian Elischer > / \ julian@elischer.org > ( OZ ) World tour 2000 > ---> X_.---._/ presently in: Budapest > v > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message -- __--_|\ Julian Elischer / \ julian@elischer.org ( OZ ) World tour 2000 ---> X_.---._/ presently in: Budapest v To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 4:23:54 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 04:23:50 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 42CFE37B400 for ; Sun, 10 Dec 2000 04:23:47 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id eBACNRL97642; Sun, 10 Dec 2000 13:23:27 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: alex@big.endian.de (Alexander Langer) Cc: arch@FreeBSD.ORG Subject: Re: inheriting the "nodump" flag ? In-Reply-To: Your message of "Sun, 10 Dec 2000 12:35:16 +0100." <20001210123516.A12435@cichlids.cichlids.com> Date: Sun, 10 Dec 2000 13:23:27 +0100 Message-ID: <97640.976451007@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <20001210123516.A12435@cichlids.cichlids.com>, Alexander Langer writ es: >Thus spake Poul-Henning Kamp (phk@FreeBSD.ORG): > >> I would like to propose that directories and files inherit the >> nodump flag if it is set on the directory they are created in. >> Comments ? protests ? > >I think it is just a logical conclusion. > >To be honest, I can't even understand how files in a directory can be >dumped if the directory itself is marked as nodump. Yeah, dump(8) violates POLA a bit there, but it is that way due to the way it is implemented. It would also be nice to be able to give dump(8) a list of exclude/include patterns: -X '*.core' -X 'ktrace.out' -I '*.rej' (volounteers welcome :-) -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 4:24:48 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 04:24:46 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 9335437B400 for ; Sun, 10 Dec 2000 04:24:45 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id eBACOeL97670; Sun, 10 Dec 2000 13:24:40 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Julian Elischer Cc: arch@FreeBSD.ORG Subject: Re: inheriting the "nodump" flag ? In-Reply-To: Your message of "Sun, 10 Dec 2000 03:47:52 PST." <3A336D68.2B454829@elischer.org> Date: Sun, 10 Dec 2000 13:24:40 +0100 Message-ID: <97668.976451080@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <3A336D68.2B454829@elischer.org>, Julian Elischer writes: >> I would like to propose that directories and files inherit the >> nodump flag if it is set on the directory they are created in. >> >> Comments ? protests ? > >I thought that it was supposed to stop the tree walker from walking >down through the tree beyond that point if it is on a directory.. > >in other words, if you set it on a directory it is supposed to >'prune' the entire tree.. sounds like a problem with the >tree walker rather than the kernel. Well, dump's tree walker is rather peculiar, so I don't blame anyone for the way it is implemented currently. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 6:44:41 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 06:44:39 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id F17B637B400 for ; Sun, 10 Dec 2000 06:44:38 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id PAA49690; Sun, 10 Dec 2000 15:44:33 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: arch@freebsd.org Subject: %a and %A formats From: Dag-Erling Smorgrav Date: 10 Dec 2000 15:44:33 +0100 Message-ID: Lines: 15 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I've added %a and %A formats to the kernel printf() and replaced most instances of inet_ntoa() in sys/netinet with %a (I'll do netinet6 later if there's interest). The patches are up on freefall: http://people.freebsd.org/~des/software/printf-20001209.diff I've probably done at least *one* thing wrong, so I'll appreciate feedback on how to improve the patch before it's committed. I'd also appreciate any information on how to teach gcc about %a and %A (currently, it thinks %a is a floating point format and complains about passing a pointer instead of a double) DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 6:46: 3 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 06:46:02 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 827B237B400 for ; Sun, 10 Dec 2000 06:46:01 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id PAA49703; Sun, 10 Dec 2000 15:46:00 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: arch@FreeBSD.ORG Subject: Re: %a and %A formats References: From: Dag-Erling Smorgrav Date: 10 Dec 2000 15:45:59 +0100 In-Reply-To: Dag-Erling Smorgrav's message of "10 Dec 2000 15:44:33 +0100" Message-ID: Lines: 9 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Dag-Erling Smorgrav writes: > http://people.freebsd.org/~des/software/printf-20001209.diff Oh, and I forgot - it builds, but hasn't been tested. You have been warned. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 7:40:26 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 07:40:24 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from peace.mahoroba.org (peace.calm.imasy.or.jp [202.227.26.34]) by hub.freebsd.org (Postfix) with ESMTP id 86FE837B400 for ; Sun, 10 Dec 2000 07:40:22 -0800 (PST) Received: from localhost (IDENT:5a9JysishX3OxgzrrIgwiA5Yx0qxtsTjXZp678vVJs+50Q+5sAlG+gK43EFScOt9@localhost [::1]) (authenticated) by peace.mahoroba.org (8.11.1/8.11.1/peace) with ESMTP/inet6 id eBAFc2E90863; Mon, 11 Dec 2000 00:38:02 +0900 (JST) (envelope-from ume@mahoroba.org) Date: Mon, 11 Dec 2000 00:38:00 +0900 (JST) Message-Id: <20001211.003800.25391776.ume@mahoroba.org> To: des@ofug.org Cc: arch@freebsd.org Subject: Re: %a and %A formats From: Hajimu UMEMOTO In-Reply-To: References: X-Mailer: xcite1.20> Mew version 1.95b38 on Emacs 20.7 / Mule 4.0 =?iso-2022-jp?B?KBskQjJWMWMbKEIp?= X-PGP-Public-Key: http://www.imasy.org/~ume/publickey.asc X-PGP-Fingerprint: 6B 0C 53 FC 5D D0 37 91 05 D0 B3 EF 36 9B 6A BC X-URL: http://www.imasy.org/~ume/ X-OS: FreeBSD 5.0-CURRENT Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG >>>>> On 10 Dec 2000 15:44:33 +0100 >>>>> Dag-Erling Smorgrav said: des> I've added %a and %A formats to the kernel printf() and replaced most des> instances of inet_ntoa() in sys/netinet with %a (I'll do netinet6 des> later if there's interest). The patches are up on freefall: des> http://people.freebsd.org/~des/software/printf-20001209.diff I don't like this. I've tired enough to address af dependent programming. What character will you introduce for IPv7 or IPv8...? Furthermore, IPv6 address has scope that IPv4 doesn't have. %a should takes struct sockaddr * for the argument instead of struct in_addr *, and should be address independent. Do you have a plan to the standard like POSIX? -- Hajimu UMEMOTO @ Internet Mutual Aid Society Yokohama, Japan ume@mahoroba.org ume@bisd.hitachi.co.jp ume@{,jp.}FreeBSD.org http://www.imasy.org/~ume/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 8: 1:25 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 08:01:23 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id EE7CB37B400 for ; Sun, 10 Dec 2000 08:01:22 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id eBAG1LL98902 for ; Sun, 10 Dec 2000 17:01:21 +0100 (CET) (envelope-from phk@critter.freebsd.dk) Cc: arch@FreeBSD.ORG Subject: Re: inheriting the "nodump" flag ? In-Reply-To: Your message of "Sun, 10 Dec 2000 12:25:04 +0100." <97286.976447504@critter> Date: Sun, 10 Dec 2000 17:01:21 +0100 Message-ID: <98900.976464081@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <97286.976447504@critter>, Poul-Henning Kamp writes: > >I don't know how many of you know or even use the "nodump" file flag. > >It's utility is somewhat limited by the fact that it is not >inherited from the parent directory. > >I like to mark my cvs tree as "nodump", there is no point in backing >it up for me. > >Unfortunately, since the nodump flag is not inherited on creation >I need to run "chflags -R nodump /home/ncvs" first every time I want >to make a backup. > >I would like to propose that directories and files inherit the >nodump flag if it is set on the directory they are created in. > >Comments ? protests ? Attached is the surprisingly simple patch to implement this in ufs. Index: ufs/ufs/ufs_vnops.c =================================================================== RCS file: /home/ncvs/src/sys/ufs/ufs/ufs_vnops.c,v retrieving revision 1.154 diff -u -r1.154 ufs_vnops.c --- ufs/ufs/ufs_vnops.c 2000/11/04 08:10:56 1.154 +++ ufs/ufs/ufs_vnops.c 2000/12/10 15:30:25 @@ -1355,6 +1355,7 @@ #endif #endif /* !SUIDDIR */ ip->i_flag |= IN_ACCESS | IN_CHANGE | IN_UPDATE; + ip->i_flags |= (dp->i_flags & UF_NODUMP); ip->i_mode = dmode; tvp->v_type = VDIR; /* Rest init'd in getnewvnode(). */ ip->i_effnlink = 2; @@ -2071,6 +2072,7 @@ return (error); ip = VTOI(tvp); ip->i_gid = pdir->i_gid; + ip->i_flags |= (pdir->i_flags & UF_NODUMP); #ifdef SUIDDIR { #ifdef QUOTA -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 8:27:35 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 08:27:34 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id DD14F37B400 for ; Sun, 10 Dec 2000 08:27:32 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id RAA50081; Sun, 10 Dec 2000 17:27:18 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Hajimu UMEMOTO Cc: arch@freebsd.org Subject: Re: %a and %A formats References: <20001211.003800.25391776.ume@mahoroba.org> From: Dag-Erling Smorgrav Date: 10 Dec 2000 17:27:18 +0100 In-Reply-To: Hajimu UMEMOTO's message of "Mon, 11 Dec 2000 00:38:00 +0900 (JST)" Message-ID: Lines: 13 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Hajimu UMEMOTO writes: > I don't like this. I've tired enough to address af dependent > programming. What character will you introduce for IPv7 or IPv8...? > Furthermore, IPv6 address has scope that IPv4 doesn't have. %a should > takes struct sockaddr * for the argument instead of struct in_addr *, > and should be address independent. Many (if not most) of the places in the kernel that need this don't have a struct sockaddr available. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 8:32:32 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 08:32:30 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 000A037B400 for ; Sun, 10 Dec 2000 08:32:25 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id eBAGW9L99151; Sun, 10 Dec 2000 17:32:09 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Dag-Erling Smorgrav Cc: Hajimu UMEMOTO , arch@FreeBSD.ORG Subject: Re: %a and %A formats In-Reply-To: Your message of "10 Dec 2000 17:27:18 +0100." Date: Sun, 10 Dec 2000 17:32:09 +0100 Message-ID: <99149.976465929@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message , Dag-Erling Smorgrav writes: >Hajimu UMEMOTO writes: >> I don't like this. I've tired enough to address af dependent >> programming. What character will you introduce for IPv7 or IPv8...? >> Furthermore, IPv6 address has scope that IPv4 doesn't have. %a should >> takes struct sockaddr * for the argument instead of struct in_addr *, >> and should be address independent. > >Many (if not most) of the places in the kernel that need this don't >have a struct sockaddr available. What we *really* need is a %{type} construct with dynmically loadable renderes for different types. Then you could printf("Deny %{proto} from ${ip} to ${ip}", ...) -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 9:26:48 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 09:26:45 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from cypherpunks.ai (cypherpunks.ai [209.88.68.47]) by hub.freebsd.org (Postfix) with ESMTP id 1F81737B401 for ; Sun, 10 Dec 2000 09:26:45 -0800 (PST) Received: from vangelderen.org (grolsch.ai [209.88.68.214]) by cypherpunks.ai (Postfix) with ESMTP id 847CE4D; Sun, 10 Dec 2000 13:26:38 -0400 (AST) Sender: gelderen@cypherpunks.ai Message-ID: <3A33BCCE.844B35B4@vangelderen.org> Date: Sun, 10 Dec 2000 13:26:38 -0400 From: "Jeroen C. van Gelderen" X-Mailer: Mozilla 4.73 [en] (X11; I; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Poul-Henning Kamp Cc: Julian Elischer , arch@FreeBSD.ORG Subject: Re: inheriting the "nodump" flag ? References: <97668.976451080@critter> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Poul-Henning Kamp wrote: > > In message <3A336D68.2B454829@elischer.org>, Julian Elischer writes: > > >> I would like to propose that directories and files inherit the > >> nodump flag if it is set on the directory they are created in. > >> > >> Comments ? protests ? > > > >I thought that it was supposed to stop the tree walker from walking > >down through the tree beyond that point if it is on a directory.. > > > >in other words, if you set it on a directory it is supposed to > >'prune' the entire tree.. sounds like a problem with the > >tree walker rather than the kernel. > > Well, dump's tree walker is rather peculiar, so I don't blame > anyone for the way it is implemented currently. This answer does not address the real point Julian is trying to make. Trying to fix a buggy dump implementation by patching the kernel is the wrong approach. It doesn't matter how peculiar the tree walker is, if it's buggy the fixing needs to be done there. It looks like NetBSD have already addressed the problem a year ago: http://lists.openresources.com/NetBSD/tech-kern/msg00453.html for the beginning of a thread discussing the problem. This was the first hit Google returned for "nodump flag". Basic research is cheap these days... A look at the NetBSD PR in question (6705) reveals: http://www.NetBSD.org/cgi-bin/query-pr-single.pl?number=6705 [...] >Release-Note: >Audit-Trail: State-Changed-From-To: open->closed State-Changed-By: bouyer State-Changed-When: Tue Mar 9 09:32:08 PST 1999 State-Changed-Why: Functionality added [to dump I presume], but differently. ^^^^^^^^^^^^^^^^^^^ It seems that fixing dump is the correct approach. Changing the semantics to nodump almost certainly is not as Julian pointed out in his other mail (<3A336EA3.9F1FE318@elischer.org>). I can now even add another reason why changing the kernel isn't such a good plan: inter-BSD compatibility. Cheers, Jeroen -- Jeroen C. van Gelderen o _ _ _ jeroen@vangelderen.org _o /\_ _ \\o (_)\__/o (_) _< \_ _>(_) (_)/<_ \_| \ _|/' \/ (_)>(_) (_) (_) (_) (_)' _\o_ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 9:29:20 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 09:29:16 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mail.interware.hu (mail.interware.hu [195.70.32.130]) by hub.freebsd.org (Postfix) with ESMTP id 3D17637B400 for ; Sun, 10 Dec 2000 09:29:16 -0800 (PST) Received: from gaborone-42.budapest.interware.hu ([195.70.52.170] helo=elischer.org) by mail.interware.hu with esmtp (Exim 3.16 #1 (Debian)) id 145AHh-00023M-00; Sun, 10 Dec 2000 18:29:14 +0100 Sender: julian@FreeBSD.ORG Message-ID: <3A33BD4B.B078C3E3@elischer.org> Date: Sun, 10 Dec 2000 09:28:43 -0800 From: Julian Elischer X-Mailer: Mozilla 4.7 [en] (X11; U; FreeBSD 5.0-CURRENT i386) X-Accept-Language: en, hu MIME-Version: 1.0 To: Poul-Henning Kamp Cc: arch@FreeBSD.ORG Subject: Re: inheriting the "nodump" flag ? References: <98900.976464081@critter> Content-Type: text/plain; charset=iso-8859-15 Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Poul-Henning Kamp wrote: > > In message <97286.976447504@critter>, Poul-Henning Kamp writes: > > > >I don't know how many of you know or even use the "nodump" file flag. > > > >It's utility is somewhat limited by the fact that it is not > >inherited from the parent directory. > > > >I like to mark my cvs tree as "nodump", there is no point in backing > >it up for me. > > > >Unfortunately, since the nodump flag is not inherited on creation > >I need to run "chflags -R nodump /home/ncvs" first every time I want > >to make a backup. > > > >I would like to propose that directories and files inherit the > >nodump flag if it is set on the directory they are created in. > > > >Comments ? protests ? > > Attached is the surprisingly simple patch to implement this in ufs. I think this badly breaks POLA and I am actively against it. As a default behaviour it is bad because: All files get broken up into several classes: 1/ old files in such a directory (dumped) 2/ new files in such a directory (not dumped) 3/ any files in an old subdirectory (dumped) 4/ any files in a new subdirectory (not dumped) This behaviour is not intuitive to the average user who will expect a 'don't dump' on a directory to prune that directory from the dump contents. to fix it you must ALWAYS do a chflags -R which is an extra step which can easily be |forgotten leading to confusion. the answer is to fix DUMP, or use `find . -flags ....|cpio` DUMP is the odd man out here by having a broken tree walker. If anyone else suggested this you'd be screaming bloody murder. > > Index: ufs/ufs/ufs_vnops.c > =================================================================== > RCS file: /home/ncvs/src/sys/ufs/ufs/ufs_vnops.c,v > retrieving revision 1.154 > diff -u -r1.154 ufs_vnops.c > --- ufs/ufs/ufs_vnops.c 2000/11/04 08:10:56 1.154 > +++ ufs/ufs/ufs_vnops.c 2000/12/10 15:30:25 > @@ -1355,6 +1355,7 @@ > #endif > #endif /* !SUIDDIR */ > ip->i_flag |= IN_ACCESS | IN_CHANGE | IN_UPDATE; > + ip->i_flags |= (dp->i_flags & UF_NODUMP); > ip->i_mode = dmode; > tvp->v_type = VDIR; /* Rest init'd in getnewvnode(). */ > ip->i_effnlink = 2; > @@ -2071,6 +2072,7 @@ > return (error); > ip = VTOI(tvp); > ip->i_gid = pdir->i_gid; > + ip->i_flags |= (pdir->i_flags & UF_NODUMP); > #ifdef SUIDDIR > { > #ifdef QUOTA > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message -- __--_|\ Julian Elischer / \ julian@elischer.org ( OZ ) World tour 2000 ---> X_.---._/ presently in: Budapest v To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 9:38: 1 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 09:37:58 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mail.interware.hu (mail.interware.hu [195.70.32.130]) by hub.freebsd.org (Postfix) with ESMTP id 3316937B400 for ; Sun, 10 Dec 2000 09:37:58 -0800 (PST) Received: from gaborone-42.budapest.interware.hu ([195.70.52.170] helo=elischer.org) by mail.interware.hu with esmtp (Exim 3.16 #1 (Debian)) id 145APy-0002gQ-00; Sun, 10 Dec 2000 18:37:46 +0100 Sender: julian@FreeBSD.ORG Message-ID: <3A33BF4B.97EE1BCC@elischer.org> Date: Sun, 10 Dec 2000 09:37:15 -0800 From: Julian Elischer X-Mailer: Mozilla 4.7 [en] (X11; U; FreeBSD 5.0-CURRENT i386) X-Accept-Language: en, hu MIME-Version: 1.0 To: "Jeroen C. van Gelderen" Cc: Poul-Henning Kamp , arch@FreeBSD.ORG Subject: Re: inheriting the "nodump" flag ? References: <97668.976451080@critter> <3A33BCCE.844B35B4@vangelderen.org> Content-Type: text/plain; charset=iso-8859-15 Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG "Jeroen C. van Gelderen" wrote: > > > This answer does not address the real point Julian is trying > to make. Trying to fix a buggy dump implementation by patching > the kernel is the wrong approach. It doesn't matter how peculiar > the tree walker is, if it's buggy the fixing needs to be done > there. thanks: well the patch is there on the URL given below. I fully support implementing it on FreeBSD as well. (with a little testing) > > It looks like NetBSD have already addressed the problem a > year ago: > > http://lists.openresources.com/NetBSD/tech-kern/msg00453.html > > for the beginning of a thread discussing the problem. This > was the first hit Google returned for "nodump flag". Basic > research is cheap these days... > > A look at the NetBSD PR in question (6705) reveals: > http://www.NetBSD.org/cgi-bin/query-pr-single.pl?number=6705 > [...] > >Release-Note: > >Audit-Trail: > State-Changed-From-To: open->closed > State-Changed-By: bouyer > State-Changed-When: Tue Mar 9 09:32:08 PST 1999 > State-Changed-Why: > Functionality added [to dump I presume], but differently. > ^^^^^^^^^^^^^^^^^^^ > > It seems that fixing dump is the correct approach. Changing > the semantics to nodump almost certainly is not as Julian > pointed out in his other mail (<3A336EA3.9F1FE318@elischer.org>). > > I can now even add another reason why changing the kernel isn't > such a good plan: inter-BSD compatibility. > > Cheers, > Jeroen > -- > Jeroen C. van Gelderen o _ _ _ > jeroen@vangelderen.org _o /\_ _ \\o (_)\__/o (_) > _< \_ _>(_) (_)/<_ \_| \ _|/' \/ > (_)>(_) (_) (_) (_) (_)' _\o_ -- __--_|\ Julian Elischer / \ julian@elischer.org ( OZ ) World tour 2000 ---> X_.---._/ presently in: Budapest v To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 9:57:30 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 09:57:29 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from puck.firepipe.net (mcut-b-167.resnet.purdue.edu [128.211.209.167]) by hub.freebsd.org (Postfix) with ESMTP id F26D537B400 for ; Sun, 10 Dec 2000 09:57:28 -0800 (PST) Received: by puck.firepipe.net (Postfix, from userid 1000) id 3F1D118C6; Sun, 10 Dec 2000 12:57:29 -0500 (EST) Date: Sun, 10 Dec 2000 12:57:29 -0500 From: Will Andrews To: Poul-Henning Kamp Cc: arch@FreeBSD.ORG Subject: Re: inheriting the "nodump" flag ? Message-ID: <20001210125729.C560@puck.firepipe.net> Reply-To: Will Andrews Mail-Followup-To: Will Andrews , Poul-Henning Kamp , arch@FreeBSD.ORG References: <97286.976447504@critter> <98900.976464081@critter> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <98900.976464081@critter>; from phk@critter.freebsd.dk on Sun, Dec 10, 2000 at 05:01:21PM +0100 X-Operating-System: FreeBSD 4.2-STABLE i386 Sender: will@puck.firepipe.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sun, Dec 10, 2000 at 05:01:21PM +0100, Poul-Henning Kamp wrote: > Attached is the surprisingly simple patch to implement this in ufs. Poul, having read the rest of this thread, I'm afraid I see more merit in fixing dump(8) rather than changing filesystem semantics. -- wca To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 11:13:14 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 11:13:12 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id B3F9C37B400 for ; Sun, 10 Dec 2000 11:13:12 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id eBAJCs802096; Sun, 10 Dec 2000 11:12:54 -0800 (PST) Date: Sun, 10 Dec 2000 11:12:54 -0800 From: Alfred Perlstein To: Dag-Erling Smorgrav Cc: arch@FreeBSD.ORG Subject: Re: %a and %A formats Message-ID: <20001210111254.F16205@fw.wintelcom.net> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from des@ofug.org on Sun, Dec 10, 2000 at 03:45:59PM +0100 Sender: bright@fw.wintelcom.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Dag-Erling Smorgrav [001210 06:46] wrote: > Dag-Erling Smorgrav writes: > > http://people.freebsd.org/~des/software/printf-20001209.diff > > Oh, and I forgot - it builds, but hasn't been tested. You have been > warned. Glancing at it, I don't see it working properly on different endianness. Should it work? -Alfred To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 11:17:20 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 11:17:19 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from relay.nuxi.com (nuxi.cs.ucdavis.edu [169.237.7.38]) by hub.freebsd.org (Postfix) with ESMTP id 299A037B400 for ; Sun, 10 Dec 2000 11:17:19 -0800 (PST) Received: from dragon.nuxi.com (Ipittythefoolthattrustsident@trang.nuxi.com [209.152.133.57]) by relay.nuxi.com (8.9.3/8.9.3) with ESMTP id LAA52945; Sun, 10 Dec 2000 11:16:59 -0800 (PST) (envelope-from obrien@NUXI.com) Received: (from obrien@localhost) by dragon.nuxi.com (8.11.1/8.11.1) id eBAJGvc80902; Sun, 10 Dec 2000 11:16:57 -0800 (PST) (envelope-from obrien) Date: Sun, 10 Dec 2000 11:16:56 -0800 From: "David O'Brien" To: Dag-Erling Smorgrav Cc: arch@freebsd.org Subject: Re: %a and %A formats Message-ID: <20001210111656.A80274@dragon.nuxi.com> Reply-To: alpha@freebsd.org References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from des@ofug.org on Sun, Dec 10, 2000 at 03:44:33PM +0100 X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD group X-Pgp-Rsa-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Rsa-Keyid: 1024/34F9F9D5 Sender: obrien@NUXI.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sun, Dec 10, 2000 at 03:44:33PM +0100, Dag-Erling Smorgrav wrote: > I'd also appreciate any information on how to teach gcc about %a and %A > (currently, it thinks %a is a floating point format and complains about > passing a pointer instead of a double) Because that is what %a and %A already are. -- -- David (obrien@FreeBSD.org) GNU is Not Unix / Linux Is Not UniX To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 11:22: 8 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 11:22:07 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 5BB5D37B400 for ; Sun, 10 Dec 2000 11:22:06 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id eBAJM2L00384; Sun, 10 Dec 2000 20:22:02 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Alfred Perlstein Cc: Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: %a and %A formats In-Reply-To: Your message of "Sun, 10 Dec 2000 11:12:54 PST." <20001210111254.F16205@fw.wintelcom.net> Date: Sun, 10 Dec 2000 20:22:02 +0100 Message-ID: <382.976476122@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <20001210111254.F16205@fw.wintelcom.net>, Alfred Perlstein writes: >* Dag-Erling Smorgrav [001210 06:46] wrote: >> Dag-Erling Smorgrav writes: >> > http://people.freebsd.org/~des/software/printf-20001209.diff >> >> Oh, and I forgot - it builds, but hasn't been tested. You have been >> warned. > >Glancing at it, I don't see it working properly on different >endianness. > >Should it work? I see nothing but regular string/char operations, where do you see endianess issues ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 11:25:30 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 11:25:29 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 4B38937B400 for ; Sun, 10 Dec 2000 11:25:28 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id eBAJPML00442; Sun, 10 Dec 2000 20:25:22 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Julian Elischer Cc: "Jeroen C. van Gelderen" , arch@FreeBSD.ORG Subject: Re: inheriting the "nodump" flag ? In-Reply-To: Your message of "Sun, 10 Dec 2000 09:37:15 PST." <3A33BF4B.97EE1BCC@elischer.org> Date: Sun, 10 Dec 2000 20:25:22 +0100 Message-ID: <440.976476322@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <3A33BF4B.97EE1BCC@elischer.org>, Julian Elischer writes: >> It looks like NetBSD have already addressed the problem a >> year ago: >> >> http://lists.openresources.com/NetBSD/tech-kern/msg00453.html >> >> for the beginning of a thread discussing the problem. This >> was the first hit Google returned for "nodump flag". Basic >> research is cheap these days... >> >> A look at the NetBSD PR in question (6705) reveals: >> http://www.NetBSD.org/cgi-bin/query-pr-single.pl?number=6705 Well, then we just need somebody to MFN it... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 11:58:12 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 11:58:10 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 7C6AB37B400 for ; Sun, 10 Dec 2000 11:58:10 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id eBAJtYg03294; Sun, 10 Dec 2000 11:55:34 -0800 (PST) Date: Sun, 10 Dec 2000 11:55:34 -0800 From: Alfred Perlstein To: Poul-Henning Kamp Cc: Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: %a and %A formats Message-ID: <20001210115534.G16205@fw.wintelcom.net> References: <20001210111254.F16205@fw.wintelcom.net> <382.976476122@critter> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <382.976476122@critter>; from phk@critter.freebsd.dk on Sun, Dec 10, 2000 at 08:22:02PM +0100 Sender: bright@fw.wintelcom.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Poul-Henning Kamp [001210 11:22] wrote: > In message <20001210111254.F16205@fw.wintelcom.net>, Alfred Perlstein writes: > >* Dag-Erling Smorgrav [001210 06:46] wrote: > >> Dag-Erling Smorgrav writes: > >> > http://people.freebsd.org/~des/software/printf-20001209.diff > >> > >> Oh, and I forgot - it builds, but hasn't been tested. You have been > >> warned. > > > >Glancing at it, I don't see it working properly on different > >endianness. > > > >Should it work? > > I see nothing but regular string/char operations, where do you > see endianess issues ? Ok, maybe i'm not getting something here: + case 'a': + p = va_arg(ap, char *); + for (n = 0; n < 4; ++n, ++p) { + tmp = (unsigned char)*p; + if (tmp > 99 || padc == '0') + PCHAR('0' + tmp / 100); + if (tmp > 9 || padc == '0') + PCHAR('0' + (tmp % 100) / 10); + PCHAR('0' + tmp % 10); + if (n < 3) + PCHAR('.'); + } + break; aren't sockaddrs in host format by the time they are printed? if they are, how do we know it's like this: a b c d and not d c b a I think I need to make some coffee or something. :) -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 12:11: 5 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 12:11:04 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 7317A37B401 for ; Sun, 10 Dec 2000 12:11:03 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id VAA51094; Sun, 10 Dec 2000 21:10:59 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Alfred Perlstein Cc: arch@FreeBSD.ORG Subject: Re: %a and %A formats References: <20001210111254.F16205@fw.wintelcom.net> From: Dag-Erling Smorgrav Date: 10 Dec 2000 21:10:59 +0100 In-Reply-To: Alfred Perlstein's message of "Sun, 10 Dec 2000 11:12:54 -0800" Message-ID: Lines: 10 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Alfred Perlstein writes: > Glancing at it, I don't see it working properly on different > endianness. You're right, I didn't think about endianness. It should be a trivial fix (as long as you don't expect it to handle NUXI-style machines) DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 12:17: 9 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 12:17:07 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from netau1.alcanet.com.au (ntp.alcanet.com.au [203.62.196.27]) by hub.freebsd.org (Postfix) with ESMTP id A02DC37B400 for ; Sun, 10 Dec 2000 12:17:05 -0800 (PST) Received: from mfg1.cim.alcatel.com.au (mfg1.cim.alcatel.com.au [139.188.23.1]) by netau1.alcanet.com.au (8.9.3 (PHNE_18979)/8.9.3) with ESMTP id HAA13557; Mon, 11 Dec 2000 07:17:02 +1100 (EDT) Received: from gsmx07.alcatel.com.au by cim.alcatel.com.au (PMDF V5.2-32 #37641) with ESMTP id <01JXKIOS89TCE7XYVS@cim.alcatel.com.au>; Mon, 11 Dec 2000 07:16:50 +1100 Received: (from jeremyp@localhost) by gsmx07.alcatel.com.au (8.11.0/8.11.0) id eBAKGx523033; Mon, 11 Dec 2000 07:16:59 +1100 (EST envelope-from jeremyp) Content-return: prohibited Date: Mon, 11 Dec 2000 07:16:59 +1100 From: Peter Jeremy Subject: Re: RANDOMDEV inspired realitycheck regarding i386/i486... In-reply-to: ; from DougB@gorean.org on Mon, Dec 04, 2000 at 07:20:49PM -0800 To: Doug Barton Cc: arch@FreeBSD.ORG Mail-followup-to: Doug Barton , arch@FreeBSD.ORG Message-id: <20001211071659.E69646@gsmx07.alcatel.com.au> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-disposition: inline User-Agent: Mutt/1.2.5i References: <20001201152137.K1474@gsmx07.alcatel.com.au> Sender: jeremyp@gsmx07.alcatel.com.au Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 2000-Dec-04 19:20:49 -0800, Doug Barton wrote: >On Fri, 1 Dec 2000, Peter Jeremy wrote: > >> On 2000-Nov-30 21:47:45 -0600, "Michael C . Wu" wrote: >> >On Fri, Dec 01, 2000 at 10:29:15AM +1100, Peter Jeremy scribbled: >> >| On 2000-Nov-14 15:08:06 +0100, Poul-Henning Kamp wrote: >> >| >Has anybody run a 486 or 386 under current recently ? >> >| >> >| X on a PRE_SMPNG 486 is painful - mouse movements no longer make >> >| the X pointer move in real time. I haven't noticed the seeding >> >| issue (probably just luck). >> > >> >PRE_SMPNG does not have the /dev/random seeding issue. >> > >> >You actually expected X to run well on a 486? :-) >> >> It used to run reasonably well (ignoring hogs like Netscape) before >> Yarrow was added. > > Have you tried updating to the latest -Current? All aspects of the >entropy harvesting have changed significantly since PRE_SMPNG. I upgraded to -current as of last Friday (this took most of the weekend on my 486DX2-50, so I didn't do much experimenting). The entropy seeding during startup takes a couple of seconds - not a problem. X seems somewhat better, though the performance is still not acceptable - the mouse movement will freeze for a second or so occasionally. (I just pull up a longish menu from fvwm and scan the mouse up and down). Overall, entropy harvesting in -current has improved significantly since PRE_SMPNG, but IMHO there is still some way to go before its acceptable on slower machines. Peter To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 20:15:29 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 20:15:27 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from cypherpunks.ai (cypherpunks.ai [209.88.68.47]) by hub.freebsd.org (Postfix) with ESMTP id 713A837B401 for ; Sun, 10 Dec 2000 20:15:26 -0800 (PST) Received: from vangelderen.org (grolsch.ai [209.88.68.214]) by cypherpunks.ai (Postfix) with ESMTP id BE08D4D; Mon, 11 Dec 2000 00:15:24 -0400 (AST) Sender: gelderen@cypherpunks.ai Message-ID: <3A3454DC.5A9E7FF1@vangelderen.org> Date: Mon, 11 Dec 2000 00:15:24 -0400 From: "Jeroen C. van Gelderen" X-Mailer: Mozilla 4.73 [en] (X11; I; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Poul-Henning Kamp Cc: Julian Elischer , arch@FreeBSD.ORG Subject: Re: inheriting the "nodump" flag ? References: <440.976476322@critter> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Hi, Poul-Henning Kamp wrote: > > In message <3A33BF4B.97EE1BCC@elischer.org>, Julian Elischer writes: This is not correct. Please be careful with your attributions... > >> It looks like NetBSD have already addressed the problem a > >> year ago: > >> > >> http://lists.openresources.com/NetBSD/tech-kern/msg00453.html > >> > >> for the beginning of a thread discussing the problem. This > >> was the first hit Google returned for "nodump flag". Basic > >> research is cheap these days... > >> > >> A look at the NetBSD PR in question (6705) reveals: > >> http://www.NetBSD.org/cgi-bin/query-pr-single.pl?number=6705 > > Well, then we just need somebody to MFN it... The NetBSD dump seems to be more evolved than the FreeBSD version. I've partially merged the NetBSD changes into our dump, just enough so that the 'nodump' flag is handled properly. I will merge the rest of the NetBSD changes as well if there is interest. A dump/fs guru will have to review these as I must have made mistakes. For now you can try the (lightly tested) patch at http://grolsch.ai/~gelderen/dump.diff It's bigger than strictly neccessary but I have tried to minimize the diffs between the two BSDs. I had some doubts about merging the byte-order support but decided that it seemed useful enough and actually didn't affect the readability of the code negatively; Hence the iswap32()s. I have some patches for the NetBSD folks as well and if both parties accept them we can have a virtually identical traverse.c on the two BSDs. What do you think? Cheers, Jeroen -- Jeroen C. van Gelderen o _ _ _ jeroen@vangelderen.org _o /\_ _ \\o (_)\__/o (_) _< \_ _>(_) (_)/<_ \_| \ _|/' \/ (_)>(_) (_) (_) (_) (_)' _\o_ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Dec 10 21:35:20 2000 From owner-freebsd-arch@FreeBSD.ORG Sun Dec 10 21:35:17 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mail.rpi.edu (mail.rpi.edu [128.113.100.7]) by hub.freebsd.org (Postfix) with ESMTP id 05ECD37B401 for ; Sun, 10 Dec 2000 21:35:17 -0800 (PST) Received: from [128.113.24.47] (gilead.acs.rpi.edu [128.113.24.47]) by mail.rpi.edu (8.9.3/8.9.3) with ESMTP id AAA98792; Mon, 11 Dec 2000 00:35:07 -0500 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: References: Date: Mon, 11 Dec 2000 00:35:05 -0500 To: Dag-Erling Smorgrav , arch@FreeBSD.ORG From: Garance A Drosihn Subject: Re: %a and %A formats Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG At 3:44 PM +0100 12/10/00, Dag-Erling Smorgrav wrote: >I've added %a and %A formats to the kernel printf() and replaced >most instances of inet_ntoa() in sys/netinet with %a (I'll do >netinet6 later if there's interest). The patches are up on >freefall: > > http://people.freebsd.org/~des/software/printf-20001209.diff [an aside for those who didn't look at the diff: %a is meant for ipv4 addresses, and %A is for ipv6 addresses...] I do not think we should go in this direction. The printf family of routines is something which needs to be fairly standard across OS's (compilers, etc), and it would be a bad idea for us to arbitrary add new opcodes to those routines. Most of the opcodes in printf are for basic C types, and not OS-dependent data structures. I notice you mentioned the 'kernel printf()', which implies it is separate from the userland printf(), but I'm still not comfortable with the idea of introducing opcodes to it. Besides, I can imagine this being useful to many userland programs, and if we were to add any opcode to the userland printf then it would make sense that the same code ends up in the kernel's printf(). >I'd also appreciate any information on how to teach gcc about >%a and %A (currently, it thinks %a is a floating point format >and complains about passing a pointer instead of a double) Chances are that is *because* %a and %A are already in use somewhere, which certainly implies that this is a bad idea. I don't think we need confusion over "which codes mean what" in different versions of printf's. If we were going to add some new opcodes to printf, make them something dramatically different. Go for '%|x|', where we could add a variety of our own opcodes between the '|'s. Thus, these would be '%|a|' and '%|A|' (or perhaps some other letter). Or perhaps we could define some other routine, which would know about networking data structures, and would have it's own set of opcodes. Use that to return a string (or write into a buffer), and use printf/syslog/etc to print out that string to the desired destination. I'm just trying to toss out some suggestions here, I haven't figured out which one (if any) I like... But I do expect that using %A and %a is going to get a little confusing. disclaimer: I don't work in the kernel, so my opinion probably doesn't amount to much on a topic like this... :-) -- Garance Alistair Drosehn = gad@eclipse.acs.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 3:27:28 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 03:27:25 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3C89B37B725 for ; Mon, 11 Dec 2000 03:24:22 -0800 (PST) Received: from mobile.wemm.org (adsl-64-163-195-99.dsl.snfc21.pacbell.net [64.163.195.99]) by mx1.FreeBSD.org (Postfix) with ESMTP id 715A46E25BD for ; Mon, 11 Dec 2000 02:01:05 -0800 (PST) Received: from netplex.com.au (localhost [127.0.0.1]) by mobile.wemm.org (8.11.1/8.11.1) with ESMTP id eBB9xiK26188; Mon, 11 Dec 2000 01:59:44 -0800 (PST) (envelope-from peter@netplex.com.au) Message-Id: <200012110959.eBB9xiK26188@mobile.wemm.org> X-Mailer: exmh version 2.2 06/23/2000 with nmh-1.0.4 To: Dag-Erling Smorgrav Cc: Alfred Perlstein , arch@FreeBSD.ORG Subject: Re: %a and %A formats In-Reply-To: Date: Mon, 11 Dec 2000 01:59:43 -0800 From: Peter Wemm Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Dag-Erling Smorgrav wrote: > Alfred Perlstein writes: > > Glancing at it, I don't see it working properly on different > > endianness. > > You're right, I didn't think about endianness. It should be a trivial > fix (as long as you don't expect it to handle NUXI-style machines) Dont forget gcc's -fformat-extensions code needs to have corresponding patches to check these keyword types. We compile the kernel with this flag specifically for this reason. It understands all the wierd formats (eg: %B etc). Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 3:27:39 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 03:27:35 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EB3D337B74E for ; Mon, 11 Dec 2000 03:24:23 -0800 (PST) Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by mx1.FreeBSD.org (Postfix) with ESMTP id 40B256E3042 for ; Mon, 11 Dec 2000 02:31:26 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id eBBAU8b24268; Mon, 11 Dec 2000 02:30:08 -0800 (PST) Date: Mon, 11 Dec 2000 02:30:08 -0800 From: Alfred Perlstein To: Peter Wemm Cc: Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: %a and %A formats Message-ID: <20001211023008.X16205@fw.wintelcom.net> References: <200012110959.eBB9xiK26188@mobile.wemm.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200012110959.eBB9xiK26188@mobile.wemm.org>; from peter@netplex.com.au on Mon, Dec 11, 2000 at 01:59:43AM -0800 Sender: bright@fw.wintelcom.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Peter Wemm [001211 01:59] wrote: > Dag-Erling Smorgrav wrote: > > Alfred Perlstein writes: > > > Glancing at it, I don't see it working properly on different > > > endianness. > > > > You're right, I didn't think about endianness. It should be a trivial > > fix (as long as you don't expect it to handle NUXI-style machines) > > Dont forget gcc's -fformat-extensions code needs to have corresponding > patches to check these keyword types. We compile the kernel with this flag > specifically for this reason. It understands all the wierd formats (eg: %B > etc). The more I think about this patch, the less I like it. While the work is good and it's probably pretty handy, the kernel really doesn't print out too many addresses afaik. A better idea would be to fix the inaddr->char* routines to be mpsafe like I've done with inet_ntoa_r(). I just don't want to wind up sometime later wondering if %Z is a good flag to use to print out an mbuf. :) -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 3:41:10 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 03:41:04 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C08A637B69B for ; Mon, 11 Dec 2000 03:41:04 -0800 (PST) Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6FEC66E251B for ; Mon, 11 Dec 2000 02:34:21 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.8.7/8.8.7) with ESMTP id VAA14078; Mon, 11 Dec 2000 21:30:57 +1100 Date: Mon, 11 Dec 2000 21:32:23 +1100 (EST) From: Bruce Evans X-Sender: bde@besplex.bde.org To: Peter Jeremy Cc: Doug Barton , arch@FreeBSD.ORG Subject: Re: RANDOMDEV inspired realitycheck regarding i386/i486... In-Reply-To: <20001211071659.E69646@gsmx07.alcatel.com.au> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Mon, 11 Dec 2000, Peter Jeremy wrote: > I upgraded to -current as of last Friday (this took most of the > weekend on my 486DX2-50, so I didn't do much experimenting). The > entropy seeding during startup takes a couple of seconds - not a > problem. I crashed -current a few times recently, and entropy seeding during startup took several minutes of leaning on the space bar. This was initially with a week old kernel and finally with a current kernel and a broken ps. Apparently there isn't a enough entropy unless ps produces a normal amount. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 3:41:20 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 03:41:16 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8E65F37B6A7 for ; Mon, 11 Dec 2000 03:41:05 -0800 (PST) Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5D60F6E306B for ; Mon, 11 Dec 2000 02:45:31 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id LAA54504; Mon, 11 Dec 2000 11:44:08 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Alfred Perlstein Cc: Peter Wemm , arch@FreeBSD.ORG Subject: Re: %a and %A formats References: <200012110959.eBB9xiK26188@mobile.wemm.org> <20001211023008.X16205@fw.wintelcom.net> From: Dag-Erling Smorgrav Date: 11 Dec 2000 11:44:07 +0100 In-Reply-To: Alfred Perlstein's message of "Mon, 11 Dec 2000 02:30:08 -0800" Message-ID: Lines: 10 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Alfred Perlstein writes: > The more I think about this patch, the less I like it. While the > work is good and it's probably pretty handy, the kernel really > doesn't print out too many addresses afaik. You'd be surprised... DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 4: 9:35 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 04:09:28 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C41B237B706 for ; Mon, 11 Dec 2000 04:08:48 -0800 (PST) Received: from relay.nuxi.com (nuxi.cs.ucdavis.edu [169.237.7.38]) by mx1.FreeBSD.org (Postfix) with ESMTP id E7E696E2E87 for ; Mon, 11 Dec 2000 01:00:27 -0800 (PST) Received: from dragon.nuxi.com (Ipittythefoolthattrustsident@trang.nuxi.com [209.152.133.57]) by relay.nuxi.com (8.9.3/8.9.3) with ESMTP id AAA56070; Mon, 11 Dec 2000 00:59:10 -0800 (PST) (envelope-from obrien@NUXI.com) Received: (from obrien@localhost) by dragon.nuxi.com (8.11.1/8.11.1) id eBB8x8N90068; Mon, 11 Dec 2000 00:59:08 -0800 (PST) (envelope-from obrien) Date: Mon, 11 Dec 2000 00:59:08 -0800 From: "David O'Brien" To: "Jeroen C. van Gelderen" Cc: Poul-Henning Kamp , Julian Elischer , arch@FreeBSD.ORG Subject: Re: inheriting the "nodump" flag ? Message-ID: <20001211005908.H89853@dragon.nuxi.com> Reply-To: arch@FreeBSD.ORG References: <440.976476322@critter> <3A3454DC.5A9E7FF1@vangelderen.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <3A3454DC.5A9E7FF1@vangelderen.org>; from jeroen@vangelderen.org on Mon, Dec 11, 2000 at 12:15:24AM -0400 X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD group X-Pgp-Rsa-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Rsa-Keyid: 1024/34F9F9D5 Sender: obrien@NUXI.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Mon, Dec 11, 2000 at 12:15:24AM -0400, Jeroen C. van Gelderen wrote: > The NetBSD dump seems to be more evolved than the FreeBSD > version. I've partially merged the NetBSD changes into our > dump, just enough so that the 'nodump' flag is handled > properly. Actually the patch has more than just 'nodump'. You've done a lot of style changes too. :-( > It's bigger than strictly neccessary but I have tried to > minimize the diffs between the two BSDs. Which should not be done at the same time. OK, I will now go out on a limb and say I'll handle the 'nodump' changes to dump on Monday. So nobody, please do the same. -- -- David (obrien@FreeBSD.org) GNU is Not Unix / Linux Is Not UniX To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 4:47:52 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 04:47:50 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from oden.exmandato.se (oden.exmandato.se [192.71.33.1]) by hub.freebsd.org (Postfix) with ESMTP id DDED937B400 for ; Mon, 11 Dec 2000 04:47:49 -0800 (PST) Received: from servicefactory.se (root@oden.exmandato.se [192.71.33.1]) by oden.exmandato.se (8.8.8/8.8.5) with ESMTP id NAA00319; Mon, 11 Dec 2000 13:44:41 +0100 (MET) Sender: jonas@oden.exmandato.se Message-ID: <3A34BE26.8F2876B@servicefactory.se> Date: Mon, 11 Dec 2000 12:44:38 +0100 From: Jonas Bulow X-Mailer: Mozilla 4.76 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Poul-Henning Kamp Cc: Dag-Erling Smorgrav , Hajimu UMEMOTO , arch@FreeBSD.ORG Subject: Re: %a and %A formats References: <99149.976465929@critter> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Poul-Henning Kamp wrote: > > In message , Dag-Erling Smorgrav writes: > >Hajimu UMEMOTO writes: > >> I don't like this. I've tired enough to address af dependent > >> programming. What character will you introduce for IPv7 or IPv8...? > >> Furthermore, IPv6 address has scope that IPv4 doesn't have. %a should > >> takes struct sockaddr * for the argument instead of struct in_addr *, > >> and should be address independent. > > > >Many (if not most) of the places in the kernel that need this don't > >have a struct sockaddr available. > > What we *really* need is a %{type} construct with dynmically > loadable renderes for different types. Something like http://www.research.att.com/sw/tools/sfio/sfio-fmt.ps ? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 7:31:11 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 07:31:10 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from cypherpunks.ai (cypherpunks.ai [209.88.68.47]) by hub.freebsd.org (Postfix) with ESMTP id 7D24137B400 for ; Mon, 11 Dec 2000 07:31:08 -0800 (PST) Received: from vangelderen.org (grolsch.ai [209.88.68.214]) by cypherpunks.ai (Postfix) with ESMTP id 2242A49; Mon, 11 Dec 2000 11:31:01 -0400 (AST) Sender: gelderen@cypherpunks.ai Message-ID: <3A34F335.CD5CCD74@vangelderen.org> Date: Mon, 11 Dec 2000 11:31:01 -0400 From: "Jeroen C. van Gelderen" X-Mailer: Mozilla 4.73 [en] (X11; I; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: arch@FreeBSD.ORG Cc: Poul-Henning Kamp , Julian Elischer Subject: Re: inheriting the "nodump" flag ? References: <440.976476322@critter> <3A3454DC.5A9E7FF1@vangelderen.org> <20001211005908.H89853@dragon.nuxi.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG David O'Brien wrote: > > On Mon, Dec 11, 2000 at 12:15:24AM -0400, Jeroen C. van Gelderen wrote: > > The NetBSD dump seems to be more evolved than the FreeBSD > > version. I've partially merged the NetBSD changes into our > > dump, just enough so that the 'nodump' flag is handled > > properly. > > Actually the patch has more than just 'nodump'. You've done a lot of > style changes too. :-( I actually meant to say that only the nodump changes were guaranteed to be merged fully. I think all of the NetBSD version ought to be merged and the chances of overlooking bits are much lower when you merge most of the code. > > It's bigger than strictly neccessary but I have tried to > > minimize the diffs between the two BSDs. > > Which should not be done at the same time. What's the reason? If our version is the same as theirs we won't have any bugs they don't have, right? > OK, I will now go out on a limb and say I'll handle the 'nodump' changes > to dump on Monday. So nobody, please do the same. What about merging the rest of the NetBSD stuff? What do you suggest? -- Jeroen C. van Gelderen o _ _ _ jeroen@vangelderen.org _o /\_ _ \\o (_)\__/o (_) _< \_ _>(_) (_)/<_ \_| \ _|/' \/ (_)>(_) (_) (_) (_) (_)' _\o_ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 7:54:36 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 07:54:34 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id C50CD37B400 for ; Mon, 11 Dec 2000 07:54:33 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id QAA55956; Mon, 11 Dec 2000 16:51:47 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: "Jeroen C. van Gelderen" Cc: arch@FreeBSD.ORG, Poul-Henning Kamp , Julian Elischer Subject: Re: inheriting the "nodump" flag ? References: <440.976476322@critter> <3A3454DC.5A9E7FF1@vangelderen.org> <20001211005908.H89853@dragon.nuxi.com> <3A34F335.CD5CCD74@vangelderen.org> From: Dag-Erling Smorgrav Date: 11 Dec 2000 16:51:46 +0100 In-Reply-To: "Jeroen C. van Gelderen"'s message of "Mon, 11 Dec 2000 11:31:01 -0400" Message-ID: Lines: 15 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG "Jeroen C. van Gelderen" writes: > David O'Brien wrote: > > On Mon, Dec 11, 2000 at 12:15:24AM -0400, Jeroen C. van Gelderen wrote: > > > It's bigger than strictly neccessary but I have tried to > > > minimize the diffs between the two BSDs. > > Which should not be done at the same time. > What's the reason? If our version is the same as theirs > we won't have any bugs they don't have, right? So you can easily see the functional changes when reading diffs, and skip revisions marked as "style only". DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 8:57: 4 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 08:57:03 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from gratis.grondar.za (grouter.grondar.za [196.7.18.65]) by hub.freebsd.org (Postfix) with ESMTP id D942037B400 for ; Mon, 11 Dec 2000 08:56:59 -0800 (PST) Received: from grondar.za (root@gratis.grondar.za [196.7.18.133]) by gratis.grondar.za (8.11.1/8.11.1) with ESMTP id eBBGukI34896; Mon, 11 Dec 2000 18:56:46 +0200 (SAST) (envelope-from mark@grondar.za) Message-Id: <200012111656.eBBGukI34896@gratis.grondar.za> To: Bruce Evans Cc: arch@FreeBSD.ORG Subject: Re: RANDOMDEV inspired realitycheck regarding i386/i486... References: In-Reply-To: ; from Bruce Evans "Mon, 11 Dec 2000 21:32:23 +1100." Date: Mon, 11 Dec 2000 18:56:50 +0200 From: Mark Murray Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > I crashed -current a few times recently, and entropy seeding during > startup took several minutes of leaning on the space bar. This was > initially with a week old kernel and finally with a current kernel > and a broken ps. Apparently there isn't a enough entropy unless ps > produces a normal amount. Bruce, Please try this; short-circuit all the /etc/rc pseudo-entropy gathering (the processes that run a load of junk to try to reseed) and replace them with something that you _know_ is short-and-quick (like 'echo "kfjgsdkjfhskdjfhksdhf" >/dev/random') and let me know how long that takes. WARNING - this will make the startup state of /dev/random dangerously predictable and repeatable. Thanks! M -- Mark Murray Join the anti-SPAM movement: http://www.cauce.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 10: 3:26 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 10:03:24 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id A02D437B400 for ; Mon, 11 Dec 2000 10:03:23 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id TAA56515; Mon, 11 Dec 2000 19:03:22 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: arch@freebsd.org Subject: Safe string formatting in the kernel From: Dag-Erling Smorgrav Date: 11 Dec 2000 19:03:21 +0100 Message-ID: Lines: 16 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I've implemented a set of functions for performing safe string formatting in the kernel, based on an initial idea (and design) by Poul-Henning. There's a patch up on freefall: http://people.freebsd.org/~des/software/sbuf-20001211.diff The current implementation is not complete (I plan to add support for automatically growing sbufs instead of failing when they fill up) but it's functional enough for most of the uses PHK and I have for it now. The code has already been reviewed by a number of people, so unless there are major objections I'll commit it some time later this week. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 11:13:32 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 11:13:30 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 4FC9737B400 for ; Mon, 11 Dec 2000 11:13:26 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id UAA56837; Mon, 11 Dec 2000 20:13:25 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel References: From: Dag-Erling Smorgrav Date: 11 Dec 2000 20:13:24 +0100 In-Reply-To: Dag-Erling Smorgrav's message of "11 Dec 2000 19:03:21 +0100" Message-ID: Lines: 10 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Dag-Erling Smorgrav writes: > http://people.freebsd.org/~des/software/sbuf-20001211.diff That patch has a bug. I put up a new one: http://people.freebsd.org/~des/software/sbuf-20001211b.diff DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 12:14:50 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 12:14:49 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 829C537B400 for ; Mon, 11 Dec 2000 12:14:48 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id VAA57116; Mon, 11 Dec 2000 21:14:47 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel References: From: Dag-Erling Smorgrav Date: 11 Dec 2000 21:14:46 +0100 In-Reply-To: Dag-Erling Smorgrav's message of "11 Dec 2000 19:03:21 +0100" Message-ID: Lines: 8 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I've added a patch that makes linprocfs use sbufs instead of sprintf(): http://people.freebsd.org/~des/software/linprocfs-20001211.diff DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 13:53: 9 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 13:53:05 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 6A37737B400 for ; Mon, 11 Dec 2000 13:53:05 -0800 (PST) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by pike.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id eBBLr3E58088 for ; Mon, 11 Dec 2000 13:53:03 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 Date: Mon, 11 Dec 2000 13:53:20 -0800 (PST) From: John Baldwin To: arch@FreeBSD.org Subject: Can !curproc touch Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I've got a question about p_cred in proc, specifically p_cred->pc_ucred. In several VOP's and other places we use p_cred->pc_ucred (aka p_ucred) as the credentials if we don't already have one. The problem arises if another process can crfree() that ucred either by a crcopy() or a direct crfree() of p_ucred. In that case, the ucred being passed around through VFS will be invalid. For example, suppose cpu A is running process P and does a VOP() using P->p_ucred. Now, suppose process Q on cpu B does a setgroups() on P, thus doing a crcopy() of p_ucred. This alone won't break things because if there is only 1 reference to a ucred, we don't crfree() it in crcopy(), thus we won't end up with an empty ucred, though the ucred will _change_ halfway through the VOP, which could be ugly. OTOH, if the ucred has a refcount > 1, then it will be crfree()'d, but there will still be a reference to it. However, if the another CPU/process releases the remaining ucred references before the VOP finishes, you can have problems. However, this can only happen if a process other than P can read or write to P->p_ucred. Candidates for this might be aio, NFS, etc. If only P can touch p_ucred, then I can leave it at is current state (k) and it doesn't need to be locked. On the other hand, if p_ucred can be read/written by someone other than P, then I need to lock accesses to p_ucred with the proc structure lock, and I need to modify consumers of ucred's as follows: VFOO(p, p->p_ucred, ...); becomes: struct ucred *uc; ... PROC_LOCK(p); uc = p->p_ucred; crhold(uc); PROC_UNLOCK(p); VFOO(p, uc, ...); crfree(uc); -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 13:58:22 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 13:58:19 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from io.yi.org (unknown [24.70.218.157]) by hub.freebsd.org (Postfix) with ESMTP id 18C9937B402 for ; Mon, 11 Dec 2000 13:58:19 -0800 (PST) Received: from io.yi.org (localhost.gvcl1.bc.wave.home.com [127.0.0.1]) by io.yi.org (Postfix) with ESMTP id 1E73EBA7D; Mon, 11 Dec 2000 13:58:16 -0800 (PST) X-Mailer: exmh version 2.1.1 10/15/1999 To: Dag-Erling Smorgrav Cc: arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel In-Reply-To: Message from Dag-Erling Smorgrav of "11 Dec 2000 20:13:24 +0100." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 11 Dec 2000 13:58:15 -0800 From: Jake Burkholder Message-Id: <20001211215816.1E73EBA7D@io.yi.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Dag-Erling Smorgrav writes: > > http://people.freebsd.org/~des/software/sbuf-20001211.diff > > That patch has a bug. I put up a new one: > > http://people.freebsd.org/~des/software/sbuf-20001211b.diff > > DES > -- > Dag-Erling Smorgrav - des@ofug.org > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message This is interesting. Some comments: +MALLOC_DEFINE(sbuf_malloc_type, "sbuf", "string buffers"); Aren't malloc types usually capital letters that look like a flag? like M_SBUF? +struct sbuf { + char *s_buf; /* storage buffer */ + size_t s_size; /* size of storage buffer */ + size_t s_len; /* current length of string */ + int s_dynamic; /* storage buffer must be freed */ + int s_done; /* sbuf is finished and read-only */ + int s_flags; /* flags */ + struct sbuf *s_next; /* next in chain */ +}; Why not make s_dynamic live in s_flags? Maybe s_done also? Maybe also have a flag S_INITED instead of using s_buf as a flag? If you're going to make a list shouldn't you use the queue(3) macros? +sbuf_putc(struct sbuf *s, int c) ... + s->s_buf[s->s_len] = 0; '\0' ? +sbuf_printf(struct sbuf *s, char *fmt, ...) ... + len = kvprintf(fmt, _sbuf_pchar, s, 10, ap); If the idea is to invent our own formats, and sbuf_printf calls kvprintf, then the new formats need to be added to kvprintf right? In that case one would be able to use them with just normal kernel printf which seems like something we're trying to avoid. Maybe some kind of printf format switch table would be useful? Like an array of function pointers that would get passed to kvprintf, where the ascii value of the format character is the index of a function to parse the format. Jake To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 14: 2:10 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 14:02:08 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 2708437B400 for ; Mon, 11 Dec 2000 14:02:08 -0800 (PST) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by pike.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id eBBM26E58594 for ; Mon, 11 Dec 2000 14:02:06 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 Date: Mon, 11 Dec 2000 14:02:22 -0800 (PST) From: John Baldwin To: arch@FreeBSD.org Subject: curpriority... Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Well, it seems that curpriority is a bit bogus right now. First of all, it is only set in msleep/mawait when a process resumes. Shouldn't it be set at the end of mi_switch() when a process resumes instead, so that it is set for every process switch? Was it only done in tsleep/await before as an optimization of some sort? Secondly, it probably needs to become a per-CPU data variable, rather than one that is global to the entire system.. Comments? -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 14:10:10 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 14:10:07 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from smtp02.primenet.com (smtp02.primenet.com [206.165.6.132]) by hub.freebsd.org (Postfix) with ESMTP id 8D7A037B400; Mon, 11 Dec 2000 14:10:07 -0800 (PST) Received: (from daemon@localhost) by smtp02.primenet.com (8.9.3/8.9.3) id PAA25135; Mon, 11 Dec 2000 15:05:23 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp02.primenet.com, id smtpdAAAnca49W; Mon Dec 11 15:05:17 2000 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id PAA29128; Mon, 11 Dec 2000 15:09:57 -0700 (MST) From: Terry Lambert Message-Id: <200012112209.PAA29128@usr08.primenet.com> Subject: Re: Can !curproc touch To: jhb@FreeBSD.ORG (John Baldwin) Date: Mon, 11 Dec 2000 22:09:57 +0000 (GMT) Cc: arch@FreeBSD.ORG In-Reply-To: from "John Baldwin" at Dec 11, 2000 01:53:20 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: tlambert@usr08.primenet.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > I've got a question about p_cred in proc, specifically p_cred->pc_ucred. In > several VOP's and other places we use p_cred->pc_ucred (aka p_ucred) as the > credentials if we don't already have one. The problem arises if another > process can crfree() that ucred either by a crcopy() or a direct crfree() of > p_ucred. I would suggest that credential instances are established separately from their use. Ideally, if I am logged in 5 places, I have exactly one instance of my credentials structure lying around. This is much more important for things like a fixed SMBFS, which uses per-user credentials across the mount in order to establish per user zccess controls, enforced by the server (the current mount can not do this, since there is no way for the kernel to query a user session about credentials). In this case, you'd expect that a credenital would be something like: struct mycred { struct mycred *nextcred; char *nonce; void *cred_for_nonce; }; When one of my login instances authenticates to a particular credential requiring system, represented by a particular nonce, then all should be authenticated (I am authenticating to the system, not to a session instance; consider an X session with multiple xterm's, where each xterm does not require seperate authentication). What this basically means is that you need a lock, or use an atomic operation, in order to maintain a reference count on the credential object. When a function enters on to dereference the credential pointer, it acquires a reference. The only remaining houskeeping is setting the reference count to one when the credential is instanced, and deleting it on the 1->0 reference transition. A lot of the issues having to do with SMP references should be handled this way; so should the vnode issues (e.g. when a vnode is referenced by the directory lookup cache, the cache should hold a reference to it; each open instance should similarly hold a reference to it, etc.). You aren't trying to protect if from manipulation, once it is established (manipulation would require a light weight lock held over the manipulation event, if it were a common case; better to set the lock only on a multiply referenced object (e.g., have something like a "has_alias" flag, and if it's not set, ignore the locking, while requiring the lock to change the flag; since the operation is atomic, and you guarantee ordering, where the lock is held, the flag is set, the lock is released, the lock is acquired for other manipulation or reading now the flag is set... you are safe to do a quick test, and unaliased operations proceed quickly). Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 14:37: 8 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 14:37:07 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id B7AB637B400 for ; Mon, 11 Dec 2000 14:37:06 -0800 (PST) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by pike.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id eBBMb0E59924; Mon, 11 Dec 2000 14:37:00 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200012112209.PAA29128@usr08.primenet.com> Date: Mon, 11 Dec 2000 14:37:16 -0800 (PST) From: John Baldwin To: Terry Lambert Subject: Re: Can !curproc touch Cc: arch@FreeBSD.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 11-Dec-00 Terry Lambert wrote: >> I've got a question about p_cred in proc, specifically p_cred->pc_ucred. In >> several VOP's and other places we use p_cred->pc_ucred (aka p_ucred) as the >> credentials if we don't already have one. The problem arises if another >> process can crfree() that ucred either by a crcopy() or a direct crfree() of >> p_ucred. [ snip ] We already share ucred's, and we already do reference counting for ucred structures. xref crfree()/crcopy()/crhold(), etc. Really, Terry, reading the code and reading the questions in detail would help here. My question is about protecting the p_ucred pointer that is part of struct proc. I'm not trying to lock the ucred itself, I'm trying to figure out how to ensure that the pointer to a ucred I get out of proc is valid when I pass it to a VOP and that it _stays_ valid the entire time. I know that if need be I can do it by protecting the p_ucred pointer with the proc lock and bumping the refcount before passing it down the VOP stack, but if I can get away w/o having to do that I'd like to avoid the cost. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 16:12:44 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 16:12:43 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id DC95C37B400 for ; Mon, 11 Dec 2000 16:12:42 -0800 (PST) Received: (qmail 93149 invoked by uid 1142); 12 Dec 2000 00:12:42 -0000 Date: 11 Dec 2000 16:12:42 -0800 Date: Mon, 11 Dec 2000 16:12:32 -0800 From: Jason Evans To: Dag-Erling Smorgrav Cc: arch@freebsd.org Subject: Re: Safe string formatting in the kernel Message-ID: <20001211161232.D2312@canonware.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from des@ofug.org on Mon, Dec 11, 2000 at 07:03:21PM +0100 Sender: jasone@magnesium.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Mon, Dec 11, 2000 at 07:03:21PM +0100, Dag-Erling Smorgrav wrote: > I've implemented a set of functions for performing safe string > formatting in the kernel, based on an initial idea (and design) by > Poul-Henning. Can you please elaborate on what this is useful for and why we need it? I skimmed through the patch, but the answer didn't jump out at me. Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 18:49:37 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 18:49:35 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from netau1.alcanet.com.au (ntp.alcanet.com.au [203.62.196.27]) by hub.freebsd.org (Postfix) with ESMTP id 220CC37B400 for ; Mon, 11 Dec 2000 18:49:34 -0800 (PST) Received: from mfg1.cim.alcatel.com.au (mfg1.cim.alcatel.com.au [139.188.23.1]) by netau1.alcanet.com.au (8.9.3 (PHNE_18979)/8.9.3) with ESMTP id NAA12812; Tue, 12 Dec 2000 13:49:27 +1100 (EDT) Received: from gsmx07.alcatel.com.au by cim.alcatel.com.au (PMDF V5.2-32 #37645) with ESMTP id <01JXMAOQDR5SEAFNJ7@cim.alcatel.com.au>; Tue, 12 Dec 2000 13:49:19 +1100 Received: (from jeremyp@localhost) by gsmx07.alcatel.com.au (8.11.0/8.11.0) id eBC2nNM29195; Tue, 12 Dec 2000 13:49:23 +1100 (EST envelope-from jeremyp) Content-return: prohibited Date: Tue, 12 Dec 2000 13:49:23 +1100 From: Peter Jeremy Subject: Re: inheriting the "nodump" flag ? In-reply-to: <3A3454DC.5A9E7FF1@vangelderen.org>; from jeroen@vangelderen.org on Mon, Dec 11, 2000 at 12:15:24AM -0400 To: "Jeroen C. van Gelderen" Cc: arch@FreeBSD.ORG Mail-followup-to: "Jeroen C. van Gelderen" , arch@FreeBSD.ORG Message-id: <20001212134923.O69646@gsmx07.alcatel.com.au> MIME-version: 1.0 Content-type: text/plain; charset=iso-8859-1 Content-disposition: inline Content-transfer-encoding: 8BIT User-Agent: Mutt/1.2.5i References: <440.976476322@critter> <3A3454DC.5A9E7FF1@vangelderen.org> Sender: jeremyp@gsmx07.alcatel.com.au Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 2000-Dec-11 00:15:24 -0400, "Jeroen C. van Gelderen" wrote: >The NetBSD dump seems to be more evolved than the FreeBSD >version. I've partially merged the NetBSD changes into our >dump, just enough so that the 'nodump' flag is handled >properly. I will merge the rest of the NetBSD changes as >well if there is interest. Some time ago, Søren suggested that the NetBSD dump has some buffering to improve its speed. I believe this would be a worthwhile addition. >I have some patches for the NetBSD folks as well and if >both parties accept them we can have a virtually identical >traverse.c on the two BSDs. IMHO, avoiding gratuitous differences between the various *BSDs is a good idea. Peter To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 18:54:57 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 18:54:55 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from citusc.usc.edu (citusc.usc.edu [128.125.38.123]) by hub.freebsd.org (Postfix) with ESMTP id 9D0D137B400 for ; Mon, 11 Dec 2000 18:54:55 -0800 (PST) Received: (from kris@localhost) by citusc.usc.edu (8.9.3/8.9.3) id SAA01782; Mon, 11 Dec 2000 18:56:10 -0800 Date: Mon, 11 Dec 2000 18:56:10 -0800 From: kris@citusc.usc.edu To: Dag-Erling Smorgrav Cc: arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel Message-ID: <20001211185610.A1741@citusc.usc.edu> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="AqsLC8rIMeq19msA" Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: ; from des@ofug.org on Mon, Dec 11, 2000 at 07:03:21PM +0100 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG --AqsLC8rIMeq19msA Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Mon, Dec 11, 2000 at 07:03:21PM +0100, Dag-Erling Smorgrav wrote: > I've implemented a set of functions for performing safe string > formatting in the kernel, based on an initial idea (and design) by > Poul-Henning. There's a patch up on freefall: I haven't reviewed this implementation, but introducing a secure string handling API into the kernel has my support as security officer. The current abuse of sprintf() in the kernel is really, really scary. Kris --AqsLC8rIMeq19msA Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6NZPKWry0BWjoQKURAg/cAKC3ed/YmIHQTM2dtfHuZF8Qo+6fygCdHLm6 ATYgkgSvJ0hYq6fHZZ2zmS8= =kcYH -----END PGP SIGNATURE----- --AqsLC8rIMeq19msA-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 19: 1:30 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 19:01:28 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (placeholder-dcat-1076843399.broadbandoffice.net [64.47.83.135]) by hub.freebsd.org (Postfix) with ESMTP id F002A37B400 for ; Mon, 11 Dec 2000 19:01:27 -0800 (PST) Received: (from dillon@localhost) by earth.backplane.com (8.11.1/8.9.3) id eBC2xfb99004; Mon, 11 Dec 2000 18:59:41 -0800 (PST) (envelope-from dillon) Date: Mon, 11 Dec 2000 18:59:41 -0800 (PST) From: Matt Dillon Message-Id: <200012120259.eBC2xfb99004@earth.backplane.com> To: kris@citusc.usc.edu Cc: Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel References: <20001211185610.A1741@citusc.usc.edu> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :On Mon, Dec 11, 2000 at 07:03:21PM +0100, Dag-Erling Smorgrav wrote: :> I've implemented a set of functions for performing safe string :> formatting in the kernel, based on an initial idea (and design) by :> Poul-Henning. There's a patch up on freefall: : :I haven't reviewed this implementation, but introducing a secure :string handling API into the kernel has my support as security :officer. The current abuse of sprintf() in the kernel is really, :really scary. : :Kris sprintf(), strcpy(), and strcat(). But why not just replace those functions with an snprintf() equivalent? I don't think we really need a dynamic string allocation mechanism in the kernel, there is virtually nowhere where it would actually be of any use. sprintf() -> snprintf(...) strcpy() -> sn_strcpy(dst, src, sizeof_destination_buffer) strcat() -> sn_strcat(dst, src, sizeof_destination_buffer) -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 19:10: 3 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 19:09:59 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from rina.r.dl.itc.u-tokyo.ac.jp (rina.r.dl.itc.u-tokyo.ac.jp [133.11.199.247]) by hub.freebsd.org (Postfix) with ESMTP id D47BD37B400 for ; Mon, 11 Dec 2000 19:09:57 -0800 (PST) Received: from rina.r.dl.itc.u-tokyo.ac.jp (localhost [127.0.0.1]) by rina.r.dl.itc.u-tokyo.ac.jp (8.11.1+3.4Wpre/3.7W-rina.r-0.1-11.01.2000) with ESMTP id eBC39rC06587; Tue, 12 Dec 2000 12:09:55 +0900 (JST) Date: Tue, 12 Dec 2000 12:09:52 +0900 Message-ID: From: Seigo Tanimura To: tanimura@r.dl.itc.u-tokyo.ac.jp Cc: dillon@earth.backplane.com, bright@wintelcom.net, arch@FreeBSD.ORG Subject: Re: Even 1GB KVA is not enough, but we have no more space In-Reply-To: In your message of "Sat, 09 Dec 2000 14:24:06 +0900" <86elzi88w9.wl@silver.carrots.uucp.r.dl.itc.u-tokyo.ac.jp> References: <20001207013611.O16205@fw.wintelcom.net> <20001207015651.P16205@fw.wintelcom.net> <200012071952.eB7JqXm11711@earth.backplane.com> <86elzi88w9.wl@silver.carrots.uucp.r.dl.itc.u-tokyo.ac.jp> User-Agent: Wanderlust/1.1.1 (Purple Rain) SEMI/1.13.7 (Awazu) FLIM/1.13.2 (Kasanui) MULE XEmacs/21.1 (patch 12) (Channel Islands) (i386--freebsd) Organization: Digital Library Research Division, Information Techinology Centre, The University of Tokyo MIME-Version: 1.0 (generated by SEMI 1.13.7 - "Awazu") Content-Type: text/plain; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sat, 09 Dec 2000 14:24:06 +0900, Seigo Tanimura said: Seigo> And the saga continues. After regulating the size of struct swblock, Seigo> ffs_vget() failed to allocalte a new vnode. At the time the PowerEdge Seigo> failed, the kernel held around 197K vnodes, which is as large as Seigo> 46MB. This time I reduced the size of kmem_map, the pool of malloc(9). Sorry, please forget this matter. The culprit truned out to be a race condition of testing-and-setting ffs_inode_hash_lock in ffs_vget(). Mutexifing the test-and-sleep-or-set and wakeup-reset operations was enough to fix the problem. Now my kernel allocates up to 316K vnodes on make buildworld and release, so malloc(9) pool of 200MB is sufficient. (316K vnodes occupy about 90-100MB in kmem_map) The updated patch is now at URI: http://people.FreeBSD.org/~tanimura/patches/kvmfix.diff which includes A. adaptation of swap metadata size, B. rejection of swapon(2) upon failure of swap zone allocation, and C. atomic test-and-set and reset of {ffs,ifs}_inode_hash_lock. I am going to commit this patch in about 6 hours from now. -- Seigo Tanimura To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 19:20:13 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 19:20:11 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from netau1.alcanet.com.au (ntp.alcanet.com.au [203.62.196.27]) by hub.freebsd.org (Postfix) with ESMTP id 4DE7F37B400 for ; Mon, 11 Dec 2000 19:20:10 -0800 (PST) Received: from mfg1.cim.alcatel.com.au (mfg1.cim.alcatel.com.au [139.188.23.1]) by netau1.alcanet.com.au (8.9.3 (PHNE_18979)/8.9.3) with ESMTP id OAA16449; Tue, 12 Dec 2000 14:20:05 +1100 (EDT) Received: from gsmx07.alcatel.com.au by cim.alcatel.com.au (PMDF V5.2-32 #37645) with ESMTP id <01JXMBQO8JPCEAFV8Y@cim.alcatel.com.au>; Tue, 12 Dec 2000 14:19:55 +1100 Received: (from jeremyp@localhost) by gsmx07.alcatel.com.au (8.11.0/8.11.0) id eBC3JwY29509; Tue, 12 Dec 2000 14:19:58 +1100 (EST envelope-from jeremyp) Content-return: prohibited Date: Tue, 12 Dec 2000 14:19:58 +1100 From: Peter Jeremy Subject: Re: Safe string formatting in the kernel In-reply-to: ; from des@ofug.org on Mon, Dec 11, 2000 at 08:13:24PM +0100 To: Dag-Erling Smorgrav Cc: arch@FreeBSD.ORG Mail-followup-to: Dag-Erling Smorgrav , arch@FreeBSD.ORG Message-id: <20001212141958.P69646@gsmx07.alcatel.com.au> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-disposition: inline User-Agent: Mutt/1.2.5i References: Sender: jeremyp@gsmx07.alcatel.com.au Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 2000-Dec-11 20:13:24 +0100, Dag-Erling Smorgrav wrote: > http://people.freebsd.org/~des/software/sbuf-20001211b.diff Overall the purpose of this isn't clear to me. It doesn't appear to have any real advantages over using the standard string functions. The main advantage I can see for having a proper set of string functions would be to support dynamic (growable) strings and sbuf uses a fixed size buffer. +struct sbuf { + char *s_buf; /* storage buffer */ + size_t s_size; /* size of storage buffer */ + size_t s_len; /* current length of string */ + int s_dynamic; /* storage buffer must be freed */ + int s_done; /* sbuf is finished and read-only */ + int s_flags; /* flags */ + struct sbuf *s_next; /* next in chain */ +}; The s_flags and s_next fields appear to be unused. What is their purpose? 2 int's (s_dynamic and s_done) to store 2 boolean values seems excessive. I agree that having a flags word is useful (even if there aren't currently any caller values for it). Once a flags field is defined, all the flags should go there. Also, I believe it would be better to have s_next adjacent to s_buf. (To avoid padding before s_next - though the struct in its current form will have 4 bytes of padding on the Alpha/IA64 anyway). + while (*ptr && s->s_len < s->s_size - 1) + s->s_buf[s->s_len++] = *ptr++; + s->s_buf[s->s_len] = '\0'; How about strlcpy()? Peter To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 20:32:56 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 20:32:51 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from assaris.sics.se (assaris.sics.se [193.10.66.234]) by hub.freebsd.org (Postfix) with ESMTP id E2F2537B402 for ; Mon, 11 Dec 2000 20:32:49 -0800 (PST) Received: (from assar@localhost) by assaris.sics.se (8.9.3/8.9.3) id FAA54822; Tue, 12 Dec 2000 05:32:48 +0100 (CET) (envelope-from assar) Sender: assar@assaris.sics.se From: assar@FreeBSD.ORG To: Matt Dillon Cc: kris@citusc.usc.edu, Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel References: <20001211185610.A1741@citusc.usc.edu> <200012120259.eBC2xfb99004@earth.backplane.com> Date: 12 Dec 2000 05:32:48 +0100 In-Reply-To: Matt Dillon's message of "Mon, 11 Dec 2000 18:59:41 -0800 (PST)" Message-ID: <5lhf4ap8cv.fsf@assaris.sics.se> Lines: 10 User-Agent: Gnus/5.070098 (Pterodactyl Gnus v0.98) Emacs/20.6 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG --=-=-= Matt Dillon writes: > strcpy() -> sn_strcpy(dst, src, sizeof_destination_buffer) > strcat() -> sn_strcat(dst, src, sizeof_destination_buffer) strlcpy and strlcat. Why keep the API different for no good reason? /assar --=-=-= Content-Disposition: attachment; filename=f-sys-d Content-Description: patch for adding strlcpy and strlcat to kernel Index: conf/files =================================================================== RCS file: /home/ncvs/src/sys/conf/files,v retrieving revision 1.451 diff -u -w -r1.451 files --- conf/files 2000/12/12 01:14:24 1.451 +++ conf/files 2000/12/12 03:48:43 @@ -717,6 +717,8 @@ libkern/strcat.c standard libkern/strcmp.c standard libkern/strcpy.c standard +libkern/strlcat.c standard +libkern/strlcpy.c standard libkern/strlen.c standard libkern/strncmp.c standard libkern/strncpy.c standard Index: sys/libkern.h =================================================================== RCS file: /home/ncvs/src/sys/sys/libkern.h,v retrieving revision 1.23 diff -u -w -r1.23 libkern.h --- sys/libkern.h 2000/09/03 11:32:07 1.23 +++ sys/libkern.h 2000/12/12 03:49:10 @@ -84,6 +84,8 @@ char *strcat __P((char *, const char *)); int strcmp __P((const char *, const char *)); char *strcpy __P((char *, const char *)); +size_t strlcat __P((char *, const char *, size_t)); +size_t strlcpy __P((char *, const char *, size_t)); size_t strlen __P((const char *)); int strncmp __P((const char *, const char *, size_t)); char *strncpy __P((char *, const char *, size_t)); --- /dev/null Tue Dec 12 05:30:54 2000 +++ sys/libkern/strlcpy.c Tue Dec 12 04:37:24 2000 @@ -0,0 +1,65 @@ +/* $OpenBSD: strlcpy.c,v 1.4 1999/05/01 18:56:41 millert Exp $ */ + +/* + * Copyright (c) 1998 Todd C. Miller + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote products + * derived from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, + * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY + * AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL + * THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, + * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, + * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; + * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, + * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR + * OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF + * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#include + +/* + * Copy src to string dst of size siz. At most siz-1 characters + * will be copied. Always NUL terminates (unless siz == 0). + * Returns strlen(src); if retval >= siz, truncation occurred. + */ +size_t strlcpy(dst, src, siz) + char *dst; + const char *src; + size_t siz; +{ + register char *d = dst; + register const char *s = src; + register size_t n = siz; + + /* Copy as many bytes as will fit */ + if (n != 0 && --n != 0) { + do { + if ((*d++ = *s++) == 0) + break; + } while (--n != 0); + } + + /* Not enough room in dst, add NUL and traverse rest of src */ + if (n == 0) { + if (siz != 0) + *d = '\0'; /* NUL-terminate dst */ + while (*s++) + ; + } + + return(s - src - 1); /* count does not include NUL */ +} --- /dev/null Tue Dec 12 05:30:54 2000 +++ sys/libkern/strlcat.c Tue Dec 12 04:37:38 2000 @@ -0,0 +1,68 @@ +/* $OpenBSD: strlcat.c,v 1.2 1999/06/17 16:28:58 millert Exp $ */ + +/* + * Copyright (c) 1998 Todd C. Miller + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote products + * derived from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, + * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY + * AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL + * THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, + * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, + * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; + * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, + * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR + * OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF + * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#include + +/* + * Appends src to string dst of size siz (unlike strncat, siz is the + * full size of dst, not space left). At most siz-1 characters + * will be copied. Always NUL terminates (unless siz == 0). + * Returns strlen(src); if retval >= siz, truncation occurred. + */ +size_t strlcat(dst, src, siz) + char *dst; + const char *src; + size_t siz; +{ + register char *d = dst; + register const char *s = src; + register size_t n = siz; + size_t dlen; + + /* Find the end of dst and adjust bytes left but don't go past end */ + while (*d != '\0' && n-- != 0) + d++; + dlen = d - dst; + n = siz - dlen; + + if (n == 0) + return(dlen + strlen(s)); + while (*s != '\0') { + if (n != 1) { + *d++ = *s; + n--; + } + s++; + } + *d = '\0'; + + return(dlen + (s - src)); /* count does not include NUL */ +} --=-=-=-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 20:55:25 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 20:55:24 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (placeholder-dcat-1076843399.broadbandoffice.net [64.47.83.135]) by hub.freebsd.org (Postfix) with ESMTP id C252437B400; Mon, 11 Dec 2000 20:55:20 -0800 (PST) Received: (from dillon@localhost) by earth.backplane.com (8.11.1/8.9.3) id eBC4tKV99946; Mon, 11 Dec 2000 20:55:20 -0800 (PST) (envelope-from dillon) Date: Mon, 11 Dec 2000 20:55:20 -0800 (PST) From: Matt Dillon Message-Id: <200012120455.eBC4tKV99946@earth.backplane.com> To: assar@FreeBSD.ORG Cc: kris@citusc.usc.edu, Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel References: <20001211185610.A1741@citusc.usc.edu> <200012120259.eBC2xfb99004@earth.backplane.com> <5lhf4ap8cv.fsf@assaris.sics.se> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG : : :--=-=-= : :Matt Dillon writes: :> strcpy() -> sn_strcpy(dst, src, sizeof_destination_buffer) :> strcat() -> sn_strcat(dst, src, sizeof_destination_buffer) : :strlcpy and strlcat. Why keep the API different for no good reason? : :/assar Yes, you are right. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 23:31:39 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 23:31:37 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id A859A37B400 for ; Mon, 11 Dec 2000 23:31:36 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id eBC7U5L18192; Tue, 12 Dec 2000 08:30:06 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Matt Dillon Cc: kris@citusc.usc.edu, Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel In-Reply-To: Your message of "Mon, 11 Dec 2000 18:59:41 PST." <200012120259.eBC2xfb99004@earth.backplane.com> Date: Tue, 12 Dec 2000 08:30:05 +0100 Message-ID: <18190.976606205@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <200012120259.eBC2xfb99004@earth.backplane.com>, Matt Dillon writes: > sprintf(), strcpy(), and strcat(). But why not just replace those > functions with an snprintf() equivalent? I don't think we really need > a dynamic string allocation mechanism in the kernel, there is virtually > nowhere where it would actually be of any use. There are several places where this new API would make the code simpler and less prone to overflowable errors. procfs and various netgraph nodes spring to mind immediately. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Dec 11 23:35:55 2000 From owner-freebsd-arch@FreeBSD.ORG Mon Dec 11 23:35:54 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 3ED2A37B400 for ; Mon, 11 Dec 2000 23:35:53 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id eBC7ZhL18371; Tue, 12 Dec 2000 08:35:43 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Peter Jeremy Cc: Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel In-Reply-To: Your message of "Tue, 12 Dec 2000 14:19:58 +1100." <20001212141958.P69646@gsmx07.alcatel.com.au> Date: Tue, 12 Dec 2000 08:35:43 +0100 Message-ID: <18369.976606543@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <20001212141958.P69646@gsmx07.alcatel.com.au>, Peter Jeremy writes: >On 2000-Dec-11 20:13:24 +0100, Dag-Erling Smorgrav wrote: >> http://people.freebsd.org/~des/software/sbuf-20001211b.diff > >Overall the purpose of this isn't clear to me. It doesn't appear to >have any real advantages over using the standard string functions. >The main advantage I can see for having a proper set of string >functions would be to support dynamic (growable) strings and sbuf uses a >fixed size buffer. The fixed size buffer is just to keep the initial implementation simple I think. Growable buffers is indeed catered for in the API (that's why the sbuf_finish() function is there. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 6:35:38 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 06:35:35 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id A6FD637B404 for ; Tue, 12 Dec 2000 06:35:34 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id PAA61075; Tue, 12 Dec 2000 15:35:24 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Jake Burkholder Cc: arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel References: <20001211215816.1E73EBA7D@io.yi.org> From: Dag-Erling Smorgrav Date: 12 Dec 2000 15:35:23 +0100 In-Reply-To: Jake Burkholder's message of "Mon, 11 Dec 2000 13:58:15 -0800" Message-ID: Lines: 68 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Jake Burkholder writes: > This is interesting. I hoped it would be :) > Aren't malloc types usually capital letters that look like a flag? > like M_SBUF? You're probably right; I didn't check. I didn't think it mattered since it's only used internally in subr_sbuf.c. > Why not make s_dynamic live in s_flags? Maybe s_done also? Good idea. Don't know why it didn't occur to me. > Maybe also have a flag S_INITED instead of using s_buf as a flag? s_buf isn't used as a flag except in assertions. > If you're going to make a list shouldn't you use the queue(3) macros? Because I don't really want a list. I'm not even sure I need s_next. I haven't decided how to implement sbuf growth yet; I added s_next because I thought there was a fair chance I'd need it and didn't want to change the struct later. But I see now that I need to modify the struct anyway, so I'll probably discard s_next and either use queue macros or something totally different. The rationale for using a chain, by the way, is that if you need to extend the buffer several times, you want to put off concatenating segments until sbuf_finish(). If it weren't for that, you could just malloc() a new buffer, copy the old data into the new buffer, free the old buffer, and point s_buf at the new buffer. > > +sbuf_putc(struct sbuf *s, int c) > ... > + s->s_buf[s->s_len] = 0; > > '\0' ? Yes, that was a slip. > +sbuf_printf(struct sbuf *s, char *fmt, ...) > ... > + len = kvprintf(fmt, _sbuf_pchar, s, 10, ap); > > If the idea is to invent our own formats, and sbuf_printf calls kvprintf, > then the new formats need to be added to kvprintf right? The idea is not to invent our own formats. At least, the idea is not to have sbuf_printf() support formats which printf() or sprintf() or log() don't. If you're thinking of my %a / %A patch, it is totally unrelated to this matter. > In that case > one would be able to use them with just normal kernel printf which seems > like something we're trying to avoid. Maybe some kind of printf format > switch table would be useful? That may be a good idea, but it's way outside the scope of the sbuf interface. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 6:42:35 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 06:42:34 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id F3F3A37B404; Tue, 12 Dec 2000 06:42:32 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id PAA61115; Tue, 12 Dec 2000 15:42:29 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: assar@FreeBSD.ORG Cc: Matt Dillon , kris@citusc.usc.edu, arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel References: <20001211185610.A1741@citusc.usc.edu> <200012120259.eBC2xfb99004@earth.backplane.com> <5lhf4ap8cv.fsf@assaris.sics.se> From: Dag-Erling Smorgrav Date: 12 Dec 2000 15:42:29 +0100 In-Reply-To: assar@FreeBSD.ORG's message of "12 Dec 2000 05:32:48 +0100" Message-ID: Lines: 22 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG assar@FreeBSD.ORG writes: > Matt Dillon writes: > > strcpy() -> sn_strcpy(dst, src, sizeof_destination_buffer) > > strcat() -> sn_strcat(dst, src, sizeof_destination_buffer) > strlcpy and strlcat. Why keep the API different for no good reason? Because there are other issues than just overflowing the buffer. There's the issue of truncation (a lot of code uses snprintf() etc. without checking if the resulting string was actually truncated, which may be a security risk of its own), and there's the issue of using large amounts of stack space for buffers (procfs and linprocfs are notorious offenders in both these areas, but they're not the only ones) Vulnerabilities were recently found in the procfs code which were successfully solved with snprintf(), but could have been (and hopefully will be) solved in a much more elegant and future-proof manner using sbufs. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 6:48:45 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 06:48:42 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 8E3CD37B400 for ; Tue, 12 Dec 2000 06:48:41 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id PAA61139; Tue, 12 Dec 2000 15:48:32 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Peter Jeremy Cc: arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel References: <20001212141958.P69646@gsmx07.alcatel.com.au> From: Dag-Erling Smorgrav Date: 12 Dec 2000 15:48:32 +0100 In-Reply-To: Peter Jeremy's message of "Tue, 12 Dec 2000 14:19:58 +1100" Message-ID: Lines: 37 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Peter Jeremy writes: > Overall the purpose of this isn't clear to me. It doesn't appear to > have any real advantages over using the standard string functions. This has been explained elsewhere in this thread both by Poul-Henning and myself. > The main advantage I can see for having a proper set of string > functions would be to support dynamic (growable) strings and sbuf uses a > fixed size buffer. Please read the comments that precede the patch. > The s_flags and s_next fields appear to be unused. What is their > purpose? This is documented in patch itself and in the comments that precede it. > 2 int's (s_dynamic and s_done) to store 2 boolean values seems > excessive. Yes, I'll move those into s_flags. > How about strlcpy()? I suppose sbuf_cat() and sbuf_cpy() could be modified to take an additional size argument (with a magic value, e.g. (size_t)(-1), meaning "the whole shebang") which would allow them to function in a manner similar to strlcat() and strlcpy(). Poul-Henning didn't specify that in his design, and I didn't think of it myself (I implemented his design almost to the letter, except for adding the flags argument to sbuf_new() and adding sbuf_setpos() and sbuf_putc()) DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 7:50: 2 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 07:49:57 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mail.interware.hu (mail.interware.hu [195.70.32.130]) by hub.freebsd.org (Postfix) with ESMTP id CC0FE37B402 for ; Tue, 12 Dec 2000 07:49:56 -0800 (PST) Received: from luanda-36.budapest.interware.hu ([195.70.51.36] helo=elischer.org) by mail.interware.hu with esmtp (Exim 3.16 #1 (Debian)) id 145rgU-0004OO-00; Tue, 12 Dec 2000 16:49:43 +0100 Sender: julian@FreeBSD.ORG Message-ID: <3A361F0E.F015A262@elischer.org> Date: Tue, 12 Dec 2000 04:50:22 -0800 From: Julian Elischer X-Mailer: Mozilla 4.7 [en] (X11; U; FreeBSD 5.0-CURRENT i386) X-Accept-Language: en, hu MIME-Version: 1.0 To: Poul-Henning Kamp Cc: Matt Dillon , kris@citusc.usc.edu, Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel References: <18190.976606205@critter> Content-Type: text/plain; charset=iso-8859-15 Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Poul-Henning Kamp wrote: > > In message <200012120259.eBC2xfb99004@earth.backplane.com>, Matt Dillon writes: > > > sprintf(), strcpy(), and strcat(). But why not just replace those > > functions with an snprintf() equivalent? I don't think we really need > > a dynamic string allocation mechanism in the kernel, there is virtually > > nowhere where it would actually be of any use. > > There are several places where this new API would make the code > simpler and less prone to overflowable errors. procfs and various > netgraph nodes spring to mind immediately. hmmm such as? > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message -- __--_|\ Julian Elischer / \ julian@elischer.org ( OZ ) World tour 2000 ---> X_.---._/ presently in: Budapest v To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 8:13:56 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 08:13:52 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id E44E537B400 for ; Tue, 12 Dec 2000 08:13:51 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id eBCGBve48906; Tue, 12 Dec 2000 17:11:57 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Julian Elischer Cc: Matt Dillon , kris@citusc.usc.edu, Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel In-Reply-To: Your message of "Tue, 12 Dec 2000 04:50:22 PST." <3A361F0E.F015A262@elischer.org> Date: Tue, 12 Dec 2000 17:11:57 +0100 Message-ID: <48904.976637517@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <3A361F0E.F015A262@elischer.org>, Julian Elischer writes: >Poul-Henning Kamp wrote: >> >> In message <200012120259.eBC2xfb99004@earth.backplane.com>, Matt Dillon writes: >> >> > sprintf(), strcpy(), and strcat(). But why not just replace those >> > functions with an snprintf() equivalent? I don't think we really need >> > a dynamic string allocation mechanism in the kernel, there is virtually >> > nowhere where it would actually be of any use. >> >> There are several places where this new API would make the code >> simpler and less prone to overflowable errors. procfs and various >> netgraph nodes spring to mind immediately. > >hmmm such as? The "config" and "status" returns. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 8:34: 2 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 08:34:00 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 2C57B37B400 for ; Tue, 12 Dec 2000 08:33:59 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id RAA61654; Tue, 12 Dec 2000 17:33:02 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Julian Elischer Cc: Poul-Henning Kamp , Matt Dillon , kris@citusc.usc.edu, arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel References: <18190.976606205@critter> <3A361F0E.F015A262@elischer.org> From: Dag-Erling Smorgrav Date: 12 Dec 2000 17:33:01 +0100 In-Reply-To: Julian Elischer's message of "Tue, 12 Dec 2000 04:50:22 -0800" Message-ID: Lines: 28 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Julian Elischer writes: > Poul-Henning Kamp wrote: > > There are several places where this new API would make the code > > simpler and less prone to overflowable errors. procfs and various > > netgraph nodes spring to mind immediately. > hmmm such as? In procfs: portions of procfs_{map,rlimit,status,vnops}.c In linprocfs: most of linprocfs_misc.c Poul-Henning also mentioned the mn(4) and musycc(4) drivers. Any part of the kernel that exports string sysctls will also benefit. Most of the magic that's necessary for writeable string sysctls can be eliminated by using sbufs in such a way that you'll be able to declare string sysctls just as easily as integer sysctls (currently, each writeable string sysctl requires ~20 lines of code). Empirical tests show that the sbuf API adds less than 2 kB of code to the kernel, and I believe (though I can't prove) that the amount of duplicated code and static buffers that can be replaced with judicious use of sbufs will more than outweigh that cost. In some cases (procfs and linprocfs at least, but probably others too) using sbufs also saves large amounts of syscall stack space. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 8:48:50 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 08:48:48 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 6418D37B402 for ; Tue, 12 Dec 2000 08:48:47 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id eBCGlqe49407; Tue, 12 Dec 2000 17:47:52 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Dag-Erling Smorgrav Cc: Julian Elischer , Matt Dillon , kris@citusc.usc.edu, arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel In-Reply-To: Your message of "12 Dec 2000 17:33:01 +0100." Date: Tue, 12 Dec 2000 17:47:52 +0100 Message-ID: <49405.976639672@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message , Dag-Erling Smorgrav writes: >Julian Elischer writes: >> Poul-Henning Kamp wrote: >> > There are several places where this new API would make the code >> > simpler and less prone to overflowable errors. procfs and various >> > netgraph nodes spring to mind immediately. >> hmmm such as? > >In procfs: portions of procfs_{map,rlimit,status,vnops}.c >In linprocfs: most of linprocfs_misc.c > >Poul-Henning also mentioned the mn(4) and musycc(4) drivers. > >Any part of the kernel that exports string sysctls will also benefit. >Most of the magic that's necessary for writeable string sysctls can be >eliminated by using sbufs in such a way that you'll be able to declare >string sysctls just as easily as integer sysctls (currently, each >writeable string sysctl requires ~20 lines of code). > >Empirical tests show that the sbuf API adds less than 2 kB of code to >the kernel, and I believe (though I can't prove) that the amount of >duplicated code and static buffers that can be replaced with judicious >use of sbufs will more than outweigh that cost. In some cases (procfs >and linprocfs at least, but probably others too) using sbufs also >saves large amounts of syscall stack space. The main thing, however, is providing a trivial string api so that we can get less overflow bugs in the future. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 9: 8:58 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 09:08:55 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 5FC6237B6A4 for ; Tue, 12 Dec 2000 09:08:48 -0800 (PST) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by pike.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id eBCH8ZE88691; Tue, 12 Dec 2000 09:08:36 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Tue, 12 Dec 2000 09:08:40 -0800 (PST) From: John Baldwin To: Seigo Tanimura Subject: Re: Even 1GB KVA is not enough, but we have no more space Cc: arch@FreeBSD.org, bright@wintelcom.net, dillon@earth.backplane.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 12-Dec-00 Seigo Tanimura wrote: > On Sat, 09 Dec 2000 14:24:06 +0900, > Seigo Tanimura said: > > Seigo> And the saga continues. After regulating the size of struct swblock, > Seigo> ffs_vget() failed to allocalte a new vnode. At the time the PowerEdge > Seigo> failed, the kernel held around 197K vnodes, which is as large as > Seigo> 46MB. This time I reduced the size of kmem_map, the pool of malloc(9). > > Sorry, please forget this matter. The culprit truned out to be a race > condition of testing-and-setting ffs_inode_hash_lock in > ffs_vget(). Mutexifing the test-and-sleep-or-set and wakeup-reset > operations was enough to fix the problem. Now my kernel allocates up > to 316K vnodes on make buildworld and release, so malloc(9) pool of > 200MB is sufficient. (316K vnodes occupy about 90-100MB in kmem_map) > > > The updated patch is now at > > URI: http://people.FreeBSD.org/~tanimura/patches/kvmfix.diff > > which includes > > A. adaptation of swap metadata size, > B. rejection of swapon(2) upon failure of swap zone allocation, and > C. atomic test-and-set and reset of {ffs,ifs}_inode_hash_lock. > > I am going to commit this patch in about 6 hours from now. Mutexes look ok to me. One thing that you might do differently is this: mtx_enter(&foo); want_wakeup = foo_lock < 0; foo_lock = 0; mtx_exit(&foo); if (want_wakeup) wakeup(&foo_lock); The reason being that when you do the wakeup(), another CPU might start executing other kernel task immediately. The first thing it will do is block on the foo mutex and get switched out until you release the mutex after wakeup returns. Granted, there is a very small window here, and if it obfuscates the code too badly, it probably isn't worth the gain, however it is something to keep in mind. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 10:20:21 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 10:20:20 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.66]) by hub.freebsd.org (Postfix) with ESMTP id C058837B400 for ; Tue, 12 Dec 2000 10:20:17 -0800 (PST) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.11.0/8.11.0) with ESMTP id eBCIKGs74897; Tue, 12 Dec 2000 11:20:16 -0700 (MST) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id LAA31234; Tue, 12 Dec 2000 11:20:14 -0700 (MST) Message-Id: <200012121820.LAA31234@harmony.village.org> To: Matt Dillon Subject: Re: Safe string formatting in the kernel Cc: kris@citusc.usc.edu, Dag-Erling Smorgrav , arch@FreeBSD.ORG In-reply-to: Your message of "Mon, 11 Dec 2000 18:59:41 PST." <200012120259.eBC2xfb99004@earth.backplane.com> References: <200012120259.eBC2xfb99004@earth.backplane.com> <20001211185610.A1741@citusc.usc.edu> Date: Tue, 12 Dec 2000 11:20:14 -0700 From: Warner Losh Sender: imp@harmony.village.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <200012120259.eBC2xfb99004@earth.backplane.com> Matt Dillon writes: : strcpy() -> sn_strcpy(dst, src, sizeof_destination_buffer) : strcat() -> sn_strcat(dst, src, sizeof_destination_buffer) We already have strlcat and strlcpy in the tree. I'd agree with the weak need for making it all dynamic. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 10:25:10 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 10:25:08 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.66]) by hub.freebsd.org (Postfix) with ESMTP id 17DE337B400; Tue, 12 Dec 2000 10:25:06 -0800 (PST) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.11.0/8.11.0) with ESMTP id eBCIP4s74927; Tue, 12 Dec 2000 11:25:04 -0700 (MST) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id LAA31285; Tue, 12 Dec 2000 11:25:03 -0700 (MST) Message-Id: <200012121825.LAA31285@harmony.village.org> To: Dag-Erling Smorgrav Subject: Re: Safe string formatting in the kernel Cc: assar@FreeBSD.ORG, Matt Dillon , kris@citusc.usc.edu, arch@FreeBSD.ORG In-reply-to: Your message of "12 Dec 2000 15:42:29 +0100." References: <20001211185610.A1741@citusc.usc.edu> <200012120259.eBC2xfb99004@earth.backplane.com> <5lhf4ap8cv.fsf@assaris.sics.se> Date: Tue, 12 Dec 2000 11:25:03 -0700 From: Warner Losh Sender: imp@harmony.village.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message Dag-Erling Smorgrav writes: : Because there are other issues than just overflowing the buffer. : There's the issue of truncation (a lot of code uses snprintf() etc. : without checking if the resulting string was actually truncated, which : may be a security risk of its own), and there's the issue of using : large amounts of stack space for buffers (procfs and linprocfs are : notorious offenders in both these areas, but they're not the only : ones) Both strl* and snprintf have a return value so that if you truncate, you can detect it and do something different in the buffer overflow case. There are many times that you want to have a bounded uppper bound on the length of the string (path names spring to mind). There have been zero problems with truncating long strings in the security lists to date. I've not yet seen anything that looks close to an attack with string truncation causing any sorts of problems. : Vulnerabilities were recently found in the procfs code which were : successfully solved with snprintf(), but could have been (and : hopefully will be) solved in a much more elegant and future-proof : manner using sbufs. Just be careful that your dynamic string growing things don't violate the hard limit invariants in the kernel. If it produces paths longer than 1023 characters, for example, it is wrong. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 12:24:51 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 12:24:50 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id D259137B400 for ; Tue, 12 Dec 2000 12:24:49 -0800 (PST) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by pike.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id eBCKOjE95641 for ; Tue, 12 Dec 2000 12:24:45 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 Date: Tue, 12 Dec 2000 12:24:50 -0800 (PST) From: John Baldwin To: arch@FreeBSD.org Subject: An opaque refcount type Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Here's another bikeshed war for everyone to get in on: I've implemented a relatively light weight and very simple opaque reference counter. It defines an opaque refcount_t type. In the INVARIANTS case, this maps to a structure that contains an integer and a mutex. The mutex is used to protect the integer count as well as several KASSERT()'s. In the non-INVARIANTS case, it maps to a single integer on all of our current platforms (x86, alpha, ia64) and is managed via atomic operations. It defines a rather simple API: void refcount_init(refcount_t *count, u_int value) - Initializes the reference counter and sets its inital count to 'value'. void refcount_destroy(refcount_t *count) - Cleans up any internals used in a refcount, frees resources, etc. void refcount_acquire(refcount_t *count) - Atomically bump the reference count by one. int refcount_release(refcount_t *count) - Atomically decerement the reference count by one and return true if the count drops to zero. The patch to do all this is found at http://www.FreeBSD.org/~jhb/patches/refcount.patch. If this goes in I'll work up a manpage to go along with it as well.. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 12:32: 1 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 12:32:00 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id C8D3537B402; Tue, 12 Dec 2000 12:31:58 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id VAA62590; Tue, 12 Dec 2000 21:31:29 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Warner Losh Cc: assar@FreeBSD.ORG, Matt Dillon , kris@citusc.usc.edu, arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel References: <20001211185610.A1741@citusc.usc.edu> <200012120259.eBC2xfb99004@earth.backplane.com> <5lhf4ap8cv.fsf@assaris.sics.se> <200012121825.LAA31285@harmony.village.org> From: Dag-Erling Smorgrav Date: 12 Dec 2000 21:31:28 +0100 In-Reply-To: Warner Losh's message of "Tue, 12 Dec 2000 11:25:03 -0700" Message-ID: Lines: 11 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Warner Losh writes: > Just be careful that your dynamic string growing things don't violate > the hard limit invariants in the kernel. If it produces paths longer > than 1023 characters, for example, it is wrong. Code manipulating path names would specify a hard upper limit of MAXPATHLEN. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 13: 0:23 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 13:00:21 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from khavrinen.lcs.mit.edu (khavrinen.lcs.mit.edu [18.24.4.193]) by hub.freebsd.org (Postfix) with ESMTP id 5A06337B400; Tue, 12 Dec 2000 13:00:20 -0800 (PST) Received: (from wollman@localhost) by khavrinen.lcs.mit.edu (8.9.3/8.9.3) id QAA53113; Tue, 12 Dec 2000 16:00:03 -0500 (EST) (envelope-from wollman) Date: Tue, 12 Dec 2000 16:00:03 -0500 (EST) From: Garrett Wollman Message-Id: <200012122100.QAA53113@khavrinen.lcs.mit.edu> To: jhb@FreeBSD.ORG Subject: Re: An opaque refcount type X-Newsgroups: mit.lcs.mail.freebsd-arch In-Reply-To: Organization: MIT Laboratory for Computer Science Cc: arch@FreeBSD.ORG Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In article you write: >an opaque refcount_t type. By definition, C typedefs can never be opaque. Only pointers to incomplete aggregates (i.e., `struct foo *' or `union bar *') are opaque in C. In any event, I think requiring a functional interface to reference counts is probably a Bad Idea. -GAWollman -- Garrett A. Wollman | O Siem / We are all family / O Siem / We're all the same wollman@lcs.mit.edu | O Siem / The fires of freedom Opinions not those of| Dance in the burning flame MIT, LCS, CRS, or NSA| - Susan Aglukark and Chad Irschick To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 13: 4: 4 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 13:04:02 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from ns.yogotech.com (ns.yogotech.com [206.127.123.66]) by hub.freebsd.org (Postfix) with ESMTP id CC11137B6A0; Tue, 12 Dec 2000 13:03:58 -0800 (PST) Received: from nomad.yogotech.com (nomad.yogotech.com [206.127.123.131]) by ns.yogotech.com (8.9.3/8.9.3) with ESMTP id OAA20177; Tue, 12 Dec 2000 14:03:53 -0700 (MST) (envelope-from nate@nomad.yogotech.com) Received: (from nate@localhost) by nomad.yogotech.com (8.8.8/8.8.8) id OAA27194; Tue, 12 Dec 2000 14:03:52 -0700 (MST) (envelope-from nate) From: Nate Williams MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <14902.37560.212440.830262@nomad.yogotech.com> Date: Tue, 12 Dec 2000 14:03:52 -0700 (MST) To: Garrett Wollman Cc: jhb@FreeBSD.ORG, arch@FreeBSD.ORG Subject: Re: An opaque refcount type In-Reply-To: <200012122100.QAA53113@khavrinen.lcs.mit.edu> References: <200012122100.QAA53113@khavrinen.lcs.mit.edu> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Reply-To: nate@yogotech.com (Nate Williams) Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > >an opaque refcount_t type. > > By definition, C typedefs can never be opaque. Only pointers to > incomplete aggregates (i.e., `struct foo *' or `union bar *') are > opaque in C. > > In any event, I think requiring a functional interface to reference > counts is probably a Bad Idea. I disagree. This is a very object oriented thing to do, and it allow you to hide the vulgarities of doing things 'safely' behind the API. On a Sparc a refcount might be 24 bits wide, on the 386/486 it can be safely 32 bits, ia64 64bits, etc... Nate To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 13: 7:55 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 13:07:50 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 630CA37B402 for ; Tue, 12 Dec 2000 13:07:47 -0800 (PST) Received: from foo.osd.bsdi.com (root@foo.osd.bsdi.com [204.216.28.137]) by pike.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id eBCL7gE97156; Tue, 12 Dec 2000 13:07:42 -0800 (PST) (envelope-from jhb@foo.osd.bsdi.com) Received: (from jhb@localhost) by foo.osd.bsdi.com (8.11.1/8.11.0) id eBCL6v320085; Tue, 12 Dec 2000 13:06:57 -0800 (PST) (envelope-from jhb) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200012122100.QAA53113@khavrinen.lcs.mit.edu> Date: Tue, 12 Dec 2000 13:06:56 -0800 (PST) Organization: BSD, Inc. From: John Baldwin To: Garrett Wollman Subject: Re: An opaque refcount type Cc: arch@FreeBSD.ORG Sender: jhb@foo.osd.bsdi.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 12-Dec-00 Garrett Wollman wrote: > In article > you write: >>an opaque refcount_t type. > > By definition, C typedefs can never be opaque. Only pointers to > incomplete aggregates (i.e., `struct foo *' or `union bar *') are > opaque in C. It's opaque in the sense that a user doesn't know what it is inside it. This means we can freely change around the implementation. For example, in the INVARIANTS case it adds in lots of extra checks, but to ensure correctness, it has to add in a mutex to use. > In any event, I think requiring a functional interface to reference > counts is probably a Bad Idea. Would you prefer C macros to static inline's then? > -GAWollman -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.Baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 13: 9:48 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 13:09:46 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from prism.flugsvamp.com (cb58709-a.mdsn1.wi.home.com [24.17.241.9]) by hub.freebsd.org (Postfix) with ESMTP id 4C9DD37B400; Tue, 12 Dec 2000 13:09:46 -0800 (PST) Received: (from jlemon@localhost) by prism.flugsvamp.com (8.11.0/8.11.0) id eBCL6UA78335; Tue, 12 Dec 2000 15:06:30 -0600 (CST) (envelope-from jlemon) Date: Tue, 12 Dec 2000 15:06:29 -0600 From: Jonathan Lemon To: John Baldwin Cc: arch@FreeBSD.ORG Subject: Re: An opaque refcount type Message-ID: <20001212150629.B37608@prism.flugsvamp.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0pre2i In-Reply-To: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, Dec 12, 2000 at 12:24:50PM -0800, John Baldwin wrote: > Here's another bikeshed war for everyone to get in on: I've implemented a > relatively light weight and very simple opaque reference counter. It defines > an opaque refcount_t type. In the INVARIANTS case, this maps to a structure I find myself wanting something more like a gated refcount; if the gate is "open", then bump the reference (all atomically), otherwise fail. However, it isn't quite clear what to do on a release, and it probably is not going to fit into this model. I can say at this point that I'm going to need something more complicated than what is proposed for reference counts on route entries and code that can be kld{load/unload}ed. -- Jonathan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 13:18:19 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 13:18:16 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id AC7B137B402; Tue, 12 Dec 2000 13:18:14 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id WAA62830; Tue, 12 Dec 2000 22:18:13 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: John Baldwin Cc: Garrett Wollman , arch@FreeBSD.ORG Subject: Re: An opaque refcount type References: From: Dag-Erling Smorgrav Date: 12 Dec 2000 22:18:12 +0100 In-Reply-To: John Baldwin's message of "Tue, 12 Dec 2000 13:06:56 -0800 (PST)" Message-ID: Lines: 13 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG John Baldwin writes: > On 12-Dec-00 Garrett Wollman wrote: > > In any event, I think requiring a functional interface to reference > > counts is probably a Bad Idea. > Would you prefer C macros to static inline's then? I'd say compromise - use functions in the INVARIANTS case, macros otherwise (I assume that the code in the non-INVARIANTS case is very short and simple) DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 13:40:11 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 13:40:09 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id CBEC537B402 for ; Tue, 12 Dec 2000 13:40:08 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id WAA62945; Tue, 12 Dec 2000 22:40:07 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: arch@freebsd.org Subject: New sbuf patch on freefall From: Dag-Erling Smorgrav Date: 12 Dec 2000 22:40:07 +0100 Message-ID: Lines: 9 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG http://people.freebsd.org/~des/software/sbuf-20001212.diff I've cleaned up the code, fixed a bug or two, removed some unnecessary code, added some more assertions, and incorporated most of the suggestions I've gotten so far, as well as cleaned up the man page. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 13:57:11 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 13:57:09 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id EEB9D37B400; Tue, 12 Dec 2000 13:57:08 -0800 (PST) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.1/8.11.1) with SMTP id eBCLv7e65525; Tue, 12 Dec 2000 16:57:08 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Tue, 12 Dec 2000 16:57:07 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: John Baldwin Cc: arch@FreeBSD.org Subject: Re: Can !curproc touch In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: robert@fledge.watson.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG My understanding is that a number of systems perform a crcopy() while holding the struct proc mutex, and get an additional ucred reference which is then passed down the VFS stack, guarantying that the credentials are used consistently. This addresses a number of problems, including the multi-threaded case where you want a system call to be processed entirely under one set of credentials, serializing requests from the perspective of the credential. The general rule for reference counts should be that the holder of the reference count is responsible for protecting it: if the owner of the reference is struct proc, then the user of that reference must protect struct proc while using the reference. If the use of the reference is relatively long-term, then an additional reference should be created and used instead (i.e., crref()), allowing the user to release the protection (mutex, lock, invariants, whatever) on the struct proc. This was the type of reference count race condition I was discussing, btw, at BSDCon while we were futzing around at the whiteboard discussing drivers and related stuff. Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 14: 0:52 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 14:00:49 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from khavrinen.lcs.mit.edu (khavrinen.lcs.mit.edu [18.24.4.193]) by hub.freebsd.org (Postfix) with ESMTP id 2325F37B400; Tue, 12 Dec 2000 14:00:46 -0800 (PST) Received: (from wollman@localhost) by khavrinen.lcs.mit.edu (8.9.3/8.9.3) id RAA53571; Tue, 12 Dec 2000 17:00:43 -0500 (EST) (envelope-from wollman) Date: Tue, 12 Dec 2000 17:00:43 -0500 (EST) From: Garrett Wollman Message-Id: <200012122200.RAA53571@khavrinen.lcs.mit.edu> To: John Baldwin Cc: arch@FreeBSD.ORG Subject: Re: An opaque refcount type In-Reply-To: References: <200012122100.QAA53113@khavrinen.lcs.mit.edu> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG < said: > It's opaque in the sense that a user doesn't know what it is inside > it. In order to include it in a data structure, the user must know what is inside it. That is the crux of the problem. We have already learned to our dismay how tangled messes of include files and other definitions can cause problems, and more importantly that having potentially-exported data structures depend on kernel compilation options. Here is an alternative formulation: #define DECLARE_REFCOUNT(prefix) \ int prefix ## _refcnt; \ struct refcount_bk * prefix ## _refcnt_bk; /* * Actually, don't use the macro above, but rather, declare your structure * as if it were expanded inline, so that your header file does not * depend on this one as a prerequisite. */ #ifdef INVARIANT_SUPPORT #define REFHOLD(count, bookkeeping) \ ((bookkeeping != 0) \ ? refcount_track(count, bookkeeping, 1) \ : atomic_increment(count)) \ #define REFDROP(count, bookkeeping) \ ((bookkeeping != 0) \ ? refcount_track(count, bookkeeping, -1); \ : atomic_decrement(count)) \ #else #define REFHOLD(count, bookkeeping) \ atomic_increment(count) #define REFDROP(count, bookkeeping) \ atomic_decrement(count) #endif #ifdef INVARIANTS struct refcount_bk *refcount_bk_init(int *countp); void refcount_bk_destroy(struct refcount_bk *bookkeeping); #else #define refcount_bk_init(countp) (struct refcount_bk *)0 #define refcount_bk_destroy(bk) /* nothing */ #endif /******* example of usage *******/ /* gronk.h */ struct gronk { /* insert real data for gronk structure */ int gronk_refcnt; struct refcount_bk *gronk_bk; }; /* gronk.c */ #include "gronk.h" #include "refcnt.h" void gronk_init(struct gronk *gronk) { /* insert real initialization */ gronk->gronk_refcnt = 0; /* or however many you want */ gronk->gronk_bk = refcount_bk_init(&gronk->gronk_refcnt); } void gronk_ref(struct gronk *gronk) { REFHOLD(gronk->gronk_refcnt, gronk->gronk_bk); } void gronk_rele(struct gronk *gronk) { if (REFDROP(gronk->gronk_refcnt, gronk->gronk_bk) == 1) gronk_free(gronk); } /************************************/ Obviously, this rouine requires the ability to allocate memory when creating the `refcount_bk' structures. This is not a serious problem. It also gives clients more flexibility with respect to determining which reference counts are checked, which may be useful for debugging. -GAWollman -- Garrett A. Wollman | O Siem / We are all family / O Siem / We're all the same wollman@lcs.mit.edu | O Siem / The fires of freedom Opinions not those of| Dance in the burning flame MIT, LCS, CRS, or NSA| - Susan Aglukark and Chad Irschick To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 14:28:34 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 14:28:32 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.66]) by hub.freebsd.org (Postfix) with ESMTP id 2826A37B400; Tue, 12 Dec 2000 14:28:30 -0800 (PST) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.11.0/8.11.0) with ESMTP id eBCMSSs75916; Tue, 12 Dec 2000 15:28:28 -0700 (MST) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id PAA33203; Tue, 12 Dec 2000 15:28:27 -0700 (MST) Message-Id: <200012122228.PAA33203@harmony.village.org> To: Dag-Erling Smorgrav Subject: Re: Safe string formatting in the kernel Cc: assar@FreeBSD.ORG, Matt Dillon , kris@citusc.usc.edu, arch@FreeBSD.ORG In-reply-to: Your message of "12 Dec 2000 21:31:28 +0100." References: <20001211185610.A1741@citusc.usc.edu> <200012120259.eBC2xfb99004@earth.backplane.com> <5lhf4ap8cv.fsf@assaris.sics.se> <200012121825.LAA31285@harmony.village.org> Date: Tue, 12 Dec 2000 15:28:27 -0700 From: Warner Losh Sender: imp@harmony.village.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message Dag-Erling Smorgrav writes: : Warner Losh writes: : > Just be careful that your dynamic string growing things don't violate : > the hard limit invariants in the kernel. If it produces paths longer : > than 1023 characters, for example, it is wrong. : : Code manipulating path names would specify a hard upper limit of : MAXPATHLEN. If there's a known, realtively small, upper limit, why does allocating it dynamically buy you when you could have a static buffer? I know that it costs you a trip into the kernel malloc routines which can be quite high at times. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 14:30:35 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 14:30:33 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mail.interware.hu (mail.interware.hu [195.70.32.130]) by hub.freebsd.org (Postfix) with ESMTP id 3139337B400 for ; Tue, 12 Dec 2000 14:30:32 -0800 (PST) Received: from bissau-13.budapest.interware.hu ([195.70.53.141] helo=elischer.org) by mail.interware.hu with esmtp (Exim 3.16 #1 (Debian)) id 145xwM-0006jE-00 for ; Tue, 12 Dec 2000 23:30:30 +0100 Sender: julian@FreeBSD.ORG Message-ID: <3A36A6E3.6507A6C0@elischer.org> Date: Tue, 12 Dec 2000 14:29:55 -0800 From: Julian Elischer X-Mailer: Mozilla 4.7 [en] (X11; U; FreeBSD 5.0-CURRENT i386) X-Accept-Language: en, hu MIME-Version: 1.0 To: arch@freebsd.org Subject: [Fwd: Re: MEXT_IS_REF broken.] Content-Type: text/plain; charset=iso-8859-15 Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG hmm having thought a bit more about this: > > http://www.FreeBSD.org/~jhb/patches/refcount.patch --------- I said: this is great but if I understand it, there is a hole... the operations are atomic, but if teh count is 1 and the release and aquire are called at around the same time, then thre are two possibilities.. the value goes 1 -> 2 -> 1 (this is ok) the value goes 1 -> 0 -> 1 ( this is NOT ok) the aquire calls atomic_add_acq_int(count, 1); which by my reading of the code last week, compiles down to lock ; addl (mumble) if the value that the aquire finds is 0 then it should fail and return without incrementing.. the returned value of 0 for the other processor will cause it to garbage collect the object in a very sort amount of time so trying to use it is a bad idea. Am I missing something here? Hopefully there si some higher level lock or something that stops you from trying to reference an object that is having its last reference removed (otherwise how did you find it?) but never-the less, it's an issue to look at. --------- end quote: thinking about this.. you should never be adding a reference to something that you don't already have a reference to, because you must have followed a reference to find the object in the first place. Thus if you are trying to add a reference and takeone away at the same time, you must have followed the reference that is being removed, which suggests a locking problem somewhere. In other words, detection of a 0 in the count at the beginning of a reference aquisition should be cause for a panic, or at least a heaavy diagnostic of some kind, and for certain it shound fail. -- __--_|\ Julian Elischer / \ julian@elischer.org ( OZ ) World tour 2000 ---> X_.---._/ presently in: Budapest v To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 14:43:36 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 14:43:35 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 1AAA637B400; Tue, 12 Dec 2000 14:43:34 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id eBCMhNe55083; Tue, 12 Dec 2000 23:43:23 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Warner Losh Cc: Dag-Erling Smorgrav , assar@FreeBSD.ORG, Matt Dillon , kris@citusc.usc.edu, arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel In-Reply-To: Your message of "Tue, 12 Dec 2000 15:28:27 MST." <200012122228.PAA33203@harmony.village.org> Date: Tue, 12 Dec 2000 23:43:22 +0100 Message-ID: <55081.976661002@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <200012122228.PAA33203@harmony.village.org>, Warner Losh writes: >In message Dag-Erling Smorgrav writes: >: Warner Losh writes: >: > Just be careful that your dynamic string growing things don't violate >: > the hard limit invariants in the kernel. If it produces paths longer >: > than 1023 characters, for example, it is wrong. >: >: Code manipulating path names would specify a hard upper limit of >: MAXPATHLEN. > >If there's a known, realtively small, upper limit, why does allocating >it dynamically buy you when you could have a static buffer? I know >that it costs you a trip into the kernel malloc routines which can be >quite high at times. I have not reread DES's implementation, but in my design doc you could initialize an sbuf with your own buffer, for exactly that reason. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 15:10:12 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 15:10:10 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 8252B37B400 for ; Tue, 12 Dec 2000 15:10:10 -0800 (PST) Received: from foo.osd.bsdi.com (root@foo.osd.bsdi.com [204.216.28.137]) by pike.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id eBCN9WE01494; Tue, 12 Dec 2000 15:09:32 -0800 (PST) (envelope-from jhb@foo.osd.bsdi.com) Received: (from jhb@localhost) by foo.osd.bsdi.com (8.11.1/8.11.0) id eBCN8kK20896; Tue, 12 Dec 2000 15:08:46 -0800 (PST) (envelope-from jhb) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Tue, 12 Dec 2000 15:08:46 -0800 (PST) Organization: BSD, Inc. From: John Baldwin To: Dag-Erling Smorgrav Subject: Re: An opaque refcount type Cc: arch@FreeBSD.ORG, Garrett Wollman Sender: jhb@foo.osd.bsdi.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 12-Dec-00 Dag-Erling Smorgrav wrote: > John Baldwin writes: >> On 12-Dec-00 Garrett Wollman wrote: >> > In any event, I think requiring a functional interface to reference >> > counts is probably a Bad Idea. >> Would you prefer C macros to static inline's then? > > I'd say compromise - use functions in the INVARIANTS case, macros > otherwise (I assume that the code in the non-INVARIANTS case is very > short and simple) Currently they are all static inlines. > DES > -- > Dag-Erling Smorgrav - des@ofug.org -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.Baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 16:22:52 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 16:22:49 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 338BD37B400; Tue, 12 Dec 2000 16:22:48 -0800 (PST) Received: from foo.osd.bsdi.com (root@foo.osd.bsdi.com [204.216.28.137]) by pike.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id eBD0MhE03982; Tue, 12 Dec 2000 16:22:43 -0800 (PST) (envelope-from jhb@foo.osd.bsdi.com) Received: (from jhb@localhost) by foo.osd.bsdi.com (8.11.1/8.11.0) id eBD0Lvl21462; Tue, 12 Dec 2000 16:21:57 -0800 (PST) (envelope-from jhb) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Tue, 12 Dec 2000 16:21:57 -0800 (PST) Organization: BSD, Inc. From: John Baldwin To: Robert Watson Subject: Re: Can !curproc touch Cc: arch@FreeBSD.ORG Sender: jhb@foo.osd.bsdi.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 12-Dec-00 Robert Watson wrote: > > My understanding is that a number of systems perform a crcopy() while > holding the struct proc mutex, and get an additional ucred reference which > is then passed down the VFS stack, guarantying that the credentials are > used consistently. This addresses a number of problems, including the > multi-threaded case where you want a system call to be processed entirely > under one set of credentials, serializing requests from the perspective of > the credential. The general rule for reference counts should be that the > holder of the reference count is responsible for protecting it: if the > owner of the reference is struct proc, then the user of that reference > must protect struct proc while using the reference. If the use of the > reference is relatively long-term, then an additional reference should be > created and used instead (i.e., crref()), allowing the user to release the > protection (mutex, lock, invariants, whatever) on the struct proc. This > was the type of reference count race condition I was discussing, btw, at > BSDCon while we were futzing around at the whiteboard discussing drivers > and related stuff. Well, currently nothing uses the struct proc mutex when touching p_ucred. This is some code from coda: /* * find root of cfs */ int coda_root(vfsp, vpp) struct mount *vfsp; struct vnode **vpp; { struct coda_mntinfo *mi = vftomi(vfsp); struct vnode **result; int error; struct proc *p = curproc; /* XXX - bnoble */ ... error = venus_root(vftomi(vfsp), p->p_ucred, p, &VFid); To fix this I could do this: struct ucred *uc; ... PROC_LOCK(p); uc = p->p_ucred; crhold(uc); PROC_UNLOCK(p); error = venus_root(...., uc, ...); crfree(uc); In place of the last line. However, note that p is always curproc. If all the other places that accees p->p_ucred only do so if p == curproc, then I don't need to use an explicit lock to protect p_ucred, as it will be implicitly locked. Thus, my question is is there a place where a process A can read or write the p_ucred of process B. If there is, then I need to use the proc mutex to protect p_ucred. If not, I can leave p_ucred alone. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.Baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 16:49: 0 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 16:48:57 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id E421A37B400; Tue, 12 Dec 2000 16:48:56 -0800 (PST) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.1/8.11.1) with SMTP id eBD0mse67674; Tue, 12 Dec 2000 19:48:55 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Tue, 12 Dec 2000 19:48:54 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: John Baldwin Cc: arch@FreeBSD.ORG Subject: Re: Can !curproc touch In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: robert@fledge.watson.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, 12 Dec 2000, John Baldwin wrote: > > My understanding is that a number of systems perform a crcopy() while > > holding the struct proc mutex, and get an additional ucred reference which > > is then passed down the VFS stack, guarantying that the credentials are > > used consistently. This addresses a number of problems, including the > > multi-threaded case where you want a system call to be processed entirely > > under one set of credentials, serializing requests from the perspective of > > the credential. The general rule for reference counts should be that the > > holder of the reference count is responsible for protecting it: if the > > owner of the reference is struct proc, then the user of that reference > > must protect struct proc while using the reference. If the use of the > > reference is relatively long-term, then an additional reference should be > > created and used instead (i.e., crref()), allowing the user to release the > > protection (mutex, lock, invariants, whatever) on the struct proc. This > > was the type of reference count race condition I was discussing, btw, at > > BSDCon while we were futzing around at the whiteboard discussing drivers > > and related stuff. > > Well, currently nothing uses the struct proc mutex when touching p_ucred. This > is some code from coda: ... > In place of the last line. However, note that p is always curproc. If > all the other places that accees p->p_ucred only do so if p == curproc, > then I don't need to use an explicit lock to protect p_ucred, as it will > be implicitly locked. Thus, my question is is there a place where a > process A can read or write the p_ucred of process B. If there is, then > I need to use the proc mutex to protect p_ucred. If not, I can leave > p_ucred alone. Whenever a process acts on another process and inter-process authorization is required, p_target->p_ucred will be dereferenced. p_ucred is also dereferenced for process information services, such as sysctl() as used by ps and other process-grubbing programs. See p_can*(), sysctl_out_proc(), procfs, et al. I'm not sure there's ever a write case, but there are many read cases by this definition. Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 18:34:31 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 18:34:28 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 901E337B402 for ; Tue, 12 Dec 2000 18:34:27 -0800 (PST) Received: (qmail 38451 invoked by uid 1142); 13 Dec 2000 02:34:26 -0000 Date: 12 Dec 2000 18:34:26 -0800 Date: Tue, 12 Dec 2000 18:34:14 -0800 From: Jason Evans To: John Baldwin Cc: arch@FreeBSD.org Subject: Re: An opaque refcount type Message-ID: <20001212183414.K2312@canonware.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from jhb@FreeBSD.org on Tue, Dec 12, 2000 at 12:24:50PM -0800 Sender: jasone@magnesium.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, Dec 12, 2000 at 12:24:50PM -0800, John Baldwin wrote: > Here's another bikeshed war for everyone to get in on: I've implemented a > relatively light weight and very simple opaque reference counter. It defines > an opaque refcount_t type. In the INVARIANTS case, this maps to a structure > that contains an integer and a mutex. The mutex is used to protect the > integer count as well as several KASSERT()'s. In the non-INVARIANTS case, it > maps to a single integer on all of our current platforms (x86, alpha, ia64) and > is managed via atomic operations. It defines a rather simple API: > > void refcount_init(refcount_t *count, u_int value) > - Initializes the reference counter and sets its inital count to 'value'. > void refcount_destroy(refcount_t *count) > - Cleans up any internals used in a refcount, frees resources, etc. > void refcount_acquire(refcount_t *count) > - Atomically bump the reference count by one. > int refcount_release(refcount_t *count) > - Atomically decerement the reference count by one and return true if the > count drops to zero. As at least John and Alfred know, I'm opposed to an atomic refcount type, for at least the following reasons: 1) The number of bits of accuracy varies from platform to platform. It seems that the least common denominator we'd need to settle on is 24 bits (sparc64), which is enough for most, but not all uses. This is the kernel we're talking about here; I don't like the uncertainty of the accuracy. Maybe 16.7 million is enough accuracy, but... 2) It is hard to say at this point since we still have a lot of conversion work to do, but I suspect that the number of places where a refcount is ideally manipulated outside of a mutex is small. 3) The refcount_init() and refcount_destroy() calls are mandatory, but are only really necessary for platforms that have to rely on mutexes for the internal implementation (or if INVARIANTS is defined). For the most part, this just amounts to extra API complexity. 4) For platforms that can't support an atomic refcount with atomic operations, we have to use mutexes. Now, instead of having using normal atomic operations for refcounts, we have a mutex structure in place of an integer. So, in some cases, this API is actually a pessimization rather than an optimization, and any structures that use a refcount_t are suddenly much larger. Naturally, this means that any structures with a refcount_t that are exposed to userland will potentially vary in size, depending on INVARIANTS. 5) At the moment we have mtx_*(), atomic_*(), [mt]sleep(), and the lockmgr. In the foreseeable future (maybe not by 5.0, but eventually) we will probably have mtx_*(), cv_*() (condition variables, replaces [mt]sleep()), and enhanced reader/writer (actually SIX) locks (replaces lockmgr). I'd like to see the number of synchronization-related interfaces get smaller and simpler, not larger. In another email thread, I demonstrated how to safely manipulate refcounts with atomic operations. The main problem to solve is how to avoid a cleanup race when decrementing the refcount to 0. The following pseudo-code demonstrates a way to avoid this race. atomic_decrement(count) if (compare_and_set(count, 0, 1)) { /* iff (count == 0) {count = 1}. */ /* Clean up. */ } This requires two atomic operations for refcount decrement, which, as John has pointed out, is the same number of locked instructions as would be required for a mutex acquire/release. However, refcount increment still only requires one locked instruction. Therefore, this method requires 50% more locked instructions than a refcount_t implementation would require on a cooperative platform, and 33% fewer locked instructions than a refcount_t implementation on an uncooperative platform. My knowledge of memory bus locking isn't fresh, but I suspect that using two locked instructions on the same address in short succession only costs significantly more than one locked instruction if there is contention. Of course, if there is contention, then an implementation that only uses one locked instruction is going to get hurt too, so I suspect that the actual performance difference between this and a streamlined refcount_t implementation is rather small. (Someone please correct me if this is incorrect.) Even if performance is significantly worse for this method, point 2 above could make it irrelevant anyway. In summary, I have a number of specific and general issues with introducing an atomic refcount type with associated functions. Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 18:35:52 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 18:35:49 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mailhub.teliauk.com (mailhub.teliauk.com [195.12.225.36]) by hub.freebsd.org (Postfix) with ESMTP id 9785537B400; Tue, 12 Dec 2000 18:35:44 -0800 (PST) Received: from d1o314.teliauk.com (root@d1o314.teliauk.com [195.12.237.81]) by mailhub.teliauk.com (8.10.1/8.10.1) with ESMTP id eBD2ZVA09950; Wed, 13 Dec 2000 02:35:43 GMT Received: from vilnya.demon.co.uk (fakeuser@t1o314p236.teliauk.com [195.12.238.236]) by d1o314.teliauk.com (8.8.8/8.8.8) with ESMTP id CAA07012; Wed, 13 Dec 2000 02:35:12 GMT Received: from haveblue (haveblue.rings [10.2.4.5]) by vilnya.demon.co.uk (Postfix) with SMTP id 64692D9D2; Wed, 13 Dec 2000 02:35:04 +0000 (GMT) Message-ID: <015a01c064ad$96667230$0504020a@haveblue> From: "Cameron Grant" To: , Subject: newpcm/kobj Date: Wed, 13 Dec 2000 02:37:07 -0000 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG in the near future, i intend to commit my kobjified newpcm. this gives us several benefits, including: * easier extensibility- new optional methods can be added to ac97/mixer/channel classes without having to fixup every driver. * forward compatibility for drivers, provided no new mandatory methods are added. however, all drivers not in the tree at this time will need to be updated. i hope to mfc to -stable in approximately one month, along with the kobj system. newbus in -stable will not be kobjified. -cg To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 19: 4: 2 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 19:03:58 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from falla.videotron.net (falla.videotron.net [205.151.222.106]) by hub.freebsd.org (Postfix) with ESMTP id 3451F37B400; Tue, 12 Dec 2000 19:03:58 -0800 (PST) Received: from modemcable213.3-201-24.mtl.mc.videotron.ca ([24.201.3.213]) by falla.videotron.net (Sun Internet Mail Server sims.3.5.1999.12.14.10.29.p8) with ESMTP id <0G5H00AHHKIEV4@falla.videotron.net>; Tue, 12 Dec 2000 22:03:51 -0500 (EST) Date: Tue, 12 Dec 2000 22:05:00 -0500 (EST) From: Bosko Milekic Subject: Re: An opaque refcount type In-reply-to: <20001212183414.K2312@canonware.com> To: Jason Evans Cc: John Baldwin , arch@FreeBSD.ORG Message-id: MIME-version: 1.0 Content-type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Okay, now that I've gotten somewhat in touch with how the implementation is suggested, I have to bring up the following point, especially since I feel it highlites an important point that Jason brings up below: Having a mutex accompany a ref. counter is really not a good idea. In the problematic case outlined earlier regarding the mbuf ref. scheme, you'll note that there is a reference counter accompanying EVERY SINGLE ext buf. If you now throw in an extra mutex in there, you'll looking at AT LEAST (NMBCLUSTERS * sizeof(mutex)) added wastage. And that's a lower bound because it only covers clusters. Not to mention that all these counters are pre-allocated, pre-wired. On 12 Dec 2000, Jason Evans wrote: > On Tue, Dec 12, 2000 at 12:24:50PM -0800, John Baldwin wrote: > > Here's another bikeshed war for everyone to get in on: I've implemented a > > relatively light weight and very simple opaque reference counter. It defines > > an opaque refcount_t type. In the INVARIANTS case, this maps to a structure > > that contains an integer and a mutex. The mutex is used to protect the > > integer count as well as several KASSERT()'s. In the non-INVARIANTS case, it > > maps to a single integer on all of our current platforms (x86, alpha, ia64) and > > is managed via atomic operations. It defines a rather simple API: > > > > void refcount_init(refcount_t *count, u_int value) > > - Initializes the reference counter and sets its inital count to 'value'. > > void refcount_destroy(refcount_t *count) > > - Cleans up any internals used in a refcount, frees resources, etc. > > void refcount_acquire(refcount_t *count) > > - Atomically bump the reference count by one. > > int refcount_release(refcount_t *count) > > - Atomically decerement the reference count by one and return true if the > > count drops to zero. > > As at least John and Alfred know, I'm opposed to an atomic refcount type, > for at least the following reasons: > > 1) The number of bits of accuracy varies from platform to platform. It > seems that the least common denominator we'd need to settle on is 24 > bits (sparc64), which is enough for most, but not all uses. This is the > kernel we're talking about here; I don't like the uncertainty of the > accuracy. Maybe 16.7 million is enough accuracy, but... > > 2) It is hard to say at this point since we still have a lot of conversion > work to do, but I suspect that the number of places where a refcount is > ideally manipulated outside of a mutex is small. > > 3) The refcount_init() and refcount_destroy() calls are mandatory, but are > only really necessary for platforms that have to rely on mutexes for the > internal implementation (or if INVARIANTS is defined). For the most > part, this just amounts to extra API complexity. > > 4) For platforms that can't support an atomic refcount with atomic > operations, we have to use mutexes. Now, instead of having using normal > atomic operations for refcounts, we have a mutex structure in place of > an integer. So, in some cases, this API is actually a pessimization > rather than an optimization, and any structures that use a refcount_t > are suddenly much larger. Naturally, this means that any structures > with a refcount_t that are exposed to userland will potentially vary in > size, depending on INVARIANTS. > > 5) At the moment we have mtx_*(), atomic_*(), [mt]sleep(), and the lockmgr. > In the foreseeable future (maybe not by 5.0, but eventually) we will > probably have mtx_*(), cv_*() (condition variables, replaces [mt]sleep()), > and enhanced reader/writer (actually SIX) locks (replaces lockmgr). > I'd like to see the number of synchronization-related interfaces get > smaller and simpler, not larger. > > In another email thread, I demonstrated how to safely manipulate refcounts > with atomic operations. The main problem to solve is how to avoid a > cleanup race when decrementing the refcount to 0. The following > pseudo-code demonstrates a way to avoid this race. > > atomic_decrement(count) > if (compare_and_set(count, 0, 1)) { /* iff (count == 0) {count = 1}. */ > /* Clean up. */ > } > > This requires two atomic operations for refcount decrement, which, as John > has pointed out, is the same number of locked instructions as would be > required for a mutex acquire/release. However, refcount increment still > only requires one locked instruction. Therefore, this method requires 50% > more locked instructions than a refcount_t implementation would require on > a cooperative platform, and 33% fewer locked instructions than a refcount_t > implementation on an uncooperative platform. > > My knowledge of memory bus locking isn't fresh, but I suspect that using > two locked instructions on the same address in short succession only costs > significantly more than one locked instruction if there is contention. Of > course, if there is contention, then an implementation that only uses one > locked instruction is going to get hurt too, so I suspect that the actual > performance difference between this and a streamlined refcount_t > implementation is rather small. (Someone please correct me if this is > incorrect.) > > Even if performance is significantly worse for this method, point 2 above > could make it irrelevant anyway. In summary, I have a number of specific > and general issues with introducing an atomic refcount type with associated > functions. > > Jason > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > > Bosko Milekic bmilekic@technokratis.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 19: 9:52 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 19:09:50 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mass.osd.bsdi.com (adsl-63-202-176-64.dsl.snfc21.pacbell.net [63.202.176.64]) by hub.freebsd.org (Postfix) with ESMTP id 1C7D037B400; Tue, 12 Dec 2000 19:09:50 -0800 (PST) Received: from mass.osd.bsdi.com (localhost [127.0.0.1]) by mass.osd.bsdi.com (8.11.1/8.11.1) with ESMTP id eBD3JH306370; Tue, 12 Dec 2000 19:19:27 -0800 (PST) (envelope-from msmith@mass.osd.bsdi.com) Message-Id: <200012130319.eBD3JH306370@mass.osd.bsdi.com> X-Mailer: exmh version 2.1.1 10/15/1999 To: Bosko Milekic Cc: Jason Evans , John Baldwin , arch@FreeBSD.ORG Subject: Re: An opaque refcount type In-reply-to: Your message of "Tue, 12 Dec 2000 22:05:00 EST." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 12 Dec 2000 19:19:17 -0800 From: Mike Smith Sender: msmith@mass.osd.bsdi.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > Okay, now that I've gotten somewhat in touch with how the implementation > is suggested, I have to bring up the following point, especially since > I feel it highlites an important point that Jason brings up below: > > Having a mutex accompany a ref. counter is really not a good idea. In > the problematic case outlined earlier regarding the mbuf ref. scheme, > you'll note that there is a reference counter accompanying EVERY SINGLE > ext buf. If you now throw in an extra mutex in there, you'll looking at > AT LEAST (NMBCLUSTERS * sizeof(mutex)) added wastage. And that's a lower > bound because it only covers clusters. Not to mention that all these > counters are pre-allocated, pre-wired. The BSD/OS folks use a neat technique where you have a small group of mutexes against which you hash a unique identifier for your object; eg. the address of the refcounter. This avoids the 1:1 noise, but still gives you a good chance of not blocking everything just because you're hitting a refcounter somewhere. -- ... every activity meets with opposition, everyone who acts has his rivals and unfortunately opponents also. But not because people want to be opponents, rather because the tasks and relationships force people to take different points of view. [Dr. Fritz Todt] V I C T O R Y N O T V E N G E A N C E To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 19:16:18 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 19:16:15 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from io.yi.org (unknown [24.70.218.157]) by hub.freebsd.org (Postfix) with ESMTP id 318B437B400; Tue, 12 Dec 2000 19:16:15 -0800 (PST) Received: from io.yi.org (localhost.gvcl1.bc.wave.home.com [127.0.0.1]) by io.yi.org (Postfix) with ESMTP id 280D9BA7D; Tue, 12 Dec 2000 19:14:56 -0800 (PST) X-Mailer: exmh version 2.1.1 10/15/1999 To: Jason Evans Cc: John Baldwin , arch@FreeBSD.ORG Subject: Re: An opaque refcount type In-Reply-To: Message from Jason Evans of "Tue, 12 Dec 2000 18:34:14 PST." <20001212183414.K2312@canonware.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 12 Dec 2000 19:14:56 -0800 From: Jake Burkholder Message-Id: <20001213031456.280D9BA7D@io.yi.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > On Tue, Dec 12, 2000 at 12:24:50PM -0800, John Baldwin wrote: > > Here's another bikeshed war for everyone to get in on: I've implemented a > > relatively light weight and very simple opaque reference counter. It defines > > an opaque refcount_t type. In the INVARIANTS case, this maps to a structure > > that contains an integer and a mutex. The mutex is used to protect the > > integer count as well as several KASSERT()'s. In the non-INVARIANTS case, it > > maps to a single integer on all of our current platforms (x86, alpha, ia64) and > > is managed via atomic operations. It defines a rather simple API: > > > > void refcount_init(refcount_t *count, u_int value) > > - Initializes the reference counter and sets its inital count to 'value'. > > void refcount_destroy(refcount_t *count) > > - Cleans up any internals used in a refcount, frees resources, etc. > > void refcount_acquire(refcount_t *count) > > - Atomically bump the reference count by one. > > int refcount_release(refcount_t *count) > > - Atomically decerement the reference count by one and return true if the > > count drops to zero. > > As at least John and Alfred know, I'm opposed to an atomic refcount type, > for at least the following reasons: > > 1) The number of bits of accuracy varies from platform to platform. It > seems that the least common denominator we'd need to settle on is 24 > bits (sparc64), which is enough for most, but not all uses. This is the > kernel we're talking about here; I don't like the uncertainty of the > accuracy. Maybe 16.7 million is enough accuracy, but... > Where does the 24 bits for sparc64 come from? According to the ultra-sparc manual it has an atomic compare and swap instruction, casx, which operates on a 64 bit value. AFAIK only sparc32 is restricted to 24 bits, because of the ldstub instruction. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 19:57:56 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 19:57:53 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 072DE37B400; Tue, 12 Dec 2000 19:57:53 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id eBD3vqg05364; Tue, 12 Dec 2000 19:57:52 -0800 (PST) Date: Tue, 12 Dec 2000 19:57:52 -0800 From: Alfred Perlstein To: Jason Evans Cc: John Baldwin , arch@FreeBSD.ORG Subject: Re: An opaque refcount type Message-ID: <20001212195752.T16205@fw.wintelcom.net> References: <20001212183414.K2312@canonware.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20001212183414.K2312@canonware.com>; from jasone@canonware.com on Tue, Dec 12, 2000 at 06:34:14PM -0800 Sender: bright@fw.wintelcom.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Jason Evans [001212 18:34] wrote: > On Tue, Dec 12, 2000 at 12:24:50PM -0800, John Baldwin wrote: > > Here's another bikeshed war for everyone to get in on: I've implemented a > > relatively light weight and very simple opaque reference counter. It defines > > an opaque refcount_t type. In the INVARIANTS case, this maps to a structure > > that contains an integer and a mutex. The mutex is used to protect the > > integer count as well as several KASSERT()'s. In the non-INVARIANTS case, it > > maps to a single integer on all of our current platforms (x86, alpha, ia64) and > > is managed via atomic operations. It defines a rather simple API: > > > > void refcount_init(refcount_t *count, u_int value) > > - Initializes the reference counter and sets its inital count to 'value'. > > void refcount_destroy(refcount_t *count) > > - Cleans up any internals used in a refcount, frees resources, etc. > > void refcount_acquire(refcount_t *count) > > - Atomically bump the reference count by one. > > int refcount_release(refcount_t *count) > > - Atomically decerement the reference count by one and return true if the > > count drops to zero. > > As at least John and Alfred know, I'm opposed to an atomic refcount type, > for at least the following reasons: > > 1) The number of bits of accuracy varies from platform to platform. It > seems that the least common denominator we'd need to settle on is 24 > bits (sparc64), which is enough for most, but not all uses. This is the > kernel we're talking about here; I don't like the uncertainty of the > accuracy. Maybe 16.7 million is enough accuracy, but... Now we have a situation where the whole mbuf subsystem needs an atomic_max or something, because the second we port to an arch that doesn't support atomic_cmpset_long() we're screwed. > 2) It is hard to say at this point since we still have a lot of conversion > work to do, but I suspect that the number of places where a refcount is > ideally manipulated outside of a mutex is small. Please look at the Linux kernel source for examples of effecient usage of refcounts. > 3) The refcount_init() and refcount_destroy() calls are mandatory, but are > only really necessary for platforms that have to rely on mutexes for the > internal implementation (or if INVARIANTS is defined). For the most > part, this just amounts to extra API complexity. It just makes sense to have these. > 4) For platforms that can't support an atomic refcount with atomic > operations, we have to use mutexes. Now, instead of having using normal > atomic operations for refcounts, we have a mutex structure in place of > an integer. So, in some cases, this API is actually a pessimization > rather than an optimization, and any structures that use a refcount_t > are suddenly much larger. Naturally, this means that any structures > with a refcount_t that are exposed to userland will potentially vary in > size, depending on INVARIANTS. We don't have much of a choice with your suggestion either. > 5) At the moment we have mtx_*(), atomic_*(), [mt]sleep(), and the lockmgr. > In the foreseeable future (maybe not by 5.0, but eventually) we will > probably have mtx_*(), cv_*() (condition variables, replaces [mt]sleep()), > and enhanced reader/writer (actually SIX) locks (replaces lockmgr). > I'd like to see the number of synchronization-related interfaces get > smaller and simpler, not larger. > > In another email thread, I demonstrated how to safely manipulate refcounts > with atomic operations. The main problem to solve is how to avoid a > cleanup race when decrementing the refcount to 0. The following > pseudo-code demonstrates a way to avoid this race. > > atomic_decrement(count) > if (compare_and_set(count, 0, 1)) { /* iff (count == 0) {count = 1}. */ > /* Clean up. */ > } > > This requires two atomic operations for refcount decrement, which, as John > has pointed out, is the same number of locked instructions as would be > required for a mutex acquire/release. However, refcount increment still > only requires one locked instruction. Therefore, this method requires 50% > more locked instructions than a refcount_t implementation would require on > a cooperative platform, and 33% fewer locked instructions than a refcount_t > implementation on an uncooperative platform. > > My knowledge of memory bus locking isn't fresh, but I suspect that using > two locked instructions on the same address in short succession only costs > significantly more than one locked instruction if there is contention. Of > course, if there is contention, then an implementation that only uses one > locked instruction is going to get hurt too, so I suspect that the actual > performance difference between this and a streamlined refcount_t > implementation is rather small. (Someone please correct me if this is > incorrect.) > > Even if performance is significantly worse for this method, point 2 above > could make it irrelevant anyway. In summary, I have a number of specific > and general issues with introducing an atomic refcount type with associated > functions. So let me summarize, your proposal is: 1) non-portable 2) slower 3) more complex and prone to error The road ahead looks like it's going to be a real joy. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 21: 1:14 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 21:01:09 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from assaris.sics.se (dyna225-119.nada.kth.se [130.237.225.119]) by hub.freebsd.org (Postfix) with ESMTP id 9974437B400 for ; Tue, 12 Dec 2000 21:01:08 -0800 (PST) Received: (from assar@localhost) by assaris.sics.se (8.9.3/8.9.3) id GAA59972; Wed, 13 Dec 2000 06:01:09 +0100 (CET) (envelope-from assar) Sender: assar@assaris.sics.se From: assar@FreeBSD.ORG To: Warner Losh Cc: Matt Dillon , kris@citusc.usc.edu, Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel References: <200012120259.eBC2xfb99004@earth.backplane.com> <20001211185610.A1741@citusc.usc.edu> <200012121820.LAA31234@harmony.village.org> Date: 13 Dec 2000 06:01:09 +0100 In-Reply-To: Warner Losh's message of "Tue, 12 Dec 2000 11:20:14 -0700" Message-ID: <5l66kozzhm.fsf@assaris.sics.se> Lines: 15 User-Agent: Gnus/5.070098 (Pterodactyl Gnus v0.98) Emacs/20.6 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG --=-=-= Warner Losh writes: > In message <200012120259.eBC2xfb99004@earth.backplane.com> Matt Dillon writes: > : strcpy() -> sn_strcpy(dst, src, sizeof_destination_buffer) > : strcat() -> sn_strcat(dst, src, sizeof_destination_buffer) > > We already have strlcat and strlcpy in the tree. There's no strlcpy/strlcat in the kernel. But I think there should be (independently of the sbuf stuff) and the appended patch adds them. Any comments/reviews? /assar --=-=-= Content-Disposition: attachment; filename=f-sys-d Index: conf/files =================================================================== RCS file: /home/ncvs/src/sys/conf/files,v retrieving revision 1.451 diff -u -w -r1.451 files --- conf/files 2000/12/12 01:14:24 1.451 +++ conf/files 2000/12/12 03:48:43 @@ -717,6 +717,8 @@ libkern/strcat.c standard libkern/strcmp.c standard libkern/strcpy.c standard +libkern/strlcat.c standard +libkern/strlcpy.c standard libkern/strlen.c standard libkern/strncmp.c standard libkern/strncpy.c standard Index: sys/libkern.h =================================================================== RCS file: /home/ncvs/src/sys/sys/libkern.h,v retrieving revision 1.23 diff -u -w -r1.23 libkern.h --- sys/libkern.h 2000/09/03 11:32:07 1.23 +++ sys/libkern.h 2000/12/12 03:49:10 @@ -84,6 +84,8 @@ char *strcat __P((char *, const char *)); int strcmp __P((const char *, const char *)); char *strcpy __P((char *, const char *)); +size_t strlcat __P((char *, const char *, size_t)); +size_t strlcpy __P((char *, const char *, size_t)); size_t strlen __P((const char *)); int strncmp __P((const char *, const char *, size_t)); char *strncpy __P((char *, const char *, size_t)); --- /dev/null Tue Dec 12 05:30:54 2000 +++ sys/libkern/strlcpy.c Tue Dec 12 04:37:24 2000 @@ -0,0 +1,65 @@ +/* $OpenBSD: strlcpy.c,v 1.4 1999/05/01 18:56:41 millert Exp $ */ + +/* + * Copyright (c) 1998 Todd C. Miller + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote products + * derived from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, + * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY + * AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL + * THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, + * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, + * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; + * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, + * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR + * OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF + * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#include + +/* + * Copy src to string dst of size siz. At most siz-1 characters + * will be copied. Always NUL terminates (unless siz == 0). + * Returns strlen(src); if retval >= siz, truncation occurred. + */ +size_t strlcpy(dst, src, siz) + char *dst; + const char *src; + size_t siz; +{ + register char *d = dst; + register const char *s = src; + register size_t n = siz; + + /* Copy as many bytes as will fit */ + if (n != 0 && --n != 0) { + do { + if ((*d++ = *s++) == 0) + break; + } while (--n != 0); + } + + /* Not enough room in dst, add NUL and traverse rest of src */ + if (n == 0) { + if (siz != 0) + *d = '\0'; /* NUL-terminate dst */ + while (*s++) + ; + } + + return(s - src - 1); /* count does not include NUL */ +} --- /dev/null Tue Dec 12 05:30:54 2000 +++ sys/libkern/strlcat.c Tue Dec 12 04:37:38 2000 @@ -0,0 +1,68 @@ +/* $OpenBSD: strlcat.c,v 1.2 1999/06/17 16:28:58 millert Exp $ */ + +/* + * Copyright (c) 1998 Todd C. Miller + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. The name of the author may not be used to endorse or promote products + * derived from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, + * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY + * AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL + * THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, + * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, + * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; + * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, + * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR + * OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF + * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#include + +/* + * Appends src to string dst of size siz (unlike strncat, siz is the + * full size of dst, not space left). At most siz-1 characters + * will be copied. Always NUL terminates (unless siz == 0). + * Returns strlen(src); if retval >= siz, truncation occurred. + */ +size_t strlcat(dst, src, siz) + char *dst; + const char *src; + size_t siz; +{ + register char *d = dst; + register const char *s = src; + register size_t n = siz; + size_t dlen; + + /* Find the end of dst and adjust bytes left but don't go past end */ + while (*d != '\0' && n-- != 0) + d++; + dlen = d - dst; + n = siz - dlen; + + if (n == 0) + return(dlen + strlen(s)); + while (*s != '\0') { + if (n != 1) { + *d++ = *s; + n--; + } + s++; + } + *d = '\0'; + + return(dlen + (s - src)); /* count does not include NUL */ +} --=-=-=-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 23:20:52 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 23:20:50 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mass.osd.bsdi.com (adsl-63-202-176-64.dsl.snfc21.pacbell.net [63.202.176.64]) by hub.freebsd.org (Postfix) with ESMTP id 175D037B400 for ; Tue, 12 Dec 2000 23:20:50 -0800 (PST) Received: from mass.osd.bsdi.com (localhost [127.0.0.1]) by mass.osd.bsdi.com (8.11.1/8.11.1) with ESMTP id eBD7UV307046 for ; Tue, 12 Dec 2000 23:30:35 -0800 (PST) (envelope-from msmith@mass.osd.bsdi.com) Message-Id: <200012130730.eBD7UV307046@mass.osd.bsdi.com> X-Mailer: exmh version 2.1.1 10/15/1999 To: arch@freebsd.org Subject: Proposed bus address typedef. Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 12 Dec 2000 23:30:31 -0800 From: Mike Smith Sender: msmith@mass.osd.bsdi.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I'd like to propose some changes to the way we represent bus addresses, to deal with situations where u_long (IMO not a good choice to begin with) is too small. Specifically, I'd like to be able to deal with x86 systems in PAE mode, where physical addresses are 36 bits in size. To achieve this, I would like to introduce a typedef, bus_addr_t, which is defined on a given platform as being large enough to contain any bus-addressable location. This type would then be used in the following applications (initially): - The resource manager - The resource list code (helpers for bus implementations) - The busspace and busdma subsystems Note that this will break binary compatibility with loadable modules, but should not cause significant problems at the source level with code currently assuming that u_long or u_int32_t is big enough (except when it isn't, of course). Comments? -- ... every activity meets with opposition, everyone who acts has his rivals and unfortunately opponents also. But not because people want to be opponents, rather because the tasks and relationships force people to take different points of view. [Dr. Fritz Todt] V I C T O R Y N O T V E N G E A N C E To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 23:24:59 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 23:24:58 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 07D3C37B402; Tue, 12 Dec 2000 23:24:58 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id eBD7OvB10790; Tue, 12 Dec 2000 23:24:57 -0800 (PST) Date: Tue, 12 Dec 2000 23:24:57 -0800 From: Alfred Perlstein To: Mike Smith Cc: arch@FreeBSD.ORG Subject: Re: Proposed bus address typedef. Message-ID: <20001212232457.X16205@fw.wintelcom.net> References: <200012130730.eBD7UV307046@mass.osd.bsdi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200012130730.eBD7UV307046@mass.osd.bsdi.com>; from msmith@FreeBSD.ORG on Tue, Dec 12, 2000 at 11:30:31PM -0800 Sender: bright@fw.wintelcom.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Mike Smith [001212 23:20] wrote: > > I'd like to propose some changes to the way we represent bus addresses, to > deal with situations where u_long (IMO not a good choice to begin with) > is too small. Sure sounds like a nescessary change. > Specifically, I'd like to be able to deal with x86 systems in PAE mode, > where physical addresses are 36 bits in size. Er, don't PAE machines use segmentation registers? There's no 64bit registers are there? If that's not true, any chance on it becoming a complile time option to save cycles on non PAE machines? -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 23:28:52 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 23:28:50 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mass.osd.bsdi.com (adsl-63-202-176-64.dsl.snfc21.pacbell.net [63.202.176.64]) by hub.freebsd.org (Postfix) with ESMTP id 92B2137B400 for ; Tue, 12 Dec 2000 23:28:49 -0800 (PST) Received: from mass.osd.bsdi.com (localhost [127.0.0.1]) by mass.osd.bsdi.com (8.11.1/8.11.1) with ESMTP id eBD7cc307071 for ; Tue, 12 Dec 2000 23:38:38 -0800 (PST) (envelope-from msmith@mass.osd.bsdi.com) Message-Id: <200012130738.eBD7cc307071@mass.osd.bsdi.com> X-Mailer: exmh version 2.1.1 10/15/1999 To: arch@FreeBSD.ORG Subject: Re: Proposed bus address typedef. In-reply-to: Your message of "Tue, 12 Dec 2000 23:24:57 PST." <20001212232457.X16205@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 12 Dec 2000 23:38:38 -0800 From: Mike Smith Sender: msmith@mass.osd.bsdi.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > * Mike Smith [001212 23:20] wrote: > > > > I'd like to propose some changes to the way we represent bus addresses, to > > deal with situations where u_long (IMO not a good choice to begin with) > > is too small. > > Sure sounds like a nescessary change. It's only "necessary" really in this one case, although I'd like to step away from "u_long" just because it's so vague. > > Specifically, I'd like to be able to deal with x86 systems in PAE mode, > > where physical addresses are 36 bits in size. > > Er, don't PAE machines use segmentation registers? There's no > 64bit registers are there? There are no 64-bit registers, no. > If that's not true, any chance on it becoming a complile time option > to save cycles on non PAE machines? That would definitely be a worthwhile optimisation, although it would lead to nasty binary compatibility issues between PAE and non-PAE driver modules, for example. -- ... every activity meets with opposition, everyone who acts has his rivals and unfortunately opponents also. But not because people want to be opponents, rather because the tasks and relationships force people to take different points of view. [Dr. Fritz Todt] V I C T O R Y N O T V E N G E A N C E To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 23:38: 1 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 23:37:58 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id 7E65F37B402; Tue, 12 Dec 2000 23:37:57 -0800 (PST) Received: from beppo (beppo [192.67.166.79]) by feral.com (8.9.3/8.9.3) with ESMTP id XAA19306; Tue, 12 Dec 2000 23:38:04 -0800 Date: Tue, 12 Dec 2000 23:37:57 -0800 (PST) From: Matthew Jacob Reply-To: mjacob@feral.com To: Mike Smith Cc: arch@FreeBSD.ORG Subject: Re: Proposed bus address typedef. In-Reply-To: <200012130730.eBD7UV307046@mass.osd.bsdi.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG A bus address is always going to be some kind of compound type. For example, a VME address is always a duple of address and address modifier. But what is you're trying to get at here? The needs of the busdma subsystem are such that a compound type is natural (and you extract the n-bit cookie to program your dma engine as needed from it- this is very similar to Solaris). But the CPU needs of bus address are different in different contexts. There's a natural virtual flat size (e.g., 32 bits for ia32, 64 bits for alpha). But there are possibly associated extra bits that don't fit conveniently in the natural pointer word size (I assume the PAE stuff is like this). But it's going to need to either strongly typed as to what to do with the extra bits or you have to be platform cognizant. I would suggest that you define bus address as an opaque handle with methods for extracting: a kernel virtual pointer to the object for the calling cpu [ note that this *could* be different for different CPUs ] an appropriate DMA value that can be used to point to the object (note that this implies seperate methods for 32 and 64 bit PCI implementations) if you *must* have a scalar that represents the raw bits (if possible) that reflect the platform physical representation of the object, pick the largest integer scalar we support (64 bits). This would, for example, have the full 40 bit address in alpha of some I/O addresses (e.g.). [ a list of other types could go here ] That is, if you're boing to break binary compatibility, break it thoroughly and leave us some headroom to DTRT with respect to objects. -matt On Tue, 12 Dec 2000, Mike Smith wrote: > > I'd like to propose some changes to the way we represent bus addresses, to > deal with situations where u_long (IMO not a good choice to begin with) > is too small. > > Specifically, I'd like to be able to deal with x86 systems in PAE mode, > where physical addresses are 36 bits in size. > > To achieve this, I would like to introduce a typedef, bus_addr_t, which > is defined on a given platform as being large enough to contain any > bus-addressable location. > > This type would then be used in the following applications (initially): > > - The resource manager > - The resource list code (helpers for bus implementations) > - The busspace and busdma subsystems > > Note that this will break binary compatibility with loadable modules, but > should not cause significant problems at the source level with code > currently assuming that u_long or u_int32_t is big enough (except when it > isn't, of course). > > Comments? > > -- > ... every activity meets with opposition, everyone who acts has his > rivals and unfortunately opponents also. But not because people want > to be opponents, rather because the tasks and relationships force > people to take different points of view. [Dr. Fritz Todt] > V I C T O R Y N O T V E N G E A N C E > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 23:46:44 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 23:46:40 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from relay.nuxi.com (nuxi.cs.ucdavis.edu [169.237.7.38]) by hub.freebsd.org (Postfix) with ESMTP id F2EF837B402; Tue, 12 Dec 2000 23:46:39 -0800 (PST) Received: from dragon.nuxi.com (Ipittythefoolthattrustsident@trang.nuxi.com [209.152.133.57]) by relay.nuxi.com (8.9.3/8.9.3) with ESMTP id XAA68401; Tue, 12 Dec 2000 23:45:36 -0800 (PST) (envelope-from obrien@NUXI.com) Received: (from obrien@localhost) by dragon.nuxi.com (8.11.1/8.11.1) id eBD7jF673930; Tue, 12 Dec 2000 23:45:15 -0800 (PST) (envelope-from obrien) Date: Tue, 12 Dec 2000 23:45:14 -0800 From: "David O'Brien" To: assar@FreeBSD.ORG Cc: arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel Message-ID: <20001212234514.A73840@dragon.nuxi.com> Reply-To: arch@FreeBSD.ORG References: <200012120259.eBC2xfb99004@earth.backplane.com> <20001211185610.A1741@citusc.usc.edu> <200012121820.LAA31234@harmony.village.org> <5l66kozzhm.fsf@assaris.sics.se> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <5l66kozzhm.fsf@assaris.sics.se>; from assar@FreeBSD.ORG on Wed, Dec 13, 2000 at 06:01:09AM +0100 X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD group X-Pgp-Rsa-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Rsa-Keyid: 1024/34F9F9D5 Sender: obrien@NUXI.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Wed, Dec 13, 2000 at 06:01:09AM +0100, assar@FreeBSD.ORG wrote: > There's no strlcpy/strlcat in the kernel. But I think there should be > (independently of the sbuf stuff) and the appended patch adds them. > Any comments/reviews? Will you also be looking into using them so we get some milage out of the extra kernel size? (size is an on the boot floppy). -- -- David (obrien@FreeBSD.org) GNU is Not Unix / Linux Is Not UniX To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Dec 12 23:59:45 2000 From owner-freebsd-arch@FreeBSD.ORG Tue Dec 12 23:59:44 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from assaris.sics.se (dyna225-119.nada.kth.se [130.237.225.119]) by hub.freebsd.org (Postfix) with ESMTP id 942AA37B402 for ; Tue, 12 Dec 2000 23:59:40 -0800 (PST) Received: (from assar@localhost) by assaris.sics.se (8.9.3/8.9.3) id IAA61308; Wed, 13 Dec 2000 08:59:48 +0100 (CET) (envelope-from assar) Sender: assar@assaris.sics.se From: assar@FreeBSD.ORG To: arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel References: <200012120259.eBC2xfb99004@earth.backplane.com> <20001211185610.A1741@citusc.usc.edu> <200012121820.LAA31234@harmony.village.org> <5l66kozzhm.fsf@assaris.sics.se> <20001212234514.A73840@dragon.nuxi.com> Date: 13 Dec 2000 08:59:47 +0100 In-Reply-To: "David O'Brien"'s message of "Tue, 12 Dec 2000 23:45:14 -0800" Message-ID: <5lvgsovjik.fsf@assaris.sics.se> Lines: 21 User-Agent: Gnus/5.070098 (Pterodactyl Gnus v0.98) Emacs/20.6 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG "David O'Brien" writes: > On Wed, Dec 13, 2000 at 06:01:09AM +0100, assar@FreeBSD.ORG wrote: > > There's no strlcpy/strlcat in the kernel. But I think there should be > > (independently of the sbuf stuff) and the appended patch adds them. > > Any comments/reviews? > > Will you also be looking into using them so we get some milage out of the > extra kernel size? (size is an on the boot floppy). Once they're there, they should of course be used. And strcpy/strcat might get removed when they're not used any longer. :-) Sizes I get on i386: text filename 111 strlcat.o 69 strlcpy.o 35 strcpy.o 43 strcat.o /assar To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 0:13:43 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 00:13:40 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from smtp05.primenet.com (smtp05.primenet.com [206.165.6.135]) by hub.freebsd.org (Postfix) with ESMTP id AEF2B37B400; Wed, 13 Dec 2000 00:13:39 -0800 (PST) Received: (from daemon@localhost) by smtp05.primenet.com (8.9.3/8.9.3) id BAA28907; Wed, 13 Dec 2000 01:10:06 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp05.primenet.com, id smtpdAAA2YaaA4; Wed Dec 13 01:10:00 2000 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id BAA25955; Wed, 13 Dec 2000 01:13:30 -0700 (MST) From: Terry Lambert Message-Id: <200012130813.BAA25955@usr08.primenet.com> Subject: Re: Can !curproc touch To: jhb@FreeBSD.org (John Baldwin) Date: Wed, 13 Dec 2000 08:13:25 +0000 (GMT) Cc: tlambert@primenet.com (Terry Lambert), arch@FreeBSD.org In-Reply-To: from "John Baldwin" at Dec 11, 2000 02:37:16 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: tlambert@usr08.primenet.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > >> I've got a question about p_cred in proc, specifically > >> p_cred->pc_ucred. In several VOP's and other places we > >> use p_cred->pc_ucred (aka p_ucred) as the credentials > >> if we don't already have one. The problem arises if > >> another process can crfree() that ucred either by a > >> crcopy() or a direct crfree() of p_ucred. This was incorrectly attributed to me. > We already share ucred's, and we already do reference counting > for ucred structures. xref crfree()/crcopy()/crhold(), etc. > Really, Terry, reading the code and reading the questions in > detail would help here. Please read my statements in the contect of the question. In particular, I think that "crfree" should be hidden in crunref() (or whatever the inverse of "crhold" is) on the 1->0 reference decrement, and "crcopy" needs to die. Sorry if that wasn't clear from my posting. > My question is about protecting the p_ucred pointer that is part > of struct proc. I'm not trying to lock the ucred itself, I'm > trying to figure out how to ensure that the pointer to a ucred > I get out of proc is valid when I pass it to a VOP and that it > _stays_ valid the entire time. I know that if need be I can do > it by protecting the p_ucred pointer with the proc lock and > bumping the refcount before passing it down the VOP stack, but > if I can get away w/o having to do that I'd like to avoid the > cost. My point was that holding the proc lock is implicit in the addition of the reference and the assignment of the pointer in the proc structure initially. I was under the impression that your function that you are entering for the purpose of dereferencing the value out of the proc structure could hold _its own_ reference to the credential on entry, releasing it on exit. Given the approach I outlined, this would prevent the 1->0 reansition, since there would be a positive reference which could not go away. Therefore the credential you entered with is valid _at least_ until you exit. The interaction of the credential modifying POSIX functions and threads is a pain. I think that any operation that is begun with a particular credential should remain with that credential until the kernel returns to user space. This implies to me that the credential reference is handled on kernel entry, and that a reference is created at that time, and that it's the reference, not the dereference value, which would be used. This kind of implies to me that, since a credential change is a rare and unlikely event, that the reference should probably be cached on a per KSE basis, rather than referenced out of the proc structure anyway. It seems to me that on exit to user space, the KSE value would be compared to the proc value, and if they did not match, a reference would be added to the proc value, subtracted from the KSE value, and the KSE value replaced. Obviously, the KSE would be locked over the subtract/replace operation, since the old credential could become invalid during this process, and you want it in a consistant state. I don't see this as being different from what I outlined previously (though more detailed, here), and I see it being very different from the current code. To my mind, doing lazy binding of the credentials in the proc structure to those referenced by the thread/KSE structure is the best way of reducing the need for locking in all but the exceptional circumstance of a credential change: since that is very rare, and requires priviledge, in any case, I really don't have a problem with an exit-to-user-space penalty that might seem a bit steep at first glance. I also think that whatever approach is used shouldn't really architect against the possibility of handing around opaque objects like NT domain credentials, Kerberos tickets, or similar things that aren't currently on FreeBSD's radar. Does my suggested approach make more sense, and appear more distinct from the current implementation, now? Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 0:25:52 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 00:25:50 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from smtp05.primenet.com (smtp05.primenet.com [206.165.6.135]) by hub.freebsd.org (Postfix) with ESMTP id 0704A37B400 for ; Wed, 13 Dec 2000 00:25:50 -0800 (PST) Received: (from daemon@localhost) by smtp05.primenet.com (8.9.3/8.9.3) id BAA01381; Wed, 13 Dec 2000 01:22:14 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp05.primenet.com, id smtpdAAAgSaORc; Wed Dec 13 01:22:09 2000 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id BAA26231; Wed, 13 Dec 2000 01:25:38 -0700 (MST) From: Terry Lambert Message-Id: <200012130825.BAA26231@usr08.primenet.com> Subject: Re: Safe string formatting in the kernel To: kris@citusc.usc.edu Date: Wed, 13 Dec 2000 08:25:37 +0000 (GMT) Cc: des@ofug.org (Dag-Erling Smorgrav), arch@FreeBSD.ORG In-Reply-To: <20001211185610.A1741@citusc.usc.edu> from "kris@citusc.usc.edu" at Dec 11, 2000 06:56:10 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: tlambert@usr08.primenet.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > I've implemented a set of functions for performing safe string > > formatting in the kernel, based on an initial idea (and design) by > > Poul-Henning. There's a patch up on freefall: > > I haven't reviewed this implementation, but introducing a secure > string handling API into the kernel has my support as security > officer. The current abuse of sprintf() in the kernel is really, > really scary. FWIW, Linux doesn't have the equivalent of a copyinstr() or other string manipulation. The only place that Linux copies strings in or out is in their path manipulation for file names (unless you count symbol resoloution via module loading). I've been a fan of this approach, ever since I fixed a memory leak in the failure path (submitted via Matt Day in 1997). It is much more robust; I've been troubled by the mount option cruft in BSD, and the more string stuff goes into the kernel, the less happy I become with it. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 0:28:31 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 00:28:30 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (placeholder-dcat-1076843399.broadbandoffice.net [64.47.83.135]) by hub.freebsd.org (Postfix) with ESMTP id E6E0037B400; Wed, 13 Dec 2000 00:28:29 -0800 (PST) Received: (from dillon@localhost) by earth.backplane.com (8.11.1/8.9.3) id eBD8SOT81109; Wed, 13 Dec 2000 00:28:24 -0800 (PST) (envelope-from dillon) Date: Wed, 13 Dec 2000 00:28:24 -0800 (PST) From: Matt Dillon Message-Id: <200012130828.eBD8SOT81109@earth.backplane.com> To: Alfred Perlstein Cc: Mike Smith , arch@FreeBSD.ORG Subject: Re: Proposed bus address typedef. References: <200012130730.eBD7UV307046@mass.osd.bsdi.com> <20001212232457.X16205@fw.wintelcom.net> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I'd prefer it not be a compile-time option. The code is complicated enough already. Pointers on IA32 are still 32 bits... just 32 bits per address space. So there is no particular need for them to be visible outside the device driver core. Most of the rest of the kernel that accesses a physical address could just as well use a page index rather then an actual address. A page index still fits in an int (i.e. vm_page_t->phys_addr could easily become a page index, or could become a machine-dependant MMU compatible value if not a page index). Most of the rest of the kernel either uses virtual addresses, which are still 32 bits, or block numbers. If the change can be made without impacting the size of heavily used system structures I'm all for it. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 0:39:16 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 00:39:14 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from smtp01.primenet.com (smtp01.primenet.com [206.165.6.131]) by hub.freebsd.org (Postfix) with ESMTP id 3D51F37B400; Wed, 13 Dec 2000 00:39:14 -0800 (PST) Received: (from daemon@localhost) by smtp01.primenet.com (8.9.3/8.9.3) id BAA07015; Wed, 13 Dec 2000 01:37:57 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp01.primenet.com, id smtpdAAA67aGNn; Wed Dec 13 01:37:48 2000 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id BAA26590; Wed, 13 Dec 2000 01:38:58 -0700 (MST) From: Terry Lambert Message-Id: <200012130838.BAA26590@usr08.primenet.com> Subject: Re: An opaque refcount type To: jhb@FreeBSD.ORG (John Baldwin) Date: Wed, 13 Dec 2000 08:38:58 +0000 (GMT) Cc: arch@FreeBSD.ORG In-Reply-To: from "John Baldwin" at Dec 12, 2000 12:24:50 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: tlambert@usr08.primenet.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > The patch to do all this is found at > http://www.FreeBSD.org/~jhb/patches/refcount.patch. If this goes in I'll work > up a manpage to go along with it as well.. Nice... Chuck, I guess it's your turn to speak up about "premature optimization"? 8-) 8-). I'm all for it, since I'd like to avoid mixing more complex semantics in with the implementation, going forward, since that's the type of thing that's very hard to "speed up later". IMO... Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 0:43:11 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 00:43:09 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from smtp04.primenet.com (smtp04.primenet.com [206.165.6.134]) by hub.freebsd.org (Postfix) with ESMTP id 47C2137B400; Wed, 13 Dec 2000 00:43:09 -0800 (PST) Received: (from daemon@localhost) by smtp04.primenet.com (8.9.3/8.9.3) id BAA16958; Wed, 13 Dec 2000 01:38:54 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp04.primenet.com, id smtpdAAA76ay6G; Wed Dec 13 01:38:45 2000 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id BAA26666; Wed, 13 Dec 2000 01:42:53 -0700 (MST) From: Terry Lambert Message-Id: <200012130842.BAA26666@usr08.primenet.com> Subject: Re: Safe string formatting in the kernel To: imp@village.org (Warner Losh) Date: Wed, 13 Dec 2000 08:42:53 +0000 (GMT) Cc: des@ofug.org (Dag-Erling Smorgrav), assar@FreeBSD.ORG, dillon@earth.backplane.com (Matt Dillon), kris@citusc.usc.edu, arch@FreeBSD.ORG In-Reply-To: <200012121825.LAA31285@harmony.village.org> from "Warner Losh" at Dec 12, 2000 11:25:03 AM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: tlambert@usr08.primenet.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Just be careful that your dynamic string growing things don't violate > the hard limit invariants in the kernel. If it produces paths longer > than 1023 characters, for example, it is wrong. Actually, this is rather broken already in the symlink following code in the lookup code, since it expands links in place with the rest of the path components, instead of properly traversing them. So the limit there is less than 1023 for some intermediate component of a path being a (large) symlink. I think the only path lengths you need to worry about are those which ware externalized into user space. This is primarily the "readlink" and mount code statuts reporting, etc.., so it's not much of a real problem. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 0:53: 1 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 00:53:00 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 2D6F137B400 for ; Wed, 13 Dec 2000 00:52:59 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id eBD8pWe79448; Wed, 13 Dec 2000 09:51:33 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Terry Lambert Cc: kris@citusc.usc.edu, des@ofug.org (Dag-Erling Smorgrav), arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel In-Reply-To: Your message of "Wed, 13 Dec 2000 08:25:37 GMT." <200012130825.BAA26231@usr08.primenet.com> Date: Wed, 13 Dec 2000 09:51:32 +0100 Message-ID: <79446.976697492@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <200012130825.BAA26231@usr08.primenet.com>, Terry Lambert writes: >I've been a fan of this approach, ever since I fixed a memory >leak in the failure path (submitted via Matt Day in 1997). It >is much more robust; I've been troubled by the mount option >cruft in BSD, and the more string stuff goes into the kernel, >the less happy I become with it. I don't necessarily see that as a bad thing :-) The main trouble is bad syscall API design: All strings should be passed by pointer+length, rather than asciiz sematics. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 0:53:18 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 00:53:16 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 621D137B400; Wed, 13 Dec 2000 00:53:15 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id JAA65356; Wed, 13 Dec 2000 09:50:43 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Poul-Henning Kamp Cc: Warner Losh , assar@FreeBSD.ORG, Matt Dillon , kris@citusc.usc.edu, arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel References: <55081.976661002@critter> From: Dag-Erling Smorgrav Date: 13 Dec 2000 09:50:42 +0100 In-Reply-To: Poul-Henning Kamp's message of "Tue, 12 Dec 2000 23:43:22 +0100" Message-ID: Lines: 13 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Poul-Henning Kamp writes: > In message <200012122228.PAA33203@harmony.village.org>, Warner Losh writes: > >If there's a known, realtively small, upper limit, why does allocating > >it dynamically buy you when you could have a static buffer? > I have not reread DES's implementation, but in my design doc you > could initialize an sbuf with your own buffer, for exactly that > reason. Yes. Please read the patch before criticizing it. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 0:59: 8 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 00:59:07 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from smtp03.primenet.com (smtp03.primenet.com [206.165.6.133]) by hub.freebsd.org (Postfix) with ESMTP id 96B4437B400; Wed, 13 Dec 2000 00:59:06 -0800 (PST) Received: (from daemon@localhost) by smtp03.primenet.com (8.9.3/8.9.3) id BAA12259; Wed, 13 Dec 2000 01:56:47 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp03.primenet.com, id smtpdAAAx.aa6x; Wed Dec 13 01:56:42 2000 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id BAA26893; Wed, 13 Dec 2000 01:58:58 -0700 (MST) From: Terry Lambert Message-Id: <200012130858.BAA26893@usr08.primenet.com> Subject: Re: An opaque refcount type To: bmilekic@technokratis.com (Bosko Milekic) Date: Wed, 13 Dec 2000 08:58:58 +0000 (GMT) Cc: jasone@canonware.com (Jason Evans), jhb@FreeBSD.ORG (John Baldwin), arch@FreeBSD.ORG In-Reply-To: from "Bosko Milekic" at Dec 12, 2000 10:05:00 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: tlambert@usr08.primenet.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Having a mutex accompany a ref. counter is really not a good idea. The mutex is solely for supporting INVARIANTS. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 1: 6:25 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 01:06:23 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from smtp04.primenet.com (smtp04.primenet.com [206.165.6.134]) by hub.freebsd.org (Postfix) with ESMTP id BFAE437B400 for ; Wed, 13 Dec 2000 01:06:22 -0800 (PST) Received: (from daemon@localhost) by smtp04.primenet.com (8.9.3/8.9.3) id CAA29010; Wed, 13 Dec 2000 02:02:10 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp04.primenet.com, id smtpdAAAtdaaE4; Wed Dec 13 02:02:02 2000 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id CAA27235; Wed, 13 Dec 2000 02:06:07 -0700 (MST) From: Terry Lambert Message-Id: <200012130906.CAA27235@usr08.primenet.com> Subject: Re: Safe string formatting in the kernel To: phk@critter.freebsd.dk (Poul-Henning Kamp) Date: Wed, 13 Dec 2000 09:06:07 +0000 (GMT) Cc: tlambert@primenet.com (Terry Lambert), kris@citusc.usc.edu, des@ofug.org (Dag-Erling Smorgrav), arch@FreeBSD.ORG In-Reply-To: <79446.976697492@critter> from "Poul-Henning Kamp" at Dec 13, 2000 09:51:32 AM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: tlambert@usr08.primenet.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > >I've been a fan of this approach, ever since I fixed a memory > >leak in the failure path (submitted via Matt Day in 1997). It > >is much more robust; I've been troubled by the mount option > >cruft in BSD, and the more string stuff goes into the kernel, > >the less happy I become with it. > > I don't necessarily see that as a bad thing :-) > > The main trouble is bad syscall API design: All strings should be > passed by pointer+length, rather than asciiz sematics. DEFINITELY. This would let you do the allocation based on peeking at the size prior to copying the whole string in. Count prefix strings are one thing the C language has been missing for years. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 1:21:27 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 01:21:24 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from smtp03.primenet.com (smtp03.primenet.com [206.165.6.133]) by hub.freebsd.org (Postfix) with ESMTP id 73A9C37B400; Wed, 13 Dec 2000 01:21:24 -0800 (PST) Received: (from daemon@localhost) by smtp03.primenet.com (8.9.3/8.9.3) id CAA16137; Wed, 13 Dec 2000 02:19:02 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp03.primenet.com, id smtpdAAAHda4EF; Wed Dec 13 02:18:58 2000 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id CAA27456; Wed, 13 Dec 2000 02:21:15 -0700 (MST) From: Terry Lambert Message-Id: <200012130921.CAA27456@usr08.primenet.com> Subject: Re: Can !curproc touch To: jhb@FreeBSD.ORG (John Baldwin) Date: Wed, 13 Dec 2000 09:21:15 +0000 (GMT) Cc: rwatson@FreeBSD.ORG (Robert Watson), arch@FreeBSD.ORG In-Reply-To: from "John Baldwin" at Dec 12, 2000 04:21:57 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: tlambert@usr08.primenet.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Well, currently nothing uses the struct proc mutex when touching > p_ucred. This is some code from coda: [ ... unprotected code ... ] > To fix this I could do this: > > struct ucred *uc; > ... > PROC_LOCK(p); > uc = p->p_ucred; > crhold(uc); > PROC_UNLOCK(p); > error = venus_root(...., uc, ...); > crfree(uc); > > In place of the last line. However, note that p is always curproc. > If all the other places that accees p->p_ucred only do so if p == > curproc, then I don't need to use an explicit lock to protect > p_ucred, as it will be implicitly locked. Thus, my question is is > there a place where a process A can read or write the p_ucred of > process B. If there is, then I need to use the proc mutex to > protect p_ucred. If not, I can leave p_ucred alone. NB: I think you meant to call crcopy() and not crhold(), right? Since p_ucred isn't opaque, you really have no choice but to protect the structure when dereferencing the pointer. The real problem here is that you haven't fixed the code, though. Consider the case of a multithreaded program, where one thread sleeps in "venus_root", _before_ dereferencing the "uc" you pass in. Meanwhile, a second thread does a setuid() call, which destroys the memory pointed to by "p->p_ucred" ... and therefore the memory pointed to by "uc". A subseqnet dereference of "uc" in "venus_root" will fail. If a dereference and adding a reference are each atomic, AND the kernel is non-preemptive, you don't need to protect the dereference, since the aggregate operation will end up being idempotent. If not, then you need to protect the pointer during the dereference of the pointer and the adding of the reference to the credential. Thus the code becomes either: struct ucred *uc; ... PROC_LOCK(p); uc = ADDREF(p->p_ucred); PROC_UNLOCK(p); error = venus_root(...., uc, ...); UNREF(uc); Or: struct ucred *uc; ... uc = ADDREF(p->p_ucred); /* dereference&increment is idempotent*/ error = venus_root(...., uc, ...); UNREF(uc); The first would be required in the INVARIANTS case, with the suggested simple reference count API that was recently discussed. Clearly, avoiding this is a good idea, if possible. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 1:21:48 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 01:21:44 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id B9EB437B69D for ; Wed, 13 Dec 2000 01:21:41 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id eBD9KIe88313; Wed, 13 Dec 2000 10:20:18 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Terry Lambert Cc: kris@citusc.usc.edu, des@ofug.org (Dag-Erling Smorgrav), arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel In-Reply-To: Your message of "Wed, 13 Dec 2000 09:06:07 GMT." <200012130906.CAA27235@usr08.primenet.com> Date: Wed, 13 Dec 2000 10:20:18 +0100 Message-ID: <88311.976699218@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <200012130906.CAA27235@usr08.primenet.com>, Terry Lambert writes: >> >I've been a fan of this approach, ever since I fixed a memory >> >leak in the failure path (submitted via Matt Day in 1997). It >> >is much more robust; I've been troubled by the mount option >> >cruft in BSD, and the more string stuff goes into the kernel, >> >the less happy I become with it. >> >> I don't necessarily see that as a bad thing :-) >> >> The main trouble is bad syscall API design: All strings should be >> passed by pointer+length, rather than asciiz sematics. > >DEFINITELY. > >This would let you do the allocation based on peeking at the >size prior to copying the whole string in. Count prefix strings >are one thing the C language has been missing for years. ...unfortunately, just like many other good things, we can't easily change the API of things like open(2)... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 1:26: 3 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 01:26:01 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from smtp04.primenet.com (smtp04.primenet.com [206.165.6.134]) by hub.freebsd.org (Postfix) with ESMTP id 4A14737B400 for ; Wed, 13 Dec 2000 01:26:01 -0800 (PST) Received: (from daemon@localhost) by smtp04.primenet.com (8.9.3/8.9.3) id CAA08250; Wed, 13 Dec 2000 02:21:49 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp04.primenet.com, id smtpdAAA6AaOaq; Wed Dec 13 02:21:43 2000 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id CAA27496; Wed, 13 Dec 2000 02:25:51 -0700 (MST) From: Terry Lambert Message-Id: <200012130925.CAA27496@usr08.primenet.com> Subject: Re: Safe string formatting in the kernel To: phk@critter.freebsd.dk (Poul-Henning Kamp) Date: Wed, 13 Dec 2000 09:25:51 +0000 (GMT) Cc: tlambert@primenet.com (Terry Lambert), kris@citusc.usc.edu, des@ofug.org (Dag-Erling Smorgrav), arch@FreeBSD.ORG In-Reply-To: <88311.976699218@critter> from "Poul-Henning Kamp" at Dec 13, 2000 10:20:18 AM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: tlambert@usr08.primenet.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > >> I don't necessarily see that as a bad thing :-) > >> > >> The main trouble is bad syscall API design: All strings should be > >> passed by pointer+length, rather than asciiz sematics. > > > >DEFINITELY. > > > >This would let you do the allocation based on peeking at the > >size prior to copying the whole string in. Count prefix strings > >are one thing the C language has been missing for years. > > ...unfortunately, just like many other good things, we can't > easily change the API of things like open(2)... Why not? The open(2) call is a library stub anyway; I'm strongly of the opinion that POSIX semantics are a near useless subset of the desirable semantics, and map a tiny amount of the problem space. They probably deserve to be in libc, rather than fossilized into the system call interface. For example, the idea of a synchronous system call is really an asynchronous call plus an aiowait on the call status structure... it would sure make it a hell of a lot easier to implement a threads library. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 2:26:43 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 02:26:38 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 55E3B37B402; Wed, 13 Dec 2000 02:26:38 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id eBDAQcl15622; Wed, 13 Dec 2000 02:26:38 -0800 (PST) Date: Wed, 13 Dec 2000 02:26:38 -0800 From: Alfred Perlstein To: arch@freebsd.org Cc: net@freebsd.org Subject: patch to cleanup inflight desciptor handling. Message-ID: <20001213022637.A16205@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: bright@fw.wintelcom.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Not a lot of people are familiar with fd passing so I'll give a short description: By using AF_UNIX sockets between processes, a process can use sendmsg() to send a filedescriptor through the socket where the other process will do a recvmsg() to pickup the descriptor. The "problem" is that if a descriptor is in transit/inflight and the sending process closes the file, it still needs to remain open for the recipient. What can happen is: process A: sendmsg(descriptor) process A: exit process B: exit without the garbage collection we'd have leaked a file descriptor inside the kernel. There's a pretty complex loop in sys/kern/uipc_usrreq.c that deals with garbage collecting these inflight descriptors. The problem with the garbage collection routine is that: 1) it's expensive as it walks all the open files in the system at least twice. 2) it's ugly/hackish 3) it will need to aquire global locks on kernel structure lists for signifigant amounts of time. 4) complicates the code because certain things need to be done out of order, ie sorflush before sofree (which does the sorflush anyway). The solution is actually taken from Linux, in Linux all network buffers have the ability to have a free routine callback done on them when a network buffer is deallocated. FreeBSD only has a free routine available for M_EXT buffers (buffers with external storage), the routine is called when (m_flags & M_EXT) != 0 && m_type != EXT_CLUSTER To achieve my goal I made it so that all fd passing requires an mbuf cluster and took responsibility for freeing the mbuf cluster in my callback. I set m_type == EXT_CMSG_DATA and provide my own free routine until the descriptors are read by the recieving process, if the descriptors are read then i restore it back to a "normal" mbuf with an attached cluster to be free()'d. Good things about this patch: 1) simplifies a) locking b) descriptor management c) the code in general 2) less latency, the gc routine can be expensive 3) some comments are added describing some other stuff that needs fixing. (problems with rfork threads) 4) shrink struct file by one int Problems with this patch: 1) most fd passing probably only sends one descriptor at time, by allocating clusters I'm wasting a lot more space, and taking more time to do the allocation. 2) the mbuf subsystem should provide macros to do what I'm doing (hijacking the free routine on a mbuf+cluster) 3) the mbuf subsystem should provide a way to get a callback on a single mbuf without a cluster attached. http://people.FreeBSD.org/~alfred/inflight.diff thanks, -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 2:32:39 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 02:32:36 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id AE7BA37B400; Wed, 13 Dec 2000 02:32:35 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id LAA65764; Wed, 13 Dec 2000 11:32:34 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: assar@FreeBSD.ORG Cc: arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel References: <200012120259.eBC2xfb99004@earth.backplane.com> <20001211185610.A1741@citusc.usc.edu> <200012121820.LAA31234@harmony.village.org> <5l66kozzhm.fsf@assaris.sics.se> <20001212234514.A73840@dragon.nuxi.com> <5lvgsovjik.fsf@assaris.sics.se> From: Dag-Erling Smorgrav Date: 13 Dec 2000 11:32:33 +0100 In-Reply-To: assar@FreeBSD.ORG's message of "13 Dec 2000 08:59:47 +0100" Message-ID: Lines: 16 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG assar@FreeBSD.ORG writes: > Sizes I get on i386: > > text filename > 111 strlcat.o > 69 strlcpy.o > 35 strcpy.o > 43 strcat.o strlcat() / strlcpy() shouldn't be more than a few bytes larger than strcat() / strcpy(), and the latter can be replaced by macros that call the former with a size of UINT_MAX. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 2:41: 5 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 02:41:03 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from rina.r.dl.itc.u-tokyo.ac.jp (rina.r.dl.itc.u-tokyo.ac.jp [133.11.199.247]) by hub.freebsd.org (Postfix) with ESMTP id 15B8237B402; Wed, 13 Dec 2000 02:41:03 -0800 (PST) Received: from rina.r.dl.itc.u-tokyo.ac.jp (localhost [127.0.0.1]) by rina.r.dl.itc.u-tokyo.ac.jp (8.11.1+3.4Wpre/3.7W-rina.r-0.1-11.01.2000) with ESMTP id eBDAdIC58393; Wed, 13 Dec 2000 19:39:20 +0900 (JST) Date: Wed, 13 Dec 2000 19:39:18 +0900 Message-ID: From: Seigo Tanimura To: jhb@FreeBSD.org Cc: tanimura@r.dl.itc.u-tokyo.ac.jp, arch@FreeBSD.org, bright@wintelcom.net, dillon@earth.backplane.com Subject: Re: Even 1GB KVA is not enough, but we have no more space In-Reply-To: In your message of "Tue, 12 Dec 2000 09:08:40 -0800 (PST)" References: User-Agent: Wanderlust/1.1.1 (Purple Rain) SEMI/1.13.7 (Awazu) FLIM/1.13.2 (Kasanui) MULE XEmacs/21.1 (patch 12) (Channel Islands) (i386--freebsd) Organization: Digital Library Research Division, Information Techinology Centre, The University of Tokyo MIME-Version: 1.0 (generated by SEMI 1.13.7 - "Awazu") Content-Type: text/plain; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, 12 Dec 2000 09:08:40 -0800 (PST), John Baldwin said: John> The reason being that when you do the wakeup(), another CPU might start John> executing other kernel task immediately. The first thing it will do is block Thanks, I added to the committed patch. tegge has point out that we ay still run into a deadlock if malloc(9) of an inode failes. So my next task is to improve vnode recycling (which depends on my learing how vnodes work) -- Seigo Tanimura To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 4: 2:38 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 04:02:31 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from whale.sunbay.crimea.ua (whale.sunbay.crimea.ua [212.110.138.65]) by hub.freebsd.org (Postfix) with ESMTP id D9AE437B400 for ; Wed, 13 Dec 2000 04:02:19 -0800 (PST) Received: (from ru@localhost) by whale.sunbay.crimea.ua (8.11.0/8.11.0) id eBDC0Tm95032; Wed, 13 Dec 2000 14:00:29 +0200 (EET) (envelope-from ru) Date: Wed, 13 Dec 2000 14:00:29 +0200 From: Ruslan Ermilov To: Dag-Erling Smorgrav Cc: arch@FreeBSD.ORG Subject: Re: New sbuf patch on freefall Message-ID: <20001213140029.A94874@sunbay.com> Mail-Followup-To: Dag-Erling Smorgrav , arch@FreeBSD.ORG References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="Nq2Wo0NMKNjxTN9z" Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from des@ofug.org on Tue, Dec 12, 2000 at 10:40:07PM +0100 Sender: ru@whale.sunbay.crimea.ua Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG --Nq2Wo0NMKNjxTN9z Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tue, Dec 12, 2000 at 10:40:07PM +0100, Dag-Erling Smorgrav wrote: > http://people.freebsd.org/~des/software/sbuf-20001212.diff > > I've cleaned up the code, fixed a bug or two, removed some unnecessary > code, added some more assertions, and incorporated most of the > suggestions I've gotten so far, as well as cleaned up the man page. > This patch to sbuf.9 manpage: - Fixes the .Dd call - Removes trailing whitespaces - Adds two missing .Ns calls - .Fa -> .Dv in one place - Removes extraneous duplication of your email address Cheers, -- Ruslan Ermilov Oracle Developer/DBA, ru@sunbay.com Sunbay Software AG, ru@FreeBSD.org FreeBSD committer, +380.652.512.251 Simferopol, Ukraine http://www.FreeBSD.org The Power To Serve http://www.oracle.com Enabling The Information Age --Nq2Wo0NMKNjxTN9z Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="sbuf.9.patch" --- sbuf.9~ Wed Dec 13 13:41:16 2000 +++ sbuf.9 Wed Dec 13 13:56:08 2000 @@ -25,7 +25,7 @@ .\" .\" $FreeBSD$ .\" -.Dd December 10th, 2000 +.Dd December 10, 2000 .Dt sbuf 9 .Os FreeBSD .Sh NAME @@ -42,25 +42,25 @@ .Nd safe string formatting .Sh SYNOPSIS .Fd #include -.Ft int +.Ft int .Fn sbuf_new "struct sbuf *s" "char *buf" "size_t length" "int flags" -.Ft int +.Ft int .Fn sbuf_setpos "struct sbuf *s" "size_t pos" -.Ft int +.Ft int .Fn sbuf_cat "struct sbuf *s" "char *str" -.Ft int +.Ft int .Fn sbuf_cpy "struct sbuf *s" "char *str" -.Ft int +.Ft int .Fn sbuf_printf "struct sbuf *s" "char *fmt" "..." -.Ft int +.Ft int .Fn sbuf_putc "struct sbuf *s" "int c" -.Ft int +.Ft int .Fn sbuf_finish "struct sbuf *s" .Ft char * .Fn sbuf_data "struct sbuf *s" -.Ft size_t +.Ft size_t .Fn sbuf_len "struct sbuf *s" -.Ft void +.Ft void .Fn sbuf_delete "struct sbuf *s" .Sh DESCRIPTION The @@ -104,7 +104,7 @@ The .Fn sbuf_setpos function sets the -.Fa sbuf 's +.Fa sbuf Ns 's current position to .Fa pos , which is a value between zero and one less than the size of the @@ -172,7 +172,7 @@ Finally, the .Fn sbuf_delete function clears the -.Fa sbuf +.Fa sbuf and frees its storage buffer if it was allocated by .Fn sbuf_new . .Sh NOTES @@ -181,7 +181,7 @@ to overflow, most subsequent operations (including .Fn sbuf_finish ) on it will fail until the -.Fa sbuf 's +.Fa sbuf Ns 's position is reset to a value between 0 and one less than the size of its storage buffer using .Fn sbuf_setpos , @@ -209,7 +209,7 @@ and .Fn sbuf_len return -.Fa NULL +.Dv NULL and 0, respectively, if the buffer overflowed. .Sh SEE ALSO .Xr printf 3 , @@ -230,4 +230,4 @@ .An Dag-Erling Co\(:idan Sm\(/orgrav Aq des@FreeBSD.org . .Pp This manual page was written by -.An Dag-Erling Co\(:idan Sm\(/orgrav Aq des@FreeBSD.org . +.An Dag-Erling Co\(:idan Sm\(/orgrav . --Nq2Wo0NMKNjxTN9z-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 4: 4:17 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 04:04:16 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id AA3CF37B400 for ; Wed, 13 Dec 2000 04:04:14 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id NAA66137; Wed, 13 Dec 2000 13:04:08 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Ruslan Ermilov Cc: arch@FreeBSD.ORG Subject: Re: New sbuf patch on freefall References: <20001213140029.A94874@sunbay.com> From: Dag-Erling Smorgrav Date: 13 Dec 2000 13:04:07 +0100 In-Reply-To: Ruslan Ermilov's message of "Wed, 13 Dec 2000 14:00:29 +0200" Message-ID: Lines: 9 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Ruslan Ermilov writes: > This patch to sbuf.9 manpage: > [...] Thanks! DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 5:46:34 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 05:46:32 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 6CAB337B400 for ; Wed, 13 Dec 2000 05:46:31 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id OAA66483; Wed, 13 Dec 2000 14:46:26 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Terry Lambert Cc: phk@critter.freebsd.dk (Poul-Henning Kamp), kris@citusc.usc.edu, arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel References: <200012130925.CAA27496@usr08.primenet.com> From: Dag-Erling Smorgrav Date: 13 Dec 2000 14:46:26 +0100 In-Reply-To: Terry Lambert's message of "Wed, 13 Dec 2000 09:25:51 +0000 (GMT)" Message-ID: Lines: 11 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Terry Lambert writes: > > ...unfortunately, just like many other good things, we can't > > easily change the API of things like open(2)... > Why not? BSD/OS (and other) binaries would stop working on FreeBSD, to name just one reason. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 5:52:57 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 05:52:55 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.66]) by hub.freebsd.org (Postfix) with ESMTP id 931F237B400; Wed, 13 Dec 2000 05:52:54 -0800 (PST) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.11.0/8.11.0) with ESMTP id eBDDqms79686; Wed, 13 Dec 2000 06:52:53 -0700 (MST) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id GAA38352; Wed, 13 Dec 2000 06:52:48 -0700 (MST) Message-Id: <200012131352.GAA38352@harmony.village.org> To: Terry Lambert Subject: Re: Safe string formatting in the kernel Cc: des@ofug.org (Dag-Erling Smorgrav), assar@FreeBSD.ORG, dillon@earth.backplane.com (Matt Dillon), kris@citusc.usc.edu, arch@FreeBSD.ORG In-reply-to: Your message of "Wed, 13 Dec 2000 08:42:53 GMT." <200012130842.BAA26666@usr08.primenet.com> References: <200012130842.BAA26666@usr08.primenet.com> Date: Wed, 13 Dec 2000 06:52:48 -0700 From: Warner Losh Sender: imp@harmony.village.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <200012130842.BAA26666@usr08.primenet.com> Terry Lambert writes: : > Just be careful that your dynamic string growing things don't violate : > the hard limit invariants in the kernel. If it produces paths longer : > than 1023 characters, for example, it is wrong. : : Actually, this is rather broken already in the symlink following : code in the lookup code, since it expands links in place with : the rest of the path components, instead of properly traversing : them. So the limit there is less than 1023 for some intermediate : component of a path being a (large) symlink. Yes, but.... The longest path that you can give the kernel or get from the kernel is 1023. If the path happens to expand longer due to symlinks, or concatination with pwd, that's OK. I don't think that the kernel constructs a full path like this internally as a string, however. The point is that there are some things that other things expect to be a fixed length and if you go around violating that you are in for trouble. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 5:59:25 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 05:59:23 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mail-relay.eunet.no (mail-relay.eunet.no [193.71.71.242]) by hub.freebsd.org (Postfix) with ESMTP id 7C7A337B402 for ; Wed, 13 Dec 2000 05:59:22 -0800 (PST) Received: from login-1.eunet.no (login-1.eunet.no [193.75.110.2]) by mail-relay.eunet.no (8.9.3/8.9.3/GN) with ESMTP id OAA71215; Wed, 13 Dec 2000 14:59:17 +0100 (CET) (envelope-from mbendiks@eunet.no) Received: from localhost (mbendiks@localhost) by login-1.eunet.no (8.9.3/8.8.8) with ESMTP id OAA37067; Wed, 13 Dec 2000 14:59:17 +0100 (CET) (envelope-from mbendiks@eunet.no) X-Authentication-Warning: login-1.eunet.no: mbendiks owned process doing -bs Date: Wed, 13 Dec 2000 14:59:17 +0100 (CET) From: Marius Bendiksen To: Julian Elischer Cc: Seigo Tanimura , arch@freebsd.org Subject: Re: Even 1GB KVA is not enough, but we have no more space In-Reply-To: <3A2F93C6.7967D1DA@elischer.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > THEORETICALLY it should be possible to put the kernel into a differnt > KV space from the processes and give it 4GB. > Practically, we'd have to do a lot to do this, and it may effect > throughout (page tables loading in and out). Isn't KV space just system pages (ie kernel adress space)? If so, then making the system handle a 4GB KV could be done by maintaining a certain portion of the KV space in the user page directory, and switching to the KV page directory on demand, at that point removing user pages. Marius To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 6:11: 3 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 06:11:00 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mail-relay.eunet.no (mail-relay.eunet.no [193.71.71.242]) by hub.freebsd.org (Postfix) with ESMTP id 9F44F37B402; Wed, 13 Dec 2000 06:10:59 -0800 (PST) Received: from login-1.eunet.no (login-1.eunet.no [193.75.110.2]) by mail-relay.eunet.no (8.9.3/8.9.3/GN) with ESMTP id PAA76585; Wed, 13 Dec 2000 15:10:58 +0100 (CET) (envelope-from mbendiks@eunet.no) Received: from localhost (mbendiks@localhost) by login-1.eunet.no (8.9.3/8.8.8) with ESMTP id PAA37143; Wed, 13 Dec 2000 15:10:58 +0100 (CET) (envelope-from mbendiks@eunet.no) X-Authentication-Warning: login-1.eunet.no: mbendiks owned process doing -bs Date: Wed, 13 Dec 2000 15:10:58 +0100 (CET) From: Marius Bendiksen To: Poul-Henning Kamp Cc: arch@freebsd.org Subject: Re: inheriting the "nodump" flag ? In-Reply-To: <97286.976447504@critter> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > I would like to propose that directories and files inherit the > nodump flag if it is set on the directory they are created in. > > Comments ? protests ? This appears entirely sensible, though you might want a sysctl to toggle the behaviour, or maybe a mount option. Alternately, you might want to put this in a seperate flag, but that mostly defeats the purpose unless you do some hairy magic to lazily propagate the real nodump flag. Marius To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 6:14:16 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 06:14:15 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mail-relay.eunet.no (mail-relay.eunet.no [193.71.71.242]) by hub.freebsd.org (Postfix) with ESMTP id 66F5437B402 for ; Wed, 13 Dec 2000 06:14:14 -0800 (PST) Received: from login-1.eunet.no (login-1.eunet.no [193.75.110.2]) by mail-relay.eunet.no (8.9.3/8.9.3/GN) with ESMTP id PAA78141; Wed, 13 Dec 2000 15:14:10 +0100 (CET) (envelope-from mbendiks@eunet.no) Received: from localhost (mbendiks@localhost) by login-1.eunet.no (8.9.3/8.8.8) with ESMTP id PAA37166; Wed, 13 Dec 2000 15:14:10 +0100 (CET) (envelope-from mbendiks@eunet.no) X-Authentication-Warning: login-1.eunet.no: mbendiks owned process doing -bs Date: Wed, 13 Dec 2000 15:14:10 +0100 (CET) From: Marius Bendiksen To: Hajimu UMEMOTO Cc: des@ofug.org, arch@freebsd.org Subject: Re: %a and %A formats In-Reply-To: <20001211.003800.25391776.ume@mahoroba.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > I don't like this. I've tired enough to address af dependent > programming. What character will you introduce for IPv7 or IPv8...? %4a, %6a, %7a, %8a ? > Furthermore, IPv6 address has scope that IPv4 doesn't have. %a should %6A to bring along scope? Remember, this is all just as a convenience. Marius To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 6:20:16 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 06:20:14 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mail-relay.eunet.no (mail-relay.eunet.no [193.71.71.242]) by hub.freebsd.org (Postfix) with ESMTP id 847F537B400 for ; Wed, 13 Dec 2000 06:20:13 -0800 (PST) Received: from login-1.eunet.no (login-1.eunet.no [193.75.110.2]) by mail-relay.eunet.no (8.9.3/8.9.3/GN) with ESMTP id PAA80746; Wed, 13 Dec 2000 15:20:12 +0100 (CET) (envelope-from mbendiks@eunet.no) Received: from localhost (mbendiks@localhost) by login-1.eunet.no (8.9.3/8.8.8) with ESMTP id PAA37220; Wed, 13 Dec 2000 15:20:12 +0100 (CET) (envelope-from mbendiks@eunet.no) X-Authentication-Warning: login-1.eunet.no: mbendiks owned process doing -bs Date: Wed, 13 Dec 2000 15:20:12 +0100 (CET) From: Marius Bendiksen To: Poul-Henning Kamp Cc: Dag-Erling Smorgrav , Hajimu UMEMOTO , arch@FreeBSD.ORG Subject: Re: %a and %A formats In-Reply-To: <99149.976465929@critter> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > What we *really* need is a %{type} construct with dynmically > loadable renderes for different types. > > Then you could printf("Deny %{proto} from ${ip} to ${ip}", ...) An alternate, though undoubtedly more cumbersome way to do this might be to use vectors of a sort: vector *v; v_create(&v, 3 /*initial*/, 0 /*maxgrow*/); v_append(v, vdesc_ipfw_proto, myproto); v_append(v, vdesc_ipfw_ip, myfromip); v_append(v, vdesc_ipfw_ip, mytoip); say("Deny \1 from \2 to \3", v); v_destroy(v); This mechanism might, however, be reused later, somewhat similar to the va functions, except that we attach arbitrary descriptors to the data that can be used for various forms of rendring and special handling. I have used things like this in my code before, and could supply the code used there if it is desireable. Marius To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 6:36: 0 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 06:35:58 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mail-relay.eunet.no (mail-relay.eunet.no [193.71.71.242]) by hub.freebsd.org (Postfix) with ESMTP id D05ED37B400 for ; Wed, 13 Dec 2000 06:35:57 -0800 (PST) Received: from login-1.eunet.no (login-1.eunet.no [193.75.110.2]) by mail-relay.eunet.no (8.9.3/8.9.3/GN) with ESMTP id PAA88042; Wed, 13 Dec 2000 15:35:53 +0100 (CET) (envelope-from mbendiks@eunet.no) Received: from localhost (mbendiks@localhost) by login-1.eunet.no (8.9.3/8.8.8) with ESMTP id PAA37292; Wed, 13 Dec 2000 15:35:53 +0100 (CET) (envelope-from mbendiks@eunet.no) X-Authentication-Warning: login-1.eunet.no: mbendiks owned process doing -bs Date: Wed, 13 Dec 2000 15:35:53 +0100 (CET) From: Marius Bendiksen To: Seigo Tanimura Cc: dillon@earth.backplane.com, bright@wintelcom.net, arch@FreeBSD.ORG Subject: Re: Even 1GB KVA is not enough, but we have no more space In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Sorry, please forget this matter. The culprit truned out to be a race > condition of testing-and-setting ffs_inode_hash_lock in > ffs_vget(). Mutexifing the test-and-sleep-or-set and wakeup-reset > operations was enough to fix the problem. Now my kernel allocates up > to 316K vnodes on make buildworld and release, so malloc(9) pool of > 200MB is sufficient. (316K vnodes occupy about 90-100MB in kmem_map) For older systems, I used to replace the wakeup with wakeup_one. I guess that would reduce lock contention too. I sent a patch on that topic a while ago to one of the lists when someone was seeing problems with ffs_vget deadlocks. Marius To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 7:12:52 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 07:12:50 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mail.interware.hu (mail.interware.hu [195.70.32.130]) by hub.freebsd.org (Postfix) with ESMTP id 2559237B402; Wed, 13 Dec 2000 07:12:49 -0800 (PST) Received: from portonovo-37.budapest.interware.hu ([195.70.60.101] helo=elischer.org) by mail.interware.hu with esmtp (Exim 3.16 #1 (Debian)) id 146DaI-0007om-00; Wed, 13 Dec 2000 16:12:46 +0100 Sender: julian@FreeBSD.ORG Message-ID: <3A378EA2.A7C14084@elischer.org> Date: Wed, 13 Dec 2000 06:58:42 -0800 From: Julian Elischer X-Mailer: Mozilla 4.7 [en] (X11; U; FreeBSD 5.0-CURRENT i386) X-Accept-Language: en, hu MIME-Version: 1.0 To: mjacob@feral.com Cc: Mike Smith , arch@FreeBSD.ORG Subject: Re: Proposed bus address typedef. References: Content-Type: text/plain; charset=iso-8859-15 Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Matthew Jacob wrote: > > A bus address is always going to be some kind of compound type. For example, a > VME address is always a duple of address and address modifier. > > But what is you're trying to get at here? The needs of the busdma subsystem > are such that a compound type is natural (and you extract the n-bit cookie to > program your dma engine as needed from it- this is very similar to Solaris). > > But the CPU needs of bus address are different in different contexts. There's > a natural virtual flat size (e.g., 32 bits for ia32, 64 bits for alpha). But > there are possibly associated extra bits that don't fit conveniently in the > natural pointer word size (I assume the PAE stuff is like this). But it's > going to need to either strongly typed as to what to do with the extra bits or > you have to be platform cognizant. Mike mentions 36 bits for ia32. I would be hesitant to make the value 36 bits for i386 because having looked at the way they do this, there is no reason they couldn't extend it to 64 bits in the 'pentiumV'. The page table entries have another 28 bits marked "reserved". > > I would suggest that you define bus address as an opaque handle with methods > for extracting: > > a kernel virtual pointer to the object for the calling cpu > [ note that this *could* be different for different CPUs ] > > an appropriate DMA value that can be used to point to the object > (note that this implies seperate methods for 32 and 64 bit PCI > implementations) > > if you *must* have a scalar that represents the raw bits (if possible) > that reflect the platform physical representation of the object, > pick the largest integer scalar we support (64 bits). This would, > for example, have the full 40 bit address in alpha of some I/O > addresses (e.g.). > > [ a list of other types could go here ] > > That is, if you're boing to break binary compatibility, break it thoroughly > and leave us some headroom to DTRT with respect to objects. > > -matt > -- __--_|\ Julian Elischer / \ julian@elischer.org ( OZ ) World tour 2000 ---> X_.---._/ presently in: Budapest v To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 7:39:29 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 07:39:27 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from aslan.scsiguy.com (aslan.scsiguy.com [63.229.232.106]) by hub.freebsd.org (Postfix) with ESMTP id C5DA137B400; Wed, 13 Dec 2000 07:39:26 -0800 (PST) Received: from aslan (localhost [127.0.0.1]) by aslan.scsiguy.com (8.11.0/8.9.3) with ESMTP id eBDFdMs27981; Wed, 13 Dec 2000 08:39:22 -0700 (MST) (envelope-from gibbs@aslan.scsiguy.com) Message-Id: <200012131539.eBDFdMs27981@aslan.scsiguy.com> X-Mailer: exmh version 2.2 06/23/2000 with nmh-1.0.4 To: John Baldwin Cc: Garrett Wollman , arch@FreeBSD.ORG Subject: Re: An opaque refcount type In-Reply-To: Your message of "Tue, 12 Dec 2000 13:06:56 PST." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 13 Dec 2000 08:39:22 -0700 From: "Justin T. Gibbs" Sender: gibbs@aslan.scsiguy.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG >It's opaque in the sense that a user doesn't know what it is inside it. This >means we can freely change around the implementation. For example, in the >INVARIANTS case it adds in lots of extra checks, but to ensure correctness, it >has to add in a mutex to use. My problem with it is that in the instances where you have to acquire a mutex anyway to manage the data, you will not want to use this interface. So, unlike say the LIST macros, there is no chance for our code to standardize on a single refcount API. -- Justin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 8: 0:40 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 08:00:39 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 40D0237B402 for ; Wed, 13 Dec 2000 08:00:38 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id RAA67070; Wed, 13 Dec 2000 17:00:35 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: arch@freebsd.org Subject: objections to sbuf? From: Dag-Erling Smorgrav Date: 13 Dec 2000 17:00:34 +0100 Message-ID: Lines: 5 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Any serious objections to committing the latest sbuf patch? DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 8: 2:45 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 08:02:43 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 0D1E237B400 for ; Wed, 13 Dec 2000 08:02:43 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id RAA67078; Wed, 13 Dec 2000 17:02:42 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: arch@freebsd.org Subject: retiring kernfs From: Dag-Erling Smorgrav Date: 13 Dec 2000 17:02:41 +0100 Message-ID: Lines: 7 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Any serious objections to retiring the (long obsolete) kernfs pseudo-file system? (I'll count any "kernfs? what's that?" replies as "no, go ahead and remove it") DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 8:12:46 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 08:12:43 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from warning.follo.net (warning.follo.net [195.204.136.30]) by hub.freebsd.org (Postfix) with ESMTP id 8421937B402 for ; Wed, 13 Dec 2000 08:12:42 -0800 (PST) Received: (from eivind@localhost) by warning.follo.net (8.9.3/8.9.3) id RAA27725; Wed, 13 Dec 2000 17:12:40 +0100 (CET) Date: Wed, 13 Dec 2000 17:12:39 +0100 From: Eivind Eklund To: Dag-Erling Smorgrav Cc: arch@FreeBSD.ORG Subject: Re: retiring kernfs Message-ID: <20001213171239.C26270@warning.follo.net> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i In-Reply-To: ; from des@ofug.org on Wed, Dec 13, 2000 at 05:02:41PM +0100 Sender: eivind@warning.follo.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Wed, Dec 13, 2000 at 05:02:41PM +0100, Dag-Erling Smorgrav wrote: > Any serious objections to retiring the (long obsolete) kernfs > pseudo-file system? (I'll count any "kernfs? what's that?" replies as > "no, go ahead and remove it") I think we should remove it. We've not recommended using it for half a decade, the relevant parts of the information is available from sysctl (or should be - if it isn't, removing kernfs is a good way to get people to tell us :-), and if we at some point want to pick it up again, it would probably be better to base it off the version in NetBSD (which has been maintained) rather than the cruft we have in our tree. The changes to date in the cruft will be available from CVS. Besides, it reduce the buildtime for LINT and the KLDs :-) Eivind. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 8:20:32 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 08:20:30 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 4288E37B400; Wed, 13 Dec 2000 08:20:29 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id RAA67186; Wed, 13 Dec 2000 17:20:23 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Eivind Eklund Cc: arch@FreeBSD.ORG Subject: Re: retiring kernfs References: <20001213171239.C26270@warning.follo.net> From: Dag-Erling Smorgrav Date: 13 Dec 2000 17:20:22 +0100 In-Reply-To: Eivind Eklund's message of "Wed, 13 Dec 2000 17:12:39 +0100" Message-ID: Lines: 12 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Eivind Eklund writes: > I think we should remove it. We've not recommended using it for half > a decade, the relevant parts of the information is available from sysctl (or > should be [...]) I believe it's all in kern.* and hw.*, except for copyright (which displays the kernel's copyright statement) and time (which gives the current time in seconds and microseconds since the epoch). DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 10: 4:24 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 10:04:21 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 8E3F337B402 for ; Wed, 13 Dec 2000 10:04:20 -0800 (PST) Received: from foo.osd.bsdi.com (root@foo.osd.bsdi.com [204.216.28.137]) by pike.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id eBDI4DE31596; Wed, 13 Dec 2000 10:04:13 -0800 (PST) (envelope-from jhb@foo.osd.bsdi.com) Received: (from jhb@localhost) by foo.osd.bsdi.com (8.11.1/8.11.0) id eBDI3QS29150; Wed, 13 Dec 2000 10:03:26 -0800 (PST) (envelope-from jhb) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200012130813.BAA25955@usr08.primenet.com> Date: Wed, 13 Dec 2000 10:03:26 -0800 (PST) Organization: BSD, Inc. From: John Baldwin To: Terry Lambert Subject: Re: Can !curproc touch Cc: arch@FreeBSD.org Sender: jhb@foo.osd.bsdi.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG >> >> I've got a question about p_cred in proc, specifically >> >> p_cred->pc_ucred. In several VOP's and other places we >> >> use p_cred->pc_ucred (aka p_ucred) as the credentials >> >> if we don't already have one. The problem arises if >> >> another process can crfree() that ucred either by a >> >> crcopy() or a direct crfree() of p_ucred. > > This was incorrectly attributed to me. ?? I wrote that. :) >> We already share ucred's, and we already do reference counting >> for ucred structures. xref crfree()/crcopy()/crhold(), etc. >> Really, Terry, reading the code and reading the questions in >> detail would help here. > > Please read my statements in the contect of the question. In > particular, I think that "crfree" should be hidden in crunref() > (or whatever the inverse of "crhold" is) on the 1->0 reference > decrement, and "crcopy" needs to die. crfree() is the opposite of crhold(). The actual FREE() of a ucred is hidden in crfree() on the 1->0 reference decrement. :) xref src/sys/kern/kern_prot.c >> My question is about protecting the p_ucred pointer that is part >> of struct proc. I'm not trying to lock the ucred itself, I'm >> trying to figure out how to ensure that the pointer to a ucred >> I get out of proc is valid when I pass it to a VOP and that it >> _stays_ valid the entire time. I know that if need be I can do >> it by protecting the p_ucred pointer with the proc lock and >> bumping the refcount before passing it down the VOP stack, but >> if I can get away w/o having to do that I'd like to avoid the >> cost. > > My point was that holding the proc lock is implicit in the > addition of the reference and the assignment of the pointer > in the proc structure initially. Erm, here, let me try to describe this better: Process P Process Q process P gets it's inital ucred from fork VOP_FOO() P passes its ucred in as the credentionals for the VOP. The VOP blocks. setgroups(P) process Q changes P's ucred. The ucred the VOP is referencing has now either changed, has been corrupted, or now is used by some other process. (This is hypothetical of course.) This can only happen if more than one process can read/write P's p_ucred. This problem occurs after process creation, not during. > I was under the impression that your function that you are > entering for the purpose of dereferencing the value out of > the proc structure could hold _its own_ reference to the > credential on entry, releasing it on exit. This is my proposed change, to grab the proc lock to protect the p_ucred pointer from changing out from under me, then use a local variable to add another reference to the ucred and bump the refcount, and then release the proc lock. After the ucred consumer (e.g. VOP) returns (it will run w/o the proc lock), it can lower the refcount. > Given the approach I outlined, this would prevent the 1->0 > reansition, since there would be a positive reference which > could not go away. Therefore the credential you entered > with is valid _at least_ until you exit. Yes, that is what I can guarantee with the above idea. However, if p_ucred is only ever accessed by curproc, then I don't actually need to protect p_ucred with the proc lock as it will never be contended. > Does my suggested approach make more sense, and appear more > distinct from the current implementation, now? I think we are saying the same thing. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.Baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 10:12:33 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 10:12:30 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 5D66437B400; Wed, 13 Dec 2000 10:12:30 -0800 (PST) Received: from foo.osd.bsdi.com (root@foo.osd.bsdi.com [204.216.28.137]) by pike.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id eBDICNE32299; Wed, 13 Dec 2000 10:12:23 -0800 (PST) (envelope-from jhb@foo.osd.bsdi.com) Received: (from jhb@localhost) by foo.osd.bsdi.com (8.11.1/8.11.0) id eBDIBbI29229; Wed, 13 Dec 2000 10:11:37 -0800 (PST) (envelope-from jhb) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200012130921.CAA27456@usr08.primenet.com> Date: Wed, 13 Dec 2000 10:11:36 -0800 (PST) Organization: BSD, Inc. From: John Baldwin To: Terry Lambert Subject: Re: Can !curproc touch Cc: arch@FreeBSD.ORG, (Robert Watson) Sender: jhb@foo.osd.bsdi.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 13-Dec-00 Terry Lambert wrote: >> Well, currently nothing uses the struct proc mutex when touching >> p_ucred. This is some code from coda: > > [ ... unprotected code ... ] > >> To fix this I could do this: >> >> struct ucred *uc; >> ... >> PROC_LOCK(p); >> uc = p->p_ucred; >> crhold(uc); >> PROC_UNLOCK(p); >> error = venus_root(...., uc, ...); >> crfree(uc); >> >> In place of the last line. However, note that p is always curproc. >> If all the other places that accees p->p_ucred only do so if p == >> curproc, then I don't need to use an explicit lock to protect >> p_ucred, as it will be implicitly locked. Thus, my question is is >> there a place where a process A can read or write the p_ucred of >> process B. If there is, then I need to use the proc mutex to >> protect p_ucred. If not, I can leave p_ucred alone. > > NB: I think you meant to call crcopy() and not crhold(), right? Hmm, actually, yes, I can probably do a uc = crcopy(p_ucred) in place of the assignment and crhold(). However, that operation isn't atomic, so theoretically p_ucred might change out from under me, which would be bad. > Since p_ucred isn't opaque, you really have no choice but to > protect the structure when dereferencing the pointer. Yes. :( > The real problem here is that you haven't fixed the code, though. > > Consider the case of a multithreaded program, where one thread > sleeps in "venus_root", _before_ dereferencing the "uc" you > pass in. Meanwhile, a second thread does a setuid() call, which > destroys the memory pointed to by "p->p_ucred" ... and therefore > the memory pointed to by "uc". A subseqnet dereference of "uc" > in "venus_root" will fail. Er, no. When the second thread does a setuid(), setuid() won't destory the ucred because it will have a refcount > 1, instead it will call crcopy(), which will make a duplicate ucred that will become p_ucred, and decrement the refcount on the original ucred. When venus_root() returns and calls crfree(), then the refcount on the original ucred will drop to 0, and _then_ it will be free'd. > If a dereference and adding a reference are each atomic, AND the > kernel is non-preemptive, you don't need to protect the dereference, > since the aggregate operation will end up being idempotent. If > not, then you need to protect the pointer during the dereference > of the pointer and the adding of the reference to the credential. Well, with SMPng, the kernel is somewhat pre-emptive (for light weight interrupt threads at least), so I think I'll be stuck using the proc lock. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.Baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 10:23:20 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 10:23:18 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 4B6FA37B400 for ; Wed, 13 Dec 2000 10:23:18 -0800 (PST) Received: from foo.osd.bsdi.com (root@foo.osd.bsdi.com [204.216.28.137]) by pike.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id eBDIMSE33042; Wed, 13 Dec 2000 10:22:28 -0800 (PST) (envelope-from jhb@foo.osd.bsdi.com) Received: (from jhb@localhost) by foo.osd.bsdi.com (8.11.1/8.11.0) id eBDILg429310; Wed, 13 Dec 2000 10:21:42 -0800 (PST) (envelope-from jhb) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200012131539.eBDFdMs27981@aslan.scsiguy.com> Date: Wed, 13 Dec 2000 10:21:41 -0800 (PST) Organization: BSD, Inc. From: John Baldwin To: "Justin T. Gibbs" Subject: Re: An opaque refcount type Cc: arch@FreeBSD.ORG, Garrett Wollman Sender: jhb@foo.osd.bsdi.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 13-Dec-00 Justin T. Gibbs wrote: >>It's opaque in the sense that a user doesn't know what it is inside it. This >>means we can freely change around the implementation. For example, in the >>INVARIANTS case it adds in lots of extra checks, but to ensure correctness, >>it >>has to add in a mutex to use. > > My problem with it is that in the instances where you have to acquire > a mutex anyway to manage the data, you will not want to use this interface. > So, unlike say the LIST macros, there is no chance for our code to > standardize > on a single refcount API. Yes, I don't like that part of it either. In order to make it fast and lightweight it is not the swiss army knife of refcounts. If we don't use this, then probably we will just fall back to using a mutex for each refcount. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.Baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 10:42:45 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 10:42:44 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (placeholder-dcat-1076843399.broadbandoffice.net [64.47.83.135]) by hub.freebsd.org (Postfix) with ESMTP id CF38237B402 for ; Wed, 13 Dec 2000 10:42:43 -0800 (PST) Received: (from dillon@localhost) by earth.backplane.com (8.11.1/8.9.3) id eBDIgB984584; Wed, 13 Dec 2000 10:42:11 -0800 (PST) (envelope-from dillon) Date: Wed, 13 Dec 2000 10:42:11 -0800 (PST) From: Matt Dillon Message-Id: <200012131842.eBDIgB984584@earth.backplane.com> To: Dag-Erling Smorgrav Cc: arch@FreeBSD.ORG Subject: Re: objections to sbuf? References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :Any serious objections to committing the latest sbuf patch? : :DES :-- :Dag-Erling Smorgrav - des@ofug.org I won't object to you comitting it, but I think it's a huge waste of effort and space, not to mention introducing yet another MALLOC allocation which can potentially deadlock the system at a critical juncture. The kernel just doesn't have any sort of serious string handling problem that using snprintf() and strlcpy() couldn't fix in a second. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 10:52:53 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 10:52:51 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from beastie.mckusick.com (beastie.mckusick.com [209.31.233.184]) by hub.freebsd.org (Postfix) with ESMTP id D1C2A37B400; Wed, 13 Dec 2000 10:52:50 -0800 (PST) Received: from beastie.mckusick.com (localhost [127.0.0.1]) by beastie.mckusick.com (8.9.3/8.9.3) with ESMTP id KAA17423; Wed, 13 Dec 2000 10:52:49 -0800 (PST) (envelope-from mckusick@beastie.mckusick.com) Message-Id: <200012131852.KAA17423@beastie.mckusick.com> To: Alfred Perlstein Subject: Re: patch to cleanup inflight desciptor handling. Cc: arch@freebsd.org, net@freebsd.org In-Reply-To: Your message of "Wed, 13 Dec 2000 02:26:38 PST." <20001213022637.A16205@fw.wintelcom.net> Date: Wed, 13 Dec 2000 10:52:49 -0800 From: Kirk McKusick Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I believe that your changes have been sorely needed for many years. While I would like to see regular mbufs given a callback mechanism, your present approach of using an mbuf cluster solves 90% of the problem. Kirk McKusick To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 10:59:10 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 10:59:09 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 3F7C337B400 for ; Wed, 13 Dec 2000 10:59:08 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id eBDIx3i35721; Wed, 13 Dec 2000 19:59:03 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Matt Dillon Cc: Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: objections to sbuf? In-Reply-To: Your message of "Wed, 13 Dec 2000 10:42:11 PST." <200012131842.eBDIgB984584@earth.backplane.com> Date: Wed, 13 Dec 2000 19:59:03 +0100 Message-ID: <35719.976733943@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <200012131842.eBDIgB984584@earth.backplane.com>, Matt Dillon writes: > I won't object to you comitting it, but I think it's a huge waste > of effort and space, not to mention introducing yet another MALLOC > allocation which can potentially deadlock the system at a critical > juncture. The kernel just doesn't have any sort of serious > string handling problem that using snprintf() and strlcpy() couldn't > fix in a second. Considering mailing list archives content, I think the "... fix in a second" is subject to some debate... A good API saves many programming and debugging hours. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 11: 4:46 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 11:04:44 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (placeholder-dcat-1076843399.broadbandoffice.net [64.47.83.135]) by hub.freebsd.org (Postfix) with ESMTP id 834C937B400 for ; Wed, 13 Dec 2000 11:04:44 -0800 (PST) Received: (from dillon@localhost) by earth.backplane.com (8.11.1/8.9.3) id eBDJ2Vx84987; Wed, 13 Dec 2000 11:02:31 -0800 (PST) (envelope-from dillon) Date: Wed, 13 Dec 2000 11:02:31 -0800 (PST) From: Matt Dillon Message-Id: <200012131902.eBDJ2Vx84987@earth.backplane.com> To: Poul-Henning Kamp Cc: Matt Dillon , Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: objections to sbuf? References: <35719.976733943@critter> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :Considering mailing list archives content, I think the "... fix in :a second" is subject to some debate... : :A good API saves many programming and debugging hours. : :-- :Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 :phk@FreeBSD.ORG | TCP/IP since RFC 956 I grepped through and looked at every sprintf, strcpy, and strcat in the kernel. It is *NOT* a big deal. It is certainly a hellofalot less work to convert those to snprintf/strlcpy/etc then to convert them to sbuf. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 11:10: 6 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 11:10:02 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 252B237B402 for ; Wed, 13 Dec 2000 11:10:00 -0800 (PST) Received: (qmail 62475 invoked by uid 1142); 13 Dec 2000 19:09:59 -0000 Date: 13 Dec 2000 11:09:59 -0800 Date: Wed, 13 Dec 2000 11:09:48 -0800 From: Jason Evans To: Matt Dillon Cc: Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: objections to sbuf? Message-ID: <20001213110948.S2312@canonware.com> References: <200012131842.eBDIgB984584@earth.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200012131842.eBDIgB984584@earth.backplane.com>; from dillon@earth.backplane.com on Wed, Dec 13, 2000 at 10:42:11AM -0800 Sender: jasone@magnesium.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Wed, Dec 13, 2000 at 10:42:11AM -0800, Matt Dillon wrote: > :Any serious objections to committing the latest sbuf patch? > > I won't object to you comitting it, but I think it's a huge waste > of effort and space, not to mention introducing yet another MALLOC > allocation which can potentially deadlock the system at a critical > juncture. The kernel just doesn't have any sort of serious > string handling problem that using snprintf() and strlcpy() couldn't > fix in a second. I agree with Matt. Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 11:12: 1 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 11:11:59 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 286D437B402 for ; Wed, 13 Dec 2000 11:11:58 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id eBDJBsi35888; Wed, 13 Dec 2000 20:11:54 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Matt Dillon Cc: Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: objections to sbuf? In-Reply-To: Your message of "Wed, 13 Dec 2000 11:02:31 PST." <200012131902.eBDJ2Vx84987@earth.backplane.com> Date: Wed, 13 Dec 2000 20:11:54 +0100 Message-ID: <35886.976734714@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <200012131902.eBDJ2Vx84987@earth.backplane.com>, Matt Dillon writes: >:Considering mailing list archives content, I think the "... fix in >:a second" is subject to some debate... >: >:A good API saves many programming and debugging hours. >: >:-- >:Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 >:phk@FreeBSD.ORG | TCP/IP since RFC 956 > > I grepped through and looked at every sprintf, strcpy, and strcat > in the kernel. It is *NOT* a big deal. It is certainly a hellofalot > less work to convert those to snprintf/strlcpy/etc then to convert > them to sbuf. I don't recall anybody mentioning much less suggesting a wholesale rewrite of every string operation in the kernel... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 11:15:50 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 11:15:48 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (placeholder-dcat-1076843399.broadbandoffice.net [64.47.83.135]) by hub.freebsd.org (Postfix) with ESMTP id C379B37B400 for ; Wed, 13 Dec 2000 11:15:47 -0800 (PST) Received: (from dillon@localhost) by earth.backplane.com (8.11.1/8.9.3) id eBDJDgK85146; Wed, 13 Dec 2000 11:13:42 -0800 (PST) (envelope-from dillon) Date: Wed, 13 Dec 2000 11:13:42 -0800 (PST) From: Matt Dillon Message-Id: <200012131913.eBDJDgK85146@earth.backplane.com> To: Poul-Henning Kamp Cc: Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: objections to sbuf? References: <35886.976734714@critter> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG : :In message <200012131902.eBDJ2Vx84987@earth.backplane.com>, Matt Dillon writes: :>:Considering mailing list archives content, I think the "... fix in :>:a second" is subject to some debate... :>: :>:A good API saves many programming and debugging hours. :>: :>:-- :>:Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 :>:phk@FreeBSD.ORG | TCP/IP since RFC 956 :> :> I grepped through and looked at every sprintf, strcpy, and strcat :> in the kernel. It is *NOT* a big deal. It is certainly a hellofalot :> less work to convert those to snprintf/strlcpy/etc then to convert :> them to sbuf. : :I don't recall anybody mentioning much less suggesting a wholesale :rewrite of every string operation in the kernel... : :-- :Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 What's the point of creating a new interface in the kernel for string handling if you don't intend to use it? -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 11:17:20 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 11:17:16 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242]) by hub.freebsd.org (Postfix) with ESMTP id A8E2937B402 for ; Wed, 13 Dec 2000 11:17:16 -0800 (PST) Received: from adlmail.cup.hp.com (adlmail.cup.hp.com [15.0.100.30]) by palrel1.hp.com (Postfix) with ESMTP id 28E26526; Wed, 13 Dec 2000 11:17:14 -0800 (PST) Received: from cup.hp.com (gauss.cup.hp.com [15.28.97.152]) by adlmail.cup.hp.com (8.9.3 (PHNE_18546)/8.9.3 SMKit7.02) with ESMTP id LAA17062; Wed, 13 Dec 2000 11:17:13 -0800 (PST) Sender: marcel@cup.hp.com Message-ID: <3A37CB39.C2E8AA67@cup.hp.com> Date: Wed, 13 Dec 2000 14:17:13 -0500 From: Marcel Moolenaar Organization: Hewlett-Packard X-Mailer: Mozilla 4.73 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Marius Bendiksen Cc: Poul-Henning Kamp , Dag-Erling Smorgrav , Hajimu UMEMOTO , arch@FreeBSD.ORG Subject: Re: %a and %A formats References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Marius Bendiksen wrote: > > vector *v; > v_create(&v, 3 /*initial*/, 0 /*maxgrow*/); > v_append(v, vdesc_ipfw_proto, myproto); > v_append(v, vdesc_ipfw_ip, myfromip); > v_append(v, vdesc_ipfw_ip, mytoip); > say("Deny \1 from \2 to \3", v); > v_destroy(v); Using indexes to refer to the arguments (vectors) has an advantage for i18n. The order of the arguments is related to the language and some translations produce better messages if the order of the arguments isn't fixed. Other than that, I think I prefer a one line printf() over a 7 line say() anytime :-) -- Marcel Moolenaar mail: marcel@cup.hp.com / marcel@FreeBSD.org tel: (408) 447-4222 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 11:20:44 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 11:20:43 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from critter.freebsd.dk (flutter.freebsd.dk [212.242.40.147]) by hub.freebsd.org (Postfix) with ESMTP id 6792937B404 for ; Wed, 13 Dec 2000 11:20:42 -0800 (PST) Received: from critter (localhost [127.0.0.1]) by critter.freebsd.dk (8.11.1/8.11.1) with ESMTP id eBDJKdi36006; Wed, 13 Dec 2000 20:20:39 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Matt Dillon Cc: Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: objections to sbuf? In-Reply-To: Your message of "Wed, 13 Dec 2000 11:13:42 PST." <200012131913.eBDJDgK85146@earth.backplane.com> Date: Wed, 13 Dec 2000 20:20:38 +0100 Message-ID: <36004.976735238@critter> From: Poul-Henning Kamp Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In message <200012131913.eBDJDgK85146@earth.backplane.com>, Matt Dillon writes: >:> I grepped through and looked at every sprintf, strcpy, and strcat >:> in the kernel. It is *NOT* a big deal. It is certainly a hellofalot >:> less work to convert those to snprintf/strlcpy/etc then to convert >:> them to sbuf. >: >:I don't recall anybody mentioning much less suggesting a wholesale >:rewrite of every string operation in the kernel... >: >:-- >:Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > > What's the point of creating a new interface in the kernel for > string handling if you don't intend to use it? We *do* intend to use it, but we don't intend to rewrite the entire kernel wholesale to use it. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 11:23:48 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 11:23:46 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 4B89337B400 for ; Wed, 13 Dec 2000 11:23:45 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id UAA68099; Wed, 13 Dec 2000 20:21:05 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Marcel Moolenaar Cc: Marius Bendiksen , Poul-Henning Kamp , Hajimu UMEMOTO , arch@FreeBSD.ORG Subject: Re: %a and %A formats References: <3A37CB39.C2E8AA67@cup.hp.com> From: Dag-Erling Smorgrav Date: 13 Dec 2000 20:21:04 +0100 In-Reply-To: Marcel Moolenaar's message of "Wed, 13 Dec 2000 14:17:13 -0500" Message-ID: Lines: 13 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Marcel Moolenaar writes: > Using indexes to refer to the arguments (vectors) has an advantage for > i18n. The order of the arguments is related to the language and some > translations produce better messages if the order of the arguments isn't > fixed. Printf() allows you to specify which argument to use for each format specifier. The kernel version doesn't support that, but I don't think it's hard to fix. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 12: 9:18 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 12:09:15 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from smtp02.primenet.com (smtp02.primenet.com [206.165.6.132]) by hub.freebsd.org (Postfix) with ESMTP id 44FA737B400 for ; Wed, 13 Dec 2000 12:09:15 -0800 (PST) Received: (from daemon@localhost) by smtp02.primenet.com (8.9.3/8.9.3) id NAA15197; Wed, 13 Dec 2000 13:04:25 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp02.primenet.com, id smtpdAAAuvaWMD; Wed Dec 13 13:04:18 2000 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id NAA10673; Wed, 13 Dec 2000 13:08:59 -0700 (MST) From: Terry Lambert Message-Id: <200012132008.NAA10673@usr08.primenet.com> Subject: Re: %a and %A formats To: marcel@cup.hp.com (Marcel Moolenaar) Date: Wed, 13 Dec 2000 20:08:59 +0000 (GMT) Cc: mbendiks@eunet.no (Marius Bendiksen), phk@critter.freebsd.dk (Poul-Henning Kamp), des@ofug.org (Dag-Erling Smorgrav), ume@mahoroba.org (Hajimu UMEMOTO), arch@FreeBSD.ORG In-Reply-To: <3A37CB39.C2E8AA67@cup.hp.com> from "Marcel Moolenaar" at Dec 13, 2000 02:17:13 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: tlambert@usr08.primenet.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Using indexes to refer to the arguments (vectors) has an advantage for > i18n. The order of the arguments is related to the language and some > translations produce better messages if the order of the arguments isn't > fixed. Having been forced down this route before, I can tell you that it is an _incredibly_ shortsighted thing to do to localize kernel messages or log messages. A company I worked for _stupidy_ localized log messages. The result of doing this was that users could read the logs in their native language, but engineers supporting problems could not do root cause analysis, since they were trained as (this is a "big surprise") engineers, and not as linguists. Obviously, this was the idea of the UI folks, who wanted to give users eye-candy so that they would be happy with their purchase, and their idea of making this happen was to let them look at the log messages as the code chugged along doing its job, with no thought to support issues. A lot of people whine about sendmail status code not being in their native language. Some SMTP servers actually translate these status codes into other languages (I've seen German and French). Ignoring, for the moment, the _fact_ that RFC821 prohibits non-ASCI characters over an SMTP command transport, there is no locale information about the person on the other end of the SMTP hose. You can't assume their language/locale like this. The technically correct thing to do is to translate error codes in the MUA by providing the correct message for a given error number, in the locale specified by the user for the MUA. In other words, making up for deficient mail clients by changing the error responses is a stupid idea (the same argument applies to decoding DSNs in a locale specific way). For log messages, the appropriate method is to create a format for logging that gives parametric, machine-parsable logs, and then to localize the logs on viewing. A secondary win to this approach is that an appropriately modified syslogd can track state transitions in the programs it is logging for, and then report overall status information (ideally, via SNMP Application or Network Services monitoring MIBs). Kernel messages are, likewise, an issue of status or exceptional condition reporting. It _may_ not be justified to go the whole AIX route and assign facility and error numbers for kernel messages... I'd argue that most of them aren't in English, anyway, and are untranslatable. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 13: 7: 3 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 13:06:59 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (placeholder-dcat-1076843399.broadbandoffice.net [64.47.83.135]) by hub.freebsd.org (Postfix) with ESMTP id ABD8A37B699; Wed, 13 Dec 2000 13:06:59 -0800 (PST) Received: (from dillon@localhost) by earth.backplane.com (8.11.1/8.9.3) id eBDL6Sg86570; Wed, 13 Dec 2000 13:06:28 -0800 (PST) (envelope-from dillon) Date: Wed, 13 Dec 2000 13:06:28 -0800 (PST) From: Matt Dillon Message-Id: <200012132106.eBDL6Sg86570@earth.backplane.com> To: Kirk McKusick Cc: Alfred Perlstein , arch@FreeBSD.ORG, net@FreeBSD.ORG Subject: Re: patch to cleanup inflight desciptor handling. References: <200012131852.KAA17423@beastie.mckusick.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :I believe that your changes have been sorely needed for many :years. While I would like to see regular mbufs given a callback :mechanism, your present approach of using an mbuf cluster :solves 90% of the problem. : : Kirk McKusick ... Aflred, be careful that you don't break things we only just fixed last year. The descriptor passing code has been broken for many years. I think the reason we have to scan the descriptor list is related to locating isolated self-referential 'loops' with descriptor passing and unix domain sockets and closing them. e.g. when you pass a descriptor for a unix-domain socket through a unix-domain socket, it is possible for the socket descriptors to reference each other and thus never have their ref count drop to 0 even when all associated processes have close()'d. This happens all the time. Be sure you don't break the fix that solves that particular problem. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 13:13:28 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 13:13:25 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from netau1.alcanet.com.au (ntp.alcanet.com.au [203.62.196.27]) by hub.freebsd.org (Postfix) with ESMTP id A5D8937B698 for ; Wed, 13 Dec 2000 13:13:24 -0800 (PST) Received: from mfg1.cim.alcatel.com.au (mfg1.cim.alcatel.com.au [139.188.23.1]) by netau1.alcanet.com.au (8.9.3 (PHNE_18979)/8.9.3) with ESMTP id IAA09090; Thu, 14 Dec 2000 08:13:17 +1100 (EDT) Received: from gsmx07.alcatel.com.au by cim.alcatel.com.au (PMDF V5.2-32 #37641) with ESMTP id <01JXORIHTGK0E7Y88Y@cim.alcatel.com.au>; Thu, 14 Dec 2000 08:13:01 +1100 Received: (from jeremyp@localhost) by gsmx07.alcatel.com.au (8.11.0/8.11.0) id eBDLDE683047; Thu, 14 Dec 2000 08:13:14 +1100 (EST envelope-from jeremyp) Content-return: prohibited Date: Thu, 14 Dec 2000 08:13:14 +1100 From: Peter Jeremy Subject: Re: %a and %A formats In-reply-to: <200012132008.NAA10673@usr08.primenet.com>; from tlambert@primenet.com on Wed, Dec 13, 2000 at 08:08:59PM +0000 To: Terry Lambert Cc: arch@FreeBSD.ORG Mail-followup-to: Terry Lambert , arch@FreeBSD.ORG Message-id: <20001214081314.D69646@gsmx07.alcatel.com.au> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii Content-disposition: inline User-Agent: Mutt/1.2.5i References: <3A37CB39.C2E8AA67@cup.hp.com> <200012132008.NAA10673@usr08.primenet.com> Sender: jeremyp@gsmx07.alcatel.com.au Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 2000-Dec-13 20:08:59 +0000, Terry Lambert wrote: >> Using indexes to refer to the arguments (vectors) has an advantage for >> i18n. The order of the arguments is related to the language and some >> translations produce better messages if the order of the arguments isn't >> fixed. > >Having been forced down this route before, I can tell you that >it is an _incredibly_ shortsighted thing to do to localize >kernel messages or log messages. I agree with this viewpoint - but since I don't read any other languages, I could be considered partisan :-). I'd be interested in opposing arguments from some EASL[*] developers. The kernel source code (including the occasional comment) is roughly English. Anyone able to understand the source code can presumably understand the messages (or at least find them with grep). (The counter-argument is that you shouldn't need to read the source to work out what a kernel message means). IMHO, one thing that developers can do to help non-English developers would be to avoid the use of abbreviations messages. In my first job, I was maintaining code where the comments were in a mixture of English and Flemish[#]. Understanding the abbreviated Flemish was not easy :-). At least with a complete word, an English-??? dictionary might provide a non-English speaker with some insight. > Some SMTP servers actually translate these status codes into other >languages (I've seen German and French). Which is why MTAs interpret the error number, rather than message. >For log messages, the appropriate method is to create a format >for logging that gives parametric, machine-parsable logs, and >then to localize the logs on viewing. Agreed. It's far better to log everything and summarise it for the sysadmin, rather than just log things that are critical. In the latter case, you will know something happened, but the back- ground information to understand why it happened is probably in the bit-bucket. >Kernel messages are, likewise, an issue of status or exceptional >condition reporting. It _may_ not be justified to go the whole >AIX route and assign facility and error numbers for kernel >messages... Having a unique error number associated with each error message makes it a lot easier to look that error message up in a manual to find out more about it. Though I suspect it'll be some time before FreeBSD has a messages manual along the lines of an Oracle or IBM product :-). > I'd argue that most of them aren't in English, >anyway, and are untranslatable. For many technical terms, other languages have inherited the English word in any case. The following messages would probably be equally [un]intelligible to a sysadmin of any linguistic background irrespective of the language they were translated into (about the only critical word would be distinguishing `master' from `slave' in the disk probe). fxp0: port 0x6c00-0x6c3f mem 0xe5800000-0xe58fffff,0xe5904000-0xe5904fff irq 9 at device 11.0 on pci0 ad2: 1222MB [2484/16/63] at ata1-master using WDMA2 arp: 192.168.156.130 moved from 08:00:2b:c4:f6:73 to 00:00:f8:10:98:05 on vlan2 [*] English as a second language [#] Spoken in parts of Belgium, very similar to Dutch. Peter To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 13:23: 1 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 13:22:55 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from Awfulhak.org (awfulhak.demon.co.uk [194.222.196.252]) by hub.freebsd.org (Postfix) with ESMTP id 9188037B402; Wed, 13 Dec 2000 13:22:53 -0800 (PST) Received: from hak.lan.Awfulhak.org (root@hak.lan.awfulhak.org [172.16.0.12]) by Awfulhak.org (8.11.1/8.11.1) with ESMTP id eBDLKcZ03838; Wed, 13 Dec 2000 21:20:39 GMT (envelope-from brian@lan.awfulhak.org) Received: from hak.lan.Awfulhak.org (brian@localhost [127.0.0.1]) by hak.lan.Awfulhak.org (8.11.1/8.11.1) with ESMTP id eBDLOBU68538; Wed, 13 Dec 2000 21:24:11 GMT (envelope-from brian@hak.lan.Awfulhak.org) Message-Id: <200012132124.eBDLOBU68538@hak.lan.Awfulhak.org> X-Mailer: exmh version 2.2 06/23/2000 with nmh-1.0.4 To: Alfred Perlstein Cc: arch@FreeBSD.ORG, net@FreeBSD.ORG, brian@Awfulhak.org Subject: Re: patch to cleanup inflight desciptor handling. In-Reply-To: Message from Alfred Perlstein of "Wed, 13 Dec 2000 02:26:38 PST." <20001213022637.A16205@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 13 Dec 2000 21:24:11 +0000 From: Brian Somers Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Not a lot of people are familiar with fd passing so I'll give > a short description: > > By using AF_UNIX sockets between processes, a process can use > sendmsg() to send a filedescriptor through the socket where the > other process will do a recvmsg() to pickup the descriptor. > > The "problem" is that if a descriptor is in transit/inflight > and the sending process closes the file, it still needs to > remain open for the recipient. > > What can happen is: > > process A: sendmsg(descriptor) > process A: exit > process B: exit > > without the garbage collection we'd have leaked a file descriptor > inside the kernel. [.....] Hmm, the last time i looked at this, I believe the whole thing was dealt with by not increasing the file descriptor reference count when it was put in the message header. If process A closed the descriptor before process B actually recvmsg()d it, it would be EBADF. The recvmsg() actually incremented the reference count. I always assumed that this was a result of not wanting to have any funny garbage collecting code ? Of course looking at the code now, it increases fp->f_count in unp_internalize(), so maybe I was on drugs then.... Assuming I wasn't on drugs (maybe the behaviour was changed - cvs annotate suggests some activity around March 9, but that was the file*/int alignment stuff), I think it would be valid to go back to the old behaviour (not increasing f_count and letting the original owner actually close the descriptor while f_msgcount != 0). Does any of this make sense ? Or am I just describing a different case (where process B doesn't exit) ? > -- > -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] > "I have the heart of a child; I keep it in a jar on my desk." -- Brian Don't _EVER_ lose your sense of humour ! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 13:43: 0 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 13:42:58 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mail.interware.hu (mail.interware.hu [195.70.32.130]) by hub.freebsd.org (Postfix) with ESMTP id AF62137B400 for ; Wed, 13 Dec 2000 13:42:56 -0800 (PST) Received: from monrovia-11.budapest.interware.hu ([195.70.53.203] helo=elischer.org) by mail.interware.hu with esmtp (Exim 3.16 #1 (Debian)) id 146Jfl-0000LB-00; Wed, 13 Dec 2000 22:42:50 +0100 Sender: julian@FreeBSD.ORG Message-ID: <3A37ED35.8AC618D5@elischer.org> Date: Wed, 13 Dec 2000 13:42:13 -0800 From: Julian Elischer X-Mailer: Mozilla 4.7 [en] (X11; U; FreeBSD 5.0-CURRENT i386) X-Accept-Language: en, hu MIME-Version: 1.0 To: Matt Dillon Cc: Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: objections to sbuf? References: <200012131842.eBDIgB984584@earth.backplane.com> Content-Type: text/plain; charset=iso-8859-15 Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Matt Dillon wrote: > > :Any serious objections to committing the latest sbuf patch? > : > :DES > :-- > :Dag-Erling Smorgrav - des@ofug.org > > I won't object to you comitting it, but I think it's a huge waste > of effort and space, not to mention introducing yet another MALLOC > allocation which can potentially deadlock the system at a critical > juncture. The kernel just doesn't have any sort of serious > string handling problem that using snprintf() and strlcpy() couldn't > fix in a second. What he said.. (I haven't see a glaring need for this yet... BTW I lost the URL to the new improved version...) > > -Matt > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message -- __--_|\ Julian Elischer / \ julian@elischer.org ( OZ ) World tour 2000 ---> X_.---._/ presently in: Budapest v To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 14:18:36 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 14:18:34 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mailhub.teliauk.com (mailhub.teliauk.com [195.12.225.36]) by hub.freebsd.org (Postfix) with ESMTP id 1D51A37B699; Wed, 13 Dec 2000 14:18:33 -0800 (PST) Received: from d1o314.teliauk.com (root@d1o314.teliauk.com [195.12.237.81]) by mailhub.teliauk.com (8.10.1/8.10.1) with ESMTP id eBDMIVA23957; Wed, 13 Dec 2000 22:18:32 GMT Received: from vilnya.demon.co.uk (fakeuser@t1o314p156.teliauk.com [195.12.238.156]) by d1o314.teliauk.com (8.8.8/8.8.8) with ESMTP id WAA09310; Wed, 13 Dec 2000 22:18:10 GMT Received: from haveblue (haveblue.rings [10.2.4.5]) by vilnya.demon.co.uk (Postfix) with SMTP id 86D4ED9B8; Wed, 13 Dec 2000 22:17:29 +0000 (GMT) Message-ID: <011501c06552$c1730c40$0504020a@haveblue> From: "Cameron Grant" To: , References: <015a01c064ad$96667230$0504020a@haveblue> Subject: Re: newpcm/kobj Date: Wed, 13 Dec 2000 22:19:27 -0000 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > in the near future, i intend to commit my kobjified newpcm. this gives us > several benefits, including: > > * easier extensibility- new optional methods can be added to > ac97/mixer/channel classes without having to fixup every driver. > > * forward compatibility for drivers, provided no new mandatory methods are > added. > > however, all drivers not in the tree at this time will need to be updated. > > i hope to mfc to -stable in approximately one month, along with the kobj > system. newbus in -stable will not be kobjified. the diff for newpcm/kobj is at http://people.freebsd.org/~cg/kobj.diff.gz -cg To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 14:19:24 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 14:19:19 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 9234437B698; Wed, 13 Dec 2000 14:19:19 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id eBDMJIx04918; Wed, 13 Dec 2000 14:19:18 -0800 (PST) Date: Wed, 13 Dec 2000 14:19:18 -0800 From: Alfred Perlstein To: Matt Dillon Cc: Kirk McKusick , arch@FreeBSD.ORG, net@FreeBSD.ORG Subject: Re: patch to cleanup inflight desciptor handling. Message-ID: <20001213141917.Q16205@fw.wintelcom.net> References: <200012131852.KAA17423@beastie.mckusick.com> <200012132106.eBDL6Sg86570@earth.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200012132106.eBDL6Sg86570@earth.backplane.com>; from dillon@earth.backplane.com on Wed, Dec 13, 2000 at 01:06:28PM -0800 Sender: bright@fw.wintelcom.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Matt Dillon [001213 13:07] wrote: > :I believe that your changes have been sorely needed for many > :years. While I would like to see regular mbufs given a callback > :mechanism, your present approach of using an mbuf cluster > :solves 90% of the problem. > : > : Kirk McKusick > > ... Aflred, be careful that you don't break things we only just fixed > last year. The descriptor passing code has been broken for many years. > > I think the reason we have to scan the descriptor list is related to > locating isolated self-referential 'loops' with descriptor passing and > unix domain sockets and closing them. e.g. when you pass a descriptor > for a unix-domain socket through a unix-domain socket, it is possible > for the socket descriptors to reference each other and thus never have > their ref count drop to 0 even when all associated processes have > close()'d. This happens all the time. Be sure you don't break the > fix that solves that particular problem. Ok, I'll see if that can happen. Basically since the reference never goes to zero on the socket, the buffers are never forced to be flushed/cleared and the mbuf will then never be free'd resulting it it leaking itself. Basically a socket hanging there with an mbuf referencing itself. I wonder if Linux fixed/has this problem. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 14:53:51 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 14:53:44 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 65D9737B404; Wed, 13 Dec 2000 14:53:42 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id eBDMrff06270; Wed, 13 Dec 2000 14:53:41 -0800 (PST) Date: Wed, 13 Dec 2000 14:53:41 -0800 From: Alfred Perlstein To: Matt Dillon Cc: Kirk McKusick , arch@FreeBSD.ORG, net@FreeBSD.ORG Subject: Re: patch to cleanup inflight desciptor handling. Message-ID: <20001213145341.S16205@fw.wintelcom.net> References: <200012131852.KAA17423@beastie.mckusick.com> <200012132106.eBDL6Sg86570@earth.backplane.com> <20001213141917.Q16205@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20001213141917.Q16205@fw.wintelcom.net>; from bright@wintelcom.net on Wed, Dec 13, 2000 at 02:19:18PM -0800 Sender: bright@fw.wintelcom.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Alfred Perlstein [001213 14:20] wrote: > * Matt Dillon [001213 13:07] wrote: > > :I believe that your changes have been sorely needed for many > > :years. While I would like to see regular mbufs given a callback > > :mechanism, your present approach of using an mbuf cluster > > :solves 90% of the problem. > > : > > : Kirk McKusick > > > > ... Aflred, be careful that you don't break things we only just fixed > > last year. The descriptor passing code has been broken for many years. > > > > I think the reason we have to scan the descriptor list is related to > > locating isolated self-referential 'loops' with descriptor passing and > > unix domain sockets and closing them. e.g. when you pass a descriptor > > for a unix-domain socket through a unix-domain socket, it is possible > > for the socket descriptors to reference each other and thus never have > > their ref count drop to 0 even when all associated processes have > > close()'d. This happens all the time. Be sure you don't break the > > fix that solves that particular problem. > > Ok, I'll see if that can happen. Basically since the reference > never goes to zero on the socket, the buffers are never forced to > be flushed/cleared and the mbuf will then never be free'd resulting > it it leaking itself. Basically a socket hanging there with an > mbuf referencing itself. > > I wonder if Linux fixed/has this problem. Ok, my patch has this problem: void parent(int con) { int fd; fd = open("/tmp/wank", O_RDONLY); send_fd_withdata(con, con, "wank", 4); sleep (5); exit(1); } void child(int con) { int fd, error; char buf[100]; sleep(5); get_fd_withdata(con, &fd, buf, sizeof(buf)); send_fd_withdata(con, fd, "foo", 3); exit(1); buf[4] = '\0'; printf("%s\n", buf); if ((error = read(fd, buf, sizeof(buf))) < 0) perror("read"); buf[sizeof(buf)-1] = '\0'; printf("%s\n", buf); } This causes a leak, I think the trick is to just always call sorflush() when the pcb is free'd. Looking at linux they still are using gc. I'll give this a lot more thought before resubmitting this idea. sorry, -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 14:58:29 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 14:58:26 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from hand.dotat.at (sfo-gw.covalent.net [207.44.198.62]) by hub.freebsd.org (Postfix) with ESMTP id 7CFD637B402; Wed, 13 Dec 2000 14:58:22 -0800 (PST) Received: from fanf by hand.dotat.at with local (Exim 3.15 #3) id 146KqP-00094D-00; Wed, 13 Dec 2000 22:57:53 +0000 Date: Wed, 13 Dec 2000 22:57:53 +0000 From: Tony Finch To: Brian Somers Cc: Alfred Perlstein , arch@FreeBSD.ORG, net@FreeBSD.ORG Subject: Re: patch to cleanup inflight desciptor handling. Message-ID: <20001213225753.B30050@hand.dotat.at> References: <200012132124.eBDLOBU68538@hak.lan.Awfulhak.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: <200012132124.eBDLOBU68538@hak.lan.Awfulhak.org> Organization: Covalent Technologies, Inc Sender: fanf@dotat.at Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Brian Somers wrote: > >Hmm, the last time i looked at this, I believe the whole thing was >dealt with by not increasing the file descriptor reference count >when it was put in the message header. If process A closed the >descriptor before process B actually recvmsg()d it, it would be >EBADF. The recvmsg() actually incremented the reference count. But it has always been documented behaviour that the receiving process gets a valid descriptor even if the sender closes it directly after sendmsging it. If this was not the case then descriptor handoff would require an "ok" reply from the receiving process before the sender could close it, which is a pain. Hmm, the only references for this I can think of are Stevens and the red & black daemon books, but I'm sure I've read a good discussion of it somewhere else. Tony. -- f.a.n.finch fanf@covalent.net dot@dotat.at "And remember my friend, future events such as these will affect you in the future." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 15:18:41 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 15:18:40 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from relay.nuxi.com (nuxi.cs.ucdavis.edu [169.237.7.38]) by hub.freebsd.org (Postfix) with ESMTP id 5B97B37B402 for ; Wed, 13 Dec 2000 15:18:40 -0800 (PST) Received: from dragon.nuxi.com (Ipittythefoolthattrustsident@trang.nuxi.com [209.152.133.57]) by relay.nuxi.com (8.9.3/8.9.3) with ESMTP id PAA73011; Wed, 13 Dec 2000 15:18:38 -0800 (PST) (envelope-from obrien@NUXI.com) Received: (from obrien@localhost) by dragon.nuxi.com (8.11.1/8.11.1) id eBDNIb184294; Wed, 13 Dec 2000 15:18:37 -0800 (PST) (envelope-from obrien) Date: Wed, 13 Dec 2000 15:18:32 -0800 From: "David O'Brien" To: Dag-Erling Smorgrav Cc: arch@freebsd.org Subject: Re: retiring kernfs Message-ID: <20001213151832.A84135@dragon.nuxi.com> Reply-To: arch@freebsd.org References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from des@ofug.org on Wed, Dec 13, 2000 at 05:02:41PM +0100 X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD group X-Pgp-Rsa-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Rsa-Keyid: 1024/34F9F9D5 Sender: obrien@NUXI.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Wed, Dec 13, 2000 at 05:02:41PM +0100, Dag-Erling Smorgrav wrote: > Any serious objections to retiring the (long obsolete) kernfs > pseudo-file system? Can you give a reason for your wanting to remove something that compiles and works? -- -- David (obrien@FreeBSD.org) GNU is Not Unix / Linux Is Not UniX To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 15:20:51 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 15:20:49 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from relay.nuxi.com (nuxi.cs.ucdavis.edu [169.237.7.38]) by hub.freebsd.org (Postfix) with ESMTP id 1A9F837B400 for ; Wed, 13 Dec 2000 15:20:49 -0800 (PST) Received: from dragon.nuxi.com (Ipittythefoolthattrustsident@trang.nuxi.com [209.152.133.57]) by relay.nuxi.com (8.9.3/8.9.3) with ESMTP id PAA73032 for ; Wed, 13 Dec 2000 15:20:48 -0800 (PST) (envelope-from obrien@NUXI.com) Received: (from obrien@localhost) by dragon.nuxi.com (8.11.1/8.11.1) id eBDNKla84314 for arch@FreeBSD.ORG; Wed, 13 Dec 2000 15:20:47 -0800 (PST) (envelope-from obrien) Date: Wed, 13 Dec 2000 15:20:47 -0800 From: "David O'Brien" To: arch@FreeBSD.ORG Subject: Re: objections to sbuf? Message-ID: <20001213152046.B84135@dragon.nuxi.com> Reply-To: arch@FreeBSD.ORG References: <35886.976734714@critter> <200012131913.eBDJDgK85146@earth.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200012131913.eBDJDgK85146@earth.backplane.com>; from dillon@earth.backplane.com on Wed, Dec 13, 2000 at 11:13:42AM -0800 X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD group X-Pgp-Rsa-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Rsa-Keyid: 1024/34F9F9D5 Sender: obrien@NUXI.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Wed, Dec 13, 2000 at 11:13:42AM -0800, Matt Dillon wrote: > :>:a second" is subject to some debate... > :>:A good API saves many programming and debugging hours. > :> > :> I grepped through and looked at every sprintf, strcpy, and strcat > :> in the kernel. It is *NOT* a big deal. It is certainly a hellofalot > :> less work to convert those to snprintf/strlcpy/etc then to convert > :> them to sbuf. > : > :I don't recall anybody mentioning much less suggesting a wholesale > :rewrite of every string operation in the kernel... > : > :-- > :Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > > What's the point of creating a new interface in the kernel for > string handling if you don't intend to use it? > > -Matt I agree 100%. If we aren't going to whole-sale convert to it then all we are doing is bloating the kernel (remember, it still has to fit on a floppy). To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 15:36:55 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 15:36:51 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id CBF2B37B402; Wed, 13 Dec 2000 15:36:50 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id eBDNan707825; Wed, 13 Dec 2000 15:36:49 -0800 (PST) Date: Wed, 13 Dec 2000 15:36:49 -0800 From: Alfred Perlstein To: Matt Dillon Cc: Kirk McKusick , arch@FreeBSD.ORG, net@FreeBSD.ORG Subject: Re: patch to cleanup inflight desciptor handling. Message-ID: <20001213153649.T16205@fw.wintelcom.net> References: <200012131852.KAA17423@beastie.mckusick.com> <200012132106.eBDL6Sg86570@earth.backplane.com> <20001213141917.Q16205@fw.wintelcom.net> <20001213145341.S16205@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20001213145341.S16205@fw.wintelcom.net>; from bright@wintelcom.net on Wed, Dec 13, 2000 at 02:53:41PM -0800 Sender: bright@fw.wintelcom.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > This causes a leak, I think the trick is to just always call sorflush() > when the pcb is free'd. Even this doesn't work. > > Looking at linux they still are using gc. I'll give this a lot > more thought before resubmitting this idea. Ok, the problem is you have 3 af_unix pairs all open between 2 processes process B sends 3 over 2 to A process B sends 2 over 3 to A process B send 2 and 3 over 1 to A process B closes 1 2 and 3 A then closes 3 2 and then 1 closing 3 and 2 doesn't cause the socketbuffer to be flushed because they are still self referencing. closing 1 causes the socketbuffer to be flushed, on flushing it comes across 2 and drops a reference but doesn't flush, it then hits 3 and drops a reference but doesn't flush. since 3 references 2 and 2 references 3 and nothing else references 2 or 3, we just leaked 2 and 3. I guess the gc has to stay. dammit. :) My apologies for wasting everyone's time here. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 15:48:11 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 15:48:08 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from hand.dotat.at (sfo-gw.covalent.net [207.44.198.62]) by hub.freebsd.org (Postfix) with ESMTP id 2923737B404; Wed, 13 Dec 2000 15:48:08 -0800 (PST) Received: from fanf by hand.dotat.at with local (Exim 3.15 #3) id 146Lcp-000Ale-00; Wed, 13 Dec 2000 23:47:55 +0000 Date: Wed, 13 Dec 2000 23:47:55 +0000 From: Tony Finch To: Alfred Perlstein Cc: Matt Dillon , Kirk McKusick , arch@FreeBSD.ORG, net@FreeBSD.ORG Subject: Re: patch to cleanup inflight desciptor handling. Message-ID: <20001213234755.U71002@hand.dotat.at> References: <200012131852.KAA17423@beastie.mckusick.com> <200012132106.eBDL6Sg86570@earth.backplane.com> <20001213141917.Q16205@fw.wintelcom.net> <20001213145341.S16205@fw.wintelcom.net> <20001213153649.T16205@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: <20001213153649.T16205@fw.wintelcom.net> Organization: Covalent Technologies, Inc Sender: fanf@dotat.at Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Alfred Perlstein wrote: > >I guess the gc has to stay. > >dammit. :) > >My apologies for wasting everyone's time here. ``One day a student came to Moon and said: "I understand how to make a better garbage collector. We must keep a reference count of the pointers to each cons." ``Moon patiently told the student the following story: ``"One day a student came to Moon and said: `I understand how to make a better garbage collector...'"'' :-) Tony. -- f.a.n.finch fanf@covalent.net dot@dotat.at "Perhaps on your way home you will pass someone in the dark, and you will never know it, for they will be from outer space." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 15:54: 4 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 15:54:00 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from Awfulhak.org (awfulhak.demon.co.uk [194.222.196.252]) by hub.freebsd.org (Postfix) with ESMTP id 0762F37B404; Wed, 13 Dec 2000 15:53:59 -0800 (PST) Received: from hak.lan.Awfulhak.org (root@hak.lan.awfulhak.org [172.16.0.12]) by Awfulhak.org (8.11.1/8.11.1) with ESMTP id eBDNplZ04667; Wed, 13 Dec 2000 23:51:47 GMT (envelope-from brian@lan.awfulhak.org) Received: from hak.lan.Awfulhak.org (brian@localhost [127.0.0.1]) by hak.lan.Awfulhak.org (8.11.1/8.11.1) with ESMTP id eBDNtJU70418; Wed, 13 Dec 2000 23:55:19 GMT (envelope-from brian@hak.lan.Awfulhak.org) Message-Id: <200012132355.eBDNtJU70418@hak.lan.Awfulhak.org> X-Mailer: exmh version 2.2 06/23/2000 with nmh-1.0.4 To: Tony Finch Cc: Brian Somers , Alfred Perlstein , arch@FreeBSD.ORG, net@FreeBSD.ORG, brian@Awfulhak.org Subject: Re: patch to cleanup inflight desciptor handling. In-Reply-To: Message from Tony Finch of "Wed, 13 Dec 2000 22:57:53 GMT." <20001213225753.B30050@hand.dotat.at> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 13 Dec 2000 23:55:19 +0000 From: Brian Somers Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Brian Somers wrote: > > > >Hmm, the last time i looked at this, I believe the whole thing was > >dealt with by not increasing the file descriptor reference count > >when it was put in the message header. If process A closed the > >descriptor before process B actually recvmsg()d it, it would be > >EBADF. The recvmsg() actually incremented the reference count. > > But it has always been documented behaviour that the receiving process > gets a valid descriptor even if the sender closes it directly after > sendmsging it. If this was not the case then descriptor handoff would > require an "ok" reply from the receiving process before the sender > could close it, which is a pain. > > Hmm, the only references for this I can think of are Stevens and the > red & black daemon books, but I'm sure I've read a good discussion of > it somewhere else. I've just looked back through my archives... the problem I'm thinking of was a different problem - where the descriptor passed was the only descriptor open for a tty whose pgrp was that of process A. A passed the descriptor to B and then exited at which point the tty (correctly) revoked all it's remaining descriptors (the one en-route or in process B). There's no way to avoid this - except by having A fork(), the child close the descriptor and continue where it left off and the parent pause() waiting for a signal from B to tell it that it's finished with that tty. This is why I implemented ``enable keep-session'' :-) > Tony. > -- > f.a.n.finch fanf@covalent.net dot@dotat.at > "And remember my friend, future events such > as these will affect you in the future." Cheers. -- Brian Don't _EVER_ lose your sense of humour ! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 16:21:29 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 16:21:27 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mail-relay.eunet.no (mail-relay.eunet.no [193.71.71.242]) by hub.freebsd.org (Postfix) with ESMTP id DB47937B404 for ; Wed, 13 Dec 2000 16:21:26 -0800 (PST) Received: from login-1.eunet.no (login-1.eunet.no [193.75.110.2]) by mail-relay.eunet.no (8.9.3/8.9.3/GN) with ESMTP id BAA72452; Thu, 14 Dec 2000 01:21:08 +0100 (CET) (envelope-from mbendiks@eunet.no) Received: from localhost (mbendiks@localhost) by login-1.eunet.no (8.9.3/8.8.8) with ESMTP id BAA41035; Thu, 14 Dec 2000 01:21:07 +0100 (CET) (envelope-from mbendiks@eunet.no) X-Authentication-Warning: login-1.eunet.no: mbendiks owned process doing -bs Date: Thu, 14 Dec 2000 01:21:07 +0100 (CET) From: Marius Bendiksen To: Marcel Moolenaar Cc: Poul-Henning Kamp , Dag-Erling Smorgrav , Hajimu UMEMOTO , arch@FreeBSD.ORG Subject: Re: %a and %A formats In-Reply-To: <3A37CB39.C2E8AA67@cup.hp.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Using indexes to refer to the arguments (vectors) has an advantage for > i18n. The order of the arguments is related to the language and some > translations produce better messages if the order of the arguments isn't > fixed. Indeed. An alternate mechanism would be to supply a (void *) key which refers to the key, and have argument/property descriptors. But that is overkill, I suppose. > Other than that, I think I prefer a one line printf() over a 7 line > say() anytime :-) And most prefer a simple 'goto bad;' over a full switch statement to handle errors. :). It is a matter of flexibility. Also, one introduces a generic way to pass vectors and variable arguments w/descriptors. I doubt code readability suffers. A number of subsystems could benefit from this, esp. if the keying was used as well. Take the vnode ops, for example. Marius To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 17:25:47 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 17:25:44 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (placeholder-dcat-1076843399.broadbandoffice.net [64.47.83.135]) by hub.freebsd.org (Postfix) with ESMTP id 0181A37B698; Wed, 13 Dec 2000 17:25:44 -0800 (PST) Received: (from dillon@localhost) by earth.backplane.com (8.11.1/8.9.3) id eBE1Pbi89951; Wed, 13 Dec 2000 17:25:37 -0800 (PST) (envelope-from dillon) Date: Wed, 13 Dec 2000 17:25:37 -0800 (PST) From: Matt Dillon Message-Id: <200012140125.eBE1Pbi89951@earth.backplane.com> To: Alfred Perlstein Cc: Kirk McKusick , arch@FreeBSD.ORG, net@FreeBSD.ORG Subject: Re: patch to cleanup inflight desciptor handling. References: <200012131852.KAA17423@beastie.mckusick.com> <200012132106.eBDL6Sg86570@earth.backplane.com> <20001213141917.Q16205@fw.wintelcom.net> <20001213145341.S16205@fw.wintelcom.net> <20001213153649.T16205@fw.wintelcom.net> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :I guess the gc has to stay. : :dammit. :) : :My apologies for wasting everyone's time here. : :-- :-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] No waste at all, Alfred, the file descriptor passing code had been broken for over 10 years precisely because of its complexity. Rewriting the GC to be more efficient essentially requires using deep graph theory to locate isolated loops of arbitrary complexity. p.s. many object oriented language garbage collectors have the same problem. create object A, create object B, A references B, B references A, drop A, drop B. A and B still have references and don't get cleaned up. Fun. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 17:31: 9 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 17:31:05 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 274D037B402; Wed, 13 Dec 2000 17:31:05 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id eBE1V4611720; Wed, 13 Dec 2000 17:31:04 -0800 (PST) Date: Wed, 13 Dec 2000 17:31:04 -0800 From: Alfred Perlstein To: Matt Dillon Cc: Kirk McKusick , arch@FreeBSD.ORG, net@FreeBSD.ORG Subject: Re: patch to cleanup inflight desciptor handling. Message-ID: <20001213173103.B16205@fw.wintelcom.net> References: <200012131852.KAA17423@beastie.mckusick.com> <200012132106.eBDL6Sg86570@earth.backplane.com> <20001213141917.Q16205@fw.wintelcom.net> <20001213145341.S16205@fw.wintelcom.net> <20001213153649.T16205@fw.wintelcom.net> <200012140125.eBE1Pbi89951@earth.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200012140125.eBE1Pbi89951@earth.backplane.com>; from dillon@earth.backplane.com on Wed, Dec 13, 2000 at 05:25:37PM -0800 Sender: bright@fw.wintelcom.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Matt Dillon [001213 17:25] wrote: > > :I guess the gc has to stay. > : > :dammit. :) > : > :My apologies for wasting everyone's time here. > : > :-- > :-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] > > No waste at all, Alfred, the file descriptor passing code had been > broken for over 10 years precisely because of its complexity. Rewriting > the GC to be more efficient essentially requires using deep graph theory > to locate isolated loops of arbitrary complexity. > > p.s. many object oriented language garbage collectors have the same > problem. create object A, create object B, A references B, B references A, > drop A, drop B. A and B still have references and don't get cleaned up. > Fun. Are you saying the code in place is broken? If so I'll spend some time looking at it and the Linux implementation to figure if at least one of us gets it right and try to find some sort of solution. Obviously the easiest way would be to disallow passing of any descriptors that have descriptors in thier socketbuffers. Since almost no one uses this code, and I hardly see a reason for allowing that type of operation (passing af_unix fds with fds in flight) it might be a good idea to just disallow that sort of operation. It would definetly simplify and probably speed up the code. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 17:43:21 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 17:43:18 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (placeholder-dcat-1076843399.broadbandoffice.net [64.47.83.135]) by hub.freebsd.org (Postfix) with ESMTP id EA65F37B698; Wed, 13 Dec 2000 17:43:17 -0800 (PST) Received: (from dillon@localhost) by earth.backplane.com (8.11.1/8.9.3) id eBE1hCx90268; Wed, 13 Dec 2000 17:43:12 -0800 (PST) (envelope-from dillon) Date: Wed, 13 Dec 2000 17:43:12 -0800 (PST) From: Matt Dillon Message-Id: <200012140143.eBE1hCx90268@earth.backplane.com> To: Alfred Perlstein Cc: Kirk McKusick , arch@FreeBSD.ORG, net@FreeBSD.ORG Subject: Re: patch to cleanup inflight desciptor handling. References: <200012131852.KAA17423@beastie.mckusick.com> <200012132106.eBDL6Sg86570@earth.backplane.com> <20001213141917.Q16205@fw.wintelcom.net> <20001213145341.S16205@fw.wintelcom.net> <20001213153649.T16205@fw.wintelcom.net> <200012140125.eBE1Pbi89951@earth.backplane.com> <20001213173103.B16205@fw.wintelcom.net> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :> No waste at all, Alfred, the file descriptor passing code had been : :Are you saying the code in place is broken? If so I'll spend some :time looking at it and the Linux implementation to figure if at :least one of us gets it right and try to find some sort of solution. No, *had*, not *has*. It isn't broken, just inefficient in certain cases due to the brute-force GC. :Obviously the easiest way would be to disallow passing of any :descriptors that have descriptors in thier socketbuffers. : :Since almost no one uses this code, and I hardly see a reason for :allowing that type of operation (passing af_unix fds with fds in :flight) it might be a good idea to just disallow that sort of :operation. : :It would definetly simplify and probably speed up the code. There's no reason to disallow that. Besides, any socket can be listen()'d after having been queued, so you aren't really preventing anything. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 17:49:13 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 17:49:08 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from ringworld.nanolink.com (unknown [195.24.48.13]) by hub.freebsd.org (Postfix) with SMTP id D688037B402 for ; Wed, 13 Dec 2000 17:48:57 -0800 (PST) Received: (qmail 3870 invoked by uid 1000); 14 Dec 2000 01:48:04 -0000 Date: Thu, 14 Dec 2000 03:48:04 +0200 From: Peter Pentchev To: arch@FreeBSD.org Subject: add -I ignoremask option to du(1) Message-ID: <20001214034803.C575@ringworld.oblivion.bg> Mail-Followup-To: arch@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Hi, Is there a reason no one has done this yet? :) I find it particularly nice for e.g. excluding CVS/ subdirs from du output. And yes, I know this can be done with a bit of find(1) hackery, but the attached patch seems almost good enough to me (modulo the **ignmasks storage; maybe a list would be in order there, but I think there would very rarely be many more than 2 or 3 ignore masks). Comments? Flames? "Shut-up-already"'s? :) G'luck, Peter -- This sentence would be seven words long if it were six words shorter. Index: src/usr.bin/du/du.1 =================================================================== RCS file: /home/ncvs/src/usr.bin/du/du.1,v retrieving revision 1.18 diff -u -r1.18 du.1 --- src/usr.bin/du/du.1 2000/11/20 19:20:41 1.18 +++ src/usr.bin/du/du.1 2000/12/14 01:38:10 @@ -41,6 +41,7 @@ .Sh SYNOPSIS .Nm .Op Fl P | Fl H | Fl L +.Op Fl I Ar mask .Op Fl a | s | d Ar depth .Op Fl c .Op Fl h | k @@ -72,6 +73,9 @@ hierarchies are not followed. .It Fl L Symbolic links on the command line and in file hierarchies are followed. +.It Fl I Ar mask +Ignore files and directories matching the specified +.Ar mask . .It Fl a Display an entry for each file in a file hierarchy. .It Fl h Index: src/usr.bin/du/du.c =================================================================== RCS file: /home/ncvs/src/usr.bin/du/du.c,v retrieving revision 1.19 diff -u -r1.19 du.c --- src/usr.bin/du/du.c 2000/03/26 14:21:57 1.19 +++ src/usr.bin/du/du.c 2000/12/14 01:38:10 @@ -54,6 +54,7 @@ #include #include +#include #include #include #include @@ -88,10 +89,16 @@ int unitp [] = { NONE, KILO, MEGA, GIGA, TERA, PETA }; +char ** ignmasks = NULL; +int igncount = 0; + int linkchk __P((FTSENT *)); static void usage __P((void)); void prthumanval __P((double)); unit_t unit_adjust __P((double *)); +void ignoreadd __P((char *)); +void ignoreclean __P((void)); +int ignorep __P((FTSENT *)); int main(argc, argv) @@ -113,11 +120,14 @@ ftsoptions = 0; depth = INT_MAX; - while ((ch = getopt(argc, argv, "HLPasd:chkrx")) != -1) + while ((ch = getopt(argc, argv, "HI:LPasd:chkrx")) != -1) switch (ch) { case 'H': Hflag = 1; break; + case 'I': + ignoreadd(optarg); + break; case 'L': if (Pflag) usage(); @@ -224,8 +234,13 @@ while ((p = fts_read(fts)) != NULL) { switch (p->fts_info) { case FTS_D: /* Ignore. */ + if (ignorep(p)) + fts_set(fts, p, FTS_SKIP); break; case FTS_DP: + if (ignorep(p)) + break; + p->fts_parent->fts_number += p->fts_number += p->fts_statp->st_blocks; @@ -249,6 +264,9 @@ rval = 1; break; default: + if (ignorep(p)) + break; + if (p->fts_statp->st_nlink > 1 && linkchk(p)) break; @@ -281,6 +299,7 @@ } } + ignoreclean(); exit(rval); } @@ -368,4 +387,47 @@ (void)fprintf(stderr, "usage: du [-H | -L | -P] [-a | -s | -d depth] [-c] [-h | -k] [-x] [file ...]\n"); exit(EX_USAGE); +} + +void +ignoreadd(mask) + char *mask; +{ + char *newmask, **newign; + unsigned l; + + l = strlen(mask) + 1; + if (newmask = (char *) malloc(l + 1), newmask == NULL) + err(1, "can't allocate memory"); + strlcpy(newmask, mask, l+1); /* strcpy? playing it safe.. */ + if (newign = realloc(ignmasks, (igncount + 1) * sizeof(*ignmasks)), + newign == NULL) + err(1, "can't allocate memory"); + newign[igncount++] = newmask; + ignmasks = newign; +} + +void +ignoreclean() +{ + int i; + + for(i = 0; i < igncount; i++) + free(ignmasks[i]); + memset(ignmasks, 0, igncount * sizeof(*ignmasks)); + free(ignmasks); +} + +int +ignorep(ent) + FTSENT *ent; +{ + int i; + + if (igncount == 0) + return 0; + for(i = 0; i < igncount; i++) + if (fnmatch(ignmasks[i], ent->fts_name, 0) != FNM_NOMATCH) + return 1; + return 0; } To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 19: 1:55 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 19:01:53 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mass.osd.bsdi.com (dhcp242.osd.bsdi.com [204.216.28.242]) by hub.freebsd.org (Postfix) with ESMTP id A216A37B404 for ; Wed, 13 Dec 2000 19:01:51 -0800 (PST) Received: from mass.osd.bsdi.com (localhost [127.0.0.1]) by mass.osd.bsdi.com (8.11.1/8.11.1) with ESMTP id eBE3Biu02171; Wed, 13 Dec 2000 19:11:45 -0800 (PST) (envelope-from msmith@mass.osd.bsdi.com) Message-Id: <200012140311.eBE3Biu02171@mass.osd.bsdi.com> X-Mailer: exmh version 2.1.1 10/15/1999 To: Matt Dillon Cc: arch@FreeBSD.ORG Subject: Re: Proposed bus address typedef. In-reply-to: Your message of "Wed, 13 Dec 2000 00:28:24 PST." <200012130828.eBD8SOT81109@earth.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 13 Dec 2000 19:11:43 -0800 From: Mike Smith Sender: msmith@mass.osd.bsdi.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Just in summary; I think I'm hesitant to do anything about this yet. Matt Jacob raises some good points about some of the more complex compound address constructs that I haven't thought through, so I will let the matter mostly lie. I'd like to address something here though: > Pointers on IA32 are still > 32 bits... just 32 bits per address space. Virtual addresses remain 32 bits. Physical addresses (which includes the address of both regions to be mapped into virtual space, as well as the targets of DMA operations) may be larger. > So there is no particular > need for them to be visible outside the device driver core. This isn't correct; the resource manager needs to be able to track these resources, and the busspace and busdma code needs to be able to handle these physical addresses. > Most of > the rest of the kernel that accesses a physical address could just as > well use a page index rather then an actual address. A page index still > fits in an int (i.e. vm_page_t->phys_addr could easily become a page > index, or could become a machine-dependant MMU compatible value if not > a page index). Most of the rest of the kernel either uses virtual > addresses, which are still 32 bits, or block numbers. If the change can > be made without impacting the size of heavily used system structures > I'm all for it. I think your bias is very heavily towards the VM system, in whose domain you are undoubtedly correct. My intention was simply to find a way for the I/O subsystem to deal with a physical address space significantly larger than the virtual address space. In hindsight, this may be too narrow a scope for such a change, however a more complex resolution would require a lot of effort to get "right" and may not be helpful given the small domain of usefulness for this. -- ... every activity meets with opposition, everyone who acts has his rivals and unfortunately opponents also. But not because people want to be opponents, rather because the tasks and relationships force people to take different points of view. [Dr. Fritz Todt] V I C T O R Y N O T V E N G E A N C E To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 21: 7:44 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 21:07:43 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from relay.nuxi.com (nuxi.cs.ucdavis.edu [169.237.7.38]) by hub.freebsd.org (Postfix) with ESMTP id E574637B402 for ; Wed, 13 Dec 2000 21:07:42 -0800 (PST) Received: from dragon.nuxi.com (Ipittythefoolthattrustsident@trang.nuxi.com [209.152.133.57]) by relay.nuxi.com (8.9.3/8.9.3) with ESMTP id VAA74476; Wed, 13 Dec 2000 21:07:41 -0800 (PST) (envelope-from obrien@NUXI.com) Received: (from obrien@localhost) by dragon.nuxi.com (8.11.1/8.11.1) id eBE57e887388; Wed, 13 Dec 2000 21:07:40 -0800 (PST) (envelope-from obrien) Date: Wed, 13 Dec 2000 21:07:40 -0800 From: "David O'Brien" To: Peter Pentchev Cc: arch@FreeBSD.org Subject: Re: add -I ignoremask option to du(1) Message-ID: <20001213210739.A87300@dragon.nuxi.com> Reply-To: arch@FreeBSD.org References: <20001214034803.C575@ringworld.oblivion.bg> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20001214034803.C575@ringworld.oblivion.bg>; from roam@orbitel.bg on Thu, Dec 14, 2000 at 03:48:04AM +0200 X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD group X-Pgp-Rsa-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Rsa-Keyid: 1024/34F9F9D5 Sender: obrien@NUXI.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Thu, Dec 14, 2000 at 03:48:04AM +0200, Peter Pentchev wrote: > Comments? Flames? "Shut-up-already"'s? :) ... > +.Op Fl I Ar mask I only ask, if you've researched this to find the prior art of which option letter is most "compatible" for some definition of the word. Pax(1) would use "-X" for this, and diff(1) both "-x" and "-X". To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 21:47:56 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 21:47:52 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (placeholder-dcat-1076843399.broadbandoffice.net [64.47.83.135]) by hub.freebsd.org (Postfix) with ESMTP id B10B037B698; Wed, 13 Dec 2000 21:47:52 -0800 (PST) Received: (from dillon@localhost) by earth.backplane.com (8.11.1/8.9.3) id eBE5lda91759; Wed, 13 Dec 2000 21:47:39 -0800 (PST) (envelope-from dillon) Date: Wed, 13 Dec 2000 21:47:39 -0800 (PST) From: Matt Dillon Message-Id: <200012140547.eBE5lda91759@earth.backplane.com> To: Brian Somers Cc: Alfred Perlstein , arch@FreeBSD.ORG, net@FreeBSD.ORG, brian@Awfulhak.org Subject: Re: patch to cleanup inflight desciptor handling. References: <200012132124.eBDLOBU68538@hak.lan.Awfulhak.org> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :Hmm, the last time i looked at this, I believe the whole thing was :dealt with by not increasing the file descriptor reference count :when it was put in the message header. If process A closed the :descriptor before process B actually recvmsg()d it, it would be :EBADF. The recvmsg() actually incremented the reference count. : :I always assumed that this was a result of not wanting to have any :funny garbage collecting code ? : :Of course looking at the code now, it increases fp->f_count in :unp_internalize(), so maybe I was on drugs then.... Yes, you were on drugs :-) The moment the descriptor is queued to the socket, the ref count is bumped. It's really a pointer to the underlying file that is queued (and whos ref count is bumped), not the descriptor number. :Assuming I wasn't on drugs (maybe the behaviour was changed - cvs :annotate suggests some activity around March 9, but that was the :file*/int alignment stuff), I think it would be valid to go back :to the old behaviour (not increasing f_count and letting the original :owner actually close the descriptor while f_msgcount != 0). : :Does any of this make sense ? Or am I just describing a different :case (where process B doesn't exit) ? Well, it sort of makes sense, but only in a very twisted fashion :-). sendmsg semantics are that once you queue the descriptor, you can indeed close() it without destroying the queued entity. Think about it... if this were not the case you would be forced to synchronize with the receiving process prior to closing the descriptor on your end to guarentee its validity on the receiving end, which would be a non-trivial piece of userland programming. If we did have those semantics then you could in fact throw the in-flight message away when the sending process went away, but you would have to implement a secondary ref count to prevent the system from throwing away the underlying file until you could clear the message(s). We have *NEVER* had those semantics nor would we ever want those semantics. Even if it were legal (which it isn't), it makes the user-level programming too complex and too fragile. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 21:59: 5 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 21:59:04 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from assaris.sics.se (h122n4fls32o892.telia.com [213.64.47.122]) by hub.freebsd.org (Postfix) with ESMTP id 158D337B400 for ; Wed, 13 Dec 2000 21:59:03 -0800 (PST) Received: (from assar@localhost) by assaris.sics.se (8.9.3/8.9.3) id GAA76442; Thu, 14 Dec 2000 06:59:09 +0100 (CET) (envelope-from assar) Sender: assar@assaris.sics.se From: assar@FreeBSD.ORG To: Peter Pentchev Cc: arch@FreeBSD.ORG Subject: Re: add -I ignoremask option to du(1) References: <20001214034803.C575@ringworld.oblivion.bg> Date: 14 Dec 2000 06:59:08 +0100 In-Reply-To: Peter Pentchev's message of "Thu, 14 Dec 2000 03:48:04 +0200" Message-ID: <5lwvd3sfv7.fsf@assaris.sics.se> Lines: 16 User-Agent: Gnus/5.070098 (Pterodactyl Gnus v0.98) Emacs/20.6 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Peter Pentchev writes: > +void > +ignoreadd(mask) > + char *mask; > +{ > + char *newmask, **newign; > + unsigned l; > + > + l = strlen(mask) + 1; > + if (newmask = (char *) malloc(l + 1), newmask == NULL) > + err(1, "can't allocate memory"); > + strlcpy(newmask, mask, l+1); /* strcpy? playing it safe.. */ Why not newmask = strdup(mask); or even do the strdup:ing below ? /assar To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 22:55:52 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 22:55:51 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (placeholder-dcat-1076843399.broadbandoffice.net [64.47.83.135]) by hub.freebsd.org (Postfix) with ESMTP id 2E9BF37B400 for ; Wed, 13 Dec 2000 22:55:51 -0800 (PST) Received: (from dillon@localhost) by earth.backplane.com (8.11.1/8.9.3) id eBE6tmP92001; Wed, 13 Dec 2000 22:55:48 -0800 (PST) (envelope-from dillon) Date: Wed, 13 Dec 2000 22:55:48 -0800 (PST) From: Matt Dillon Message-Id: <200012140655.eBE6tmP92001@earth.backplane.com> To: Marius Bendiksen Cc: Julian Elischer , Seigo Tanimura , arch@FreeBSD.ORG Subject: Re: Even 1GB KVA is not enough, but we have no more space References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :> THEORETICALLY it should be possible to put the kernel into a differnt :> KV space from the processes and give it 4GB. :> Practically, we'd have to do a lot to do this, and it may effect :> throughout (page tables loading in and out). : :Isn't KV space just system pages (ie kernel adress space)? If so, then making :the system handle a 4GB KV could be done by maintaining a certain portion of :the KV space in the user page directory, and switching to the KV page directory :on demand, at that point removing user pages. : :Marius We're probably not going to ever do anything like this, because manipulating page directory entries or the whole VM space on a user->supervisor or supervisor->user transition flushes the TLB and seriously degrades performance. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 23:27:44 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 23:27:40 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from hand.dotat.at (sfo-gw.covalent.net [207.44.198.62]) by hub.freebsd.org (Postfix) with ESMTP id 2E17537B400; Wed, 13 Dec 2000 23:27:40 -0800 (PST) Received: from fanf by hand.dotat.at with local (Exim 3.15 #3) id 146Snh-00003b-00; Thu, 14 Dec 2000 07:27:37 +0000 Date: Thu, 14 Dec 2000 07:27:37 +0000 From: Tony Finch To: Matt Dillon Cc: Alfred Perlstein , Kirk McKusick , arch@FreeBSD.ORG, net@FreeBSD.ORG Subject: Re: patch to cleanup inflight desciptor handling. Message-ID: <20001214072737.A92196@hand.dotat.at> References: <200012131852.KAA17423@beastie.mckusick.com> <200012132106.eBDL6Sg86570@earth.backplane.com> <20001213141917.Q16205@fw.wintelcom.net> <20001213145341.S16205@fw.wintelcom.net> <20001213153649.T16205@fw.wintelcom.net> <200012140125.eBE1Pbi89951@earth.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: <200012140125.eBE1Pbi89951@earth.backplane.com> Organization: Covalent Technologies, Inc Sender: fanf@dotat.at Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Matt Dillon wrote: > > No waste at all, Alfred, the file descriptor passing code had been > broken for over 10 years precisely because of its complexity. Rewriting > the GC to be more efficient essentially requires using deep graph theory > to locate isolated loops of arbitrary complexity. Most efficient GCers don't involve much graph theory (the notable exception is concurrent collectors); instead they rely on various strategies to drastically reduce the proportion of the arena that they need to examine in most GC runs. In principle mark-sweep collectors are as simple as they get, but unp_gc suffers from the interaction with refcounting. You can use the idea of scanning less of the arena to improve unp_gc as follows. I suggest that you keep two additional lists: one of open unix domain sockets, and one of in-flight sockets. Instead of the existing breadth-first search of the whole file table at the start of unp_gc, it should first clear the mark on each descriptor on the in-flight list, then do a depth-first search of all the descriptors reachable from the unix domain sockets list, marking each one. The loop after the big comment in unp_gc should then scan the in-flight list looking for unmarked descriptors instead of the whole file table. The descriptor freeing loops stay as they are now. I think this should solve the problem at hand, i.e. a lock being held on an important resource while something complicated is being done; instead you would hold locks on two much less important lists (the unix domain list and the in-flight list). > p.s. many object oriented language garbage collectors have the same > problem. create object A, create object B, A references B, B references A, > drop A, drop B. A and B still have references and don't get cleaned up. > Fun. Most modern GCers don't use reference counting partly for that reason, and partly because the overhead of maintaining reference counts is too great. Tony. -- f.a.n.finch fanf@covalent.net dot@dotat.at "Well, as long as they can think we'll have our problems. But those whom we're using cannot think." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Dec 13 23:51:31 2000 From owner-freebsd-arch@FreeBSD.ORG Wed Dec 13 23:51:29 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from ringworld.nanolink.com (unknown [195.24.48.13]) by hub.freebsd.org (Postfix) with SMTP id 1979B37B400 for ; Wed, 13 Dec 2000 23:51:28 -0800 (PST) Received: (qmail 4882 invoked by uid 1000); 14 Dec 2000 07:50:34 -0000 Date: Thu, 14 Dec 2000 09:50:34 +0200 From: Peter Pentchev To: arch@FreeBSD.org Subject: Re: add -I ignoremask option to du(1) Message-ID: <20001214095033.D575@ringworld.oblivion.bg> Mail-Followup-To: arch@FreeBSD.org References: <20001214034803.C575@ringworld.oblivion.bg> <20001213210739.A87300@dragon.nuxi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20001213210739.A87300@dragon.nuxi.com>; from obrien@FreeBSD.org on Wed, Dec 13, 2000 at 09:07:40PM -0800 Sender: roam@ringworld.nanolink.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Wed, Dec 13, 2000 at 09:07:40PM -0800, David O'Brien wrote: > On Thu, Dec 14, 2000 at 03:48:04AM +0200, Peter Pentchev wrote: > > Comments? Flames? "Shut-up-already"'s? :) > ... > > +.Op Fl I Ar mask > > I only ask, if you've researched this to find the prior art of which > option letter is most "compatible" for some definition of the word. > Pax(1) would use "-X" for this, and diff(1) both "-x" and "-X". Well, "-x" was what I thought of first, but it was already taken, and with a longtime Unix history, too. "-I" is what cvs(1) uses. I could make it "-i mask" and add "-I maskfile" to match diff(1) though; or would "-I mask", "-X maskfile" be more appropriate? Or drop the maskfile idea at all, and use "-X".. I was also thinking of adding something like -i or -U to specify an 'unignore' mask - names to process even if they match an -I option. diff(1) does not have this at all. cvs(1) uses "-I !mask", but this could make things hard for people used to cvs's also accepting "-I ! mask" - getopt() would choke on that. G'luck, Peter PS. Hm a look at pax(1) and its history suggests that its "-X" has been used for 'no filesystem traversal' ever since its conception, or at least importing into FreeBSD.. This sentence no verb. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Dec 14 0: 1: 0 2000 From owner-freebsd-arch@FreeBSD.ORG Thu Dec 14 00:00:58 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from ringworld.nanolink.com (unknown [195.24.48.13]) by hub.freebsd.org (Postfix) with SMTP id A740B37B400 for ; Thu, 14 Dec 2000 00:00:56 -0800 (PST) Received: (qmail 5002 invoked by uid 1000); 14 Dec 2000 08:00:07 -0000 Date: Thu, 14 Dec 2000 10:00:06 +0200 From: Peter Pentchev To: arch@FreeBSD.org Subject: Re: add -I ignoremask option to du(1) Message-ID: <20001214100006.E575@ringworld.oblivion.bg> Mail-Followup-To: arch@FreeBSD.org References: <20001214034803.C575@ringworld.oblivion.bg> <5lwvd3sfv7.fsf@assaris.sics.se> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <5lwvd3sfv7.fsf@assaris.sics.se>; from assar@FreeBSD.ORG on Thu, Dec 14, 2000 at 06:59:08AM +0100 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Thu, Dec 14, 2000 at 06:59:08AM +0100, assar@FreeBSD.ORG wrote: > Peter Pentchev writes: > > +void > > +ignoreadd(mask) > > + char *mask; > > +{ > > + char *newmask, **newign; > > + unsigned l; > > + > > + l = strlen(mask) + 1; > > + if (newmask = (char *) malloc(l + 1), newmask == NULL) > > + err(1, "can't allocate memory"); > > + strlcpy(newmask, mask, l+1); /* strcpy? playing it safe.. */ > > Why not newmask = strdup(mask); or even do the strdup:ing below ? Ah.. my oversight, thanks! I'll submit a new diff as soon as there's some consensus on which option letter(s) should be used, and should there be an 'unignore' option, too (see my reply to David O'Brien's message in the thread). G'luck, Peter -- I've heard that this sentence is a rumor. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Dec 14 3:36:36 2000 From owner-freebsd-arch@FreeBSD.ORG Thu Dec 14 03:36:31 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 719BC37B400 for ; Thu, 14 Dec 2000 03:36:29 -0800 (PST) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id MAA71424; Thu, 14 Dec 2000 12:36:27 +0100 (CET) (envelope-from des@ofug.org) Sender: des@ofug.org X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: arch@freebsd.org Subject: Re: retiring kernfs References: <20001213151832.A84135@dragon.nuxi.com> From: Dag-Erling Smorgrav Date: 14 Dec 2000 12:36:27 +0100 In-Reply-To: "David O'Brien"'s message of "Wed, 13 Dec 2000 15:18:32 -0800" Message-ID: Lines: 18 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG "David O'Brien" writes: > On Wed, Dec 13, 2000 at 05:02:41PM +0100, Dag-Erling Smorgrav wrote: > > Any serious objections to retiring the (long obsolete) kernfs > > pseudo-file system? > Can you give a reason for your wanting to remove something that compiles > and works? It's superfluous (all the information it provides is available through sysctl(8)), deprecated (sources with better memory than me tell me that it's been recommended against since 2.1.0), and a potential security hole as it hasn't been maintained since 1996 (except for being kept compilable across kernel API changes), and I can practically guarantee (without inspecting the source code) that it does Very Wrong Things with e.g. prisons. Oh, and nobody uses it. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Dec 14 4:16:24 2000 From owner-freebsd-arch@FreeBSD.ORG Thu Dec 14 04:16:21 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mail-relay.eunet.no (mail-relay.eunet.no [193.71.71.242]) by hub.freebsd.org (Postfix) with ESMTP id D11A337B404 for ; Thu, 14 Dec 2000 04:16:14 -0800 (PST) Received: from login-1.eunet.no (login-1.eunet.no [193.75.110.2]) by mail-relay.eunet.no (8.9.3/8.9.3/GN) with ESMTP id NAA14576; Thu, 14 Dec 2000 13:16:12 +0100 (CET) (envelope-from mbendiks@eunet.no) Received: from localhost (mbendiks@localhost) by login-1.eunet.no (8.9.3/8.8.8) with ESMTP id NAA43785; Thu, 14 Dec 2000 13:16:12 +0100 (CET) (envelope-from mbendiks@eunet.no) X-Authentication-Warning: login-1.eunet.no: mbendiks owned process doing -bs Date: Thu, 14 Dec 2000 13:16:09 +0100 (CET) From: Marius Bendiksen To: Matt Dillon Cc: Julian Elischer , Seigo Tanimura , arch@FreeBSD.ORG Subject: Re: Even 1GB KVA is not enough, but we have no more space In-Reply-To: <200012140655.eBE6tmP92001@earth.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > We're probably not going to ever do anything like this, because > manipulating page directory entries or the whole VM space on a > user->supervisor or supervisor->user transition flushes > the TLB and seriously degrades performance. Hence my reference to doing this on-demand. If the kernel could be taught about virtual/physical paging issues (ie. could be aware of whether objects are currently in the KV space), then you could avoid this except in the few cases where you hit the 1G limit. Marius To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Dec 14 5:23:20 2000 From owner-freebsd-arch@FreeBSD.ORG Thu Dec 14 05:23:19 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from lists01.iafrica.com (lists01.iafrica.com [196.7.0.141]) by hub.freebsd.org (Postfix) with ESMTP id 0D7D737B400 for ; Thu, 14 Dec 2000 05:23:17 -0800 (PST) Received: from nwl.fw.uunet.co.za ([196.31.2.162]) by lists01.iafrica.com with esmtp (Exim 3.12 #2) id 146YLo-0003YU-00; Thu, 14 Dec 2000 15:23:12 +0200 Received: (from nobody@localhost) by nwl.fw.uunet.co.za (8.8.8/8.6.9) id PAA22877; Thu, 14 Dec 2000 15:23:09 +0200 (SAST) Received: by nwl.fw.uunet.co.za via recvmail id 22785; Thu Dec 14 15:22:37 2000 Received: from sheldonh (helo=axl.fw.uunet.co.za) by axl.fw.uunet.co.za with local-esmtp (Exim 3.16 #1) id 146YLE-000EOA-00; Thu, 14 Dec 2000 15:22:36 +0200 From: Sheldon Hearn To: Peter Pentchev Cc: arch@freebsd.org Subject: Re: add -I ignoremask option to du(1) In-reply-to: Your message of "Thu, 14 Dec 2000 03:48:04 +0200." <20001214034803.C575@ringworld.oblivion.bg> Date: Thu, 14 Dec 2000 15:22:36 +0200 Message-ID: <55313.976800156@axl.fw.uunet.co.za> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Thu, 14 Dec 2000 03:48:04 +0200, Peter Pentchev wrote: > Is there a reason no one has done this yet? :) I think we're getting totally out of hand with respect to teaching utilities how to do each others' jobs for one another. Recursion, exclusion and pattern matching are all creeping into utilities that once did one job well. These things are all handled very well by find and the shell. So the only question is whether we shrug and continue down this road, or limit ourselves to only those silly enhancements that we deem necessary for compatibility with other platforms. Personally, I prefer the latter. Ciao, Sheldon. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Dec 14 6:18:37 2000 From owner-freebsd-arch@FreeBSD.ORG Thu Dec 14 06:18:32 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mail.interware.hu (mail.interware.hu [195.70.32.130]) by hub.freebsd.org (Postfix) with ESMTP id B68C737B404; Thu, 14 Dec 2000 06:18:30 -0800 (PST) Received: from kairo-18.budapest.interware.hu ([195.70.50.82] helo=elischer.org) by mail.interware.hu with esmtp (Exim 3.16 #1 (Debian)) id 146ZD8-0002OO-00; Thu, 14 Dec 2000 15:18:19 +0100 Sender: julian@FreeBSD.ORG Message-ID: <3A38D684.F241D6B9@elischer.org> Date: Thu, 14 Dec 2000 06:17:40 -0800 From: Julian Elischer X-Mailer: Mozilla 4.7 [en] (X11; U; FreeBSD 5.0-CURRENT i386) X-Accept-Language: en, hu MIME-Version: 1.0 To: Alfred Perlstein Cc: Matt Dillon , Kirk McKusick , arch@FreeBSD.ORG, net@FreeBSD.ORG Subject: Re: patch to cleanup inflight desciptor handling. References: <200012131852.KAA17423@beastie.mckusick.com> <200012132106.eBDL6Sg86570@earth.backplane.com> <20001213141917.Q16205@fw.wintelcom.net> <20001213145341.S16205@fw.wintelcom.net> Content-Type: text/plain; charset=iso-8859-15 Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Alfred Perlstein wrote: > > * Alfred Perlstein [001213 14:20] wrote: > > * Matt Dillon [001213 13:07] wrote: > > > :I believe that your changes have been sorely needed for many > > > :years. While I would like to see regular mbufs given a callback > > > :mechanism, your present approach of using an mbuf cluster > > > :solves 90% of the problem. > > > : > > > : Kirk McKusick > > > > > > ... Aflred, be careful that you don't break things we only just fixed > > > last year. The descriptor passing code has been broken for many years. > > > > > > I think the reason we have to scan the descriptor list is related to > > > locating isolated self-referential 'loops' with descriptor passing and > > > unix domain sockets and closing them. e.g. when you pass a descriptor > > > for a unix-domain socket through a unix-domain socket, it is possible > > > for the socket descriptors to reference each other and thus never have > > > their ref count drop to 0 even when all associated processes have > > > close()'d. This happens all the time. Be sure you don't break the > > > fix that solves that particular problem. > > > > Ok, I'll see if that can happen. Basically since the reference > > never goes to zero on the socket, the buffers are never forced to > > be flushed/cleared and the mbuf will then never be free'd resulting > > it it leaking itself. Basically a socket hanging there with an > > mbuf referencing itself. > > > > I wonder if Linux fixed/has this problem. > > Ok, my patch has this problem: > > void > parent(int con) > { > int fd; > > fd = open("/tmp/wank", O_RDONLY); > send_fd_withdata(con, con, "wank", 4); > sleep (5); > exit(1); > > } > > void > child(int con) > { > int fd, error; > char buf[100]; > > sleep(5); > get_fd_withdata(con, &fd, buf, sizeof(buf)); > send_fd_withdata(con, fd, "foo", 3); > exit(1); > buf[4] = '\0'; > printf("%s\n", buf); > if ((error = read(fd, buf, sizeof(buf))) < 0) > perror("read"); > buf[sizeof(buf)-1] = '\0'; > printf("%s\n", buf); > > > } > > This causes a leak, I think the trick is to just always call sorflush() > when the pcb is free'd. that's what I was trying to point out to you.... It get's more complicate when you have 2 pipes, and each has the filedescriptors for the other one in it. > > Looking at linux they still are using gc. I'll give this a lot > more thought before resubmitting this idea. > > sorry, > -- > -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] > "I have the heart of a child; I keep it in a jar on my desk." > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-net" in the body of the message -- __--_|\ Julian Elischer / \ julian@elischer.org ( OZ ) World tour 2000 ---> X_.---._/ presently in: Budapest v To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Dec 14 6:20:14 2000 From owner-freebsd-arch@FreeBSD.ORG Thu Dec 14 06:20:10 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from ringworld.nanolink.com (ringworld.nanolink.com [195.24.48.189]) by hub.freebsd.org (Postfix) with SMTP id 53FE137B400 for ; Thu, 14 Dec 2000 06:20:08 -0800 (PST) Received: (qmail 2495 invoked by uid 1000); 14 Dec 2000 14:19:07 -0000 Date: Thu, 14 Dec 2000 16:19:07 +0200 From: Peter Pentchev To: Sheldon Hearn Cc: arch@freebsd.org Subject: Re: add -I ignoremask option to du(1) Message-ID: <20001214161907.D369@ringworld.oblivion.bg> Mail-Followup-To: Sheldon Hearn , arch@freebsd.org References: <20001214034803.C575@ringworld.oblivion.bg> <55313.976800156@axl.fw.uunet.co.za> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <55313.976800156@axl.fw.uunet.co.za>; from sheldonh@uunet.co.za on Thu, Dec 14, 2000 at 03:22:36PM +0200 Sender: roam@ringworld.nanolink.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Thu, Dec 14, 2000 at 03:22:36PM +0200, Sheldon Hearn wrote: > > > On Thu, 14 Dec 2000 03:48:04 +0200, Peter Pentchev wrote: > > > Is there a reason no one has done this yet? :) > > I think we're getting totally out of hand with respect to teaching > utilities how to do each others' jobs for one another. I'm aware that danger lieth along that path. However, see below. > Recursion, exclusion and pattern matching are all creeping into > utilities that once did one job well. These things are all handled very > well by find and the shell. In general, I agree. However, consider the problem at hand: getting information about the filesystem usage of files and directories down a tree, excluding certain files. Let's consider an example when all files and directories matching either of '*.c' and 'CVS' patterns need to be excluded. I mentioned in my first email that this could be done through some find(1) hackery. I can see several possible problems there - feel free to correct my admittedly not-thoroughly-informed opinions on any of those. 1. Non-obvious syntax. OK, I'll admit that this was the main reason - I was almost stumped when I had to find out how to do it :) And I am still somewhat doubtful as to whether I have really found a way. The best I've come up with so far is: find path ( ( -name *.c -or -name CVS ) -and -prune ) -or -print or -ls instead of -print. With -print, du(1) descends directories, showing space allocated to ignored files, too. With -ls, I can easily awk out the filesize and name, but I'd have to do additional scripting to keep track of directory tree totals. 2. With find(1) and du(1) combined, there might be a heavy duplication of effort involved - traversing a directory tree twice. This does not happen with find -ls | awk, but see above for the directory tree totals :( 3. I'd think the fts routines that du(1) uses could be more efficient than having find(1) pass its output through (possibly blocking) pipes to whatever happens to be on the other end. Note that this is a *highly* uninformed opinion though, and I should probably not be mentioning it at all :) > So the only question is whether we shrug and continue down this road, or > limit ourselves to only those silly enhancements that we deem necessary > for compatibility with other platforms. As I mentioned above, I agree in principle; just not in this particular case. This might very well turn into creeping featurism (which I know will not be allowed :), but I like telling myself that it's not all that bad :) G'luck, Peter -- I am the thought you are now thinking. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Dec 14 9:33:24 2000 From owner-freebsd-arch@FreeBSD.ORG Thu Dec 14 09:33:22 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from homer.softweyr.com (bsdconspiracy.net [208.187.122.220]) by hub.freebsd.org (Postfix) with ESMTP id 4E14637B400 for ; Thu, 14 Dec 2000 09:33:11 -0800 (PST) Received: from [127.0.0.1] (helo=softweyr.com ident=Fools trust ident!) by homer.softweyr.com with esmtp (Exim 3.16 #1) id 146c5H-0000Jk-00; Thu, 14 Dec 2000 10:22:24 -0700 Sender: wes@FreeBSD.ORG Message-ID: <3A3901CF.7B09AE6F@softweyr.com> Date: Thu, 14 Dec 2000 10:22:23 -0700 From: Wes Peters Organization: Softweyr LLC X-Mailer: Mozilla 4.75 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Sheldon Hearn Cc: Peter Pentchev , arch@freebsd.org Subject: Re: add -I ignoremask option to du(1) References: <55313.976800156@axl.fw.uunet.co.za> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Sheldon Hearn wrote: > > On Thu, 14 Dec 2000 03:48:04 +0200, Peter Pentchev wrote: > > > Is there a reason no one has done this yet? :) > > I think we're getting totally out of hand with respect to teaching > utilities how to do each others' jobs for one another. > > Recursion, exclusion and pattern matching are all creeping into > utilities that once did one job well. These things are all handled very > well by find and the shell. > > So the only question is whether we shrug and continue down this road, or > limit ourselves to only those silly enhancements that we deem necessary > for compatibility with other platforms. > > Personally, I prefer the latter. UNIX was doomed the moment somebody added sorting code to ls(1). Resist feeping creaturism. -- "Where am I, and what am I doing in this handbasket?" Wes Peters Softweyr LLC wes@softweyr.com http://softweyr.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Dec 14 10:10:52 2000 From owner-freebsd-arch@FreeBSD.ORG Thu Dec 14 10:10:51 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from puck.firepipe.net (poynting.physics.purdue.edu [128.210.146.58]) by hub.freebsd.org (Postfix) with ESMTP id 1659337B400 for ; Thu, 14 Dec 2000 10:10:51 -0800 (PST) Received: by puck.firepipe.net (Postfix, from userid 1000) id 320CC18DB; Thu, 14 Dec 2000 13:10:50 -0500 (EST) Date: Thu, 14 Dec 2000 13:10:50 -0500 From: Will Andrews To: Peter Pentchev Cc: arch@FreeBSD.ORG Subject: Re: add -I ignoremask option to du(1) Message-ID: <20001214131050.L1873@puck.firepipe.net> Reply-To: Will Andrews Mail-Followup-To: Will Andrews , Peter Pentchev , arch@FreeBSD.ORG References: <20001214034803.C575@ringworld.oblivion.bg> <5lwvd3sfv7.fsf@assaris.sics.se> <20001214100006.E575@ringworld.oblivion.bg> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20001214100006.E575@ringworld.oblivion.bg>; from roam@orbitel.bg on Thu, Dec 14, 2000 at 10:00:06AM +0200 X-Operating-System: FreeBSD 4.2-STABLE i386 Sender: will@puck.firepipe.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Thu, Dec 14, 2000 at 10:00:06AM +0200, Peter Pentchev wrote: > Ah.. my oversight, thanks! I'll submit a new diff as soon as there's > some consensus on which option letter(s) should be used, and should > there be an 'unignore' option, too (see my reply to David O'Brien's > message in the thread). Don't forget to check the results of the strdup.. if it is NULL you should err out somehow. -- wca To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Dec 14 10:29:59 2000 From owner-freebsd-arch@FreeBSD.ORG Thu Dec 14 10:29:50 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from hand.dotat.at (sfo-gw.covalent.net [207.44.198.62]) by hub.freebsd.org (Postfix) with ESMTP id 6E97937B402; Thu, 14 Dec 2000 10:29:49 -0800 (PST) Received: from fanf by hand.dotat.at with local (Exim 3.15 #3) id 146d8N-000Ll1-00; Thu, 14 Dec 2000 18:29:39 +0000 Date: Thu, 14 Dec 2000 18:29:39 +0000 From: Tony Finch To: Matt Dillon Cc: Alfred Perlstein , Kirk McKusick , arch@FreeBSD.ORG, net@FreeBSD.ORG Subject: Re: patch to cleanup inflight desciptor handling. Message-ID: <20001214182939.B92196@hand.dotat.at> References: <200012131852.KAA17423@beastie.mckusick.com> <200012132106.eBDL6Sg86570@earth.backplane.com> <20001213141917.Q16205@fw.wintelcom.net> <20001213145341.S16205@fw.wintelcom.net> <20001213153649.T16205@fw.wintelcom.net> <200012140125.eBE1Pbi89951@earth.backplane.com> <20001214072737.A92196@hand.dotat.at> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: <20001214072737.A92196@hand.dotat.at> Organization: Covalent Technologies, Inc Sender: fanf@dotat.at Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Tony Finch wrote: > >Instead of the existing breadth-first search of the whole file table >at the start of unp_gc, it should first clear the mark on each >descriptor on the in-flight list, then do a depth-first search of all >the descriptors reachable from the unix domain sockets list, marking >each one. Actually that isn't right. The order of the search doesn't matter; what is important is that the trick that the existing code uses (with the FDEFER flag and unp_defer) to perform the scan using no additional space won't work: you must use a more conventional scanning algorithm instead. But anyway, I have found a much better algorithm in the garbage collection book [1]. The algorithm you want [2] isn't directly described in the book; instead they explain a lazy variant [3] which has some additional mechanism to delay running the GC in order to amortise the cost, which is not something we want to do here. AFAICT though the core is the same, so I'll describe it in pseudocode below. It scans the theoretical minimum number of file descriptors :-) When closing a unix domain descriptor with descriptors in flight, do this: count_local_refs(fp); find_external_refs(fp); close_unreffed(fp); count_local_refs counts all the references within the graph of fds reachable from the fd we are closing, taking into account loops: [The foreach syntax is an abbreviation for code like lines 1120 to 1150 of uipc_usrreq.c.] count_local_refs(struct file *fp) { if (!(fp->f_flag & FMAYBEFREE)) { fp->f_flag |= FMAYBEFREE; foreach child (in_flight(fp)) { child->f_count--; count_local_refs(child); } } } find_external_refs finds subgraphs that are reachable externally and marks them as such: find_external_refs(struct file *fp) { if (fp->f_flag & FMAYBEFREE) if (fp->f_count) mark_external_refs(fp); else { fp->f_flag |= FREALLYFREE; foreach child (in_flight(fp)) find_external_refs(child); } } mark_external_refs(struct file *fp) { fp->f_flag &= ~FMAYBEFREE; foreach child (in_flight(fp)) { child->f_count++; if (fp->f_flag & FMAYBEFREE) mark_external_refs(child); } } close_unreffed then closes all the unreferenced descriptors. close_unreffed(struct file *fp) { if (fp->f_flag & FREALLYFREE) { fp->f_flag &= ~(FMAYBEFREE|FREALLYFREE); foreach child (in_flight(fp)) close_unreffed(child); /* XXX: here we probably need to set fp->f_count to 1 to avoid a panic */ close(fp); } } There are a few problems with this as I have described it, but I think they are reasonably easily fixed: * The recursive algorithms may overflow the kernel stack so they should be replaced with iterative ones. * The way it fools around with refcounts will cause trouble if any other processes do things with the descriptors that the algorithm is dealing with; perhaps the first pass through the graph should lock all the descriptors to avoid this. * It will fail if it is run concurrently on two overlapping graphs; there should be a mutex around the unix domain socket close code to prevent this happening. * The unix domain socket close code must detect when it is called recursively by this code (i.e. fp->f_flag & REALLYFREE) and avoid calling the gc again recursively. Here are the references. According to someone at the San Francisco Public Library there aren't any copies of Information Processing Letters in the Bay Area, which I find hard to believe since it implies that there aren't any good computer science or academic periodical libraries here. [1] R. E. Jones, R. D. Lins: "Garbage Collection"; Wiley (Chichester, 1996). [2] A. D. Martinez, R. Wachenauzer, R. D. Lins: "Cyclic Reference Counting with Local Mark-Scan"; Information Processing Letters 34:31-35, 1990 [3] R. D. Lins: "Cyclic Reference Counting with Local Mark-Scan"; Information Processing Letters 44(4):215-220, 1992; University of Kent at Canterbury Computing Laboratory Technical Report 75, July 1990. [4] R. D. Lins, M. A. Vasques: University of Kent at Canterbury Computing Laboratory Technical Report 92, August 1991. Tony. -- f.a.n.finch fanf@covalent.net dot@dotat.at "Because all you of Earth are idiots!" To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Dec 14 12:24:20 2000 From owner-freebsd-arch@FreeBSD.ORG Thu Dec 14 12:24:18 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.66]) by hub.freebsd.org (Postfix) with ESMTP id 1188337B400 for ; Thu, 14 Dec 2000 12:24:13 -0800 (PST) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.11.0/8.11.0) with ESMTP id eBEKO3s87527 for ; Thu, 14 Dec 2000 13:24:10 -0700 (MST) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id NAA49965 for ; Thu, 14 Dec 2000 13:24:02 -0700 (MST) Message-Id: <200012142024.NAA49965@harmony.village.org> To: arch@freebsd.org Subject: Minor proposed change to suspend/resume Date: Thu, 14 Dec 2000 13:24:02 -0700 From: Warner Losh Sender: imp@harmony.village.org Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In talking about some improvements to cardbus that Justin is making, he's managed to convince me that allowing for suspend/resume in pccard/cardbus is a good idea. Between us we've hammered out a protocol that should work and will degenerate to the old behavior. First, the suspend routine will be responsible for putting the device into a state that can be suspended. This is as it is today. If it cannot do so because the device is busy or otherwise temporary unable to suspend, it should return EBUSY. If it cannot do so because it has no way of detecting the device is the same on resume as before, then it should return ENODEV. For devices that return ENODEV, the parent bus will detach that child before the suspend. For resume, the bus will make its best effort to see if the device is the same as it was when it left. This check is intended to be really simple (eg make sure that the vendor, device, subvendor and subdevice are the same as before on PCI). If the bus thinks the card might be the same one, it will call the device's resume routine. The device will check to make sure it is the "same one" that was there before (eg for NIC cards that it has the same MAC address, for sio maybe that it is still a 16550). If not, it returns an error (I suggest ENXIO). If successful, it will then generally need to initialize its hardware since suspend/resume will generally reset the hardware. This is bus specific, but is a common trait of all busses current extant, except maybe usb, so it should be viewed as advice. If the resume routine returns an error, the parent bus will detach the device. Parent busses need not participate in this protocol if they don't want to. It is only for those buses that have removable parts. pccard and cardbus are top of the list, but PCI might be on the list depending on how CompactPCI's hot plug stuff is integrated into the kernel in the future. There's also mini-pci in notebooks. Suspend to disk, pop out the minipci card and resume is physically possible. The above proposal is a minor change to the current way that we're doing suspend and resume. Right now it is simpler: default to a routine that always returns 0 (this won't change), non-zero on suspend means abort the suspend and call everybody's resume routine who has been suspended so far. I'm not sure what errors on resume are supposed to mean, but the current bus_generic_resume does nothing on errors. I have added conventions: The return value from suspend can be meaningful. The failure of resume means detach the device and reprobe that slot. Comments? Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Dec 14 13:55:25 2000 From owner-freebsd-arch@FreeBSD.ORG Thu Dec 14 13:55:23 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 2D24937B400 for ; Thu, 14 Dec 2000 13:55:23 -0800 (PST) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by pike.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id eBELtCE87174 for ; Thu, 14 Dec 2000 13:55:12 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 Date: Thu, 14 Dec 2000 13:55:36 -0800 (PST) From: John Baldwin To: arch@FreeBSD.org Subject: kthread API Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Hey all, I'd like to make some of the kernel API's a little more consistent as far as in their naming. The first one I'd like to tackle is the kthread API. Specifically, I'd like to stick the whole API under a kthread_* API. This would result in the following: Old Name New Name -------- -------- kproc_start kproc_start shutdown_kproc kproc_shutdown kthread_create kthread_create kthread_exit kthread_exit resume_kproc kthread_resume suspend_kproc kthread_suspend kproc_suspend_loop kthread_suspend_check The description of all of these can be found in kthread(9). I kept the kproc_ prefix for start and shutdown because those functions are wrappers used specifically with the kernel daemons (pagedaemon, vmdaemon, etc.) whereas the other functions may operate on any kernel thread. I'm not opposed to sticking those under kthread_* if that is what everyone else wants however. After kthread_*, I plan to probably stick the software interrupt API under a swi_* namespace. If I get this committed, I'll fix update the manpage as well. The patch can be found at http://www.FreeBSD.org/~jhb/patches/kthread.patch. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Dec 14 14: 9:24 2000 From owner-freebsd-arch@FreeBSD.ORG Thu Dec 14 14:09:20 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from hand.dotat.at (sfo-gw.covalent.net [207.44.198.62]) by hub.freebsd.org (Postfix) with ESMTP id 30D7D37B402; Thu, 14 Dec 2000 14:09:19 -0800 (PST) Received: from fanf by hand.dotat.at with local (Exim 3.15 #3) id 146gYl-0003CX-00; Thu, 14 Dec 2000 22:09:07 +0000 Date: Thu, 14 Dec 2000 22:09:07 +0000 From: Tony Finch To: Matt Dillon Cc: Alfred Perlstein , Kirk McKusick , arch@FreeBSD.ORG, net@FreeBSD.ORG Subject: Re: patch to cleanup inflight desciptor handling. Message-ID: <20001214220907.I92196@hand.dotat.at> References: <200012131852.KAA17423@beastie.mckusick.com> <200012132106.eBDL6Sg86570@earth.backplane.com> <20001213141917.Q16205@fw.wintelcom.net> <20001213145341.S16205@fw.wintelcom.net> <20001213153649.T16205@fw.wintelcom.net> <200012140125.eBE1Pbi89951@earth.backplane.com> <20001214072737.A92196@hand.dotat.at> <20001214182939.B92196@hand.dotat.at> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: <20001214182939.B92196@hand.dotat.at> Organization: Covalent Technologies, Inc Sender: fanf@dotat.at Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Tony Finch wrote: > >R. D. Lins: "Cyclic Reference Counting with Local Mark-Scan"; >Information Processing Letters 44(4):215-220, 1992; University of Kent >at Canterbury Computing Laboratory Technical Report 75, July 1990. http://citeseer.nj.nec.com/lins90cyclic.html Tony. -- f.a.n.finch fanf@covalent.net dot@dotat.at "Then they attacked a town. A small town, I'll admit. But nevertheless a town of people. People who died." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Dec 14 15: 8:50 2000 From owner-freebsd-arch@FreeBSD.ORG Thu Dec 14 15:08:46 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from smtp02.primenet.com (smtp02.primenet.com [206.165.6.132]) by hub.freebsd.org (Postfix) with ESMTP id 4A34E37B400; Thu, 14 Dec 2000 15:08:46 -0800 (PST) Received: (from daemon@localhost) by smtp02.primenet.com (8.9.3/8.9.3) id QAA01598; Thu, 14 Dec 2000 16:03:58 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp02.primenet.com, id smtpdAAAVva4ed; Thu Dec 14 16:03:52 2000 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id QAA15345; Thu, 14 Dec 2000 16:08:36 -0700 (MST) From: Terry Lambert Message-Id: <200012142308.QAA15345@usr08.primenet.com> Subject: Re: patch to cleanup inflight desciptor handling. To: bright@wintelcom.net (Alfred Perlstein) Date: Thu, 14 Dec 2000 23:08:36 +0000 (GMT) Cc: dillon@earth.backplane.com (Matt Dillon), mckusick@mckusick.com (Kirk McKusick), arch@FreeBSD.ORG, net@FreeBSD.ORG In-Reply-To: <20001213141917.Q16205@fw.wintelcom.net> from "Alfred Perlstein" at Dec 13, 2000 02:19:18 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: tlambert@usr08.primenet.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Ok, I'll see if that can happen. Basically since the reference > never goes to zero on the socket, the buffers are never forced to > be flushed/cleared and the mbuf will then never be free'd resulting > it it leaking itself. Basically a socket hanging there with an > mbuf referencing itself. > > I wonder if Linux fixed/has this problem. SVR4 and Solaris avoid the problem entirely by ensuring that each reference to a vnode pointer counts as an "open", and the vnode can not be discarded until a 1->0 reference count change (grep for VHOLD/VRELE/VREF in the Solaris headers). This also makes it easier to do kernel level file I/O, since all you need is a vp that has a reference added to it, and you aren't stuck needing a process so that you can have an open file table so you can hold a descriptor reference. A trade off to this approach is that you have to supply the offset on reads and writes, instead of assuming advancement of an automatically maintained f_offset value. That's actually a positive, in my book. So when you are passing the fd, you are actually passing a vp; this is not aproblem, since the semantics are equivalent to "dup", not "dup2", so you don't get to pick your target descriptor, anyway. One real PITA that you will see, if you get into this code at all, is that the lock references don't get reassigned, so they all go away ("POSIX me _harder_!") when the passer does the close, which is probably the wrong thing. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Dec 14 15:16:41 2000 From owner-freebsd-arch@FreeBSD.ORG Thu Dec 14 15:16:37 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from hand.dotat.at (sfo-gw.covalent.net [207.44.198.62]) by hub.freebsd.org (Postfix) with ESMTP id 74DAE37B400; Thu, 14 Dec 2000 15:16:37 -0800 (PST) Received: from fanf by hand.dotat.at with local (Exim 3.15 #3) id 146hbx-0005Rn-00; Thu, 14 Dec 2000 23:16:29 +0000 Date: Thu, 14 Dec 2000 23:16:29 +0000 From: Tony Finch To: Terry Lambert Cc: Alfred Perlstein , Matt Dillon , Kirk McKusick , arch@FreeBSD.ORG, net@FreeBSD.ORG Subject: Re: patch to cleanup inflight desciptor handling. Message-ID: <20001214231629.K92196@hand.dotat.at> References: <20001213141917.Q16205@fw.wintelcom.net> <200012142308.QAA15345@usr08.primenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: <200012142308.QAA15345@usr08.primenet.com> Organization: Covalent Technologies, Inc Sender: fanf@dotat.at Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Terry Lambert wrote: > >SVR4 and Solaris avoid the problem entirely by ensuring that >each reference to a vnode pointer counts as an "open", and >the vnode can not be discarded until a 1->0 reference count >change (grep for VHOLD/VRELE/VREF in the Solaris headers). FreeBSD does this too. It doesn't solve the circular reference problem, though (hence the AI Koan). Tony. -- f.a.n.finch fanf@covalent.net dot@dotat.at "Then they attacked a town. A small town, I'll admit. But nevertheless a town of people. People who died." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Dec 14 15:30:11 2000 From owner-freebsd-arch@FreeBSD.ORG Thu Dec 14 15:30:09 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mass.osd.bsdi.com (adsl-63-202-177-71.dsl.snfc21.pacbell.net [63.202.177.71]) by hub.freebsd.org (Postfix) with ESMTP id D33BE37B698; Thu, 14 Dec 2000 15:30:08 -0800 (PST) Received: from mass.osd.bsdi.com (localhost [127.0.0.1]) by mass.osd.bsdi.com (8.11.1/8.11.1) with ESMTP id eBENeJo01069; Thu, 14 Dec 2000 15:40:19 -0800 (PST) (envelope-from msmith@mass.osd.bsdi.com) Message-Id: <200012142340.eBENeJo01069@mass.osd.bsdi.com> X-Mailer: exmh version 2.1.1 10/15/1999 To: John Baldwin Cc: arch@FreeBSD.org Subject: Re: kthread API In-reply-to: Your message of "Thu, 14 Dec 2000 13:55:36 PST." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 14 Dec 2000 15:40:19 -0800 From: Mike Smith Sender: msmith@mass.osd.bsdi.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > I'd like to make some of the kernel API's a little more consistent as > far as in their naming. The first one I'd like to tackle is the kthread > API. Specifically, I'd like to stick the whole API under a kthread_* API. I like this. 8) -- ... every activity meets with opposition, everyone who acts has his rivals and unfortunately opponents also. But not because people want to be opponents, rather because the tasks and relationships force people to take different points of view. [Dr. Fritz Todt] V I C T O R Y N O T V E N G E A N C E To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Dec 14 15:37:31 2000 From owner-freebsd-arch@FreeBSD.ORG Thu Dec 14 15:37:27 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from smtp05.primenet.com (smtp05.primenet.com [206.165.6.135]) by hub.freebsd.org (Postfix) with ESMTP id 356D237B400; Thu, 14 Dec 2000 15:37:27 -0800 (PST) Received: (from daemon@localhost) by smtp05.primenet.com (8.9.3/8.9.3) id QAA23890; Thu, 14 Dec 2000 16:33:51 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp05.primenet.com, id smtpdAAA2ba4LU; Thu Dec 14 16:33:45 2000 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id QAA15971; Thu, 14 Dec 2000 16:37:16 -0700 (MST) From: Terry Lambert Message-Id: <200012142337.QAA15971@usr08.primenet.com> Subject: Re: patch to cleanup inflight desciptor handling. To: dot@dotat.at (Tony Finch) Date: Thu, 14 Dec 2000 23:37:15 +0000 (GMT) Cc: tlambert@primenet.com (Terry Lambert), bright@wintelcom.net (Alfred Perlstein), dillon@earth.backplane.com (Matt Dillon), mckusick@mckusick.com (Kirk McKusick), arch@FreeBSD.ORG, net@FreeBSD.ORG In-Reply-To: <20001214231629.K92196@hand.dotat.at> from "Tony Finch" at Dec 14, 2000 11:16:29 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: tlambert@usr08.primenet.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > >SVR4 and Solaris avoid the problem entirely by ensuring that > >each reference to a vnode pointer counts as an "open", and > >the vnode can not be discarded until a 1->0 reference count > >change (grep for VHOLD/VRELE/VREF in the Solaris headers). > > FreeBSD does this too. It doesn't solve the circular reference > problem, though (hence the AI Koan). I think in-progress passes will have their references freed when the receiving end goes away. This still leaves the possibility of a circular reference being passed while being passed until the process exits, but I don't think you need to care about that. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Dec 14 15:45:45 2000 From owner-freebsd-arch@FreeBSD.ORG Thu Dec 14 15:45:41 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from smtp04.primenet.com (smtp04.primenet.com [206.165.6.134]) by hub.freebsd.org (Postfix) with ESMTP id F36F937B402; Thu, 14 Dec 2000 15:45:40 -0800 (PST) Received: (from daemon@localhost) by smtp04.primenet.com (8.9.3/8.9.3) id QAA21835; Thu, 14 Dec 2000 16:41:26 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp04.primenet.com, id smtpdAAAduaaFQ; Thu Dec 14 16:41:21 2000 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id QAA16165; Thu, 14 Dec 2000 16:45:27 -0700 (MST) From: Terry Lambert Message-Id: <200012142345.QAA16165@usr08.primenet.com> Subject: Re: patch to cleanup inflight desciptor handling. To: dot@dotat.at (Tony Finch) Date: Thu, 14 Dec 2000 23:45:27 +0000 (GMT) Cc: tlambert@primenet.com (Terry Lambert), bright@wintelcom.net (Alfred Perlstein), dillon@earth.backplane.com (Matt Dillon), mckusick@mckusick.com (Kirk McKusick), arch@FreeBSD.ORG, net@FreeBSD.ORG, dot@dotat.at (Tony Finch) In-Reply-To: <20001214233914.L92196@hand.dotat.at> from "Tony Finch" at Dec 14, 2000 11:39:14 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: tlambert@usr08.primenet.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > >This still leaves the possibility of a circular reference being > >passed while being passed until the process exits, but I don't > >think you need to care about that. > > That's the whole point of this conversation! I don't think you need to care about it. There are valid reasons for doing the passing, and a core dump of one process in a pair involved in a pass could leave the thing in limbo. But it is resolved on process exit. I don't think you need to be overly concerned about the thing being in limbo, since if both processes exit, it's cleaned up. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Dec 14 15:52:18 2000 From owner-freebsd-arch@FreeBSD.ORG Thu Dec 14 15:52:13 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 50BBC37B402; Thu, 14 Dec 2000 15:52:13 -0800 (PST) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id eBENldm19603; Thu, 14 Dec 2000 15:47:39 -0800 (PST) Date: Thu, 14 Dec 2000 15:47:39 -0800 From: Alfred Perlstein To: Terry Lambert Cc: Tony Finch , Matt Dillon , Kirk McKusick , arch@FreeBSD.ORG, net@FreeBSD.ORG Subject: Re: patch to cleanup inflight desciptor handling. Message-ID: <20001214154739.A19572@fw.wintelcom.net> References: <20001214233914.L92196@hand.dotat.at> <200012142345.QAA16165@usr08.primenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200012142345.QAA16165@usr08.primenet.com>; from tlambert@primenet.com on Thu, Dec 14, 2000 at 11:45:27PM +0000 Sender: bright@fw.wintelcom.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG * Terry Lambert [001214 15:45] wrote: > > >This still leaves the possibility of a circular reference being > > >passed while being passed until the process exits, but I don't > > >think you need to care about that. > > > > That's the whole point of this conversation! > > I don't think you need to care about it. There are valid reasons > for doing the passing, and a core dump of one process in a pair > involved in a pass could leave the thing in limbo. But it is > resolved on process exit. > > I don't think you need to be overly concerned about the thing > being in limbo, since if both processes exit, it's cleaned up. No it's not: Ok, the problem is you have 3 af_unix pairs all open between 2 processes process B sends 3 over 2 to A process B sends 2 over 3 to A process B send 2 and 3 over 1 to A process B closes 1 2 and 3 A then closes 3 2 and then 1 closing 3 and 2 doesn't cause the socketbuffer to be flushed because they are still self referencing. closing 1 causes the socketbuffer to be flushed, on flushing it comes across 2 and drops a reference but doesn't flush, it then hits 3 and drops a reference but doesn't flush. since 3 references 2 and 2 references 3 and nothing else references 2 or 3, we just leaked 2 and 3. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Dec 14 16:11:13 2000 From owner-freebsd-arch@FreeBSD.ORG Thu Dec 14 16:11:09 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from earth.backplane.com (placeholder-dcat-1076843399.broadbandoffice.net [64.47.83.135]) by hub.freebsd.org (Postfix) with ESMTP id C24EC37B402; Thu, 14 Dec 2000 16:11:09 -0800 (PST) Received: (from dillon@localhost) by earth.backplane.com (8.11.1/8.9.3) id eBF0B2R99195; Thu, 14 Dec 2000 16:11:02 -0800 (PST) (envelope-from dillon) Date: Thu, 14 Dec 2000 16:11:02 -0800 (PST) From: Matt Dillon Message-Id: <200012150011.eBF0B2R99195@earth.backplane.com> To: Terry Lambert Cc: dot@dotat.at (Tony Finch), tlambert@primenet.com (Terry Lambert), bright@wintelcom.net (Alfred Perlstein), mckusick@mckusick.com (Kirk McKusick), arch@FreeBSD.ORG, net@FreeBSD.ORG, dot@dotat.at (Tony Finch) Subject: Re: patch to cleanup inflight desciptor handling. References: <200012142345.QAA16165@usr08.primenet.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I think there is some confusion over ref counts here. I'm going to try to be clear: You *cannot* use a 1->0 transition on a ref count to cleanup self referential loops in socket message queues from file descriptor passing. Because no 1->0 transition will ever occur, even if both processes exit. This is because the file underlying the descriptor gets its ref count bumped when the *underyling* file is queued over the socket. We are *not* going to remove the ref count bumping. That's silly. We need to track references on the underlying file no matter who is referencing it.. whether it be a process or a message sitting in a socket queue. We are *not* going create a separate ref count field just to track socket queue references, because this breaks the file descriptor passing semantics... it is perfectly legal for the sending process to go away (exit, crash, whatever) after passing a descriptor prior to the receiving process recvmsg()ing the descriptor. The descriptor is not associated with the receiving process's descriptor resources until the receiving process pulls the message... this is necessary, otherwise the receiving process has no ability to control its file descriptors (descriptors would be able to pop up, at any time, outside of the receiving processes control). We are *not* going to allow the descriptors sitting in isolated loops to leak the system to death by ignoring them. There is no simple solution to the garbage collection problem. That's why the current inefficient, slow, spegetti that is the current GC code is still being used. We may be able to make it more efficient, but short of some genius spending a long time figuring out the perfect solution there aren't going to be any rewrites. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Dec 14 16:18:47 2000 From owner-freebsd-arch@FreeBSD.ORG Thu Dec 14 16:18:44 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from hand.dotat.at (sfo-gw.covalent.net [207.44.198.62]) by hub.freebsd.org (Postfix) with ESMTP id BC37B37B402; Thu, 14 Dec 2000 16:18:43 -0800 (PST) Received: from fanf by hand.dotat.at with local (Exim 3.15 #3) id 146iZz-0007V7-00; Fri, 15 Dec 2000 00:18:31 +0000 Date: Fri, 15 Dec 2000 00:18:31 +0000 From: Tony Finch To: Matt Dillon Cc: Terry Lambert , Alfred Perlstein , Kirk McKusick , arch@FreeBSD.ORG, net@FreeBSD.ORG Subject: Re: patch to cleanup inflight desciptor handling. Message-ID: <20001215001831.N92196@hand.dotat.at> References: <200012142345.QAA16165@usr08.primenet.com> <200012150011.eBF0B2R99195@earth.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: <200012150011.eBF0B2R99195@earth.backplane.com> Organization: Covalent Technologies, Inc Sender: fanf@dotat.at Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Matt Dillon wrote: > > We are *not* going create a separate ref count field just to track > socket queue references, because this breaks the file descriptor passing > semantics... There is an f_msgcount field already but isn't used for the sort of half-baked hack at the problem which you are railing against :-) > There is no simple solution to the garbage collection problem. That's > why the current inefficient, slow, spegetti that is the current GC code > is still being used. We may be able to make it more efficient, but short > of some genius spending a long time figuring out the perfect solution > there aren't going to be any rewrites. From looking at the GC literature the best known solution to this problem is the one I posted earlier today. Tony. -- f.a.n.finch fanf@covalent.net dot@dotat.at "If I didn't see it with my own eyes I would never have believed it!" To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Dec 14 16:48:27 2000 From owner-freebsd-arch@FreeBSD.ORG Thu Dec 14 16:48:25 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mail-relay.eunet.no (mail-relay.eunet.no [193.71.71.242]) by hub.freebsd.org (Postfix) with ESMTP id 7FB5737B400 for ; Thu, 14 Dec 2000 16:48:24 -0800 (PST) Received: from login-1.eunet.no (login-1.eunet.no [193.75.110.2]) by mail-relay.eunet.no (8.9.3/8.9.3/GN) with ESMTP id BAA70761; Fri, 15 Dec 2000 01:48:20 +0100 (CET) (envelope-from mbendiks@eunet.no) Received: from localhost (mbendiks@localhost) by login-1.eunet.no (8.9.3/8.8.8) with ESMTP id BAA48207; Fri, 15 Dec 2000 01:48:20 +0100 (CET) (envelope-from mbendiks@eunet.no) X-Authentication-Warning: login-1.eunet.no: mbendiks owned process doing -bs Date: Fri, 15 Dec 2000 01:48:20 +0100 (CET) From: Marius Bendiksen To: Sheldon Hearn Cc: Peter Pentchev , arch@freebsd.org Subject: Re: add -I ignoremask option to du(1) In-Reply-To: <55313.976800156@axl.fw.uunet.co.za> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > So the only question is whether we shrug and continue down this road, or > limit ourselves to only those silly enhancements that we deem necessary > for compatibility with other platforms. You are ignoring the option of standardizing all the utilities to do these things through a shared library. Marius To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Dec 14 20:20:31 2000 From owner-freebsd-arch@FreeBSD.ORG Thu Dec 14 20:20:29 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from sr14.nsw-remote.bigpond.net.au (sr14.nsw-remote.bigpond.net.au [24.192.3.29]) by hub.freebsd.org (Postfix) with ESMTP id 1428437B402 for ; Thu, 14 Dec 2000 20:20:27 -0800 (PST) Received: from areilly.bpc-users.org (CPE-144-132-234-126.nsw.bigpond.net.au [144.132.234.126]) by sr14.nsw-remote.bigpond.net.au (Pro-8.9.3/8.9.3) with SMTP id PAA28862 for ; Fri, 15 Dec 2000 15:20:20 +1100 (EDT) Received: (qmail 15037 invoked by uid 1000); 15 Dec 2000 04:20:18 -0000 From: "Andrew Reilly" Date: Fri, 15 Dec 2000 15:20:17 +1100 To: Terry Lambert Cc: Poul-Henning Kamp , kris@citusc.usc.edu, Dag-Erling Smorgrav , arch@FreeBSD.ORG Subject: Re: Safe string formatting in the kernel Message-ID: <20001215152017.A14921@gurney.reilly.home> References: <79446.976697492@critter> <200012130906.CAA27235@usr08.primenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200012130906.CAA27235@usr08.primenet.com>; from tlambert@primenet.com on Wed, Dec 13, 2000 at 09:06:07AM +0000 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Wed, Dec 13, 2000 at 09:06:07AM +0000, Terry Lambert wrote: > This would let you do the allocation based on peeking at the > size prior to copying the whole string in. Count prefix strings > are one thing the C language has been missing for years. Counted strings are good (lets you put arbitrary binary data into them, for one thing). Putting the count into a control structure as has been suggested here, or passed as another (separate) argument are both also good. Mangling the malloc'd data block so that the first word is a length, irrespective of whatever the data type is, is doubleplus ungood. (That mightn't have been what you meant by count _prefix_ strings, and I apologise if it wasn't, but it sounded like it to me.) -- Andrew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Dec 15 0:55:45 2000 From owner-freebsd-arch@FreeBSD.ORG Fri Dec 15 00:55:43 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from lists01.iafrica.com (lists01.iafrica.com [196.7.0.141]) by hub.freebsd.org (Postfix) with ESMTP id D18F037B402 for ; Fri, 15 Dec 2000 00:55:36 -0800 (PST) Received: from nwl.fw.uunet.co.za ([196.31.2.162]) by lists01.iafrica.com with esmtp (Exim 3.12 #2) id 146qeK-0006ol-00; Fri, 15 Dec 2000 10:55:32 +0200 Received: (from nobody@localhost) by nwl.fw.uunet.co.za (8.8.8/8.6.9) id KAA09679; Fri, 15 Dec 2000 10:55:30 +0200 (SAST) Received: by nwl.fw.uunet.co.za via recvmail id 9461; Fri Dec 15 10:54:45 2000 Received: from sheldonh (helo=axl.fw.uunet.co.za) by axl.fw.uunet.co.za with local-esmtp (Exim 3.16 #1) id 146qdZ-000J3Y-00; Fri, 15 Dec 2000 10:54:45 +0200 From: Sheldon Hearn To: Marius Bendiksen Cc: Peter Pentchev , arch@freebsd.org Subject: Re: add -I ignoremask option to du(1) In-reply-to: Your message of "Fri, 15 Dec 2000 01:48:20 +0100." Date: Fri, 15 Dec 2000 10:54:45 +0200 Message-ID: <73255.976870485@axl.fw.uunet.co.za> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, 15 Dec 2000 01:48:20 +0100, Marius Bendiksen wrote: > > So the only question is whether we shrug and continue down this road, or > > limit ourselves to only those silly enhancements that we deem necessary > > for compatibility with other platforms. > > You are ignoring the option of standardizing all the utilities to do these > things through a shared library. First, let me make clear that I have read Peter's rebuttal and I can see where he's coming from. I don't like it, but I have no technical objections that I can level against his argument. So what I say now is in your response to your mail and is not an attempt to drag Peter's proposal down. Now... I'm not ignoring the option of using shared libraries for implementation of common functions. However, we're talking about user interface here, which has nothing to do with implementation. The point is that we're gradually brewing a toolbox of spanners with saw edges and hammers with screwdriver attachments. And you activate the screwdriver on your hammer differently from the way you activate it on your spirit level. Not to mention that FreeBSD's hammer screwdriver isn't activated the way ${some_other}NIX's hammer screwdriver is. :-) Unless the UNIX philosophy of "each tool does one job well" has changed, let's not flog this one too much. Let's just leave it at this: When considering extensions to existing utilities, make sure that you have good reason beyond an aversion to a few extra keystrokes or ignorance of other utilities. Ciao, Sheldon. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Dec 15 1:47:29 2000 From owner-freebsd-arch@FreeBSD.ORG Fri Dec 15 01:47:28 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from ns5.pacific.net.au (ns5.pacific.net.au [203.143.252.30]) by hub.freebsd.org (Postfix) with ESMTP id 4AC6437B402; Fri, 15 Dec 2000 01:47:26 -0800 (PST) Received: from dungeon.home (ppp168.dyn249.pacific.net.au [203.143.249.168]) by ns5.pacific.net.au (8.9.0/8.9.1) with ESMTP id UAA03464; Fri, 15 Dec 2000 20:47:21 +1100 (EST) Received: from dungeon.home (localhost [127.0.0.1]) by dungeon.home (8.11.1/8.9.3) with ESMTP id eBF89CC12453; Fri, 15 Dec 2000 18:09:13 +1000 (EST) (envelope-from mckay) Message-Id: <200012150809.eBF89CC12453@dungeon.home> To: John Baldwin Cc: arch@FreeBSD.ORG Subject: Re: kthread API References: In-Reply-To: from John Baldwin at "Thu, 14 Dec 2000 21:55:36 +0000" Date: Fri, 15 Dec 2000 18:09:12 +1000 From: Stephen McKay Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Thursday, 14th December 2000, John Baldwin wrote: >I'd like to make some of the kernel API's a little more consistent as far as >in their naming. The first one I'd like to tackle is the kthread API. I support this, and have made similar suggestions in the past, where I was partially successful. After you've fixed the ones you want, consider changing vm_object_{set,clear}_flag to vm_object_flag_{set,clear} to match the (good) naming style used by similar routines from vm_page.h. Stephen. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Dec 15 6: 5:18 2000 From owner-freebsd-arch@FreeBSD.ORG Fri Dec 15 06:05:16 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mail.imp.ch (mail.imp.ch [157.161.1.2]) by hub.freebsd.org (Postfix) with ESMTP id 5E0E837B402 for ; Fri, 15 Dec 2000 06:05:15 -0800 (PST) Received: from levais.imp.ch (levais.imp.ch [157.161.4.66]) by mail.imp.ch (8.11.1/8.11.1) with ESMTP id eBFE5D205100 for ; Fri, 15 Dec 2000 15:05:14 +0100 (CET) (envelope-from Martin.Blapp@imp.ch) Date: Fri, 15 Dec 2000 15:08:48 +0100 (CET) From: Martin Blapp To: arch@freebsd.org Subject: _DIAGASSERT() Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Hi, I'm porting code from NetBSD and ask myself if I should remove some userland-code _DIAGASSERT()'s ... There is nothing appropriate in FreeBSD like diagassert() ... In NetBSD this is defined therefore : include/assert.h:# define _DIAGASSERT(e) ((e) ? (void)0 : __diagassert(__FILE__, __LINE__, "e")) and in libc/gen/assert.c : void __diagassert(file, line, failedexpr) const char *file, *failedexpr; int line; { /* * XXX: check $DIAGASSERT here, and do user-defined actions */ (void)fprintf(stderr, "%s: assertion \"%s\" failed: file \"%s\", line %d\n", __progname, failedexpr, file, line); syslog(LOG_DEBUG|LOG_USER, "assertion \"%s\" failed: file \"%s\", line %d", failedexpr, file, line); return; } Cheers Martin Martin Blapp, mb@imp.ch ------------------------------------------------ Improware AG, UNIX solution and service provider Zurlindenstrasse 29, 4133 Pratteln, Switzerland Phone: +41 79 370 26 05, Fax: +41 61 826 93 01 ------------------------------------------------ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Dec 15 6:34: 1 2000 From owner-freebsd-arch@FreeBSD.ORG Fri Dec 15 06:33:58 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from warning.follo.net (warning.follo.net [195.204.136.30]) by hub.freebsd.org (Postfix) with ESMTP id C320037B400 for ; Fri, 15 Dec 2000 06:33:48 -0800 (PST) Received: (from eivind@localhost) by warning.follo.net (8.9.3/8.9.3) id PAA77514; Fri, 15 Dec 2000 15:33:45 +0100 (CET) Date: Fri, 15 Dec 2000 15:33:45 +0100 From: Eivind Eklund To: Martin Blapp Cc: arch@FreeBSD.ORG Subject: Re: _DIAGASSERT() Message-ID: <20001215153345.A76398@warning.follo.net> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i In-Reply-To: ; from mb@imp.ch on Fri, Dec 15, 2000 at 03:08:48PM +0100 Sender: eivind@warning.follo.net Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, Dec 15, 2000 at 03:08:48PM +0100, Martin Blapp wrote: > > Hi, > > I'm porting code from NetBSD and ask myself if I should remove > some userland-code _DIAGASSERT()'s ... > > There is nothing appropriate in FreeBSD like diagassert() ... > > In NetBSD this is defined therefore : > > include/assert.h:# define _DIAGASSERT(e) ((e) ? (void)0 > : __diagassert(__FILE__, __LINE__, "e")) > > and in libc/gen/assert.c : > > void > __diagassert(file, line, failedexpr) > const char *file, *failedexpr; > int line; > { > /* > * XXX: check $DIAGASSERT here, and do user-defined > actions > */ > (void)fprintf(stderr, > "%s: assertion \"%s\" failed: file \"%s\", line %d\n", > __progname, failedexpr, file, line); > syslog(LOG_DEBUG|LOG_USER, > "assertion \"%s\" failed: file \"%s\", line %d", > failedexpr, file, line); > return; > } The relevant question is really "Should we add this, and if so in what exact form - or should we just drop the internal rigidity tests NetBSD has added?" I think the right answer is to add it, but with LOG_ERR instead of LOG_DEBUG - failure of the internal invariants in the code is something we want the user to see, not syslogd to discard. Eivind. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Dec 15 21:43:42 2000 From owner-freebsd-arch@FreeBSD.ORG Fri Dec 15 21:43:40 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from citusc.usc.edu (citusc.usc.edu [128.125.38.123]) by hub.freebsd.org (Postfix) with ESMTP id AA36637B400 for ; Fri, 15 Dec 2000 21:43:40 -0800 (PST) Received: (from kris@localhost) by citusc.usc.edu (8.9.3/8.9.3) id VAA10594; Fri, 15 Dec 2000 21:44:55 -0800 Date: Fri, 15 Dec 2000 21:44:55 -0800 From: Kris Kennaway To: Martin Blapp Cc: arch@FreeBSD.ORG Subject: Re: _DIAGASSERT() Message-ID: <20001215214455.A10562@citusc.usc.edu> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="SUOF0GtieIMvvwua" Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: ; from mb@imp.ch on Fri, Dec 15, 2000 at 03:08:48PM +0100 Sender: kris@citusc.usc.edu Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG --SUOF0GtieIMvvwua Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Dec 15, 2000 at 03:08:48PM +0100, Martin Blapp wrote: >=20 > Hi, >=20 > I'm porting code from NetBSD and ask myself if I should remove > some userland-code _DIAGASSERT()'s ... >=20 > There is nothing appropriate in FreeBSD like diagassert() ... Submit patches to add it :-) Kris --SUOF0GtieIMvvwua Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6OwFXWry0BWjoQKURAv66AKC2ZqRI3ozX8mw/cgO0m+yJG788bgCgxLIA PK0bAdu+Gp2Rs5BAcuOZiwQ= =QMbz -----END PGP SIGNATURE----- --SUOF0GtieIMvvwua-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Dec 16 8:53:20 2000 From owner-freebsd-arch@FreeBSD.ORG Sat Dec 16 08:53:19 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id C9FE737B400; Sat, 16 Dec 2000 08:53:16 -0800 (PST) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.9.3/8.8.7) with ESMTP id DAA26605; Sun, 17 Dec 2000 03:53:13 +1100 Date: Sun, 17 Dec 2000 03:53:12 +1100 (EST) From: Bruce Evans X-Sender: bde@besplex.bde.org To: John Baldwin Cc: arch@FreeBSD.ORG Subject: Re: kthread API In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Thu, 14 Dec 2000, John Baldwin wrote: > I'd like to make some of the kernel API's a little more consistent as far as in > their naming. The first one I'd like to tackle is the kthread API. > Specifically, I'd like to stick the whole API under a kthread_* API. This > would result in the following: > > Old Name New Name > -------- -------- > kproc_start kproc_start > shutdown_kproc kproc_shutdown > kthread_create kthread_create > kthread_exit kthread_exit > resume_kproc kthread_resume > suspend_kproc kthread_suspend > kproc_suspend_loop kthread_suspend_check "kthread" is a silly abbreviation, like "vmemory" would be for "virtual memory". Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Dec 16 10:48:11 2000 From owner-freebsd-arch@FreeBSD.ORG Sat Dec 16 10:48:09 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 18F9437B400 for ; Sat, 16 Dec 2000 10:48:05 -0800 (PST) Received: from foo.osd.bsdi.com (root@foo.osd.bsdi.com [204.216.28.137]) by pike.osd.bsdi.com (8.11.1/8.9.3) with ESMTP id eBGIlmE60193; Sat, 16 Dec 2000 10:47:48 -0800 (PST) (envelope-from jhb@foo.osd.bsdi.com) Received: (from jhb@localhost) by foo.osd.bsdi.com (8.11.1/8.11.0) id eBGIkv562533; Sat, 16 Dec 2000 10:46:57 -0800 (PST) (envelope-from jhb) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Sat, 16 Dec 2000 10:46:57 -0800 (PST) Organization: BSD, Inc. From: John Baldwin To: Bruce Evans Subject: Re: kthread API Cc: arch@FreeBSD.ORG Sender: jhb@foo.osd.bsdi.com Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On 16-Dec-00 Bruce Evans wrote: > On Thu, 14 Dec 2000, John Baldwin wrote: > >> I'd like to make some of the kernel API's a little more consistent as far as >> in >> their naming. The first one I'd like to tackle is the kthread API. >> Specifically, I'd like to stick the whole API under a kthread_* API. This >> would result in the following: >> >> Old Name New Name >> -------- -------- >> kproc_start kproc_start >> shutdown_kproc kproc_shutdown >> kthread_create kthread_create >> kthread_exit kthread_exit >> resume_kproc kthread_resume >> suspend_kproc kthread_suspend >> kproc_suspend_loop kthread_suspend_check > > "kthread" is a silly abbreviation, like "vmemory" would be for "virtual > memory". It has prior precedent in that it is already used, and that both NetBSD and OpenBSD use kthread_* for their kthread API. > Bruce -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.Baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Dec 16 16:44:45 2000 From owner-freebsd-arch@FreeBSD.ORG Sat Dec 16 16:44:43 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from palrel3.hp.com (palrel3.hp.com [156.153.255.226]) by hub.freebsd.org (Postfix) with ESMTP id 0565C37B400 for ; Sat, 16 Dec 2000 16:44:43 -0800 (PST) Received: from adlmail.cup.hp.com (adlmail.cup.hp.com [15.0.100.30]) by palrel3.hp.com (Postfix) with ESMTP id 5EC8F8537 for ; Sat, 16 Dec 2000 16:44:33 -0800 (PST) Received: from cup.hp.com (p1000180.nsr.hp.com [15.109.0.180]) by adlmail.cup.hp.com (8.9.3 (PHNE_18546)/8.9.3 SMKit7.02) with ESMTP id QAA18771 for ; Sat, 16 Dec 2000 16:44:32 -0800 (PST) Sender: marcel@cup.hp.com Message-ID: <3A3C0C70.848F5057@cup.hp.com> Date: Sat, 16 Dec 2000 16:44:32 -0800 From: Marcel Moolenaar Organization: Hewlett-Packard X-Mailer: Mozilla 4.73 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: arch@FreeBSD.org Subject: A perlified gensetdefs Content-Type: multipart/mixed; boundary="------------34B57FFE194884664E44F09C" Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG This is a multi-part message in MIME format. --------------34B57FFE194884664E44F09C Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Hi, In order to reduce the number of ELF specific tools, I rewrote gensetdefs(1) in perl, using objdump(1). When put in /sys/kern, we further reduce the chicken/egg problem when building a kernel. This definitely helps porting FreeBSD. take a look at it and let me know if it's a good idea or not. Any perl specific improvements, including style, are welcomed as well. I'm no Perl programmer by any means... -- Marcel Moolenaar mail: marcel@cup.hp.com / marcel@FreeBSD.org tel: (408) 447-4222 --------------34B57FFE194884664E44F09C Content-Type: application/x-perl; name="gensetdefs.pl" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="gensetdefs.pl" #!/usr/bin/perl my %sz = (); my $ptrsz = 0; my $objdump = $ENV{OBJDUMP}; $objdump =~ s/^$/objdump/; foreach $objfile (@ARGV) { open(SECTION, "$objdump -h $objfile |"); while($line =
) { ($idx, $name, $size, $vma, $lma, $ofs, $align) = split(" ", $line); if ($name =~ /^.set./) { $name =~ s/^.set.//; $sz{$name} = $sz{$name} + eval "0x$size"; if ($ptrsz < eval $align) { $ptrsz = eval $align; } } } close SECTION; } open(SETDEFS_H, "| sort > setdefs.h"); while (($name, $size) = each %sz) { my $elts = $size / $ptrsz; print SETDEFS_H "DEFINE_SET($name, $elts);\n"; } close SETDEFS_H; open(SETDEF0_C, "> setdef0.c"); print SETDEF0_C "/* THIS FILE IS GENERATED, DO NOT EDIT. */\n\n"; print SETDEF0_C "#define DEFINE_SET(set, count)\t\t\t\\\n"; print SETDEF0_C "__asm__(\".section .set.\" #set \",\\\"aw\\\"\");\t\\\n"; print SETDEF0_C "__asm__(\".globl \" #set);\t\t\t\\\n"; print SETDEF0_C "__asm__(\".type \" #set \",\@object\");\t\t\\\n"; print SETDEF0_C "__asm__(\".p2align 3\");\t\t\t\t\\\n"; print SETDEF0_C "__asm__(#set \":\");\t\t\t\t\\\n"; print SETDEF0_C "__asm__(\".quad \" #count);\t\t\t\\\n"; print SETDEF0_C "__asm__(\".previous\")\n\n"; print SETDEF0_C "#include \"setdefs.h\"\n"; close SETDEF0_C; open(SETDEF1_C, "> setdef1.c"); print SETDEF1_C "/* THIS FILE IS GENERATED, DO NOT EDIT. */\n\n"; print SETDEF1_C "#define DEFINE_SET(set, count)\t\t\t\\\n"; print SETDEF1_C "__asm__(\".section .set.\" #set \",\\\"aw\\\"\");\t\\\n"; print SETDEF1_C "__asm__(\".quad 0\");\t\t\t\t\\\n"; print SETDEF1_C "__asm__(\".previous\")\n\n"; print SETDEF1_C "#include \"setdefs.h\"\n"; close SETDEF1_C; --------------34B57FFE194884664E44F09C-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Dec 16 19:40:41 2000 From owner-freebsd-arch@FreeBSD.ORG Sat Dec 16 19:40:39 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mail.jeamland.net (rafe.jeamland.net [203.18.243.114]) by hub.freebsd.org (Postfix) with ESMTP id E647E37B400 for ; Sat, 16 Dec 2000 19:40:38 -0800 (PST) Received: by mail.jeamland.net (Postfix, from userid 1000) id 98BFF70601; Sun, 17 Dec 2000 14:40:36 +1100 (EST) Date: Sun, 17 Dec 2000 14:40:36 +1100 From: Benno Rice To: Marcel Moolenaar Cc: arch@FreeBSD.org Subject: Re: A perlified gensetdefs Message-ID: <20001217144036.A73676@rafe.jeamland.net> References: <3A3C0C70.848F5057@cup.hp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <3A3C0C70.848F5057@cup.hp.com>; from marcel@cup.hp.com on Sat, Dec 16, 2000 at 04:44:32PM -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sat, Dec 16, 2000 at 04:44:32PM -0800, Marcel Moolenaar wrote: > Hi, > > In order to reduce the number of ELF specific tools, I rewrote > gensetdefs(1) in perl, using objdump(1). When put in /sys/kern, we > further reduce the chicken/egg problem when building a kernel. This > definitely helps porting FreeBSD. > > take a look at it and let me know if it's a good idea or not. Any perl > specific improvements, including style, are welcomed as well. I'm no > Perl programmer by any means... I don't have that much objection to it, as it makes supporting big-endian architectures (eg, PowerPC =)) a fair bit easier. I have however made the following changes to the script: * Turned on warnings and 'use strict', and made all changes to silence the resultant warnings (mainly use of 'my' in more places). * Renamed some variables to make their purpose clearer. * Reworked a few blocks to make them easier to comprehend. * Added some comments. I also changed the two-column indent to four, but you're welcome to change that back, bikesheds permitting. =) Also, since you allow the environment variable OBJDUMP to override the name of the objdump program used, can I assume that you no longer object to my use of NM in genassym and lorder? -- Benno Rice benno@FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Dec 16 20: 7:52 2000 From owner-freebsd-arch@FreeBSD.ORG Sat Dec 16 20:07:50 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242]) by hub.freebsd.org (Postfix) with ESMTP id 8209D37B400; Sat, 16 Dec 2000 20:07:50 -0800 (PST) Received: from adlmail.cup.hp.com (adlmail.cup.hp.com [15.0.100.30]) by palrel1.hp.com (Postfix) with ESMTP id 33336155E4; Sat, 16 Dec 2000 20:07:25 -0800 (PST) Received: from cup.hp.com (p1000180.nsr.hp.com [15.109.0.180]) by adlmail.cup.hp.com (8.9.3 (PHNE_18546)/8.9.3 SMKit7.02) with ESMTP id UAA21163; Sat, 16 Dec 2000 20:07:24 -0800 (PST) Sender: marcel@cup.hp.com Message-ID: <3A3C3BFC.1F806316@cup.hp.com> Date: Sat, 16 Dec 2000 20:07:24 -0800 From: Marcel Moolenaar Organization: Hewlett-Packard X-Mailer: Mozilla 4.73 [en] (X11; U; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Benno Rice Cc: arch@FreeBSD.org Subject: Re: A perlified gensetdefs References: <3A3C0C70.848F5057@cup.hp.com> <20001217144036.A73676@rafe.jeamland.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Benno Rice wrote: > > * Turned on warnings and 'use strict', and made all changes to silence > the resultant warnings (mainly use of 'my' in more places). > * Renamed some variables to make their purpose clearer. > * Reworked a few blocks to make them easier to comprehend. > * Added some comments. Sounds good. > I also changed the two-column indent to four, but you're welcome to change that > back, bikesheds permitting. =) No, 4 is better. I still have to configure my editor... > Also, since you allow the environment variable OBJDUMP to override the name > of the objdump program used, can I assume that you no longer object to my use > of NM in genassym and lorder? Yep. I did some experimentation with the prefix method, just to get some feel for it and the more I worked with it, the more I disliked it. -- Marcel Moolenaar mail: marcel@cup.hp.com / marcel@FreeBSD.org tel: (408) 447-4222 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Dec 16 20:18: 3 2000 From owner-freebsd-arch@FreeBSD.ORG Sat Dec 16 20:17:59 2000 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mail.jeamland.net (rafe.jeamland.net [203.18.243.114]) by hub.freebsd.org (Postfix) with ESMTP id D72C537B400 for ; Sat, 16 Dec 2000 20:17:58 -0800 (PST) Received: by mail.jeamland.net (Postfix, from userid 1000) id 56FBC70601; Sun, 17 Dec 2000 15:17:57 +1100 (EST) Date: Sun, 17 Dec 2000 15:17:57 +1100 From: Benno Rice To: Marcel Moolenaar Cc: arch@FreeBSD.org Subject: Re: A perlified gensetdefs Message-ID: <20001217151757.D73676@rafe.jeamland.net> References: <3A3C0C70.848F5057@cup.hp.com> <20001217144036.A73676@rafe.jeamland.net> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="fUYQa+Pmc3FrFX/N" Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20001217144036.A73676@rafe.jeamland.net>; from benno@FreeBSD.org on Sun, Dec 17, 2000 at 02:40:36PM +1100 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG --fUYQa+Pmc3FrFX/N Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Sun, Dec 17, 2000 at 02:40:36PM +1100, Benno Rice wrote: > > I don't have that much objection to it, as it makes supporting big-endian > architectures (eg, PowerPC =)) a fair bit easier. I have however made > the following changes to the script: > * Turned on warnings and 'use strict', and made all changes to silence > the resultant warnings (mainly use of 'my' in more places). > * Renamed some variables to make their purpose clearer. > * Reworked a few blocks to make them easier to comprehend. > * Added some comments. Oops, and this time I actually attached the script. -- Benno Rice benno@FreeBSD.org --fUYQa+Pmc3FrFX/N Content-Type: application/x-perl Content-Disposition: attachment; filename="gensetdefs.pl" #!/usr/bin/perl -w use strict; my %sets = (); my $pointersize = 0; my $objdump = 'objdump'; # Allow objdump to be overridden by the environment. if (defined $ENV{'OBJDUMP'}) { $objdump = $ENV{'OBJDUMP'}; } # Run objdump over each object file to find all defined linker sets. foreach my $objfile (@ARGV) { open(SECTION, "$objdump -h $objfile |"); while(my $line =
) { my ($index, $name, $size, $vma, $lma, $offset, $align) = split(" ", $line); next if not defined $name; if ($name =~ /^.set./) { # We've found a set. $name =~ s/^.set.//; # Initialise it if needed. if (not defined $sets{$name}) { $sets{$name} = 0; } # Add the size of this entry. $sets{$name} = $sets{$name} + eval "0x$size"; if ($pointersize < eval $align) { $pointersize = eval $align; } } } close SECTION; } # Generate our list of set definitions my @setdefs; while (my ($name, $size) = each %sets) { my $elements = $size / $pointersize; push @setdefs, "DEFINE_SET($name, $elements);\n"; } # Create setdefs.h open(SETDEFS_H, "> setdefs.h"); foreach my $setdef (sort @setdefs) { print SETDEFS_H $setdef; } close SETDEFS_H; # Create setdef0.c open(SETDEF0_C, "> setdef0.c"); print SETDEF0_C < setdef1.c"); print SETDEF1_C <