From owner-freebsd-audit@FreeBSD.ORG Sun May 2 10:19:23 2004 Return-Path: Delivered-To: freebsd-audit@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4426616A4CE; Sun, 2 May 2004 10:19:23 -0700 (PDT) Received: from harmony.village.org (rover.bsdimp.com [204.144.255.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4CB1443D53; Sun, 2 May 2004 10:19:22 -0700 (PDT) (envelope-from imp@bsdimp.com) Received: from localhost (warner@rover2.village.org [10.0.0.1]) by harmony.village.org (8.12.10/8.12.9) with ESMTP id i42HJLZJ023554; Sun, 2 May 2004 11:19:21 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Sun, 02 May 2004 11:19:55 -0600 (MDT) Message-Id: <20040502.111955.111273448.imp@bsdimp.com> To: phk@phk.freebsd.dk From: "M. Warner Losh" In-Reply-To: <30608.1083480938@critter.freebsd.dk> References: <20040501223031.GA10624@melusine.cuivre.fr.eu.org> <30608.1083480938@critter.freebsd.dk> X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: current@freebsd.org cc: audit@freebsd.org cc: pb@freebsd.org cc: thomas@freebsd.org Subject: Re: Mounting root through devfs X-BeenThere: freebsd-audit@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD Security Audit List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 May 2004 17:19:23 -0000 In message: <30608.1083480938@critter.freebsd.dk> "Poul-Henning Kamp" writes: : In message <20040501223031.GA10624@melusine.cuivre.fr.eu.org>, Thomas Quinot writes: : : >Review, testing and comments would be very much appreciated. : > : >If I understand the comment just above the definition of g_dev_clone, : >integration of this patch would allow removal of a questionable hack. : : yes, this looks a fair bit but less but not quite not questionable : than the current root-mounting hack :-) "It sucks less than ever before" might not be inappropriate :-) Warner From owner-freebsd-audit@FreeBSD.ORG Mon May 3 05:21:14 2004 Return-Path: Delivered-To: freebsd-audit@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A876516A4CE; Mon, 3 May 2004 05:21:14 -0700 (PDT) Received: from melusine.cuivre.fr.eu.org (melusine.cuivre.fr.eu.org [82.225.155.84]) by mx1.FreeBSD.org (Postfix) with ESMTP id 519BD43D5C; Mon, 3 May 2004 05:21:14 -0700 (PDT) (envelope-from thomas@FreeBSD.ORG) Received: by melusine.cuivre.fr.eu.org (Postfix, from userid 1000) id 80B1E2A42B; Mon, 3 May 2004 14:21:16 +0200 (CEST) Date: Mon, 3 May 2004 14:21:16 +0200 From: Thomas Quinot To: current@freebsd.org, audit@freebsd.org, phk@freebsd.org, pb@freebsd.org Message-ID: <20040503122116.GA47228@melusine.cuivre.fr.eu.org> References: <20040501223031.GA10624@melusine.cuivre.fr.eu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040501223031.GA10624@melusine.cuivre.fr.eu.org> X-message-flag: WARNING! Using Outlook can damage your computer. User-Agent: Mutt/1.5.6i Subject: Re: Mounting root through devfs X-BeenThere: freebsd-audit@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD Security Audit List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 May 2004 12:21:14 -0000 * Thomas Quinot, 2004-05-02 : > An interesting use of this option is to mount a root file system by > FFS volume label (using GEOM_VOL), without needing to know what hardware > identifier will be assigned to it. Hum, actually, further testing revealed that this should have worked with -CURRENT without resorting to any patch. I must have messed up something when initially determining that this did not work. > If I understand the comment just above the definition of g_dev_clone, > integration of this patch would allow removal of a questionable hack. The patch could still be an improvement from an architectural perspective for this reason. Thomas. -- Thomas.Quinot@Cuivre.FR.EU.ORG From owner-freebsd-audit@FreeBSD.ORG Wed May 5 18:34:46 2004 Return-Path: Delivered-To: freebsd-audit@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1A72816A4CF; Wed, 5 May 2004 18:34:46 -0700 (PDT) Received: from spiff.melthusia.org (spiff.melthusia.org [207.67.244.17]) by mx1.FreeBSD.org (Postfix) with ESMTP id CE5BA43D41; Wed, 5 May 2004 18:34:45 -0700 (PDT) (envelope-from gtetlow@spiff.melthusia.org) Received: from spiff.melthusia.org (gtetlow@localhost [127.0.0.1]) by spiff.melthusia.org (8.12.10/8.12.10) with ESMTP id i461UPGc029815; Wed, 5 May 2004 18:30:25 -0700 (PDT) (envelope-from gtetlow@spiff.melthusia.org) Received: (from gtetlow@localhost) by spiff.melthusia.org (8.12.10/8.12.10/Submit) id i461UPfh029814; Wed, 5 May 2004 18:30:25 -0700 (PDT) (envelope-from gtetlow) Date: Wed, 5 May 2004 18:30:25 -0700 From: Gordon Tetlow To: Thomas Quinot Message-ID: <20040506013024.GG10016@spiff.melthusia.org> References: <20040501223031.GA10624@melusine.cuivre.fr.eu.org> <20040503122116.GA47228@melusine.cuivre.fr.eu.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="6v9BRtpmy+umdQlo" Content-Disposition: inline In-Reply-To: <20040503122116.GA47228@melusine.cuivre.fr.eu.org> X-Habeas-SWE-1: winter into spring X-Habeas-SWE-2: brightly anticipated X-Habeas-SWE-3: like Habeas SWE (tm) X-Habeas-SWE-4: Copyright 2002 Habeas (tm) X-Habeas-SWE-5: Sender Warranted Email (SWE) (tm). The sender of this X-Habeas-SWE-6: email in exchange for a license for this Habeas X-Habeas-SWE-7: warrant mark warrants that this is a Habeas Compliant X-Habeas-SWE-8: Message (HCM) and not spam. Please report use of this X-Habeas-SWE-9: mark in spam to . User-Agent: Mutt/1.5.5.1i cc: pb@freebsd.org cc: phk@freebsd.org cc: current@freebsd.org cc: audit@freebsd.org Subject: Re: Mounting root through devfs X-BeenThere: freebsd-audit@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD Security Audit List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 May 2004 01:34:46 -0000 --6v9BRtpmy+umdQlo Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, May 03, 2004 at 02:21:16PM +0200, Thomas Quinot wrote: > * Thomas Quinot, 2004-05-02 : >=20 > > An interesting use of this option is to mount a root file system by > > FFS volume label (using GEOM_VOL), without needing to know what hardware > > identifier will be assigned to it. >=20 > Hum, actually, further testing revealed that this should have worked > with -CURRENT without resorting to any patch. I must have messed up > something when initially determining that this did not work. I was gonna say, you don't need a patch for /dev/vol/root to work. The reason you may have been deceived is because you have to set the label on the root (which can only be done in single user when / is RO), then reboot the machine. Even then, /dev/vol/root doesn't show up because GEOM spoils the geom_vol_ffs class instance from /dev/ad0s1a due to an open write on /dev/ad0s1a (which geom_vol_ffs sits on top of). It takes an act of faith to set /dev/vol/root in your /etc/fstab, reboot, and hopefully everything works. -gordon --6v9BRtpmy+umdQlo Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.3 (FreeBSD) iD8DBQFAmZUwRu2t9DV9ZfsRAlrwAJ9giVlAA1tZx76uakK6YLnmGXEHYgCeLxWa Fzphvw8yNfqEaJaC7rsIfWc= =OM5/ -----END PGP SIGNATURE----- --6v9BRtpmy+umdQlo-- From owner-freebsd-audit@FreeBSD.ORG Sat May 8 05:55:50 2004 Return-Path: Delivered-To: freebsd-audit@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1726416A4CE; Sat, 8 May 2004 05:55:50 -0700 (PDT) Received: from mail.musha.org (daemon.musha.org [210.189.104.8]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5D50E43D48; Sat, 8 May 2004 05:55:49 -0700 (PDT) (envelope-from knu@iDaemons.org) Received: from archon.local.idaemons.org (archon.local.idaemons.org [192.168.1.32]) by mail.musha.org (Postfix) with ESMTP id 00FD6ACD7; Sat, 8 May 2004 21:55:48 +0900 (JST) Date: Sat, 08 May 2004 21:55:48 +0900 Message-ID: <86smebgjsb.knu@iDaemons.org> From: "Akinori MUSHA" To: current@FreeBSD.org, audit@FreeBSD.org Organization: Associated I. Daemons X-PGP-Public-Key: finger knu@FreeBSD.org X-PGP-Fingerprint: 081D 099C 1705 861D 4B70 B04A 920B EFC7 9FD9 E1EE MIME-Version: 1.0 (generated by EMIKO 1.14.1 - "Choanoflagellata") Content-Type: text/plain; charset=US-ASCII Subject: making test(1)'s -nt/-ot ignore nanosecond fractions X-BeenThere: freebsd-audit@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD Security Audit List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 May 2004 12:55:50 -0000 Hello, I faced an odd situation regarding file timestamps: knu@archon[2]% sh $ ls a b a b $ touch -r a b $ [ a -nt b ] && echo "D'oh!" D'oh! $ touch -r a a $ [ a -nt b ] || echo "Nah, how did it fix?" Nah, how did it fix? (Note that this is only rarely reproducible) I tracked it down and found out some facts: - File timestamps are recorded in nanoseconds on a UFS, and sometimes a file's timestamp actually has >1000 nanosecond (= >1 microsecond) value. - Stat(2) offers a file's mtime in nanoseconds through st_mtimespec.tv_nsec. - sh(1)'s builtin test command, which entity is test(1), compares times strictly in nanoseconds when seconds match. - On the other hand, there is no API to set a file's mtime in nanoseconds; utimes(2) takes time values in microseconds, and utime(3), which just calls utimes, takes time values in seconds. - Which means touch(1) or any other userland tool can only set a file's mtime in microseconds at most. - So, copying timestamps from a file "a" to a file "b" leaves a under-microsecond fraction only in "a", and after the copy "a" is "newer" than "b" by test(1). In order to fix this, we could consider adding a syscall or extend utimes(2) to take nanoseconds, but I'm afraid it would take a long time before it becomes available on -STABLE and the API spread over third-party applications. So, I'd suggest the following simple fix to make test(1) ignore such nanosecond fractions as a temporary measure: Index: test.c =================================================================== RCS file: /mirror/freebsd/ncvs/root/src/src/bin/test/test.c,v retrieving revision 1.52 diff -u -r1.52 test.c --- test.c 19 Aug 2002 09:19:31 -0000 1.52 +++ test.c 7 May 2004 17:06:54 -0000 @@ -527,7 +527,7 @@ if (b1.st_mtimespec.tv_sec < b2.st_mtimespec.tv_sec) return 0; - return (b1.st_mtimespec.tv_nsec > b2.st_mtimespec.tv_nsec); + return (b1.st_mtimespec.tv_nsec / 1000 > b2.st_mtimespec.tv_nsec / 1000); } static int Fortunately, bash and zsh behave the same way, as nanosecond times support is so heavily system dependent they give up to care about, which means the above change only makes test(1) compatible with other shells and would cause any harm. IMHO, no userland tool should care about nanosecond fractions when there is no API to set times in nanoseconds. Opinions? -- / /__ __ Akinori.org / MUSHA.org / ) ) ) ) / FreeBSD.org / Ruby-lang.org Akinori MUSHA aka / (_ / ( (__( @ iDaemons.org / and.or.jp "It seems to me as we make our own few circles 'round the sun We get it backwards and our seven years go by like one" From owner-freebsd-audit@FreeBSD.ORG Sat May 8 07:48:33 2004 Return-Path: Delivered-To: freebsd-audit@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7554916A4CE; Sat, 8 May 2004 07:48:33 -0700 (PDT) Received: from mailout1.pacific.net.au (mailout1.pacific.net.au [61.8.0.84]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5721B43D60; Sat, 8 May 2004 07:48:32 -0700 (PDT) (envelope-from bde@zeta.org.au) Received: from mailproxy1.pacific.net.au (mailproxy1.pacific.net.au [61.8.0.86])i48EmU4u021171; Sun, 9 May 2004 00:48:30 +1000 Received: from gamplex.bde.org (katana.zip.com.au [61.8.7.246]) i48EmRI2005609; Sun, 9 May 2004 00:48:28 +1000 Date: Sun, 9 May 2004 00:48:27 +1000 (EST) From: Bruce Evans X-X-Sender: bde@gamplex.bde.org To: Akinori MUSHA In-Reply-To: <86smebgjsb.knu@iDaemons.org> Message-ID: <20040509000147.U4217@gamplex.bde.org> References: <86smebgjsb.knu@iDaemons.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: current@freebsd.org cc: audit@freebsd.org Subject: Re: making test(1)'s -nt/-ot ignore nanosecond fractions X-BeenThere: freebsd-audit@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD Security Audit List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 May 2004 14:48:33 -0000 On Sat, 8 May 2004, Akinori MUSHA wrote: > I faced an odd situation regarding file timestamps: > > knu@archon[2]% sh > $ ls a b > a b > $ touch -r a b > $ [ a -nt b ] && echo "D'oh!" > D'oh! > $ touch -r a a > $ [ a -nt b ] || echo "Nah, how did it fix?" > Nah, how did it fix? > > (Note that this is only rarely reproducible) Hopefully never if the reference file was created by ffs and vfs.timestamp_precision is the default. > I tracked it down and found out some facts: > > - File timestamps are recorded in nanoseconds on a UFS, and > sometimes a file's timestamp actually has >1000 nanosecond > (= >1 microsecond) value. ffs uses vfs_timestamp() which gives a timestamp with the precision specified by vfs.timestamp_precision. The default is 0 (TSP_SEC), which means that timestamps on files are normally in seconds with nanoseconds part 0. This can be changed easily using sysctl, but changing the precision to the highest (nanoseconds) gives the bugs being discussed. Changing it to microseconds precision is safer, since utimes(2) (but not utime(2) supports this precision. The only other way to get ffs timestamps with a nonzero nanseconds part is to use utimes(), but this gives microseconds precision which utimes() can copy later. > [... details deleted] > - Which means touch(1) or any other userland tool can only set a > file's mtime in microseconds at most. > ... > In order to fix this, we could consider adding a syscall or extend > utimes(2) to take nanoseconds, but I'm afraid it would take a long > time before it becomes available on -STABLE and the API spread over > third-party applications. So, I'd suggest the following simple fix to > make test(1) ignore such nanosecond fractions as a temporary measure: It's already taken more than 10 years for for the API to make null progress matching the nanoseconds filesystem timestamps :(. I think timestamps with a precision smaller than 1 microsecond should just not be made (don't support it in vfs_timestamp(), and fix all the file systems that use nanotime() directly). Another problem with file times is that they can be older than the time at which they are made, due to shortcuts in time_second and getnanotime(). E.g., if the current time is N.009999 seconds so that time(3) returns N, then time_second may be (N-1) seconds, so file times may be 1 second older than the time at which they were made if vfs.timestamp_precision is the default. Similarly, if vfs.timestamp_precision is 1 (TSP_HZ), then file times may be 1/HZ seconds older than the time at which they were made. These bugs are missing if vfs.timestamp_precision is 2 (TSP_USEC) or 3 (TSP_NSEC). I've never needed more that seconds resolution AFAIK, but I sometimes use vfs.timestamp_precision=3 to avoid the bugs. A vfs.timestamp_precision of 1 (TSP_HZ) is more broken than I remembered. It doesn't round the nanoseconds part to a multiple of 1/HZ or a multiple of 1000, so it gives timestamps that utimes() can't preserve for no benefit. Bruce From owner-freebsd-audit@FreeBSD.ORG Sat May 8 14:22:47 2004 Return-Path: Delivered-To: freebsd-audit@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4F5CA16A4CE; Sat, 8 May 2004 14:22:47 -0700 (PDT) Received: from mail.musha.org (daemon.musha.org [210.189.104.8]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0F15643D3F; Sat, 8 May 2004 14:22:46 -0700 (PDT) (envelope-from knu@iDaemons.org) Received: from archon.local.idaemons.org (archon.local.idaemons.org [192.168.1.32]) by mail.musha.org (Postfix) with ESMTP id F25B9ACD7; Sun, 9 May 2004 06:22:44 +0900 (JST) Date: Sun, 09 May 2004 06:22:44 +0900 Message-ID: <86oeoyhavv.knu@iDaemons.org> From: "Akinori MUSHA" To: Bruce Evans In-Reply-To: <20040509000147.U4217@gamplex.bde.org> References: <86smebgjsb.knu@iDaemons.org> <20040509000147.U4217@gamplex.bde.org> Organization: Associated I. Daemons X-PGP-Public-Key: finger knu@FreeBSD.org X-PGP-Fingerprint: 081D 099C 1705 861D 4B70 B04A 920B EFC7 9FD9 E1EE MIME-Version: 1.0 (generated by EMIKO 1.14.1 - "Choanoflagellata") Content-Type: text/plain; charset=US-ASCII cc: current@freebsd.org cc: audit@freebsd.org Subject: Re: making test(1)'s -nt/-ot ignore nanosecond fractions X-BeenThere: freebsd-audit@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD Security Audit List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 May 2004 21:22:47 -0000 Hello, At Sun, 9 May 2004 00:48:27 +1000 (EST), Bruce Evans wrote: > > - File timestamps are recorded in nanoseconds on a UFS, and > > sometimes a file's timestamp actually has >1000 nanosecond < > > (= >1 microsecond) value. < > > ffs uses vfs_timestamp() which gives a timestamp with the precision > specified by vfs.timestamp_precision. The default is 0 (TSP_SEC), > which means that timestamps on files are normally in seconds with > nanoseconds part 0. This can be changed easily using sysctl, but > changing the precision to the highest (nanoseconds) gives the bugs > being discussed. Changing it to microseconds precision is safer, > since utimes(2) (but not utime(2) supports this precision. > > The only other way to get ffs timestamps with a nonzero nanseconds > part is to use utimes(), but this gives microseconds precision which > utimes() can copy later. Hm, I never changed vfs.timestamp_precision to anything other than the default (nor even knew of it), but I acutually observed some files having ...032 and ...256 nanoseconds on an NFS exported FFS on 4-STABLE. I'm not sure if the files were created directly on FFS or via NFS. Does that mean there could be a bug somewhere around nanotime() calls? -- > > [... details deleted] > > - Which means touch(1) or any other userland tool can only set a > > file's mtime in microseconds at most. > > ... > > In order to fix this, we could consider adding a syscall or extend > > utimes(2) to take nanoseconds, but I'm afraid it would take a long > > time before it becomes available on -STABLE and the API spread over > > third-party applications. So, I'd suggest the following simple fix to > > make test(1) ignore such nanosecond fractions as a temporary measure: > > It's already taken more than 10 years for for the API to make null > progress matching the nanoseconds filesystem timestamps :(. > > I think timestamps with a precision smaller than 1 microsecond should > just not be made (don't support it in vfs_timestamp(), and fix all the > file systems that use nanotime() directly). > > Another problem with file times is that they can be older than the > time at which they are made, due to shortcuts in time_second and > getnanotime(). E.g., if the current time is N.009999 seconds so that > time(3) returns N, then time_second may be (N-1) seconds, so file times > may be 1 second older than the time at which they were made if > vfs.timestamp_precision is the default. Similarly, if > vfs.timestamp_precision is 1 (TSP_HZ), then file times may be 1/HZ > seconds older than the time at which they were made. These bugs are > missing if vfs.timestamp_precision is 2 (TSP_USEC) or 3 (TSP_NSEC). > I've never needed more that seconds resolution AFAIK, but I sometimes > use vfs.timestamp_precision=3 to avoid the bugs. > > A vfs.timestamp_precision of 1 (TSP_HZ) is more broken than I remembered. > It doesn't round the nanoseconds part to a multiple of 1/HZ or a multiple > of 1000, so it gives timestamps that utimes() can't preserve for no > benefit. -- / /__ __ Akinori.org / MUSHA.org / ) ) ) ) / FreeBSD.org / Ruby-lang.org Akinori MUSHA aka / (_ / ( (__( @ iDaemons.org / and.or.jp "It seems to me as we make our own few circles 'round the sun We get it backwards and our seven years go by like one"