From owner-freebsd-fs@FreeBSD.ORG Wed Jan 26 07:47:02 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4D80316A4CE for ; Wed, 26 Jan 2005 07:47:02 +0000 (GMT) Received: from hotmail.com (bay15-f12.bay15.hotmail.com [65.54.185.12]) by mx1.FreeBSD.org (Postfix) with ESMTP id C0E0343D39 for ; Wed, 26 Jan 2005 07:47:01 +0000 (GMT) (envelope-from ppmmssgg@hotmail.com) Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Tue, 25 Jan 2005 23:47:01 -0800 Message-ID: Received: from 220.194.103.228 by by15fd.bay15.hotmail.msn.com with HTTP; Wed, 26 Jan 2005 07:46:40 GMT X-Originating-IP: [220.194.103.228] X-Originating-Email: [ppmmssgg@hotmail.com] X-Sender: ppmmssgg@hotmail.com From: =?gb2312?B?wfUg0e8=?= To: freebsd-fs@freebsd.org Date: Wed, 26 Jan 2005 07:46:40 +0000 Mime-Version: 1.0 Content-Type: text/plain; charset=gb2312; format=flowed X-OriginalArrivalTime: 26 Jan 2005 07:47:01.0404 (UTC) FILETIME=[38852DC0:01C5037B] X-Mailman-Approved-At: Wed, 26 Jan 2005 13:23:51 +0000 Subject: Use loader to load kernel.gz and mfsroot.gz in ext2 filesystem? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2005 07:47:02 -0000 Hi,sorry to bother you^-^ I have a problem about the loader of freebsd I put loader,kernel.gz and mfsroot.gz in an ext2(dos) file system on harddisk,and hope to directly use grub or use grub and loader(or other means?) to boot the system. When I use grub to transfer loader,I find it(freebsd loader) can't read files in ext2(dos) file system,and when i use grub 0.93 of redhat 9.0 to directly load mfsroot.gz by "initrd /mfsroot.gz",it fails with "linux kernel must be loaded before initrd". My goal is to put install.cfg,freebsd kernel and mfsroot on harddisk,then boot from harddisk into freebsd,and install it from original freebsd cd. Any one helps? any answers are appreciated.^-^ _________________________________________________________________ 与联机的朋友进行交流，请使用 MSN Messenger: http://messenger.msn.com/cn From owner-freebsd-fs@FreeBSD.ORG Wed Jan 26 14:00:59 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 73E2E16A4CE for ; Wed, 26 Jan 2005 14:00:59 +0000 (GMT) Received: from web41203.mail.yahoo.com (web41203.mail.yahoo.com [66.218.93.36]) by mx1.FreeBSD.org (Postfix) with SMTP id 3A06B43D1D for ; Wed, 26 Jan 2005 14:00:59 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 19163 invoked by uid 60001); 26 Jan 2005 14:00:59 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=jCKEaIeMRwkLrh30lgkuN+8EYNbpBFEA73h7FnTsUdCOv9OZP4YpchAn+QjQy4cGLZrXNBEEQhY4QtVqBMRvhur4Pl+Vh1wRa8XN4FKkRLjWiaPsasisA5WtCph9EWEx6V7BNSiT/9TGUl/FwhbqqtSXtE+OlTMgrkVKgYHOAo0= ; Message-ID: <20050126140058.19161.qmail@web41203.mail.yahoo.com> Received: from [213.54.133.150] by web41203.mail.yahoo.com via HTTP; Wed, 26 Jan 2005 06:00:58 PST Date: Wed, 26 Jan 2005 06:00:58 -0800 (PST) From: Arne "W鰎ner" To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: ufs+softupdates / consistency X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2005 14:00:59 -0000 Hi! On http://e2fsprogs.sourceforge.net/ext2intro.html I found the strings "BSD-like synchronous updates" "it can cause corruption in the user data" . On http://www.mckusick.com/softdep/ I did not find such a statement. Are soft updates safe for user data? I do not really understand, what the first www page means... Maybe they mean, that the new file size (that would be meta data, I think) is written before the user data, so that the file contains undetermined data in its tail. I would be glad, if somebody tells me, if ufs with soft updates writes file size and block information after user data writes have been committed? Bye Arne __________________________________ Do you Yahoo!? Yahoo! Mail - You care about security. So do we. http://promotions.yahoo.com/new_mail From owner-freebsd-fs@FreeBSD.ORG Wed Jan 26 17:25:27 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C197116A4CE for ; Wed, 26 Jan 2005 17:25:27 +0000 (GMT) Received: from VARK.MIT.EDU (VARK.MIT.EDU [18.95.3.179]) by mx1.FreeBSD.org (Postfix) with ESMTP id 69B0A43D4C for ; Wed, 26 Jan 2005 17:25:27 +0000 (GMT) (envelope-from das@FreeBSD.ORG) Received: from VARK.MIT.EDU (localhost [127.0.0.1]) by VARK.MIT.EDU (8.13.1/8.13.1) with ESMTP id j0QHPfj1014083; Wed, 26 Jan 2005 12:25:41 -0500 (EST) (envelope-from das@FreeBSD.ORG) Received: (from das@localhost) by VARK.MIT.EDU (8.13.1/8.13.1/Submit) id j0QHPfa9014082; Wed, 26 Jan 2005 12:25:41 -0500 (EST) (envelope-from das@FreeBSD.ORG) Date: Wed, 26 Jan 2005 12:25:41 -0500 From: David Schultz To: Arne WXrner Message-ID: <20050126172541.GA13950@VARK.MIT.EDU> Mail-Followup-To: Arne WXrner , freebsd-fs@FreeBSD.ORG References: <20050126140058.19161.qmail@web41203.mail.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20050126140058.19161.qmail@web41203.mail.yahoo.com> cc: freebsd-fs@FreeBSD.ORG Subject: Re: ufs+softupdates / consistency X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2005 17:25:27 -0000 On Wed, Jan 26, 2005, Arne WXrner wrote: > Hi! > > On > http://e2fsprogs.sourceforge.net/ext2intro.html > I found the strings > "BSD-like synchronous updates" > "it can cause corruption in the user data" . > > On > http://www.mckusick.com/softdep/ > I did not find such a statement. > > Are soft updates safe for user data? I do not really understand, > what the first www page means... Maybe they mean, that the new > file size (that would be meta data, I think) is written before the > user data, so that the file contains undetermined data in its > tail. The ext2fs paper you refer to was published at about the same time as Ganger and Patt's Soft Updates paper, so I think it's safe to say that the authors of the former didn't know about Soft Updates. The comments you refer to that seem to imply that synchronous updates are unsafe and asynchronous updates are safer are wrong in general (synchronous updates are safer), but the authors may be referring to bugs in the ext2fs implementation at that time. Soft Updates, in contrast, provides asynchronous updates, issued in an order that makes them safe. From owner-freebsd-fs@FreeBSD.ORG Wed Jan 26 17:46:05 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E0BEF16A4CE for ; Wed, 26 Jan 2005 17:46:05 +0000 (GMT) Received: from web41205.mail.yahoo.com (web41205.mail.yahoo.com [66.218.93.38]) by mx1.FreeBSD.org (Postfix) with SMTP id BBF2E43D1F for ; Wed, 26 Jan 2005 17:46:05 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 1703 invoked by uid 60001); 26 Jan 2005 17:46:04 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=KsZkXXyEWTZUnMHPB3DOgnFeLDGhZ31MQioS2bTZNG56aa8MxOgwCYzpivmDsRPppblxLabrjEkGzTjsPY1b6k7NX1AuboB3QWC3BwODToFPkmDzSgLBNoeqUGbk83rn1FQOIKqwJHnn3zzZ+/R2t5ZVIXtDrMoITP7ZpNLFls0= ; Message-ID: <20050126174604.1701.qmail@web41205.mail.yahoo.com> Received: from [213.54.133.150] by web41205.mail.yahoo.com via HTTP; Wed, 26 Jan 2005 09:46:04 PST Date: Wed, 26 Jan 2005 09:46:04 -0800 (PST) From: Arne "W鰎ner" To: freebsd-fs@FreeBSD.ORG In-Reply-To: <20050126172541.GA13950@VARK.MIT.EDU> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: Re: ufs+softupdates / consistency X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2005 17:46:06 -0000 --- David Schultz wrote: > On Wed, Jan 26, 2005, Arne WXrner wrote: > > On > > http://e2fsprogs.sourceforge.net/ext2intro.html > > I found the strings > > "BSD-like synchronous updates" > > "it can cause corruption in the user data" . > > On > > http://www.mckusick.com/softdep/ > > I did not find such a statement. > > Are soft updates safe for user data? I do not really > > understand, what the first www page means... Maybe they mean, > > that the new file size (that would be meta data, I think) is > > written before the user data, so that the file contains > > undetermined data in its tail. > > The comments you refer to that seem to imply that synchronous > updates are unsafe and asynchronous updates are safer are wrong > in general (synchronous updates are safer), but the authors may > be referring to bugs in the ext2fs implementation at that time. > Soft Updates, in contrast, provides asynchronous updates, issued > in an order that makes them safe. > I would be glad, if somebody explains me, why ext2fs/async in Linux kernel 2.4.27 (KNOPPIX V3.7) is much faster (about 4 times faster) than a ufs with soft updates on the same slice of the hard disc? Is it due to consistency reasons? In case of a ext2fs/sync in my Linux setting Linux was about 4 times slower. Are we already trying to issue write order requests for the disc blocks (whose write order is arbitrary) sorted by sector number (in order to move the disc heads as less as possible)? The disc write cache could do that, but I disabled it in order to decrease the probability of inconsistency. -Arne __________________________________ Do you Yahoo!? Yahoo! Mail - Find what you need with new enhanced search. http://info.mail.yahoo.com/mail_250 From owner-freebsd-fs@FreeBSD.ORG Wed Jan 26 18:20:55 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4EF2816A4CE for ; Wed, 26 Jan 2005 18:20:55 +0000 (GMT) Received: from bingnet2.cc.binghamton.edu (bingnet2.cc.binghamton.edu [128.226.1.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id A607343D41 for ; Wed, 26 Jan 2005 18:20:52 +0000 (GMT) (envelope-from zzhang@cs.binghamton.edu) Received: from opal (opal.cs.binghamton.edu [128.226.123.101]) j0QIKo28013845; Wed, 26 Jan 2005 13:20:51 -0500 (EST) Date: Wed, 26 Jan 2005 13:20:49 -0500 (EST) From: Zhihui Zhang X-Sender: zzhang@opal To: =?X-UNKNOWN?Q?Arne_W=F6rner?= In-Reply-To: <20050126174604.1701.qmail@web41205.mail.yahoo.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=X-UNKNOWN Content-Transfer-Encoding: QUOTED-PRINTABLE cc: freebsd-fs@FreeBSD.ORG Subject: Re: ufs+softupdates / consistency X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2005 18:20:55 -0000 No file system is super for ALL benchmarks. Maybe you should say something about your application, its access pattern, file count, file sizes, read/write ratio, etc. On Wed, 26 Jan 2005, Arne W=F6rner wrote: > --- David Schultz wrote: > > On Wed, Jan 26, 2005, Arne WXrner wrote: > > > On > > > http://e2fsprogs.sourceforge.net/ext2intro.html > > > I found the strings > > > "BSD-like synchronous updates" > > > "it can cause corruption in the user data" . > > > On > > > http://www.mckusick.com/softdep/ > > > I did not find such a statement. > > > Are soft updates safe for user data? I do not really > > > understand, what the first www page means... Maybe they mean, > > > that the new file size (that would be meta data, I think) is > > > written before the user data, so that the file contains > > > undetermined data in its tail. > >=20 > > The comments you refer to that seem to imply that synchronous > > updates are unsafe and asynchronous updates are safer are wrong > > in general (synchronous updates are safer), but the authors may > > be referring to bugs in the ext2fs implementation at that time. > > Soft Updates, in contrast, provides asynchronous updates, issued > > in an order that makes them safe. > >=20 > I would be glad, if somebody explains me, why ext2fs/async in > Linux kernel 2.4.27 (KNOPPIX V3.7) is much faster (about 4 times > faster) than a ufs with soft updates on the same slice of the hard > disc? >=20 > Is it due to consistency reasons? In case of a ext2fs/sync in my > Linux setting Linux was about 4 times slower. >=20 > Are we already trying to issue write order requests for the disc > blocks (whose write order is arbitrary) sorted by sector number > (in order to move the disc heads as less as possible)? The disc > write cache could do that, but I disabled it in order to decrease > the probability of inconsistency. >=20 > -Arne >=20 >=20 >=20 > =09=09 > __________________________________=20 > Do you Yahoo!?=20 > Yahoo! Mail - Find what you need with new enhanced search. > http://info.mail.yahoo.com/mail_250 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >=20 From owner-freebsd-fs@FreeBSD.ORG Wed Jan 26 18:41:23 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F1D0716A4CF for ; Wed, 26 Jan 2005 18:41:23 +0000 (GMT) Received: from web41215.mail.yahoo.com (web41215.mail.yahoo.com [66.218.93.48]) by mx1.FreeBSD.org (Postfix) with SMTP id AA30443D5A for ; Wed, 26 Jan 2005 18:41:23 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 86910 invoked by uid 60001); 26 Jan 2005 18:41:23 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=1U9864dPf08scd+kPRIy6+/BWHLXnB4HmPztGuiLCVXey5eC+yxoHi1I5bGsUUzOepfWhp7sVyuiFC+rs7wV//0UslMatTyhpvavSoKFvGL963yGYm5G6Ds2bgBBkyOiUbnWG13ZCi7TEuSk88aB4kHxjlPYeoQfKWLLjKIctTU= ; Message-ID: <20050126184123.86908.qmail@web41215.mail.yahoo.com> Received: from [213.54.133.150] by web41215.mail.yahoo.com via HTTP; Wed, 26 Jan 2005 10:41:23 PST Date: Wed, 26 Jan 2005 10:41:23 -0800 (PST) From: Arne "W鰎ner" To: Zhihui Zhang In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii cc: freebsd-fs@FreeBSD.ORG Subject: Re: ufs+softupdates / consistency X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2005 18:41:24 -0000 --- Zhihui Zhang wrote: > On Wed, 26 Jan 2005, Arne W鰎ner wrote: > > I would be glad, if somebody explains me, why ext2fs/async in > > Linux kernel 2.4.27 (KNOPPIX V3.7) is much faster (about 4 > > times faster) than a ufs with soft updates on the same slice of > > the hard disc? > > > > Is it due to consistency reasons? In case of a ext2fs/sync in > > my Linux setting Linux was about 4 times slower. > > > > Are we already trying to issue write order requests for the > > disc blocks (whose write order is arbitrary) sorted by sector > > number (in order to move the disc heads as less as possible)? > > The disc write cache could do that, but I disabled it in order > > to decrease the probability of inconsistency. > > > No file system is super for ALL benchmarks. Maybe you should > say something about your application, its access pattern, file > count, file sizes, read/write ratio, etc. > Ok Reading a special device (like /dev/ad0s2) in FreeBSD R5.3 is as fast as in KNOPPIX. But writing a single new file with big block size (32k-128k) and 1000 blocks is much different (depends on Linux/FBSD and sync/async). And I do not know now, why that is so. For further information see http://home.tiscali.de/cmdr_faako/ , if you want, please. -Arne W鰎ner __________________________________ Do you Yahoo!? The all-new My Yahoo! - What will yours do? http://my.yahoo.com From owner-freebsd-fs@FreeBSD.ORG Wed Jan 26 21:24:22 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4904516A4CE for ; Wed, 26 Jan 2005 21:24:22 +0000 (GMT) Received: from web41211.mail.yahoo.com (web41211.mail.yahoo.com [66.218.93.44]) by mx1.FreeBSD.org (Postfix) with SMTP id E1B3143D4C for ; Wed, 26 Jan 2005 21:24:21 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 66504 invoked by uid 60001); 26 Jan 2005 21:24:21 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=HwvIOlWVnCNGgV8XOzwcLwzyXOsAz9Jd87MM4urgzQ0U1X3OG8a+reXqrwT4CL61b/yEF7tFnwTHL0hcJfz8xgGx4kAr6DnstDw7b79kvVKzRtyesQhJLV/sPwpGIvqtRwiSE9IVOlBzBbAeGfUGsSykv7V2TjLGKhxUX4VNvxg= ; Message-ID: <20050126212421.66502.qmail@web41211.mail.yahoo.com> Received: from [83.129.184.184] by web41211.mail.yahoo.com via HTTP; Wed, 26 Jan 2005 13:24:21 PST Date: Wed, 26 Jan 2005 13:24:21 -0800 (PST) From: Arne "W鰎ner" To: freebsd-fs@freebsd.org In-Reply-To: <20050126210209.GA69421@isilon.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: Re: ufs+softupdates / consistency X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2005 21:24:22 -0000 --- Justin Husted wrote: > You probably want to compare to ext3 on linux, too, since that's > what everyone is using these days due to the journaling. > Oops. I did the Linux tests on an ext3fs (/dev/hda2 on /a type ext3 (rw,sync)). Sorry. -Arne __________________________________ Do you Yahoo!? Yahoo! Mail - You care about security. So do we. http://promotions.yahoo.com/new_mail From owner-freebsd-fs@FreeBSD.ORG Wed Jan 26 21:28:25 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2411516A4CE for ; Wed, 26 Jan 2005 21:28:25 +0000 (GMT) Received: from VARK.MIT.EDU (VARK.MIT.EDU [18.95.3.179]) by mx1.FreeBSD.org (Postfix) with ESMTP id C057243D48 for ; Wed, 26 Jan 2005 21:28:24 +0000 (GMT) (envelope-from das@FreeBSD.ORG) Received: from VARK.MIT.EDU (localhost [127.0.0.1]) by VARK.MIT.EDU (8.13.1/8.13.1) with ESMTP id j0QLScYo061479; Wed, 26 Jan 2005 16:28:38 -0500 (EST) (envelope-from das@FreeBSD.ORG) Received: (from das@localhost) by VARK.MIT.EDU (8.13.1/8.13.1/Submit) id j0QLScde061478; Wed, 26 Jan 2005 16:28:38 -0500 (EST) (envelope-from das@FreeBSD.ORG) Date: Wed, 26 Jan 2005 16:28:38 -0500 From: David Schultz To: Arne WXrner Message-ID: <20050126212838.GA61425@VARK.MIT.EDU> Mail-Followup-To: Arne WXrner , freebsd-fs@FreeBSD.ORG References: <20050126172541.GA13950@VARK.MIT.EDU> <20050126174604.1701.qmail@web41205.mail.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20050126174604.1701.qmail@web41205.mail.yahoo.com> cc: freebsd-fs@FreeBSD.ORG Subject: Re: ufs+softupdates / consistency X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2005 21:28:25 -0000 On Wed, Jan 26, 2005, Arne WXrner wrote: > I would be glad, if somebody explains me, why ext2fs/async in > Linux kernel 2.4.27 (KNOPPIX V3.7) is much faster (about 4 times > faster) than a ufs with soft updates on the same slice of the hard > disc? > > Is it due to consistency reasons? In case of a ext2fs/sync in my > Linux setting Linux was about 4 times slower. ext2fs mounted async does not provide consistency; in fact the state of the disk can be almost arbitrarily inconsistent at any given time. Soft updates is supposed to provide performance comparable to async writes without the inconsistency problem. I'm not sure what it is about your setup that causes such a disparity. (Many factors such as the FS block size and ATA write caching can make a big difference.) By the way, ext3fs uses journalling, which provides metadata consistency in a very different way from soft updates. You might also want to experiment with that to see if it works better for your workload. > Are we already trying to issue write order requests for the disc > blocks (whose write order is arbitrary) sorted by sector number > (in order to move the disc heads as less as possible)? The disc > write cache could do that, but I disabled it in order to decrease > the probability of inconsistency. Hopefully you disabled it on both FreeBSD and Linux, so you're comparing apples to apples... From owner-freebsd-fs@FreeBSD.ORG Wed Jan 26 23:03:46 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B927916A4CE for ; Wed, 26 Jan 2005 23:03:46 +0000 (GMT) Received: from web41213.mail.yahoo.com (web41213.mail.yahoo.com [66.218.93.46]) by mx1.FreeBSD.org (Postfix) with SMTP id 731F443D31 for ; Wed, 26 Jan 2005 23:03:46 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 7960 invoked by uid 60001); 26 Jan 2005 23:03:46 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=fp/+X+s6ErbOHLXcHjwHAXtRIq7eaxgBsv5L8iF58X5KLPtaiuMddzpl9yBm69XQMT7BRPk7nqVqTzxlWGFWHwiXKKeE6bBLxu+maoGvvruJm1XNSqqmcbeE2S0kD/SRY+llOVPmmKtMAmOsyqqabIjYaDGDykOeqrusO/8+Yeo= ; Message-ID: <20050126230346.7958.qmail@web41213.mail.yahoo.com> Received: from [83.129.184.184] by web41213.mail.yahoo.com via HTTP; Wed, 26 Jan 2005 15:03:46 PST Date: Wed, 26 Jan 2005 15:03:46 -0800 (PST) From: Arne "W鰎ner" To: David Schultz In-Reply-To: <20050126212838.GA61425@VARK.MIT.EDU> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii cc: freebsd-fs@FreeBSD.ORG Subject: Re: ufs+softupdates / consistency X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2005 23:03:46 -0000 --- David Schultz wrote: > ext2fs mounted async does not provide consistency; in fact the > state of the disk can be almost arbitrarily inconsistent at any > given time. Soft updates is supposed to provide performance > comparable to async writes without the inconsistency problem. > I'm not sure what it is about your setup that causes such a > disparity. (Many factors such as the FS block size and ATA write > caching can make a big difference.) > Somebody in list freebsd-performance@ opened a thread "FreeBSD 5.3 I/O Performance / Linux 2.6.10 | Continued Discussion", where he and one other state, that FreeBSD R4.11 beforms much better (twice faster) than FreeBSD R5.3. Could the disparity I saw be caused by the SMPng project in R5.3? I have to admit, that I never tested ext2fs but ext3fs with unknown fs parameters (I just used mkfs_ext3fs (or so?)). > By the way, ext3fs uses journalling, which provides metadata > consistency in a very different way from soft updates. You > might also want to experiment with that to see if it works better > for your workload. > Hmm... I do not understand this hint. Does FreeBSD offer a journaling file system? I liked the idea of soft updates quite much, because it saves disc write accesses (as far as I understood it), so that I would like to continue to use ufs+softupd... > > Are we already trying to issue write order requests for the > > disc blocks (whose write order is arbitrary) sorted by sector > > number (in order to move the disc heads as less as possible)? > > The disc write cache could do that, but I disabled it in order > > to decrease the probability of inconsistency. > > Hopefully you disabled it on both FreeBSD and Linux, so you're > comparing apples to apples... > During the tests I enabled write cache in both settings, because I did not know how to turn it off in KNOPPIX... Isn't it possible to simulate the hard disc write cache in kernel? I could try to write some code, but I am quite unfamiliar with kernel program writing. Maybe this is not a fs@ subject but more a performance@ subject? -Arne __________________________________ Do you Yahoo!? Meet the all-new My Yahoo! - Try it today! http://my.yahoo.com From owner-freebsd-fs@FreeBSD.ORG Thu Jan 27 00:40:02 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EAF0716A4CE for ; Thu, 27 Jan 2005 00:40:02 +0000 (GMT) Received: from VARK.MIT.EDU (VARK.MIT.EDU [18.95.3.179]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6C4BF43D55 for ; Thu, 27 Jan 2005 00:40:00 +0000 (GMT) (envelope-from das@FreeBSD.ORG) Received: from VARK.MIT.EDU (localhost [127.0.0.1]) by VARK.MIT.EDU (8.13.1/8.13.1) with ESMTP id j0R0eDNN062659; Wed, 26 Jan 2005 19:40:13 -0500 (EST) (envelope-from das@FreeBSD.ORG) Received: (from das@localhost) by VARK.MIT.EDU (8.13.1/8.13.1/Submit) id j0R0eDDu062658; Wed, 26 Jan 2005 19:40:13 -0500 (EST) (envelope-from das@FreeBSD.ORG) Date: Wed, 26 Jan 2005 19:40:13 -0500 From: David Schultz To: Arne WXrner Message-ID: <20050127004013.GA62561@VARK.MIT.EDU> Mail-Followup-To: Arne WXrner , freebsd-fs@FreeBSD.ORG References: <20050126212838.GA61425@VARK.MIT.EDU> <20050126230346.7958.qmail@web41213.mail.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20050126230346.7958.qmail@web41213.mail.yahoo.com> cc: freebsd-fs@FreeBSD.ORG Subject: Re: ufs+softupdates / consistency X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2005 00:40:03 -0000 On Wed, Jan 26, 2005, Arne WXrner wrote: > --- David Schultz wrote: > > ext2fs mounted async does not provide consistency; in fact the > > state of the disk can be almost arbitrarily inconsistent at any > > given time. Soft updates is supposed to provide performance > > comparable to async writes without the inconsistency problem. > > I'm not sure what it is about your setup that causes such a > > disparity. (Many factors such as the FS block size and ATA write > > caching can make a big difference.) > > > Somebody in list freebsd-performance@ opened a thread "FreeBSD 5.3 > I/O Performance / Linux 2.6.10 | Continued Discussion", where he > and one other state, that FreeBSD R4.11 beforms much better (twice > faster) than FreeBSD R5.3. > > Could the disparity I saw be caused by the SMPng project in R5.3? It's certainly possible. > > By the way, ext3fs uses journalling, which provides metadata > > consistency in a very different way from soft updates. You > > might also want to experiment with that to see if it works > better > > for your workload. > > > Hmm... I do not understand this hint. > > Does FreeBSD offer a journaling file system? No, although an interest in UFS+journalling has been expressed by someone who could make it happen. Also, Jean-Sebastien Pedron has been working on a port of ReiserFS to FreeBSD. > > > Are we already trying to issue write order requests for the > > > disc blocks (whose write order is arbitrary) sorted by sector > > > number (in order to move the disc heads as less as possible)? > > > The disc write cache could do that, but I disabled it in order > > > to decrease the probability of inconsistency. > > > > Hopefully you disabled it on both FreeBSD and Linux, so you're > > comparing apples to apples... > > > During the tests I enabled write cache in both settings, because I > did not know how to turn it off in KNOPPIX... > > Isn't it possible to simulate the hard disc write cache in kernel? The write caching will make a big difference. Sorry, but I don't know how to turn it off under Linux. Perhaps you could turn it on under FreeBSD for the purpose of your performance analysis. From owner-freebsd-fs@FreeBSD.ORG Thu Jan 27 01:42:51 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1E6A516A4CE for ; Thu, 27 Jan 2005 01:42:51 +0000 (GMT) Received: from web41204.mail.yahoo.com (web41204.mail.yahoo.com [66.218.93.37]) by mx1.FreeBSD.org (Postfix) with SMTP id CE1DA43D46 for ; Thu, 27 Jan 2005 01:42:50 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 57724 invoked by uid 60001); 27 Jan 2005 01:42:50 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=rEDoahc+PPb6jHrc1jSCZis553HBB78rkP6gmUWO5Od0GEzRrMQNh40bquLLM0eP4Mwp4NPIRSJSY8kL8Q232wZA+Pcq1X/jSbxeJe2FUKcsSnZc5/oB/ImujcPCA5e6TrWcLcGrSUwSaoWiGEzuxB9ZyX+XziPBrpc7SWy7SeQ= ; Message-ID: <20050127014250.57722.qmail@web41204.mail.yahoo.com> Received: from [213.54.140.239] by web41204.mail.yahoo.com via HTTP; Wed, 26 Jan 2005 17:42:50 PST Date: Wed, 26 Jan 2005 17:42:50 -0800 (PST) From: Arne "W鰎ner" To: David Schultz In-Reply-To: <20050127004013.GA62561@VARK.MIT.EDU> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii cc: freebsd-fs@FreeBSD.ORG Subject: Re: ufs+softupdates / consistency X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2005 01:42:51 -0000 --- David Schultz wrote: > On Wed, Jan 26, 2005, Arne WXrner wrote: > > > > The disc write cache could do that, but I disabled it in > > > > order to decrease the probability of inconsistency. > > > > > > Hopefully you disabled it on both FreeBSD and Linux, so > > > you're comparing apples to apples... > > > > > During the tests I enabled write cache in both settings, > > because I did not know how to turn it off in KNOPPIX... > > > > Isn't it possible to simulate the hard disc write cache in > > kernel? > > The write caching will make a big difference. Sorry, but I > don't know how to turn it off under Linux. Perhaps you could > turn it on under FreeBSD for the purpose of your performance > analysis. > Yes, I did. I enabled it in both test settings (KNOPPIX and FreeBSD). But I set hw.ata.wc to 0 in my every day setting. I just tried "dd of=a if=/dev/zero bs=32k count=1000" on a file system with hard disc write cache enabled and disabled and I saw no difference (appr. 5 Mbyte/sec, which is about 5 times less than a read from that file after the kernel cache was clear by other reads). Can somebody explain me, why hw.ata.wc does not change write speed? I would prefer hw.ata.wc=0, because it would be part of a damage avoidance system (I do not use an UPS and I have some write accesses over quite long periods of time caused by my TV cards (sys/dev/bktr)). Can somebody explain me, why write speed is so much slower than read speed (even with hard disc write cache)? I tried an UFS1 file system mounted async for another test. And the write speed was still about 5 Mbyte/sec. Can somebody explain me, what async filesystem I/O is (somehow my english is not sufficient to find that out)? -Arne __________________________________ Do you Yahoo!? Take Yahoo! Mail with you! Get it on your mobile phone. http://mobile.yahoo.com/maildemo From owner-freebsd-fs@FreeBSD.ORG Thu Jan 27 03:00:26 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DAF8116A4CE for ; Thu, 27 Jan 2005 03:00:26 +0000 (GMT) Received: from dyson.jdyson.com (dsl-static-206-246-160-137.iquest.net [206.246.160.137]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2B20743D31 for ; Thu, 27 Jan 2005 03:00:26 +0000 (GMT) (envelope-from toor@dyson.jdyson.com) Received: from dyson.jdyson.com (localhost [127.0.0.1]) by dyson.jdyson.com (8.12.8/8.9.3) with ESMTP id j0R30PNF000714 for ; Wed, 26 Jan 2005 22:00:25 -0500 (EST) (envelope-from toor@dyson.jdyson.com) Received: (from toor@localhost) by dyson.jdyson.com (8.12.8/8.12.8/Submit) id j0R30PE7000713 for freebsd-fs@freebsd.org; Wed, 26 Jan 2005 22:00:25 -0500 (EST) Message-Id: <200501270300.j0R30PE7000713@dyson.jdyson.com> To: freebsd-fs@freebsd.org Date: Wed, 26 Jan 2005 22:00:24 -0500 (EST) From: jdy@jdyson.com X-Mailer: ELM [version 2.4ME+ PL54 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit cc: dyson@iquest.net Subject: Write caching on FreeBSD X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: jdy@jdyson.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2005 03:00:27 -0000 Quick comment: When doing/updating the write code, I carefully limited the amount of pending write data so that the system wouldn't 'go to sleep' while trying to sync out massive amounts of data. For convienience, this amount of write space was limited by the disk cache buffer representation. My original disk cache code wasn't limited to the cache buffers, and would actually create large numbers of dirty pages. This allowed for faster performance, except under load. I was interested in the high load performance conditions, and some mods have apparently been made to support more pending writes (by increasing the disk buffer cache representations?) Anyway, it seems like FreeBSD currently has a good compromise (better than my original, but also better than a no-holds barred scheme that creates lots of dirty pages.) John From owner-freebsd-fs@FreeBSD.ORG Thu Jan 27 03:25:10 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 14E0516A4CF for ; Thu, 27 Jan 2005 03:25:10 +0000 (GMT) Received: from mail.freebsd.org.cn (dns3.freebsd.org.cn [61.129.66.75]) by mx1.FreeBSD.org (Postfix) with SMTP id 319B043D48 for ; Thu, 27 Jan 2005 03:25:03 +0000 (GMT) (envelope-from delphij@frontfree.net) Received: (qmail 51259 invoked by uid 0); 27 Jan 2005 03:16:59 -0000 Received: from unknown (HELO beastie.frontfree.net) (219.239.99.7) by mail.freebsd.org.cn with SMTP; 27 Jan 2005 03:16:59 -0000 Received: from localhost (localhost.frontfree.net [127.0.0.1]) by beastie.frontfree.net (Postfix) with ESMTP id 7624513142B; Thu, 27 Jan 2005 11:24:55 +0800 (CST) Received: from beastie.frontfree.net ([127.0.0.1]) by localhost (beastie.frontfree.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 13497-15; Thu, 27 Jan 2005 11:24:43 +0800 (CST) Received: from localhost.localdomain (unknown [61.135.152.194]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by beastie.frontfree.net (Postfix) with ESMTP id 50FC01311FC; Thu, 27 Jan 2005 11:24:41 +0800 (CST) From: Xin LI To: Arne WXrner In-Reply-To: <20050127014250.57722.qmail@web41204.mail.yahoo.com> References: <20050127014250.57722.qmail@web41204.mail.yahoo.com> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-JixoUVyPnVtgYtd5GEfX" Organization: The FreeBSD Simplified Chinese Project Date: Thu, 27 Jan 2005 11:23:26 +0800 Message-Id: <1106796206.623.35.camel@spirit> Mime-Version: 1.0 X-Mailer: Evolution 2.0.3 FreeBSD GNOME Team Port X-Virus-Scanned: by amavisd-new at frontfree.net cc: freebsd-fs@FreeBSD.ORG cc: David Schultz Subject: Re: ufs+softupdates / consistency X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: delphij@delphij.net List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2005 03:25:10 -0000 --=-JixoUVyPnVtgYtd5GEfX Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi, Arne, =E5=9C=A8 2005-01-26=E4=B8=89=E7=9A=84 17:42 -0800=EF=BC=8CArne WXrner=E5= =86=99=E9=81=93=EF=BC=9A [snip] > Yes, I did. I enabled it in both test settings (KNOPPIX and > FreeBSD). But I set hw.ata.wc to 0 in my every day setting. >=20 > I just tried "dd of=3Da if=3D/dev/zero bs=3D32k count=3D1000" on a file > system with hard disc write cache enabled and disabled and I saw > no difference (appr. 5 Mbyte/sec, which is about 5 times less than > a read from that file after the kernel cache was clear by other > reads). >=20 > Can somebody explain me, why hw.ata.wc does not change write > speed? That's because write caching do little help for sequence writes. The goal of write cache is that the driver can re-order writes so it can reduce the unnecessary movements of heads. In a sequence write, this is unnecessary and writing cache make only a little benefit at the beginning, then it will be flushed again and again as disk writes are much slower. However, unfortunately, ATA writing cache does not have "tag" feature like SCSI devices usually offer. Instead of giving interrupt when a tag is committed to disk, ATA disks simply tell the operating system "Yes, the data is already written" and this causes problem for both SoftUpdates and Journalling (which REQUIRES that the journal to be written before any metadata updates it represents). So it's wise to turn hw.ata.wc=3D0 if the disk is supposed to store some important data which is being changed from time to time, and you have turned on SoftUpdates or Journalling. > I would prefer hw.ata.wc=3D0, because it would be part of a damage > avoidance system (I do not use an UPS and I have some write > accesses over quite long periods of time caused by my TV cards > (sys/dev/bktr)). >=20 > Can somebody explain me, why write speed is so much slower than > read speed (even with hard disc write cache)? This is common case caused by physical constraints I guess :-) > I tried an UFS1 file system mounted async for another test. And > the write speed was still about 5 Mbyte/sec. Well, I haven't benchmarked it by myself, however, the author of SoftUpdates claims that the benchmark should be 95% or so as you async mount a UFS file system. BTW: I think 5MB/s of write speed is somewhat too slow for an IDE device, you may want to check the cable, etc. > Can somebody explain me, what async filesystem I/O is (somehow my > english is not sufficient to find that out)? Traditionally, file systems use synchronous writes of metadata in order to guarantee consistency of metadata. In order to get best performance, however, the writes to data should be written in an order that makes minimal disk head moves. A asynchronous mounted file system won't synchronously (N.B. Waits the write to be completed, rather than to continue and allow subsequent data to be added to the write queue) write metadata, which makes it possible to write all data in the "best performance" order. SoftUpdates and Journaling techniques makes a trade off of the traditional scenario and asynchronous (as we can see it's not safe if the system crashes, which can lead to arbitrary inconsistency of your file system). SoftUpdates guarantees that the metadata writes are in a "right" order, say, nothing will be referenced before they gets initialized, this guarantee means that the file system only have "recoverable" inconsistency, like leak of space, etc., after a crash. Journaling means that you write something describes that what metadata will be written so metadata writes can be asynchronous. After a system crash, something that checks the transaction log must be executed and roll back what is half-committed in order to get the file system clean again. Unfortunately, there are many journaling implementations that does not guarantee the transaction logs to be written before actually updating meta, rendering journaling useless. The current FreeBSD SoftUpdates implementation also has a flaw that on large disks it's still painful to check the file system (even it's running in the background). A potential solution is to change the file system layout to make the "dirty bit" local to allocation groups, which may finally lead to a new file system. Cheers, --=20 Xin LI http://www.delphij.net/ --=-JixoUVyPnVtgYtd5GEfX Content-Type: application/pgp-signature; name=signature.asc Content-Description: =?UTF-8?Q?=E8=BF=99=E6=98=AF=E4=BF=A1=E4=BB=B6=E7=9A=84=E6=95=B0?= =?UTF-8?Q?=E5=AD=97=E7=AD=BE=E5=90=8D=E9=83=A8?= =?UTF-8?Q?=E5=88=86?= -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (FreeBSD) iD8DBQBB+F6u/cVsHxFZiIoRAn2hAKCKCfaI61I3CsgW+wNeVgIlnv04TwCfdF6N D0R7D99yLHFtZ8aK3CElNuI= =xKp3 -----END PGP SIGNATURE----- --=-JixoUVyPnVtgYtd5GEfX-- From owner-freebsd-fs@FreeBSD.ORG Thu Jan 27 04:04:18 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D34BE16A4CE for ; Thu, 27 Jan 2005 04:04:18 +0000 (GMT) Received: from web41205.mail.yahoo.com (web41205.mail.yahoo.com [66.218.93.38]) by mx1.FreeBSD.org (Postfix) with SMTP id 9809E43D39 for ; Thu, 27 Jan 2005 04:04:18 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 26915 invoked by uid 60001); 27 Jan 2005 04:04:18 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=5wG6AFe7klxSwKKUcjhKDCg6lvSuOwxvUccvfeWJpI+/uWCTZxCyYO/nVF3X3qIUwT1bdiN0VXgtZ7Fe4GiYFrm0CMFvK9roocpFf0zAqSheOnXangxqA01wTQnKmig8slGHlDfyvzIkPYEV0bJ1CJoUErB8CBf25cxpg1GKWeY= ; Message-ID: <20050127040418.26913.qmail@web41205.mail.yahoo.com> Received: from [213.54.140.239] by web41205.mail.yahoo.com via HTTP; Wed, 26 Jan 2005 20:04:18 PST Date: Wed, 26 Jan 2005 20:04:18 -0800 (PST) From: Arne "W鰎ner" To: delphij@delphij.net In-Reply-To: <1106796206.623.35.camel@spirit> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii cc: freebsd-fs@FreeBSD.ORG Subject: Re: ufs+softupdates / consistency X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2005 04:04:18 -0000 --- Xin LI wrote: > 鍦 2005-01-26涓夌殑 17:42 -0800锛孉rne WXrner鍐欓亾锛 > > Can somebody explain me, why write speed is so much slower > > than read speed (even with hard disc write cache)? > > This is common case caused by physical constraints I guess :-) > > I tried an UFS1 file system mounted async for another test. > > And the write speed was still about 5 Mbyte/sec. > > Well, I haven't benchmarked it by myself, however, the author of > SoftUpdates claims that the benchmark should be 95% or so as you > async mount a UFS file system. BTW: I think 5MB/s of write > speed is somewhat too slow for an IDE device, you may want to > check the cable, etc. > Hmm. I tried a "dd if=/dev/zero of=a bs=64k count=1000 ; sync" with KNOPPIX and the same slice/cable and ext3fs (mkfs with defaults) and it was about 4 times faster than FreeBSD R5.3. Furthermore the read speed is about 25Mbyte/sec, so that the cable seems to be ok. Funnily my other hard disc (SAMSUNG SP1604N), which uses the same cable, is much faster: write (bs=64k count=1000 if=/dev/zero): appr. 48Mbyte/sec read (bs=64k count=3975 if=): appr. 38Mbyte/sec (???????????) I do not understand this different behaviour... *sniff* -Arne __________________________________ Do you Yahoo!? Yahoo! Mail - Helps protect you from nasty viruses. http://promotions.yahoo.com/new_mail From owner-freebsd-fs@FreeBSD.ORG Thu Jan 27 16:32:51 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A180816A4CE; Thu, 27 Jan 2005 16:32:51 +0000 (GMT) Received: from mailout2.pacific.net.au (mailout2.pacific.net.au [61.8.0.85]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0A5DA43D45; Thu, 27 Jan 2005 16:32:51 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailproxy1.pacific.net.au (mailproxy1.pacific.net.au [61.8.0.86])j0RGWnHn022819; Fri, 28 Jan 2005 03:32:49 +1100 Received: from katana.zip.com.au (katana.zip.com.au [61.8.7.246]) j0RGWlVU032439; Fri, 28 Jan 2005 03:32:48 +1100 Date: Fri, 28 Jan 2005 03:32:47 +1100 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: David Schultz In-Reply-To: <20050126172541.GA13950@VARK.MIT.EDU> Message-ID: <20050128031722.C58410@delplex.bde.org> References: <20050126140058.19161.qmail@web41203.mail.yahoo.com> <20050126172541.GA13950@VARK.MIT.EDU> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-fs@freebsd.org Subject: Re: ufs+softupdates / consistency X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2005 16:32:51 -0000 On Wed, 26 Jan 2005, David Schultz wrote: > On Wed, Jan 26, 2005, Arne WXrner wrote: > > On > > http://e2fsprogs.sourceforge.net/ext2intro.html > > I found the strings > > "BSD-like synchronous updates" > > "it can cause corruption in the user data" . > > > > On > > http://www.mckusick.com/softdep/ > > I did not find such a statement. > > ... > > The ext2fs paper you refer to was published at about the same time > as Ganger and Patt's Soft Updates paper, so I think it's safe to > say that the authors of the former didn't know about Soft Updates. > The comments you refer to that seem to imply that synchronous > updates are unsafe and asynchronous updates are safer are wrong in > general (synchronous updates are safer), but the authors may be > referring to bugs in the ext2fs implementation at that time. > Soft Updates, in contrast, provides asynchronous updates, issued > in an order that makes them safe. I think part of the argument for async updates being safer (than sync metadata and async data) is that the latter gives a larger window where the data pointed to by the metadata might be garbage (because the pointers to it are up to date but the data might not be). Syncing everything at once gives a smaller window, especially if the sync is in a burst like FreeBSD tries not to do. http://e2fsprogs.sourceforge.net/ext2intro.html doesn't seem to be making exactly that argument. Completing the above quote gives: "it can cause corruption in the user data which will not be flagged by the file system checker." This just says that fsck cannot even guess that the data is corrupt when the metadata is not corrupt. Using async for both gives a better chance that either both are corrupt or neither is corrupt. Bruce From owner-freebsd-fs@FreeBSD.ORG Thu Jan 27 17:39:23 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3256116A579 for ; Thu, 27 Jan 2005 17:39:23 +0000 (GMT) Received: from mailout2.pacific.net.au (mailout2.pacific.net.au [61.8.0.85]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7FF3F43D41 for ; Thu, 27 Jan 2005 17:39:22 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailproxy2.pacific.net.au (mailproxy2.pacific.net.au [61.8.0.87])j0RHcpHn030013; Fri, 28 Jan 2005 04:38:51 +1100 Received: from katana.zip.com.au (katana.zip.com.au [61.8.7.246]) j0RHcm8w011665; Fri, 28 Jan 2005 04:38:49 +1100 Date: Fri, 28 Jan 2005 04:38:48 +1100 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: =?X-UNKNOWN?Q?Arne_W=F6rner?= In-Reply-To: <20050127040418.26913.qmail@web41205.mail.yahoo.com> Message-ID: <20050128040944.G58526@delplex.bde.org> References: <20050127040418.26913.qmail@web41205.mail.yahoo.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=X-UNKNOWN Content-Transfer-Encoding: QUOTED-PRINTABLE cc: freebsd-fs@freebsd.org cc: delphij@delphij.net Subject: Re: ufs+softupdates / consistency X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2005 17:39:23 -0000 On Wed, 26 Jan 2005, Arne W=F6rner wrote: > --- Xin LI wrote: > > =E5=9C=A8 2005-01-26=E4=B8=89=E7=9A=84 17:42 -0800=EF=BC=8CArne WXrner= =E5=86=99=E9=81=93=EF=BC=9A > > > Can somebody explain me, why write speed is so much slower > > > than read speed (even with hard disc write cache)? > > > > This is common case caused by physical constraints I guess :-) There should be little difference for most disks. Maybe you don't actually have the write cache enabled. Some firmware is rumoured mishandle the wc flag. > > > I tried an UFS1 file system mounted async for another test. > > > And the write speed was still about 5 Mbyte/sec. > > > > Well, I haven't benchmarked it by myself, however, the author of > > SoftUpdates claims that the benchmark should be 95% or so as you > > async mount a UFS file system. For me, it used to be about 1.5 times slower for the benchmark of writing a copy of /usr/src, but it is now 3 times slower for this, and it is even slightly slower than soft updates without async. > > BTW: I think 5MB/s of write > > speed is somewhat too slow for an IDE device, you may want to > > check the cable, etc. > > > Hmm. I tried a "dd if=3D/dev/zero of=3Da bs=3D64k count=3D1000 ; sync" > with KNOPPIX and the same slice/cable and ext3fs (mkfs with > defaults) and it was about 4 times faster than FreeBSD R5.3. A count of 1000 is far too small for this benchmark. FreeBSD will start writing data after accumulating at most 128K (64K?), but Linux (at least old versions) may cache everything. The "sync" part of the above command doesn't actually sync the data, at least under FreeBSD where sync just starts syncing of the data. The speed of a correct version of this benchmark doesn't depend much on either asyncness or soft-updateness. Asyncness in FreeBSD mainly affects metadata, but the above writes mainly data. > Furthermore the read speed is about 25Mbyte/sec, so that the cable > seems to be ok. I get 24MB/sec without soft updates or async, 25MB/sec without soft updates but with async, to separate and almost full file systems on an old disk than can only do 36MB max. FreeBSD normally does well on this benchmark. 24MB/sec is very good considering that the file systems are old and full (and thus fragmented). The performance for many small files can be much worse (10% of the disk bandwidth is too common). This is where non-sync metadata updates can help a lot. > Funnily my other hard disc (SAMSUNG SP1604N), which uses the same > cable, is much faster: > write > (bs=3D64k count=3D1000 if=3D/dev/zero): > appr. 48Mbyte/sec > read > (bs=3D64k count=3D3975 if=3D): > appr. 38Mbyte/sec (???????????) > I do not understand this different behaviour... *sniff* This is normal behaviour. It sure looks like the other disk doesn't handle writes very well. Write caching shouldn't affect this much. FreeBSD should write blocks with size mostly 64K or 128K no matter what the dd block size is, and it is up to the disk to handle such sizes well. Use iostat to see block sizes related to those seen by the disk. Bruce From owner-freebsd-fs@FreeBSD.ORG Fri Jan 28 00:06:42 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6870A16A4CE for ; Fri, 28 Jan 2005 00:06:42 +0000 (GMT) Received: from web41204.mail.yahoo.com (web41204.mail.yahoo.com [66.218.93.37]) by mx1.FreeBSD.org (Postfix) with SMTP id 0378C43D48 for ; Fri, 28 Jan 2005 00:06:42 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 11486 invoked by uid 60001); 28 Jan 2005 00:06:41 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=kbivopwtIsPrDaWkulQTIheAns+W6pT08IgENy8cTw6WqEKgK1bfIZake8F3iqCp5TiLnuIonJRyMvxIjfeDVSmkhCQWulGH030Z6q6/2n2PODXRTLBDL1PZCJmpRb6s0BeYTrL2elpPxmNyPC9eGae9cMRL/v26crqmLZaMF98= ; Message-ID: <20050128000641.11484.qmail@web41204.mail.yahoo.com> Received: from [213.54.138.99] by web41204.mail.yahoo.com via HTTP; Thu, 27 Jan 2005 16:06:41 PST Date: Thu, 27 Jan 2005 16:06:41 -0800 (PST) From: Arne "W鰎ner" To: freebsd-fs@FreeBSD.ORG In-Reply-To: <20050127040418.26913.qmail@web41205.mail.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: Re: ufs+softupdates / consistency X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2005 00:06:42 -0000 I would like to add, that I just tried my little performance test with KNOPPIX and fsync/openO_SYNC in a little C program. The results were even better (just about 3 seconds for 1000 64k blocks). With dd and the sync call it was about 6 seconds (btw.: Linux's sync waits until the harddisc acknowledged the write order requests). Since Linux's time command said, that the process was waiting for 90% of its alive-time, I believe, that the kernel was idle, too, because 1. sending the write order request is done quite quickly (I think), and 2. most of the time hard disc is busy with re-educating the magnetic layer of its discs without talking to the kernel, so that I do not fully understand the argument, that we should write just little chunks of data to the disc (is it because concurrent read/write requests (e. g.: a user process, that wants to read something, while the kernel syncs 64MB as soon as possible) might be delayed?). I would be glad, if somebody could tell me, if there is any work in progress or completed, that allows my old SEAGATE disc to perform as good as Linux allows it to perform. If not, I would like to fix the problem. -Arne W鰎ner __________________________________ Do you Yahoo!? Meet the all-new My Yahoo! - Try it today! http://my.yahoo.com From owner-freebsd-fs@FreeBSD.ORG Fri Jan 28 19:44:40 2005 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 55AFF16A4CE for ; Fri, 28 Jan 2005 19:44:40 +0000 (GMT) Received: from darkness.comp.waw.pl (darkness.comp.waw.pl [195.117.238.136]) by mx1.FreeBSD.org (Postfix) with ESMTP id D976643D3F for ; Fri, 28 Jan 2005 19:44:39 +0000 (GMT) (envelope-from pjd@darkness.comp.waw.pl) Received: by darkness.comp.waw.pl (Postfix, from userid 1009) id 7B74AACBD6; Fri, 28 Jan 2005 20:44:37 +0100 (CET) Date: Fri, 28 Jan 2005 20:44:37 +0100 From: Pawel Jakub Dawidek To: "Frank J. Beckmann" Message-ID: <20050128194437.GC795@darkness.comp.waw.pl> References: <200412092318.22325.frank@barda.agala.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="C+dxB3FEDSxkOkFg" Content-Disposition: inline In-Reply-To: <200412092318.22325.frank@barda.agala.net> User-Agent: Mutt/1.4.2i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 5.2.1-RC2 i386 cc: freebsd-fs@freebsd.org Subject: Re: How to enable ACLs on /? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2005 19:44:40 -0000 --C+dxB3FEDSxkOkFg Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Dec 09, 2004 at 11:18:18PM +0100, Frank J. Beckmann wrote: +> Hi, +>=20 +> maybe it ist a faq, but I didn't find it in the list archive. How do I e= nable=20 +> ACLs on / in FreeBSD 5.3? I tried to boot into single user mode and did= =20 +> tunefs -a enable /dev/ad4s2a and ACLs are enabled. But they are not enab= led=20 +> anymore as soon as I mount /. But as long as / is not mounted, the ACLs = show=20 +> to be enabled. There was a bug, which I fixed in -CURRENT, it will be MFCed soon. http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/ufs/ffs/ffs_vfsops.c?rev=3D1.= 269&content-type=3Dtext/x-cvsweb-markup http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/ufs/ffs/ffs_vfsops.c.diff?r1= =3D1.268&r2=3D1.269 Temporary you can add 'acls' option for root file system into your /etc/fstab. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --C+dxB3FEDSxkOkFg Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (FreeBSD) iD8DBQFB+pYlForvXbEpPzQRAlHdAJ9yZULpS6p5FyH2rbUwbrscLr1UqQCfUA89 1XqrlCjWlqBpoysLudtT+iE= =XOyN -----END PGP SIGNATURE----- --C+dxB3FEDSxkOkFg--