From owner-freebsd-fs Sun Mar 16 19:46: 4 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DDC6237B401 for ; Sun, 16 Mar 2003 19:46:02 -0800 (PST) Received: from mta04bw.bigpond.com (mta04bw.bigpond.com [139.134.6.87]) by mx1.FreeBSD.org (Postfix) with ESMTP id A856D43FB1 for ; Sun, 16 Mar 2003 19:46:00 -0800 (PST) (envelope-from areilly@bigpond.net.au) Received: from areilly.bpc-users.org ([144.135.24.78]) by mta04bw.bigpond.com (Netscape Messaging Server 4.15 mta04bw Jul 16 2002 22:47:55) with SMTP id HBVJSM00.0YE for ; Mon, 17 Mar 2003 13:45:58 +1000 Received: from CPE-144-132-188-183.nsw.bigpond.net.au ([144.132.188.183]) by bwmam04bpa.bigpond.com(MailRouter V3.2g 35/1489518); 17 Mar 2003 13:46:23 Received: (qmail 79324 invoked from network); 17 Mar 2003 03:45:58 -0000 Received: from localhost (HELO ?127.0.0.1?) (andrew@127.0.0.1) by localhost with SMTP; 17 Mar 2003 03:45:58 -0000 Subject: mount_smbfs in FreeBSD-STABLE From: Andrew Reilly To: Boris Popov Cc: fs@freebsd.org Content-Type: text/plain Organization: Message-Id: <1047872758.78456.29.camel@gurney.reilly.home> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.2.2 Date: 17 Mar 2003 14:45:58 +1100 Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Hi Boris, Firstly, enormous thanks for getting mount_smbfs going in the first place: it's tremendously useful. I have three questions, though, two of which may not be the domain of mount_smbfs at all: 1) Is it possible to use fstab for smbfs mounts when the name of the filesystem to be mounted contains a space? I haven't figured out how to escape a space there, yet. 2) Is it possible to use amd or something similar to auto-mount smbfs/cifs file systems? 3) Is it possible to have mount_smbfs authenticate with a windows domain login, rather than a workgrop login? Thanks in advance, -- Andrew Reilly To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sun Mar 16 23:43:47 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8D65537B401; Sun, 16 Mar 2003 23:43:45 -0800 (PST) Received: from mail-out1.apple.com (mail-out1.apple.com [17.254.0.52]) by mx1.FreeBSD.org (Postfix) with ESMTP id C04EA43FAF; Sun, 16 Mar 2003 23:43:44 -0800 (PST) (envelope-from conrad@apple.com) Received: from mailgate1.apple.com (A17-128-100-225.apple.com [17.128.100.225]) by mail-out1.apple.com (8.12.8/8.12.8) with ESMTP id h2H7hiOr004602; Sun, 16 Mar 2003 23:43:44 -0800 (PST) Received: from scv1.apple.com (scv1.apple.com) by mailgate1.apple.com (Content Technologies SMTPRS 4.2.1) with ESMTP id ; Sun, 16 Mar 2003 23:43:39 -0800 Received: from apple.com (minshallidsl3.apple.com [17.219.180.28]) by scv1.apple.com (8.11.3/8.11.3) with ESMTP id h2H7hh619506; Sun, 16 Mar 2003 23:43:43 -0800 (PST) Date: Sun, 16 Mar 2003 23:43:44 -0800 Subject: Re: mount_smbfs in FreeBSD-STABLE Content-Type: text/plain; charset=US-ASCII; format=flowed Mime-Version: 1.0 (Apple Message framework v551) Cc: Conrad Minshall To: Andrew Reilly , Boris Popov , fs@FreeBSD.ORG From: Conrad Minshall In-Reply-To: <1047872758.78456.29.camel@gurney.reilly.home> Message-Id: <2E513E7C-584C-11D7-8BB2-000A95757E40@apple.com> Content-Transfer-Encoding: 7bit X-Mailer: Apple Mail (2.551) Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org For item 1)... the SMB URL definition allows %nn escapes, as usual. So if the FreeBSD version of mount_smbfs supports it, you can specify spaces with a "%20". If it doesn't support it, you might drop an "unpercent" function into mount_smbfs. Here's such a function... "Darwin" has more source code context if it is needed... /* * Removes the "%" escape sequences from a URL component. * See IETF RFC 2396. */ char * unpercent(char * component) { unsigned char c, *s; unsigned hi, lo; if (component) for (s = component; (c = *s); s++) { if (c != '%') continue; if ((hi = xtoi(s[1])) > 15 || (lo = xtoi(s[2])) > 15) continue; /* ignore invalid escapes */ s[0] = hi*16 + lo; /* * This was strcpy(s + 1, s + 3); * But nowadays leftward overlapping copies are * officially undefined in C. Ours seems to * work or not depending upon alignment. */ memmove(s+1, s+3, strlen(s+3) + 1); } return (component); } On Sunday, March 16, 2003, at 07:45 PM, Andrew Reilly wrote: > Hi Boris, > > Firstly, enormous thanks for getting mount_smbfs going in the first > place: it's tremendously useful. > > I have three questions, though, two of which may not be the domain of > mount_smbfs at all: > > 1) Is it possible to use fstab for smbfs mounts when the name of the > filesystem to be mounted contains a space? I haven't figured out how > to > escape a space there, yet. > > 2) Is it possible to use amd or something similar to auto-mount > smbfs/cifs file systems? > > 3) Is it possible to have mount_smbfs authenticate with a windows > domain > login, rather than a workgrop login? > > Thanks in advance, > > -- > Andrew Reilly > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-fs" in the body of the message > > -- Conrad Minshall conrad@apple.com 408-446-2323 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 7:14:24 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5129937B414; Mon, 17 Mar 2003 07:14:16 -0800 (PST) Received: from mailgw.cscoms.com (mailgw.cscoms.com [202.183.255.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1F96C43F93; Mon, 17 Mar 2003 07:13:01 -0800 (PST) (envelope-from job2546@thaimail.com) Received: from cscoms.com (mail.cscoms.com [202.183.255.23]) by mailgw.cscoms.com (8.12.8/8.12.3) with ESMTP id h2HF1hil061448; Mon, 17 Mar 2003 22:01:44 +0700 (ICT) Received: from ME (dial-144.ras-21.bkk.c.cscoms.com [203.170.145.144]) by cscoms.com (8.12.8/8.12.3) with SMTP id h2HEptwo024896; Mon, 17 Mar 2003 21:51:56 +0700 (GMT) Date: Mon, 17 Mar 2003 21:51:55 +0700 (GMT) Message-Id: <200303171451.h2HEptwo024896@cscoms.com> From: job2546@thaimail.com Subject: "ถ้าคุณยังทำสิ่งที่คุณทำอยู่วันนี้ พรุ่งนี้ก็จะเหมือนวันนี้ X-Priority: 1 (Highest) Reply-To: job2546@thaimail.com X-Mailer: Microsoft Outlook Express 5.00.2615.200 MIME-Version: 1.0 Content-type: multipart/mixed; boundary="#MYBOUNDARY#" X-Virus-Scanned: by amavisd-milter (http://amavis.org/) To: undisclosed-recipients: ; Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org --#MYBOUNDARY# Content-Type: text/plain; charset=ansi Content-Transfer-Encoding: 8bit "หากคุณล้มเหลวที่จะวางแผน ย่อมแปลว่าคุณวางแผนที่จะล้มเหลว" จิม โรห์น นักปรัชญาอันดับ 1 ของโลก เช่น คุณคิดว่าในชีวิตนี้เราคงไม่มีทางรวย คุณก็ไจะไม่มีทางรวยเลย หรือ "คุณคิดว่าสักวันถึงฉันต้องรวยแน่ๆ" จิม โรห์น บอกว่า "ถ้าคุณยังทำสิ่งที่คุณทำอยู่ทุกวันนี้ อีก 3 ปีข้างหน้าลองคิดดูว่า คุณจะมีโอกาสรวยได้หรือไม่" "ถ้าคำตอบคือ ใช่ คุณกำลังจะรวย" ก็ยินดีกับคุณด้วยครับคุณกำลังจะรวยแล้ว "แต่ถ้าคำตอบคือ ไม่ คุณไม่สามารถรวยได้" คุณต้องเปลี่ยนอะไรสักอย่างในชีวิตคุณแล้ว จิม โรห์น บอกอีกว่า "ถ้าคุณยังทำสิ่งที่คุณทำอยู่วันนี้ พรุ่งนี้ก็จะเหมือนวันนี้ ไปเรื่อยๆไม่มีที่สิ้นสุด" หมายความว่า -ถ้าวันนี้คุณยังต้องวิ่งหาเงิน จ่ายหนี้ต่างๆ -ถ้าวันนี้คุณยังถูกเจ้านายกดขี่ ใช้งานอย่างหนัก -ถ้าวันนี้คุณยังหาทางออกไม่ได้ ลองเปิดโอกาสให้ตัวเองดู เปิดใจของคุณให้กว้างแล้วเดินตามเรามาหรือปล่อยให้โอกาสนี้หลุดลอยไป ============================================================ คุณสามารถเข้าไปดูรายละเอียดเพิ่มเติมและกรอกข้อมูลเพื่อขอรับข้อมูลเบื้องต้นฟรี ! ได้ที่ http://www.geocities.com/thaigetrich/easywork ============================================================ ขออภัยหากข้อความนี้ถูกส่งไปยังคุณโดยบังเอิญ หากคุณไม่ต้องการรับข้อความนี้อีกกรุณา mail มาที่ www.ecommerce.web1000.com/unsub --#MYBOUNDARY#-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 7:48:18 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A6E2837B401 for ; Mon, 17 Mar 2003 07:48:17 -0800 (PST) Received: from rms21.rommon.net (rms21.rommon.net [193.64.42.200]) by mx1.FreeBSD.org (Postfix) with ESMTP id DA3F443F85 for ; Mon, 17 Mar 2003 07:48:15 -0800 (PST) (envelope-from pete@he.iki.fi) Received: from PHE (h93.vuokselantie10.fi [193.64.42.147]) by rms21.rommon.net (8.12.6/8.12.6) with SMTP id h2HFmEQB032379 for ; Mon, 17 Mar 2003 17:48:14 +0200 (EET) (envelope-from pete@he.iki.fi) Message-ID: <008601c2ec9c$9ea015e0$932a40c1@PHE> From: "Petri Helenius" To: Subject: md Date: Mon, 17 Mar 2003 17:48:13 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Is it possible to allocate a file to be used as swap as /dev/md0 and then allocate a filesystem (namely /tmp) with /dev/md1 with backingstore set to swap? Trying this hung the system on seemingly infinite disk wait. Pete To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 7:57:10 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8304937B401 for ; Mon, 17 Mar 2003 07:57:09 -0800 (PST) Received: from rms21.rommon.net (rms21.rommon.net [193.64.42.200]) by mx1.FreeBSD.org (Postfix) with ESMTP id 618CA43F3F for ; Mon, 17 Mar 2003 07:57:08 -0800 (PST) (envelope-from pete@he.iki.fi) Received: from PHE (h93.vuokselantie10.fi [193.64.42.147]) by rms21.rommon.net (8.12.6/8.12.6) with SMTP id h2HFmEQB032379 for ; Mon, 17 Mar 2003 17:48:14 +0200 (EET) (envelope-from pete@he.iki.fi) Message-ID: <008601c2ec9c$9ea015e0$932a40c1@PHE> From: "Petri Helenius" To: Subject: md Date: Mon, 17 Mar 2003 17:48:13 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Is it possible to allocate a file to be used as swap as /dev/md0 and then allocate a filesystem (namely /tmp) with /dev/md1 with backingstore set to swap? Trying this hung the system on seemingly infinite disk wait. Pete To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 8:17:43 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 918A637B401 for ; Mon, 17 Mar 2003 08:17:42 -0800 (PST) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id B30EA43FA3 for ; Mon, 17 Mar 2003 08:17:40 -0800 (PST) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.8/8.12.8) with ESMTP id h2HGHcYl012116; Mon, 17 Mar 2003 17:17:39 +0100 (CET) (envelope-from phk@phk.freebsd.dk) To: "Petri Helenius" Cc: freebsd-fs@FreeBSD.ORG Subject: Re: md From: "Poul-Henning Kamp" In-Reply-To: Your message of "Mon, 17 Mar 2003 17:48:13 +0200." <008601c2ec9c$9ea015e0$932a40c1@PHE> Date: Mon, 17 Mar 2003 17:17:38 +0100 Message-ID: <12115.1047917858@critter.freebsd.dk> Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org In message <008601c2ec9c$9ea015e0$932a40c1@PHE>, "Petri Helenius" writes: > >Is it possible to allocate a file to be used as swap as /dev/md0 and then >allocate a filesystem (namely /tmp) with /dev/md1 with backingstore set to swap? > >Trying this hung the system on seemingly infinite disk wait. Once you start to do things like this, you usually get what you ask for :-) That said, in theory you should not get into trouble this way, but there may be something I overlook... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 10:34:11 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4993637B401 for ; Mon, 17 Mar 2003 10:34:10 -0800 (PST) Received: from rms21.rommon.net (rms21.rommon.net [193.64.42.200]) by mx1.FreeBSD.org (Postfix) with ESMTP id C41CB43F3F for ; Mon, 17 Mar 2003 10:34:08 -0800 (PST) (envelope-from pete@he.iki.fi) Received: from PHE (h93.vuokselantie10.fi [193.64.42.147]) by rms21.rommon.net (8.12.6/8.12.6) with SMTP id h2HIY5m4002842; Mon, 17 Mar 2003 20:34:05 +0200 (EET) (envelope-from pete@he.iki.fi) Message-ID: <00db01c2ecb3$ca4ed3e0$932a40c1@PHE> From: "Petri Helenius" To: "Poul-Henning Kamp" Cc: References: <12115.1047917858@critter.freebsd.dk> Subject: Re: md Date: Mon, 17 Mar 2003 20:34:04 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 8bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org > > Once you start to do things like this, you usually get what you ask > for :-) This kind of happened since I first configured more swap from a file and were happy for a while. Then I realized that Iดm running out of space in /tmp (which used to be swap-backed memory filesystem in the 4.X days) and did the obvious, generate swap based memory filesystem. > > That said, in theory you should not get into trouble this way, but there > may be something I overlook... > As a datapoint, I have some swap which is "regular" swap (256M) and then 2G off a file. Pete To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 12:22:40 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3D38637B404; Mon, 17 Mar 2003 12:22:38 -0800 (PST) Received: from sccrmhc02.attbi.com (sccrmhc02.attbi.com [204.127.202.62]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4320843FAF; Mon, 17 Mar 2003 12:22:37 -0800 (PST) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by sccrmhc02.attbi.com (sccrmhc02) with ESMTP id <2003031720223600200rqence>; Mon, 17 Mar 2003 20:22:36 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id MAA70592; Mon, 17 Mar 2003 12:22:34 -0800 (PST) Date: Mon, 17 Mar 2003 12:22:33 -0800 (PST) From: Julian Elischer To: FreeBSD current users Cc: fs@freebsd.org Subject: Anyone working on fsck? Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Is there anyone working on fsck? Recent timings with a fast machine with 1TB filesystems show that it takes abuot 6 hours to fsck such a filesystem (on a fast array with a lot of RAM) This is with a version of fsck that already has some locally developed speedups and changes. I have not dared time the standard one yet. Pass 1 takes 2 hours on its own. but the cpu is only about 5% busy. Looking at the access patterns on the drive suggests that there MAY be some chance to speed this up using such things reading in teh active cylinder group in advance, and performing the indirect block checks in another thread/process. Does anyone have any plans to work on this sort of thing? On another issue: The memory requirements for fsck are about 700MB+ per TB of filesystem on FreeBSD4.x With the advent of UFS2 this increases, and the filesyste,s can also get bigger. This means that a 2+TB UFS2 filesystem will be IMPOSSIBLE to check on an i386 machine. One solution is to implement an "offline, non-in-place" filesystem checker using all those techniques we all learned in CS101 relating to mag-tape merge sorts etc. basically you'd have to have a small disk partition set asside to hold working files, to which teh checker would write files of records detailing block numbers etc. then you would do merge or block sorts on those files to get them in various orders (depending on fields in the records) And recombine them to find such things as: multiple referenced blocks, and other file inconsitancies. It wouldn;t be super fast but at least it COULD be used to check a 30TB array, where the in-memory version would beed a process VM space of 24MB which is clearly impossible on a x86. Julian To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 12:33: 1 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9C10737B401; Mon, 17 Mar 2003 12:33:00 -0800 (PST) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3ABD043F85; Mon, 17 Mar 2003 12:33:00 -0800 (PST) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id 223DA2ED412; Mon, 17 Mar 2003 12:33:00 -0800 (PST) Date: Mon, 17 Mar 2003 12:33:00 -0800 From: Alfred Perlstein To: Julian Elischer Cc: FreeBSD current users , fs@freebsd.org Subject: Re: Anyone working on fsck? Message-ID: <20030317203300.GQ4145@elvis.mu.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4i Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org * Julian Elischer [030317 12:22] wrote: > > Is there anyone working on fsck? > Recent timings with a fast machine with 1TB filesystems > show that it takes abuot 6 hours to fsck such a filesystem > (on a fast array with a lot of RAM) > > This is with a version of fsck that already has some locally developed > speedups and changes. I have not dared time the standard one yet. Is this with or without the intentional delay introduced in order to avoid monopolizing the disk in background mode? -Alfred To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 12:40:20 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 92FBC37B401; Mon, 17 Mar 2003 12:40:19 -0800 (PST) Received: from sccrmhc03.attbi.com (sccrmhc03.attbi.com [204.127.202.63]) by mx1.FreeBSD.org (Postfix) with ESMTP id B451A43F3F; Mon, 17 Mar 2003 12:40:16 -0800 (PST) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by sccrmhc03.attbi.com (sccrmhc03) with ESMTP id <2003031720401500300kvef5e>; Mon, 17 Mar 2003 20:40:15 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id MAA70726; Mon, 17 Mar 2003 12:40:14 -0800 (PST) Date: Mon, 17 Mar 2003 12:40:12 -0800 (PST) From: Julian Elischer To: Alfred Perlstein Cc: FreeBSD current users , fs@freebsd.org Subject: Re: Anyone working on fsck? In-Reply-To: <20030317203300.GQ4145@elvis.mu.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org this is a full 100% forground fsck -y On Mon, 17 Mar 2003, Alfred Perlstein wrote: > * Julian Elischer [030317 12:22] wrote: > > > > Is there anyone working on fsck? > > Recent timings with a fast machine with 1TB filesystems > > show that it takes abuot 6 hours to fsck such a filesystem > > (on a fast array with a lot of RAM) > > > > This is with a version of fsck that already has some locally developed > > speedups and changes. I have not dared time the standard one yet. > > Is this with or without the intentional delay introduced in order > to avoid monopolizing the disk in background mode? > > -Alfred > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 12:45:20 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 169A037B401; Mon, 17 Mar 2003 12:45:19 -0800 (PST) Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by mx1.FreeBSD.org (Postfix) with ESMTP id 53B7343FB1; Mon, 17 Mar 2003 12:45:18 -0800 (PST) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by rwcrmhc52.attbi.com (rwcrmhc52) with ESMTP id <2003031720451705200ptglre>; Mon, 17 Mar 2003 20:45:17 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id MAA70776; Mon, 17 Mar 2003 12:45:17 -0800 (PST) Date: Mon, 17 Mar 2003 12:45:15 -0800 (PST) From: Julian Elischer To: Alfred Perlstein Cc: FreeBSD current users , fs@freebsd.org Subject: Re: Anyone working on fsck? In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org I might add that the test filesystem was 95% full with about 8,000,000 directories on it. It was populated with multiple copies of /bin and /etc as a test set :-) On Mon, 17 Mar 2003, Julian Elischer wrote: > this is a full 100% forground fsck -y > > On Mon, 17 Mar 2003, Alfred Perlstein wrote: > > > * Julian Elischer [030317 12:22] wrote: > > > > > > Is there anyone working on fsck? > > > Recent timings with a fast machine with 1TB filesystems > > > show that it takes abuot 6 hours to fsck such a filesystem > > > (on a fast array with a lot of RAM) > > > > > > This is with a version of fsck that already has some locally developed > > > speedups and changes. I have not dared time the standard one yet. > > > > Is this with or without the intentional delay introduced in order > > to avoid monopolizing the disk in background mode? > > > > -Alfred > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-fs" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 12:58:37 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8333837B401; Mon, 17 Mar 2003 12:58:35 -0800 (PST) Received: from energyhq.homeip.net (213-97-200-73.uc.nombres.ttd.es [213.97.200.73]) by mx1.FreeBSD.org (Postfix) with ESMTP id 737FF43F75; Mon, 17 Mar 2003 12:58:31 -0800 (PST) (envelope-from flynn@energyhq.homeip.net) Received: from christine.energyhq.tk (christine.energyhq.tk [192.168.100.1]) by energyhq.homeip.net (Postfix) with SMTP id 19444AF5D3; Mon, 17 Mar 2003 21:58:29 +0100 (CET) Date: Mon, 17 Mar 2003 21:59:53 +0100 From: Miguel Mendez To: Julian Elischer Cc: FreeBSD current users , fs@freebsd.org Subject: Re: Anyone working on fsck? Message-Id: <20030317215953.49aa5cd1.flynn@energyhq.homeip.net> In-Reply-To: References: X-Mailer: Sylpheed version 0.8.11claws (GTK+ 1.2.10; i386--netbsdelf) X-Face: 1j}k*2E>Y\+C~E|/wehi[:dCM,{N7/uE3o# P,{t7gA/qnovFDDuyQV.1hdT7&#d)q"xY33}{_GS>kk'S{O]nE$A`T|\4&p\&mQyexOLb8}FO List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org --=.uaQk/vJHQv,bDY Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit On Mon, 17 Mar 2003 12:22:33 -0800 (PST) Julian Elischer wrote: Howdy, > It wouldn;t be super fast but at least it COULD be used to check a > 30TB array, where the in-memory version would beed a process VM space > of 24MB which is clearly impossible on a x86. I'm sure most of us have computers with more than 24MB :-P, sorry couldn't resist. It's an interesting problem anyway, does anyone have insight on how other OSs and FS handle these kind of problems? e.g. Solaris with vxfs or Linux with XFS/JFS/this week's FS. Cheers, -- Miguel Mendez - flynn@energyhq.homeip.net GPG Public Key :: http://energyhq.homeip.net/files/pubkey.txt EnergyHQ :: http://www.energyhq.tk Of course it runs NetBSD! Tired of Spam? -> http://www.trustic.com --=.uaQk/vJHQv,bDY Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (NetBSD) iD8DBQE+djdSnLctrNyFFPERAqy7AKC6Lc7SZyZ21M7uR5hIJvW9WM2ZMgCgyOZo n5PuqPcZMcHt8unWJbUPPOc= =hFif -----END PGP SIGNATURE----- --=.uaQk/vJHQv,bDY-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 13:27: 1 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B4C1A37B404; Mon, 17 Mar 2003 13:26:59 -0800 (PST) Received: from thunderer.cnchost.com (thunderer.concentric.net [207.155.252.72]) by mx1.FreeBSD.org (Postfix) with ESMTP id CDE1543F75; Mon, 17 Mar 2003 13:26:58 -0800 (PST) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (adsl-209-204-185-216.sonic.net [209.204.185.216]) by thunderer.cnchost.com id QAA23205; Mon, 17 Mar 2003 16:26:56 -0500 (EST) [ConcentricHost SMTP Relay 1.15] Message-ID: <200303172126.QAA23205@thunderer.cnchost.com> To: Julian Elischer Cc: FreeBSD current users , fs@FreeBSD.ORG Subject: Re: Anyone working on fsck? In-reply-to: Your message of "Mon, 17 Mar 2003 12:22:33 PST." Date: Mon, 17 Mar 2003 13:26:56 -0800 From: Bakul Shah Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org UFS is the real problem here, not fsck. Its tradeoffs for improving normal access latencies may have been right in the past but not for modern big disks. The seek time & RPM have not improved very much in the past 20 years while disk capacity has increased by a factor of about 20,000 (and GB/$ even more). IMHO there is not much you can do at the fsck level -- you stil have to visit all the cyl groups and what not. Even a factor of 10 improvement in fsck means 36 minutes which is far too long. Keeping track of 67M to 132M blocks and (assuming avg file size of 8k to 16k) something like 60M to 80M files takes quite a bit of time when you are also seeking all over the disk. A few ideas: When you have about 67M (2^26) files, ideally you want to *avoid* checking as many as you can. Given access times, you are only going to be able to do a few hundred disk accesses at most in a minute. So you are going to have only a few files/dirs that may be inconsistent in case of a crash. Why not keep track of that somehow? If you need about 1GB of space to store the state of a TB file system that needs to be checked, may be it _should_ be *stored* in a contiguous area on the FS itself. 1GB is about 0.1% of space. Typically only a few cyl grps may be inconsistent in case of a crash. May be some info about which cyl groups need to be checked can be stored so that brute force checking of all grps can be avoided. Typically a file will be stored in one or a small number of cyl groups. If that info. is stored somewhere it can speed things up. Extant based allocation will reduce the number of indirect blocks. But may be this is not such a big issue if most of your files fit in a few blocks. Anyway, support for all of these have to be done in the filesystem first before fsck can benefit. If instead you spend time "optimizing" just fsck, you will likely make it far more complex (and potentially harder to get right). To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 13:39:15 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 28A1D37B401; Mon, 17 Mar 2003 13:39:14 -0800 (PST) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 242A243F85; Mon, 17 Mar 2003 13:39:13 -0800 (PST) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.8/8.12.8) with ESMTP id h2HLd2Yl054292; Mon, 17 Mar 2003 22:39:03 +0100 (CET) (envelope-from phk@phk.freebsd.dk) To: Bakul Shah Cc: Julian Elischer , FreeBSD current users , fs@FreeBSD.ORG Subject: Re: Anyone working on fsck? From: "Poul-Henning Kamp" In-Reply-To: Your message of "Mon, 17 Mar 2003 13:26:56 PST." <200303172126.QAA23205@thunderer.cnchost.com> Date: Mon, 17 Mar 2003 22:39:02 +0100 Message-ID: <54291.1047937142@critter.freebsd.dk> Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org In message <200303172126.QAA23205@thunderer.cnchost.com>, Bakul Shah writes: >UFS is the real problem here, not fsck. Its tradeoffs for >improving normal access latencies may have been right in the >past but not for modern big disks. The seek time & RPM have >not improved very much in the past 20 years while disk >capacity has increased by a factor of about 20,000 (and GB/$ >even more). IMHO there is not much you can do at the fsck >level -- you stil have to visit all the cyl groups and what >not. Even a factor of 10 improvement in fsck means 36 >minutes which is far too long. Now, before we go off and design YABFS, can we just get real for a second ? I have been tending UNIX computers of all sorts for many years and there is one bit of wisdom that has yet to fail me: Every now and then, boot in single-user and run full fsck on all filesystems. If this had failed to be productive, I would have given up the habit years ago, but it is still a good idea it seems. Personally, I think background-fsck is close to the ideal situation since I can skip the "boot in single-user" part of the above profylactic. If you start to implement any sort of journaling (that is what you talked about in your email), you might as well just stop right at the "clean" bit, and avoid the complexity. Optimizing fsck is a valid project, I just wish it would be somebody who would also finish the last 30% who would do it. Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 13:52: 6 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 20E0737B404; Mon, 17 Mar 2003 13:52:04 -0800 (PST) Received: from rwcrmhc53.attbi.com (rwcrmhc53.attbi.com [204.127.198.39]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8A5CC43F75; Mon, 17 Mar 2003 13:52:03 -0800 (PST) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by rwcrmhc53.attbi.com (rwcrmhc53) with ESMTP id <2003031721520205300ji8mre>; Mon, 17 Mar 2003 21:52:03 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id NAA71305; Mon, 17 Mar 2003 13:52:01 -0800 (PST) Date: Mon, 17 Mar 2003 13:51:59 -0800 (PST) From: Julian Elischer To: Bakul Shah Cc: FreeBSD current users , fs@FreeBSD.ORG Subject: Re: Anyone working on fsck? In-Reply-To: <200303172126.QAA23205@thunderer.cnchost.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Thanks for your thoughts. . Some good points.. On Mon, 17 Mar 2003, Bakul Shah wrote: > UFS is the real problem here, not fsck. Its tradeoffs for > improving normal access latencies may have been right in the > past but not for modern big disks. The seek time & RPM have > not improved very much in the past 20 years while disk > capacity has increased by a factor of about 20,000 (and GB/$ > even more). IMHO there is not much you can do at the fsck > level -- you stil have to visit all the cyl groups and what > not. Even a factor of 10 improvement in fsck means 36 > minutes which is far too long. Keeping track of 67M to 132M > blocks and (assuming avg file size of 8k to 16k) something > like 60M to 80M files takes quite a bit of time when you are > also seeking all over the disk. Puting indirect blocks nearer the cylinder group metadata for the group that contains the inode would be nice :-) > > A few ideas: > > When you have about 67M (2^26) files, ideally you want to > *avoid* checking as many as you can. Given access times, you > are only going to be able to do a few hundred disk accesses > at most in a minute. So you are going to have only a few > files/dirs that may be inconsistent in case of a crash. Why > not keep track of that somehow? In the case of HW failure on a raid you REALLY need to checke everything. You don't trust anything.. > > If you need about 1GB of space to store the state of a TB > file system that needs to be checked, may be it _should_ be > *stored* in a contiguous area on the FS itself. 1GB is about > 0.1% of space. It is no trouble for us to set asside a separate filesystem, or partition for this.. > > Typically only a few cyl grps may be inconsistent in case of > a crash. May be some info about which cyl groups need to be > checked can be stored so that brute force checking of all > grps can be avoided. > > Typically a file will be stored in one or a small number of > cyl groups. If that info. is stored somewhere it can speed > things up. > > Extant based allocation will reduce the number of indirect > blocks. But may be this is not such a big issue if most of > your files fit in a few blocks. > > Anyway, support for all of these have to be done in the > filesystem first before fsck can benefit. > > If instead you spend time "optimizing" just fsck, you will > likely make it far more complex (and potentially harder to > get right). > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 13:56:29 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3EE6737B401; Mon, 17 Mar 2003 13:56:28 -0800 (PST) Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9035C43F3F; Mon, 17 Mar 2003 13:56:27 -0800 (PST) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by rwcrmhc51.attbi.com (rwcrmhc51) with ESMTP id <20030317215627051004i5b8e>; Mon, 17 Mar 2003 21:56:27 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id NAA71343; Mon, 17 Mar 2003 13:56:26 -0800 (PST) Date: Mon, 17 Mar 2003 13:56:24 -0800 (PST) From: Julian Elischer To: Bakul Shah Cc: FreeBSD current users , fs@FreeBSD.ORG Subject: Re: Anyone working on fsck? In-Reply-To: <200303172126.QAA23205@thunderer.cnchost.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Mon, 17 Mar 2003, Bakul Shah wrote: > Anyway, support for all of these have to be done in the > filesystem first before fsck can benefit. yep > If instead you spend time "optimizing" just fsck, you will > likely make it far more complex (and potentially harder to > get right). You talk like I have a choice :-) I cannot change ufs/ffs and even if I could the clients wouldn't go for it. The problem space is Fsck of UFS/FFS partitions is too slow for 200GB+ filesystems. The solution space can not contain any answer that includes redefining UFS/FFS. Welcome to the real world. :-) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 14: 5:18 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C19A437B401; Mon, 17 Mar 2003 14:05:16 -0800 (PST) Received: from goliath.cnchost.com (goliath.cnchost.com [207.155.252.47]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3716443F93; Mon, 17 Mar 2003 14:05:16 -0800 (PST) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (adsl-209-204-185-216.sonic.net [209.204.185.216]) by goliath.cnchost.com id RAA18117; Mon, 17 Mar 2003 17:05:02 -0500 (EST) [ConcentricHost SMTP Relay 1.15] Message-ID: <200303172205.RAA18117@goliath.cnchost.com> To: "Poul-Henning Kamp" Cc: Julian Elischer , FreeBSD current users , fs@FreeBSD.ORG Subject: Re: Anyone working on fsck? In-reply-to: Your message of "Mon, 17 Mar 2003 22:39:02 +0100." <54291.1047937142@critter.freebsd.dk> Date: Mon, 17 Mar 2003 14:05:02 -0800 From: Bakul Shah Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org > Now, before we go off and design YABFS, can we just get real for > a second ? I leave it to others to design YAFS, I just wanted to complain about this one :-) Every few years I seriously look at speeding up fsck but give up. I remember even asking about it a few years ago on one of these groups. > I have been tending UNIX computers of all sorts for many years and > there is one bit of wisdom that has yet to fail me: > > Every now and then, boot in single-user and run full fsck > on all filesystems. > > If this had failed to be productive, I would have given up the > habit years ago, but it is still a good idea it seems. Even now I use fsck in forground since the background fsck was not stable enough the last time I used it. But I remember thinking fsck was taking too long for as long as I have used it (since 1981). > Personally, I think background-fsck is close to the ideal situation > since I can skip the "boot in single-user" part of the above > profylactic. Anything that runs for half hour or more in fg is likely to take longer in bg. What happens if the system crashes again before it finishes? Will bg fsck handle that? Am I right in thinking that it can not save files in /lost+found? In general I am very uneasy with bg fsck -- when I am validating something I don't want to be creating new stuff. > If you start to implement any sort of journaling (that is what you > talked about in your email), you might as well just stop right at > the "clean" bit, and avoid the complexity. No, I didn't suggest journaling, I suggested storing all state in a contiguous area (or a small number of such areas). indirect blocks, keeping track of free blocks, etc. You can still do a completely exhaustive fsck but it won't be exhausting to you. Journaling is also a good idea:-) > Optimizing fsck is a valid project, I just wish it would be somebody > who would also finish the last 30% who would do it. I am skeptical you will get more than a factor of 2 improvement without changing the FS (but hey, that is 3 hours for Julian so I am sure he will be happy with that!). To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 14: 8:49 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 48A5237B404; Mon, 17 Mar 2003 14:08:48 -0800 (PST) Received: from goliath.cnchost.com (goliath.cnchost.com [207.155.252.47]) by mx1.FreeBSD.org (Postfix) with ESMTP id 77F5A43F93; Mon, 17 Mar 2003 14:08:46 -0800 (PST) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (adsl-209-204-185-216.sonic.net [209.204.185.216]) by goliath.cnchost.com id RAA27811; Mon, 17 Mar 2003 17:08:32 -0500 (EST) [ConcentricHost SMTP Relay 1.15] Message-ID: <200303172208.RAA27811@goliath.cnchost.com> To: Julian Elischer Cc: Bakul Shah , FreeBSD current users , fs@FreeBSD.ORG Subject: Re: Anyone working on fsck? In-reply-to: Your message of "Mon, 17 Mar 2003 13:56:24 PST." Date: Mon, 17 Mar 2003 14:08:31 -0800 From: Bakul Shah Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org > You talk like I have a choice :-) > I cannot change ufs/ffs and even if I could the clients wouldn't go for > it. What about changing the size of block size or cyl grp size? Do they change things much? > The problem space is > > Fsck of UFS/FFS partitions is too slow for 200GB+ filesystems. > The solution space can not contain any answer that includes redefining > UFS/FFS. Welcome to the real world. :-) I am so glad I have a separate machine for every few GB of disk space :-) So may be you can have on a multi-processor solution. I'll try to come up with more useful suggestions given your constraints.... To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 14:13: 9 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ADC7837B401; Mon, 17 Mar 2003 14:13:07 -0800 (PST) Received: from odin.ac.hmc.edu (Odin.AC.HMC.Edu [134.173.32.75]) by mx1.FreeBSD.org (Postfix) with ESMTP id D876243F3F; Mon, 17 Mar 2003 14:13:06 -0800 (PST) (envelope-from brdavis@odin.ac.hmc.edu) Received: from odin.ac.hmc.edu (IDENT:brdavis@localhost.localdomain [127.0.0.1]) by odin.ac.hmc.edu (8.12.3/8.12.3) with ESMTP id h2HMD2Ju032666; Mon, 17 Mar 2003 14:13:02 -0800 Received: (from brdavis@localhost) by odin.ac.hmc.edu (8.12.3/8.12.3/Submit) id h2HMD2uG032665; Mon, 17 Mar 2003 14:13:02 -0800 Date: Mon, 17 Mar 2003 14:13:02 -0800 From: Brooks Davis To: Julian Elischer Cc: Alfred Perlstein , FreeBSD current users , fs@FreeBSD.ORG Subject: Re: Anyone working on fsck? Message-ID: <20030317141302.A29659@Odin.AC.HMC.Edu> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="MGYHOYXEY6WxJCY8" Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: ; from julian@elischer.org on Mon, Mar 17, 2003 at 12:45:15PM -0800 X-Virus-Scanned: by amavisd-milter (http://amavis.org/) on odin.ac.hmc.edu Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org --MGYHOYXEY6WxJCY8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Mar 17, 2003 at 12:45:15PM -0800, Julian Elischer wrote: > I might add that the test filesystem was 95% full with about 8,000,000 > directories on it. It was populated with multiple copies of /bin=20 > and /etc as a test set :-) How much like you're real file mix does this look? If your real mix doesn't require this many files it may not be so bad. I've got an 800GB SCSI-IDE RAID box[0] with a single UFS file system with about 520GB of mirrors on it and it fscks in ~40min. I am still intrested in improvements to fsck since I'm planning to buy several systems with two 1.4TB IDE RAID5 arrays in them soon. -- Brooks Promise UltraTrak RM8000 with 8 120GB disks in a RAID5. --=20 Any statement of the form "X is the one, true Y" is FALSE. PGP fingerprint 655D 519C 26A7 82E7 2529 9BF0 5D8E 8BE9 F238 1AD4 --MGYHOYXEY6WxJCY8 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (GNU/Linux) iD8DBQE+dkhtXY6L6fI4GtQRArRWAKDMtFV/aZqKwFVwB/rux+KlGzVMzgCeIEt/ 8as0z3liFaKRqiF5My9eVMg= =Su6T -----END PGP SIGNATURE----- --MGYHOYXEY6WxJCY8-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 14:25:47 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C343C37B401; Mon, 17 Mar 2003 14:25:46 -0800 (PST) Received: from rwcrmhc53.attbi.com (rwcrmhc53.attbi.com [204.127.198.39]) by mx1.FreeBSD.org (Postfix) with ESMTP id 306A943F3F; Mon, 17 Mar 2003 14:25:44 -0800 (PST) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by rwcrmhc53.attbi.com (rwcrmhc53) with ESMTP id <2003031722254305300da3qje>; Mon, 17 Mar 2003 22:25:43 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id OAA71643; Mon, 17 Mar 2003 14:25:32 -0800 (PST) Date: Mon, 17 Mar 2003 14:25:30 -0800 (PST) From: Julian Elischer To: Bakul Shah Cc: FreeBSD current users , fs@FreeBSD.ORG Subject: Re: Anyone working on fsck? In-Reply-To: <200303172205.RAA18117@goliath.cnchost.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Mon, 17 Mar 2003, Bakul Shah wrote: > > Now, before we go off and design YABFS, can we just get real for > > a second ? > > > I am skeptical you will get more than a factor of 2 > improvement without changing the FS (but hey, that is 3 hours > for Julian so I am sure he will be happy with that!). I doubt I can get even 30% but I did just cut off 12% off pass1 by adding a pre-read of the cylinder group. (2 hours -> 1hour 45 minutes) (it's a start) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 14:31: 9 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1901F37B401; Mon, 17 Mar 2003 14:31:08 -0800 (PST) Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id B73D343FB1; Mon, 17 Mar 2003 14:31:06 -0800 (PST) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id h2HMUua90483; Mon, 17 Mar 2003 17:30:56 -0500 (EST) (envelope-from jroberson@chesapeake.net) Date: Mon, 17 Mar 2003 17:30:56 -0500 (EST) From: Jeff Roberson To: Brooks Davis Cc: Julian Elischer , Alfred Perlstein , FreeBSD current users , Subject: Re: Anyone working on fsck? In-Reply-To: <20030317141302.A29659@Odin.AC.HMC.Edu> Message-ID: <20030317172953.O66343-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Mon, 17 Mar 2003, Brooks Davis wrote: > On Mon, Mar 17, 2003 at 12:45:15PM -0800, Julian Elischer wrote: > > I might add that the test filesystem was 95% full with about 8,000,000 > > directories on it. It was populated with multiple copies of /bin > > and /etc as a test set :-) > > How much like you're real file mix does this look? If your real mix > doesn't require this many files it may not be so bad. I've got an 800GB > SCSI-IDE RAID box[0] with a single UFS file system with about 520GB of > mirrors on it and it fscks in ~40min. > > I am still intrested in improvements to fsck since I'm planning to buy > several systems with two 1.4TB IDE RAID5 arrays in them soon. > For these types of systems doing a block caching layer with a prefetch that understands how many spindles there are would be a huge benefit. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 15:35:19 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ABFC437B401; Mon, 17 Mar 2003 15:35:17 -0800 (PST) Received: from c3po.skynet.be (c3po.skynet.be [195.238.3.237]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3C85A43F3F; Mon, 17 Mar 2003 15:35:16 -0800 (PST) (envelope-from brad.knowles@skynet.be) Received: from [10.0.1.2] (105.195-200-80.adsl.skynet.be [80.200.195.105]) by c3po.skynet.be (8.12.8/8.12.8/Skynet-OUT-2.21) with ESMTP id h2HNYxxq010529; Tue, 18 Mar 2003 00:35:03 +0100 (MET) (envelope-from ) Mime-Version: 1.0 X-Sender: bs663385@pop.skynet.be Message-Id: In-Reply-To: <54291.1047937142@critter.freebsd.dk> References: <54291.1047937142@critter.freebsd.dk> Date: Mon, 17 Mar 2003 23:39:59 +0100 To: "Poul-Henning Kamp" From: Brad Knowles Subject: Re: Anyone working on fsck? Cc: Bakul Shah , Julian Elischer , FreeBSD current users , fs@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org At 10:39 PM +0100 2003/03/17, Poul-Henning Kamp wrote: > Optimizing fsck is a valid project, I just wish it would be somebody > who would also finish the last 30% who would do it. Just what are you saying? Is Julian Elischer not the right person to be working on this, because he has a history of not finishing the last 30% of something? -- Brad Knowles, "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin, Historical Review of Pennsylvania. GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+ !w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++) tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 15:48:45 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6161137B401; Mon, 17 Mar 2003 15:48:40 -0800 (PST) Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.80]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7B9C843F85; Mon, 17 Mar 2003 15:48:38 -0800 (PST) (envelope-from grog@lemis.com) Received: by wantadilla.lemis.com (Postfix, from userid 1004) id 8C5F451A62; Tue, 18 Mar 2003 10:18:36 +1030 (CST) Date: Tue, 18 Mar 2003 10:18:36 +1030 From: Greg 'groggy' Lehey To: Poul-Henning Kamp Cc: Bakul Shah , Julian Elischer , FreeBSD current users , fs@FreeBSD.ORG Subject: Re: Anyone working on fsck? Message-ID: <20030317234836.GG9422@wantadilla.lemis.com> References: <200303172126.QAA23205@thunderer.cnchost.com> <54291.1047937142@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="sClP8c1IaQxyux9v" Content-Disposition: inline In-Reply-To: <54291.1047937142@critter.freebsd.dk> User-Agent: Mutt/1.4i Organization: The FreeBSD Project Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.FreeBSD.org/ X-PGP-Fingerprint: 9A1B 8202 BCCE B846 F92F 09AC 22E6 F290 507A 4223 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org --sClP8c1IaQxyux9v Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Monday, 17 March 2003 at 22:39:02 +0100, Poul-Henning Kamp wrote: > In message <200303172126.QAA23205@thunderer.cnchost.com>, Bakul Shah writes: > >> UFS is the real problem here, not fsck. Its tradeoffs for >> improving normal access latencies may have been right in the >> past but not for modern big disks. The seek time & RPM have >> not improved very much in the past 20 years while disk >> capacity has increased by a factor of about 20,000 (and GB/$ >> even more). IMHO there is not much you can do at the fsck >> level -- you stil have to visit all the cyl groups and what >> not. Even a factor of 10 improvement in fsck means 36 >> minutes which is far too long. > > Now, before we go off and design YABFS, can we just get real for > a second ? > > I have been tending UNIX computers of all sorts for many years and > there is one bit of wisdom that has yet to fail me: > > Every now and then, boot in single-user and run full fsck > on all filesystems. > > If this had failed to be productive, I would have given up the > habit years ago, but it is still a good idea it seems. > > Personally, I think background-fsck is close to the ideal situation > since I can skip the "boot in single-user" part of the above > profylactic. > > If you start to implement any sort of journaling (that is what you > talked about in your email), you might as well just stop right at > the "clean" bit, and avoid the complexity. > > Optimizing fsck is a valid project, I just wish it would be somebody > who would also finish the last 30% who would do it. Poul-Henning, how can you justify the second half of that sentence? I take exception to the implications. In case anybody is in any doubt, I've heard you say this sort of thing about julian before. Please don't do it again. This is without my core hat. As most people here know, core has warned you about this kind of behaviour multiple times before. What I say here in no way prejudices what core may decide to do about the incident. Greg -- See complete headers for address and phone numbers --sClP8c1IaQxyux9v Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.0 (FreeBSD) iD8DBQE+dl7UIubykFB6QiMRAthBAKCoHeaOjJA0BhrK3DGHsMo4/mxeRQCcCk+0 vFHr1g1z5HkRNh25+bE9yd4= =nfRO -----END PGP SIGNATURE----- --sClP8c1IaQxyux9v-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 16: 5:25 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9111237B401; Mon, 17 Mar 2003 16:05:23 -0800 (PST) Received: from rwcrmhc53.attbi.com (rwcrmhc53.attbi.com [204.127.198.39]) by mx1.FreeBSD.org (Postfix) with ESMTP id 01D1143F75; Mon, 17 Mar 2003 16:05:23 -0800 (PST) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by rwcrmhc53.attbi.com (rwcrmhc53) with ESMTP id <20030318000522053001ukfge>; Tue, 18 Mar 2003 00:05:22 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id QAA72331; Mon, 17 Mar 2003 16:05:21 -0800 (PST) Date: Mon, 17 Mar 2003 16:05:19 -0800 (PST) From: Julian Elischer To: Brooks Davis Cc: Alfred Perlstein , FreeBSD current users , fs@FreeBSD.ORG Subject: Re: Anyone working on fsck? In-Reply-To: <20030317141302.A29659@Odin.AC.HMC.Edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Mon, 17 Mar 2003, Brooks Davis wrote: > On Mon, Mar 17, 2003 at 12:45:15PM -0800, Julian Elischer wrote: > > I might add that the test filesystem was 95% full with about 8,000,000 > > directories on it. It was populated with multiple copies of /bin > > and /etc as a test set :-) > > How much like you're real file mix does this look? If your real mix > doesn't require this many files it may not be so bad. I've got an 800GB > SCSI-IDE RAID box[0] with a single UFS file system with about 520GB of > mirrors on it and it fscks in ~40min. This is typical of the real file mix, actually it's probably a bit pesimistic but only by a small margin.. (I needed 'worst case' figures) > > I am still intrested in improvements to fsck since I'm planning to buy > several systems with two 1.4TB IDE RAID5 arrays in them soon. > > -- Brooks > > Promise UltraTrak RM8000 with 8 120GB disks in a RAID5. > > -- > Any statement of the form "X is the one, true Y" is FALSE. > PGP fingerprint 655D 519C 26A7 82E7 2529 9BF0 5D8E 8BE9 F238 1AD4 > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 19:43: 8 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D904737B401; Mon, 17 Mar 2003 19:43:03 -0800 (PST) Received: from bluejay.mail.pas.earthlink.net (bluejay.mail.pas.earthlink.net [207.217.120.218]) by mx1.FreeBSD.org (Postfix) with ESMTP id 08E5B43F3F; Mon, 17 Mar 2003 19:43:03 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0217.cvx22-bradley.dialup.earthlink.net ([209.179.198.217] helo=mindspring.com) by bluejay.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18v806-0000xC-00; Mon, 17 Mar 2003 19:42:55 -0800 Message-ID: <3E76956E.5E0FCC6B@mindspring.com> Date: Mon, 17 Mar 2003 19:41:34 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Bakul Shah Cc: Julian Elischer , FreeBSD current users , fs@FreeBSD.ORG Subject: Re: Anyone working on fsck? References: <200303172126.QAA23205@thunderer.cnchost.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a41dbd95575530f85a57bab592527d699e350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Bakul Shah wrote: > UFS is the real problem here, not fsck. Its tradeoffs for > improving normal access latencies may have been right in the > past but not for modern big disks. The seek time & RPM have > not improved very much in the past 20 years while disk > capacity has increased by a factor of about 20,000 (and GB/$ > even more). IMHO there is not much you can do at the fsck > level -- you stil have to visit all the cyl groups and what > not. Even a factor of 10 improvement in fsck means 36 > minutes which is far too long. Keeping track of 67M to 132M > blocks and (assuming avg file size of 8k to 16k) something > like 60M to 80M files takes quite a bit of time when you are > also seeking all over the disk. Sorry, but the track-to-track seek latency optimizations you are referring to are turned off, given the newfs defaults, and have been for a very long time. Please see revision 1.7 of /usr/src/sbin/newfs/newfs.c: revision 1.7 date: 1995/02/05 08:42:31; author: phk; state: Exp; lines: +13 -2 Basically, the problem is that for a BG fsck, it's not possible to lock access on a cylinder group basis, and then there is a hell of a lot of data on that drive. What Julian needs is a Journalling or Log-structured FS. Even that will not save him in some failure cases, unless the hardware has a CMOS data area that can be written with the failure cause, even if it's a double panic (must distinguish power failure from kernel panic, as kernel panics result from corrupt kernel data, which means you must check all files). > When you have about 67M (2^26) files, ideally you want to > *avoid* checking as many as you can. Given access times, you > are only going to be able to do a few hundred disk accesses > at most in a minute. So you are going to have only a few > files/dirs that may be inconsistent in case of a crash. Why > not keep track of that somehow? The BG fsck code specifically *only* checks the CG bitmap allocations. Basically, this means that anything that might be using the CG has to be checked to see if it references blocks in the bitmaps. The easiest way to do this is to provide some sort of domain control, so that you limit the files you have to check to a smaller set of files per CG, so that you don't have to check all the files to check a particular CG -- and then preload the CG's for the sets of files you *do* have to check. Another way of saying this is... don't put all your space in a single FS. 8-). The problem with having to (effectively) read every inode and direct block on the disk is really insurmountable, I think. There are some FS design changes that could be made to "fix" the issue, by partitioning CG's by locality, and then providing locality forward references as a seperate bitmap, and forcing files to stick to their locality until they can't, (basically, the AIX GFS PP and LP voume management trick), but that would require a layout change and an additional dependency insertion. Too bad the soft updates implementation isn't a generic graph dependency mechanism, with node/node dependency resolvers that get registered for edges, or you could do this easily (as well as implying edges between stacking layers, and exporting a transaction interface to user space). > Typically only a few cyl grps may be inconsistent in case of > a crash. May be some info about which cyl groups need to be > checked can be stored so that brute force checking of all > grps can be avoided. This would work well... under laboratory test conditions. In the field, if the machine has a more general workload (or even a heterogeneous workload, but with a hell of a lot of files, like a big mail server), this falls down, as the number of bits marking "unupdated cylinder groups" becomes large. ...AND... The problem is still that you must scan everything on the disk (practically) to identify the inode or indirect block that references the block on the cylinder roup in question, and THAT's the problem. If you knew a small set of CG's, that needed checked, vs. "all of them", it would *still* bust the cache, which is what takes all the time. Assume, on average, each file goes into indirect blocks. That's basically an N^3 algorithm to find a given block for a given CG; and the problem is the cache busting. Reducing this to N^2*M, where M < N, or even M << N, will not significantly speed up the process; e.g.: ,-------.***** |+++++++|***** * Where you miss |+++,--------. + Where you have to repeat `---| | ****| | ****| | ****`--------' ...So even if you pick your repetition area right (you could have accidently picked the big one instead of the small one, if the files were very large, or there was a very large number of small ones), you don't save a lot. > Typically a file will be stored in one or a small number of > cyl groups. If that info. is stored somewhere it can speed > things up. The problem is reverse mapping a bit in a CG bitmap to the file that reference it... 8^p. > Extant based allocation will reduce the number of indirect > blocks. But may be this is not such a big issue if most of > your files fit in a few blocks. Extents would save considerably, since you could target specific cylinger group bitmaps, cached in RAM, by section, for processing, and the amount of references you had to make to know if a given inode had an extent covering a region controlled by a given bitmap would be reduced. However, this would only be a win, if you knew, a priori, that you were storing mostly very big files. Even so, you are still scanning every inode on the system to get the extent list in order to get the CG intersection to verify a given bit. > Anyway, support for all of these have to be done in the > filesystem first before fsck can benefit. > > If instead you spend time "optimizing" just fsck, you will > likely make it far more complex (and potentially harder to > get right). Multithreading fsck would be an incredibly bad idea. Being able to control the locality of the search process is your primary means of controlling the amount of cache busting that can happen (see the "squares" diagram, above). The win you get, if any, has to come from preloading CG bitmaps into RAM, effectively, so that the process doesn't end up swapping. Multithreading this destroys your locality of reference. The 10 second delay in starting fack's on different FS's in fsck right now is probably well past time to be removed. The "-l" option would have been useful here, but it's been removed. If the disks are on the same spindle, you still destroy locality by seeking between regions while it's going. BTW: someone needs to update the man page, particularly the "SYNOPSIS". If this is a BG fsck on an FS, you might want to consider something like an "MADV_WIRE" argument to madvise(2), for the CG bitmaps in core, so that the pages don't end up being forced in and out of swap through normal cache effects. For this to work well, you would want to make it a root-only option, and to restrict its size to a fraction of physical memory. Personally, I recommend using a different FS, if you are going to create a honing big FS as a single partition. 8-(. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 19:45:43 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2996E37B401; Mon, 17 Mar 2003 19:45:42 -0800 (PST) Received: from bluejay.mail.pas.earthlink.net (bluejay.mail.pas.earthlink.net [207.217.120.218]) by mx1.FreeBSD.org (Postfix) with ESMTP id 972EE43FA3; Mon, 17 Mar 2003 19:45:41 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0217.cvx22-bradley.dialup.earthlink.net ([209.179.198.217] helo=mindspring.com) by bluejay.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18v82l-0001Gd-00; Mon, 17 Mar 2003 19:45:40 -0800 Message-ID: <3E769614.B2BC8660@mindspring.com> Date: Mon, 17 Mar 2003 19:44:20 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Julian Elischer Cc: Bakul Shah , FreeBSD current users , fs@FreeBSD.ORG Subject: Re: Anyone working on fsck? References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a43f68e6a0fb7949f1ec9797fa53012868350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Julian Elischer wrote: > The problem space is > > Fsck of UFS/FFS partitions is too slow for 200GB+ filesystems. > > The solution space can not contain any answer that includes redefining > UFS/FFS. Welcome to the real world. :-) "Use smaller than 200GB+ filesystems". 8-). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 19:51: 0 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E391237B401; Mon, 17 Mar 2003 19:50:58 -0800 (PST) Received: from bluejay.mail.pas.earthlink.net (bluejay.mail.pas.earthlink.net [207.217.120.218]) by mx1.FreeBSD.org (Postfix) with ESMTP id 277A243F93; Mon, 17 Mar 2003 19:50:58 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0217.cvx22-bradley.dialup.earthlink.net ([209.179.198.217] helo=mindspring.com) by bluejay.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18v87h-0001xJ-00; Mon, 17 Mar 2003 19:50:46 -0800 Message-ID: <3E769745.FA122633@mindspring.com> Date: Mon, 17 Mar 2003 19:49:25 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Bakul Shah Cc: Poul-Henning Kamp , Julian Elischer , FreeBSD current users , fs@FreeBSD.ORG Subject: Re: Anyone working on fsck? References: <200303172205.RAA18117@goliath.cnchost.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a43f68e6a0fb7949f13380241dbafb9eb7350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Bakul Shah wrote: > > I have been tending UNIX computers of all sorts for many years and > > there is one bit of wisdom that has yet to fail me: > > > > Every now and then, boot in single-user and run full fsck > > on all filesystems. > > > > If this had failed to be productive, I would have given up the > > habit years ago, but it is still a good idea it seems. > > Even now I use fsck in forground since the background fsck > was not stable enough the last time I used it. But I > remember thinking fsck was taking too long for as long as I > have used it (since 1981). If your problem is "availability", then any "time > 0" counts as "too long". Taking the whole system offline, while the rest of the world has gone into an electronic transaction frenzy because some world event, and running a BG fsck is not really an option. > Anything that runs for half hour or more in fg is likely to > take longer in bg. What happens if the system crashes again > before it finishes? Will bg fsck handle that? Am I right in > thinking that it can not save files in /lost+found? It can, but it has to be changed; it's pretty ugly. Your best bet is to precreate "lost+found", and then modify the code to allow you to ftruncate(2) a directory large, forcibly allocating backing blocks for it (that was my last workaround to the problem). > > Optimizing fsck is a valid project, I just wish it would be somebody > > who would also finish the last 30% who would do it. > > I am skeptical you will get more than a factor of 2 > improvement without changing the FS (but hey, that is 3 hours > for Julian so I am sure he will be happy with that!). I'm skeptical you will get a factor of 2. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 20: 0:17 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A1A7537B404; Mon, 17 Mar 2003 20:00:16 -0800 (PST) Received: from bluejay.mail.pas.earthlink.net (bluejay.mail.pas.earthlink.net [207.217.120.218]) by mx1.FreeBSD.org (Postfix) with ESMTP id AF97143FB1; Mon, 17 Mar 2003 20:00:15 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0217.cvx22-bradley.dialup.earthlink.net ([209.179.198.217] helo=mindspring.com) by bluejay.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18v8Gg-00036q-00; Mon, 17 Mar 2003 20:00:03 -0800 Message-ID: <3E769970.3A05ECAF@mindspring.com> Date: Mon, 17 Mar 2003 19:58:40 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Jeff Roberson Cc: Brooks Davis , Julian Elischer , Alfred Perlstein , FreeBSD current users , fs@FreeBSD.ORG Subject: Re: Anyone working on fsck? References: <20030317172953.O66343-100000@mail.chesapeake.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a46deee49447c996d677be39db25741d92548b785378294e88350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Jeff Roberson wrote: > On Mon, 17 Mar 2003, Brooks Davis wrote: > > I am still intrested in improvements to fsck since I'm planning to buy > > several systems with two 1.4TB IDE RAID5 arrays in them soon. > > For these types of systems doing a block caching layer with a prefetch > that understands how many spindles there are would be a huge benefit. I call that layer "Vinum" or "RAIDFrame", since that's a job I expect that code to do for me. 8-). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 20: 2:46 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F2CE937B401; Mon, 17 Mar 2003 20:02:44 -0800 (PST) Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0309F43F3F; Mon, 17 Mar 2003 20:02:44 -0800 (PST) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id h2I42c868107; Mon, 17 Mar 2003 23:02:39 -0500 (EST) (envelope-from jroberson@chesapeake.net) Date: Mon, 17 Mar 2003 23:02:38 -0500 (EST) From: Jeff Roberson To: Terry Lambert Cc: Brooks Davis , Julian Elischer , Alfred Perlstein , FreeBSD current users , Subject: Re: Anyone working on fsck? In-Reply-To: <3E769970.3A05ECAF@mindspring.com> Message-ID: <20030317230147.I66343-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Mon, 17 Mar 2003, Terry Lambert wrote: > Jeff Roberson wrote: > > On Mon, 17 Mar 2003, Brooks Davis wrote: > > > I am still intrested in improvements to fsck since I'm planning to buy > > > several systems with two 1.4TB IDE RAID5 arrays in them soon. > > > > For these types of systems doing a block caching layer with a prefetch > > that understands how many spindles there are would be a huge benefit. > > I call that layer "Vinum" or "RAIDFrame", since that's a job I > expect that code to do for me. 8-). They are not responsible for data caching. Only informing the upper layers how many spindles they have. Software RAID should be a transform only in my opinion. There is no reason to have duplicate block caches in system memory. Cheers, Jeff To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 20:33:55 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E6CE137B401; Mon, 17 Mar 2003 20:33:53 -0800 (PST) Received: from bluejay.mail.pas.earthlink.net (bluejay.mail.pas.earthlink.net [207.217.120.218]) by mx1.FreeBSD.org (Postfix) with ESMTP id 54A2C43F93; Mon, 17 Mar 2003 20:33:53 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0217.cvx22-bradley.dialup.earthlink.net ([209.179.198.217] helo=mindspring.com) by bluejay.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18v8nG-00073H-00; Mon, 17 Mar 2003 20:33:43 -0800 Message-ID: <3E76A151.320A1E12@mindspring.com> Date: Mon, 17 Mar 2003 20:32:17 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Jeff Roberson Cc: Brooks Davis , Julian Elischer , Alfred Perlstein , FreeBSD current users , fs@FreeBSD.ORG Subject: Re: Anyone working on fsck? References: <20030317230147.I66343-100000@mail.chesapeake.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4797becd7d142051c76c6a303e25dd15f666fa475841a1c7a350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Jeff Roberson wrote: > On Mon, 17 Mar 2003, Terry Lambert wrote: > > Jeff Roberson wrote: > > > On Mon, 17 Mar 2003, Brooks Davis wrote: > > > > I am still intrested in improvements to fsck since I'm planning to buy > > > > several systems with two 1.4TB IDE RAID5 arrays in them soon. > > > > > > For these types of systems doing a block caching layer with a prefetch > > > that understands how many spindles there are would be a huge benefit. > > > > I call that layer "Vinum" or "RAIDFrame", since that's a job I > > expect that code to do for me. 8-). > > They are not responsible for data caching. Only informing the upper > layers how many spindles they have. Software RAID should be a transform > only in my opinion. There is no reason to have duplicate block caches in > system memory. Let's turn that around: "There is no reason to have duplicate spindle knowledge". Actually, I was not suggesting duplicate block caches, I was suggesting cache attribution by spindle by the code that knows what block lives on what spindle. Even so, for RAID, this is generally problematic, because there's multiple locations for the block: where it lives, where it's mirrored, where it's parity block lives, etc.. Ideally, these are all different spindles, so the problem can't be fixed by a simple cache. 8-(. You would need a chain of spindle references, supplied by the code that knows the spindle, hung off the cache object. Gets ugly fast. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 20:35:56 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D393B37B401 for ; Mon, 17 Mar 2003 20:35:54 -0800 (PST) Received: from panzer.kdm.org (panzer.kdm.org [216.160.178.169]) by mx1.FreeBSD.org (Postfix) with ESMTP id AA77D43F85 for ; Mon, 17 Mar 2003 20:35:53 -0800 (PST) (envelope-from ken@panzer.kdm.org) Received: from panzer.kdm.org (localhost [127.0.0.1]) by panzer.kdm.org (8.12.6/8.12.5) with ESMTP id h2I4ZI8U075391; Mon, 17 Mar 2003 21:35:18 -0700 (MST) (envelope-from ken@panzer.kdm.org) Received: (from ken@localhost) by panzer.kdm.org (8.12.6/8.12.5/Submit) id h2I4ZHGB075390; Mon, 17 Mar 2003 21:35:18 -0700 (MST) (envelope-from ken) Date: Mon, 17 Mar 2003 21:35:17 -0700 From: "Kenneth D. Merry" To: Jeff Roberson Cc: Terry Lambert , Brooks Davis , Julian Elischer , Alfred Perlstein , FreeBSD current users , fs@FreeBSD.ORG Subject: Re: Anyone working on fsck? Message-ID: <20030317213517.A75211@panzer.kdm.org> References: <3E769970.3A05ECAF@mindspring.com> <20030317230147.I66343-100000@mail.chesapeake.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <20030317230147.I66343-100000@mail.chesapeake.net>; from jroberson@chesapeake.net on Mon, Mar 17, 2003 at 11:02:38PM -0500 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Mon, Mar 17, 2003 at 23:02:38 -0500, Jeff Roberson wrote: > On Mon, 17 Mar 2003, Terry Lambert wrote: > > Jeff Roberson wrote: > > > On Mon, 17 Mar 2003, Brooks Davis wrote: > > > > I am still intrested in improvements to fsck since I'm planning to buy > > > > several systems with two 1.4TB IDE RAID5 arrays in them soon. > > > > > > For these types of systems doing a block caching layer with a prefetch > > > that understands how many spindles there are would be a huge benefit. > > > > I call that layer "Vinum" or "RAIDFrame", since that's a job I > > expect that code to do for me. 8-). > > They are not responsible for data caching. Only informing the upper > layers how many spindles they have. Software RAID should be a transform > only in my opinion. There is no reason to have duplicate block caches in > system memory. There are times when a software RAID layer should do some caching. Hopefully your software RAID layer will be integrated enough into the system that it can avoid doing any copies, though. The place where you really want some sort of caching is when you try to coalesce buffers to get full stripe writes with RAID-5. Otherwise, you have to do read-modify-write, which is more expensive. There are other cases where caching is needed for RAID-1 and RAID-5, but they generally require specialized hardware. (Hint: why do many controllers that support RAID-1 and RAID-5 have battery backed caches?) Other than stripe coalescing, and those cases that come up when you have specialized hardware, you're right, RAID should be a transform that doesn't require copying. (Caching is another story.) Ken -- Kenneth Merry ken@kdm.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 20:59:54 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D3F8D37B401; Mon, 17 Mar 2003 20:59:50 -0800 (PST) Received: from goliath.cnchost.com (goliath.cnchost.com [207.155.252.47]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1CE2843F75; Mon, 17 Mar 2003 20:59:50 -0800 (PST) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (adsl-209-204-185-216.sonic.net [209.204.185.216]) by goliath.cnchost.com id XAA15021; Mon, 17 Mar 2003 23:59:46 -0500 (EST) [ConcentricHost SMTP Relay 1.15] Message-ID: <200303180459.XAA15021@goliath.cnchost.com> To: Terry Lambert Cc: Julian Elischer , FreeBSD current users , fs@FreeBSD.ORG Subject: Re: Anyone working on fsck? In-reply-to: Your message of "Mon, 17 Mar 2003 19:41:34 PST." <3E76956E.5E0FCC6B@mindspring.com> Date: Mon, 17 Mar 2003 20:59:45 -0800 From: Bakul Shah Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org > > UFS is the real problem here, not fsck. Its tradeoffs for > > improving normal access latencies may have been right in the > > past but not for modern big disks. ... > Sorry, but the track-to-track seek latency optimizations you > are referring to are turned off, given the newfs defaults, and > have been for a very long time. I was thinking of the basic idea of cylinder groups as good for normal load, not so good for fsck when you have too many CGs. I wasn't thinking of what fsck does or does not do. > Basically, the problem is that for a BG fsck, it's not possible > to lock access on a cylinder group basis, and then there is a hell > of a lot of data on that drive. Note that Julian said 6 hours to fsck a TB in the normal foreground mode. > What Julian needs is a Journalling or Log-structured FS. All that Julian wants is a faster fsck without mucking with the FS! While I agree with you that you do need a full consistency check, it is worth thinking about how one can avoid that whenever possible. For example, if you can know where the disk head is at the time of crash (based on what blocks were being written) it should be possible to avoid a full check. > The easiest way to do this is to provide some sort of domain > control, so that you limit the files you have to check to a > smaller set of files per CG, so that you don't have to check > all the files to check a particular CG -- and then preload the > CG's for the sets of files you *do* have to check. If you have to visit a CG (during fsck) you have already paid the cost of the seek and rotational latency. Journalling wouldn't help here if you still have a zillion CGs. > Another way of saying this is... don't put all your space in a > single FS. 8-). Or in effect treat each CG (or a group of CGs) as a self contained filesystem (for the purpose of physical allocation) and maintain explicit import/export lists for files that span them. > The problem with having to (effectively) read every inode and > direct block on the disk is really insurmountable, I think. That is why I was suggesting putting them in one (or small number of) contiguous area(s). On a modern ATA100 or better disk you can read a GB in under a minute. Once the data is in-core you can divide up the checking to multiple processors. This is sort of like a distributed graph collection: you only need to worry about graphs that cross a node boundary. Most structures wille contained in one node. Even for UFS it is probably worth dividing fsck in two or more processes, one doing IO, one or more doing computation. > > Typically only a few cyl grps may be inconsistent in case of > > a crash. May be some info about which cyl groups need to be > > checked can be stored so that brute force checking of all > > grps can be avoided. > > This would work well... under laboratory test conditions. In > the field, if the machine has a more general workload (or even > a heterogeneous workload, but with a hell of a lot of files, > like a big mail server), this falls down, as the number of bits > marking "unupdated cylinder groups" becomes large. Possible -- it is one of the ideas I can think of. I'd have to actually model it or simulate it beyond handwaving to know one way or other. May be useful in conjunction with other ideas. > ...AND... The problem is still that you must scan everything on > the disk (practically) to identify the inode or indirect block > that references the block on the cylinder roup in question, and > THAT's the problem. If you knew a small set of CG's, that needed > checked, vs. "all of them", it would *still* bust the cache, which > is what takes all the time. > Assume, on average, each file goes into indirect blocks. On my machine the average file size is 21KB (averaged over 4,000,000 files). Even with 8KB blocksize very few will have indirect blocks. I don't know how typical my file size distribution is but I suspect the average case is probably smaller files (I store lots of datasheets, manuals, databases, PDFs, MP3s, cvs repositories, compressed tars of old stuff). But in any case wouldn't going forward from inodes make more sense? This is like a standard tracing garbage collection algorithm. Blocks that are not reachable are free. Even for a 1 TB system with 8K blocks you need 2^(40-13-3) == 16Mbytes bitmap or some multiple if you want more than 1 bit of state. > The problem is reverse mapping a bit in a CG bitmap to the file > that reference it... 8^p. Why would you want to do that?! > Multithreading fsck would be an incredibly bad idea. It depends on the actual algorithm. > Personally, I recommend using a different FS, if you are going > to create a honing big FS as a single partition. 8-(. There are other issues with smaller partitions. I'd rather have a single logical file system where all the space can be used. If physically it is implemented as a number of smaller systems that is okay. Also note that now people can create big honking files with video streaming at the rate of 2GB per hour and even then you are compromising on quality a bit. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 22:18:12 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A411037B401; Mon, 17 Mar 2003 22:18:10 -0800 (PST) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id DC3ED43F75; Mon, 17 Mar 2003 22:18:08 -0800 (PST) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.8/8.12.8) with ESMTP id h2I6HwYl046518; Tue, 18 Mar 2003 07:17:58 +0100 (CET) (envelope-from phk@phk.freebsd.dk) To: Brad Knowles Cc: Bakul Shah , Julian Elischer , FreeBSD current users , fs@FreeBSD.ORG Subject: Re: Anyone working on fsck? From: "Poul-Henning Kamp" In-Reply-To: Your message of "Mon, 17 Mar 2003 23:39:59 +0100." Date: Tue, 18 Mar 2003 07:17:58 +0100 Message-ID: <46517.1047968278@critter.freebsd.dk> Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org In message , Brad Knowles writes: >At 10:39 PM +0100 2003/03/17, Poul-Henning Kamp wrote: > >> Optimizing fsck is a valid project, I just wish it would be somebody >> who would also finish the last 30% who would do it. > > Just what are you saying? Is Julian Elischer not the right >person to be working on this, because he has a history of not >finishing the last 30% of something? Yes. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 22:24: 4 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 754C237B401; Mon, 17 Mar 2003 22:24:03 -0800 (PST) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6942643F75; Mon, 17 Mar 2003 22:24:02 -0800 (PST) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.8/8.12.8) with ESMTP id h2I6NuYl047573; Tue, 18 Mar 2003 07:23:56 +0100 (CET) (envelope-from phk@phk.freebsd.dk) To: "Greg 'groggy' Lehey" Cc: Bakul Shah , Julian Elischer , FreeBSD current users , fs@FreeBSD.org Subject: Re: Anyone working on fsck? From: "Poul-Henning Kamp" In-Reply-To: Your message of "Tue, 18 Mar 2003 10:18:36 +1030." <20030317234836.GG9422@wantadilla.lemis.com> Date: Tue, 18 Mar 2003 07:23:56 +0100 Message-ID: <47572.1047968636@critter.freebsd.dk> Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org In message <20030317234836.GG9422@wantadilla.lemis.com>, "Greg 'groggy' Lehey" writes: >> Optimizing fsck is a valid project, I just wish it would be somebody >> who would also finish the last 30% who would do it. > >Poul-Henning, how can you justify the second half of that sentence? I >take exception to the implications. In case anybody is in any doubt, >I've heard you say this sort of thing about julian before. Please >don't do it again. I'll stop as soon as KSE is finished, fair ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 22:48: 7 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6B42137B401; Mon, 17 Mar 2003 22:48:05 -0800 (PST) Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.80]) by mx1.FreeBSD.org (Postfix) with ESMTP id D6FA943F75; Mon, 17 Mar 2003 22:48:03 -0800 (PST) (envelope-from grog@lemis.com) Received: by wantadilla.lemis.com (Postfix, from userid 1004) id 2806151A62; Tue, 18 Mar 2003 17:18:01 +1030 (CST) Date: Tue, 18 Mar 2003 17:18:01 +1030 From: Greg 'groggy' Lehey To: Poul-Henning Kamp Cc: Bakul Shah , Julian Elischer , FreeBSD Developers , fs@FreeBSD.org Subject: Interdeveloper relations (was: Anyone working on fsck?) Message-ID: <20030318064801.GO84879@wantadilla.lemis.com> References: <20030317234836.GG9422@wantadilla.lemis.com> <47572.1047968636@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="V7BlxAaPrdhzdIM1" Content-Disposition: inline In-Reply-To: <47572.1047968636@critter.freebsd.dk> User-Agent: Mutt/1.4i Organization: The FreeBSD Project Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.FreeBSD.org/ X-PGP-Fingerprint: 9A1B 8202 BCCE B846 F92F 09AC 22E6 F290 507A 4223 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org --V7BlxAaPrdhzdIM1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tuesday, 18 March 2003 at 7:23:56 +0100, Poul-Henning Kamp wrote: > In message <20030317234836.GG9422@wantadilla.lemis.com>, "Greg 'groggy' Lehey" > writes: > >>> Optimizing fsck is a valid project, I just wish it would be somebody >>> who would also finish the last 30% who would do it. >> >> Poul-Henning, how can you justify the second half of that sentence? I >> take exception to the implications. In case anybody is in any doubt, >> I've heard you say this sort of thing about julian before. Please >> don't do it again. > > I'll stop as soon as KSE is finished, fair ? No. That has nothing to do with the issue. In case you still don't understand, this has nothing to do with your opinion of julian, to which you're welcome. What is not acceptable is that, again and again, you choose to express it in this way. Greg -- See complete headers for address and phone numbers --V7BlxAaPrdhzdIM1 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.0 (FreeBSD) iD8DBQE+dsEhIubykFB6QiMRAqsfAJ9BERDDnJijT+G4XOTwLb04+QMpuwCfZ+HZ LA903WYGtwHXbhy8b2nM3OI= =VOQs -----END PGP SIGNATURE----- --V7BlxAaPrdhzdIM1-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 22:51:12 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 61E7537B401; Mon, 17 Mar 2003 22:51:05 -0800 (PST) Received: from puffin.mail.pas.earthlink.net (puffin.mail.pas.earthlink.net [207.217.120.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8675443F85; Mon, 17 Mar 2003 22:51:04 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0055.cvx22-bradley.dialup.earthlink.net ([209.179.198.55] helo=mindspring.com) by puffin.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18vAvv-0003T4-00; Mon, 17 Mar 2003 22:50:48 -0800 Message-ID: <3E76C15F.D3C7F9A7@mindspring.com> Date: Mon, 17 Mar 2003 22:49:03 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Bakul Shah Cc: Julian Elischer , FreeBSD current users , fs@FreeBSD.ORG Subject: Re: Anyone working on fsck? References: <200303180459.XAA15021@goliath.cnchost.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4ad7bdad5ce164efe02ed0a715bafa1ef666fa475841a1c7a350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Bakul Shah wrote: > > Sorry, but the track-to-track seek latency optimizations you > > are referring to are turned off, given the newfs defaults, and > > have been for a very long time. > > I was thinking of the basic idea of cylinder groups as good > for normal load, not so good for fsck when you have too many > CGs. I wasn't thinking of what fsck does or does not do. Number of CG's is irrelevent, really. You can read 10 of the bitmaps at a time and precache them, or you can read 1000 of them and precache them, the difference is going to be minimal. The thing that takes the time is the forward traversal to do the reverse lookup. Julian has the right idea about precaching; I suggested it, too, before reading his message, but after a certain level, there is a point of diminishing returns. Getting rid of cylinder groups just pushes the problem up a level, it doesn't eliminate it. No cylinder group bitmap to be the last thing updated? OK, then whatever becomes the last thing updated, instead of that, becomes the reverse lookup bottleneck. > > Basically, the problem is that for a BG fsck, it's not possible > > to lock access on a cylinder group basis, and then there is a hell > > of a lot of data on that drive. > > Note that Julian said 6 hours to fsck a TB in the normal > foreground mode. Yes, I know. I'm aware. He has a lot of data to transfer in from the disk in order to do the reverse lookup, with that much data on the disk. He could change the code so that that everything that wasn't in use was zeroed. That would help, since then he could just create a second CG map in an empty file by traversing all inodes and indirect blocks for non-zero values, not caring if the indirect blocks were referenced by inodes. Assuming indirect blocks and data blocks were distinguishable. Like I said, there are a lot of FS changes that could help. > > What Julian needs is a Journalling or Log-structured FS. > > All that Julian wants is a faster fsck without mucking with > the FS! And I'd like to regrow all my teeth that have ever had dental work done on them. There are some things you can't have. 8-). > While I agree with you that you do need a full consistency > check, it is worth thinking about how one can avoid that > whenever possible. For example, if you can know where the > disk head is at the time of crash (based on what blocks were > being written) it should be possible to avoid a full check. You can buy hardware that can do this. Sun sold it in the 1980's, and people have made NVRAM-backed caching disk controllers for forever, since then. > > The easiest way to do this is to provide some sort of domain > > control, so that you limit the files you have to check to a > > smaller set of files per CG, so that you don't have to check > > all the files to check a particular CG -- and then preload the > > CG's for the sets of files you *do* have to check. > > If you have to visit a CG (during fsck) you have already paid > the cost of the seek and rotational latency. No. The cost of the fsck is *not* the reading of the cylinder group bitmaps. You can cache a fixed number of those. The problem is in not knowing which inodes and indirect blocks contain direct and indirect block references to the cylinder groups you happen to have in cache. In other words, the cost is in enumerating all the allocated blocks in all of the files. The *reason* it doesn't matter in the CG bitmap case, is that a bitmap can only tell "allocated" vs. "unallocated"; there is no third state that means "I haven't checked this bit yet". So you can only load up however many bitmaps you are going to check in a given pass (hopefully they all fit in memory, but if they don't fitting in the address space does you no good, because they still aren't tri-state), and then iterate every single bit reference in existance, and compare them to the ones you have, and clear them if they aren't referenced by *anyone*. In other words, this is *always* a cylinder group gitmap bit-major operations (as in "row-major" vs. "column-major" arrays in C and FORTRAN). > Journalling wouldn't help here if you still have a zillion CGs. Yes, it would. If you had Journalling, you wouldn't have CG bitmaps to worry about, no matter how many CG's you had. The *only* thing you would have that you cared about is the "operation started" and "operation complete" stamps on a journal entry, and you'd *only* care whether the former was greater or less than than the latter. And then you'd write a field of "last valid journal entry flip-flop" into the oldest of the two flip-flop fields with a journal reference and the current date stamp, and you'd be done. The only thing you'd care about was how many "last valid" flip-flops you had, which would be totally unrelated to cylinder groups, because it's a file-major idea (and you have to traverse the files anyway). > > Another way of saying this is... don't put all your space in a > > single FS. 8-). > > Or in effect treat each CG (or a group of CGs) as a self > contained filesystem (for the purpose of physical allocation) > and maintain explicit import/export lists for files that span > them. I already said that. 8-) 8-). > > The problem with having to (effectively) read every inode and > > direct block on the disk is really insurmountable, I think. > > That is why I was suggesting putting them in one (or small > number of) contiguous area(s). On a modern ATA100 or better > disk you can read a GB in under a minute. Once the data is > in-core you can divide up the checking to multiple > processors. This is sort of like a distributed graph > collection: you only need to worry about graphs that cross a > node boundary. Most structures wille contained in one node. There are a couple of FS's which already do this. One of them is Eric H. Herrin II and Raphael A. Finkel's "Viva" out of the University of Kentucky. That was a 1993 project. Another is "Stride". Actually, the BSD 4.0 FS did a similar thing, in keeping different data types on different areas of the disk. If you read the BSD FFS paper, though, you'll see why the moved away from that. 8-). One of the reasons is fragility, which modern disks *certainly* do not address, no matter what other "advances" they have: lose a single track on an IDE due to a power failure during write, you lose all your data. > Even for UFS it is probably worth dividing fsck in two or > more processes, one doing IO, one or more doing computation. ?????? So adding more non-locality accesses to a system that is slow as a result of non-locality acceses helps how? 8-) 8-). > > ...AND... The problem is still that you must scan everything on > > the disk (practically) to identify the inode or indirect block > > that references the block on the cylinder roup in question, and > > THAT's the problem. If you knew a small set of CG's, that needed > > checked, vs. "all of them", it would *still* bust the cache, which > > is what takes all the time. > > > Assume, on average, each file goes into indirect blocks. > > On my machine the average file size is 21KB (averaged over > 4,000,000 files). Even with 8KB blocksize very few will have > indirect blocks. I don't know how typical my file size > distribution is but I suspect the average case is probably > smaller files (I store lots of datasheets, manuals, > databases, PDFs, MP3s, cvs repositories, compressed tars of > old stuff). > > But in any case wouldn't going forward from inodes make more > sense? This is like a standard tracing garbage collection > algorithm. Blocks that are not reachable are free. Even for > a 1 TB system with 8K blocks you need 2^(40-13-3) == 16Mbytes > bitmap or some multiple if you want more than 1 bit of state. There are a lot of smart things that can be done... if you are willing to change the FS. Most of them have been done already, at least 10 years ago. 8-) 8-). > > The problem is reverse mapping a bit in a CG bitmap to the file > > that reference it... 8^p. > > Why would you want to do that?! That's exactly what you are doing, when you are trying to decide to clear a bit in the CG. The tristate problem. You have to know that it's *not* refernced. For something that's references, on average, you have to look at 1/2 of your search set. For something that's not, you have to look at 100% of your search set. That's why negative caching in a directory name lookup cache is twice as valuable as positive caching. 8-) 8-). > > Multithreading fsck would be an incredibly bad idea. > > It depends on the actual algorithm. Right. For example, a "multithreaded algorithm"... ;^). > > Personally, I recommend using a different FS, if you are going > > to create a honing big FS as a single partition. 8-(. > > There are other issues with smaller partitions. I'd rather > have a single logical file system where all the space can be > used. If physically it is implemented as a number of smaller > systems that is okay. Also note that now people can create > big honking files with video streaming at the rate of 2GB per > hour and even then you are compromising on quality a bit. Sure. I expected that, given Julian's current line of work, and the size of the FS, that what we were talking about was honking big files representing image captures of all documents in a large document set. This also made me think that the reason for the huge partition was to be able to handle "single very large file" instances, more than "a huge number of little files", so that he couldn't break them up into reasonable sizes. It was also the reason for the "smiley" on my suggestion that he use smaller partition sizes. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 22:53:46 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 07A2137B404; Mon, 17 Mar 2003 22:53:44 -0800 (PST) Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.80]) by mx1.FreeBSD.org (Postfix) with ESMTP id 735C243F85; Mon, 17 Mar 2003 22:53:41 -0800 (PST) (envelope-from grog@lemis.com) Received: by wantadilla.lemis.com (Postfix, from userid 1004) id 6002851A62; Tue, 18 Mar 2003 17:23:39 +1030 (CST) Date: Tue, 18 Mar 2003 17:23:39 +1030 From: Greg 'groggy' Lehey To: Jeff Roberson Cc: Terry Lambert , Brooks Davis , Julian Elischer , Alfred Perlstein , FreeBSD current users , fs@FreeBSD.ORG Subject: Software RAID caching? (was: Anyone working on fsck?) Message-ID: <20030318065339.GQ84879@wantadilla.lemis.com> References: <3E769970.3A05ECAF@mindspring.com> <20030317230147.I66343-100000@mail.chesapeake.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="uRjmd8ppyyws0Tml" Content-Disposition: inline In-Reply-To: <20030317230147.I66343-100000@mail.chesapeake.net> User-Agent: Mutt/1.4i Organization: The FreeBSD Project Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.FreeBSD.org/ X-PGP-Fingerprint: 9A1B 8202 BCCE B846 F92F 09AC 22E6 F290 507A 4223 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org --uRjmd8ppyyws0Tml Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Monday, 17 March 2003 at 23:02:38 -0500, Jeff Roberson wrote: > On Mon, 17 Mar 2003, Terry Lambert wrote: > >> Jeff Roberson wrote: >>> On Mon, 17 Mar 2003, Brooks Davis wrote: >>>> I am still intrested in improvements to fsck since I'm planning to buy >>>> several systems with two 1.4TB IDE RAID5 arrays in them soon. >>> >>> For these types of systems doing a block caching layer with a prefetch >>> that understands how many spindles there are would be a huge benefit. >> >> I call that layer "Vinum" or "RAIDFrame", since that's a job I >> expect that code to do for me. 8-). > > They are not responsible for data caching. Only informing the upper > layers how many spindles they have. Software RAID should be a transform > only in my opinion. There is no reason to have duplicate block caches in > system memory. Agreed. Vinum doesn't cache. There is one case, though, where it could be argued that it's worthwhile, namely in RAID-[45] parity blocks. Greg -- See complete headers for address and phone numbers --uRjmd8ppyyws0Tml Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.0 (FreeBSD) iD8DBQE+dsJzIubykFB6QiMRAnxRAKCchEJe3ef58LaGhKV3ryDQ2KWSuwCffM0d IDGm/1c4dUXxIubQoi4Uq90= =TNJT -----END PGP SIGNATURE----- --uRjmd8ppyyws0Tml-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Mon Mar 17 23:45:58 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9642A37B401; Mon, 17 Mar 2003 23:45:57 -0800 (PST) Received: from canning.wemm.org (canning.wemm.org [192.203.228.65]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1727A43F93; Mon, 17 Mar 2003 23:45:57 -0800 (PST) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by canning.wemm.org (Postfix) with ESMTP id E89E32A8C1; Mon, 17 Mar 2003 23:45:56 -0800 (PST) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: "Poul-Henning Kamp" Cc: Julian Elischer , FreeBSD current users , fs@FreeBSD.org Subject: Re: Anyone working on fsck? In-Reply-To: <47572.1047968636@critter.freebsd.dk> Date: Mon, 17 Mar 2003 23:45:56 -0800 From: Peter Wemm Message-Id: <20030318074556.E89E32A8C1@canning.wemm.org> Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org "Poul-Henning Kamp" wrote: > In message <20030317234836.GG9422@wantadilla.lemis.com>, "Greg 'groggy' Lehey " > writes: > > >> Optimizing fsck is a valid project, I just wish it would be somebody > >> who would also finish the last 30% who would do it. > > > >Poul-Henning, how can you justify the second half of that sentence? I > >take exception to the implications. In case anybody is in any doubt, > >I've heard you say this sort of thing about julian before. Please > >don't do it again. > > I'll stop as soon as KSE is finished, fair ? Poul-Henning.. this is a bit of a cheap shot. Your point may be valid, but this isn't the way to express it as it just turns into a 'phk is being mean again' flamewar and the message gets lost in the noise. Anyway, the deadline for KSE to be demonstrated as robust in order to avoid getting disabled for 5.x is getting closer. I'm glad it was going to be finished inside 2 months, starting about 18 months ago. (See the 5.x release milestones for the actual deadline, June 30 if I recall correctly.) Cheers, -Peter -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Mar 18 1:59:20 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6680437B401; Tue, 18 Mar 2003 01:59:19 -0800 (PST) Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id 539FE43FA3; Tue, 18 Mar 2003 01:59:18 -0800 (PST) (envelope-from des@ofug.org) Received: by flood.ping.uio.no (Postfix, from userid 2602) id 09B035308; Tue, 18 Mar 2003 10:59:16 +0100 (CET) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Julian Elischer Cc: FreeBSD current users , fs@freebsd.org Subject: Re: Anyone working on fsck? From: des@ofug.org (Dag-Erling =?iso-8859-1?q?Sm=F8rgrav?=) Date: Tue, 18 Mar 2003 10:59:14 +0100 In-Reply-To: (Julian Elischer's message of "Mon, 17 Mar 2003 12:22:33 -0800 (PST)") Message-ID: User-Agent: Gnus/5.090015 (Oort Gnus v0.15) Emacs/21.2 References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Julian Elischer writes: > [fsck is impossibly slow on multi-TB filesystems] Let's rather work on getting a working log-structured filesystem committed so we don't *need* fsck for filesystems that large. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Mar 18 2: 8:59 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DBC3237B401; Tue, 18 Mar 2003 02:08:57 -0800 (PST) Received: from energyhq.homeip.net (213-97-200-73.uc.nombres.ttd.es [213.97.200.73]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3E6A743FA3; Tue, 18 Mar 2003 02:08:56 -0800 (PST) (envelope-from flynn@energyhq.homeip.net) Received: from christine.energyhq.tk (christine.energyhq.tk [192.168.100.1]) by energyhq.homeip.net (Postfix) with SMTP id AF475AF5D3; Tue, 18 Mar 2003 11:08:54 +0100 (CET) Date: Tue, 18 Mar 2003 11:08:52 +0100 From: Miguel Mendez To: des@ofug.org, ?=@energyhq.homeip.net (Dag-Erling =?ISO-8859-1?Q?Sm=F8rgrav) Cc: Julian Elischer , FreeBSD current users , fs@freebsd.org Subject: Re: Anyone working on fsck? Message-Id: <20030318110852.438ffd2e.flynn@energyhq.homeip.net> In-Reply-To: References: X-Mailer: Sylpheed version 0.8.11claws (GTK+ 1.2.10; i386--netbsdelf) X-Face: 1j}k*2E>Y\+C~E|/wehi[:dCM,{N7/uE3o# P,{t7gA/qnovFDDuyQV.1hdT7&#d)q"xY33}{_GS>kk'S{O]nE$A`T|\4&p\&mQyexOLb8}FO List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org --SN=.He0IVEaPvcsM Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On Tue, 18 Mar 2003 10:59:14 +0100 des@ofug.org (Dag-Erling Sm=F8rgrav) wrote: Howdy, =20 > Let's rather work on getting a working log-structured filesystem > committed so we don't *need* fsck for filesystems that large. Rather than reinventing the wheel, how about porting NetBSD's LFS to FreeBS= D? Granted it's not 100% reliable yet, but it's improving. I'd rather have = that than wait for JFS4BSD to be ready. In fact, I'd volunteer to work with= my (limited) knowledge. Cheers, --=20 Miguel Mendez - flynn@energyhq.homeip.net GPG Public Key :: http://energyhq.homeip.net/files/pubkey.txt EnergyHQ :: http://www.energyhq.tk Of course it runs NetBSD! Tired of Spam? -> http://www.trustic.com --SN=.He0IVEaPvcsM Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (NetBSD) iD8DBQE+dvA0nLctrNyFFPERApdaAKCuepZnbAY62q9m+d0pJMgbG9WIxQCcCjZa qT4TEPJd6DUUVFLZVIAYIuY= =tYa5 -----END PGP SIGNATURE----- --SN=.He0IVEaPvcsM-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Mar 18 6:34:37 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 445CB37B404; Tue, 18 Mar 2003 06:34:36 -0800 (PST) Received: from picard.skynet.be (picard.skynet.be [195.238.3.88]) by mx1.FreeBSD.org (Postfix) with ESMTP id B29DD43F85; Tue, 18 Mar 2003 06:34:30 -0800 (PST) (envelope-from brad.knowles@skynet.be) Received: from [10.0.1.2] (180.91-201-80.adsl.skynet.be [80.201.91.180]) by picard.skynet.be (8.12.8/8.12.8/Skynet-OUT-2.21) with ESMTP id h2IEYEEJ006892; Tue, 18 Mar 2003 15:34:14 +0100 (MET) (envelope-from ) Mime-Version: 1.0 X-Sender: bs663385@pop.skynet.be Message-Id: In-Reply-To: <3E76A151.320A1E12@mindspring.com> References: <20030317230147.I66343-100000@mail.chesapeake.net> <3E76A151.320A1E12@mindspring.com> Date: Tue, 18 Mar 2003 14:59:51 +0100 To: Terry Lambert From: Brad Knowles Subject: Re: Anyone working on fsck? Cc: Jeff Roberson , Brooks Davis , Julian Elischer , Alfred Perlstein , FreeBSD current users , fs@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org At 8:32 PM -0800 2003/03/17, Terry Lambert wrote: > Even so, for RAID, this is generally problematic, because there's > multiple locations for the block: where it lives, where it's mirrored, > where it's parity block lives, etc.. Ideally, these are all different > spindles, so the problem can't be fixed by a simple cache. 8-(. It seems to me that there could be a logical cache which could be provided by the RAID code, which would sit above the physical block locations, the mirrors of the blocks, the parity data, etc.... Or is this the part that is provided by the filesystem? -- Brad Knowles, "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin, Historical Review of Pennsylvania. GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+ !w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++) tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Mar 18 6:34:55 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C2B4537B401; Tue, 18 Mar 2003 06:34:52 -0800 (PST) Received: from picard.skynet.be (picard.skynet.be [195.238.3.88]) by mx1.FreeBSD.org (Postfix) with ESMTP id C14D243F3F; Tue, 18 Mar 2003 06:34:50 -0800 (PST) (envelope-from brad.knowles@skynet.be) Received: from [10.0.1.2] (180.91-201-80.adsl.skynet.be [80.201.91.180]) by picard.skynet.be (8.12.8/8.12.8/Skynet-OUT-2.21) with ESMTP id h2IEYEEN006892; Tue, 18 Mar 2003 15:34:20 +0100 (MET) (envelope-from ) Mime-Version: 1.0 X-Sender: bs663385@pop.skynet.be Message-Id: In-Reply-To: <3E76C15F.D3C7F9A7@mindspring.com> References: <200303180459.XAA15021@goliath.cnchost.com> <3E76C15F.D3C7F9A7@mindspring.com> Date: Tue, 18 Mar 2003 15:26:12 +0100 To: Terry Lambert From: Brad Knowles Subject: Re: Anyone working on fsck? Cc: Bakul Shah , Julian Elischer , FreeBSD current users , fs@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org At 10:49 PM -0800 2003/03/17, Terry Lambert wrote: > Yes, I know. I'm aware. He has a lot of data to transfer in > from the disk in order to do the reverse lookup, with that much > data on the disk. I'm confused. In situations where you need to do reverse lookups, don't you normally tend to create doubly-linked lists, or other structures that you can traverse quickly and efficiently? > He could change the code so that that everything that wasn't in > use was zeroed. That would help, since then he could just create > a second CG map in an empty file by traversing all inodes and > indirect blocks for non-zero values, not caring if the indirect > blocks were referenced by inodes. Assuming indirect blocks and > data blocks were distinguishable. > > Like I said, there are a lot of FS changes that could help. I'm not sure I understand how this would be a filesystem change. Since this second CG map would be used only during fsck and not otherwise present on the system, couldn't you just throw it away when you were done, resulting in a filesystem that was structurally unchanged from before? > The *reason* it doesn't matter in the CG bitmap case, is that a > bitmap can only tell "allocated" vs. "unallocated"; there is no > third state that means "I haven't checked this bit yet". > > So you can only load up however many bitmaps you are going to > check in a given pass (hopefully they all fit in memory, but > if they don't fitting in the address space does you no good, > because they still aren't tri-state), and then iterate every > single bit reference in existance, and compare them to the ones > you have, and clear them if they aren't referenced by *anyone*. Seems to me that you could easily solve this with a second CG map (as you proposed above), that starts of initially zeroed for all blocks, and as you go through the "normal" CG maps, you turn on all the corresponding bits in the corresponding second map as you touch the blocks that are referenced. This should be a single linear pass, and then you'd be done. If you had one I/O process and multiple worker processes actually scanning the CG maps and updating the second copy, this should just absolutely *fly*. What am I missing here? -- Brad Knowles, "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin, Historical Review of Pennsylvania. GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+ !w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++) tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Mar 18 6:35: 2 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DE73F37B401; Tue, 18 Mar 2003 06:35:00 -0800 (PST) Received: from picard.skynet.be (picard.skynet.be [195.238.3.88]) by mx1.FreeBSD.org (Postfix) with ESMTP id C3D2443FB1; Tue, 18 Mar 2003 06:34:58 -0800 (PST) (envelope-from brad.knowles@skynet.be) Received: from [10.0.1.2] (180.91-201-80.adsl.skynet.be [80.201.91.180]) by picard.skynet.be (8.12.8/8.12.8/Skynet-OUT-2.21) with ESMTP id h2IEYEEL006892; Tue, 18 Mar 2003 15:34:16 +0100 (MET) (envelope-from ) Mime-Version: 1.0 X-Sender: bs663385@pop.skynet.be Message-Id: In-Reply-To: <46517.1047968278@critter.freebsd.dk> References: <46517.1047968278@critter.freebsd.dk> Date: Tue, 18 Mar 2003 15:04:27 +0100 To: "Poul-Henning Kamp" From: Brad Knowles Subject: Re: Anyone working on fsck? Cc: Brad Knowles , Bakul Shah , Julian Elischer , FreeBSD current users , fs@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org At 7:17 AM +0100 2003/03/18, Poul-Henning Kamp wrote: >>> Optimizing fsck is a valid project, I just wish it would be somebody >>> who would also finish the last 30% who would do it. >> >> Just what are you saying? Is Julian Elischer not the right >>person to be working on this, because he has a history of not >>finishing the last 30% of something? > > Yes. Can you substantiate this rather extreme claim? Can you point to specific messages in the archives that demonstrate this? -- Brad Knowles, "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin, Historical Review of Pennsylvania. GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+ !w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++) tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Mar 18 12:14:14 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 43B0237B404; Tue, 18 Mar 2003 12:14:13 -0800 (PST) Received: from mailsrv.otenet.gr (mailsrv.otenet.gr [195.170.0.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id BBB1B43FE0; Tue, 18 Mar 2003 12:14:08 -0800 (PST) (envelope-from keramida@ceid.upatras.gr) Received: from gothmog.gr (patr530-a151.otenet.gr [212.205.215.151]) by mailsrv.otenet.gr (8.12.8/8.12.8) with ESMTP id h2IKDM5u020671; Tue, 18 Mar 2003 22:13:34 +0200 (EET) Received: from gothmog.gr (gothmog [127.0.0.1]) by gothmog.gr (8.12.8/8.12.8) with ESMTP id h2IKDJam003345; Tue, 18 Mar 2003 22:13:19 +0200 (EET) (envelope-from keramida@ceid.upatras.gr) Received: (from giorgos@localhost) by gothmog.gr (8.12.8/8.12.8/Submit) id h2IKD3v4003344; Tue, 18 Mar 2003 22:13:03 +0200 (EET) (envelope-from keramida@ceid.upatras.gr) Date: Tue, 18 Mar 2003 22:13:03 +0200 From: Giorgos Keramidas To: Bakul Shah Cc: Poul-Henning Kamp , Julian Elischer , current@FreeBSD.ORG, fs@FreeBSD.ORG Subject: Re: Anyone working on fsck? Message-ID: <20030318201303.GD1825@gothmog.gr> References: <54291.1047937142@critter.freebsd.dk> <200303172205.RAA18117@goliath.cnchost.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200303172205.RAA18117@goliath.cnchost.com> X-RAVMilter-Version: 8.4.2(snapshot 20021217) (terpsi) Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On 2003-03-17 14:05, Bakul Shah wrote: >> If you start to implement any sort of journaling (that is what you >> talked about in your email), you might as well just stop right at >> the "clean" bit, and avoid the complexity. > > No, I didn't suggest journaling, I suggested storing all state in a > contiguous area (or a small number of such areas). indirect blocks, > keeping track of free blocks, etc. You can still do a completely > exhaustive fsck but it won't be exhausting to you. This has two disadvantages though. One of them is that you'll end up redefining UFS which Julian prohibited. The other one is that you risk corruption if this area (or some of these areas) of the disk has hardware problems. One area only is entirely out of question. If it breaks, you lost it all. Add too many of these areas and you waste a lot of disk space. Of course, there probably is some percentage that can give us all the benefits of accessing only a fraction of the total disk space and still be small enough to be ok. It can even be made tunable with tunefs, as so many other things. But if this is done, it should be done after quite a bit of research. And you still end up modifying UFS. - Giorgos To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Mar 18 14:43:57 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 979D537B404; Tue, 18 Mar 2003 14:43:50 -0800 (PST) Received: from stork.mail.pas.earthlink.net (stork.mail.pas.earthlink.net [207.217.120.188]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9CEA143F3F; Tue, 18 Mar 2003 14:43:49 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0203.cvx22-bradley.dialup.earthlink.net ([209.179.198.203] helo=mindspring.com) by stork.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18vPnu-0001Io-00; Tue, 18 Mar 2003 14:43:31 -0800 Message-ID: <3E77A0C0.B1BEA2D7@mindspring.com> Date: Tue, 18 Mar 2003 14:42:08 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Brad Knowles Cc: Bakul Shah , Julian Elischer , FreeBSD current users , fs@FreeBSD.ORG Subject: Re: Anyone working on fsck? References: <200303180459.XAA15021@goliath.cnchost.com> <3E76C15F.D3C7F9A7@mindspring.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4c02ff085df222b2c13f20f3d35e2cfb1350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Brad Knowles wrote: > At 10:49 PM -0800 2003/03/17, Terry Lambert wrote: > > Yes, I know. I'm aware. He has a lot of data to transfer in > > from the disk in order to do the reverse lookup, with that much > > data on the disk. > > I'm confused. In situations where you need to do reverse > lookups, don't you normally tend to create doubly-linked lists, or > other structures that you can traverse quickly and efficiently? Yes and no. "Yes", if that usage is typical. "No", if it's do extraordinary that it's to the point of "infintessimally small". This is a case of optimizing the success code path at the expense of the failure code path, and is not only accepted practice, it's good practice. The specific issue here is that the CG bitmaps normally exist to give layout hints to the block allocator, which would normally take a huge amount of traversal in order to find free blocks on its own. They exist, in other words, to find free blocks fast. The problem that happens is when the data in a bitmap is stale; but it can only be stale in a particular way: when a block is freed back to the FS, the last operation which occurs, in the ordered update of the metadata, is to mark it free in the CG bitmap itself. So the failure case from which the recovery is proceeding is "a failure to mark a block which is otherwise unallocated as unallocated in the CG bitmap". What this boils down to is looking to see if your table *doesn't* contain an entry, after recovery from a catastropic failure, where the only thing that could be out of date is there being an entry in the table that shouldn't be there. Meanwhile, when performing allocations, you are looking for the 0 bits... the entries which are *not* there. Because of this, it's safe to go through and use the FS, and make new allocations, since the table can only have erroneously set bits... meaning you cannot allocate a block that's not allocated. You will *never* make the mistake of allocating a block that's *already* allocated. So the only thing you care about on recovery, which can, in theory, take as much time as necessary to do it's job, is clearing the bits that aren't valid. Meanwhile, new allocations are occurring all the time, since the FS in question is "live". So what you do is bound the problem set by establishing a "snapshot" of the FS, which, itself, adds considerable overhead, and then you allow any allocations you want to to continue in the live FS. This is because the bits you want to clear will not be validly set by the live FS: you can clear them any time, and it will be valid to clear them in both the snapshot and in the live FS, without risking the integrity of the live FS. Another way of bounding the problem would be to "read lock" the CG's whose bitmaps you are currently examining, and any new allocations which attempt to fall in these locked regions are redirected. There are a couple of downsides to that approach, too, but it can be made to work. In any case, whether you are operating on a live FS or a snapshot, or performing a foreground fsck on a quiescent/RO/unmounted FS, the problem is the same: you must select a CG, cache a copy of it in memory, create a "scratch" copy, and then examine all block allocations, everywhere, and where they indicate a block allocated in the cylinder group in hand, clear the bit in the scratch copy. The set of bits you are left with is the set of bits that are not actaully allocated. At this point, you lock the live FS version of the CG (if you are not operating on a read-locked CG *in* a live FS, of course), and AND the real CG with the NOT of the copy, and use that to replace the old bitmap... thus clearing the "phantom allocations". Then you go onto the next CG bitmap. Make sense now? > > He could change the code so that that everything that wasn't in > > use was zeroed. That would help, since then he could just create > > a second CG map in an empty file by traversing all inodes and > > indirect blocks for non-zero values, not caring if the indirect > > blocks were referenced by inodes. Assuming indirect blocks and > > data blocks were distinguishable. > > > > Like I said, there are a lot of FS changes that could help. > > I'm not sure I understand how this would be a filesystem change. > Since this second CG map would be used only during fsck and not > otherwise present on the system, couldn't you just throw it away when > you were done, resulting in a filesystem that was structurally > unchanged from before? Newfs would have to do a heck of a lot more writing than before. Freeing would have to write zeroed information. Data blocks that might be used as indirect blocks would have to be zeroed. Etc.. Basically, you would eat the penalty as a small additional overhead during normal operation, to reduce the cost by one order, in case of failure. For a really, really high failure cost at a given known probability of failure, this cost might amortize out as cheaper, in terms of overall operational cost. > > The *reason* it doesn't matter in the CG bitmap case, is that a > > bitmap can only tell "allocated" vs. "unallocated"; there is no > > third state that means "I haven't checked this bit yet". [ ... ] > > Seems to me that you could easily solve this with a second CG map > (as you proposed above), that starts of initially zeroed for all > blocks, and as you go through the "normal" CG maps, you turn on all > the corresponding bits in the corresponding second map as you touch > the blocks that are referenced. No; it's more tricky; see my example above. Your scratch block has to start with all the bits set that are already set in the CG bitmap, not with all zeros. The reason for this is complicated, but what it boils down to is that you need to be able to make additional allocations, and you need to know the difference between new additional allocations which occurred as a result of operations in files/areas you have already traversed in the currently occurring pass. Otherwise, you can't just read-lock the CG bitmap, you must write-lock it as well. Using the snapshot approach is one way of avoiding this issue. > This should be a single linear pass, and then you'd be done. If > you had one I/O process and multiple worker processes actually > scanning the CG maps and updating the second copy, this should just > absolutely *fly*. > > What am I missing here? The fact that for a 1TB FS, you don't have enough virtual address space to hold all the CG bitmaps and all the scratch memory and the mapped buffers of the FS data you are traversing in your head at once? 8-) 8-). Even if you did, you'd end up paging, and you are actually worse off doing that, then you are doing multiple linear passes, after breaking the CG's into blocks of CG's to check in linear runs. That's why I suggested an MADV_WIRE for mapping memory to contain the CG bitmaps you are interested in perusing per pass. The reason you are worse off comes down to the fact that, on a linear traversal of disk data which is so much larger than your available cache, you will *never* have a case where the block you are interested in looking at is already in cache. One of the "sneaky" ways of dealing with this is to use an "elevator" algorithm: for one CG set, scan forward through the FS; for the next, scan backward. This means you will get cache effects, since the data you examine will be the data you just examined, until you run out of LRU, and then you degrade to having to read all data in. An even more "sneaky" way would be to *know* your LRU list length, and start a *forward* scan from there. That would be 2X as effective, since you would get a 100% cache hit, at least for the length of the LRU, until you, once again, end up in the degenerate case. This is one of the reasons VMS had the concept of being able to know both your "working set quota" and your "current working set size". Another "sneaky" approach, which I've already suggested, is to reduce the size of the working set you care about, by increasing the locality between "random" FS data and the CG bitmap, so that any allocations you care about will be in a smaller portion of the disk. This is automatic, in the case of "make smaller partitions"; it requires a change in disk layout policy otherwise, and that change could damage overall performance, in the same way "growfs" does, by increasing regional fragmentation. Even so, there's no way to keep it from breaking down for very large files which spanned a (for lack of better terminology) "cylinder group group" boundary. Even a "sliding group window" would only save you another 50% of the time (assume average locality is pinned to the inode location). It's a pretty intractable problem, with existing FFS structure; you can get minor performance wins, by holding multiple CG's in hand to do the job (maybe all of them, if the FS is small enough, which would make it 1 pass), but for a very large FS, you are pretty much toast, unless you are willing to change FS layout *some*. I rather expect that UFS2 will end up changing layout, as people start using it for very large filesystems. Actually, thinking about it, you could bound it to a single non-CG bitmap metadata traversal in all recovery situations... if you were willing to change FS layout, in a probably compatible way. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Mar 18 16:25:21 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 75D2937B401; Tue, 18 Mar 2003 16:25:20 -0800 (PST) Received: from vador.skynet.be (vador.skynet.be [195.238.3.236]) by mx1.FreeBSD.org (Postfix) with ESMTP id E151F43FB1; Tue, 18 Mar 2003 16:25:18 -0800 (PST) (envelope-from brad.knowles@skynet.be) Received: from [10.0.1.2] (86.11-136-217.adsl.skynet.be [217.136.11.86]) by vador.skynet.be (8.12.8/8.12.8/Skynet-OUT-2.21) with ESMTP id h2J0P3Xv007339; Wed, 19 Mar 2003 01:25:03 +0100 (MET) (envelope-from ) Mime-Version: 1.0 X-Sender: bs663385@pop.skynet.be Message-Id: In-Reply-To: <3E77A0C0.B1BEA2D7@mindspring.com> References: <200303180459.XAA15021@goliath.cnchost.com> <3E76C15F.D3C7F9A7@mindspring.com> <3E77A0C0.B1BEA2D7@mindspring.com> Date: Wed, 19 Mar 2003 01:15:46 +0100 To: Terry Lambert From: Brad Knowles Subject: Re: Anyone working on fsck? Cc: Brad Knowles , Bakul Shah , Julian Elischer , FreeBSD current users , fs@FreeBSD.ORG Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org At 2:42 PM -0800 2003/03/18, Terry Lambert wrote: > Make sense now? No. However, I am now convinced that I don't understand enough of how the filesystem works to even be able to ask the simplest of questions about how this process can be improved. So, I will now shut up. -- Brad Knowles, "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin, Historical Review of Pennsylvania. GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+ !w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++) tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Mar 18 16:34:57 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 605F337B401; Tue, 18 Mar 2003 16:34:55 -0800 (PST) Received: from stork.mail.pas.earthlink.net (stork.mail.pas.earthlink.net [207.217.120.188]) by mx1.FreeBSD.org (Postfix) with ESMTP id AEC7B43F3F; Tue, 18 Mar 2003 16:34:52 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0182.cvx22-bradley.dialup.earthlink.net ([209.179.198.182] helo=mindspring.com) by stork.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18vRXb-0000jk-00; Tue, 18 Mar 2003 16:34:48 -0800 Message-ID: <3E77BAD3.12447056@mindspring.com> Date: Tue, 18 Mar 2003 16:33:23 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Brad Knowles Cc: Bakul Shah , Julian Elischer , FreeBSD current users , fs@FreeBSD.ORG Subject: Re: Anyone working on fsck? References: <200303180459.XAA15021@goliath.cnchost.com> <3E76C15F.D3C7F9A7@mindspring.com> <3E77A0C0.B1BEA2D7@mindspring.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a453fd58d6e942106ea548b70a2c2bd86e350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Brad Knowles wrote: > At 2:42 PM -0800 2003/03/18, Terry Lambert wrote: > > Make sense now? > > No. > > However, I am now convinced that I don't understand enough of how > the filesystem works to even be able to ask the simplest of questions > about how this process can be improved. So, I will now shut up. You should read the "A Fast Filesystem for UNIX" paper by McKusick, et. al.. The Viva FS paper I mentioned the other day is also good background on placement policies. Actually, I have thought out a (relatively) simple way of making all CG fsck's take a single pass, but it requires access to some free blocks on the FS itself, and it has a worrisome amount of writes that end up happening to the same region of the disk, so I don't know what it means for the failure rate. The only space I can see to grab for it is something like the quota stuff, or to steal inode #1 from the whiteouts. But the resulting fsck could be *guaranteed* to take only a single pass through the data. Further, it could be bounded, to ensure the fsck process does not need to swap data sets in hand (but compared to the single pass optimization, going from O(3)->O(1), that's just gravy). It would only require a single bit change to the superblock, too, if the space was found. Perhaps there is a "don't care" area available for this, like I suggested before UFS2 was started? -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Tue Mar 18 17: 6:41 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A5F3A37B401 for ; Tue, 18 Mar 2003 17:06:40 -0800 (PST) Received: from geekpunk.net (adsl-32-194-137.bna.bellsouth.net [67.32.194.137]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9FC1E43F85 for ; Tue, 18 Mar 2003 17:06:39 -0800 (PST) (envelope-from bandix@geekpunk.net) Received: from localhost.my.domain (taran [127.0.0.1]) by geekpunk.net (8.12.6/8.12.6) with ESMTP id h2IKPLhI032009; Tue, 18 Mar 2003 14:25:21 -0600 (CST) (envelope-from bandix@geekpunk.net) Received: (from bandix@localhost) by localhost.my.domain (8.12.6/8.12.6/Submit) id h2IKPGeL032008; Tue, 18 Mar 2003 14:25:16 -0600 (CST) (envelope-from bandix) Date: Tue, 18 Mar 2003 14:25:16 -0600 From: "Brandon D. Valentine" To: Miguel Mendez Cc: des@ofug.org, fs@freebsd.org Subject: Re: Anyone working on fsck? Message-ID: <20030318202516.GB25577@geekpunk.net> References: <20030318110852.438ffd2e.flynn@energyhq.homeip.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030318110852.438ffd2e.flynn@energyhq.homeip.net> User-Agent: Mutt/1.4i Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org [ Cc: trimmed. ] On Tue, Mar 18, 2003 at 11:08:52AM +0100, Miguel Mendez wrote: > On Tue, 18 Mar 2003 10:59:14 +0100 Dag-Erling Sm?rgrav wrote: > > Let's rather work on getting a working log-structured filesystem > > committed so we don't *need* fsck for filesystems that large. > > Rather than reinventing the wheel, how about porting NetBSD's LFS to > FreeBSD? Granted it's not 100% reliable yet, but it's improving. I'd > rather have that than wait for JFS4BSD to be ready. In fact, I'd > volunteer to work with my (limited) knowledge. While I do not have the time nor expertise to work on any such project at this time I would *love* to see FreeBSD developers work with the NetBSD folks toward reviving LFS. I think there is a definite demand for a BSD-licensed log-structured filesystem and the most logical place to begin is with the extant example. Brandon D. Valentine -- brandon@dvalentine.com http://www.geekpunk.net Pseudo-Random Googlism: nashville is considered to have a retail trade area with a radius of approximately 100 miles and encompassing a population of more than two million To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Wed Mar 19 7:22:13 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B0C2537B407; Wed, 19 Mar 2003 07:22:11 -0800 (PST) Received: from mailgw.cscoms.com (mailgw.cscoms.com [202.183.255.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id C01F243F75; Wed, 19 Mar 2003 07:22:06 -0800 (PST) (envelope-from wowwwhealthy@thaimail.com) Received: from cscoms.com (mail.cscoms.com [202.183.255.23]) by mailgw.cscoms.com (8.12.8/8.12.3) with ESMTP id h2JFDqil015445; Wed, 19 Mar 2003 22:13:53 +0700 (ICT) Received: from ME (dial-255.ras-7.bkk.c.cscoms.com [203.170.141.193]) by cscoms.com (8.12.8/8.12.3) with SMTP id h2JF9rwo010446; Wed, 19 Mar 2003 22:09:56 +0700 (GMT) Date: Wed, 19 Mar 2003 22:09:53 +0700 (GMT) Message-Id: <200303191509.h2JF9rwo010446@cscoms.com> From: wowwwhealthy@thaimail.com Subject: ท่านทราบหรือไม่ว่าคนอ้วนจะเสี่ยงต่อการเป็นเบาหวานมากกว่าคนน้ำหนักปกติถึง 30 เท่า X-Priority: 1 (Highest) Reply-To: wowwwhealthy@thaimail.com X-Mailer: Microsoft Outlook Express 5.00.2615.200 MIME-Version: 1.0 Content-type: multipart/mixed; boundary="#MYBOUNDARY#" X-Virus-Scanned: by amavisd-milter (http://amavis.org/) To: undisclosed-recipients: ; Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org --#MYBOUNDARY# Content-Type: text/plain; charset=ansi Content-Transfer-Encoding: 8bit คนไทยกำลังเป็นโรคอ้วนมากขึ้นทุกที พ.อ.หญิง รศ. พ.ญ. พรฑิตา ชัยอำนวย ผู้อำนวยการเวชศาสตร์ฟื้นฟู โรงพยาบาลพระมงกุฏเกล้า บรรยายเรื่อง "กินอย่างไรให้ห่างไกลโรคหัวใจและโรคอ้วน" ในตอนหนึ่งของการบรรยาย ผู้บรรยายกล่าวว่า "สิ่งที่พึงตระหนักคือ ผู้ชายไม่ควรให้รอบเอวเกิน 36 นิ้ว หญิงไม่ควรเกิน 32 นิ้ว ถ้ามากกว่านี้ต้องเร่งลดน้ำหนัก" เพราะถ้าหากท่านวัดรอบเอวแล้วได้ตัวเลขเกินกว่ามาตราฐานนี้ แสดงว่าท่านกำลังเป็นโรคอ้วน คนเป็นโรคอ้วนมีความเสี่ยงที่จะต้อง พบกับโรคร้ายต่างๆ มากมาย นับตั้งแต่ โรคหัวใจ เบาหวาน ไขมันในเลือดสูง ความดันโลหิตสูง อัมพาต และท่านอาจจะหยุดหายใจขณะหลับ จนเกิดภาวะพร่องออกซิเจน ตื่นนอนจะมีอาการมึน เป็นต้อหินง่ายเนื่องจากเลือดขาดออกซิเจน เป็นโรคข้อ เพราะแบกรับน้ำหนักมาก เป็นเกาต์ มะเร็ง นิ่วในถุงน้ำดี มีลูกยาก โรคเกี่ยวกับระบบหายใจ โรคถุงน้ำดี ท่านทราบหรือไม่ว่าคนอ้วนจะเสี่ยงต่อการเป็นเบาหวานมากกว่าคนน้ำหนักปกติถึง 30 เท่า เสี่ยงเป็นโรคหลอดเลือดหัวใจตีบกว่าคนทั่วไป 15 เท่า โรคอัมพาต 11 เท่า โรคมะเร็งลำไส้ 2 เท่า คนเป็นโรคอ้วนเป็นโรคร้ายตายง่ายอย่างนี้ถ้าไม่เรียกคนที่มีรอบเอวเกินมาตราฐานว่า รอบเอวมรณะ แล้วจะเรียกว่าอะไรล่ะครับ วิธีถอดห่วงยาง (ลดเอว) คุณหมอบอกว่า วิธีรักษาโรคอ้วนสามารถทำได้ด้วยการควบคุมแคลอรีของอาหารที่รับประทาน คือพยายามให้ลดลงวันละ 600 แคลอรี ซึ่งภายใน 7 วันท่านจะสามารถลดน้ำหนักได้0.6 กิโลกรัม เพราะไขมัน 1 กิโลกรัม เท่ากับ 7,000 แคลอรี ประการที่สำคัญ ต้องออกกำลังกายอย่างสม่ำเสมอทุกวันครับ อย่างน้อย 20 นาที ถ้าออกกำลังกายได้ 60 นาทีจะยิ่งเป็นผลดี คุณหมอบอกว่าเราควรให้สนใจใฝ่ศึกษาหาความรู้ด้านโภชนาการให้มาก ๆ คือให้ศึกษาว่าอาหารชนิดไหนให้พลังงานน้อย พลังมากแค่ไหน และ ควรสร้างความสุขที่ได้บริโภคอาหารไขมันต่ำ ๆ ในการรับประทานอาหารควรเคี้ยวให้ช้าๆ จะรู้สึกอิ่มทั้งๆ ที่บริโภคน้อย อ้อ ! budpage แนะนำว่าท่านควรซื้อสายวัดมาเก็บไว้สักเส้นนะครับ ทุก ๆ เช้าคอยวัดเอวตัวเอง และควรจดบันทึกเป็นสถิติไว้ทุกวันด้วย สนุกดีครับ อีกทั้งยังเป็นการท้าทายให้เรามีความตื่นตัวที่จะลดความอ้วนอยู่เสมออีกด้วย สุดท้ายนี้ขอให้ทุก ๆ ท่านมีรอบเอวในระดับมาตราฐานที่ปลอดภัยกันทุก ๆ ท่านนะครับ (จบบทความนี้ เวบมาสเตอร์ คงขอตัวไปซื้อสายวัดมาควบคุมน้ำหนักด้วยคนนะครับ สวัสดี ) ขอเชิญชาวพุทธมาช่วยกันระดมความคิดในหัวข้อ "วิธีออกกำลังกายให้สนุก" ลองมาดูกันว่าใครจะมีวิธีการเด็ดๆ ที่จะทำให้คนอ้วนอยากออกกำลังกายโดยไม่ฝืนใจกันบ้าง แล้วพบกันใหม่กับเอกสารสาระที่มีประโยชน์ฉบับหน้าค่ะ ***************************************************************** ถ้าท่านต้องการข้อมูลที่มีประโยชน์ในด้านโภชนาการเพื่อสุขภาพที่แข็งแรง หรือต้องการลดน้ำหนักโดยวิธีธรรมชาติ สามารถขอข้อมูลได้จาก ... http://www.geocities.com/healthclub999/easythin ***************************************************************** หากไม่ต้องการรับข้อมูลข่าวสารอีก กรุณาเข้าไปแจ้งที่ www.unsubhealthclub.web1000.com --#MYBOUNDARY#-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Thu Mar 20 12:53:23 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 50A0837B401 for ; Thu, 20 Mar 2003 12:53:22 -0800 (PST) Received: from filer.fsl.cs.sunysb.edu (filer.fsl.cs.sunysb.edu [130.245.126.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id E081D43F3F for ; Thu, 20 Mar 2003 12:53:19 -0800 (PST) (envelope-from ezk@fsl.cs.sunysb.edu) Received: from agora.fsl.cs.sunysb.edu (IDENT:Gf8K6Nw2kMQ/AqItNfCqfYxCIkR42BVW@agora.fsl.cs.sunysb.edu [130.245.126.12]) by filer.fsl.cs.sunysb.edu (8.12.5/8.12.8) with ESMTP id h2KKqush008641; Thu, 20 Mar 2003 15:52:56 -0500 Received: from agora.fsl.cs.sunysb.edu (IDENT:x/0dvmeQW1g4BZyCeNE+gEICLjU0klcw@localhost.localdomain [127.0.0.1]) by agora.fsl.cs.sunysb.edu (8.12.8/8.12.8) with ESMTP id h2KKqwb9019148; Thu, 20 Mar 2003 15:52:58 -0500 Received: (from ezk@localhost) by agora.fsl.cs.sunysb.edu (8.12.8/8.12.8/Submit) id h2KKqwTf019144; Thu, 20 Mar 2003 15:52:58 -0500 Date: Thu, 20 Mar 2003 15:52:58 -0500 Message-Id: <200303202052.h2KKqwTf019144@agora.fsl.cs.sunysb.edu> From: Erez Zadok To: freebsd-fs@FreeBSD.ORG Subject: cannot append on 5.0-RELEASE w/ NFSv2 Cc: X-MailKey: Erez Zadok Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org I'm not sure if this is already known, but I came across what I think is an NFSv2 client-side bug in 5.0-RELEASE. My NFS server is Linux red hat 8.0 w/ latest errata kernel, w/ export options "rw,async". When my fbsd5 box mounts with -o nfsv2, the following /bin/sh (and /bin/bash) append command fails with "Permission Denied": echo foo >> bar when "bar" is NFS mounted from the linux server. I remounted the server w/ nfsv3 (took out the -o nfsv2 option) and appending works well. I didn't try -current or other NFS servers. Cheers, Erez. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Thu Mar 20 14:12:39 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C951B37B401 for ; Thu, 20 Mar 2003 14:12:38 -0800 (PST) Received: from smtp02.syd.iprimus.net.au (smtp04.syd.iprimus.net.au [210.50.76.52]) by mx1.FreeBSD.org (Postfix) with ESMTP id EC22843F75 for ; Thu, 20 Mar 2003 14:12:37 -0800 (PST) (envelope-from tim@dilbert.robbins.dropbear.id.au) Received: from dilbert.robbins.dropbear.id.au (210.50.36.32) by smtp02.syd.iprimus.net.au (7.0.008) id 3E796B430001F4C9; Fri, 21 Mar 2003 09:11:54 +1100 Received: from dilbert.robbins.dropbear.id.au (qn7dwusm1lb48tm3@localhost [127.0.0.1]) by dilbert.robbins.dropbear.id.au (8.12.8/8.12.8) with ESMTP id h2KMBoWa001823; Fri, 21 Mar 2003 09:11:50 +1100 (EST) (envelope-from tim@dilbert.robbins.dropbear.id.au) Received: (from tim@localhost) by dilbert.robbins.dropbear.id.au (8.12.8/8.12.8/Submit) id h2KMBo1c001822; Fri, 21 Mar 2003 09:11:50 +1100 (EST) Date: Fri, 21 Mar 2003 09:11:49 +1100 From: Tim Robbins To: Erez Zadok Cc: freebsd-fs@FreeBSD.org Subject: Re: cannot append on 5.0-RELEASE w/ NFSv2 Message-ID: <20030321091149.A1061@dilbert.robbins.dropbear.id.au> References: <200303202052.h2KKqwTf019144@agora.fsl.cs.sunysb.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <200303202052.h2KKqwTf019144@agora.fsl.cs.sunysb.edu>; from ezk@cs.sunysb.edu on Thu, Mar 20, 2003 at 03:52:58PM -0500 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Thu, Mar 20, 2003 at 03:52:58PM -0500, Erez Zadok wrote: > I'm not sure if this is already known, but I came across what I think is an > NFSv2 client-side bug in 5.0-RELEASE. My NFS server is Linux red hat 8.0 w/ > latest errata kernel, w/ export options "rw,async". > > When my fbsd5 box mounts with -o nfsv2, the following /bin/sh (and > /bin/bash) append command fails with "Permission Denied": > > echo foo >> bar > > when "bar" is NFS mounted from the linux server. > > I remounted the server w/ nfsv3 (took out the -o nfsv2 option) and appending > works well. I recognise this as being similar to a bug in smbfs that I fixed just before 5.0 was released. I'll fix NFS later today. Tim To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Fri Mar 21 8:40:28 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5AED937B401; Fri, 21 Mar 2003 08:40:26 -0800 (PST) Received: from mailgw.cscoms.com (mailgw.cscoms.com [202.183.255.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8142A43FDD; Fri, 21 Mar 2003 08:40:17 -0800 (PST) (envelope-from FreeBooklet@thaimail.com) Received: from cscoms.com (mail.cscoms.com [202.183.255.23]) by mailgw.cscoms.com (8.12.8/8.12.3) with ESMTP id h2LGZJET032180; Fri, 21 Mar 2003 23:35:36 +0700 (ICT) Received: from ME (dial-49.ras-7.bkk.c.cscoms.com [203.170.129.49]) by cscoms.com (8.12.8/8.12.3) with SMTP id h2LGV3wo017054; Fri, 21 Mar 2003 23:31:07 +0700 (GMT) Date: Fri, 21 Mar 2003 23:31:03 +0700 (GMT) Message-Id: <200303211631.h2LGV3wo017054@cscoms.com> From: FreeBooklet@thaimail.com Subject: แจกฟรี ! หนังสือคู่มือคนเคยจน สำหรับผู้สนใจ.... X-Priority: 1 (Highest) Reply-To: FreeBooklet@thaimail.com X-Mailer: Microsoft Outlook Express 5.00.2615.200 MIME-Version: 1.0 Content-type: multipart/mixed; boundary="#MYBOUNDARY#" X-Virus-Scanned: by amavisd-milter (http://amavis.org/) To: undisclosed-recipients: ; Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org --#MYBOUNDARY# Content-Type: text/plain; charset=ansi Content-Transfer-Encoding: 8bit >>>>ทำงานที่ยากที่สุดก่อน >>>> >>ผมยิ่งมีชีวิตอยู่นานเท่าไหร่ ผมยิ่งมั่นใจมากขึ้นเท่านั้น >>ว่าความแตกต่างอันยิ่งใหญ่ระหว่างมนุษย์... >>ระหว่าคนที่อ่อนแอและคนทีอำนาจ....... >>ระหว่างคนที่ยิ่งใหญ่และคนที่ไม่สำคัญ ก็คือ >>เรี่ยวแรงของ....ความตั้งใจแน่วแน่ที่ไม่อาจทำลายได้.... >>จุดประสงค์ที่เมื่อตั้งขึ้นแล้ว ถ้าไม่ตายก็ต้องชนะ >>-เซอร์โธมัส ฟาวเวล บั๊กซ์ตัน >>หนึ่งในเทคนิคที่ดีที่สุดในการเอาชนะนิสัยผัดวันประกันพรุ่ง >>และทำงานได้มากขึ้นและเร็วขึ้นก็คือลงมือทำงานที่ยากที่สุดของคุณก่อน >>นี่คือการ " กินกบของคุณ " ที่แท้จริง มันเป็นทักษะส่วนบุคคลในการบริหาร >>ที่ยากที่สุดและสำคัญที่สุดเริ่มต้นตอนเช้าด้วยงานที่ใหญ่ที่สุดและสำคัญที่สุด >>คือ สิ่งตรงข้ามกับที่คนส่วนใหญ่ทำ ระเบียบวินัยนี้จะทำให้คุณเลิกนิสัย ผัดวัน >>ประกันพรุ่งและทำให้อนาคตอยู่ในกำมือคุณ >>>>>>>>การเริ่มต้นแต่ละวันด้วยงานที่ยากที่สุดเป็นการเริ่มต้นแบบก้าวกระโดด >>ที่ดี คุณจะมีไฟมากขึ้น และจะทำงานได้ผลดีมากขึ้น >>>>>>>>ในวันที่คุณเริ่มลงมือทำงานสำคัญโดยทันทีทันควัน คุณจะรู้สึกดีกับตัว >>คุณเองและกับงานของคุณมากกว่าคนอื่นๆ คุณจะรู้สึกมีอำนาจมากขึ้น ควบคุม >>ตัวเองได้มากขึ้นและรับผิดชอบดูแลชีวิตตัวเองได้มากกว่าเวลาอื่น >>>>>>>สร้างนิสัยเริ่มทำงานที่ยากที่สุดก่อนแล้วคุณจะไม่ต้องมองย้อนกลับ >>คุณจะกลายเป็น หนึ่งในคนที่มีประสิทธิภาพมากที่สุดในคนรุ่นคุณ............... >>กินกบตัวนั้นซะ!!! จงมองตัวเองว่าเป็นงานที่กำลังคืบหน้า จงเทใจให้กับการเพาะนิสัย >>เป็นคนมีผลงานสูงด้วยการฝึกซ้ำแล้วซ้ำเล่าจนกระทั่งมันกลายเป็นเรื่องอัตโนมัติและ >>กลายเป็นเรื่องง่าย >>>>>>>>หนึ่งในวลีที่มีอนุภาพมากที่สุดซึ่งคุณสามารถเรียนรู้และนำมาใช้ได้ก็คือ >>" เพื่อวันนี้เท่านั้น! "อย่าวิตกเรื่องการเปลี่ยนแปลงชีวิตตัวเอง ถ้ามันฟังเหมือนเป็น >>ความคิดที่ดี จงทำมัน" เพื่อวันนี้เท่านั้น" >>>>>>>>บอกกับตัวเองว่า " เพื่อวันนี้เท่านั้น ฉันจะวางแผนเตรียมการ และเริ่มต้นงาน >>ที่ยากที่สุดก่อนจะทำอย่างอื่น "แล้วคุณจะต้องทึ่งกับความแตกต่างที่เกิดขึ้นในชีวิตคุณ ---------------------------------------------------------------------------------------------- คุณ พร้อมแล้วรึยัง กับรูปแบบการทำงานง่ายๆจากที่บ้าน Click Here! www.geocities.com/thaigetrich/easywork , หรือ Tel. 0-2277-7850 กด 25 ----------------------------------------------------------------------------------------------- ขออภัยหากเป็นการรบกวน และหากไม่ต้องการให้ส่งข่าวสารมายังท่านอีก กรุณาเมลล์มาที่ easywork@maildozy.com หัวข้อ unsub --#MYBOUNDARY#-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sat Mar 22 7:17:47 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6317137B401; Sat, 22 Mar 2003 07:17:46 -0800 (PST) Received: from comp.chem.msu.su (comp-ext.chem.msu.su [158.250.32.157]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0932E43F75; Sat, 22 Mar 2003 07:17:45 -0800 (PST) (envelope-from yar@comp.chem.msu.su) Received: from comp.chem.msu.su (localhost [127.0.0.1]) by comp.chem.msu.su (8.12.3/8.12.3) with ESMTP id h2MFGugO034603; Sat, 22 Mar 2003 18:17:42 +0300 (MSK) (envelope-from yar@comp.chem.msu.su) Received: (from yar@localhost) by comp.chem.msu.su (8.12.3/8.12.3/Submit) id h2MFGuJ1034602; Sat, 22 Mar 2003 18:16:56 +0300 (MSK) (envelope-from yar) Date: Sat, 22 Mar 2003 18:16:56 +0300 From: Yar Tikhiy To: ppc@freebsd.org, fs@freebsd.org Subject: HFS/HFS Plus driver and tools for 5.x are available Message-ID: <20030322151656.GA34184@comp.chem.msu.su> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.3i Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Hello, I'm glad to offer to the community an HFS and HFS Plus filesystem module and tools, ported from Darwin to FreeBSD 5. Of course, it's a rather early version that deserves thorough testing. Please see http://people.freebsd.org/~yar/hfs/ for details. I'm looking forward for your feedback. -- Yar To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sat Mar 22 13:55:44 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0C59637B401; Sat, 22 Mar 2003 13:55:42 -0800 (PST) Received: from obsecurity.dyndns.org (adsl-63-207-60-150.dsl.lsan03.pacbell.net [63.207.60.150]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0C1CF43F85; Sat, 22 Mar 2003 13:55:41 -0800 (PST) (envelope-from kris@obsecurity.org) Received: from rot13.obsecurity.org (rot13.obsecurity.org [10.0.0.5]) by obsecurity.dyndns.org (Postfix) with ESMTP id ADC6B66B9B; Sat, 22 Mar 2003 13:55:40 -0800 (PST) Received: by rot13.obsecurity.org (Postfix, from userid 1000) id 934321282; Sat, 22 Mar 2003 13:55:40 -0800 (PST) Date: Sat, 22 Mar 2003 13:55:40 -0800 From: Kris Kennaway To: kirk@mckusick.com, fs@FreeBSD.org, current@FreeBSD.org Subject: panic: softdep_disk_io_initiation: read Message-ID: <20030322215540.GA30318@rot13.obsecurity.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Dxnq1zWXvFF0Q93v" Content-Disposition: inline User-Agent: Mutt/1.4i Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org --Dxnq1zWXvFF0Q93v Content-Type: text/plain; charset=us-ascii Content-Disposition: inline I've got 3 of these so far (2 on alpha, 1 on i386) since updating to current as of 17 March. panic: softdep_disk_io_initiation: read Debugger("panic") Stopped at Debugger+0x54: xchgl %ebx,in_Debugger.0 db> trace Debugger(c041df8a,c04935e0,c0430d64,d7a50b20,1) at Debugger+0x54 panic(c0430d64,d7a50b38,c029caa9,c04b8e80,ce4fa730) at panic+0xab softdep_disk_io_initiation(ce4fa730,10012,c51c936c,34e,20020080) at softdep_disk_io_initiation+0xac cluster_wbuild(c51c936c,4000,53,0,2) at cluster_wbuild+0x3ed vfs_bio_awrite(ce62d5a8,0,c0431afe,d4,c02afa16) at vfs_bio_awrite+0x1f9 ffs_fsync(d7a50c48,12,c41164b0,484,0) at ffs_fsync+0x2b2 ffs_sync(c4128e00,3,c14e7e00,c41164b0,c4128e00) at ffs_sync+0x18e sync_fsync(d7a50cd0,20002,c41164b0,6ae,0) at sync_fsync+0x16a sched_sync(0,d7a50d48,c041ba89,363,0) at sched_sync+0x17e fork_exit(c02aed50,0,d7a50d48) at fork_exit+0xc4 fork_trampoline() at fork_trampoline+0x1a --- trap 0x1, eip = 0, esp = 0xd7a50d7c, ebp = 0 --- db> #0 doadump () at /a/asami/portbuild/i386/src-client/sys/kern/kern_shutdown.c:239 #1 0xc0165bf5 in db_fncall (dummy1=1016, dummy2=0, dummy3=1016, dummy4=0xd7a50948 "") at /a/asami/portbuild/i386/src-client/sys/ddb/db_command.c:546 #2 0xc0165972 in db_command (last_cmdp=0xc0443f60, cmd_table=0x0, aux_cmd_tablep=0xc043dac4, aux_cmd_tablep_end=0xc043dac8) at /a/asami/portbuild/i386/src-client/sys/ddb/db_command.c:346 #3 0xc0165a86 in db_command_loop () at /a/asami/portbuild/i386/src-client/sys/ddb/db_command.c:470 #4 0xc016880a in db_trap (type=3, code=0) at /a/asami/portbuild/i386/src-client/sys/ddb/db_trap.c:72 #5 0xc03c2ab1 in kdb_trap (type=3, code=0, regs=0xd7a50a9c) at /a/asami/portbuild/i386/src-client/sys/i386/i386/db_interface.c:166 #6 0xc03d3d1f in trap (frame= {tf_fs = -1069481960, tf_es = 16, tf_ds = -1068826608, tf_edi = -1005493072, tf_esi = 256, tf_ebp = -677049624, tf_isp = -677049656, tf_ebx = 0, tf_edx = 0, tf_ecx = 1, tf_eax = 18, tf_trapno = 3, tf_err = 0, tf_eip = -1069798060, tf_cs = 8, tf_eflags = 646, tf_esp = -1069321409, tf_ss = -1069424758}) at /a/asami/portbuild/i386/src-client/sys/i386/i386/trap.c:602 #7 0xc03c4408 in calltrap () at {standard input}:96 #8 0xc025a3bb in panic (fmt=0x0) at /a/asami/portbuild/i386/src-client/sys/kern/kern_shutdown.c:528 #9 0xc0367ecc in softdep_disk_io_initiation (bp=0xce4fa730) at /a/asami/portbuild/i386/src-client/sys/ufs/ffs/ffs_softdep.c:3466 #10 0xc02a5b3d in cluster_wbuild (vp=0xc51c936c, size=16384, start_lbn=84, len=2) at buf.h:422 #11 0xc029ee79 in vfs_bio_awrite (bp=0xce62d5a8) at /a/asami/portbuild/i386/src-client/sys/kern/vfs_bio.c:1682 #12 0xc036f872 in ffs_fsync (ap=0xd7a50c48) at /a/asami/portbuild/i386/src-client/sys/ufs/ffs/ffs_vnops.c:255 #13 0xc036ea1e in ffs_sync (mp=0xc4128e00, waitfor=3, cred=0xc14e7e00, td=0xc41164b0) at vnode_if.h:612 #14 0xc02b21ca in sync_fsync (ap=0xd7a50cd0) at /a/asami/portbuild/i386/src-client/sys/kern/vfs_subr.c:3493 #15 0xc02aeece in sched_sync () at vnode_if.h:612 #16 0xc0246ac4 in fork_exit (callout=0xc02aed50 , arg=0x0, frame=0x0) at /a/asami/portbuild/i386/src-client/sys/kern/kern_fork.c:875 (kgdb) Would you like me to examine the contents of the stack frames for you? Kris --Dxnq1zWXvFF0Q93v Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (FreeBSD) iD8DBQE+fNvcWry0BWjoQKURApU6AKCVixd4xrW4WwIH2FO+hec9IITeUACg3amK ys9lzcFpFdhfpZ2H4QK8I4E= =jxu9 -----END PGP SIGNATURE----- --Dxnq1zWXvFF0Q93v-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sat Mar 22 18:17:49 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 89A8837B401; Sat, 22 Mar 2003 18:17:47 -0800 (PST) Received: from sasami.jurai.net (sasami.jurai.net [66.92.160.223]) by mx1.FreeBSD.org (Postfix) with ESMTP id D7F7F43FB1; Sat, 22 Mar 2003 18:17:46 -0800 (PST) (envelope-from mdodd@FreeBSD.ORG) Received: from sasami.jurai.net (sasami.jurai.net [66.92.160.223]) by sasami.jurai.net (8.12.8/8.12.8) with ESMTP id h2N2HjCi024216; Sat, 22 Mar 2003 21:17:46 -0500 (EST) (envelope-from mdodd@FreeBSD.ORG) Date: Sat, 22 Mar 2003 21:17:45 -0500 (EST) From: "Matthew N. Dodd" X-X-Sender: winter@sasami.jurai.net To: Yar Tikhiy Cc: ppc@FreeBSD.ORG, fs@FreeBSD.ORG Subject: Re: HFS/HFS Plus driver and tools for 5.x are available In-Reply-To: <20030322151656.GA34184@comp.chem.msu.su> Message-ID: <20030322211504.X8716@sasami.jurai.net> References: <20030322151656.GA34184@comp.chem.msu.su> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Sat, 22 Mar 2003, Yar Tikhiy wrote: > I'm glad to offer to the community an HFS and HFS Plus filesystem > module and tools, ported from Darwin to FreeBSD 5. Of course, it's > a rather early version that deserves thorough testing. > > Please see http://people.freebsd.org/~yar/hfs/ for details. > > I'm looking forward for your feedback. It appears to work. Is there any chance you can use separate out the mask into file and directory modes (like mount_smbfs(8))? Good job, and thanks btw! -- | Matthew N. Dodd | '78 Datsun 280Z | '75 Volvo 164E | FreeBSD/NetBSD | | winter@jurai.net | 2 x '84 Volvo 245DL | ix86,sparc,pmax | | http://www.jurai.net/~winter | For Great Justice! | ISO8802.5 4ever | To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sat Mar 22 23:45:16 2003 Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DB6E837B401; Sat, 22 Mar 2003 23:45:13 -0800 (PST) Received: from obsecurity.dyndns.org (adsl-63-207-60-150.dsl.lsan03.pacbell.net [63.207.60.150]) by mx1.FreeBSD.org (Postfix) with ESMTP id 333F943FA3; Sat, 22 Mar 2003 23:45:13 -0800 (PST) (envelope-from kris@obsecurity.org) Received: from rot13.obsecurity.org (rot13.obsecurity.org [10.0.0.5]) by obsecurity.dyndns.org (Postfix) with ESMTP id 021A066B9B; Sat, 22 Mar 2003 23:45:13 -0800 (PST) Received: by rot13.obsecurity.org (Postfix, from userid 1000) id D33BE1283; Sat, 22 Mar 2003 23:45:12 -0800 (PST) Date: Sat, 22 Mar 2003 23:45:12 -0800 From: Kris Kennaway To: fs@FreeBSD.org, iedowse@FreeBSD.org, peter@FreeBSD.org Subject: NFS panic on 4.8-pre Message-ID: <20030323074512.GA21092@rot13.obsecurity.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="YiEDa0DAkWCtVeE4" Content-Disposition: inline User-Agent: Mutt/1.4i Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org --YiEDa0DAkWCtVeE4 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline I just got the following on a package build system I set up earlier today. It's running FreeBSD 4.8-PRERELEASE #2: Thu Feb 27 19:18:58 GMT 2003, and was doing a lot of loopback NFS traffic (5-6 instances of 3 nfs mounted filesystems). Unfortunately I don't yet have a kernel.debug to get a proper traceback :( Kris IdlePTD at phsyical address 0x002f3000 initial pcb at physical address 0x00277500 panicstr: page fault panic messages: --- Fatal trap 12: page fault while in kernel mode fault virtual address = 0x0 fault code = supervisor write, page not present instruction pointer = 0x8:0xc02212e7 stack pointer = 0x10:0xfd1ddca4 frame pointer = 0x10:0xfd1ddcd0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 703 (find) interrupt mask = none trap number = 12 panic: page fault #0 0xc015b926 in dumpsys () #1 0xc015b6f7 in boot () #2 0xc015bb35 in panic () #3 0xc02227ff in trap_fatal () #4 0xc02224ad in trap_pfault () #5 0xc0222053 in trap () #6 0xc02212e7 in generic_bzero () #7 0xc01d74df in nfs_lookup () #8 0xc0187bf1 in lookup () #9 0xc01876ec in namei () #10 0xc018d3f1 in lstat () #11 0xc0222ab5 in syscall2 () #12 0xc0215b25 in Xint0x80_syscall () #13 0x480a0a01 in ?? () #14 0x480a02a2 in ?? () #15 0x804971a in ?? () #16 0x804b9a0 in ?? () #17 0x80493ce in ?? () (kgdb) I couldn't find a kernel.debug though :-( Can you build one? Kris ----- End forwarded message ----- --YiEDa0DAkWCtVeE4 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (FreeBSD) iD8DBQE+fWYIWry0BWjoQKURAqHNAJ9uVdvpA9ZTUT292Xzevta4kpVTDQCffDlZ pEd/L1Qt8ZZ2iO3B6fWAVxI= =x6Ek -----END PGP SIGNATURE----- --YiEDa0DAkWCtVeE4-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message