From owner-freebsd-fs Wed Dec 26 11:15:17 2001 Delivered-To: freebsd-fs@freebsd.org Received: from hawk.prod.itd.earthlink.net (hawk.mail.pas.earthlink.net [207.217.120.22]) by hub.freebsd.org (Postfix) with ESMTP id 3FB7437B416; Wed, 26 Dec 2001 11:15:14 -0800 (PST) Received: from 1cust25.tnt2.ogden.ut.da.uu.net ([63.39.41.25] helo=mindspring.com) by hawk.prod.itd.earthlink.net with esmtp (Exim 3.33 #1) id 16JJW4-0001mn-00; Wed, 26 Dec 2001 11:15:04 -0800 Message-ID: <3C2A21B9.428C3800@mindspring.com> Date: Wed, 26 Dec 2001 11:15:05 -0800 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Alfred Perlstein Cc: Ian Dowse , mckusick@freebsd.org, fs@freebsd.org Subject: Re: fsck and predictive readahead? References: <20011222053306.Y48837@elvis.mu.org> <200112221353.aa41047@salmon.maths.tcd.ie> <20011222150814.Z48837@elvis.mu.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Alfred Perlstein wrote: > ./fsck_ffs -d -n /vol/spare 24.50s user 4.72s system 19% cpu 2:30.73 total > > No matter how you order the IO, fsck is going to have to wait for read(2) > to return. If we can offload that waiting to a child process we may > be able to fix this. > > Is there any detailed commenting on the sources available, they are > quite readable, but still very terse. A more in depth explanation > of each function would really help. Do you know of a paper, manpage > or do you have the time to sprinkle some commentary into the code? There's a paper on fsck in /usr/share, and it's also in the Usenix published books (I can share them with you, if you need me to). The easiest way to handle this would be with a windowed mmap(), with the other process being given either atonomy to read one byte per page, predictively, or the calculation ahead. Then the pages would be in core by the time the first process needed them, and then all we are talking about is mapping. This really looks like a job for rfork(), rather than another process, or true threads, so that the address space is shared, and there is not a remapping requirement (PTE thrashing would potentially slow you down with any other approach). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Wed Dec 26 11:20:38 2001 Delivered-To: freebsd-fs@freebsd.org Received: from elvis.mu.org (elvis.mu.org [216.33.66.196]) by hub.freebsd.org (Postfix) with ESMTP id D803537B417; Wed, 26 Dec 2001 11:20:34 -0800 (PST) Received: by elvis.mu.org (Postfix, from userid 1192) id 8750981D01; Wed, 26 Dec 2001 13:20:34 -0600 (CST) Date: Wed, 26 Dec 2001 13:20:34 -0600 From: Alfred Perlstein To: Terry Lambert Cc: Ian Dowse , mckusick@freebsd.org, fs@freebsd.org Subject: Re: fsck and predictive readahead? Message-ID: <20011226132034.N91594@elvis.mu.org> References: <20011222053306.Y48837@elvis.mu.org> <200112221353.aa41047@salmon.maths.tcd.ie> <20011222150814.Z48837@elvis.mu.org> <3C2A21B9.428C3800@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <3C2A21B9.428C3800@mindspring.com>; from tlambert2@mindspring.com on Wed, Dec 26, 2001 at 11:15:05AM -0800 Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org * Terry Lambert [011226 13:15] wrote: > Alfred Perlstein wrote: > > ./fsck_ffs -d -n /vol/spare 24.50s user 4.72s system 19% cpu 2:30.73 total > > > > No matter how you order the IO, fsck is going to have to wait for read(2) > > to return. If we can offload that waiting to a child process we may > > be able to fix this. > > > > Is there any detailed commenting on the sources available, they are > > quite readable, but still very terse. A more in depth explanation > > of each function would really help. Do you know of a paper, manpage > > or do you have the time to sprinkle some commentary into the code? > > There's a paper on fsck in /usr/share, and it's also in the Usenix > published books (I can share them with you, if you need me to). Thanks, I'll have a look at the paper, I'll let you know if I need more from your references. > The easiest way to handle this would be with a windowed mmap(), > with the other process being given either atonomy to read one > byte per page, predictively, or the calculation ahead. Then > the pages would be in core by the time the first process needed > them, and then all we are talking about is mapping. > > This really looks like a job for rfork(), rather than another > process, or true threads, so that the address space is shared, > and there is not a remapping requirement (PTE thrashing would > potentially slow you down with any other approach). Agreed, first I'd like to see how it performs if I can get the IO done. Perhaps aio, but that's kinda icky. :) -- -Alfred Perlstein [alfred@freebsd.org] 'Instead of asking why a piece of software is using "1970s technology," start asking why software is ignoring 30 years of accumulated wisdom.' Tax deductable donations for FreeBSD: http://www.freebsdfoundation.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Wed Dec 26 19:12:56 2001 Delivered-To: freebsd-fs@freebsd.org Received: from holly.dyndns.org (adsl-208-191-149-232.dsl.hstntx.swbell.net [208.191.149.232]) by hub.freebsd.org (Postfix) with ESMTP id D5D6037B416 for ; Wed, 26 Dec 2001 19:12:53 -0800 (PST) Received: (from chris@localhost) by holly.dyndns.org (8.11.6/8.9.3) id fBR3E0X10691 for freebsd-fs@FreeBSD.org; Wed, 26 Dec 2001 21:14:01 -0600 (CST) (envelope-from chris) Date: Wed, 26 Dec 2001 21:13:59 -0600 From: Chris Costello To: freebsd-fs@FreeBSD.org Subject: [chris@FreeBSD.org: cvs commit: src/share/man/man4 Makefile ffs.4 ufs.4] Message-ID: <20011226211359.B961@holly.calldei.com> Reply-To: chris@FreeBSD.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org As part of my work, some existing documentation for UFS and FFS were necessary, and I'm interested in review on the accuracy, layout, and general usefulness of the content, as well as anything else you have to say about it. Please Cc any reviews/corrections to me directly as well, so that I may get to them ASAP. Thanks! ----- Forwarded message from Chris Costello ----- From: Chris Costello Date: Wed, 26 Dec 2001 19:05:44 -0800 (PST) To: cvs-committers@FreeBSD.org, cvs-all@FreeBSD.org Subject: cvs commit: src/share/man/man4 Makefile ffs.4 ufs.4 chris 2001/12/26 19:05:44 PST Modified files: share/man/man4 Makefile Added files: share/man/man4 ffs.4 ufs.4 Log: Document UFS and FFS options and features. Obtained from: TrustedBSD Project Sponsored by: DARPA, NAI Labs Revision Changes Path 1.144 +2 -0 src/share/man/man4/Makefile 1.1 +114 -0 src/share/man/man4/ffs.4 (new) 1.1 +158 -0 src/share/man/man4/ufs.4 (new) ----- End forwarded message ----- -- +-------------------+---------------------------------------+ | Chris Costello | Some programming languages manage to | | chris@FreeBSD.org | absorb change but withstand progress. | +-------------------+---------------------------------------+ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Wed Dec 26 20:31:43 2001 Delivered-To: freebsd-fs@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 37C8537B419; Wed, 26 Dec 2001 20:31:40 -0800 (PST) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.6/8.11.5) with SMTP id fBR4VVD53292; Wed, 26 Dec 2001 23:31:32 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Wed, 26 Dec 2001 23:31:31 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: Alfred Perlstein Cc: Terry Lambert , Ian Dowse , mckusick@freebsd.org, fs@freebsd.org Subject: Re: fsck and predictive readahead? In-Reply-To: <20011226132034.N91594@elvis.mu.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Wed, 26 Dec 2001, Alfred Perlstein wrote: > > The easiest way to handle this would be with a windowed mmap(), > > with the other process being given either atonomy to read one > > byte per page, predictively, or the calculation ahead. Then > > the pages would be in core by the time the first process needed > > them, and then all we are talking about is mapping. > > > > This really looks like a job for rfork(), rather than another > > process, or true threads, so that the address space is shared, > > and there is not a remapping requirement (PTE thrashing would > > potentially slow you down with any other approach). > > Agreed, first I'd like to see how it performs if I can get the IO done. > Perhaps aio, but that's kinda icky. :) There were a number of pretty neat papers done on the topic of read-ahead by the folks at CMU's PDL (Parallel Data Labatory), including a paper on balancing resource allocation (in particular memory) between various sorts of tasks: straight read-ahead, caching, and possibly reliable prefetch data. They define API's to allow a process to report prospective up-coming read and write operations, which allows the operating system to manage resources better, as well as evaluate the correctness of the prefetch requests. If we're going to try at a more general solution here, investigating some of that work would probably make sense: since the application can't measure available cache/etc resources well, especially when running on a live system via background fsck, it will have to rely on the kernel to DTRT when it comes to not letting it get ahead of itself -- you could easily see fsck slowing itself down if it assumes the wrong things about how large that read-ahead window can get. If you want specific papers, I can dig up the references. Generally, it's work by Gibson, et al. Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Thu Dec 27 11: 6:35 2001 Delivered-To: freebsd-fs@freebsd.org Received: from temphost.dragondata.com (temphost.dragondata.com [63.167.131.128]) by hub.freebsd.org (Postfix) with ESMTP id B486E37B417 for ; Thu, 27 Dec 2001 11:06:32 -0800 (PST) Received: (from toasty@localhost) by temphost.dragondata.com (8.11.3/8.11.3) id fBRJD1K19224 for fs@freebsd.org; Thu, 27 Dec 2001 13:13:01 -0600 (CST) (envelope-from toasty) From: Kevin Day Message-Id: <200112271913.fBRJD1K19224@temphost.dragondata.com> Subject: signedness in fsck To: fs@freebsd.org Date: Thu, 27 Dec 2001 13:13:01 -0600 (CST) X-Mailer: ELM [version 2.5 PL5] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org (originally sent to hackers, was suggested I send here instead, sorry for the crosspost) I think I've found a bug in fsck. Some invalid values in cylinder groups don't get fixed. >From pass5.c: if (cg->cg_rotor < newcg->cg_ndblk) newcg->cg_rotor = cg->cg_rotor; else newcg->cg_rotor = 0; if (cg->cg_frotor < newcg->cg_ndblk) newcg->cg_frotor = cg->cg_frotor; else newcg->cg_frotor = 0; if (cg->cg_irotor < newcg->cg_niblk) newcg->cg_irotor = cg->cg_irotor; else newcg->cg_irotor = 0; All those values in the cg's are int32_t's. (signed). Somehow I ended up with 5 cyl groups that have "irotor" to some random negative value. fsck doesn't see a problem with them, but I get a panic when a create gets called: #5 0xc0207e6b in trap (frame={tf_fs = -1072234480, tf_es = -345702384, tf_ds = -1072234480, tf_edi = -750092288, tf_esi = -955801600, tf_ebp = -345663052, tf_isp = -345663072, tf_ebx = 177188131, tf_edx = -750091519, tf_ecx = 255, tf_eax = -927279650, tf_trapno = 12, tf_err = 0, tf_eip = -1071824083, tf_cs = 8, tf_eflags = 66183, tf_esp = -345663008, tf_ss = -1071971432}) at ../../i386/i386/trap.c:448 #6 0xc01d432d in skpc (mask0=255, size=177188131, cp0=0xc8bad5de
) at ../../libkern/skpc.c:50 #7 0xc01b0398 in ffs_nodealloccg (ip=0xc7cb6300, cg=6921, ipref=27906609, mode=33188) at ../../ufs/ffs/ffs_alloc.c:1347 #8 0xc01af567 in ffs_hashalloc (ip=0xc7cb6300, cg=6921, pref=27906609, size=33188, allocator=0xc01b0258 ) at ../../ufs/ffs/ffs_alloc.c:863 #9 0xc01aeffc in ffs_valloc (pvp=0xebe8d980, mode=33188, cred=0xc77fd584, vpp=0xeb659a5c) at ../../ufs/ffs/ffs_alloc.c:607 #10 0xc01c1b0f in ufs_makeinode (mode=33188, dvp=0xebe8d980, vpp=0xeb659c64, cnp=0xeb659c78) at ../../ufs/ufs/ufs_vnops.c:2097 #11 0xc01bf4c4 in ufs_create (ap=0xeb659bfc) at ../../ufs/ufs/ufs_vnops.c:194 Here's the cyl group: $2 = {cg_firstfield = 0, cg_magic = 590421, cg_time = 1009434417, cg_cgx = 9165, cg_ncyl = 16, cg_niblk = 4032, cg_ndblk = 32768, cg_cs = { cs_ndir = 60, cs_nbfree = 2740, cs_nifree = 1987, cs_nffree = 679}, cg_rotor = 1288, cg_frotor = 5112, cg_irotor = -2036988557, cg_frsum = {0, 80, 153, 45, 14, 5, 7, 5}, cg_btotoff = 168, cg_boff = 232, cg_iusedoff = 264, cg_freeoff = 768, cg_nextfreeoff = 5436, cg_clustersumoff = 4860, cg_clusteroff = 4924, cg_nclusterblks = 4096, cg_sparecon = {0 }, cg_space = "\016"} Notice cg_irotor is completely out of whack, but fsck thinks the FS is clean. I wrote a patch that made fsck detect this, but... why are these signed values anyway, other than historical reasons? -- Kevin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message From owner-freebsd-fs Sat Dec 29 0:37:17 2001 Delivered-To: freebsd-fs@freebsd.org Received: from stmpy-2.cais.net (stmpy-2.cais.net [205.252.14.72]) by hub.freebsd.org (Postfix) with ESMTP id CA1AF37B417 for ; Sat, 29 Dec 2001 00:37:06 -0800 (PST) Received: from merry-xmastoall ([63.222.160.34]) by stmpy-2.cais.net (8.11.1/8.11.1) with SMTP id fBT8aNi82861; Sat, 29 Dec 2001 03:36:26 -0500 (EST) Message-Id: <200112290836.fBT8aNi82861@stmpy-2.cais.net> From: loseweight20012000@yahoo.com To: Subject: ULTIMATE WEIGHT LOSS PRODUCT !!! Date: Sat, 29 Dec 2001 00:44:48 -0600 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_22C5_000053D7.00000DEF" X-Priority: 3 X-MSMail-Priority: Normal Reply-To: loseweight20012000@yahoo.com Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org ------=_NextPart_000_22C5_000053D7.00000DEF Content-Type: text/html; No Way?

No Way? Yes Whey!
Introducing Our Whey of Life to A 
$50,000 - $100,000 A Year Income!
Maximize Your Lean To Fat Ratio With the Real Food Lean Maximizer Weight Loss System
  • Cut Your Grocery Bill In Half 
  • Experience Weight Loss That Stays Off. 
  • Increase Fat Loss, Increase Lean Muscle, Increase Metabolism, Increase Endurance and Strength Gain (Documented Up To 430%). 
  • Reduces Food Craving, Promotes Normal Blood Sugar Levels and Helps Support Immune System and Increases Cell Hydration. 
  • Our Products Are Safe, Effective and 100% GUARANTEED! 

Learn How our products are effective 
  • Earn Up To 42% Fast Start Commissions 
  • Earn Residual Income From 5x5 Expandable Matrix With 100% Matching Bonuses On Your Personally Sponsored. 

If you'd like to learn more about our products Call 1-800-891-4074 
Are you interested in building extra residual income? Contact us NOW!

Email us your Name, Phone Number,& Best Time to Call. Someone will call you in 24hrs. Thanks

mailto:lose30lbs2002@yahoo.com-Subject=moreinfo

 


Please discontinue advertising to me..mailto:kdsfdfss@yahoo.com?subject=Remove
To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message