From owner-freebsd-fs@FreeBSD.ORG Wed Nov 10 17:05:37 2010 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 484B31065673; Wed, 10 Nov 2010 17:05:37 +0000 (UTC) (envelope-from sarawgi.aditya@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 0E96D8FC12; Wed, 10 Nov 2010 17:05:36 +0000 (UTC) Received: by pwi10 with SMTP id 10so193682pwi.13 for ; Wed, 10 Nov 2010 09:05:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:date:from:to:cc:subject :message-id:references:mime-version:content-type:content-disposition :in-reply-to:user-agent; bh=doHDc3Be3vx7yab1G/OcJncfuNovjKpeU+Flb4/uKlM=; b=JnNtUfxvt0j3a8aifyoTbRW3guaws2mGb44PW6o32bNFEsBOScpubiKvwITxnbWrxy ONhx7ae0jtDDi6GPhxbas19cd7o3eEcI4Y0foz3mMXN27s6/+GbN/nDDYVmBwXPLf5Jb hR1biIFMH6HD6InArH1lxLrwIMsnARxy9DcYQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=q/nwmuknIjTdD9DtVV6kJfIyKWKMI1pOAj3VTvQ1UCh3jPZJXBgI1GjTGFI+M1/ua9 rW17iWTlgQ5XS2IhEYt3qMTNSDRi6YziD/AW8kr9foDAI2D640MJNSmJ926Dby0ihS6I lFnAVAmbus+Mb6zypcYpvKQNg+2mXmtGSZs4s= Received: by 10.42.177.73 with SMTP id bh9mr1657654icb.162.1289408734429; Wed, 10 Nov 2010 09:05:34 -0800 (PST) Received: from earth ([183.87.49.208]) by mx.google.com with ESMTPS id gy41sm1082604ibb.23.2010.11.10.09.05.32 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 10 Nov 2010 09:05:33 -0800 (PST) Date: Wed, 10 Nov 2010 22:37:22 +0530 From: Aditya Sarawgi To: Doug Barton Message-ID: <20101110170719.GA1573@earth> References: <20100929031825.L683@besplex.bde.org> <20100929084801.M948@besplex.bde.org> <20100929041650.GA1553@aditya> <201009290917.05269.jhb@freebsd.org> <20100929202526.GA1564@aditya> <4CD0A3E8.4080304@FreeBSD.org> <4CD201AE.3040409@FreeBSD.org> <20101108174327.GC2066@earth> <4CD9E535.8000801@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4CD9E535.8000801@FreeBSD.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: fs@freebsd.org Subject: Re: ext2fs now extremely slow X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Nov 2010 17:05:37 -0000 On Tue, Nov 09, 2010 at 04:20:05PM -0800, Doug Barton wrote: > On 11/08/2010 09:43, Aditya Sarawgi wrote: > > On Wed, Nov 03, 2010 at 05:43:26PM -0700, Doug Barton wrote: > > >> Regarding stability, sometimes (but not always) when I'm doing the above > >> listed disk-intensive things on an otherwise idle system I've had the > >> system lock up. Not panic, not reboot, just wedge. I'm running X when > >> this happens, so I'm not 100% sure that the disk activity is the > >> culprit, but it seems very suspicious. Yesterday was a very bad day, I > >> had to do 3 tries to get all the way through a buildworld/kernel, mostly > >> because the last 2 crashes resulted in my /usr/src (which is actually > >> /home/svn/head) and /usr/obj (/home/obj-9) directories getting corrupted > >> respectively. Today (running r214694) has actually been quite good, > >> although I haven't tried a buildworld yet. > >> > > > > I am not sure if this is the right use case for ext2fs > > Can you expand on that? What about it do you see as problematic? > ext2fs is not a native filesystem. It is relatively slower than UFS may have some other problems like deadlocks which are not yet discovered. May make the data inconsistent due to lack of facilities like journaling. It is only meant to make your data in linux partitions accessible. > >>> You can test Zheng's preallocation patch for ext2fs, there is a > >>> serious lack of testers for that. > >> > >> I would be happy to do that, but my reading of this thread last month > >> didn't produce a clear "try this version of the patch" neon sign. > >> Various people referred to suggestions, updates, etc. If someone could > >> provide a URL for the right patch to try, as well as a suggestion for > >> benchmarking methodology, I'll be glad to do so. > >> > > > > I have attached the patch. > > Thanks for that. I'm curious though whether this is the latest version > of the patch with the suggested improvements from earlier in this thread? > There will be only style and some comment fixes in the new patch. > > Some primitive testing like copying files, > > untaring etc and comparing with the existing ext2fs will do. If you > > are looking to do a full fledged benchmarking then I would suggest > > iozone, blogbench, dbench etc. > > Sorry, I am not a filesystem person, so if you want me to do any real > benchmarking you're going to have to give me details ... Install this > program, run this test, etc. > Install dbench, blogbench from ports or packages and follow this http://wiki.freebsd.org/SOC2010ZhengLiu > Meanwhile I finally got around to setting up my 8.1-RELEASE partition on > this same system and particularly with cvsup it's very noticeable that > ext2fs in -current is MUCH slower than in RELENG_8. I'll do some before > and after tests on -current, then I'll do the same thing on 8.1 and see > how the numbers compare. > Yes, we know there are some scope of improvements. So the BSDL ext2fs lacks preallocation which used to preallocate some blocks which improved the sequential write performance. This problem is solved by Zheng's reservation window work. The other issue like Bruce mentioned is that some of the blocks in between are skipped by the block allocator algorithm. We intend to have fixes for both of these before 9 is released. Can you also mail me privately the output of dumpe2fs Thanks Aditya Sarawgi