From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 6 10:56:01 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id DD53C8F for ; Wed, 6 Mar 2013 10:56:01 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [188.252.31.196]) by mx1.freebsd.org (Postfix) with ESMTP id 3A208163 for ; Wed, 6 Mar 2013 10:56:00 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5) with ESMTP id r26AtsYG002299; Wed, 6 Mar 2013 11:55:54 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5/Submit) with ESMTP id r26Atr36002296; Wed, 6 Mar 2013 11:55:54 +0100 (CET) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Wed, 6 Mar 2013 11:55:53 +0100 (CET) From: Wojciech Puchar To: Mark Saad Subject: Re: gjournal +UFS - anyone actually use it? In-Reply-To: <15BA8CB5-A10A-4FA9-AC5F-1BA2B7720E5E@longcount.org> Message-ID: References: <15BA8CB5-A10A-4FA9-AC5F-1BA2B7720E5E@longcount.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Wed, 06 Mar 2013 11:55:54 +0100 (CET) Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Mar 2013 10:56:01 -0000 > I was using it to store large MySQL myisam tables , speed was acceptable at the time . I never had any fs corruption and it worked as expected . > > At the time I set it up I remember there was some chatter about how slow gjournal was compared to ufs with softupdate . did some tests yesterday on 25GB partition. i simulated journal on SSD using 2GB malloc backed ramdisk (md0). UFS partition mounted async as gjournal recommends. test: unpacking kernel sources to multiple directories until disk gets full simulates write heavy I/O. gjournal+SATA drive (seagate constellation ES 500GB) with write cache disabled - 14 minutes 20 seconds. write cache enabled - 14 minutes 5 seconds (nearly no difference). UFS+journalled softupdates, no gjournal, disk write cache on - 26 minutes 44 seconds. disk write cache off - was too lazy to wait after an hour. With gjournal it is not only faster, but it doesn't make other I/O activity crawling. > > Fast forward to today I almost always use ufs with softupdate journal , new in FreeBSD 9.0 and available as a patch to 6.x, 7.x , and 8.x The problem is as follows: SU+J makes sure that metadata will get consistent. NOT DATA. And this is quite a mess if you get UPS failure under high load. gjournal does journal everything. > This is better supported now , as more people use it in new 9.x builds . > >> i think about journal on SSD. > > I believe this is only and option in geom journal , SSDs are not expensive today. i can get 128GB SSD and create 20GB journal just to limit wear. and possibly use the rest of SSD to store read-intensive data. the way gjournal writes to journal device (sequential) is very friendly for SSD too. Small SLC-based SSD would be best but i don't see anything on the market with acceptable price for now. > I am not sure if you can relocate a suj journal to an alternate disk. no you can't. But still you will end with consistent metadata but not consistent data. I recovered it but still it took a time and lots of checking. gjournal doesn't seem to be elegant in case of journal failure (i simulated it with forced removal of ramdisk with mdconfig). TONS of messages in logs, but still - no data loss, just you have to shutdown system, boot from pendrive, remove journal, fsck (just for sure), and then add journal again