From owner-freebsd-performance@FreeBSD.ORG Wed Sep 23 16:30:30 2009 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 11452106568B for ; Wed, 23 Sep 2009 16:30:30 +0000 (UTC) (envelope-from oliver@FreeBSD.org) Received: from avocado.salatschuessel.net (avocado.salatschuessel.net [78.111.72.186]) by mx1.freebsd.org (Postfix) with SMTP id 638138FC21 for ; Wed, 23 Sep 2009 16:30:29 +0000 (UTC) Received: (qmail 16701 invoked by uid 89); 23 Sep 2009 16:03:46 -0000 Received: from unknown (HELO kartoffel.salatschuessel.net) (78.111.72.187) by avocado.salatschuessel.net with SMTP; 23 Sep 2009 16:03:46 -0000 Date: Wed, 23 Sep 2009 18:03:47 +0200 From: Oliver Lehmann To: freebsd-performance@freebsd.org Message-Id: <20090923180347.ee0ae193.oliver@FreeBSD.org> X-Mailer: Sylpheed 2.7.1 (GTK+ 2.16.6; amd64-portbld-freebsd7.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: gjournal, hardware raid, software raid X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2009 16:30:30 -0000 Hi, I'll have soon the following System configuration: two 250GB harddisks (WD2500KS) - configured as RAID1 using gmirror - disks are attached to a Promise SATA300 TX2plus - are supposed to hold the whole base system+swap four 1TB harddisks (WD10EADS) - configured as RAID5 using a 3ware 3500-4LP with a battery backup unit having write cache enabled. - will hold one big (around 3TB) partition using GPT The system itself has 2GB of RAM and is an old dual PIII 850 running on an intel L440GX+ I now plan to setup gjournal for the 3TB partition to slow down the fsck run times. The system crashes from time to time so this is vital for me ;) I thought about what would be the best configuration here and I saw 3 solutions: 1) have one partition on the 3TB RAID sharing data and journal 2) have two partitions, one holding the journal, one holding the data. Both on the 3TB RAID. Preferable the journal at the "beginning" of the disks. 3) same partitions like above but the journal partition will be on the gmirror RAID1 holding the base system. I like the 3rd option because on a heavy I/O load on the RAID5 the 3ware must not switch between data and journal and can stick to handling the data only. The Promise controller will then stick to handle the data. Looks like some I/O balancing for me... I maybe want gjournal for the /usr partition of the RAID1 as well. So I would then have two 6GB (2GB*3) journaling partitions on the RAID1 if I go for option 3. What do you guys think. Please keep me CCed - I'm not subscribed. -- Oliver Lehmann http://www.pofo.de/ http://wishlist.ans-netz.de/ From owner-freebsd-performance@FreeBSD.ORG Wed Sep 23 17:34:32 2009 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AAE8A1065672; Wed, 23 Sep 2009 17:34:32 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-yx0-f171.google.com (mail-yx0-f171.google.com [209.85.210.171]) by mx1.freebsd.org (Postfix) with ESMTP id 514A18FC08; Wed, 23 Sep 2009 17:34:32 +0000 (UTC) Received: by yxe1 with SMTP id 1so1121213yxe.3 for ; Wed, 23 Sep 2009 10:34:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=paSRMj5M1ea2r4YsiDGahahO5LAAJ1mc9ub79/sg/zM=; b=VhjFkkA+XRCQi3ehK/faD7IoRAFoDttGNz4rU8UdF3KgGpi589M5VQlj4BU8H7EviW TxLX+QmJGcLfcmqmjhI8BLfSuGryRLyuQJqb0HgHLICpmZEG0GiqoVKVMPEK5Fu9m+8X uYdj/fW50Qucr+XTCcWGJqfvXooto2sAFkA4k= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=QMXV5DoigfwzZptoPp0SJoTPYpcGYfRfBPUIgx+iNtacH0WRdV4TU92LyvChr5mw8r pYGT6Rbf/ILwIYCsvGw1o/JgdnPartjlJVjR9OLq6dIHT8KP3e5ODhxUe4IpxsW/+2kE GzrHzOHdZk5Za+GNip2QSvTjNicmAY9x7LGWU= MIME-Version: 1.0 Received: by 10.150.234.5 with SMTP id g5mr1986026ybh.339.1253725645986; Wed, 23 Sep 2009 10:07:25 -0700 (PDT) In-Reply-To: <20090923180347.ee0ae193.oliver@FreeBSD.org> References: <20090923180347.ee0ae193.oliver@FreeBSD.org> Date: Wed, 23 Sep 2009 10:07:25 -0700 Message-ID: From: Freddie Cash To: Oliver Lehmann Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-performance@freebsd.org Subject: Re: gjournal, hardware raid, software raid X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2009 17:34:32 -0000 On Wed, Sep 23, 2009 at 9:03 AM, Oliver Lehmann wrote: > I'll have soon the following System configuration: > > two 250GB harddisks (WD2500KS) > - configured as RAID1 using gmirror > - disks are attached to a Promise SATA300 TX2plus > - are supposed to hold the whole base system+swap > > four 1TB harddisks (WD10EADS) > - configured as RAID5 using a 3ware 3500-4LP with a > battery backup unit having write cache enabled. > - will hold one big (around 3TB) partition using GPT > > The system itself has 2GB of RAM and is an old dual PIII 850 running on > an intel L440GX+ > > I now plan to setup gjournal for the 3TB partition to slow down the fsck > run times. The system crashes from time to time so this is vital for me ;) > > I thought about what would be the best configuration here and I saw 3 > solutions: > > 1) have one partition on the 3TB RAID sharing data and journal > 2) have two partitions, one holding the journal, one holding the data. > Both on the 3TB RAID. Preferable the journal at the "beginning" of the > disks. > 3) same partitions like above but the journal partition will be on the > gmirror RAID1 holding the base system. > > I like the 3rd option because on a heavy I/O load on the RAID5 the 3ware > must not switch between data and journal and can stick to handling the > data only. The Promise controller will then stick to handle the data. > Looks like some I/O balancing for me... > > I maybe want gjournal for the /usr partition of the RAID1 as well. So I > would then have two 6GB (2GB*3) journaling partitions on the RAID1 if I > go for option 3. > > I've never done anything with gjournal, so can't really comment on what would be the better setup for it. However, I would like to point out that there's a fourth option: - configure the RAID controller to use 4 Single Disk arrays (or JBOD, not sure when 3Ware added Single Disk support) - use ZFS to create a raidz dataset using the 4 drives You now have a 3 TB pool of storage, without having to fight with GPT, without worrying about the "raid5 write hole", without worrying about fsck at boot, and without trying to figure out how to do journalling. You'll still get the benefits of the cache and BBU on the RAID controller, along with all the alerts from 3dm2, but it will be (in essence) acting as an IDE controller instead of a RAID controller. ZFS will handle the RAID, along with a lot of other niceties that come along with it (journalling/copy-on-write, snapshots, compression, end-to-end data integrity, etc). Of course, since this is a 32-bit system with only 2 GB of RAM, you'd need to do some tuning (via /boot/loader.conf) to make it work well. But it's a viable option, IMO/IME (I do something similar using a P4 system with 2 GB of RAM, at home). -- Freddie Cash fjwcash@gmail.com From owner-freebsd-performance@FreeBSD.ORG Wed Sep 23 17:45:28 2009 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7706E10656B1 for ; Wed, 23 Sep 2009 17:45:28 +0000 (UTC) (envelope-from oliver@FreeBSD.org) Received: from avocado.salatschuessel.net (avocado.salatschuessel.net [78.111.72.186]) by mx1.freebsd.org (Postfix) with SMTP id BF94D8FC1E for ; Wed, 23 Sep 2009 17:45:27 +0000 (UTC) Received: (qmail 18135 invoked by uid 89); 23 Sep 2009 17:45:26 -0000 Received: from unknown (HELO kartoffel.salatschuessel.net) (78.111.72.187) by avocado.salatschuessel.net with SMTP; 23 Sep 2009 17:45:26 -0000 Date: Wed, 23 Sep 2009 19:45:27 +0200 From: Oliver Lehmann To: Freddie Cash Message-Id: <20090923194527.0b3cfbb2.oliver@FreeBSD.org> In-Reply-To: References: <20090923180347.ee0ae193.oliver@FreeBSD.org> X-Mailer: Sylpheed 2.7.1 (GTK+ 2.16.6; amd64-portbld-freebsd7.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-performance@freebsd.org Subject: Re: gjournal, hardware raid, software raid X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2009 17:45:28 -0000 Freddie Cash wrote: > I've never done anything with gjournal, so can't really comment on what > would be the better setup for it. However, I would like to point out that > there's a fourth option: > - configure the RAID controller to use 4 Single Disk arrays (or JBOD, not > sure when 3Ware added Single Disk support) > - use ZFS to create a raidz dataset using the 4 drives Uh... Ok, this might be an option generally but honestly not for me ;) Considering the money I've spent 3 years ago for buying the RAID controller and considering the load it takes from the system handling the RAID I think it is in the best interest for the system to keep the RAID-5 3ware internally. As you might have seen the system is not the fastest one with not that much of memory - so whatever load I can delegate I'm try to delegate off the CPUs. Today I would probably not spend the money buying a RAID controller again but use indeed ZFS instead. But back the time there was only (g)vinum - and I never liked it. -- Oliver Lehmann http://www.pofo.de/ http://wishlist.ans-netz.de/