From owner-freebsd-stable@FreeBSD.ORG Sun Apr 20 02:12:26 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3BF5A1065670 for ; Sun, 20 Apr 2008 02:12:26 +0000 (UTC) (envelope-from matheusber@gmail.com) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.179]) by mx1.freebsd.org (Postfix) with ESMTP id 082D78FC18 for ; Sun, 20 Apr 2008 02:12:25 +0000 (UTC) (envelope-from matheusber@gmail.com) Received: by wa-out-1112.google.com with SMTP id k17so1833414waf.3 for ; Sat, 19 Apr 2008 19:12:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=f3BFJrNxD8PLqKDbDP+dHvDtbToTIhxD7U2O7Aj+JLQ=; b=SC8oewGp7KscPnPnQG5/a81X/sFsRKwzuaYHIPGu7EnQbV2nEYXsHxDl5Xeq/kE/TwKFN6d7Jn/Ojx60TXxveuz6Vy4cVB1Go69Slyi3p0JKj5yQh2B2IXmRl5ZoLIfLFreiFuHy630lPlRWFik1/VfcxmL0sQlydLaeX0OenBU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=BuzIYLbEA/SVfLWB2f68lvSxk5Z+P66MAVUmqjeRAg4QFY/WT2vzmhaYjaf1clrq7rtjZy/j6LiWVawDKBl/ucuRN2vWsBlshDv4hggMtmv6iQ2unOpl7T7EKmgbtVp/UFYmTwx63ITv8fZY7iYMjLAAfc6VBJjhtFajp9PE6DM= Received: by 10.114.191.1 with SMTP id o1mr4310349waf.205.1208657545219; Sat, 19 Apr 2008 19:12:25 -0700 (PDT) Received: by 10.114.241.9 with HTTP; Sat, 19 Apr 2008 19:12:25 -0700 (PDT) Message-ID: <4956a5e50804191912g52833c35q868827dc2b54e5ae@mail.gmail.com> Date: Sat, 19 Apr 2008 23:12:25 -0300 From: Nenhum_de_Nos To: "Arnaud Houdelette" In-Reply-To: <47FCBAFB.9060508@tzim.net> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: <396418019.20080409104542@serebryakov.spb.ru> <47FCBAFB.9060508@tzim.net> Cc: lev@freebsd.org, freebsd-stable@freebsd.org Subject: Re: [SO]HO Software RAID5 server: which implementation should I choice? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Apr 2008 02:12:26 -0000 On Wed, Apr 9, 2008 at 9:47 AM, Arnaud Houdelette wrote: > Lev Serebryakov a =E9crit : > > > > > Hello, freebsd-stable. > > > > Does somebody use some software RAID5 on FreeBSD in real production > > system? > > > > I want to build storage server for my home: RAW photos, multi-layer > > PhotoShop files and FLAC-encoded music consume a lot of space, and > > they should be availible both from desktop & notebook. > > > > Also, all photo-content is unique, so I need some insuranse from > > single HDD crash. I understand, that I will not safe from fire, PSU > > failure and thing slike this. > > > > I selected hardware platform: Intel Q35-based MoBo with 6xSATA-II port= s > > (all of them is chipset-based, so no SiliconImage/JMicron/Whatever > > crappy controllers), some low-end Core2Duo, 2Gb of memory. > > Storage will be 5x500Gb WD HDDs for RAID + one small HDD for boot, > system, > > swap, etc. I want to have 2Tb (ok, not real Tb, I know) of "protected" > > storage. > > I want to have maximum speed via 1Gb network, because graphic files > > are big and should open fast. Not as fast as local ones, I understand > > that, but speeds about 12-15Mb/s is not enough for sure :) > > > > Only problem I see: which software RAID5 solution should I prefer? > > FreeBSD-based, of course! > > > > I see these variants: > > > > (1) FreeBSD 6(7?) + graid3. Slow, one disk for checksums is bottleneck= , > > as far as I understand. > > > > (2) FreeBSD 6(7?) + gvinum/radi5. Is it stable enough?! Is it complete= ? > > when I try it about 6 months ago in VMWare installation with 5 > > virtual disks, I got panics and strange behaviour after "crashing" > > one of virtual disks. > > > > (3) FreeBSD 6(7?) + graid5. Again, is it stable enough? There are > > THREE versions of it. Which one should I prefer? There was long > > thread about it some times ago without any clear conclusion. Does > > something changed? > > > > (4) FreeBSD 7 + ZFS "zraid". And again: stability. Too many messages > > about locks, crashes, etc. Code is experemental. Is it only for > > 32 bit systems? > > > > (5) Do I miss something? > > > > (6) Solaris + ZFS? I don't want it, I know a little about Solaris > > administaration, and I already have FreeBSD servers and routers. > > > > I know, that 3ware or Areca controllers are very good. I know, that > > "gmirror" is very stable. But these variants are too expensive for > > home server :( > > > > Does somebody use some software RAID5 on FreeBSD in real production? > > Any advices? > > > > > > Hi ! > > I personally use the 3 option for my personal Home File Server. I got > approximatly the same usage for the file server (mostly video, music, > photo). I built my own about 12 month ago. > I reviewed the about the same variants as the one you propose : > (1) Discarded for performance issues. Raid3 is slow. Really. > (2) raid5/vinum is also slow. And as I understood at that time, recover= y > from lost hard drive wasn't easy enough for the freebsd niubee I was the= n. > (4) ZFS wasn't there yet. But I did test it on a test VMWare, and wasn't > convinced (mostly stability and memory issues). > > So I use geom_raid5. I sticked to the main distributions, which seemed m= ore > stable at the moment. The kernel module is fairly simple to build/install= . > Performance is (very) good for a software raid. > I successfully switched the raid array from an i386 6.2-RELEASE to an an > amd64 7.0-RELEASE (with motherboard and CPU change) without any assle. > > For the moment, I use one big UFS+SU (and snapshots) on the whole array.= I > successfuly tried unpplugging then replugging on of the drives, suddent > power loss, using the array with a missing disk (degraded mode). All did > work fine. (still, I use an UPS on the file server). > > The sole issue I had is with ataidle. I had to patch ata-disc.c to incre= ase > the IO timeout. Without, the raid5 module detected temporary disk loss an= d > constantly launched rebuilds of the array. > > With 7.0, I wondered if I should use gjournal, but I'm not sure if it's > really the way to go on a file system dedicated to store many big files. = So > I stick to soft updates. > > Current configuration is : > / on a 2GB usb key > /tmp on memory > ports and source trees (and some portsnap stuff) on a small disk > 4x250 GB sata for the raid5 array. > AMD A64 3200+ and 512 GB DDRII > Realtek Gigabit nics. > > Copy from raid5 to /dev/null gives about 100MB/s > Copy from /dev/random to raid5 about 40MB/s > > I use samba shares. I get about 40MB/s in both ways from another compute= r > on the network (enabling jumbo-frames gives a big boost). > > Hope my own story can help you in any way. > > Regards, > > Arnaud Houdelette I know its been quite some time from the mail, but if you could say where to find this module. eikipedia says its on freebsd 7 but there is not this module for me (/boot/kernel/ there is no raid5 file). everytime I search the internet I find old stuff about it. If you could point me the site/article/anything :) thanks, matheus --=20 We will call you cygnus, The God of balance you shall be