From owner-freebsd-geom@FreeBSD.ORG Fri Apr 24 21:23:53 2009 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A7733106564A for ; Fri, 24 Apr 2009 21:23:53 +0000 (UTC) (envelope-from erleya@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.157]) by mx1.freebsd.org (Postfix) with ESMTP id 2EDD48FC13 for ; Fri, 24 Apr 2009 21:23:52 +0000 (UTC) (envelope-from erleya@gmail.com) Received: by fg-out-1718.google.com with SMTP id l26so221500fgb.12 for ; Fri, 24 Apr 2009 14:23:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:from:to :content-type:content-transfer-encoding:mime-version:subject:date :x-mailer; bh=bZj9shl+PPIU+pyfVaASglUEXnxVvK0IoV7FxfLpJ2k=; b=dzUmS/rokv8sZRUB3hMIjBtNhl98+9h95LcJIMzKBY74Z5TzviLME6C8/3QkA1Sjs5 G3XVDAZ6sPjzIsE3oUA4JMZx33wvxTs8cJ0YIt9fNa66vIGAHYiMCjoK4WOfqsmk6zcg Jwhr3Z4sD1xCVqPYxZ5wlmpxafIna1Jbqu+Us= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:from:to:content-type:content-transfer-encoding :mime-version:subject:date:x-mailer; b=fO+qI59FrFpAxwoMad8KaCd9t9c7CvEDm5AyalQkqxLLa36tIKHZ/Xr9CiyZwqPDlr Inm7lGeCbgsTTnNQSfFEzsu82E9hOqn+FfvQ/w9ubBa9UwnR0sXlrfV87Rp5XVv0m1Io av2VNN0cddKsJLxb1DiMXRO81nnq8FWxzXdBc= Received: by 10.86.70.20 with SMTP id s20mr1170712fga.1.1240606437104; Fri, 24 Apr 2009 13:53:57 -0700 (PDT) Received: from ?192.168.0.2? (lab75-4-82-245-153-214.fbx.proxad.net [82.245.153.214]) by mx.google.com with ESMTPS id d4sm1885377fga.9.2009.04.24.13.53.55 (version=TLSv1/SSLv3 cipher=RC4-MD5); Fri, 24 Apr 2009 13:53:56 -0700 (PDT) Message-Id: From: Alexei Troussov To: freebsd-geom@freebsd.org Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Fri, 24 Apr 2009 22:53:53 +0200 X-Mailer: Apple Mail (2.930.3) Subject: gjournal+gstripe+gmirror question X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Apr 2009 21:23:53 -0000 Hello All, I decided to build multi-terabyte data storage (file server) using 4 x 1Tb disks. No special raid controller is used, FreeBSD 7.1 is running on another disk, so the disk array is going to be used for file storage only. So, I tried several combinations of gmirror/gstripe and put gjournal on it: 1) two mirrors and stripe on them 2) two stripes and mirror on them Both configurations gave excellent I/O speed (around 100Mb/sec), and I chose the first one as it needs less time to resync disks when one of the mirrors fails. To make picture more complete, I didn't try graid3 because it seems to load CPU more than configs above, which is not good for my barebone :) Now when my top-level gstripe gets "gjournal"-ed, I see relatively poor write speed around 20-30 Mb/sec. As far as I understand from man gjournal and some posts here, performance usually drops about 40-50%, but what I have is much worse... Are there a better way to configure/tune the whole thing? Thanks in advance! PS Here is my current playground config: -------- > gmirror list Geom name: gm0 State: COMPLETE Components: 1 Balance: round-robin Slice: 4096 Flags: NONE GenID: 0 SyncID: 1 ID: 2111539126 Providers: 1. Name: mirror/gm0 Mediasize: 1000204885504 (932G) Sectorsize: 512 Mode: r1w1e2 Consumers: 1. Name: ad10 Mediasize: 1000204886016 (932G) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Priority: 0 Flags: NONE GenID: 0 SyncID: 1 ID: 3300348150 2. Name: ad12 Mediasize: 1000204886016 (932G) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Priority: 0 Flags: NONE GenID: 0 SyncID: 1 ID: 4291633068 Geom name: gm1 State: COMPLETE Components: 2 Balance: round-robin Slice: 4096 Flags: NONE GenID: 0 SyncID: 1 ID: 835525163 Providers: 1. Name: mirror/gm1 Mediasize: 1000204885504 (932G) Sectorsize: 512 Mode: r1w1e2 Consumers: 1. Name: ad11 Mediasize: 1000204886016 (932G) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Priority: 0 Flags: NONE GenID: 0 SyncID: 1 ID: 1770536936 2. Name: ad13 Mediasize: 1000204886016 (932G) Sectorsize: 512 Mode: r1w1e1 State: ACTIVE Priority: 0 Flags: NONE GenID: 0 SyncID: 1 ID: 922042393 > gstripe list Geom name: gs0 State: UP Status: Total=2, Online=2 Type: AUTOMATIC Stripesize: 131072 ID: 1106325435 Providers: 1. Name: stripe/gs0 Mediasize: 2000409591808 (1.8T) Sectorsize: 512 Mode: r1w1e1 Consumers: 1. Name: mirror/gm0 Mediasize: 1000204885504 (932G) Sectorsize: 512 Mode: r1w1e2 Number: 0 2. Name: mirror/gm1 Mediasize: 1000204885504 (932G) Sectorsize: 512 Mode: r1w1e2 Number: 1 -------- Have a nice day, Alex