From owner-freebsd-questions@FreeBSD.ORG Sat Jan 6 06:22:31 2007 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3C87116A40F for ; Sat, 6 Jan 2007 06:22:31 +0000 (UTC) (envelope-from ahnjoan@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.173]) by mx1.freebsd.org (Postfix) with ESMTP id CD74F13C441 for ; Sat, 6 Jan 2007 06:22:30 +0000 (UTC) (envelope-from ahnjoan@gmail.com) Received: by ug-out-1314.google.com with SMTP id o2so5238230uge for ; Fri, 05 Jan 2007 22:22:29 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=WAAET3V6yBcVoT2quTgcUT4HbSmJRkgUAJfWfu2mFBnv5VSKd0Q+sh2PnDvEYlRHmqlEjs7Gc6FnsuSxGOafhVNbK6HsFdOWCGHphXqh20ZU+ALyGIfPoY3RIlLdXkGvTfiVB0ARknN6n1psG3trq1mEf1YY92r162rtiqdpGLo= Received: by 10.78.160.2 with SMTP id i2mr7060815hue.1168062922416; Fri, 05 Jan 2007 21:55:22 -0800 (PST) Received: by 10.78.11.6 with HTTP; Fri, 5 Jan 2007 21:55:22 -0800 (PST) Message-ID: <5e575c8a0701052155i7ee630e6k6a35d3b982534620@mail.gmail.com> Date: Sat, 6 Jan 2007 00:55:22 -0500 From: "Ahnjoan Amous" To: freebsd-questions@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=WINDOWS-1252; format=flowed Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Subject: disappointing rebuild speed using ggatec and ggated X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Jan 2007 06:22:31 -0000 I'm attempting a RAID 1 mirror with ggate[c,d] between two hosts using 6.1-release and am disappointed by the rebuild rate. I have tried using Ethernet cards supported by fxp and re drivers, have read advice on how to tune to improve network performance. I have also attempted ggate[c,d] configuration using an md rather than ad device but can't seem to figure out what might cause the problem being encountered. Searching the lists and web has lead make the following modifications. *** These are quadruple what the defaults were on my system*** sysctl net.inet.tcp.sendspace=3D131072 sysctl net.inet.tcp.recvspace=3D262144 sysctl kern.ipc.maxsockbuf=3D1048576 sysctl kern.ipc.nmbclusters=3D35584 *** Kernel config *** options HZ=3D8000 options DEVICE_POLLING *** ifconfig*** enable POLLING on each machine with re0 With the following modifications, my rebuild has improved. Instead of stalling out at 2% (of a 146G mirror) the rebuild seems to steadily continue albeit very slowly. Today an entire manual rebuild (non auto sync for gm device) took over 17 hours over a crossover cable between two "re" interfaces. I don't expect that the rebuild would happen at wire speed but guess it might be possible to finish in less than 4 hours. I do not have the gm device or ggate device mounted on the primary host during the rebuild. I am not connecting to either the primary or failover host for any services. ( I hope this communicates that both hosts are not taxed by external sources ) Using "systat =96vm 1" shows that the machine comes close to 0% idle while the mirror is actually rebuilding. Using "gstat" shows that the ggate device on the primary comes close to 100% busy while the mirror is actually rebuilding. For some reason the rebuild seems to stop (systat =96vm 1 goes to 80% idle, and gstat goes to 0% busy for ggate) every once in a while. I can't pin point why it stops rebuilding and hope that I have just missed someone commenting that I should do something obvious to fix this issue. Thanks Ahnjoan