From owner-freebsd-stable@FreeBSD.ORG Fri Jan 18 19:18:05 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AFD2416A468 for ; Fri, 18 Jan 2008 19:18:05 +0000 (UTC) (envelope-from tom@tomjudge.com) Received: from smtp801.mail.ird.yahoo.com (smtp801.mail.ird.yahoo.com [217.146.188.61]) by mx1.freebsd.org (Postfix) with SMTP id ECEF413C4D3 for ; Fri, 18 Jan 2008 19:18:04 +0000 (UTC) (envelope-from tom@tomjudge.com) Received: (qmail 60533 invoked from network); 18 Jan 2008 19:18:03 -0000 Received: from unknown (HELO ?192.168.1.2?) (thomasjudge@btinternet.com@86.140.145.126 with plain) by smtp801.mail.ird.yahoo.com with SMTP; 18 Jan 2008 19:18:03 -0000 X-YMail-OSG: a04_GeAVM1k07b3tpM3z3a2IXrFhcSoTPNDbD9sN_EW1sWbN X-Yahoo-Newman-Property: ymail-3 Message-ID: <4790FB84.1040409@tomjudge.com> Date: Fri, 18 Jan 2008 19:18:28 +0000 From: Tom Judge User-Agent: Thunderbird 1.5.0.13 (X11/20070824) MIME-Version: 1.0 To: Ulrich Spoerlein References: <39FB5CF3-F2F4-401B-9D6D-7796608152E5@ish.com.au> <4FF9842D-ADC9-4A99-9DC4-E0FE1CC9CDCF@stromnet.se> <20080116222729.GB1529@roadrunner.spoerlein.net> <134BD86C-19CF-42A7-9190-FAD3BB564A06@stromnet.se> <7ad7ddd90801170030l790810b5r5bb156e3cda286b1@mail.gmail.com> <73ED6B7D-49C5-4C69-8BE2-27F0BDAA246E@stromnet.se> <7ad7ddd90801180128m114b0491p7d8f44ae7b1b4cb7@mail.gmail.com> In-Reply-To: <7ad7ddd90801180128m114b0491p7d8f44ae7b1b4cb7@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-stable@freebsd.org, =?ISO-8859-1?Q?Johan_Str=F6m?= Subject: Re: Backup solution suggestions [ggated] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jan 2008 19:18:05 -0000 Ulrich Spoerlein wrote: > On Jan 18, 2008 9:11 AM, Johan Ström wrote: >> Your no,barely, bad hell no seems to fit pretty good.. I did some >> testing during the night with the above (non-production) setup. >> What I did was doing some rsyncing over the night: >> >> while true ; do >> echo "`date` Clearing vmail" >> logfile >> rm -rf vmail >> echo "`date` Starting rsync" >> logfile >> rsync -vr /usr/var/vmail . |tee -a logfile >> echo "`date` Rsync finished " >> logfile >> done >> >> I started this at ~02.0. The results? A freshly rebooted 6.2 (6.2- >> RELEASE-p6 FreeBSD 6.2-RELEASE-p6 #0: Fri Jul 27 15:47:50 UTC 2007) >> box in the morning.. >> [...] >> What I dont have is a coredump, judging from dmesg -a savecore wasnt >> even run.. running it now, 5 hours later, didnt find any cores. >> >> The other end (7.0 server) wasnt affected at all. >> >> Not realy sure what it had been doing, because looking at my >> bandwidth graphs from the switch, nothing was done at all.. It didnt >> even go through one iteration of rsync... ~7.5k files/directorys >> seems to have been transfered, then the log doesnt say more. But >> according to the BW graph, after ~03.00 no traffic was sent at all... >> >> Some known bug with 6.2? > > There was some ggatec problems with TCP and/or sockets, I think they > have been mostly resolved post-6.2. If you want to pursue this further > (it *would* be a cool setup, no doubt) I'd suggest three things: > - Update to 6.3 > - Leave GELI out of the loop for now (only do ggate, with random data perhaps) > - Build a kernel *without* options PREEMPTION > Hi, We have 4 production High Avaliability NFS clusters running GMirror+GGate+LinuxHA (2 Nodes per cluster) on RELENG_6_2. This setup has proved very stable for us you have to do some tuning though: /etc/sysctl.conf: net.inet.tcp.sendspace=1048576 net.inet.tcp.recvspace=1048576 kern.ipc.maxsockbuf=2049152 /boot/loader.conf: kern.ipc.nmbclusters="32768" Command line options to ggate[cd] ggate[dc]_buf_size="1310720" ggatec_timeout="5" ggatec_queue_size="2048" Cluster node uptimes range from 40-160 days with the last reboots being caused by power problems not FreeBSD issues. The problems may be in the tuning or with geli, personally I would leave geli out at try with the above configuration. Then try above with geli to see what the problem is. Tom