From owner-freebsd-cluster@FreeBSD.ORG Tue May 10 19:07:55 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 733D116A4CE for ; Tue, 10 May 2005 19:07:55 +0000 (GMT) Received: from pop-a065d01.pas.sa.earthlink.net (pop-a065d01.pas.sa.earthlink.net [207.217.121.248]) by mx1.FreeBSD.org (Postfix) with ESMTP id C943743D95 for ; Tue, 10 May 2005 19:07:53 +0000 (GMT) (envelope-from alcentore@real-storage.com) Received: from fl-67-76-218-72.sta.sprint-hsd.net ([67.76.218.72] helo=anubis) by pop-a065d01.pas.sa.earthlink.net with smtp (Exim 3.36 #10) id 1DVa5B-0004qk-00 for freebsd-cluster@freebsd.org; Tue, 10 May 2005 12:07:53 -0700 From: "Anthony Centore" To: Date: Tue, 10 May 2005 15:10:19 -0400 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 1 (Highest) X-MSMail-Priority: High X-Mailer: Microsoft Outlook IMO, Build 9.0.6604 (9.0.2911.0) Importance: High X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 Subject: iSCSI and clustering with FreeBSD X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 May 2005 19:07:55 -0000 Peter, How far along are you in your freebsd iscsi development ? Does an iscsi driver for freebsd exist ? Appreciate your time. Anthony L Centore II -- No virus found in this outgoing message. Checked by AVG Anti-Virus. Version: 7.0.308 / Virus Database: 266.11.8 - Release Date: 5/10/2005 From owner-freebsd-cluster@FreeBSD.ORG Wed May 11 12:11:27 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4D06F16A4CE for ; Wed, 11 May 2005 12:11:27 +0000 (GMT) Received: from torch.higis.ru (gate.higis.ru [81.195.168.122]) by mx1.FreeBSD.org (Postfix) with ESMTP id 345A343D70 for ; Wed, 11 May 2005 12:11:24 +0000 (GMT) (envelope-from dimma@torch.higis.ru) Received: from localhost (localhost [127.0.0.1]) by torch.higis.ru (8.12.10/8.12.10) with ESMTP id j4BCBMxI011332 for ; Wed, 11 May 2005 16:11:22 +0400 (MSD) (envelope-from dimma@torch.higis.ru) Received: from torch.higis.ru ([127.0.0.1]) by localhost (torch.higis.ru [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 11291-01 for ; Wed, 11 May 2005 16:11:19 +0400 (MSD) Received: from torch.higis.ru (localhost [127.0.0.1]) by torch.higis.ru (8.12.10/8.12.10) with ESMTP id j4BCBGiV011329 for ; Wed, 11 May 2005 16:11:16 +0400 (MSD) (envelope-from dimma@torch.higis.ru) Received: (from dimma@localhost) by torch.higis.ru (8.12.10/8.12.10/Submit) id j4BCBGU4011328 for freebsd-cluster@freebsd.org; Wed, 11 May 2005 16:11:16 +0400 (MSD) (envelope-from dimma) Date: Wed, 11 May 2005 16:11:16 +0400 From: Dmitriy Kirhlarov To: freebsd-cluster@freebsd.org Message-ID: <20050511121115.GA2731@torch.higis.ru> References: <20050429183405.M60641@bmyster.com> Mime-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <20050429183405.M60641@bmyster.com> User-Agent: Mutt/1.5.6i X-Virus-Scanned: by amavisd-new at higis.ru Subject: Re: Clustering with Freebsd X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2005 12:11:27 -0000 Hi Brent! On Fri, 29 Apr 2005, Brent wrote: > I have a webhosting business all runnning on various Freebsd i386 boxes. I > was wondering if there are any good howto's on doing this ? The services id > like to cluster are, > apache > sendmail > bind 9.2.3 > UW imap replace to cyrus imapd. read about gmirror, carp, ggated. By. Dmitriy From owner-freebsd-cluster@FreeBSD.ORG Wed May 11 12:49:59 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4CE4416A4CE for ; Wed, 11 May 2005 12:49:59 +0000 (GMT) Received: from mh2.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8505243D6D for ; Wed, 11 May 2005 12:49:58 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh2.centtech.com (8.13.1/8.13.1) with ESMTP id j4BCnvU4077732; Wed, 11 May 2005 07:49:57 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <4281FF74.6060005@centtech.com> Date: Wed, 11 May 2005 07:49:56 -0500 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.7) Gecko/20050504 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Dmitriy Kirhlarov References: <20050429183405.M60641@bmyster.com> <20050511121115.GA2731@torch.higis.ru> In-Reply-To: <20050511121115.GA2731@torch.higis.ru> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-cluster@freebsd.org Subject: Re: Clustering with Freebsd X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2005 12:49:59 -0000 Dmitriy Kirhlarov wrote: > Hi Brent! > > On Fri, 29 Apr 2005, Brent wrote: > > >>I have a webhosting business all runnning on various Freebsd i386 boxes. I >>was wondering if there are any good howto's on doing this ? The services id >>like to cluster are, >>apache >>sendmail >>bind 9.2.3 >>UW imap > > > replace to cyrus imapd. > > read about gmirror, carp, ggated. How would gmirror and ggated help? Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology A lost ounce of gold may be found, a lost moment of time never. ------------------------------------------------------------------------ From owner-freebsd-cluster@FreeBSD.ORG Wed May 11 12:40:58 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5D7ED16A4D0 for ; Wed, 11 May 2005 12:40:58 +0000 (GMT) Received: from web15508.mail.cnb.yahoo.com (web15508.mail.cnb.yahoo.com [202.165.102.37]) by mx1.FreeBSD.org (Postfix) with SMTP id 14D5A43D48 for ; Wed, 11 May 2005 12:40:57 +0000 (GMT) (envelope-from shiner_chen@yahoo.com.cn) Message-ID: <20050511124052.40055.qmail@web15508.mail.cnb.yahoo.com> Received: from [61.187.15.109] by web15508.mail.cnb.yahoo.com via HTTP; Wed, 11 May 2005 20:40:52 CST Date: Wed, 11 May 2005 20:40:52 +0800 (CST) From: shiner chen To: freebsd-cluster@freebsd.org MIME-Version: 1.0 X-Mailman-Approved-At: Wed, 11 May 2005 13:17:04 +0000 Content-Type: text/plain; charset=gb2312 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.1 Subject: the web-cluster through the kld! X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2005 12:40:58 -0000 I will implement the web-cluster by the loadable module based on the tcp/ip stack code ,but ,the symbol definitions in the loadable module will conflict with the orignal network stack when the module is loaded. how do i resolve it? does the loadable module have independent namespace? i remmeber that the symorder can resolve it in freebsd 2.2.6 ,but i don't find it in freebsd 5.3! can you help me ? thanks shiner may 11th 2005 --------------------------------- Do You Yahoo!? 注册世界一流品质的雅虎免费电邮 From owner-freebsd-cluster@FreeBSD.ORG Wed May 11 14:07:06 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 574BF16A4D0 for ; Wed, 11 May 2005 14:07:06 +0000 (GMT) Received: from torch.higis.ru (gate.higis.ru [81.195.168.122]) by mx1.FreeBSD.org (Postfix) with ESMTP id 128F143D6B for ; Wed, 11 May 2005 14:07:05 +0000 (GMT) (envelope-from dimma@torch.higis.ru) Received: from localhost (localhost [127.0.0.1]) by torch.higis.ru (8.12.10/8.12.10) with ESMTP id j4BE73xI013738 for ; Wed, 11 May 2005 18:07:03 +0400 (MSD) (envelope-from dimma@torch.higis.ru) Received: from torch.higis.ru ([127.0.0.1]) by localhost (torch.higis.ru [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 13584-03 for ; Wed, 11 May 2005 18:07:00 +0400 (MSD) Received: from torch.higis.ru (localhost [127.0.0.1]) by torch.higis.ru (8.12.10/8.12.10) with ESMTP id j4BE6wiV013735 for ; Wed, 11 May 2005 18:06:58 +0400 (MSD) (envelope-from dimma@torch.higis.ru) Received: (from dimma@localhost) by torch.higis.ru (8.12.10/8.12.10/Submit) id j4BE6wKF013734 for freebsd-cluster@freebsd.org; Wed, 11 May 2005 18:06:58 +0400 (MSD) (envelope-from dimma) Date: Wed, 11 May 2005 18:06:58 +0400 From: Dmitriy Kirhlarov To: freebsd-cluster@freebsd.org Message-ID: <20050511140657.GB2731@torch.higis.ru> References: <20050429183405.M60641@bmyster.com> <20050511121115.GA2731@torch.higis.ru> <4281FF74.6060005@centtech.com> Mime-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <4281FF74.6060005@centtech.com> User-Agent: Mutt/1.5.6i X-Virus-Scanned: by amavisd-new at higis.ru Subject: Re: Clustering with Freebsd X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2005 14:07:06 -0000 Hi Eric! On Wed, 11 May 2005, Eric Anderson wrote: > Dmitriy Kirhlarov wrote: > >Hi Brent! > > > >On Fri, 29 Apr 2005, Brent wrote: > > > > > >>I have a webhosting business all runnning on various Freebsd i386 boxes. > >>I was wondering if there are any good howto's on doing this ? The > >>services id like to cluster are, > >>apache > >>sendmail > >>bind 9.2.3 > >>UW imap > > > > > >replace to cyrus imapd. > > > >read about gmirror, carp, ggated. > > How would gmirror and ggated help? Replicate data between two hosts, but it not very fine solution -- cached data not replicate to second node, when first node die. By. Dmitriy From owner-freebsd-cluster@FreeBSD.ORG Wed May 11 14:10:09 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1E90216A4CE for ; Wed, 11 May 2005 14:10:09 +0000 (GMT) Received: from mh2.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6F55543D2D for ; Wed, 11 May 2005 14:10:08 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh2.centtech.com (8.13.1/8.13.1) with ESMTP id j4BEA7a3078440; Wed, 11 May 2005 09:10:07 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <4282123E.6080909@centtech.com> Date: Wed, 11 May 2005 09:10:06 -0500 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.7) Gecko/20050504 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Dmitriy Kirhlarov References: <20050429183405.M60641@bmyster.com> <20050511121115.GA2731@torch.higis.ru> <4281FF74.6060005@centtech.com> <20050511140657.GB2731@torch.higis.ru> In-Reply-To: <20050511140657.GB2731@torch.higis.ru> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-cluster@freebsd.org Subject: Re: Clustering with Freebsd X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2005 14:10:09 -0000 Dmitriy Kirhlarov wrote: > Hi Eric! > > On Wed, 11 May 2005, Eric Anderson wrote: > > >>Dmitriy Kirhlarov wrote: >> >>>Hi Brent! >>> >>>On Fri, 29 Apr 2005, Brent wrote: >>> >>> >>> >>>>I have a webhosting business all runnning on various Freebsd i386 boxes. >>>>I was wondering if there are any good howto's on doing this ? The >>>>services id like to cluster are, >>>>apache >>>>sendmail >>>>bind 9.2.3 >>>>UW imap >>> >>> >>>replace to cyrus imapd. >>> >>>read about gmirror, carp, ggated. >> >>How would gmirror and ggated help? > > > Replicate data between two hosts, but it not very fine solution -- cached data not replicate to second node, when first node die. And it would only be for failover, not for load distribution/balancing. We really need a clustered filesystem for FreeBSD.. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology A lost ounce of gold may be found, a lost moment of time never. ------------------------------------------------------------------------ From owner-freebsd-cluster@FreeBSD.ORG Wed May 11 15:25:20 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 81D8A16A4CE for ; Wed, 11 May 2005 15:25:20 +0000 (GMT) Received: from priv-edtnes57.telusplanet.net (outbound01.telus.net [199.185.220.220]) by mx1.FreeBSD.org (Postfix) with ESMTP id BAF7943D64 for ; Wed, 11 May 2005 15:25:19 +0000 (GMT) (envelope-from Clusters@mpecsinc.com) Received: from MPECSSTN01 ([142.179.185.119]) by priv-edtnes57.telusplanet.netESMTP <20050511152519.HCIA26100.priv-edtnes57.telusplanet.net@MPECSSTN01> for ; Wed, 11 May 2005 09:25:19 -0600 From: "Phil E." To: "FreeBSD Clustering List" Date: Wed, 11 May 2005 09:25:43 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.6353 thread-index: AcVWPCPioj0a+GI7QmaPYxOMp2t4gAAAGJvg X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2527 In-Reply-To: <4282123E.6080909@centtech.com> Message-Id: <20050511152519.HCIA26100.priv-edtnes57.telusplanet.net@MPECSSTN01> Subject: Introduction & RE: Clustering with Freebsd X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2005 15:25:20 -0000 Hello all, My name is Philip Elder. I reside in St. Albert Alberta, Canada. My wife and I run our own I.T. business here primarily focused on Windows infrastructure though I still use the command line more than the GUI. :D I am pretty green when it comes to FreeBSD. I am learning, slowly. My goal is to build clusters, cluster based storage solutions, and some other ideas jumping around in my mind. I chose FreeBSD because of its stability. BTW, I do not have any programming experience at this time but I am willing to learn to as well. I do have a fair amount of server based infrastructure design, implementation, and support experience though (I've been in the industry since the early 1990's). I have been doing a lot of research and am wondering if perhaps this may be a place to start as far as a file system foundation: http://wiki.ethereal.com/ATA_20Over_20Ethernet Here is more from the developer: http://www.coraid.com/documents/SATAEtherDrive.pdf Please feel free to let me know if I am way off base here as this is a good way for me to learn! ;) Thanks! Philip Elder. -----Original Message----- From: owner-freebsd-cluster@freebsd.org [mailto:owner-freebsd-cluster@freebsd.org] On Behalf Of Eric Anderson Sent: Wednesday, May 11, 2005 8:10 AM To: Dmitriy Kirhlarov Cc: freebsd-cluster@freebsd.org Subject: Re: Clustering with Freebsd Dmitriy Kirhlarov wrote: > Hi Eric! > > On Wed, 11 May 2005, Eric Anderson wrote: ... > Replicate data between two hosts, but it not very fine solution -- cached data not replicate to second node, when first node die. And it would only be for failover, not for load distribution/balancing. We really need a clustered filesystem for FreeBSD.. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology A lost ounce of gold may be found, a lost moment of time never. ------------------------------------------------------------------------ _______________________________________________ freebsd-cluster@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-cluster To unsubscribe, send any mail to "freebsd-cluster-unsubscribe@freebsd.org" From owner-freebsd-cluster@FreeBSD.ORG Wed May 11 15:37:12 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7E80016A4CE for ; Wed, 11 May 2005 15:37:12 +0000 (GMT) Received: from tinker.exit.com (tinker.exit.com [206.223.0.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 24EF443D8B for ; Wed, 11 May 2005 15:37:12 +0000 (GMT) (envelope-from frank@exit.com) Received: from realtime.exit.com (realtime [206.223.0.5]) by tinker.exit.com (8.13.3/8.13.3) with ESMTP id j4BFbB9V081743; Wed, 11 May 2005 08:37:11 -0700 (PDT) (envelope-from frank@exit.com) Received: from realtime.exit.com (localhost [127.0.0.1]) by realtime.exit.com (8.13.3/8.13.3) with ESMTP id j4BFbBDG012508; Wed, 11 May 2005 08:37:11 -0700 (PDT) (envelope-from frank@realtime.exit.com) Received: (from frank@localhost) by realtime.exit.com (8.13.3/8.12.9/Submit) id j4BFbBRG012507; Wed, 11 May 2005 08:37:11 -0700 (PDT) (envelope-from frank) From: Frank Mayhar Message-Id: <200505111537.j4BFbBRG012507@realtime.exit.com> In-Reply-To: <20050511152519.HCIA26100.priv-edtnes57.telusplanet.net@MPECSSTN01> To: "Phil E." Date: Wed, 11 May 2005 08:37:11 -0700 (PDT) X-Copyright0: Copyright 2005 Frank Mayhar. All Rights Reserved. X-Copyright1: Permission granted for electronic reproduction as Usenet News or email only. X-Mailer: ELM [version 2.4ME+ PL119 (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII cc: FreeBSD Clustering List Subject: Re: Introduction & RE: Clustering with Freebsd X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: frank@exit.com List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2005 15:37:12 -0000 Phil E. wrote: > I have been doing a lot of research and am wondering if perhaps this may be > a place to start as far as a file system foundation: > > http://wiki.ethereal.com/ATA_20Over_20Ethernet > > Here is more from the developer: > > http://www.coraid.com/documents/SATAEtherDrive.pdf There are Linux-based clustered filesystems already extant. It's generally more efficient, I think, to do the clustering at a higher level than down at the device level (I haven't read those documents, but "ATA over Ethernet" sounds pretty low-level to me). You might take a look at the stuff in OpenSSI, http://www.openssi.org/, for a few pointers or ideas. Certainly there's a clustered file system there but it's by no means the only one. -- Frank Mayhar frank@exit.com http://www.exit.com/ Exit Consulting http://www.gpsclock.com/ http://www.exit.com/blog/frank/ From owner-freebsd-cluster@FreeBSD.ORG Wed May 11 15:41:57 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 628F616A4CE for ; Wed, 11 May 2005 15:41:57 +0000 (GMT) Received: from ms-dienst.rz.rwth-aachen.de (ms-2.rz.RWTH-Aachen.DE [134.130.3.131]) by mx1.FreeBSD.org (Postfix) with ESMTP id F3DEE43D73 for ; Wed, 11 May 2005 15:41:56 +0000 (GMT) (envelope-from chris@haakonia.hitnet.rwth-aachen.de) Received: from r220-1 (r220-1.rz.RWTH-Aachen.DE [134.130.3.31]) by ms-dienst.rz.rwth-aachen.de (iPlanet Messaging Server 5.2 Patch 2 (built Jul 14 2004)) with ESMTP id <0IGC00BDL0XV4E@ms-dienst.rz.rwth-aachen.de> for freeBSD-Cluster@freebsd.org; Wed, 11 May 2005 17:41:56 +0200 (MEST) Received: from relay.rwth-aachen.de ([134.130.3.1]) by r220-1 (MailMonitor for SMTP v1.2.2 ) ; Wed, 11 May 2005 17:41:55 +0200 (MEST) Received: from haakonia.hitnet.rwth-aachen.de (haakonia.hitnet.RWTH-Aachen.DE [137.226.181.92])j4BFfr6E022307; Wed, 11 May 2005 17:41:54 +0200 (MEST) Received: by haakonia.hitnet.rwth-aachen.de (Postfix, from userid 1001) id ADC142845D; Wed, 11 May 2005 17:41:53 +0200 (CEST) Date: Wed, 11 May 2005 17:41:53 +0200 From: Christian Brueffer In-reply-to: <20050511152519.HCIA26100.priv-edtnes57.telusplanet.net@MPECSSTN01> To: "Phil E." Message-id: <20050511154153.GD1232@unixpages.org> MIME-version: 1.0 Content-type: multipart/signed; boundary="2iBwrppp/7QCDedR"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-disposition: inline User-Agent: Mutt/1.5.6i X-Operating-System: FreeBSD 5.4-STABLE X-PGP-Key: http://people.FreeBSD.org/~brueffer/brueffer.key.asc X-PGP-Fingerprint: A5C8 2099 19FF AACA F41B B29B 6C76 178C A0ED 982D References: <4282123E.6080909@centtech.com> <20050511152519.HCIA26100.priv-edtnes57.telusplanet.net@MPECSSTN01> cc: FreeBSD Clustering List Subject: Re: Introduction & RE: Clustering with Freebsd X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2005 15:41:57 -0000 --2iBwrppp/7QCDedR Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, May 11, 2005 at 09:25:43AM -0600, Phil E. wrote: > Hello all, >=20 > My name is Philip Elder. I reside in St. Albert Alberta, Canada. >=20 > My wife and I run our own I.T. business here primarily focused on Windows > infrastructure though I still use the command line more than the GUI. :D >=20 > I am pretty green when it comes to FreeBSD. I am learning, slowly. My goal > is to build clusters, cluster based storage solutions, and some other ide= as > jumping around in my mind. I chose FreeBSD because of its stability. BTW,= I > do not have any programming experience at this time but I am willing to > learn to as well. I do have a fair amount of server based infrastructure > design, implementation, and support experience though (I've been in the > industry since the early 1990's). >=20 > I have been doing a lot of research and am wondering if perhaps this may = be > a place to start as far as a file system foundation: >=20 > http://wiki.ethereal.com/ATA_20Over_20Ethernet >=20 > Here is more from the developer: >=20 > http://www.coraid.com/documents/SATAEtherDrive.pdf >=20 > Please feel free to let me know if I am way off base here as this is a go= od > way for me to learn! ;) >=20 Hi Philip, the author of that ATA over Ethernet driver has been a committer for a few month and will probably import it into the source tree. - Christian --=20 Christian Brueffer chris@unixpages.org brueffer@FreeBSD.org GPG Key: http://people.freebsd.org/~brueffer/brueffer.key.asc GPG Fingerprint: A5C8 2099 19FF AACA F41B B29B 6C76 178C A0ED 982D --2iBwrppp/7QCDedR Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.6 (FreeBSD) iD8DBQFCgifBbHYXjKDtmC0RAoSuAKDZHjyMN87SoJD6sjNWftBjZJ2wQwCg9ZeD I58mmr19H7u7f8RW/0XTU1g= =Y86P -----END PGP SIGNATURE----- --2iBwrppp/7QCDedR-- From owner-freebsd-cluster@FreeBSD.ORG Wed May 11 16:00:44 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D63D516A4CE for ; Wed, 11 May 2005 16:00:44 +0000 (GMT) Received: from isncom.com (mail.isncom.com [216.22.101.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7D49243D72 for ; Wed, 11 May 2005 16:00:44 +0000 (GMT) (envelope-from gesperon@isncom.com) Received: from [192.168.7.19] (helo=isncom.com) by isncom.com with esmtp (Exim 4.30) id 1DVtAp-0004W3-Ni for freebsd-cluster@freebsd.org; Wed, 11 May 2005 11:30:59 -0400 Received: from [192.168.1.113] [192.168.1.113] by isncom.com with ESMTP (SMTPD32-7.07) id AD5E24D40152; Wed, 11 May 2005 10:57:34 -0400 In-Reply-To: <4282123E.6080909@centtech.com> References: <20050429183405.M60641@bmyster.com> <20050511121115.GA2731@torch.higis.ru> <4281FF74.6060005@centtech.com> <20050511140657.GB2731@torch.higis.ru> <4282123E.6080909@centtech.com> Mime-Version: 1.0 (Apple Message framework v728) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <44018F61-2594-4D10-B593-7D80AC9F5891@isncom.com> Content-Transfer-Encoding: 7bit From: Gabor Esperon Date: Wed, 11 May 2005 10:58:11 -0400 To: Eric Anderson X-Mailer: Apple Mail (2.728) X-Scan-Signature: 015602a392025d3fa82dd4c9564f3617 cc: freebsd-cluster@freebsd.org Subject: Re: Clustering with Freebsd X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2005 16:00:45 -0000 On May 11, 2005, at 10:10 AM, Eric Anderson wrote: > Dmitriy Kirhlarov wrote: > >> Hi Eric! >> On Wed, 11 May 2005, Eric Anderson wrote: >> >>> Dmitriy Kirhlarov wrote: >>> >>> >>>> Hi Brent! >>>> >>>> On Fri, 29 Apr 2005, Brent wrote: >>>> >>>> >>>> >>>> >>>>> I have a webhosting business all runnning on various Freebsd >>>>> i386 boxes. I was wondering if there are any good howto's on >>>>> doing this ? The services id like to cluster are, >>>>> apache >>>>> sendmail >>>>> bind 9.2.3 >>>>> UW imap >>>>> >>>> >>>> >>>> replace to cyrus imapd. >>>> >>>> read about gmirror, carp, ggated. >>>> >>> >>> How would gmirror and ggated help? >>> >> Replicate data between two hosts, but it not very fine solution -- >> cached data not replicate to second node, when first node die. >> > > And it would only be for failover, not for load distribution/ > balancing. > > We really need a clustered filesystem for FreeBSD.. > > Eric > I using ggated with freevrrpd for cluster failover. The freevrrpd allow you to define scripts to execute whatever the main host in the cluster goes down or up, so you can mount and execute startup scripts for your services (ej: /usr/local/etc/rc.d/apache2.sh start), and stop the service and umount when the main host goes up again. From owner-freebsd-cluster@FreeBSD.ORG Wed May 11 16:01:07 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 583E916A4E4 for ; Wed, 11 May 2005 16:01:07 +0000 (GMT) Received: from omega3.ebaseweb.com (omega3.ebaseweb.com [63.82.229.215]) by mx1.FreeBSD.org (Postfix) with ESMTP id D440E43D7F for ; Wed, 11 May 2005 16:01:06 +0000 (GMT) (envelope-from clewis@ebaseweb.com) Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Date: Wed, 11 May 2005 11:01:05 -0500 Message-ID: <3EA9E48804411F41941A18169042D10401469D@omega3.ebaseweb.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Introduction & RE: Clustering with Freebsd Thread-Index: AcVWP/4qHcyc6Fq1RA6O7jaX7iP8KwAAGfPA From: "Craig Lewis" To: "FreeBSD Clustering List" Subject: RE: Introduction & RE: Clustering with Freebsd X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2005 16:01:07 -0000 Brilliant, this sounds like a great alternative to iscsi! ? So potentially the 6.0 release might contain aoe? On roughly that time scale? I have had extensive discussion on another list about building a free bsd based mail server. Since currently our mail server is not that heavily loaded I was thinking I might could get away with gigabit, NFS, and maildir format to 'share' the filesystem between the nodes. Then as we grew as a company, maybe we could better afford network attatched storage, and maybe there would be better support in bsd for things like iscsi or now AOE. Yes, I know the TCPIP stack and gigabit would consume lots of cpu, but both of these are getting cheaper and cheaper, and again.. .we are not even close to a high load situation.=20 Any one have comments, experience, or suggestions on the above, or an even better way to share file systems on io intensive servers like mail servers. ? I would be interested to hear any summary reports on using network attatched storage on freebsd. i.e. lessons learned. -----Original Message----- From: owner-freebsd-cluster@freebsd.org [mailto:owner-freebsd-cluster@freebsd.org] On Behalf Of Christian Brueffer Sent: Wednesday, May 11, 2005 10:42 AM To: Phil E. Cc: FreeBSD Clustering List Subject: Re: Introduction & RE: Clustering with Freebsd On Wed, May 11, 2005 at 09:25:43AM -0600, Phil E. wrote: > Hello all, >=20 > My name is Philip Elder. I reside in St. Albert Alberta, Canada. >=20 > My wife and I run our own I.T. business here primarily focused on Windows > infrastructure though I still use the command line more than the GUI. :D >=20 > I am pretty green when it comes to FreeBSD. I am learning, slowly. My goal > is to build clusters, cluster based storage solutions, and some other ideas > jumping around in my mind. I chose FreeBSD because of its stability. BTW, I > do not have any programming experience at this time but I am willing to > learn to as well. I do have a fair amount of server based infrastructure > design, implementation, and support experience though (I've been in the > industry since the early 1990's). >=20 > I have been doing a lot of research and am wondering if perhaps this may be > a place to start as far as a file system foundation: >=20 > http://wiki.ethereal.com/ATA_20Over_20Ethernet >=20 > Here is more from the developer: >=20 > http://www.coraid.com/documents/SATAEtherDrive.pdf >=20 > Please feel free to let me know if I am way off base here as this is a good > way for me to learn! ;) >=20 Hi Philip, the author of that ATA over Ethernet driver has been a committer for a few month and will probably import it into the source tree. - Christian --=20 Christian Brueffer chris@unixpages.org brueffer@FreeBSD.org GPG Key: http://people.freebsd.org/~brueffer/brueffer.key.asc GPG Fingerprint: A5C8 2099 19FF AACA F41B B29B 6C76 178C A0ED 982D From owner-freebsd-cluster@FreeBSD.ORG Wed May 11 22:38:26 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 25DDA16A4F3 for ; Wed, 11 May 2005 22:38:26 +0000 (GMT) Received: from mailhub2.uq.edu.au (mailhub2.uq.edu.au [130.102.149.128]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3B4F143D66 for ; Wed, 11 May 2005 22:38:25 +0000 (GMT) (envelope-from m.swinbourne@uq.edu.au) Received: from smtp2.uq.edu.au (newsmtp2.uq.edu.au [130.102.149.129]) by mailhub2.uq.edu.au (8.13.3/8.13.3) with ESMTP id j4BMcNe4083700 for ; Thu, 12 May 2005 08:38:23 +1000 (EST) Received: from jemima.soe.uq.edu.au (jemima.soe.uq.edu.au [130.102.4.196]) by smtp2.uq.edu.au (8.13.3/8.13.3) with ESMTP id j4BMcNCD083697 for ; Thu, 12 May 2005 08:38:23 +1000 (EST) Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Date: Thu, 12 May 2005 08:38:23 +1000 Message-ID: <5B5BBC864783B7439264DD5FF664F2A969F31D@jemima.soe.uq.edu.au> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Introduction & RE: Clustering with Freebsd Thread-Index: AcVWP/4qHcyc6Fq1RA6O7jaX7iP8KwAAGfPAAA5e/LA= From: "Matthew SWINBOURNE" To: "FreeBSD Clustering List" X-Scanned-By: MIMEDefang 2.51 on UQ Mailhub on 130.102.149.128 Subject: RE: Introduction & RE: Clustering with Freebsd X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 May 2005 22:38:26 -0000 Hi All, Something to think about along these lines are TCP Off-Loader (TOE) cards. If you are seriously looking at ATA over ethernet or something similar on any production scale, then these cards are a life saver. Reduce the CPU overhead of running storage via TCP/IP to almost negligable.=20 Cheers, Matt -----Original Message----- From: owner-freebsd-cluster@freebsd.org [mailto:owner-freebsd-cluster@freebsd.org] On Behalf Of Craig Lewis Sent: Thursday, 12 May 2005 2:01 AM To: FreeBSD Clustering List Subject: RE: Introduction & RE: Clustering with Freebsd Brilliant, this sounds like a great alternative to iscsi! ? So potentially the 6.0 release might contain aoe? On roughly that time scale? I have had extensive discussion on another list about building a free bsd based mail server. Since currently our mail server is not that heavily loaded I was thinking I might could get away with gigabit, NFS, and maildir format to 'share' the filesystem between the nodes. Then as we grew as a company, maybe we could better afford network attatched storage, and maybe there would be better support in bsd for things like iscsi or now AOE. Yes, I know the TCPIP stack and gigabit would consume lots of cpu, but both of these are getting cheaper and cheaper, and again.. .we are not even close to a high load situation.=20 Any one have comments, experience, or suggestions on the above, or an even better way to share file systems on io intensive servers like mail servers. ? I would be interested to hear any summary reports on using network attatched storage on freebsd. i.e. lessons learned. -----Original Message----- From: owner-freebsd-cluster@freebsd.org [mailto:owner-freebsd-cluster@freebsd.org] On Behalf Of Christian Brueffer Sent: Wednesday, May 11, 2005 10:42 AM To: Phil E. Cc: FreeBSD Clustering List Subject: Re: Introduction & RE: Clustering with Freebsd On Wed, May 11, 2005 at 09:25:43AM -0600, Phil E. wrote: > Hello all, >=20 > My name is Philip Elder. I reside in St. Albert Alberta, Canada. >=20 > My wife and I run our own I.T. business here primarily focused on Windows > infrastructure though I still use the command line more than the GUI. :D >=20 > I am pretty green when it comes to FreeBSD. I am learning, slowly. My goal > is to build clusters, cluster based storage solutions, and some other ideas > jumping around in my mind. I chose FreeBSD because of its stability. BTW, I > do not have any programming experience at this time but I am willing to > learn to as well. I do have a fair amount of server based infrastructure > design, implementation, and support experience though (I've been in the > industry since the early 1990's). >=20 > I have been doing a lot of research and am wondering if perhaps this may be > a place to start as far as a file system foundation: >=20 > http://wiki.ethereal.com/ATA_20Over_20Ethernet >=20 > Here is more from the developer: >=20 > http://www.coraid.com/documents/SATAEtherDrive.pdf >=20 > Please feel free to let me know if I am way off base here as this is a good > way for me to learn! ;) >=20 Hi Philip, the author of that ATA over Ethernet driver has been a committer for a few month and will probably import it into the source tree. - Christian --=20 Christian Brueffer chris@unixpages.org brueffer@FreeBSD.org GPG Key: http://people.freebsd.org/~brueffer/brueffer.key.asc GPG Fingerprint: A5C8 2099 19FF AACA F41B B29B 6C76 178C A0ED 982D _______________________________________________ freebsd-cluster@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-cluster To unsubscribe, send any mail to "freebsd-cluster-unsubscribe@freebsd.org" From owner-freebsd-cluster@FreeBSD.ORG Thu May 12 02:16:33 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7503616A4CE for ; Thu, 12 May 2005 02:16:33 +0000 (GMT) Received: from smtp.263.net (mx01.263.net.cn [211.150.96.22]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6B04043D31 for ; Thu, 12 May 2005 02:16:30 +0000 (GMT) (envelope-from yfyoufeng@263.net) Received: from [10.217.12.198] (localhost [127.0.0.1]) by smtp.263.net (Postfix) with ESMTP id 37CA6C34A3; Thu, 12 May 2005 10:16:25 +0800 (CST) (envelope-from yfyoufeng@263.net) Received: from [10.217.12.198] (unknown [61.135.152.194]) by antispam-2 (Coremail:263(050316)) with SMTP id FAE/AHi8gkKYAZjC.1 for ; Thu, 12 May 2005 10:16:25 +0800 (CST) X-TEBIE-Originating-IP: [61.135.152.194] From: yf-263 To: Brent In-Reply-To: <20050429183405.M60641@bmyster.com> References: <20050429183405.M60641@bmyster.com> Content-Type: text/plain; charset=UTF-8 Organization: Unix-driver.org Date: Thu, 12 May 2005 10:15:55 +0800 Message-Id: <1115864155.3938.13.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.0.4 (2.0.4-4) Content-Transfer-Encoding: 8bit cc: freebsd-cluster@freebsd.org Subject: Re: Clustering with Freebsd X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: yfyoufeng@263.net List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 May 2005 02:16:33 -0000 Please read the Google.com's GoogleFS. And now we are working on implementing a GoogleFS like Cluster Storage for my Employer, which is target for the web, mail, IM, Netdisk, etc. services, and we have more than 1,000 million users. For GoogleFS, the upper layer is like an asymmetric cluster filesystem as www.lustre.org; and in block layer, like V9FS' Venti. What is also our target ;) Deeper info is welcomed to be discussed :) 鍦 2005-04-29浜旂殑 15:02 -0400锛孊rent鍐欓亾锛 > I have a webhosting business all runnning on various Freebsd i386 boxes. I > was wondering if there are any good howto's on doing this ? The services id > like to cluster are, > apache > sendmail > bind 9.2.3 > UW imap > > >From what i understand there are a couple ways to do this. The load balancer > method where one machine acts as a load balancer for nodes that run your > services like apache, sendmail, DNS. So when load becomes so high...you just > add another machine to help take off some of the load or when a box fails the > rest of the machines in the cluster pick up in its place. > > the other way is to do it like Solaris does with veritas cluster services. > where each machine runs a service and is monitored by the cluster and in the > event of a failure...the service & IP & all would get moved to another box .. > > any help would be great thank you > > > -- > Brent Bailey > _______________________________________________ > freebsd-cluster@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-cluster > To unsubscribe, send any mail to "freebsd-cluster-unsubscribe@freebsd.org" > -- yf-263 Unix-driver.org From owner-freebsd-cluster@FreeBSD.ORG Thu May 12 02:18:10 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2601C16A4D0 for ; Thu, 12 May 2005 02:18:10 +0000 (GMT) Received: from smtp.263.net (smtp.x263.net [211.150.96.25]) by mx1.FreeBSD.org (Postfix) with ESMTP id B6C2E43D78 for ; Thu, 12 May 2005 02:18:09 +0000 (GMT) (envelope-from yfyoufeng@263.net) Received: from [10.217.12.198] (mta5 [127.0.0.1]) by smtp.263.net (Postfix) with ESMTP id CA8222F7B4; Thu, 12 May 2005 10:18:07 +0800 (CST) (envelope-from yfyoufeng@263.net) Received: from [10.217.12.198] (unknown [61.135.152.194]) by antispam-4 (Coremail:263(050316)) with SMTP id myDIAd+8gkLlIZjC.1 for ; Thu, 12 May 2005 10:18:08 +0800 (CST) X-TEBIE-Originating-IP: [61.135.152.194] From: yf-263 To: Eric Anderson In-Reply-To: <4282123E.6080909@centtech.com> References: <20050429183405.M60641@bmyster.com> <4281FF74.6060005@centtech.com><4282123E.6080909@centtech.com> Content-Type: text/plain; charset=utf-8 Organization: Unix-driver.org Date: Thu, 12 May 2005 10:17:38 +0800 Message-Id: <1115864258.3938.15.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.0.4 (2.0.4-4) Content-Transfer-Encoding: 8bit cc: =?UTF-8?Q?=E5=8D=9A=E5=A3=AB_=E5=94=90_=E8=8D=A3=E9=94=8B_?= =?UTF-8?Q?=E6=99=9A?= =?UTF-8?Q?=E8=BE=88?= cc: freebsd-cluster@freebsd.org Subject: Re: Clustering with Freebsd X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: yfyoufeng@263.net List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 May 2005 02:18:10 -0000 鍦 2005-05-11涓夌殑 09:10 -0500锛孍ric Anderson鍐欓亾锛 > Dmitriy Kirhlarov wrote: > > Hi Eric! > > > > On Wed, 11 May 2005, Eric Anderson wrote: > > > > > >>Dmitriy Kirhlarov wrote: > >> > >>>Hi Brent! > >>> > >>>On Fri, 29 Apr 2005, Brent wrote: > >>> > >>> > >>> > >>>>I have a webhosting business all runnning on various Freebsd i386 boxes. > >>>>I was wondering if there are any good howto's on doing this ? The > >>>>services id like to cluster are, > >>>>apache > >>>>sendmail > >>>>bind 9.2.3 > >>>>UW imap > >>> > >>> > >>>replace to cyrus imapd. > >>> > >>>read about gmirror, carp, ggated. > >> > >>How would gmirror and ggated help? > > > > > > Replicate data between two hosts, but it not very fine solution -- cached data not replicate to second node, when first node die. > > And it would only be for failover, not for load distribution/balancing. > > We really need a clustered filesystem for FreeBSD.. Forget recite that we are also implementing it on FreeBSD ;) For the HA and HP purpose. > > Eric > > > -- yf-263 Unix-driver.org From owner-freebsd-cluster@FreeBSD.ORG Thu May 12 02:26:34 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2CB3B16A4CE for ; Thu, 12 May 2005 02:26:34 +0000 (GMT) Received: from smtp.263.net (mx01.263.net.cn [211.150.96.22]) by mx1.FreeBSD.org (Postfix) with ESMTP id DCF9243D39 for ; Thu, 12 May 2005 02:26:32 +0000 (GMT) (envelope-from yfyoufeng@263.net) Received: from [10.217.12.198] (localhost [127.0.0.1]) by smtp.263.net (Postfix) with ESMTP id D8646C35BD; Thu, 12 May 2005 10:26:29 +0800 (CST) (envelope-from yfyoufeng@263.net) Received: from [10.217.12.198] (unknown [61.135.152.194]) by antispam-2 (Coremail:263(050316)) with SMTP id JAFAANW+gkLxAZjC.1 for ; Thu, 12 May 2005 10:26:30 +0800 (CST) X-TEBIE-Originating-IP: [61.135.152.194] From: yf-263 To: Craig Lewis In-Reply-To: <3EA9E48804411F41941A18169042D10401469D@omega3.ebaseweb.com> References: <3EA9E48804411F41941A18169042D10401469D@omega3.ebaseweb.com> Content-Type: text/plain; charset=utf-8 Organization: Unix-driver.org Date: Thu, 12 May 2005 10:25:56 +0800 Message-Id: <1115864757.3938.19.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.0.4 (2.0.4-4) Content-Transfer-Encoding: 8bit cc: FreeBSD Clustering List Subject: RE: Introduction & RE: Clustering with Freebsd X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: yfyoufeng@263.net List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 May 2005 02:26:34 -0000 鍦 2005-05-11涓夌殑 11:01 -0500锛孋raig Lewis鍐欓亾锛 > Brilliant, this sounds like a great alternative to iscsi! ? So > potentially the 6.0 release might contain aoe? On roughly that time > scale? > > I have had extensive discussion on another list about building a free > bsd based mail server. Since currently our mail server is not that Can you tell where is the discussion ? > heavily loaded I was thinking I might could get away with gigabit, NFS, > and maildir format to 'share' the filesystem between the nodes. Then as That's what we are developing now. > we grew as a company, maybe we could better afford network attatched > storage, and maybe there would be better support in bsd for things like Yes, also the unlimited-disk-space netdisk we supply ;) > iscsi or now AOE. Yes, I know the TCPIP stack and gigabit would consume > lots of cpu, but both of these are getting cheaper and cheaper, and > again.. .we are not even close to a high load situation. > > Any one have comments, experience, or suggestions on the above, or an > even better way to share file systems on io intensive servers like mail > servers. ? I would be interested to hear any summary reports on using > network attatched storage on freebsd. i.e. lessons learned. We drink the same water :) > > > > -----Original Message----- > From: owner-freebsd-cluster@freebsd.org > [mailto:owner-freebsd-cluster@freebsd.org] On Behalf Of Christian > Brueffer > Sent: Wednesday, May 11, 2005 10:42 AM > To: Phil E. > Cc: FreeBSD Clustering List > Subject: Re: Introduction & RE: Clustering with Freebsd > > On Wed, May 11, 2005 at 09:25:43AM -0600, Phil E. wrote: > > Hello all, > > > > My name is Philip Elder. I reside in St. Albert Alberta, Canada. > > > > My wife and I run our own I.T. business here primarily focused on > Windows > > infrastructure though I still use the command line more than the GUI. > :D > > > > I am pretty green when it comes to FreeBSD. I am learning, slowly. My > goal > > is to build clusters, cluster based storage solutions, and some other > ideas > > jumping around in my mind. I chose FreeBSD because of its stability. > BTW, I > > do not have any programming experience at this time but I am willing > to > > learn to as well. I do have a fair amount of server based > infrastructure > > design, implementation, and support experience though (I've been in > the > > industry since the early 1990's). > > > > I have been doing a lot of research and am wondering if perhaps this > may be > > a place to start as far as a file system foundation: > > > > http://wiki.ethereal.com/ATA_20Over_20Ethernet > > > > Here is more from the developer: > > > > http://www.coraid.com/documents/SATAEtherDrive.pdf > > > > Please feel free to let me know if I am way off base here as this is a > good > > way for me to learn! ;) > > > > Hi Philip, > > the author of that ATA over Ethernet driver has been a committer for a > few month and will probably import it into the source tree. > > - Christian > -- yf-263 Unix-driver.org From owner-freebsd-cluster@FreeBSD.ORG Thu May 12 03:31:36 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 26FC116A4CE for ; Thu, 12 May 2005 03:31:36 +0000 (GMT) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7617843D46 for ; Thu, 12 May 2005 03:31:35 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [192.168.42.23] (andersonbox3.centtech.com [192.168.42.23]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id j4C3VN1S014537; Wed, 11 May 2005 22:31:29 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <4282CE09.6000802@centtech.com> Date: Wed, 11 May 2005 22:31:21 -0500 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.7) Gecko/20050504 X-Accept-Language: en-us, en MIME-Version: 1.0 To: yfyoufeng@263.net References: <20050429183405.M60641@bmyster.com> <20050511121115.GA2731@torch.higis.ru> <4281FF74.6060005@centtech.com> <20050511140657.GB2731@torch.higis.ru> <4282123E.6080909@centtech.com> <1115864258.3938.15.camel@localhost.localdomain> In-Reply-To: <1115864258.3938.15.camel@localhost.localdomain> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Virus-Scanned: ClamAV 0.82/875/Tue May 10 06:27:59 2005 on mh1.centtech.com X-Virus-Status: Clean cc: =?UTF-8?B?5Y2a5aOrIOWUkCDojaPplIsg5pma6L6I?= cc: freebsd-cluster@freebsd.org Subject: Re: Clustering with Freebsd X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 May 2005 03:31:36 -0000 yf-263 wrote: > =E5=9C=A8 2005-05-11=E4=B8=89=E7=9A=84 09:10 -0500=EF=BC=8CEric Anderso= n=E5=86=99=E9=81=93=EF=BC=9A >=20 >>Dmitriy Kirhlarov wrote: >> >>>Hi Eric! >>> >>>On Wed, 11 May 2005, Eric Anderson wrote: >>> >>> >>> >>>>Dmitriy Kirhlarov wrote: >>>> >>>> >>>>>Hi Brent! >>>>> >>>>>On Fri, 29 Apr 2005, Brent wrote: >>>>> >>>>> >>>>> >>>>> >>>>>>I have a webhosting business all runnning on various Freebsd i386 b= oxes.=20 >>>>>>I was wondering if there are any good howto's on doing this ? The=20 >>>>>>services id like to cluster are, >>>>>>apache >>>>>>sendmail >>>>>>bind 9.2.3 >>>>>>UW imap >>>>> >>>>> >>>>>replace to cyrus imapd. >>>>> >>>>>read about gmirror, carp, ggated. >>>> >>>>How would gmirror and ggated help? >>> >>> >>>Replicate data between two hosts, but it not very fine solution -- cac= hed data not replicate to second node, when first node die. >> >>And it would only be for failover, not for load distribution/balancing.= >> >>We really need a clustered filesystem for FreeBSD.. >=20 >=20 > Forget recite that we are also implementing it on FreeBSD ;) For the HA= > and HP purpose. For my purposes, I need to be able to serve data on that filesystem via=20 NFS - will that be possible too? Will this code be open source? I'd actually like to see a FreeBSD port of GFS.. Eric --=20 ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology A lost ounce of gold may be found, a lost moment of time never. ------------------------------------------------------------------------ From owner-freebsd-cluster@FreeBSD.ORG Thu May 12 03:44:25 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2BC4316A4CE for ; Thu, 12 May 2005 03:44:25 +0000 (GMT) Received: from smtp.263.net (263.net.cn [211.150.96.22]) by mx1.FreeBSD.org (Postfix) with ESMTP id E69D543D81 for ; Thu, 12 May 2005 03:44:23 +0000 (GMT) (envelope-from yfyoufeng@263.net) Received: from [10.217.12.198] (localhost [127.0.0.1]) by smtp.263.net (Postfix) with ESMTP id 44588C35F2; Thu, 12 May 2005 11:44:21 +0800 (CST) (envelope-from yfyoufeng@263.net) Received: from [10.217.12.198] (unknown [61.135.152.194]) by antispam-2 (Coremail:263(050316)) with SMTP id BRE+ABXRgkKcA5jC.1 for ; Thu, 12 May 2005 11:44:21 +0800 (CST) X-TEBIE-Originating-IP: [61.135.152.194] From: yf-263 To: Eric Anderson In-Reply-To: <4282CE09.6000802@centtech.com> References: <20050429183405.M60641@bmyster.com> <4281FF74.6060005@centtech.com><4282123E.6080909@centtech.com> <1115864258.3938.15.camel@localhost.localdomain> <4282CE09.6000802@centtech.com> Content-Type: text/plain; charset=UTF-8 Organization: Unix-driver.org Date: Thu, 12 May 2005 11:43:51 +0800 Message-Id: <1115869431.3938.46.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.0.4 (2.0.4-4) Content-Transfer-Encoding: 8bit cc: freebsd-cluster@freebsd.org Subject: Re: Clustering with Freebsd X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: yfyoufeng@263.net List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 May 2005 03:44:25 -0000 鍦 2005-05-11涓夌殑 22:31 -0500锛孍ric Anderson鍐欓亾锛 > yf-263 wrote: > > 鍦 2005-05-11涓夌殑 09:10 -0500锛孍ric Anderson鍐欓亾锛 > > > >>Dmitriy Kirhlarov wrote: > >> > >>>Hi Eric! > >>> > >>>On Wed, 11 May 2005, Eric Anderson wrote: > >>> > >>> > >>> > >>>>Dmitriy Kirhlarov wrote: > >>>> > >>>> > >>>>>Hi Brent! > >>>>> > >>>>>On Fri, 29 Apr 2005, Brent wrote: > >>>>> > >>>>> > >>>>> > >>>>> > >>>>>>I have a webhosting business all runnning on various Freebsd i386 boxes. > >>>>>>I was wondering if there are any good howto's on doing this ? The > >>>>>>services id like to cluster are, > >>>>>>apache > >>>>>>sendmail > >>>>>>bind 9.2.3 > >>>>>>UW imap > >>>>> > >>>>> > >>>>>replace to cyrus imapd. > >>>>> > >>>>>read about gmirror, carp, ggated. > >>>> > >>>>How would gmirror and ggated help? > >>> > >>> > >>>Replicate data between two hosts, but it not very fine solution -- cached data not replicate to second node, when first node die. > >> > >>And it would only be for failover, not for load distribution/balancing. > >> > >>We really need a clustered filesystem for FreeBSD.. > > > > > > Forget recite that we are also implementing it on FreeBSD ;) For the HA > > and HP purpose. > > For my purposes, I need to be able to serve data on that filesystem via Use our FS, you will not need NFS anymore, cause it is also a distributed FS. And it will also reduce the cost to 1/3 ~ 1/4. > NFS - will that be possible too? Will this code be open source? Now we are considering the possibility of open source to gain a well project life :) > > I'd actually like to see a FreeBSD port of GFS.. Before I have worked on porting Redhat GFS to Darwin, which use FreeBSD FS infrastructure. And the porting works on CML ;) > > Eric > > > -- yf-263 Unix-driver.org From owner-freebsd-cluster@FreeBSD.ORG Thu May 12 03:45:58 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4178816A4CE for ; Thu, 12 May 2005 03:45:58 +0000 (GMT) Received: from mh2.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id BD12643D69 for ; Thu, 12 May 2005 03:45:57 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [192.168.42.23] (andersonbox3.centtech.com [192.168.42.23]) by mh2.centtech.com (8.13.1/8.13.1) with ESMTP id j4C3jvLa086893 for ; Wed, 11 May 2005 22:45:57 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <4282D173.1040805@centtech.com> Date: Wed, 11 May 2005 22:45:55 -0500 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.7) Gecko/20050504 X-Accept-Language: en-us, en MIME-Version: 1.0 To: FreeBSD Clustering List References: <3EA9E48804411F41941A18169042D10401469D@omega3.ebaseweb.com> <1115864757.3938.19.camel@localhost.localdomain> In-Reply-To: <1115864757.3938.19.camel@localhost.localdomain> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: FreeBSD Clustering wishlist - Was: Introduction & RE: Clustering with Freebsd X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 May 2005 03:45:58 -0000 Ok - I'm changing the subject here in an attempt to gather information. Here's my wishlist: FreeBSD have a 'native' clustered filesystem. This is different than shared media (we already can do that over fiber channel, ggated, soon iscsi and AOE). This would allow multiple servers to access the same data read/write - highly important for load balancing applications like web servers, mail servers, and NFS servers. Online growable filesystem. I know I can growfs a filesystem now, but doing online while data is being used is *insanely* useful. Reiserfs and Polyserve's FS (a clustered filesystem, not open-source) do this well. FreeBSD's UFS2 made to do journaling. There's already someone working on this. I believe the above mean that we need a distributed lock manager too, so might as well add that to my wishlist. Single filesystem limits set very high - 16TB would be a good minimum. Vinum/geom (?) made to allow added a couple more 'disks' - be it a real scsi device, or another vinum device - to existing vinum's, so I can extend my vinum stripe, raid, concat, etc to a larger volume size, without worrying about which disk is where. I want to stripe mirrors of raids, and raid striped mirrors of stripes. I know it sounds crazy, but I really *do* have uses for all this. :) We currently pay lots of money every year (enough to pay an engineers salary) for support and maintenance with Polyserve. They make a good product (we need it for the clustered filesystem and NFS distributed lock manager stuff) - I'd much rather see that go to FreeBSD. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology A lost ounce of gold may be found, a lost moment of time never. ------------------------------------------------------------------------ From owner-freebsd-cluster@FreeBSD.ORG Thu May 12 03:49:08 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6931B16A4CE for ; Thu, 12 May 2005 03:49:08 +0000 (GMT) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id E839043D83 for ; Thu, 12 May 2005 03:49:07 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [192.168.42.23] (andersonbox3.centtech.com [192.168.42.23]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id j4C3n47Q014639; Wed, 11 May 2005 22:49:04 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <4282D22E.9030204@centtech.com> Date: Wed, 11 May 2005 22:49:02 -0500 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.7) Gecko/20050504 X-Accept-Language: en-us, en MIME-Version: 1.0 To: yfyoufeng@263.net References: <20050429183405.M60641@bmyster.com> <20050511121115.GA2731@torch.higis.ru> <4281FF74.6060005@centtech.com> <20050511140657.GB2731@torch.higis.ru> <4282123E.6080909@centtech.com> <1115864258.3938.15.camel@localhost.localdomain> <4282CE09.6000802@centtech.com> <1115869431.3938.46.camel@localhost.localdomain> In-Reply-To: <1115869431.3938.46.camel@localhost.localdomain> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Virus-Scanned: ClamAV 0.82/875/Tue May 10 06:27:59 2005 on mh1.centtech.com X-Virus-Status: Clean cc: freebsd-cluster@freebsd.org Subject: Re: Clustering with Freebsd X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 May 2005 03:49:08 -0000 yf-263 wrote: > =E5=9C=A8 2005-05-11=E4=B8=89=E7=9A=84 22:31 -0500=EF=BC=8CEric Anderso= n=E5=86=99=E9=81=93=EF=BC=9A >=20 >>yf-263 wrote: >> >>>=E5=9C=A8 2005-05-11=E4=B8=89=E7=9A=84 09:10 -0500=EF=BC=8CEric Anders= on=E5=86=99=E9=81=93=EF=BC=9A >>> >>> >>>>Dmitriy Kirhlarov wrote: >>>> >>>> >>>>>Hi Eric! >>>>> >>>>>On Wed, 11 May 2005, Eric Anderson wrote: >>>>> >>>>> >>>>> >>>>> >>>>>>Dmitriy Kirhlarov wrote: >>>>>> >>>>>> >>>>>> >>>>>>>Hi Brent! >>>>>>> >>>>>>>On Fri, 29 Apr 2005, Brent wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>I have a webhosting business all runnning on various Freebsd i386= boxes.=20 >>>>>>>>I was wondering if there are any good howto's on doing this ? The= =20 >>>>>>>>services id like to cluster are, >>>>>>>>apache >>>>>>>>sendmail >>>>>>>>bind 9.2.3 >>>>>>>>UW imap >>>>>>> >>>>>>> >>>>>>>replace to cyrus imapd. >>>>>>> >>>>>>>read about gmirror, carp, ggated. >>>>>> >>>>>>How would gmirror and ggated help? >>>>> >>>>> >>>>>Replicate data between two hosts, but it not very fine solution -- c= ached data not replicate to second node, when first node die. >>>> >>>>And it would only be for failover, not for load distribution/balancin= g. >>>> >>>>We really need a clustered filesystem for FreeBSD.. >>> >>> >>>Forget recite that we are also implementing it on FreeBSD ;) For the H= A >>>and HP purpose. >> >>For my purposes, I need to be able to serve data on that filesystem via= =20 >=20 >=20 > Use our FS, you will not need NFS anymore, cause it is also a > distributed FS. And it will also reduce the cost to 1/3 ~ 1/4. I have about 1000 high-end clients using the data - mostly linux, but=20 also HP-UX, solaris, and openbsd. Would it work on all those just as=20 well? I put a *huge* strain on servers. >>NFS - will that be possible too? Will this code be open source? >=20 >=20 > Now we are considering the possibility of open source to gain a well > project life :)=20 I'm listening!! :) >>I'd actually like to see a FreeBSD port of GFS.. >=20 >=20 > Before I have worked on porting Redhat GFS to Darwin, which use FreeBSD= > FS infrastructure. And the porting works on CML ;) I don't know what CML is off the top of my half awake brain, but I'd=20 like to hear more.. Eric --=20 ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology A lost ounce of gold may be found, a lost moment of time never. ------------------------------------------------------------------------ From owner-freebsd-cluster@FreeBSD.ORG Thu May 12 04:34:40 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 02DD716A4CE for ; Thu, 12 May 2005 04:34:40 +0000 (GMT) Received: from smtp.263.net (mx01.263.net.cn [211.150.96.22]) by mx1.FreeBSD.org (Postfix) with ESMTP id B41A543D82 for ; Thu, 12 May 2005 04:34:38 +0000 (GMT) (envelope-from yfyoufeng@263.net) Received: from [10.217.12.198] (localhost [127.0.0.1]) by smtp.263.net (Postfix) with ESMTP id 6CEA4C3792; Thu, 12 May 2005 12:34:35 +0800 (CST) (envelope-from yfyoufeng@263.net) Received: from [10.217.12.198] (unknown [61.135.152.194]) by antispam-2 (Coremail:263(050316)) with SMTP id cqA1ANvcgkJMBJjC.1 for ; Thu, 12 May 2005 12:34:35 +0800 (CST) X-TEBIE-Originating-IP: [61.135.152.194] From: yf-263 To: Eric Anderson In-Reply-To: <4282D173.1040805@centtech.com> References: <3EA9E48804411F41941A18169042D10401469D@omega3.ebaseweb.com> <1115864757.3938.19.camel@localhost.localdomain> <4282D173.1040805@centtech.com> Content-Type: text/plain; charset=UTF-8 Organization: Unix-driver.org Date: Thu, 12 May 2005 12:34:03 +0800 Message-Id: <1115872443.3924.13.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.0.4 (2.0.4-4) Content-Transfer-Encoding: 8bit cc: FreeBSD Clustering List Subject: Re: FreeBSD Clustering wishlist - Was: Introduction & RE: Clustering with Freebsd X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: yfyoufeng@263.net List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 May 2005 04:34:40 -0000 鍦 2005-05-11涓夌殑 22:45 -0500锛孍ric Anderson鍐欓亾锛 > Ok - I'm changing the subject here in an attempt to gather information. > > Here's my wishlist: > > FreeBSD have a 'native' clustered filesystem. This is different than > shared media (we already can do that over fiber channel, ggated, soon Yes, the clustered filesystem will not run on SAN, since that will give a high cost. > iscsi and AOE). This would allow multiple servers to access the same > data read/write - highly important for load balancing applications like > web servers, mail servers, and NFS servers. http://www.netapp.com/tech_library/3022.html <-- this article give some info about the small file operations among the web, mail, IM, netdisk, blog, etc. service. and that's our DFS targets at ;) > > Online growable filesystem. I know I can growfs a filesystem now, but > doing online while data is being used is *insanely* useful. Reiserfs > and Polyserve's FS (a clustered filesystem, not open-source) do this well. Yes, we also support that with our insanely mechanism. And you know in the current clustered fs, as GoogleFS, Lustre, etc. which can be built on online growfs. That's also our way to do it. > > FreeBSD's UFS2 made to do journaling. There's already someone working > on this. Good news. > > I believe the above mean that we need a distributed lock manager too, so > might as well add that to my wishlist. By the specific application & services, we can easily remove the distributed lock manager easily with upper layer way. You can read the GoogleFS paper to get some further info. > > Single filesystem limits set very high - 16TB would be a good minimum. The limits can be removed. > > Vinum/geom (?) made to allow added a couple more 'disks' - be it a real > scsi device, or another vinum device - to existing vinum's, so I can > extend my vinum stripe, raid, concat, etc to a larger volume size, > without worrying about which disk is where. I want to stripe mirrors of > raids, and raid striped mirrors of stripes. I know it sounds crazy, but > I really *do* have uses for all this. :) Yes, that's Lustre's way, and we also add a logical disk layer to support it. > > We currently pay lots of money every year (enough to pay an engineers > salary) for support and maintenance with Polyserve. They make a good Would you like to persuade to pay us for the developing ;) > > product (we need it for the clustered filesystem and NFS distributed > lock manager stuff) - I'd much rather see that go to FreeBSD. >From my current employer's calculation, what we do will save the cost of one zero, i.e. from 10 to 1 $, multi plus the 1,000+ users we have. So that's a really worthy project for the world. At last, any help & donate & contribute among the requirements & tech. domains are great appreciated ! > > Eric > > > -- yf-263 Unix-driver.org From owner-freebsd-cluster@FreeBSD.ORG Thu May 12 07:39:41 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 56B5C16A4CE for ; Thu, 12 May 2005 07:39:41 +0000 (GMT) Received: from iota.root-servers.ch (iota.root-servers.ch [193.41.193.195]) by mx1.FreeBSD.org (Postfix) with SMTP id 32C9743D62 for ; Thu, 12 May 2005 07:39:40 +0000 (GMT) (envelope-from gabriel_ambuehl@buz.ch) Received: (qmail 38205 invoked from network); 12 May 2005 07:39:38 -0000 Received: from 80-218-107-8.dclient.hispeed.ch (HELO ?10.2.2.101?) (80.218.107.8) by 0 with SMTP; 12 May 2005 07:39:38 -0000 Message-ID: <4283083E.9000005@buz.ch> Date: Thu, 12 May 2005 09:39:42 +0200 From: Gabriel Ambuehl User-Agent: Mozilla Thunderbird 1.0.2 (X11/20050404) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Matthew SWINBOURNE References: <5B5BBC864783B7439264DD5FF664F2A969F31D@jemima.soe.uq.edu.au> In-Reply-To: <5B5BBC864783B7439264DD5FF664F2A969F31D@jemima.soe.uq.edu.au> X-Enigmail-Version: 0.90.2.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit cc: FreeBSD Clustering List Subject: Re: Introduction & RE: Clustering with Freebsd X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 May 2005 07:39:41 -0000 Matthew SWINBOURNE wrote: >Hi All, > >Something to think about along these lines are TCP Off-Loader (TOE) >cards. If you are seriously looking at ATA over ethernet or something >similar on any production scale, then these cards are a life saver. >Reduce the CPU overhead of running storage via TCP/IP to almost >negligable. > > > > > As I understand, this is the whole point of ATA over Ethernet (vs iSCSI): it doesn't actually use TCP/IP, it runs on raw ethernet (and thus isn't routeable but that's probably more of a feature than a bug). From owner-freebsd-cluster@FreeBSD.ORG Thu May 12 15:53:49 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5C32516A4CE for ; Thu, 12 May 2005 15:53:49 +0000 (GMT) Received: from ms-dienst.rz.rwth-aachen.de (ms-2.rz.RWTH-Aachen.DE [134.130.3.131]) by mx1.FreeBSD.org (Postfix) with ESMTP id C0B0B43D76 for ; Thu, 12 May 2005 15:53:48 +0000 (GMT) (envelope-from chris@haakonia.hitnet.rwth-aachen.de) Received: from r220-1 (r220-1.rz.RWTH-Aachen.DE [134.130.3.31]) by ms-dienst.rz.rwth-aachen.de (iPlanet Messaging Server 5.2 Patch 2 (built Jul 14 2004)) with ESMTP id <0IGD008JJW5N72@ms-dienst.rz.rwth-aachen.de> for freeBSD-Cluster@freebsd.org; Thu, 12 May 2005 17:53:47 +0200 (MEST) Received: from relay.rwth-aachen.de ([134.130.3.1]) by r220-1 (MailMonitor for SMTP v1.2.2 ) ; Thu, 12 May 2005 17:53:46 +0200 (MEST) Received: from haakonia.hitnet.rwth-aachen.de (haakonia.hitnet.RWTH-Aachen.DE [137.226.181.92])j4CFrkxK001883; Thu, 12 May 2005 17:53:46 +0200 (MEST) Received: by haakonia.hitnet.rwth-aachen.de (Postfix, from userid 1001) id ED4E828442; Thu, 12 May 2005 17:53:45 +0200 (CEST) Date: Thu, 12 May 2005 17:53:45 +0200 From: Christian Brueffer In-reply-to: <3EA9E48804411F41941A18169042D10401469D@omega3.ebaseweb.com> To: Craig Lewis Message-id: <20050512155345.GA5298@unixpages.org> MIME-version: 1.0 Content-type: multipart/signed; boundary=DocE+STaALJfprDB; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-disposition: inline User-Agent: Mutt/1.5.6i X-Operating-System: FreeBSD 5.4-STABLE X-PGP-Key: http://people.FreeBSD.org/~brueffer/brueffer.key.asc X-PGP-Fingerprint: A5C8 2099 19FF AACA F41B B29B 6C76 178C A0ED 982D References: <3EA9E48804411F41941A18169042D10401469D@omega3.ebaseweb.com> cc: FreeBSD Clustering List Subject: Re: Introduction & RE: Clustering with Freebsd X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 May 2005 15:53:49 -0000 --DocE+STaALJfprDB Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, May 11, 2005 at 11:01:05AM -0500, Craig Lewis wrote: > Brilliant, this sounds like a great alternative to iscsi! ? So > potentially the 6.0 release might contain aoe? On roughly that time > scale? >=20 Considering that the feature freeze for 6.0 is not too far away, I'd say the chances are pretty slim. - Christian --=20 Christian Brueffer chris@unixpages.org brueffer@FreeBSD.org GPG Key: http://people.freebsd.org/~brueffer/brueffer.key.asc GPG Fingerprint: A5C8 2099 19FF AACA F41B B29B 6C76 178C A0ED 982D --DocE+STaALJfprDB Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.6 (FreeBSD) iD8DBQFCg3wJbHYXjKDtmC0RAmAtAJ4lsUyUZb/BXL1gliEeN6Ma4sWuOwCgsu0I F0Wo5fSOxu8AsmhZ1B/5I2s= =nFF+ -----END PGP SIGNATURE----- --DocE+STaALJfprDB-- From owner-freebsd-cluster@FreeBSD.ORG Thu May 12 23:10:53 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D196916A4CE for ; Thu, 12 May 2005 23:10:53 +0000 (GMT) Received: from mailhub2.uq.edu.au (mailhub2.uq.edu.au [130.102.149.128]) by mx1.FreeBSD.org (Postfix) with ESMTP id E5B4F43D7D for ; Thu, 12 May 2005 23:10:52 +0000 (GMT) (envelope-from m.swinbourne@uq.edu.au) Received: from smtp2.uq.edu.au (newsmtp2.uq.edu.au [130.102.149.129]) by mailhub2.uq.edu.au (8.13.3/8.13.3) with ESMTP id j4CNApYf009636 for ; Fri, 13 May 2005 09:10:51 +1000 (EST) Received: from jemima.soe.uq.edu.au (jemima.soe.uq.edu.au [130.102.4.196]) by smtp2.uq.edu.au (8.13.3/8.13.3) with ESMTP id j4CNAp8l009632 for ; Fri, 13 May 2005 09:10:51 +1000 (EST) Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Date: Fri, 13 May 2005 09:10:51 +1000 Message-ID: <5B5BBC864783B7439264DD5FF664F2A969F3F2@jemima.soe.uq.edu.au> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: Introduction & RE: Clustering with Freebsd Thread-Index: AcVWxceOq7v1CI07SyyeUmY+aiz/ywAgddTA From: "Matthew SWINBOURNE" To: "FreeBSD Clustering List" X-Scanned-By: MIMEDefang 2.51 on UQ Mailhub on 130.102.149.128 Subject: RE: Introduction & RE: Clustering with Freebsd X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 May 2005 23:10:53 -0000 Cool, Sorry, I wasn't aware of that. With iSCSI these cards definitely help, but it there's no stack used for the AOE stuff then cool. Am I right in thinking then, that most of the increase in load is interupt load for I/O on the adapter? Matt -----Original Message----- From: Gabriel Ambuehl [mailto:gabriel_ambuehl@buz.ch]=20 Sent: Thursday, 12 May 2005 5:40 PM To: Matthew SWINBOURNE Cc: FreeBSD Clustering List Subject: Re: Introduction & RE: Clustering with Freebsd Matthew SWINBOURNE wrote: >Hi All, > >Something to think about along these lines are TCP Off-Loader (TOE)=20 >cards. If you are seriously looking at ATA over ethernet or something=20 >similar on any production scale, then these cards are a life saver. >Reduce the CPU overhead of running storage via TCP/IP to almost=20 >negligable. > > > > =20 > As I understand, this is the whole point of ATA over Ethernet (vs iSCSI): it doesn't actually use TCP/IP, it runs on raw ethernet (and thus isn't routeable but that's probably more of a feature than a bug). From owner-freebsd-cluster@FreeBSD.ORG Fri May 13 12:12:08 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 208CC16A4CE for ; Fri, 13 May 2005 12:12:08 +0000 (GMT) Received: from torch.higis.ru (gate.higis.ru [81.195.168.122]) by mx1.FreeBSD.org (Postfix) with ESMTP id 09D6343D3F for ; Fri, 13 May 2005 12:12:07 +0000 (GMT) (envelope-from dimma@torch.higis.ru) Received: from localhost (localhost [127.0.0.1]) by torch.higis.ru (8.12.10/8.12.10) with ESMTP id j4DCC4xI048584 for ; Fri, 13 May 2005 16:12:05 +0400 (MSD) (envelope-from dimma@torch.higis.ru) Received: from torch.higis.ru ([127.0.0.1]) by localhost (torch.higis.ru [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 48305-05 for ; Fri, 13 May 2005 16:11:58 +0400 (MSD) Received: from torch.higis.ru (localhost [127.0.0.1]) by torch.higis.ru (8.12.10/8.12.10) with ESMTP id j4DCBviV048579 for ; Fri, 13 May 2005 16:11:57 +0400 (MSD) (envelope-from dimma@torch.higis.ru) Received: (from dimma@localhost) by torch.higis.ru (8.12.10/8.12.10/Submit) id j4DCBvwc048578 for freebsd-cluster@freebsd.org; Fri, 13 May 2005 16:11:57 +0400 (MSD) (envelope-from dimma) Date: Fri, 13 May 2005 16:11:57 +0400 From: Dmitriy Kirhlarov To: freebsd-cluster@freebsd.org Message-ID: <20050513121156.GD23893@torch.higis.ru> References: <20050429183405.M60641@bmyster.com> <20050511121115.GA2731@torch.higis.ru> <4281FF74.6060005@centtech.com> <20050511140657.GB2731@torch.higis.ru> <4282123E.6080909@centtech.com> Mime-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <4282123E.6080909@centtech.com> User-Agent: Mutt/1.5.6i X-Virus-Scanned: by amavisd-new at higis.ru Subject: Re: Clustering with Freebsd X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 May 2005 12:12:08 -0000 Hi Eric! On Wed, 11 May 2005, Eric Anderson wrote: > Dmitriy Kirhlarov wrote: > >Hi Eric! > > > >On Wed, 11 May 2005, Eric Anderson wrote: > > > > > >>Dmitriy Kirhlarov wrote: > >> > >>>Hi Brent! > >>> > >>>On Fri, 29 Apr 2005, Brent wrote: > >>> > >>> > >>> > >>>>I have a webhosting business all runnning on various Freebsd i386 > >>>>boxes. I was wondering if there are any good howto's on doing this ? > >>>>The services id like to cluster are, > >>>>apache > >>>>sendmail > >>>>bind 9.2.3 > >>>>UW imap > >>> > >>> > >>>replace to cyrus imapd. > >>> > >>>read about gmirror, carp, ggated. > >> > >>How would gmirror and ggated help? > > > > > >Replicate data between two hosts, but it not very fine solution -- cached > >data not replicate to second node, when first node die. > > And it would only be for failover, not for load distribution/balancing. carp can be used as solution, as I understand. By. Dmitriy From owner-freebsd-cluster@FreeBSD.ORG Fri May 13 16:20:57 2005 Return-Path: Delivered-To: freebsd-cluster@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1364A16A4CE for ; Fri, 13 May 2005 16:20:57 +0000 (GMT) Received: from ms-smtp-04-eri0.southeast.rr.com (ms-smtp-04-lbl.southeast.rr.com [24.25.9.103]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5EF6B43D79 for ; Fri, 13 May 2005 16:20:56 +0000 (GMT) (envelope-from jjhernan@nc.rr.com) Received: from ms-mss-01-ce0-1 ([10.10.5.78])j4DGKoL4006481 for ; Fri, 13 May 2005 12:20:53 -0400 (EDT) Received: from southeast.rr.com (localhost [127.0.0.1]) by ms-mss-01.southeast.rr.com (iPlanet Messaging Server 5.2 HotFix 2.04 (built Feb 8 2005)) with ESMTP id <0IGF0024AS2Q5O@ms-mss-01.southeast.rr.com> for freebsd-cluster@freebsd.org; Fri, 13 May 2005 12:20:50 -0400 (EDT) Received: from [10.10.1.25] (Forwarded-For: [63.239.87.162]) by ms-mss-01.southeast.rr.com (mshttpd); Fri, 13 May 2005 12:20:50 -0400 Date: Fri, 13 May 2005 12:20:50 -0400 From: jjhernan@nc.rr.com To: freebsd-cluster@freebsd.org Message-id: <8cffa78cacd3.8cacd38cffa7@southeast.rr.com> MIME-version: 1.0 X-Mailer: iPlanet Messenger Express 5.2 HotFix 2.04 (built Feb 8 2005) Content-type: text/plain; charset=us-ascii Content-language: en Content-transfer-encoding: 7BIT Content-disposition: inline X-Accept-Language: en Priority: normal X-Virus-Scanned: Symantec AntiVirus Scan Engine Subject: Re: Clustering with Freebsd X-BeenThere: freebsd-cluster@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Clustering FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 May 2005 16:20:57 -0000 As a naive individual... (maybe not a good way to start, he-he!) I think about how useful something like this will be for me to utilize the HDrives from all my FreeBSD computers. Cool! It also makes wonder why stop there? If we can do ATA commands over Ethernet then we can do others, like PCI, USB, etc. Here is an interesting link as I google for something like this: http://www.hacker-technology.com/3801.html?*session*id*key*=*session*id*val* Again, naively, we will be turning a new leaf in the computing industry when we can use ethernet as a computer bus to any device/peripheral available in a motherboard. If this happens in the near future, I should be able to configure a 6-cpu computer, with loads of memory, and hard drives in no time. Looks like a new FreeBSD project if one is not on the way already. Jorge ----- Original Message ----- From: Dmitriy Kirhlarov Date: Friday, May 13, 2005 8:11 am Subject: Re: Clustering with Freebsd > Hi Eric! > > On Wed, 11 May 2005, Eric Anderson wrote: > > > Dmitriy Kirhlarov wrote: > > >Hi Eric! > > > > > >On Wed, 11 May 2005, Eric Anderson wrote: > > > > > > > > >>Dmitriy Kirhlarov wrote: > > >> > > >>>Hi Brent! > > >>> > > >>>On Fri, 29 Apr 2005, Brent wrote: > > >>> > > >>> > > >>> > > >>>>I have a webhosting business all runnning on various Freebsd > i386 > > >>>>boxes. I was wondering if there are any good howto's on doing > this ? > > >>>>The services id like to cluster are, > > >>>>apache > > >>>>sendmail > > >>>>bind 9.2.3 > > >>>>UW imap > > >>> > > >>> > > >>>replace to cyrus imapd. > > >>> > > >>>read about gmirror, carp, ggated. > > >> > > >>How would gmirror and ggated help? > > > > > > > > >Replicate data between two hosts, but it not very fine solution - > - cached > > >data not replicate to second node, when first node die. > > > > And it would only be for failover, not for load > distribution/balancing. > carp can be used as solution, as I understand. > > By. > Dmitriy > _______________________________________________ > freebsd-cluster@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-cluster > To unsubscribe, send any mail to "freebsd-cluster- > unsubscribe@freebsd.org"