From owner-freebsd-fs@FreeBSD.ORG Mon Jun 20 01:37:29 2005 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D3C4816A41C for ; Mon, 20 Jun 2005 01:37:29 +0000 (GMT) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.FreeBSD.org (Postfix) with ESMTP id 88D3B43D49 for ; Mon, 20 Jun 2005 01:37:29 +0000 (GMT) (envelope-from scottl@samsco.org) Received: from [10.11.36.231] ([145.253.106.162]) (authenticated bits=0) by pooker.samsco.org (8.13.3/8.13.3) with ESMTP id j5K1fCos049893; Sun, 19 Jun 2005 19:41:14 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <42B61DCD.4030307@samsco.org> Date: Sun, 19 Jun 2005 19:37:17 -0600 From: Scott Long User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.8) Gecko/20050615 X-Accept-Language: en-us, en MIME-Version: 1.0 To: emartinez@crockettint.com References: <20050617211305.6E04A33F81@mxc1.crockettint.com> In-Reply-To: <20050617211305.6E04A33F81@mxc1.crockettint.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.0 required=3.8 tests=none autolearn=failed version=3.0.2 X-Spam-Checker-Version: SpamAssassin 3.0.2 (2004-11-16) on pooker.samsco.org Cc: freebsd-fs@freebsd.org Subject: Re: UFS2+Softupdates Corruption Regardless on Seven various systems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2005 01:37:30 -0000 Edgar Martinez wrote: > All, > > > > I have a network of FBSD boxen running 5.3 w/ 2x PATA WD1200JB Drives and a > Promise Fastrack TX2 controller in mirror. The systems mainly just pass > internet traffic and rarely ever touch the disks. After running for a few > weeks -> months.the disks become corrupted forcing a manual fsck from single > user mode. And since the system is thousands of miles away, it can become > painful to walk someone with a language barrier thru that. > > > > Question is WHY does this occur? > > How can you avoid this? > > What can you do to remotely fix the issue? > > Any proactive maintenance I need to be doing? > > Did I mention I would like to know WHY? > This certainly sounds like a bug, and is not something that people normally see. When and how do you notice the corruption? Does it have a particular pattern? Would it be possible to try a different brand of disk controller in order to rule out the driver being buggy? Scott From owner-freebsd-fs@FreeBSD.ORG Tue Jun 21 14:36:02 2005 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0AB1316A41C for ; Tue, 21 Jun 2005 14:36:02 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from mh2.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id A56A443D55 for ; Tue, 21 Jun 2005 14:36:01 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh2.centtech.com (8.13.1/8.13.1) with ESMTP id j5LEa0V3058425 for ; Tue, 21 Jun 2005 09:36:00 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <42B825CC.806@centtech.com> Date: Tue, 21 Jun 2005 09:35:56 -0500 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.8) Gecko/20050603 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: Cluster Filesystem for FreeBSD - any interest? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jun 2005 14:36:02 -0000 This is something I've brought up before on other lists, but I'm curious if anyone is interested in developing a BSD licensed clustered filesystem for FreeBSD (and anyone else)? If there are enough developers, we can get the details ironed out and start real software for this. What is everyone's thoughts on this? Who's interested? Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology A lost ounce of gold may be found, a lost moment of time never. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Tue Jun 21 14:45:08 2005 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E3FC416A41F for ; Tue, 21 Jun 2005 14:45:08 +0000 (GMT) (envelope-from rees@citi.umich.edu) Received: from citi.umich.edu (citi.umich.edu [141.211.133.111]) by mx1.FreeBSD.org (Postfix) with ESMTP id C72D143D53 for ; Tue, 21 Jun 2005 14:45:08 +0000 (GMT) (envelope-from rees@citi.umich.edu) Received: from citi.umich.edu (dumaguete.citi.umich.edu [141.211.133.51]) by citi.umich.edu (Postfix) with ESMTP id 538A01BBAA for ; Tue, 21 Jun 2005 10:45:08 -0400 (EDT) To: freebsd-fs@freebsd.org From: Jim Rees In-Reply-To: Eric Anderson, Tue, 21 Jun 2005 09:35:56 CDT Date: Tue, 21 Jun 2005 10:45:08 -0400 Sender: rees@citi.umich.edu Message-Id: <20050621144508.538A01BBAA@citi.umich.edu> Subject: Re: Cluster Filesystem for FreeBSD - any interest? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jun 2005 14:45:09 -0000 Did you have something specific in mind? One possibility would be pNFS. We've already got a start on NFSv4. From owner-freebsd-fs@FreeBSD.ORG Tue Jun 21 15:21:28 2005 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4F2E916A41C for ; Tue, 21 Jun 2005 15:21:28 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from mh2.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0E4AE43D48 for ; Tue, 21 Jun 2005 15:21:27 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh2.centtech.com (8.13.1/8.13.1) with ESMTP id j5LFLRnO059061; Tue, 21 Jun 2005 10:21:27 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <42B83073.3080606@centtech.com> Date: Tue, 21 Jun 2005 10:21:23 -0500 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.8) Gecko/20050603 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jim Rees References: <20050621144508.538A01BBAA@citi.umich.edu> In-Reply-To: <20050621144508.538A01BBAA@citi.umich.edu> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: Cluster Filesystem for FreeBSD - any interest? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jun 2005 15:21:28 -0000 Jim Rees wrote: > Did you have something specific in mind? One possibility would be pNFS. > We've already got a start on NFSv4. For high performance NFS serving mostly. So other NFS versions don't help me too much. Shared storage is great, but with only one machine able to serve the data on it, it becomes much less useful. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology A lost ounce of gold may be found, a lost moment of time never. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Wed Jun 22 01:44:23 2005 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9518B16A41C for ; Wed, 22 Jun 2005 01:44:23 +0000 (GMT) (envelope-from yfyoufeng@263.net) Received: from smtp.263.net (mx01.263.net.cn [211.150.96.22]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4F44243D49 for ; Wed, 22 Jun 2005 01:44:22 +0000 (GMT) (envelope-from yfyoufeng@263.net) Received: from [10.217.12.184] (localhost [127.0.0.1]) by smtp.263.net (Postfix) with ESMTP id 15EACC3684; Wed, 22 Jun 2005 09:44:12 +0800 (CST) (envelope-from yfyoufeng@263.net) Received: from [10.217.12.184] (unknown [61.135.152.194]) by antispam-2 (Coremail:263(050316)) with SMTP id M+FIAGvCuEI+F5jC.1 for ; Wed, 22 Jun 2005 09:44:12 +0800 (CST) X-TEBIE-Originating-IP: [61.135.152.194] From: yf-263 To: Eric Anderson In-Reply-To: <42B825CC.806@centtech.com> References: <42B825CC.806@centtech.com> Content-Type: text/plain; charset=UTF-8 Organization: Unix-driver.org Date: Wed, 22 Jun 2005 09:43:48 +0800 Message-Id: <1119404628.2297.4.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.2.2 (2.2.2-5) Content-Transfer-Encoding: 8bit Cc: freebsd-fs@freebsd.org Subject: Re: Cluster Filesystem for FreeBSD - any interest? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: yfyoufeng@263.net List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2005 01:44:23 -0000 在 2005-06-21二的 09:35 -0500，Eric Anderson写道： > This is something I've brought up before on other lists, but I'm curious > if anyone is interested in developing a BSD licensed clustered > filesystem for FreeBSD (and anyone else)? > > If there are enough developers, we can get the details ironed out and > start real software for this. > > What is everyone's thoughts on this? Who's interested? I have the great interests on it. But as you see in http://www.clusterfs.com/lustre_is_oss.html #In what way does Lustre have "open-development"? "Our experience in developing distributed file systems for more than 10 years has been that the open source community contributes relatively few bug fixes." So the work need a strong support from its experiences. Also I have opened a discuss on following link http://bbs.chinaunix.net/forum/viewtopic.php?t=544517 > > Eric > > > -- yf-263 Unix-driver.org From owner-freebsd-fs@FreeBSD.ORG Wed Jun 22 02:37:30 2005 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6E65E16A41C for ; Wed, 22 Jun 2005 02:37:30 +0000 (GMT) (envelope-from afields@afields.ca) Received: from afields.ca (afields.ca [216.194.67.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 453EA43D49 for ; Wed, 22 Jun 2005 02:37:30 +0000 (GMT) (envelope-from afields@afields.ca) Received: from afields.ca (localhost.afields.ca [127.0.0.1]) by afields.ca (8.12.11/8.12.11) with ESMTP id j5M2bSef020075; Tue, 21 Jun 2005 22:37:28 -0400 (EDT) (envelope-from afields@afields.ca) Received: (from afields@localhost) by afields.ca (8.12.11/8.12.11/Submit) id j5M2bSn6020074; Tue, 21 Jun 2005 22:37:28 -0400 (EDT) (envelope-from afields) Date: Tue, 21 Jun 2005 22:37:27 -0400 From: Allan Fields To: Eric Anderson Message-ID: <20050622023727.GA19408@afields.ca> References: <42B825CC.806@centtech.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <42B825CC.806@centtech.com> User-Agent: Mutt/1.4i Cc: freebsd-fs@freebsd.org Subject: Re: Cluster Filesystem for FreeBSD - any interest? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2005 02:37:30 -0000 On Tue, Jun 21, 2005 at 09:35:56AM -0500, Eric Anderson wrote: > This is something I've brought up before on other lists, but I'm curious > if anyone is interested in developing a BSD licensed clustered > filesystem for FreeBSD (and anyone else)? A few questions: Could this be done as a stackable file system (vnode layer distributed file system) or did you have something else in mind (i.e. specifically a full implementation of a network filesystem including storage layer)? Why not a port of an existing network filesystem say from Linux? (A BSD rewrite could be done, if the code was GPLed.) Would cross-platform capabilities make sense? How do you see this comparing to device-level solutions? I know the argument can be made to implement file systems/storage abstractions at multiple layers, but I thought I might ask. The other thing is there a wealth of filesystem papers out there, any in specific caught your eye? > Eric > > > > -- > ------------------------------------------------------------------------ > Eric Anderson Sr. Systems Administrator Centaur Technology > A lost ounce of gold may be found, a lost moment of time never. > ------------------------------------------------------------------------ -- Allan Fields From owner-freebsd-fs@FreeBSD.ORG Wed Jun 22 11:53:36 2005 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CD08916A41C for ; Wed, 22 Jun 2005 11:53:36 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8B99943D4C for ; Wed, 22 Jun 2005 11:53:36 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id j5MBrQik089969; Wed, 22 Jun 2005 06:53:31 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <42B95132.6090907@centtech.com> Date: Wed, 22 Jun 2005 06:53:22 -0500 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.8) Gecko/20050603 X-Accept-Language: en-us, en MIME-Version: 1.0 To: yfyoufeng@263.net References: <42B825CC.806@centtech.com> <1119404628.2297.4.camel@localhost.localdomain> In-Reply-To: <1119404628.2297.4.camel@localhost.localdomain> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Virus-Scanned: ClamAV 0.82/950/Tue Jun 21 19:48:34 2005 on mh1.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: Cluster Filesystem for FreeBSD - any interest? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2005 11:53:36 -0000 yf-263 wrote: > =E5=9C=A8 2005-06-21=E4=BA=8C=E7=9A=84 09:35 -0500=EF=BC=8CEric Anderso= n=E5=86=99=E9=81=93=EF=BC=9A >=20 >>This is something I've brought up before on other lists, but I'm curiou= s=20 >>if anyone is interested in developing a BSD licensed clustered=20 >>filesystem for FreeBSD (and anyone else)? >> >>If there are enough developers, we can get the details ironed out and=20 >>start real software for this. >> >>What is everyone's thoughts on this? Who's interested? >=20 >=20 > I have the great interests on it. >=20 > But as you see in http://www.clusterfs.com/lustre_is_oss.html #In what= > way does Lustre have "open-development"? >=20 > "Our experience in developing distributed file systems for more than 10= > years has been that the open source community contributes relatively fe= w > bug fixes." >=20 > So the work need a strong support from its experiences. >=20 > Also I have opened a discuss on following link > http://bbs.chinaunix.net/forum/viewtopic.php?t=3D544517 Unfortunately, I can't read chinese. I understand that many of those projects fail because of a lack of bug=20 reporting, but I believe that is partly due to the userbase of the=20 current cluster filesystems (being nearly all linux users, most of them=20 in a lab environment, etc), and also the lack of solid docs. Lustre is = a very interesting package, but it doesn't work on FreeBSD. I could use = GFS, or luster, or another one with Linux, but I want to use FreeBSD. Now, to be clear, I'm talking about a clustered filesystem, not a=20 distributed filesystem. Eric --=20 ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology A lost ounce of gold may be found, a lost moment of time never. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Wed Jun 22 12:09:42 2005 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2F95A16A41F for ; Wed, 22 Jun 2005 12:09:42 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id D439443D58 for ; Wed, 22 Jun 2005 12:09:41 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id j5MC9c0K090076; Wed, 22 Jun 2005 07:09:38 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <42B954FE.2070406@centtech.com> Date: Wed, 22 Jun 2005 07:09:34 -0500 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.8) Gecko/20050603 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Allan Fields References: <42B825CC.806@centtech.com> <20050622023727.GA19408@afields.ca> In-Reply-To: <20050622023727.GA19408@afields.ca> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.82/950/Tue Jun 21 19:48:34 2005 on mh1.centtech.com X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org Subject: Re: Cluster Filesystem for FreeBSD - any interest? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2005 12:09:42 -0000 Allan Fields wrote: > On Tue, Jun 21, 2005 at 09:35:56AM -0500, Eric Anderson wrote: > >>This is something I've brought up before on other lists, but I'm curious >>if anyone is interested in developing a BSD licensed clustered >>filesystem for FreeBSD (and anyone else)? > > > A few questions: > > Could this be done as a stackable file system (vnode layer distributed > file system) or did you have something else in mind (i.e. specifically > a full implementation of a network filesystem including storage > layer)? Hmm. I'm not sure if it can or not. I'll try to explain what I'm dreaming of. I currently have about 1000 clients needing access to the same pools of data (read/write) all the time. The data changes constantly. There is a lot of this data. We use NFS currently. FreeBSD is *very* fast and stable at serving NFS data. The problem is, that even though it is very fast and stable, I still cannot pump out enough bits fast enough with one machine, and if that one machine fails (hardware problems, etc), then all my machines are hung waiting for me to bring it back online. So, what I would love to have, is this kind of setup: shared media storage (fibre channel SAN, iscsi, or something like ggated possibly), connected up to a cluster of hosts running FreeBSD. Each FreeBSD server has access to the logical disks, same partitions, and can mount them all r/w. Now, I can kind of do this now, however there are obviously some issues with this currently. I want all machines in this cluster to be able to serve the data via NFS (or http, or anything else for that matter really - if you can make NFS work, anything will pretty much work) simultaneously from the same partitions, and see writes immediately as the other hosts in the cluster commit them. I currently have a solution just like this for Linux - Polyserve (http://www.polyserve.com) has a clustered filesystem for linux, that works very well. I've even tried to convince them to port it to FreeBSD, but it falls on deaf ears, so it's time to make our own. > Why not a port of an existing network filesystem say from Linux? > (A BSD rewrite could be done, if the code was GPLed.) Would > cross-platform capabilities make sense? That would work fine I'm sure - but I have found some similar threads in the past that claim it would be just as hard and time consuming to port one as it would be to create one from scratch. Cross platform capabilities would be great, but I'm mostly interested in getting FreeBSD into this arena (as it will soon be an extremely important one to be in). > How do you see this comparing to device-level solutions? I know > the argument can be made to implement file systems/storage > abstractions at multiple layers, but I thought I might ask. I'm not sure of a device level solution that does this. I think the OS has to know to commit the meta-data to a journal, or otherwise let the other machines know about locking, etc, in order for this to work. > The other thing is there a wealth of filesystem papers out there, > any in specific caught your eye? No - can you point me to some? I'll be honest here - I'm not a code developer. I would love to learn some C here, and 'just do it', but filesystems aren't exactly simple, so I'm looking for a group of people that would love to code up something amazing like this - I'll support the developers and hopefully learn something in the process. My goal personally would be to do anything I could to make the developers work most productively, and do testing. I can probably provide equipment, and a good testbed for it. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology A lost ounce of gold may be found, a lost moment of time never. ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Wed Jun 22 14:45:45 2005 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EA01016A41C for ; Wed, 22 Jun 2005 14:45:45 +0000 (GMT) (envelope-from bakul@bitblocks.com) Received: from gate.bitblocks.com (bitblocks.com [209.204.185.216]) by mx1.FreeBSD.org (Postfix) with ESMTP id A55AD43D4C for ; Wed, 22 Jun 2005 14:45:45 +0000 (GMT) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (localhost [127.0.0.1]) by gate.bitblocks.com (8.13.3/8.13.1) with ESMTP id j5MEje5P097719; Wed, 22 Jun 2005 07:45:40 -0700 (PDT) (envelope-from bakul@bitblocks.com) Message-Id: <200506221445.j5MEje5P097719@gate.bitblocks.com> To: Eric Anderson In-reply-to: Your message of "Wed, 22 Jun 2005 07:09:34 CDT." <42B954FE.2070406@centtech.com> Date: Wed, 22 Jun 2005 07:45:40 -0700 From: Bakul Shah Cc: freebsd-fs@freebsd.org Subject: Re: Cluster Filesystem for FreeBSD - any interest? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2005 14:45:46 -0000 > Hmm. I'm not sure if it can or not. I'll try to explain what I'm > dreaming of. I currently have about 1000 clients needing access to the > same pools of data (read/write) all the time. The data changes > constantly. There is a lot of this data. We use NFS currently. Sounds like you want SGI's clustered xfs.... > I'll be honest here - I'm not a code developer. I would love to learn > some C here, and 'just do it', but filesystems aren't exactly simple, so > I'm looking for a group of people that would love to code up something > amazing like this - I'll support the developers and hopefully learn > something in the process. My goal personally would be to do anything I > could to make the developers work most productively, and do testing. I > can probably provide equipment, and a good testbed for it. If you are not a seasoned programmer in _some_ language, this will not be easy at all. One suggestion is to develop an abstract model of what a CFS is. Coming up with a clear detailed precise specification is not an easy task either but it has to be done and if you can do it, it will be immensely helpful all around. You will truly understand what you are doing, you have a basis for evaluating design choices, you will have made choices before writing any code, you can write test cases, writing code is far easier etc. etc. Google for clustered filesystems. The citeseer site has some papers as well. A couple FS specific suggestions: - perhaps clustering can be built on top of existing filesystems. Each machine's local filesystem is considered a cache and you use some sort of cache coherency protocol. That way you don't have to deal with filesystem allocation and layout issues. - a network wide stable storage `disk' may be easier to do given GEOM. There are atleast N copies of each data block. Data may be cached locally at any site but writing data is done as a distributed transaction. So again cache coherency is needed. A network RAID if you will! But again, let me stress that one must have a clear *model* of the problem being solved. Getting distributed programs right is very hard even at an abstract model level. Debugging a distributed program that doesn't have a clear model is, well, for masochists (nothing against them -- I bet even they'd rather get their pain some other way:-) From owner-freebsd-fs@FreeBSD.ORG Fri Jun 24 18:16:55 2005 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A588116A41C for ; Fri, 24 Jun 2005 18:16:55 +0000 (GMT) (envelope-from emartinez@crockettint.com) Received: from mxc1.crockettint.com (chameleon1.crockettint.com [206.224.72.162]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7A44F43D1D for ; Fri, 24 Jun 2005 18:16:55 +0000 (GMT) (envelope-from emartinez@crockettint.com) Received: from localhost (localhost.crockettint.com [127.0.0.1]) by mxc1.crockettint.com (Postfix) with ESMTP id 1369B33F63; Fri, 24 Jun 2005 13:16:54 -0500 (CDT) Received: from mxc1.crockettint.com ([127.0.0.1]) by localhost (mxc1.crockettint.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 20243-06; Fri, 24 Jun 2005 13:16:51 -0500 (CDT) Received: from memnoch (unknown [12.44.178.253]) by mxc1.crockettint.com (Postfix) with ESMTP id 7306A33F5F; Fri, 24 Jun 2005 13:16:51 -0500 (CDT) From: "Edgar Martinez" To: "'Scott Long'" Date: Fri, 24 Jun 2005 13:16:32 -0500 Organization: Crockett International MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.6353 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2527 In-Reply-To: <42B61DCD.4030307@samsco.org> Thread-Index: AcV1OJ87POHBOngqT2OFUxxIBzpcMQDsCa8Q Message-Id: <20050624181651.7306A33F5F@mxc1.crockettint.com> X-Virus-Scanned: amavisd-new at crockettint.com Cc: freebsd-fs@freebsd.org Subject: RE: UFS2+Softupdates Corruption Regardless on Seven various systems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: emartinez@crockettint.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Jun 2005 18:16:55 -0000 Sorry for the delay, I am currently in clovis at a training, but will respond soon. Thanks for the response! -----Original Message----- From: Scott Long [mailto:scottl@samsco.org] Sent: Sunday, June 19, 2005 8:37 PM To: emartinez@crockettint.com Cc: freebsd-fs@freebsd.org Subject: Re: UFS2+Softupdates Corruption Regardless on Seven various systems Edgar Martinez wrote: > All, > > > > I have a network of FBSD boxen running 5.3 w/ 2x PATA WD1200JB Drives and a > Promise Fastrack TX2 controller in mirror. The systems mainly just pass > internet traffic and rarely ever touch the disks. After running for a few > weeks -> months.the disks become corrupted forcing a manual fsck from single > user mode. And since the system is thousands of miles away, it can become > painful to walk someone with a language barrier thru that. > > > > Question is WHY does this occur? > > How can you avoid this? > > What can you do to remotely fix the issue? > > Any proactive maintenance I need to be doing? > > Did I mention I would like to know WHY? > This certainly sounds like a bug, and is not something that people normally see. When and how do you notice the corruption? Does it have a particular pattern? Would it be possible to try a different brand of disk controller in order to rule out the driver being buggy? Scott