From owner-freebsd-current@FreeBSD.ORG Fri May 27 02:50:24 2005 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 806CF16A41C for ; Fri, 27 May 2005 02:50:24 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from mh2.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 33EDA43D49 for ; Fri, 27 May 2005 02:50:23 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [192.168.42.25] ([192.168.42.25]) by mh2.centtech.com (8.13.1/8.13.1) with ESMTP id j4R2oBwX059464; Thu, 26 May 2005 21:50:22 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <42968AD4.3020603@centtech.com> Date: Thu, 26 May 2005 21:49:56 -0500 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.7) Gecko/20050504 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Scott Long References: <4295D51F.50106@centtech.com> <429606D9.6080602@cs.tu-berlin.de> <42960ACB.7090801@cs.tu-berlin.de> <42960CFE.4060307@centtech.com> <42960F8F.2050109@samsco.org> <42961195.30608@centtech.com> <429613FB.80100@samsco.org> In-Reply-To: <429613FB.80100@samsco.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Current Subject: Re: Disable read/write caching to disk? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 May 2005 02:50:24 -0000 Scott Long wrote: > Eric Anderson wrote: > >> Scott Long wrote: >> >>> Eric Anderson wrote: >>> >>>> Bjoern Koenig wrote: >>>> >>>>> Bjoern Koenig wrote: >>>>> >>>>>> Eric Anderson wrote: >>>>>> >>>>>>> Is it possible to disable all read and write caching to a disk? >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> You can disable write the cache by adding the line >>>>>> >>>>>> hw.ata.wc="0" >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> I assumed that you use ATA. If you use SCSI devices then read at >>>>> least the manpages da(4) and camcontrol(8). >>>> >>>> >>>> >>>> >>>> >>>> Thanks.. I've just read (quickly) both man pages. It seems as >>>> though you are suggesting disabling the physical disk caching, which >>>> should not make a difference in my case. The disk would report >>>> whatever it needs to report to either host, and those should be in >>>> sync. >>>> >>>> When I mount the filesystem on host B ro, it shows me the filesystem >>>> as of the time that I mounted it ro. Any subsequent changes on host >>>> A (which has it mounted rw) are not seem on host B unless I unmount >>>> and mount again on host B. This seems like a FreeBSD feature and >>>> not a general scsi feature. >>>> >>>> Eric >>>> >>>> >>>> >>>> >>> >>> You simply cannot disable OS caching in FreeBSD. It's a fundamental >>> part of the block I/O and VM layers. There are filesystems like GFS >>> that deal with the issue of directly connecting more than one computer >>> to a disk or set of disks, and there are distributed filesystems like >>> AFS and Coda that deal with making the storage on multiple computers >>> appear as a single network filesystem. Unfortunately, no port of GFS >>> has been done yet, and I estimate that such a port would take 4-6 >>> months. >> >> >> >> Thanks Scott. I know of GFS, and would *love* to see it ported to >> FreeBSD. I wonder if there is a group of developers that would be >> willing to do that? > > > I'd love to do it sometime. Some diligence would have to be done on the > license and possible patents, though. I'm not sure what RedHat's stance > is on sharing in this regard. If you are serious about working on it, I could look into the RedHat side of things. This is something I really think is important for FreeBSD (in the future). If GFS isn't right, maybe another clustering shared file system, or a Global UFS (I don't even know if that's possible). >> I understand what I am doing is 'illegal', but I'm wondering why the >> ro mount only sees the changes from the time of ro mount. Mounting rw >> also shows the same thing. >> >> Do you think it's a caching issue, or something with UFS that makes it >> work this way? >> > > Even on a read-only mount, reads are cached. Cached blocks are only > evicted when there is memory pressure or when the OS specifically > invalidates them, and it's rather unpredictable when this will happen. I see. So most likely the lack of updates being visible on host B is due to read caches. >> I'm in no way advocating doing this, nor am I saying 'it should work' >> - I'm trying to learn more, understand it, and maybe use it as a >> failover mechanism. > > > The only thing that is moderately workable is to have all client > machines mount the FS read-only, and have none of them do any writing > to it. > >> >> Does anyone know the real dangers of forcing an unclean UFS filesystem >> mounted rw and skipping the fsck? > > > Assuming that the previous shutdown allowed all cached metadata in the > disk to get to the platters in the proper order (which is not terribly > true with ATA), the main inconsistency that you'll have is that deleted > files might still have allocated inodes and thus will consume space on > the filesystem even though they aren't accessible. This is the premise > that background fsck operates on, btw. > > However, there is a real possibility for there being other > inconsistencies that are not safe to run with. So it sounds dangerous, but not disastrous.. Sounds like soft-updates would help this alot, so I'll turn them back on for this filesystem (I typically do use it). At a minimum, it would be awesome to even have a way to do one host rw and several doing ro. Think of the case of a web server farm, where it's nearly all reads. Thanks for the details and information! Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology A lost ounce of gold may be found, a lost moment of time never. ------------------------------------------------------------------------