From owner-freebsd-net@FreeBSD.ORG Mon Aug 14 15:55:53 2006 Return-Path: X-Original-To: freebsd-net@FreeBSD.ORG Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B3F6216A4DD for ; Mon, 14 Aug 2006 15:55:53 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (lurza.secnetix.de [83.120.8.8]) by mx1.FreeBSD.org (Postfix) with ESMTP id 498F343D6D for ; Mon, 14 Aug 2006 15:55:51 +0000 (GMT) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (mdazwt@localhost [127.0.0.1]) by lurza.secnetix.de (8.13.4/8.13.4) with ESMTP id k7EFtiUu092648 for ; Mon, 14 Aug 2006 17:55:49 +0200 (CEST) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.13.4/8.13.1/Submit) id k7EFthXw092647; Mon, 14 Aug 2006 17:55:43 +0200 (CEST) (envelope-from olli) Date: Mon, 14 Aug 2006 17:55:43 +0200 (CEST) Message-Id: <200608141555.k7EFthXw092647@lurza.secnetix.de> From: Oliver Fromme To: freebsd-net@FreeBSD.ORG In-Reply-To: <44E07636.4090403@fsn.hu> X-Newsgroups: list.freebsd-net User-Agent: tin/1.8.0-20051224 ("Ronay") (UNIX) (FreeBSD/4.11-STABLE (i386)) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.1.2 (lurza.secnetix.de [127.0.0.1]); Mon, 14 Aug 2006 17:55:49 +0200 (CEST) Cc: Subject: Re: Redundant/failover NFS servers - stale NFS file handle X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: freebsd-net@FreeBSD.ORG List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Aug 2006 15:55:53 -0000 Attila Nagy wrote: > I would like to run diskless clients (they are actually servers) from > FreeBSD, but I don't like having a SPoF at the NFS server level and > don't want to use expensive out of the box solutions, like a NAS with a > SAN behind it. We use NetApp Filer clusters (NAS) for that purpose. They aren't cheap, but they work very well. > So in theory, having two FreeBSD boxes, both with the NFS service on a > CARP-based virtual IP would perfectly fit my needs. > > The only problem is that NFS encodes some information in the > filehandles, so when I'm doing a failover with the NFS clients (bringing > the carp interface down on the master server), I get "Stale NFS file > handle". That's to be expected. NFS file handles are based on the inode number. That means if you want to have a fail-over that's transparent for the client, your NFS servers would need to have the same inode numebrs for their files. Normally, the only way to achieve that is to duplicate the file system from the master to the slaves using dd(1). However, dd(1) has several drawbacks: First, it takes a long time, because it copies everything, including blocks that aren't allocated at all. Second (and most important), if you copy a live file system (i.e. one that is mounted read+write) with dd(1), the copy won't be in a consistent state and will at the very least need to be fsck(8)ed (and If you're unlucky, even fsck(8) won't be able to fix it). One solution to the latter problem might be to take a snapshot of the filesystem (see mksnap_ffs(8)) and the copy that snapshot (which is read-only) with dd(1). If the contents of the NFS exported file system don't change very often, that might be a workable solution. There's another possibility, but I haven't tried it for myself, so it's just theory. :-) You can try to put geom_mirror (see gmirror(8)) on top of geom_gate (see ggated(8), ggatec(8)). Then you will have a RAID1 with one component local and the other component remote. However, I think it only works reliably in read-only mode. > As for the client side, Solaris has the capability of doing NFS client > failover (reported to have some problems, but for now I would have only > FreeBSD clients), and AMD has multiple server support, but I don't know > how does that work with FreeBSD diskless boots yet. (root FS on NFS) I don't know if this is an option for you, but you can also put a minimal root file system into the kernel (md file system), just sufficient to get networking + AMD running, and mount everything else via NFS. Another possibility is to put a CompactFlash card to boot from into the machines (could be read-only). CF cards and CF-IDE/ATA adapters are fairly cheap nowadays. 512 MB cards are about 20 Euros over here, and that's more than enough to contain a root FS and even a bit more for convenience. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. "That's what I love about GUIs: They make simple tasks easier, and complex tasks impossible." -- John William Chambless