Date: Fri, 14 Nov 2008 11:05:32 +0100 From: Ivan Voras <ivoras@freebsd.org> To: freebsd-current@freebsd.org Subject: Re: GEOM hangover disables NFS Message-ID: <gfjigc$pvk$1@ger.gmane.org> In-Reply-To: <20081112235903.GA19865@troutmask.apl.washington.edu> References: <20081112235903.GA19865@troutmask.apl.washington.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigDCE8722E0DA187436CA00B64 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Steve Kargl wrote: > On three nodes in my cluster (nodes n17, n18, and n19), I had > GEOM use /dev/ad4s1e for tests with gmirror and ggated/ggatec. > I found that GEOM was insufficient for my needs and decided=20 > to return the 3 partitions to NFS-exported partitions. It seems > that once GEOM touches a partition, the partition can no longer > be used by NFS. >=20 > I'll illustrute the problem with n17:/dev/ad4s1e. In what follows, > n10 is the master node. Both n10 and n17 have brand new worlds > and kernels from about 45 minutes ago. >=20 > n10:kargl[203] ssh n17 > n17:kargl[201] df > Filesystem 1M-blocks Used Avail Capacity Mounted on > /dev/ad4s1a 247 104 123 46% / > devfs 0 0 0 100% /dev > /dev/ad4s1e 222780 0 204958 0% /data > /dev/ad4s1d 3962 182 3463 5% /usr > n10:/home 193947 92855 85576 52% /home > n10:/usr/local 19832 10494 7750 58% /usr/local >=20 > n17:kargl[202] tail -1 /etc/exports > /data -alldirs node10 node21 >=20 > The above is after a 'newfs -U /dev/ad4s1e' and a reboot. >=20 > n10:root[244] ls / | grep -E ^n > n11/ n12/ n13/ n14/ n15/ n16/ n17/ n18/ n19/ n20/ n21/ >=20 > n10:root[245] mount_nfs -o tcp n17:/data /n17 > n10:root[246] mount -v | grep n17 > n17:/data on /n17 (nfs, fsid 0eff000303000000) > n10:root[247] ls /n17 > ls: /n17: Input/output error > n10:root[248] ls / | grep -E ^n > ls: n17: Input/output error > n11/ n12/ n13/ n14/ n15/ n16/ n18/ n19/ n20/ n21/ >=20 > n10:root[251] umount /n17 > n10:root[252] ls / | grep -E ^n > n11/ n12/ n13/ n14/ n15/ n16/ n17/ n18/ n19/ n20/ n21/ >=20 > So, how does one exorcise GEOM from /dev/ad4s1e? Your message is very confusing. Here's some information that could help y= ou: * "GEOM" is just a name for "system of handling disk-like devices in FreeBSD". If you "remove GEOM" (which would probably be hard to do and would involve heavily modifying the kernel source) you will have no access to your disk drives and other similar devices. * Using ggated/ggatec you can either: - mount the file system exactly once as rw and none as ro (doesn't matter if the file system is on local node or exported, the total is important) - mount the file system as ro any number of times and none as rw. (In other words, only one machine can have the file system mounted for reading and writing. Once a file system has been mounted as rw, no other machines can do anything with it. This has nothing to do with NFS as NFS operates on top of a mounted file system and inherits its rules.) * You can remove gmirror metadata on devices with "gmirror clean". ggated and ggatec don't store metadata on devices. --------------enigDCE8722E0DA187436CA00B64 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJHU1sldnAQVacBcgRAne+AJ0cD8iDoC4GmERh1qgxfqpFpubQygCg9hrE h18N38tJ+83yWY2hbkzJysw= =tPMI -----END PGP SIGNATURE----- --------------enigDCE8722E0DA187436CA00B64--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?gfjigc$pvk$1>