From owner-freebsd-fs@FreeBSD.ORG Tue Dec 9 03:25:21 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1B2AA6F6 for ; Tue, 9 Dec 2014 03:25:21 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id BD415EC5 for ; Tue, 9 Dec 2014 03:25:20 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AtcEABdrhlSDaFve/2dsb2JhbABSB4NYWASDAcMJhg0CgUMBAQEBAX2EAgEBAQMBI1YZAhgCAg0ZAlkGE4gwCKJhnGaXEAEBAQEBAQEBAgEBAQEBAQEBARmBJo4zCw8BFAEzB4JvgUcFiUKKIAeBdJMvhAwhMIEDAR8DH34BAQE X-IronPort-AV: E=Sophos;i="5.07,542,1413259200"; d="scan'208";a="174827258" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 08 Dec 2014 22:24:06 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 7D63AAEA32; Mon, 8 Dec 2014 22:24:06 -0500 (EST) Date: Mon, 8 Dec 2014 22:24:06 -0500 (EST) From: Rick Macklem To: Mark Schouten Message-ID: <891022143.8046492.1418095446417.JavaMail.root@uoguelph.ca> In-Reply-To: <710130010-1872@kerio.tuxis.nl> Subject: Re: Mountd, why not use the '-S' flag by default MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Dec 2014 03:25:21 -0000 Mark Schouten wrote: > Hi, >=20 >=20 > I'm using a FreeBSD nfs-server as storage for my Linux KVM-based > VPS-platform. The images reside on the NFS-server. >=20 >=20 > I'm been noticing errors in my VPS disks when running 'zfs set > sharenfs=3DXYZ', probably because of reloads of mountd. >=20 >=20 > While trying to debug that, I ran acros this message in mountd(8): >=20 > =C2=A0 =C2=A0 =C2=A0-S =C2=A0 =C2=A0 =C2=A0Tell mountd to suspend/resume = execution of the nfsd > =C2=A0 =C2=A0 =C2=A0threads when- > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ever the exports list is = being reloaded. =C2=A0This avoids > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0intermit- > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0tent access errors for cl= ients that do NFS RPCs while > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0the exports > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0are being reloaded, but i= ntroduces a delay in RPC > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0response while > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0the reload is in progress= . =C2=A0If mountd crashes while an > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0exports > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0load is in progress, moun= td must be restarted to get the > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0nfsd > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0threads running again, if= this option is used. >=20 >=20 > I can't think of a reason why you wouldn't want to use -S by > default.. An '/etc/rc.d/mountd reload' without it causes even my > running Bonnie on a normal NFS-share (not via a diskimage) to stop > with 'input/output error'. Can someone enlighten me with the > drawbacks of using -S ? >=20 Well, there are a couple of things: With "-S" all nfsd threads get suspended/resumed whenever exports changes. This can result in a "pause" in NFS server response and that might be considered a POLA violation. It only works for the new NFS server and not the old one and the old one is still used by some. If it was the default, then the old and new NFS servers would have had different behaviour. (Again, this could be considered a POLA violation.) When "-S" was introduced by me, it was done as a "stop gap", since I had thought that mountd would eventually be replaced by nfse (and nfse did allow exports to be updated "atomically" so the problem didn't occur). It now appears that no variant of nfse will end up in FreeBSD. The last one is noted in the description. If, for some reason, mountd crashes during a reload, then all the nfsd threads could be stuck suspended. (I don't know if this occurs in practice.) Basically, I am a coward w.r.t. POLA and almost never change a default. (The one case I did change was making rsize, wsize default to MAX_BSIZE instead of 32K. By some strange twist of fate, this caused a lot of grief, since there was a bug related to TSO segments just under 64K for network interfaces that are limited to 32 transmit segments. I am still saying "disable TSO" to people running older FreeBSD systems because of this.;-) rick >=20 > Met vriendelijke groeten, >=20 > -- > Kerio Operator in de Cloud? https://www.kerioindecloud.nl/ > Mark Schouten | Tuxis Internet Engineering > KvK:=C2=A061527076=C2=A0| http://www.tuxis.nl/ > T: 0318 200208 | info@tuxis.nl