From owner-freebsd-fs@freebsd.org Fri Jul 3 00:51:59 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 09167993A67 for ; Fri, 3 Jul 2015 00:51:59 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id A6E302D79; Fri, 3 Jul 2015 00:51:58 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2DGBABK3JVV/61jaINVBoNmXwaDGbojgWQKhS5KAoIKEwEBAQEBAQGBCoQjAQEBAwEBAQEgKyALEAIBCA4KAgINGQICJwEJJgIECAcEARwEh3kDCggNtxGQMA2FYAEBAQEGAQEBAQEdgSGKKYJNgVYQAgEFCAEONAeCaIFDBYwXh3uEYYJZgV2ECUSDUI8eg1sCJoQWIjEHf0GBBAEBAQ X-IronPort-AV: E=Sophos;i="5.15,396,1432612800"; d="scan'208";a="223561723" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-annu.net.uoguelph.ca with ESMTP; 02 Jul 2015 20:51:43 -0400 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 6744115F533; Thu, 2 Jul 2015 20:51:43 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id yFGAulvIrRXU; Thu, 2 Jul 2015 20:51:42 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 52DC115F54D; Thu, 2 Jul 2015 20:51:42 -0400 (EDT) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id GOQkAL2s6YM4; Thu, 2 Jul 2015 20:51:42 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 32A6115F533; Thu, 2 Jul 2015 20:51:42 -0400 (EDT) Date: Thu, 2 Jul 2015 20:51:42 -0400 (EDT) From: Rick Macklem To: Ahmed Kamal Cc: Julian Elischer , freebsd-fs@freebsd.org, Xin LI Message-ID: <2010996878.3611963.1435884702063.JavaMail.zimbra@uoguelph.ca> In-Reply-To: References: <1491630362.2785531.1435799383802.JavaMail.zimbra@uoguelph.ca> <5594B008.10202@freebsd.org> <1022558302.2863702.1435838360534.JavaMail.zimbra@uoguelph.ca> <791936587.3443190.1435873993955.JavaMail.zimbra@uoguelph.ca> Subject: Re: Linux NFSv4 clients are getting (bad sequence-id error!) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.12] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF34 (Win)/8.0.9_GA_6191) Thread-Topic: Linux NFSv4 clients are getting (bad sequence-id error!) Thread-Index: qJtbvx6IOu1CAIPoeaFQqFZzEklYJg== X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Jul 2015 00:51:59 -0000 Ahmed Kamal wrote: > PS: Today (after adjusting tcp.highwater) I didn't get any screaming > reports from users about hung vnc sessions. So maybe just maybe, linux > clients are able to somehow recover from this bad sequence messages. I > could still see the bad sequence error message in logs though > > Why isn't the highwater tunable set to something better by default ? I mean > this server is certainly not under a high or unusual load (it's only 40 PCs > mounting from it) > > On Fri, Jul 3, 2015 at 1:15 AM, Ahmed Kamal > wrote: > > > Thanks all .. I understand now we're doing the "right thing" .. Although > > if mounting keeps wedging, I will have to solve it somehow! Either using > > Xin's patch .. or Upgrading RHEL to 6.x and using NFS4.1. > > > > Regarding Xin's patch, is it possible to build the patched nfsd code, as a > > kernel module ? I'm looking to minimize my delta to upstream. > > Yes, you can build the nfsd as a module. If your kernel config does not include "options NFSD" the module will get loaded/used. It is also possible to replace the module without rebooting, but you need to kill of the nfsd daemon then kldunload nfsd.ko and replace nfsd.ko with the new one. (In /boot/.) > > Also would adopting Xin's patch and hiding it behind a > > kern.nfs.allow_linux_broken_client be an option (I'm probably not the last > > person on earth to hit this) ? > > If it fixes your problem, I think this is reasonable. I'm also hoping that someone that works on the Linux client reports if/when this was changed. rick > > Thanks a lot for all the help! > > > > On Thu, Jul 2, 2015 at 11:53 PM, Rick Macklem > > wrote: > > > >> Ahmed Kamal wrote: > >> > Appreciating the fruitful discussion! Can someone please explain to me, > >> > what would happen in the current situation (linux client doing this > >> > skip-by-1 thing, and freebsd not doing it) ? What is the effect of that? > >> Well, as you've seen, the Linux client doesn't function correctly against > >> the FreeBSD server (and probably others that don't support this > >> "skip-by-1" > >> case). > >> > >> > What do users see? Any chances of data loss? > >> Hmm. Mostly it will cause Opens to fail, but I can't guess what the Linux > >> client behaviour is after receiving NFS4ERR_BAD_SEQID. You're the guy > >> observing > >> it. > >> > >> > > >> > Also, I find it strange that netapp have acknowledged this is a bug on > >> > their side, which has been fixed since then! > >> Yea, I think Netapp screwed up. For some reason their server allowed this, > >> then was fixed to not allow it and then someone decided that was broken > >> and > >> reversed it. > >> > >> > I also find it strange that I'm the first to hit this :) Is no one > >> running > >> > nfs4 yet! > >> > > >> Well, it seems to be slowly catching on. I suspect that the Linux client > >> mounting a Netapp is the most common use of it. Since it appears that they > >> flip flopped w.r.t. who's bug this is, it has probably persisted. > >> > >> It may turn out that the Linux client has been fixed or it may turn out > >> that most servers allowed this "skip-by-1" even though David Noveck (one > >> of the main authors of the protocol) seems to agree with me that it should > >> not be allowed. > >> > >> It is possible that others have bumped into this, but it wasn't isolated > >> (I wouldn't have guessed it, so it was good you pointed to the RedHat > >> discussion) > >> and they worked around it by reverting to NFSv3 or similar. > >> The protocol is rather complex in this area and changed completely for > >> NFSv4.1, > >> so many have also probably moved onto NFSv4.1 where this won't be an > >> issue. > >> (NFSv4.1 uses sessions to provide exactly once RPC semantics and doesn't > >> use > >> these seqid fields.) > >> > >> This is all just mho, rick > >> > >> > On Thu, Jul 2, 2015 at 1:59 PM, Rick Macklem > >> wrote: > >> > > >> > > Julian Elischer wrote: > >> > > > On 7/2/15 9:09 AM, Rick Macklem wrote: > >> > > > > I am going to post to nfsv4@ietf.org to see what they say. Please > >> > > > > let me know if Xin Li's patch resolves your problem, even though I > >> > > > > don't believe it is correct except for the UINT32_MAX case. Good > >> > > > > luck with it, rick > >> > > > and please keep us all in the loop as to what they say! > >> > > > > >> > > > the general N+2 bit sounds like bullshit to me.. its always N+1 in a > >> > > > number field that has a > >> > > > bit of slack at wrap time (probably due to some ambiguity in the > >> > > > original spec). > >> > > > > >> > > Actually, since N is the lock op already done, N + 1 is the next lock > >> > > operation in order. Since lock ops need to be strictly ordered, > >> allowing > >> > > N + 2 (which means N + 2 would be done before N + 1) makes no sense. > >> > > > >> > > I think the author of the RFC meant that N + 2 or greater fails, but > >> it > >> > > was poorly worded. > >> > > > >> > > I will pass along whatever I get from nfsv4@ietf.org. (There is an > >> archive > >> > > of it somewhere, but I can't remember where.;-) > >> > > > >> > > rick > >> > > _______________________________________________ > >> > > freebsd-fs@freebsd.org mailing list > >> > > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > >> > > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > >> > > > >> > > >> > > > > >