From owner-freebsd-stable@freebsd.org Thu Dec 19 09:50:46 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E10C61DB675 for ; Thu, 19 Dec 2019 09:50:46 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47dnF20YVmz4TdR for ; Thu, 19 Dec 2019 09:50:46 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: by mailman.nyi.freebsd.org (Postfix) id 0D3931DB674; Thu, 19 Dec 2019 09:50:46 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0D0131DB673 for ; Thu, 19 Dec 2019 09:50:46 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47dnF04fk9z4TdG for ; Thu, 19 Dec 2019 09:50:43 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=cs.huji.ac.il; s=57791128; h=To:References:Message-Id:Content-Transfer-Encoding:Cc:Date:In-Reply-To:From:Subject:Mime-Version:Content-Type; bh=aDThAax3PE8mzGIUS3o3WR1MoQkbKkHWRt1c5rQJG/s=; b=dcz90JnOXBxBN/7PAutgRxx877QKZmGj2Veolzt0PwsUJbiDK8D4kJ+K+Yo/sm4bddUWOhuccEzaqfuVK/ssCQC27fxQkd2pXjMn1V8sc81t9EJSju64MJgl6Wat78B1q9lZzkbKCsoEaQ68NuKe/lIyWVvU4eLNYXwQtrXPXBRFGOLiDBtvvMhGvQh9p6vTaKG0i6YpC0XL22uIe2MgsIC/VCJdMmhOcA8rNozzakjzxLMSjWT20VYAju2F0F88wc4F9bisqdZWI7DZb2LeNw6RFVUVMx5Hs7MwZej4+JpsuKKn+jAe4B4EL4+2hn1zXWOS17MtcwPkqR63PqenBA==; Received: from bach.cs.huji.ac.il ([132.65.80.20]) by kabab.cs.huji.ac.il with esmtp id 1ihsS8-0009FR-1Z; Thu, 19 Dec 2019 11:50:40 +0200 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: Re: nfs lockd errors after NetApp software upgrade. From: Daniel Braniss In-Reply-To: Date: Thu, 19 Dec 2019 11:50:39 +0200 Cc: Richard P Mackerras , "stable@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <854B6E5A-C6BC-44B3-A656-FC9B8EF19881@cs.huji.ac.il> References: <0121E289-D2AE-44BA-ADAC-4814CAEE676F@cs.huji.ac.il> To: Rick Macklem X-Mailer: Apple Mail (2.3445.9.1) X-Rspamd-Queue-Id: 47dnF04fk9z4TdG X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=cs.huji.ac.il header.s=57791128 header.b=dcz90JnO; dmarc=pass (policy=none) header.from=huji.ac.il; spf=none (mx1.freebsd.org: domain of danny@cs.huji.ac.il has no SPF policy when checking 132.65.116.210) smtp.mailfrom=danny@cs.huji.ac.il X-Spamd-Result: default: False [-3.66 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; R_DKIM_ALLOW(-0.20)[cs.huji.ac.il:s=57791128]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(-1.36)[ip: (-3.28), ipnet: 132.64.0.0/13(-1.99), asn: 378(-1.60), country: IL(0.05)]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[cs.huji.ac.il:+]; DMARC_POLICY_ALLOW(-0.50)[huji.ac.il,none]; RCVD_IN_DNSWL_NONE(0.00)[210.116.65.132.list.dnswl.org : 127.0.10.0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:378, ipnet:132.64.0.0/13, country:IL]; FREEMAIL_CC(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Dec 2019 09:50:46 -0000 > On 19 Dec 2019, at 02:22, Rick Macklem wrote: >=20 > Richard P Mackerras wrote: >=20 >> Hi, >> What software version is the NetApp using? >> Is the exported volume big? >> Is the vserver configured for 64bit identifiers? >>=20 >> If you enable NFS V4.0 or 4.1 other NFS clients using defaults might = mount NFSv4.x >unexpectedly after a reboot so you need to watch that. > The FreeBSD client always uses NFSv3 mounts by default. To get NFSv4 = you must > explicitly specify the "nfsv4" or "vers=3D4" mount option. For = NFSv4.1, you must > also specify "minorversion=3D1=E2=80=9D. >=20 > The Linux distros I am familiar with will use the highest NFS version = supported by > the server by default. (I suspect some are using NFSv4.1 without = realizing it, > which isn't necessarily bad.) >=20 > nfsstat -m > will show you which version is actually in use for both FreeBSD and = Linux. >=20 all mounts are nfsv3/tcp the error is also appearing on freebsd-11.2-stable, I=E2=80=99m now = checking if it=E2=80=99s also happening on 12.1 btw, the NetApp version is 9.3P17 cheers, danny > rick >=20 > Cheers >=20 > Richard > (NetApp admin) >=20 > On Wed, 18 Dec 2019 at 15:46, Daniel Braniss = > wrote: >=20 >=20 >> On 18 Dec 2019, at 16:55, Rick Macklem = > wrote: >>=20 >> Daniel Braniss wrote: >>=20 >>> Hi, >>> The server with the problems is running FreeBSD 11.1 stable, it was = working fine for >several months, >>> but after a software upgrade of our NetAPP server it=E2=80=99s = reporting many lockd errors >and becomes catatonic, >>> ... >>> Dec 18 13:11:02 moo-09 kernel: nfs server fr-06:/web/www: lockd not = responding >>> Dec 18 13:11:45 moo-09 last message repeated 7 times >>> Dec 18 13:12:55 moo-09 last message repeated 8 times >>> Dec 18 13:13:10 moo-09 kernel: nfs server fr-06:/web/www: lockd is = alive again >>> Dec 18 13:13:10 moo-09 last message repeated 8 times >>> Dec 18 13:13:29 moo-09 kernel: sonewconn: pcb 0xfffff8004cc051d0: = Listen queue >overflow: 194 already in queue awaiting acceptance (1 = occurrences) >>> Dec 18 13:14:29 moo-09 kernel: sonewconn: pcb 0xfffff8004cc051d0: = Listen queue >overflow: 193 already in queue awaiting acceptance (3957 = occurrences) >>> Dec 18 13:15:29 moo-09 kernel: sonewconn: pcb 0xfffff8004cc051d0: = Listen queue >overflow: 193 already in queue awaiting acceptance =E2=80=A6= >> Seems like their software upgrade didn't improve handling of NLM = RPCs? >> Appears to be handling RPCs slowly and/or intermittently. Note that = no one >> tests it with IPv6, so at least make sure you are still using IPv4 = for the mounts and >> try and make sure IP broadcast works between client and Netapp. I = think the NLM >> and NSM (rpc.statd) still use IP broadcast sometimes. >>=20 > we are ipv4 - we have our own class c :-) >> Maybe the network guys can suggest more w.r.t. why, but as I've = stated before, >> the NLM is a fundamentally broken protocol which was never published = by Sun, >> so I suggest you avoid using it if at all possible. > well, at the moment the ball is on NetAPP court, and switching to = NFSv4 at the moment is out of the question, it=E2=80=99s > a production server used by several thousand students. >=20 >>=20 >> - If the locks don't need to be seen by other clients, you can just = use the "nolockd" >> mount option. >> or >> - If locks need to be seen by other clients, try NFSv4 mounts. Netapp = filers >> should support NFSv4.1, which is a much better protocol that = NFSv4.0. >>=20 >> Good luck with it, rick > thanks > danny >=20 >> =E2=80=A6 >> any ideas? >>=20 >> thanks, >> danny >>=20 >> _______________________________________________ >> freebsd-stable@freebsd.org mailing = list >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org" >=20 > _______________________________________________ > freebsd-stable@freebsd.org mailing = list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org"