From owner-freebsd-stable@freebsd.org Wed Dec 18 16:06:34 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D416F1E20B6 for ; Wed, 18 Dec 2019 16:06:34 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 47dKd63v0kz4kfF for ; Wed, 18 Dec 2019 16:06:34 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: by mailman.nyi.freebsd.org (Postfix) id 83A591E20B5; Wed, 18 Dec 2019 16:06:34 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 836641E20B4 for ; Wed, 18 Dec 2019 16:06:34 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 47dKd55Gznz4kfD for ; Wed, 18 Dec 2019 16:06:33 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=cs.huji.ac.il; s=57791128; h=References:To:Cc:In-Reply-To:Date:Subject:Mime-Version:Content-Type:Message-Id:From; bh=R2PPf7p/waSs1s183gJiCLcMmSYK2ZhtcMcpuY+A72w=; b=SP2TwWilRQ4s+yV2PuozE7W8tmmfdfpX1/U1VIUyi38x9Wxp+ziKRkfRDr7Lkmdley4gFWEvVqXFVtNueMXgH+B84HFX1s0eVBDcfcV5z2KjAQeTmEcywwalp4W6tfjnOhcaIn4ObP0kD9ooOFqVyTrMNH0EQjFofhAcXqQBXso2DW0kuLkP19MI0vsBJoyv7HEzKpPwjPvS/69YMkiunJ0jHPc+cRWXTNIIMp2k+kjLo1j5LuFVOnT+CPsqVHBMgB3q9NBgRJ2q7gYZ53iARA6nYJJR4GYxkXvq5+E/7/dj35JABV7nWrysdZkEtSqRfkhND1CzE70FSHzznkZ5sg==; Received: from macmini.bk.cs.huji.ac.il ([132.65.179.19]) by kabab.cs.huji.ac.il with esmtp id 1ihbqJ-000F2h-O5; Wed, 18 Dec 2019 18:06:31 +0200 From: Daniel Braniss Message-Id: <06DD810E-3945-40AD-A1A2-578DF4FB1150@cs.huji.ac.il> Mime-Version: 1.0 (Mac OS X Mail 13.0 \(3608.40.2.2.4\)) Subject: Re: nfs lockd errors after NetApp software upgrade. Date: Wed, 18 Dec 2019 18:06:31 +0200 In-Reply-To: Cc: Rick Macklem , "stable@freebsd.org" To: Richard P Mackerras References: <0121E289-D2AE-44BA-ADAC-4814CAEE676F@cs.huji.ac.il> X-Mailer: Apple Mail (2.3608.40.2.2.4) X-Rspamd-Queue-Id: 47dKd55Gznz4kfD X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=cs.huji.ac.il header.s=57791128 header.b=SP2TwWil; dmarc=pass (policy=none) header.from=huji.ac.il; spf=none (mx1.freebsd.org: domain of danny@cs.huji.ac.il has no SPF policy when checking 132.65.116.210) smtp.mailfrom=danny@cs.huji.ac.il X-Spamd-Result: default: False [-3.55 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; R_DKIM_ALLOW(-0.20)[cs.huji.ac.il:s=57791128]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; IP_SCORE(-1.25)[ip: (-2.96), ipnet: 132.64.0.0/13(-1.86), asn: 378(-1.49), country: IL(0.05)]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[cs.huji.ac.il:+]; DMARC_POLICY_ALLOW(-0.50)[huji.ac.il,none]; RCVD_IN_DNSWL_NONE(0.00)[210.116.65.132.list.dnswl.org : 127.0.10.0]; R_SPF_NA(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:378, ipnet:132.64.0.0/13, country:IL]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2019 16:06:34 -0000 > On 18 Dec 2019, at 17:58, Richard P Mackerras = wrote: >=20 > Hi, > What software version is the NetApp using? the very latest :-), but will try and find out later. > Is the exported volume big? about 500G, but many files as far as I know, only accessed by one host running the web app - = moodle. > Is the vserver configured for 64bit identifiers what the issue here? > ? >=20 > If you enable NFS V4.0 or 4.1 other NFS clients using defaults might = mount NFSv4.x unexpectedly after a reboot so you need to watch that.=20 >=20 > Cheers=20 >=20 > Richard=20 > (NetApp admin) >=20 > On Wed, 18 Dec 2019 at 15:46, Daniel Braniss > wrote: >=20 >=20 > > On 18 Dec 2019, at 16:55, Rick Macklem > wrote: > >=20 > > Daniel Braniss wrote: > >=20 > >> Hi, > >> The server with the problems is running FreeBSD 11.1 stable, it was = working fine for >several months, > >> but after a software upgrade of our NetAPP server it=E2=80=99s = reporting many lockd errors >and becomes catatonic, > >> ... > >> Dec 18 13:11:02 moo-09 kernel: nfs server fr-06:/web/www: lockd not = responding > >> Dec 18 13:11:45 moo-09 last message repeated 7 times > >> Dec 18 13:12:55 moo-09 last message repeated 8 times > >> Dec 18 13:13:10 moo-09 kernel: nfs server fr-06:/web/www: lockd is = alive again > >> Dec 18 13:13:10 moo-09 last message repeated 8 times > >> Dec 18 13:13:29 moo-09 kernel: sonewconn: pcb 0xfffff8004cc051d0: = Listen queue >overflow: 194 already in queue awaiting acceptance (1 = occurrences) > >> Dec 18 13:14:29 moo-09 kernel: sonewconn: pcb 0xfffff8004cc051d0: = Listen queue >overflow: 193 already in queue awaiting acceptance (3957 = occurrences) > >> Dec 18 13:15:29 moo-09 kernel: sonewconn: pcb 0xfffff8004cc051d0: = Listen queue >overflow: 193 already in queue awaiting acceptance =E2=80=A6= > > Seems like their software upgrade didn't improve handling of NLM = RPCs? > > Appears to be handling RPCs slowly and/or intermittently. Note that = no one > > tests it with IPv6, so at least make sure you are still using IPv4 = for the mounts and > > try and make sure IP broadcast works between client and Netapp. I = think the NLM > > and NSM (rpc.statd) still use IP broadcast sometimes. > >=20 > we are ipv4 - we have our own class c :-) > > Maybe the network guys can suggest more w.r.t. why, but as I've = stated before, > > the NLM is a fundamentally broken protocol which was never published = by Sun, > > so I suggest you avoid using it if at all possible. > well, at the moment the ball is on NetAPP court, and switching to = NFSv4 at the moment is out of the question, it=E2=80=99s > a production server used by several thousand students. >=20 > >=20 > > - If the locks don't need to be seen by other clients, you can just = use the "nolockd" > > mount option. > > or > > - If locks need to be seen by other clients, try NFSv4 mounts. = Netapp filers > > should support NFSv4.1, which is a much better protocol that = NFSv4.0. > >=20 > > Good luck with it, rick > thanks > danny >=20 > > =E2=80=A6 > > any ideas? > >=20 > > thanks, > > danny > >=20 > > _______________________________________________ > > freebsd-stable@freebsd.org = mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-stable = > > To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org = " >=20 > _______________________________________________ > freebsd-stable@freebsd.org mailing = list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable = > To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org = "