From owner-freebsd-stable@freebsd.org Wed Dec 18 15:58:39 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 09CDE1E1D13 for ; Wed, 18 Dec 2019 15:58:39 +0000 (UTC) (envelope-from mack63richard@gmail.com) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47dKRy4VGRz4k4n for ; Wed, 18 Dec 2019 15:58:38 +0000 (UTC) (envelope-from mack63richard@gmail.com) Received: by mailman.nyi.freebsd.org (Postfix) id 976061E1D12; Wed, 18 Dec 2019 15:58:38 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 971811E1D11 for ; Wed, 18 Dec 2019 15:58:38 +0000 (UTC) (envelope-from mack63richard@gmail.com) Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47dKRx4hRXz4k4m for ; Wed, 18 Dec 2019 15:58:37 +0000 (UTC) (envelope-from mack63richard@gmail.com) Received: by mail-wr1-x435.google.com with SMTP id b6so2881236wrq.0 for ; Wed, 18 Dec 2019 07:58:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=rcCGtHgoWDc8yxmnUqPTMVUKIxwZHpXEEoEb08ieNIs=; b=i2B7q097fzQBMi7GdhUALoeBKBRT9H0PQaOuNoky4FNVXU65iAVJ3ngz2XRBauXg8M PJH+w8n3dmO5bd6o5VQfBc1JwMPHDTGI/8Q1UKM5/LECuCdJBE2QPqfqzLgBRw+xNou3 /4AUu4GcVfZ81et0HtHDSobSxxABMVO7BM0MsmvvGGVqrpd4KRjFQkg+orvt5ZVt+2Z/ hG785CSmz/0uTt2cSs3wuCamJdMnqLb0nKMjwWALWI6fIh1Wk99pzR3keNpLul3D/oBH 6G5xfkG0291LIPlk8qbd6ugdZhsZ4AS/Lgk1lKGQuffnsDcaOQFSGl4nt6VYQwynLVqB z4MQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=rcCGtHgoWDc8yxmnUqPTMVUKIxwZHpXEEoEb08ieNIs=; b=KAJdXOg1l7NI2FFGHqF8K2B5OBdZUDqOPTliYObuJ2jTrBQj0fk68SHcrItf6jufHv hDauM9tSOGHqFq7fkEaL5Kj1gbW7jEdy0exxviWEb+NfARkfbqWEgN+MmHXUkw0WowT6 x+kZxEYcvJuQlpSFxRfIPfVBdzqRrm8hVyzimOfFxjaPprVpae4s5G6fCSpmX++Xt3V9 G32MHAv/ETcgJ8tYQ1KtXNFBIrUNRS0HGStqPbyalqxT1mAvtXu08ACr6/IKJjUjUCxK aLl5lxMXuX7/4sjK3wXP7RTr7jxOXYkR7evG3bVhcprSpkM+0+01roE3cMj6/CH/xUZj dxkA== X-Gm-Message-State: APjAAAV6ZpoveZUT2o2NjJDdWhhQjvky5GCSDrj4OvqTldXJ6dpqrsL9 wzbFOuV7sj4O4FckFQpuPZ4itzymb+/dbfgnA6Bn9g== X-Google-Smtp-Source: APXvYqz/cnxM9P5EGVXGj4Y7NK43w585u25fkkNaJpEB+rwdk2aQH5n17BSJr6/CnfI81ccrMle/A60FEBMUe65MB8o= X-Received: by 2002:a5d:6144:: with SMTP id y4mr3694322wrt.367.1576684715421; Wed, 18 Dec 2019 07:58:35 -0800 (PST) MIME-Version: 1.0 References: <0121E289-D2AE-44BA-ADAC-4814CAEE676F@cs.huji.ac.il> In-Reply-To: <0121E289-D2AE-44BA-ADAC-4814CAEE676F@cs.huji.ac.il> From: Richard P Mackerras Date: Wed, 18 Dec 2019 15:58:24 +0000 Message-ID: Subject: Re: nfs lockd errors after NetApp software upgrade. To: Daniel Braniss Cc: Rick Macklem , "stable@freebsd.org" X-Rspamd-Queue-Id: 47dKRx4hRXz4k4m X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=i2B7q097; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of mack63richard@gmail.com designates 2a00:1450:4864:20::435 as permitted sender) smtp.mailfrom=mack63richard@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[stable@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_SOME(0.00)[]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[5.3.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(0.00)[ip: (-9.44), ipnet: 2a00:1450::/32(-2.65), asn: 15169(-1.90), country: US(-0.05)]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Dec 2019 15:58:39 -0000 Hi, What software version is the NetApp using? Is the exported volume big? Is the vserver configured for 64bit identifiers? If you enable NFS V4.0 or 4.1 other NFS clients using defaults might mount NFSv4.x unexpectedly after a reboot so you need to watch that. Cheers Richard (NetApp admin) On Wed, 18 Dec 2019 at 15:46, Daniel Braniss wrote: > > > > On 18 Dec 2019, at 16:55, Rick Macklem wrote: > > > > Daniel Braniss wrote: > > > >> Hi, > >> The server with the problems is running FreeBSD 11.1 stable, it was > working fine for >several months, > >> but after a software upgrade of our NetAPP server it=E2=80=99s reporti= ng many > lockd errors >and becomes catatonic, > >> ... > >> Dec 18 13:11:02 moo-09 kernel: nfs server fr-06:/web/www: lockd not > responding > >> Dec 18 13:11:45 moo-09 last message repeated 7 times > >> Dec 18 13:12:55 moo-09 last message repeated 8 times > >> Dec 18 13:13:10 moo-09 kernel: nfs server fr-06:/web/www: lockd is > alive again > >> Dec 18 13:13:10 moo-09 last message repeated 8 times > >> Dec 18 13:13:29 moo-09 kernel: sonewconn: pcb 0xfffff8004cc051d0: > Listen queue >overflow: 194 already in queue awaiting acceptance (1 > occurrences) > >> Dec 18 13:14:29 moo-09 kernel: sonewconn: pcb 0xfffff8004cc051d0: > Listen queue >overflow: 193 already in queue awaiting acceptance (3957 > occurrences) > >> Dec 18 13:15:29 moo-09 kernel: sonewconn: pcb 0xfffff8004cc051d0: > Listen queue >overflow: 193 already in queue awaiting acceptance =E2=80= =A6 > > Seems like their software upgrade didn't improve handling of NLM RPCs? > > Appears to be handling RPCs slowly and/or intermittently. Note that no > one > > tests it with IPv6, so at least make sure you are still using IPv4 for > the mounts and > > try and make sure IP broadcast works between client and Netapp. I think > the NLM > > and NSM (rpc.statd) still use IP broadcast sometimes. > > > we are ipv4 - we have our own class c :-) > > Maybe the network guys can suggest more w.r.t. why, but as I've stated > before, > > the NLM is a fundamentally broken protocol which was never published by > Sun, > > so I suggest you avoid using it if at all possible. > well, at the moment the ball is on NetAPP court, and switching to NFSv4 a= t > the moment is out of the question, it=E2=80=99s > a production server used by several thousand students. > > > > > - If the locks don't need to be seen by other clients, you can just use > the "nolockd" > > mount option. > > or > > - If locks need to be seen by other clients, try NFSv4 mounts. Netapp > filers > > should support NFSv4.1, which is a much better protocol that NFSv4.0. > > > > Good luck with it, rick > thanks > danny > > > =E2=80=A6 > > any ideas? > > > > thanks, > > danny > > > > _______________________________________________ > > freebsd-stable@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.or= g > " > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >