From nobody Sat Jul 16 14:19:10 2022 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4LlVhy0qmgz4Wf9H for ; Sat, 16 Jul 2022 14:19:14 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-YT3-obe.outbound.protection.outlook.com (mail-yt3can01on2044.outbound.protection.outlook.com [40.107.115.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LlVhw5FHcz3T6p for ; Sat, 16 Jul 2022 14:19:12 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hpQYlCQRSEAvwXg2bRrPHPDRBt+eqm3YSHCHfCOCp24XxkEdKi/6LiBv3Rodf1UAPAMrFPYdVFa5N2nLmG9sw181bZxwD8IyY1sLQSXjc6PQ4PWeYpLjPtWW/sdRgYK36cALeKeLMOqMJoRB+39qEYmdAP72twcxIArNVxcBi62E7BIqP1pXQurOd7moygOSYxQvrM5YzKvzSrsYz3vopF4VUvgTsLdZHCKEfxk9gm6EFC0QOXHu8f72vJ5bO42CJwFo1HjqM+ZVQV5sjb3/siyU1ceuWg3ZiYGuS8Ng2eEMOPhxMdUSZBcqvcEaFzbY+fFhq56uCrtFQvwGpuMRIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/h6vzi7170M4MOVDcpaXQqXwkqA8hhpXafz1zt6YcBg=; b=lwCZ02fKWo/MbVInT8PsVVvypGNEwLp5UJuUmDZbVmp/ZYloZC3z9RN4BuRhP6wlNQ0agoMOlXNDNkWWn0Ca4XJjkZzVCi3q4wuELDkwo4OQYo/IIlYzbX1qQSRC68hLdfe4oJ1xQpOhvJKI7OZVoVeB1iUYV0YY3QmP+nJTB7l1rbthy4bTOpwPUcb3m2LnOsk5Xa2Qnae6TjwLkf1PTkecI6rS74yhLKUXODLXXkea7gIFaMaIfH7LCEr/7e9iZjdk8VCbGXAkh6tMUEcZO+ZyS9j8JccP2Ztj0BApkE+jgnmRRY1Wj4zmISAEXnAbItwP0ebhe2gylmcmLg33AA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uoguelph.ca; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/h6vzi7170M4MOVDcpaXQqXwkqA8hhpXafz1zt6YcBg=; b=LNnJonaF4PqZInFse7QtHGkFq/24m8PG1yT34KzdDTBhvUEGA/aVDqjggKpSMRverLVidrtuqPDh30mf15mWUEfQJgquvpTbk7g3OnT2FRXIbglCOxdIy6q8OmBQgt2I5kHW0o9nMshLEAoMgo0dwXTStD4SXwJrstCIkOO5S0N44s2g55jHM5jECXy26ec5kjqt3JDS+UBzwRS+FnI4iuPqQ+YCrkOhDGmhWf1qrx0QnzhouAt5KTaIZFB3FAROBot8PN3OM1140m3L+i5/n9UZiLoVrqyKcAMCi/+jUJnal8AfMDzMdDvm9BxhbcI2swToyOZxA7d34R8FeqhS8Q== Received: from YQBPR0101MB9742.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c01:81::14) by YQBPR0101MB8928.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c01:5c::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5438.14; Sat, 16 Jul 2022 14:19:10 +0000 Received: from YQBPR0101MB9742.CANPRD01.PROD.OUTLOOK.COM ([fe80::51a6:243b:f3db:61fe]) by YQBPR0101MB9742.CANPRD01.PROD.OUTLOOK.COM ([fe80::51a6:243b:f3db:61fe%7]) with mapi id 15.20.5438.021; Sat, 16 Jul 2022 14:19:10 +0000 From: Rick Macklem To: Peter , "freebsd-stable@freebsd.org" Subject: Re: nfs stalls client: nfsrv_cache_session: no session Thread-Topic: nfs stalls client: nfsrv_cache_session: no session Thread-Index: AQHYmQ4r/JoiRFCEK06ukXc4DHs0E62BChDp Date: Sat, 16 Jul 2022 14:19:10 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: suggested_attachment_session_id: 94d14104-e420-433e-a752-d96d94d19ed0 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: a4919598-92fa-4198-a0ff-08da673626c9 x-ms-traffictypediagnostic: YQBPR0101MB8928:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: AZ13U7Nbz2DY0CZFkDh4CgKyt9AUcD5CVNw38PZmXoWVAc5dlgY2YEhTiGTA62Qj/U9qSvD3Y91hKycOfpiFUan/HbhHTUI4EdZjSy9jpYMAWEAenVtx0ZDj7uAJ3rxl2oms8qVDunRsvgfz3izcQUtNBwXWKxkjZMb1nIrBWAZhA/0+upITuZEhjBD/fmpQVJbK6sN11DE8zL1O9Eaprgfcjut1Kb16+nqk73X2f5wjGwDHezGxL/qPFVN1RdmOJmYdeEo908xN8nSWPzEQd/FdzO+CSMfFe7FdxEoRMbL7SIJ5xYQMxXQjkdqKAsH8N/NX2JX7NbKlf5OUzqaTg0hKZ0sEAjgVC9pH2zUEN9uOiO9VEvVqp8PoF5WkJW5nx+OuADegRrbaFp/9NWhz/+tCPLzSO+XLdV3OHD6DpF5Hz7aWtRViEpcsPcFzFa7zdWc9ivFSYjhcMS+y0TABD/QyUCcohKRA6ZxrAo4yEJKDz0zZN9y1/LePfyWO5t5kGw0h1cRS57sOnSFjyABuFV1Ml/4DLtmJdbmq7wTMPL8p3+FgtQLdUnjFpiBA/GoWj/eXT8xDOp8LPih9ije9oFT6JS1zr1W1RQuchuHWbboKO2OObKvptE0c92lXLbDlGOs/St5yv9AE9t8bTEl4P2NAMWYHfK1Un2J292oWzMlQK18KaGvg8wYhJdY0hkI9y859oBtGYyPoDc2HRnj+NOdxzlUPdfX1U/9/8tSRIG+htmtf78Rh27jcoxssNHvKi2W/JTR3V0Cx5PfO/h3aDFiPi9UENlHeJhEwX7bBYfGhyPO+HQCxeFp+zhuSUbFP x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:YQBPR0101MB9742.CANPRD01.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230016)(4636009)(376002)(366004)(136003)(396003)(346002)(39860400002)(5660300002)(8936002)(86362001)(41320700001)(2906002)(52536014)(33656002)(91956017)(110136005)(55016003)(786003)(316002)(8676002)(66476007)(66446008)(64756008)(66946007)(66556008)(76116006)(9686003)(6506007)(122000001)(478600001)(71200400001)(41300700001)(186003)(7696005)(38070700005)(38100700002)(99936003)(83380400001);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?LoENZT6VrOyOzNEj0rKBYMbA6D3kaB9eWx/6lZtcgLFJc1SqClJd4mlpTO?= =?iso-8859-1?Q?bbkfbSva90WwW6KnH6QEvowgrsrkKlo1rwRrFfcrDDJGhPQT7Ge9XS0unL?= =?iso-8859-1?Q?mJ7QYU0AFtGZumOZZZS/DoN0K1QvoRtxszekhV3EPBhJUOvkK+41VplKHK?= =?iso-8859-1?Q?+gLSEPruSgvWt61CHdCnp+szI42jx2hHWrgRg/vZLnAbI5l8nwzYEzBBSz?= =?iso-8859-1?Q?rQ2Sm6kSSQA9/W1Wq6RySLs+SzQAiiQuvuNgbFJigLM88jCu8ah08OQgRL?= =?iso-8859-1?Q?N6w5d0sS8dlQpNpOiqsZSgS3eV28RixtpcfDjYwbK21FP0g6EHIgpCrNF1?= =?iso-8859-1?Q?1+3IeGcFHKgyjzim/McvO1vEfOKk9YflQ4xID2IRt4MChCAn2YqnHBQ0xc?= =?iso-8859-1?Q?PXGMR2H25Wgo3n0A8+nbbiw85LCK5LoQN7TbUzLhPhUrtJaK60FzBALGq3?= =?iso-8859-1?Q?fNCk54gz/Gap0NxYXbjm/FgfM+L9FMO0CPjWf6RW9eky9sOhWj8zFBgDij?= =?iso-8859-1?Q?qCkEItf9YFM2OJhqZ/nEKcQcQbm/OVwrQUkJxZGB8Y7CNRWULKQCy/RRiJ?= =?iso-8859-1?Q?7bHKcBMOdcM5XPqU/fiBVwUxYu8E8/Jb6JynVY0a4RbCoo+pQ1iuqiRGiL?= =?iso-8859-1?Q?f/I8mjeS7vcUSs6wCQzZdeEEexF5rmBZfPCIXybtjeOMfydS5a7pEvswAE?= =?iso-8859-1?Q?XDsiqy0A1f6/6PInT5TMUYMPOVyXZfk1+v7lmPmlc24S2BZQ4qrFSMJRoa?= =?iso-8859-1?Q?2vtxM2pTgawBWvBXXj8rgnsLZRci81DmgobIaQNrmOdk1/5yUxaIksSIYn?= =?iso-8859-1?Q?ddiGrBHNbBflTwQdKvpkratHcuk10Dm2BUZAyqKTB970lUE7ro6DHhXrU6?= =?iso-8859-1?Q?3Y1EFh5us3VOAYnINJO4LeGXYAjnCHQJMBLLNf7x5zjoEjF1h1bpxIQebh?= =?iso-8859-1?Q?wgweYU2uLiR54mP0VIYorfrNbpG6PLh5eLKyliF26MIi9duNU2kwaWGNAf?= =?iso-8859-1?Q?0FkAEHVNqvAJQPcTaefXrR4AlbHS4Ty7NyVmoJteSH+YIuiEs5sXl1hAbV?= =?iso-8859-1?Q?A5oXCpmKGMjBXaX8t+UUXVP1teEg+GOwsuynNbi0MTHwu4po8xPB7mtaXm?= =?iso-8859-1?Q?3L+9iNPJy47Z+u4/baU913OOX7ESOIIdIKkxfPWXm7oklgpeOqFLB+Vnpj?= =?iso-8859-1?Q?/iqMMCHz+1eBCiwHiAomnS7mciPEA9WqqYim4hQ7WI8n+oNz6WXeW06ehE?= =?iso-8859-1?Q?W2Zz4q3XZ1ILN5NBbFJ/1MEB/CyovKYmPbwxpUEbEp4wRGiy+YB6d8xTtr?= =?iso-8859-1?Q?Jxr6Dk1IV/aeiQk+7a3XTF5uVZewOl3PIVFyOdpH6CeDblJ75uBAEkCBJQ?= =?iso-8859-1?Q?aHj/iefyS3gtizHm6T7rR9HIale0cyz1UwikEJlPesoiOn1Z/F57CHx/Su?= =?iso-8859-1?Q?XO9WnSGtKkBmHjYYFAfWmvZfc9k6B/dLeTvCOdbs7MvrhNgeQ9cB7nel4N?= =?iso-8859-1?Q?y6+ZonKoa6B8ye0f2Nk/whZA2hJxrcSsp6mDesPfbObRoYaqjtdWXDYD/+?= =?iso-8859-1?Q?M01DZIo+u5MSEl/LPkusl2ryeYsbhULlk6kRROB73YHKUoDnvduINyuBWM?= =?iso-8859-1?Q?8PIhlJcdjNUZAxMpPZ2Zl07l38FxYVofvvH3wvUB1d3Acd1173RFZfNQ0c?= =?iso-8859-1?Q?zP1mcAZc3eH3dGVz76qy4zZ04ptwZBnBlpxa6XMG?= Content-Type: multipart/mixed; boundary="_002_YQBPR0101MB97429BFA1F580F82F4E4A01DDD8A9YQBPR0101MB9742_" List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: YQBPR0101MB9742.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: a4919598-92fa-4198-a0ff-08da673626c9 X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Jul 2022 14:19:10.2520 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: yKFWlLU+ny6AMekbrdVR5+As4Dz3qXvsU3N9YqGGxXBB86BFSMjQrrx+iI1f8Oukqs8a4DgdFgIiGGJ45bEM0g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQBPR0101MB8928 X-Rspamd-Queue-Id: 4LlVhw5FHcz3T6p X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=uoguelph.ca header.s=selector2 header.b=LNnJonaF; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=none) header.from=uoguelph.ca; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 40.107.115.44 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-6.00 / 15.00]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca:dkim]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[uoguelph.ca,none]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector2]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; TO_DN_EQ_ADDR_SOME(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:8075, ipnet:40.104.0.0/14, country:US]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[40.107.115.44:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[uoguelph.ca:+]; FROM_HAS_DN(0.00)[]; FREEFALL_USER(0.00)[rmacklem]; RCPT_COUNT_TWO(0.00)[2]; TO_DN_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; HAS_ATTACHMENT(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[40.107.115.44:from] X-ThisMailContainsUnwantedMimeParts: N --_002_YQBPR0101MB97429BFA1F580F82F4E4A01DDD8A9YQBPR0101MB9742_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Peter wrote:=0A= > Hija,=0A= > I have a problem with NFSv4:=0A= > =0A= > The configuration:=0A= > Server Rel. 13.1-RC2=0A= > nfs_server_enable=3D"YES"=0A= > nfs_server_flags=3D"-u -t --minthreads 2 --maxthreads 20 -h ..."=0A= > mountd_enable=3D"YES"=0A= > mountd_flags=3D"-S -p 803 -h ..."=0A= > rpc_lockd_enable=3D"YES"=0A= > rpc_lockd_flags=3D"-h ..."=0A= > rpc_statd_enable=3D"YES"=0A= > rpc_statd_flags=3D"-h ..."=0A= > rpcbind_enable=3D"YES"=0A= > rpcbind_flags=3D"-h ..."=0A= > nfsv4_server_enable=3D"YES"=0A= > sysctl vfs.nfs.enable_uidtostring=3D1=0A= > sysctl vfs.nfsd.enable_stringtouid=3D1=0A= > =0A= > Client bhyve Rel. 13.1-RELEASE on the same system=0A= > nfs_client_enable=3D"YES"=0A= > nfs_access_cache=3D"600"=0A= > nfs_bufpackets=3D"32"=0A= > nfscbd_enable=3D"YES"=0A= > =0A= > Mount-options: nfsv4,readahead=3D1,rw,async=0A= > =0A= > =0A= > Access to the share suddenly stalled. Server reports this in messages,=0A= > every second:=0A= > nfsrv_cache_session: no session IPaddr=3D192.168...=0A= The attached little patch might help. It will soon be in stable/13, but is = not=0A= in releng/13.1.=0A= It fixes the only way I am aware of that the client's "nfscl" thread=0A= can get "stuck" on an old session and not do session recovery.=0A= It might be worth applying it to the client.=0A= =0A= This still doesn't explain how the session got broken in the first place.= =0A= =0A= rick=0A= =0A= Restarting nfsd and mountd didn't help, only now the client started to=0A= also report in messages, every second:=0A= nfs server 192.168...:/var/sysup/mnt/tmp.6.56160: is alive again=0A= =0A= Mounting the same share anew to a different place works fine.=0A= =0A= The network babble is this, every second:=0A= NFS request xid 1678997001 212 getattr fh 0,6/2=0A= NFS reply xid 1678997001 reply ok 52 getattr ERROR: unk 10052=0A= =0A= Forensics: I tried to build openoffice on that share, a couple of=0A= times. So there was a bit of traffic, and some things may have=0A= overflown.=0A= =0A= There seems to be no way to recover, only crashing the client.=0A= =0A= =0A= =0A= --_002_YQBPR0101MB97429BFA1F580F82F4E4A01DDD8A9YQBPR0101MB9742_ Content-Type: application/octet-stream; name="defunct-releng13.1.patch" Content-Description: defunct-releng13.1.patch Content-Disposition: attachment; filename="defunct-releng13.1.patch"; size=620; creation-date="Sat, 16 Jul 2022 14:19:06 GMT"; modification-date="Sat, 16 Jul 2022 14:19:06 GMT" Content-Transfer-Encoding: base64 LS0tIHN5cy9mcy9uZnNjbGllbnQvbmZzX2NscnBjb3BzLmMub3JpZwkyMDIyLTA3LTE2IDA3OjA0 OjI2LjUxNzM2NTAwMCAtMDcwMAorKysgc3lzL2ZzL25mc2NsaWVudC9uZnNfY2xycGNvcHMuYwky MDIyLTA3LTE2IDA3OjA0OjI2LjUxNjU3NTAwMCAtMDcwMApAQCAtMTA2MCw4ICsxMDYwLDE0IEBA IG5mc3JwY19zZXRjbGllbnQoc3RydWN0IG5mc21vdW50ICpubXAsIHN0cnVjdCBuZnNjbGNsaQog CQkJICogaW4tcHJvZ3Jlc3MgUlBDcy4KIAkJCSAqLwogCQkJdHNlcCA9IE5VTEw7Ci0JCQlpZiAo VEFJTFFfRklSU1QoJm5tcC0+bm1fc2VzcykgIT0gTlVMTCkKKwkJCWlmIChUQUlMUV9GSVJTVCgm bm1wLT5ubV9zZXNzKSAhPSBOVUxMKSB7CisJCQkJLyoKKwkJCQkgKiBNYXJrIHRoZSBvbGQgc2Vz c2lvbiBkZWZ1bmN0LiAgTmVlZGVkCisJCQkJICogd2hlbiBjYWxsZWQgZnJvbSBuZnNjbF9oYXNl eHBpcmVkKCkuCisJCQkJICovCiAJCQkJdHNlcCA9IE5GU01OVF9NRFNTRVNTSU9OKG5tcCk7CisJ CQkJdHNlcC0+bmZzZXNzX2RlZnVuY3QgPSAxOworCQkJfQogCQkJVEFJTFFfSU5TRVJUX0hFQUQo Jm5tcC0+bm1fc2VzcywgZHNwLAogCQkJICAgIG5mc2NsZHNfbGlzdCk7CiAJCQkvKgo= --_002_YQBPR0101MB97429BFA1F580F82F4E4A01DDD8A9YQBPR0101MB9742_--