From owner-freebsd-fs@freebsd.org Thu Feb 11 21:54:30 2021 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CBD85537C54 for ; Thu, 11 Feb 2021 21:54:30 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-eopbgr670068.outbound.protection.outlook.com [40.107.67.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Dc9QF5PLVz4t8g; Thu, 11 Feb 2021 21:54:29 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OZQsSoX4YE2PRfxuVFbVSMMl78j9kVFcq5QRxMzgwNtwVDVsE7CCNiRaXg2Fm92tuRoG6NOvkSzjQwpEi+lMCvWsxo4GPR0cc8jTttXKlJ3ygmdO4bA/PaIAovxYarI4BoN5oyWJfxps5s/vVoRjuzQX3wFXKzpkSZ9+vlNXBSeTAdPF7fv+yezUoWDP9nmZxJMwl+2H2pFuWlfI45+39tuIQmpSmvDBVRGxmI5vfhxnOLd/NvdsXWkTNmBXnleDfooYYM4Y/uJYvx/m8BF+f3tiDhZ02Xyx8SFz0YYouEOij4w9GvTGW2lA1nfqoQNszg6xVpjSJjNbOLONUkTQxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6gzuyQtxkNq0Kwctzr6ElCa7yjxvZ3m6oCd/FsBRRRk=; b=T4Ufo0FW5OjzckkXIiBVBEq4RdccepPlXEOq1hVnvjjw+gigvodQ90LKNkfYqtdot+rGwb4gP/BEjkAxxr3sZmz5+fjZYkJpworz6VnJhIyS5x+QjaQ0jMpD627lGfRirOx85unNaNDU2YlWKovtt902kz28LnQ+5Sz8GguBG/m0ORFPuu2ND2a9vD8TS1WRYomuy1ZQhxdViacgu/clRGfoDbnzHUTOLqqqnkBi2XFLnF3qrDwA2BdoSBV9vLmSe0oChkTewRax2VskZ8Y3c6GB0mYK+BsLrJhMJRdBo/d6cHJUGtNt4QreWN1eYEw7donwKDbhFdHSZLJ7Pxi/GA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uoguelph.ca; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6gzuyQtxkNq0Kwctzr6ElCa7yjxvZ3m6oCd/FsBRRRk=; b=YNhzNTlipHvsjFIL92mkOFDwIaDPcK+7xoNKuRvCbiECSjDn8Ygi52LsPAvCXqW2kmu+AiTEH3lNkRPobw2bWVFP7ro3gMkn+SmDj+k5xCZHTd9j7B6/KrC7H1kM/8i3M8i4B/p/BiXHRCTH7S1Pj/tLyralrZ5XL44mrwAlRUHwes3shxG0Qwcp5C+1CQkjBHEIXdDT/vXlJ7iXuBEPOZndxLtCu4UmJdvW9fkAObC5vZrkw/WEKR4e+Fv4CjVUowN7haeO4QMLwAlTwXMXnU8YH35ZgUm/rZW/TKe+QbfSO89MfumHYYrRqaj6fbRwnz/93CfIV+KHB4M60vMLRA== Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:19::29) by YQXPR01MB2487.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:51::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3825.23; Thu, 11 Feb 2021 21:54:27 +0000 Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::6073:6fc0:5ddf:dc8a]) by YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::6073:6fc0:5ddf:dc8a%7]) with mapi id 15.20.3825.030; Thu, 11 Feb 2021 21:54:27 +0000 From: Rick Macklem To: Alan Somers , freebsd-fs Subject: Re: NFS delegations don't expire after unmounting client Thread-Topic: NFS delegations don't expire after unmounting client Thread-Index: AQHXAKqojQBgScYUMUKehj4UI3CsMKpTfXiC Date: Thu, 11 Feb 2021 21:54:27 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 38f1503a-03df-4eae-a075-08d8ced79a90 x-ms-traffictypediagnostic: YQXPR01MB2487: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: EiLqqCmo8195S92MfrznzlCBidEQAS6DcMwS2cIyhR9I2c46r5JnXScR2IYVf/3a6QoNUoBG7P8xnPcHcdt0p4xA6Xb8fQACQ0ygHDZs7lCsdAjBxe5WtM3gnStkyqDu48gQrYXzvpMEvAhefOEQ39YkazXTBqH0DuhyJv/qZWmjogQ3p25StN41D96CjIgCI4DpmN8LuCv4Sh/kcktBXyZiOxXXrDgNKZ/7D8ZpThMnn+zQFVohVkkl0i0MVcQwhEy5uHWcT4IdiEZmphNoOU5eeRnOfLEAZilGiFXVGzN4cgn3druyF+Au5jOHHeM/g4uAUv597PMzRKOGzbgMOhb/sqy2YdAJ6x7l3iCcfc6i76GYTBeOrWePgQDySFAHElhaAC4ENDpVJGGL4rmKUOWuFst5lVDuewxxXpzX1zYQtUAT2YWK9SGiYYuVBYx0Y0hKRn1xg6YpnGEltb0yqWChfwBV8m65GEJic6v+pDXvm8MZiLP8Hp3jSz6U87ssVCuacwbdWYNu4z3DqvZ+CUj9vzCsPvVA1IR1z4+wF9gDIh0m7YK1oC0f+G88BkiXpQ26J+8Rhxok5lgVlEtMrOe+ofx2N/zMcGf5cBi7L2yTHDBvJkfmaRjB+tKb6MfDAoHlkk+pJvUWjsjAx7Dv+g== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(366004)(346002)(39860400002)(136003)(376002)(396003)(9686003)(966005)(186003)(83380400001)(55016002)(8936002)(71200400001)(86362001)(786003)(316002)(91956017)(66946007)(6506007)(110136005)(8676002)(478600001)(76116006)(450100002)(2906002)(7696005)(66556008)(66476007)(66446008)(64756008)(33656002)(52536014)(5660300002)(21314003); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?iso-8859-1?Q?mgXCHJetEWPXuOcAa+D9zu+9WXSvpYu+LRlcv+23k1mX/zh29AMUu6urMS?= =?iso-8859-1?Q?RKjcow4H/DMV9mxLtdEa5a7S/N+myJ910syABkRJbFsZQX36t9q+U9O+Sz?= =?iso-8859-1?Q?I0df0USax41M8aER76GO4LuhUODuJ5TZwudHHMApYiMW9vO12JUpzNImbl?= =?iso-8859-1?Q?5t+i/WOSvAJUtWfxTL316Habqki2H3onMdBs46M5pB+onddkm1DyJUlHpV?= =?iso-8859-1?Q?n7rdTAJwK6v0eorGLJD0lMyfG2f2G1hPt+/iZSqRsNMjZMiK91tTYzqmjy?= =?iso-8859-1?Q?gMcjNtiTIKSGwYwvNZ3hSW/FECgnLGe12kKr8j+nQ11qGnvWDh/6oMxawR?= =?iso-8859-1?Q?F2vLa8JLfVLlP+R2cQQsTnhURQPIkvsQzSOOPvgNevq99hWKLkmE/3SSDu?= =?iso-8859-1?Q?2dywGo7jZvkcyP0yG0rmCSRbCxT2Fwh74foa5mZSppXaUXWiHaMrGDDAmt?= =?iso-8859-1?Q?PAVKy+QLFU4bEMG/TryomlTP+aiXEAvM8b5Hqx6VgJC9Vqc5hFXOiTGFJi?= =?iso-8859-1?Q?N0vMJJpUYTjSDUG2Aj0aUd9b2DIknq5xP9RXUViin1TBzYVeurwcNJGDHZ?= =?iso-8859-1?Q?zyI3Nzmd+vSD6wjX2gnGvGYG/NDPAhv0OURGB6jrtpq+BsmHhJgUIpZ1kN?= =?iso-8859-1?Q?rpOP6vw/3VMWkCZfJNIaCuF3FGozPmLE7YpI+uiBtXShTqRKkcnJULQL5y?= =?iso-8859-1?Q?3Yv4zrSGyAiLcrOjuIHM3odx3rRUPCPX2RrHUZdTx2sl896gQj1Rl1DBLU?= =?iso-8859-1?Q?fODz8J21UMGwxosK6RdtvWoKvGCKxqFNoAGA0qWSZKrIETw6Gm798rDYhD?= =?iso-8859-1?Q?VHB+4y5mG+P2nlHj3O4jnpUxOmy/BBupxE6A5UIJs+HLxdhF33Uuc1ZjaL?= =?iso-8859-1?Q?JMRwER9xmneETVxfCUgFUKbbC+y+IT0lk86BL+00X4WpW2E0CcsA+pcpDU?= =?iso-8859-1?Q?MzgPclm3gBonYdGFGJTSL9QWt72knCtpD0JuxKEXaz2WkU6ucHrqAVggGL?= =?iso-8859-1?Q?4EC78pvSsd9JkXSxFfLNgD8gpJ0IdVDlNhbxE4vVC6xI3owxhA4+Al3hno?= =?iso-8859-1?Q?Sj52knuL5mWPClRhzJjNF0lXCZ7nBEjyJArjEwABYHBFsxa0HXSc2/lrf0?= =?iso-8859-1?Q?B2LhtV8HZlmz0bdIXu3GEyrdg0elZC3qShQWcrlCGGhqoEugsfoTp5OI6n?= =?iso-8859-1?Q?3c1pkCgFUHBABZBlqPyuStvlhOWfv6T5UkpsdwAGeTjE1EqeGXQ3YgfgZs?= =?iso-8859-1?Q?vgRhAPkD625ELtuckrs6lqwOV3IkmDn0/ghQMeUVAjm20NiVotUG5yH8GG?= =?iso-8859-1?Q?WQV3P9DYzkVwjIgRzzgQl9XXJgGsDE/+ygcw46+F1UomWEcsuyh+teLTg6?= =?iso-8859-1?Q?40d5GqAMP/1MoONso9aUQKVS8lTR6FB4WGx+GyxcRjrK8WzsPxSVg=3D?= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: 38f1503a-03df-4eae-a075-08d8ced79a90 X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Feb 2021 21:54:27.8675 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: BLdbBKF7sI3lvIfOpxVEnYe4up4j29icgxzGgVy0rtLPYeXoxmT1zgDe6Lu8Bdoc/7xaOHvg43N2bsZhWfzkvg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQXPR01MB2487 X-Rspamd-Queue-Id: 4Dc9QF5PLVz4t8g X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=uoguelph.ca header.s=selector1 header.b=YNhzNTli; arc=pass (microsoft.com:s=arcselector9901:i=1); dmarc=pass (policy=none) header.from=uoguelph.ca; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 40.107.67.68 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-6.10 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; RBL_DBL_DONT_QUERY_IPS(0.00)[40.107.67.68:from]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector1]; FREEFALL_USER(0.00)[rmacklem]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; MIME_GOOD(-0.10)[text/plain]; NEURAL_HAM_LONG(-1.00)[-1.000]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca:dkim]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[uoguelph.ca:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[uoguelph.ca,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RWL_MAILSPIKE_POSSIBLE(0.00)[40.107.67.68:from]; SPAMHAUS_ZRD(0.00)[40.107.67.68:from:127.0.2.255]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:8075, ipnet:40.104.0.0/14, country:US]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; MAILMAN_DEST(0.00)[freebsd-fs]; RCVD_IN_DNSWL_LOW(-0.10)[40.107.67.68:from] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Feb 2021 21:54:30 -0000 Alan Somers wrote:=0A= >I have several Linux 5.9.15 clients mounting NFS 4.1 served from a FreeBSD= =0A= >12.2-RELEASE server. Today, most of those clients' mounts hung, and their= =0A= >dmesg displayed "nfs: server XXX not responding, still trying".=0A= Usually these messages indicate a networking layer problem.=0A= Next time (or if it still happening), check basic network connectivity.=0A= Also, if any net interface does not handle TSO correctly for an RPC=0A= message near 64Kbytes in size (the nasty one is where the NFS RPC is=0A= less than 64K by less than 14bytes, so when the MAC layer header is=0A= prepended, the total exceed 64K), you can get "stuck" TCP connections.=0A= Most FreeBSD net chip drivers should be fixed for this, but I wouldn't be= =0A= surprised if there are some broken ones out there.=0A= --> Disabling TSO fixes the problem in this case.=0A= =0A= rick=0A= =0A= But one=0A= client kept running fine. nfsdumpstate on the server showed that that=0A= client, and that one only, had 2 delegations. It also had 1 OpenOwner, 1= =0A= Open, and the CB flags set. It was the only client that had CB set. On=0A= the theory that its delegation callbacks weren't working, I tried=0A= unmounting all of its NFS shares. That worked, but to my surprise=0A= nfsdumpstate showed no change! I could see that the lease time recorded in= =0A= /var/run/nfs-stablerestart was 120s, and I must've waited about 30m in all= =0A= before disabling delegations, unmounting everything, and returning to NFS= =0A= v3. So my questions are, what can cause a delegation to linger around long= =0A= after it should've expired, and what else can I do to debug this problem if= =0A= it recurs?=0A= =0A= -Alan=0A= _______________________________________________=0A= freebsd-fs@freebsd.org mailing list=0A= https://lists.freebsd.org/mailman/listinfo/freebsd-fs=0A= To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"=0A= =0A=