From owner-freebsd-net@freebsd.org Thu Mar 18 20:55:07 2021 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8B8155AD327 for ; Thu, 18 Mar 2021 20:55:07 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-eopbgr670046.outbound.protection.outlook.com [40.107.67.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F1fRZ5Cjlz4mcK for ; Thu, 18 Mar 2021 20:55:06 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Z181yS3RMQpLPF3gVF41MVegFHodwqllMhdKStCNJk+LCty+t4W/dZKqU151FXtpoEsxEGnB+Y20ea+gKVR4E3J++jbm7Cb/J4KKSgyileZPvaxt2OurEFTTyafKOrEPTvhw4T1Ih9ydgKImGNeiJwuM7rShANa8GBp/goZhwPl6uRMffD02bxAI1VgybvUios3t3xH9gejjWHEfh7GYXtYGJLeG3aWMSbOOGHJ8THfRci3Baz50Mw+2LKHZeP82GLNCvLp+eoS/QrEQyeNHfggp3Auyi+ktigxr8hpVSszjRpFrqkFh2UB8OeZ8wdLRzogmcWqnMd2Nb8FIcGE27Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h1+Y7dUTw816tZRoYHWFLpLo91Xr4wUfI/SCT4WMAZg=; b=R8nSNTMZn7ROi93NslcZMgLNudu/n6uwHVcxfwDDs1MywLLf+XQ3u/hwOsx/VE/Jpp/SuXufvlA/+Sn1Xeek/Xc4FhMCIs0jUfbYRlXD06Bb9TkQSfO3djgKxjW4TE5ahIOWCc14M735wKKsWDlMMBmAeOr29zYf0Us/6/AJDcxkOnI6cpdFbb4SPnANPfHQ0dNwfJRP5pqFBs9hPeZklz5Mg28RpsrKUoDyPryZ6wp0DJWMN3G2sRq2E+KzP7EUTETXk+DZuyHhP3mc0IccVQed1zdNPQxTmLlCc8VRB2uyswmeWaB1FwVM9aHh+88GWYE+b3jH9cDHce+Z1I0AkQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uoguelph.ca; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h1+Y7dUTw816tZRoYHWFLpLo91Xr4wUfI/SCT4WMAZg=; b=dOhEtic5yoeV6oaswCC9BVFg/ARYdQRC5KxOywTpm5X7xOStYODuNIYtRFOZUtGSftstAzyAPOZ6Ca5PZB6284lgfdVtvBY9OgC8MlPPmMKdH1ETTwPil9tYmBIrU2uugZpkRORWCDt/oE2btZEN8zF2aHt7qxX6LgDUziGAWvyRTQqIvVGL9rDfOsMnd/g3+fXcoX8JgOuZFasYntNiZhmlWxjNdU3qsxgoDuT9Kp6gVKdR+rUjf37W2wLQgC+8OXln5eEH4LNSGQgtPpqCFFEgaRZAU91U5D1UZQvKdRXdUIvIpqm/dMK6o3r7q1Cb/0yru8AaGsGLVKPA0pduDg== Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:19::29) by YQXPR0101MB0743.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:1c::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3890.28; Thu, 18 Mar 2021 20:55:05 +0000 Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::6073:6fc0:5ddf:dc8a]) by YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::6073:6fc0:5ddf:dc8a%7]) with mapi id 15.20.3933.032; Thu, 18 Mar 2021 20:55:05 +0000 From: Rick Macklem To: Michael Tuexen , "Scheffenegger, Richard" CC: Jason Breitman , "freebsd-net@freebsd.org" Subject: Re: NFS Mount Hangs Thread-Topic: NFS Mount Hangs Thread-Index: AQHXG1G2D7AHBwtmAkS1jBAqNNo2I6qIqQ+SgAEIlICAAAOAgIAAhBk7 Date: Thu, 18 Mar 2021 20:55:05 +0000 Message-ID: References: <3750001D-3F1C-4D9A-A9D9-98BCA6CA65A4@tildenparkcapital.com> <33693DE3-7FF8-4FAB-9A75-75576B88A566@tildenparkcapital.com> , <2890D243-AF46-43A4-A1AD-CB0C3481511D@lurchi.franken.de> In-Reply-To: <2890D243-AF46-43A4-A1AD-CB0C3481511D@lurchi.franken.de> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 52ed7b2e-6d57-437e-fd01-08d8ea501b64 x-ms-traffictypediagnostic: YQXPR0101MB0743: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: SWRhHOcCanjf8U15AlG9HkE7YvfeviM3qnUiIr5ewr8edgHlMBz/tOLmz/4gN13FkqhiEAIcRVpUXGBCH67MFOHH64w+ryxpBwbEr9t5oj3tT2Y6XYr/L4SWUN2S1LORA5zULbAv2xGRUN8tWCYjVK88AJW/G67UVT0c1Ok/qqi56WkcFZ58GRVOo0RIjSPYH3GVoxYNl6bjXbiKWnmJHso8mnAEP/87/aqAGx1zB9QFqPbjrf0pjTEqoskbTPe73otOBAc38dxjMfRFrirNvn5CJto1Fzb0wdvaKhX25WVzFqpaHZBBzLuy+lR96q2Nrs95h/RYho00FeAJbEa4QHreAUqSdw7M2jw/kSe4evDXtOpyYcEVE5gCMPal5FUf+i6hoLCayrpDgQdNTBxf8r9xbXF0TmbPwAOAT5Dc7iKfeusSd0/j0cKbiaGk3yAwmWYc0Tz1ukGQByHQbhWI0d5Toeyry6vZGAahu+Pgns6LL2sWaajMWwVRsYoMZ0ab452h2+JF9XujNu0xIEJW48ijfzTcZtWz/6HLAczpuhxye0GVpgqDhK21zeS/KUNPmkvLcnfsH6VMJlYek14p7i7IxVTeDTLra+tUD9z1HslaPkLwt0PYUc2R0AeJlfwhdd28eLHo/KFPt7GNdNApwSthJhGXmqa7p3J3NMO5IhQ= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(396003)(376002)(366004)(346002)(136003)(39860400002)(54906003)(66556008)(110136005)(7116003)(186003)(2906002)(3480700007)(83380400001)(8676002)(86362001)(316002)(71200400001)(8936002)(38100700001)(55016002)(478600001)(33656002)(64756008)(52536014)(4326008)(6506007)(66946007)(66446008)(91956017)(76116006)(9686003)(7696005)(66476007)(786003)(5660300002)(966005); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?iso-8859-1?Q?hFbN+AydQfS6pB7c32bULwLQSj1rv+fWne4Zf26iKDIYnC5wE5to20ud2A?= =?iso-8859-1?Q?lqG11F2bdK8SoDKCSbpihCMY7yoiB94KilgZ8iuE+VonSWfeBjesBbi1pr?= =?iso-8859-1?Q?tfaki9+QjBpwL37N3soK3Enchk+MoO57RmT1HXdCY0j+n12rFa9RcSxJj/?= =?iso-8859-1?Q?OyM0UcF2KMts2fa0LdhzR4Xmg5weeF+v6+nB4sDsWJXp8Pyl/w4DDJHLA0?= =?iso-8859-1?Q?G/xbpgU3lFUxtBuSib7mxuF7K2Mc/mWMzCApmcZKSNrTBXKZhwVwQ0qJ2B?= =?iso-8859-1?Q?VJHAEOAmaxKOX1XnWf8ahnz3QycWhVFxv3GsSMG0umIA8uC1HztmaWpIUD?= =?iso-8859-1?Q?e6DwIvhOEf6SIPsGF0KwM6ccFnZIs5OMUe32AzMXI7hGZ8WjqaJ8ctj8rl?= =?iso-8859-1?Q?PRNm0Zu3FudI7CoyExB0CW6iP645WZaZ/+Gz2bk289qq/RK2qMjAHqY6AT?= =?iso-8859-1?Q?+Ajbvof6mWajzeNFSvNgfwFH9M7NoKmDxokPv+PJR/OwbgSchuX9FlQmma?= =?iso-8859-1?Q?ZsGXtXVm3RzgkLO+brc5jwdYomylJwNcaD2lM1R7/WRKb+ieFUYhrDhxwE?= =?iso-8859-1?Q?YTP4Pg8BiGMMYoFr5LZ/A6cHvS0xmp1EJDPSWdwY6gCLZytb8OE+ajCORx?= =?iso-8859-1?Q?9BS2pd8m6OslCbgDHS6r85MY1p6PAQ3r/k6eRK7Ji86UD7l9xOARa7G8Ln?= =?iso-8859-1?Q?J+S8pR7ejo478hzM0lLDaHAhXOlhncu0mnte4fCprHfCN7XgkcsE9RWF+X?= =?iso-8859-1?Q?BQri3SE7b8xFwI9B5b+EvB7osnvGEkO12bLaSwkFPAVL3CyWMNaeJrUQCq?= =?iso-8859-1?Q?4rH4qykBojRC6vkwkaxElrkWk2VZIpoKOH6ob0BG87hfcpKFND3SaM/6uO?= =?iso-8859-1?Q?6mYvqiiRlYbDJi3gzXOWZC9Au+DHjQ4k44z1HRTorxZJ0KhPC9VwzEpyZ7?= =?iso-8859-1?Q?DZmueP3wSbG2UAfaZkNzuAPtuXprkOyqPgx3P6HBybAuyhCpM9wsgjybiN?= =?iso-8859-1?Q?fXPwH5GNAFdqHRM4sjVpavkeFouIPy4fH/c+g6gwjIds43R2svq71J6nZD?= =?iso-8859-1?Q?u/2A2yeKZiz6sGvtYTCPXDTULIE9PX56mosPNIZjSnFeb84Lw6YwMw3W6z?= =?iso-8859-1?Q?Q2gKK8sKw2YBeuyl56T6QS92OmT4SQYwe43DXS4NTx/w1JXBraxtl03dbQ?= =?iso-8859-1?Q?cSLcM9eaveBptMbEnG+iA5RiQytlNADLEfYCQFeplfhoCunyBwzHeyfWBc?= =?iso-8859-1?Q?KugD7zReV1vwCydgZruwgwu1V8DUx1K5K/ZtvwxWBTxzvNUmYnEp8nVlId?= =?iso-8859-1?Q?pM1UYXKdug4rwnMnwbpKyQ5S/KyIZyW3FuLL7GeoWaDwiZJ9gDylEkW1CT?= =?iso-8859-1?Q?aELRNRNsFZE52I3qbff4h04I4LiadNGpYq84yNk8oUnOU0J+Cg7vk=3D?= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: 52ed7b2e-6d57-437e-fd01-08d8ea501b64 X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Mar 2021 20:55:05.0361 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: ojlZnKNyq7M97ZGv6vTvFc1hk5Nx18GKCkRdH6s0ahm5bZflkLaurmUySw9aI5UurBKwEKUf4zLgMOT07A5V1A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQXPR0101MB0743 X-Rspamd-Queue-Id: 4F1fRZ5Cjlz4mcK X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=uoguelph.ca header.s=selector1 header.b=dOhEtic5; arc=pass (microsoft.com:s=arcselector9901:i=1); dmarc=pass (policy=none) header.from=uoguelph.ca; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 40.107.67.46 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-6.10 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[uoguelph.ca:+]; DMARC_POLICY_ALLOW(-0.50)[uoguelph.ca,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCVD_IN_DNSWL_LOW(-0.10)[40.107.67.46:from]; RCVD_TLS_LAST(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[40.107.67.46:from]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:8075, ipnet:40.104.0.0/14, country:US]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector1]; FREEFALL_USER(0.00)[rmacklem]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[40.107.67.46:from:127.0.2.255]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[40.107.67.46:from]; MAILMAN_DEST(0.00)[freebsd-net] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Mar 2021 20:55:07 -0000 Michael Tuexen wrote:=0A= >> On 18. Mar 2021, at 13:42, Scheffenegger, Richard wrote:=0A= >>=0A= >>>> Output from the NFS Client when the issue occurs # netstat -an | grep= =0A= >>>> NFS.Server.IP.X=0A= >>>> tcp 0 0 NFS.Client.IP.X:46896 NFS.Server.IP.X:2049 = FIN_WAIT2=0A= >>> I'm no TCP guy. Hopefully others might know why the client would be stu= ck in FIN_WAIT2 (I vaguely recall this means it is waiting for a fin/ack, b= ut could be wrong?)=0A= >>=0A= >> When the client is in Fin-Wait2 this is the state you end up when the Cl= ient side actively close() the tcp session, and then the server also ACKed = the FIN.=0A= Jason noted:=0A= =0A= >When the issue occurs, this is what I see on the NFS Server.=0A= >tcp4 0 0 NFS.Server.IP.X.2049 NFS.Client.IP.X.51550 CL= OSE_WAIT=0A= >=0A= >which corresponds to the state on the client side. The server received the= FIN=0A= >from the client and acked it.=0A= >The server is waiting for a close call to happen.=0A= >So the question is: Is the server also closing the connection?=0A= Did you mean to say "client closing the connection here?"=0A= =0A= The server should call soclose() { it never calls soshutdown() } when=0A= soreceive(with MSG_WAIT) returns 0 bytes or an error that indicates=0A= the socket is broken.=0A= --> The soreceive() call is triggered by an upcall for the rcv side of the = socket.=0A= So, are you saying the FreeBSD NFS server did not call soclose() for this c= ase?=0A= =0A= rick=0A= =0A= Best regards=0A= Michael=0A= > This will last for ~2 min or so, but is asynchronous. However, the same 4= -tuple can not be reused during this time.=0A= >=0A= > With other words, from the socket / TCP, a properly executed active close= () will end up in this state. (If the other side initiated the close, a pas= sive close, will not end in this state)=0A= >=0A= >=0A= > _______________________________________________=0A= > freebsd-net@freebsd.org mailing list=0A= > https://lists.freebsd.org/mailman/listinfo/freebsd-net=0A= > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"=0A= =0A= =0A=