From owner-freebsd-net@freebsd.org Fri Mar 19 15:58:05 2021 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9D1E4578F62 for ; Fri, 19 Mar 2021 15:58:05 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-eopbgr670073.outbound.protection.outlook.com [40.107.67.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F27pN3cL4z3DS5; Fri, 19 Mar 2021 15:58:03 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Y2WPs3IhEaGA/iH0Jj33jGPoIkzmdWsBH6ZjhmlJVLEls++gkvGC5+fzOJ8fL0sUW6xuKrGRZ2O7JFH0BdX2DGU/+Jc4pBmX1YYYB6poiw0966GcSyJSqiOty3bTl2Qr0SNc4rm99ADPZNfroeAlc9zGt4/T9ZPgshOPt60uahlavoqhFZQ/3Lj3HGOKddqe8AF5P5kuAcrMXqr/S7f2vtkSYgq5R1WdbsEfbzVhgp8fJJipZcmudP4g34n9Tne0j1CQ1QmC/VTU3FdQNhXTwIyaDX0Mv+yLJ5EhKXTVu49qGR4rrCKi3T5EaXLEoIIe5d1Jgrtr6xryTnxgTfplGQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8J2Yrj4xVoKWlqCLPSRviHtVLOngwbvu0KjGSmYL2Ag=; b=J9bRuUDCjZilEplVO0GEqZwPFYTMlo+8PSRNT2IEVFjs1bhg2Ut25Kn/RCaucXqZdoWGt8YszdrgxPGk2ex6R4SNtIdUZfxlE/Tmbe2paNp7iU6WZ2OrSl8A0ZsCHmbihPYoXOjVGQj0F3KpZ492/YdhdNQEy9CP1h/8pm/8LicPXSJMlMn0AzaZ0w4ZIbBKUf6O7K6xFN1VCALT1TXDiZiWcpnZrN9yzneI+ieYvc85DL7oUInzTwq9djsWDtwgklbZ640dMz7yO9Qc1dbC6lKG7MJ3mR1px/O75BEfevu2flGu9qQCQVvmh67t7X9qqiO2VwlMFfhUgZFrejHweA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uoguelph.ca; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8J2Yrj4xVoKWlqCLPSRviHtVLOngwbvu0KjGSmYL2Ag=; b=pFpbay64jZaT6wRZsZhwrwiRfI13mVvBubBxd+JjSfdJRuXUSBeNncJFX49qkUgrGSLhmWy8uMsLd1oogXJywffNN0v3/4F+cmvvCdVw4rv0m89Y87L3WsbqxFWe4Vp4lCKl/jQMvXB7WGWdydsiJdGjpOrNu/L1GUXBoybzBP0e/l2JcuDI+9OHJBjM9pF1QaC9PsdLEHPl+69U1mnEMkE4r6SAOB9QKz/1M+dLb61vQuyc85a+uQk5aO3Wf35b1Gn7UpLhvPIxrCGeWwVisXE8ylbBtcDI+bozKjLi/EGaxWX0wZY3epQJN4H1MSB4f1FQi37eDPsGVxNAPWTC+w== Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:19::29) by YQXPR01MB4328.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c01:10::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3933.32; Fri, 19 Mar 2021 15:58:02 +0000 Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::6073:6fc0:5ddf:dc8a]) by YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::6073:6fc0:5ddf:dc8a%7]) with mapi id 15.20.3933.033; Fri, 19 Mar 2021 15:58:02 +0000 From: Rick Macklem To: "tuexen@freebsd.org" CC: "Scheffenegger, Richard" , "freebsd-net@freebsd.org" , Alexander Motin Subject: Re: NFS Mount Hangs Thread-Topic: NFS Mount Hangs Thread-Index: AQHXG1G2D7AHBwtmAkS1jBAqNNo2I6qIqQ+SgAEIlICAAAOAgIAAhBk7gAAEF4CAATUdPQ== Date: Fri, 19 Mar 2021 15:58:02 +0000 Message-ID: References: <3750001D-3F1C-4D9A-A9D9-98BCA6CA65A4@tildenparkcapital.com> <33693DE3-7FF8-4FAB-9A75-75576B88A566@tildenparkcapital.com> <2890D243-AF46-43A4-A1AD-CB0C3481511D@lurchi.franken.de> , <9EE3DFAC-72B0-4256-B57C-DE6AA811413C@freebsd.org> In-Reply-To: <9EE3DFAC-72B0-4256-B57C-DE6AA811413C@freebsd.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 24327320-8994-4686-0064-08d8eaefc6c4 x-ms-traffictypediagnostic: YQXPR01MB4328: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4502; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ow2EeCHMPoDeLFomg8gQzRhk/2feiqYChlzF2moQJsSbbrFiDKo+wSOcI+6NXjpcUIkYfy4K17+h3Xm/hpQNfDqtBclq4t0ZYcWi2WVQH008C6FqgrBHu1z3Pj5N0P5Wljl9V/i/AQBlt4QaZm9Al7CgHOE83TGYyymrwpJH7WJs4ctoD28y48F1uPduaHfAqM1wJfl2t1BEjsolFf9abOLzSo0lIeNjmO1HkDxhmJ5mrJZZ4rFnUe17yQrMdCEczRv/ltO9UAwvFnVQEpvtcR4vD4gM69I0h5n3VbVBEdXPGkjq38lR5qjXOfKEdmNbwzO0RDvbgWaSLbdTFico25BrEUiv2JQ+NYkzySCq2X76qy5JQirQWS4IXn84wY0Qs8GQ2HHQjp/kIUH9jBZAXLiTN3mEh/LajEkqn1LTfGzKA119naDBR+HVLwAjMgMWD3BDRLdKfXlKEuiRxEGx4f2793UCDriC1KrPUC14BnvFIf6MxHTR3BzP1OKDJQWc4c0gPNLWwhk3qrJr7vRImUNYAcTM2o54IZvK4GVtznc4tUrvHf2aytTHbRG2SG5Hcw31ZHZ42tjEfUX+VL0FzGGAjnpFpLOs+Gv1aXN57vmKcsLWS4f2903NM6U1Q3GiZfOi+Ipf6a6AZN2rxvXiFiHJ/0hoTLAiZz7W9/WpA4DXwzhMi6MsFlWwJXTl3GePyZb4x4exPNrbmz/X8EGqUAPWjxaveDHjPYbzN7ud30Y= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(136003)(346002)(376002)(39860400002)(396003)(366004)(66616009)(83380400001)(66946007)(66476007)(76116006)(66556008)(186003)(478600001)(6506007)(38100700001)(2906002)(64756008)(33656002)(54906003)(66446008)(99936003)(316002)(966005)(52536014)(86362001)(7696005)(55016002)(4326008)(6916009)(3480700007)(71200400001)(5660300002)(7116003)(8676002)(9686003)(786003)(8936002)(91956017); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?iso-8859-1?Q?Eurzpm0Az0tn0nRs+z1M78w/ZYW+YQNoQQX2Qe7cfwr/IrHGOy9MIEjPW1?= =?iso-8859-1?Q?6NwrLHUjQ23Tgedlg7YREJBo9/djtdKzZ7KaJO5f7+NFblr2dVXUpd37bz?= =?iso-8859-1?Q?Ugfsh4KdWTOwtWro927tSnnXvvl9wIkjHECgvbuLf6Rpepez3x2oxBcFy2?= =?iso-8859-1?Q?IWmsigdGOZC9pHY5y52W+RKQXaUfXxzxTB7rdBKKLC6Nm2JOvn9JzrsXZW?= =?iso-8859-1?Q?OkWCyIpOVidujpPsWVSCgJq0bPswz2svSgQi1lSU5xQ6MNcbGIe8HtDK0+?= =?iso-8859-1?Q?8DFAQioM/2T959J/dR8wmyANiT0xgbQcMgmkI2ZNFxvhFC2kbYsqI4JGnR?= =?iso-8859-1?Q?oD3v3al+m8rrKMIIs1Tsr+hGFq3dKPnbdaD9o+J5APQHzAnVR8OtaWRIa7?= =?iso-8859-1?Q?f4qM02RVtBSfVRm4HJABt0tC1Os25UBVGt4oOMLGievSysgd5+u8qyfocf?= =?iso-8859-1?Q?BARL7abThn0fF7bsotjKOkx6H0dFAHwvbs7Vyua/YemoJIKTd1T3208wDQ?= =?iso-8859-1?Q?nKYmzOdaL4VwMuPD4vWeAcUOQoA+1q+5T2Eh95iitMxbTq6hkmKURI9HkV?= =?iso-8859-1?Q?ukiWFecs+dOj0wQpTfdXGeK6t3GFjzyGRg4HEWxrOnNI6u7kNmzV4cl4mf?= =?iso-8859-1?Q?/2XHHgVRUOhoOE7fq7yQWkG2c0WwVqi+m1QRMILLKhI8+h1gzT9YVBNL+m?= =?iso-8859-1?Q?rrmt8sfKnzC/MrAfbwS2r46V6XAT8uP9sGPKD7gO/DEwWKtnt3LUpuP6H1?= =?iso-8859-1?Q?vB4wGFOEt1GoX1MM/be/YXTbPm7iaw/BYQFtsz2O0lvCL90pKEAXzYQT1n?= =?iso-8859-1?Q?G47+kKtuofTpc2J6pTM0Zbhl6ySWaEgw5Yop5ZUxyWC+Y9EIeGJBLHPD/z?= =?iso-8859-1?Q?rBPYAXTqTtmzwL03/r0O0TImpJrmoF874WtZ4dluLJLmdu2PmA1kn3tz9V?= =?iso-8859-1?Q?0f8DBMAg/bYCZM1was3qpYgbUcwKSjL5VixTzZEGtDLvpR+YDtiGGd1x4Y?= =?iso-8859-1?Q?rRKFsZjBefLz38W1ZVI+RW365WL/J9aMeh5WAt6pvzbro49cBf00ccbdaz?= =?iso-8859-1?Q?tXcU9b3rXrCglukKoz2pnZVJLTX2ELx6j6QhexEy/2gYV8/f6K2h0iZCrP?= =?iso-8859-1?Q?T0d1Dv8umurMCMieMvDz2S09ldoo8zyzbZ8TqbnA+GYonQUBSMX7/vcavS?= =?iso-8859-1?Q?aclptr7QkAbTvTNcCz21Q4cY4l47tVBp7xcHSEUITCpgRHjyhAkc7UYcOl?= =?iso-8859-1?Q?hRGoRrz0B/T171xIFCgaq7IxRXfNvih2OJm3wZTldPkrSeydKhJ3DbO8Wn?= =?iso-8859-1?Q?+1NFgCqfhmnWdax1VeCphpZCSK0VktYKvwUpU1hrSyH5MpLsMpyL4nVjzu?= =?iso-8859-1?Q?1T8t2wRiNEYru03aQtBUNsj9JqhJxiZkPLKi8DaIx8qkPWFeNqXJw=3D?= x-ms-exchange-transport-forked: True Content-Type: multipart/mixed; boundary="_002_YQXPR0101MB0968E1537E26CDBDC31C58E5DD689YQXPR0101MB0968_" MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: 24327320-8994-4686-0064-08d8eaefc6c4 X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Mar 2021 15:58:02.5012 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 1h4pxIbnCEldZCwgefqeYFS+y1SegT4mIm3g45oHAJ+FBvDfs0mqw0pdIDsYRTkKVcCZuyMQlQXc0BD6JKJvlA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQXPR01MB4328 X-Rspamd-Queue-Id: 4F27pN3cL4z3DS5 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=uoguelph.ca header.s=selector1 header.b=pFpbay64; arc=pass (microsoft.com:s=arcselector9901:i=1); dmarc=pass (policy=none) header.from=uoguelph.ca; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 40.107.67.73 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-6.10 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; HAS_ATTACHMENT(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[uoguelph.ca:+]; DMARC_POLICY_ALLOW(-0.50)[uoguelph.ca,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCVD_IN_DNSWL_LOW(-0.10)[40.107.67.73:from]; RCVD_TLS_LAST(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[40.107.67.73:from]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:8075, ipnet:40.104.0.0/14, country:US]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector1]; FROM_EQ_ENVFROM(0.00)[]; FREEFALL_USER(0.00)[rmacklem]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; NEURAL_HAM_LONG(-1.00)[-1.000]; SPAMHAUS_ZRD(0.00)[40.107.67.73:from:127.0.2.255]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[40.107.67.73:from]; MAILMAN_DEST(0.00)[freebsd-net] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Mar 2021 15:58:05 -0000 --_002_YQXPR0101MB0968E1537E26CDBDC31C58E5DD689YQXPR0101MB0968_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Michael Tuexen wrote:=0A= >> On 18. Mar 2021, at 21:55, Rick Macklem wrote:=0A= >>=0A= >> Michael Tuexen wrote:=0A= >>>> On 18. Mar 2021, at 13:42, Scheffenegger, Richard wrote:=0A= >>>>=0A= >>>>>> Output from the NFS Client when the issue occurs # netstat -an | gre= p=0A= >>>>>> NFS.Server.IP.X=0A= >>>>>> tcp 0 0 NFS.Client.IP.X:46896 NFS.Server.IP.X:2049 = FIN_WAIT2=0A= >>>>> I'm no TCP guy. Hopefully others might know why the client would be s= tuck in FIN_WAIT2 (I vaguely recall this means it is waiting for a fin/ack,= but could be wrong?)=0A= >>>>=0A= >>>> When the client is in Fin-Wait2 this is the state you end up when the = Client side actively close() the tcp session, and then the server also ACKe= d the FIN.=0A= >> Jason noted:=0A= >>=0A= >>> When the issue occurs, this is what I see on the NFS Server.=0A= >>> tcp4 0 0 NFS.Server.IP.X.2049 NFS.Client.IP.X.51550 = CLOSE_WAIT=0A= >>>=0A= >>> which corresponds to the state on the client side. The server received = the FIN=0A= >>> from the client and acked it.=0A= >>> The server is waiting for a close call to happen.=0A= >>> So the question is: Is the server also closing the connection?=0A= >> Did you mean to say "client closing the connection here?"=0A= >Yes.=0A= >>=0A= >> The server should call soclose() { it never calls soshutdown() } when=0A= >> soreceive(with MSG_WAIT) returns 0 bytes or an error that indicates=0A= >> the socket is broken.=0A= Btw, I looked and the soreceive() is done with MSG_DONTWAIT, but the=0A= EWOULDBLOCK is handled appropriately.=0A= =0A= >> --> The soreceive() call is triggered by an upcall for the rcv side of t= he socket.=0A= >> So, are you saying the FreeBSD NFS server did not call soclose() for thi= s case?=0A= >Yes. If the state at the server side is CLOSE_WAIT, no close call has happ= ened yet.=0A= >The FIN from the client was received, it was ACKED, but no close() call=0A= >(or shutdown(..., SHUT_WR) or shutdown(..., SHUT_RDWR)) was issued. Theref= ore,=0A= >no FIN was sent and the client should be in the FINWAIT-2 state. This was = also=0A= >reported. So the reported states are consistent.=0A= For a test, I commented out the soclose() call in the server side krpc and,= when I=0A= dismounted, it did leave the server socket in CLOSE_WAIT.=0A= For the FreeBSD client, it did the dismount and the socket was in FIN_WAIT2= =0A= for a little while and then disappeared (someone mentioned a short timeout= =0A= and that seems to be the case).=0A= I might argue that the Linux client should not get hung when this occurs,= =0A= but there does appear to be an issue on the FreeBSD end.=0A= =0A= So it does appear you have a case where the soclose() call is not happening= =0A= on the FreeBSD NFS server. I am a little surprised since I don't think I've= =0A= heard of this before and the code is at least 10years old (at least the par= ts=0A= related to this).=0A= =0A= For the soclose() to not happen, the reference count on the socket=0A= structure cannot have gone to zero. (ie a SVC_RELEASE() was missed)=0A= Upon code inspection, I was not able to spot a reference counting bug.=0A= (Not too surprising, since a reference counting bug should have shown=0A= up long ago.)=0A= =0A= The only thing I spotted that could conceivably explain this is that the=0A= function svc_vc_stat() which returns the indication that the socket has=0A= been closed at the other end did not bother to do any locking when=0A= it checked the status. (I am not yet sure if this could result in the=0A= status of XPRT_DIED being missed by the call, but if so, that would=0A= result in the soclose() call not happening.)=0A= =0A= I have attached a small patch, which I think is safe, that adds locking=0A= to svc_vc_stat(),which I am hoping you can try at some point.=0A= (I realize this is difficult for a production server, but...)=0A= I have tested it a little and will test it some more, to try and ensure=0A= it does not break anything.=0A= =0A= I have also cc'd mav@, since he's the guy who last worked on this=0A= code, in case he has any insight w.r.t. how the soclose() might get=0A= missed (or any other way the server socket gets stuck in CLOSE_WAIT).=0A= =0A= rick=0A= ps: I'll create a PR for this, so that it doesn't get forgotten.=0A= =0A= Best regards=0A= Michael=0A= =0A= >=0A= > rick=0A= >=0A= > Best regards=0A= > Michael=0A= >> This will last for ~2 min or so, but is asynchronous. However, the same = 4-tuple can not be reused during this time.=0A= >>=0A= >> With other words, from the socket / TCP, a properly executed active clos= e() will end up in this state. (If the other side initiated the close, a pa= ssive close, will not end in this state)=0A= >>=0A= >>=0A= >> _______________________________________________=0A= >> freebsd-net@freebsd.org mailing list=0A= >> https://lists.freebsd.org/mailman/listinfo/freebsd-net=0A= >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"= =0A= >=0A= >=0A= > _______________________________________________=0A= > freebsd-net@freebsd.org mailing list=0A= > https://lists.freebsd.org/mailman/listinfo/freebsd-net=0A= > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"=0A= =0A= _______________________________________________=0A= freebsd-net@freebsd.org mailing list=0A= https://lists.freebsd.org/mailman/listinfo/freebsd-net=0A= To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"=0A= =0A= --_002_YQXPR0101MB0968E1537E26CDBDC31C58E5DD689YQXPR0101MB0968_ Content-Type: application/octet-stream; name="xprtdied.patch" Content-Description: xprtdied.patch Content-Disposition: attachment; filename="xprtdied.patch"; size=921; creation-date="Fri, 19 Mar 2021 15:50:41 GMT"; modification-date="Fri, 19 Mar 2021 15:50:41 GMT" Content-Transfer-Encoding: base64 LS0tIHN5cy9ycGMvc3ZjX3ZjLmMueHBydGRpZWQJMjAyMS0wMy0xOSAwNjozNDowNi4yOTc1NDUw MDAgLTA3MDAKKysrIHN5cy9ycGMvc3ZjX3ZjLmMJMjAyMS0wMy0xOSAwNzowMjozOS45OTcyMTIw MDAgLTA3MDAKQEAgLTU1OSwxOSArNTU5LDI2IEBAIHN0YXRpYyBlbnVtIHhwcnRfc3RhdAogc3Zj X3ZjX3N0YXQoU1ZDWFBSVCAqeHBydCkKIHsKIAlzdHJ1Y3QgY2ZfY29ubiAqY2Q7CisJZW51bSB4 cHJ0X3N0YXQgcnN0YXQ7CiAKIAljZCA9IChzdHJ1Y3QgY2ZfY29ubiAqKSh4cHJ0LT54cF9wMSk7 CiAKKwlyc3RhdCA9IFhQUlRfSURMRTsKKwlzeF94bG9jaygmeHBydC0+eHBfbG9jayk7CiAJaWYg KGNkLT5zdHJtX3N0YXQgPT0gWFBSVF9ESUVEKQotCQlyZXR1cm4gKFhQUlRfRElFRCk7CisJCXJz dGF0ID0gWFBSVF9ESUVEOworCWVsc2UgaWYgKGNkLT5tcmVxICE9IE5VTEwgJiYgY2QtPnJlc2lk ID09IDAgJiYgY2QtPmVvcikKKwkJcnN0YXQgPSBYUFJUX01PUkVSRVFTOwogCi0JaWYgKGNkLT5t cmVxICE9IE5VTEwgJiYgY2QtPnJlc2lkID09IDAgJiYgY2QtPmVvcikKLQkJcmV0dXJuIChYUFJU X01PUkVSRVFTKTsKKwlpZiAocnN0YXQgPT0gWFBSVF9JRExFKSB7CisJCVNPQ0tCVUZfTE9DSygm eHBydC0+eHBfc29ja2V0LT5zb19yY3YpOworCQlpZiAoc29yZWFkYWJsZSh4cHJ0LT54cF9zb2Nr ZXQpKQorCQkJcnN0YXQgPSBYUFJUX01PUkVSRVFTOworCQlTT0NLQlVGX1VOTE9DSygmeHBydC0+ eHBfc29ja2V0LT5zb19yY3YpOworCX0KKwlzeF94dW5sb2NrKCZ4cHJ0LT54cF9sb2NrKTsKIAot CWlmIChzb3JlYWRhYmxlKHhwcnQtPnhwX3NvY2tldCkpCi0JCXJldHVybiAoWFBSVF9NT1JFUkVR Uyk7Ci0KLQlyZXR1cm4gKFhQUlRfSURMRSk7CisJcmV0dXJuIChyc3RhdCk7CiB9CiAKIHN0YXRp YyBib29sX3QK --_002_YQXPR0101MB0968E1537E26CDBDC31C58E5DD689YQXPR0101MB0968_--