From owner-freebsd-current@freebsd.org Mon Apr 19 15:03:38 2021 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 66DD05F511A for ; Mon, 19 Apr 2021 15:03:38 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-eopbgr670060.outbound.protection.outlook.com [40.107.67.60]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FP97F1Z8tz4W8Q; Mon, 19 Apr 2021 15:03:36 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IrpJMqlYFt4jMXQ6abSIleF7wqGfdxbISk4c8NAvmkAPr5F7JX4PCSKYt64+SClmiRkqJk1BR3JQqWZi8Y/YlUCYYNveEhWgD/rt86KBjrqI/jp9DWk2R7hPYNKFRT2Cc+1/mKPGs864MSda4L+7hvCyk3hvgJUG5yEZmpKYUQ1X6XfUpLWLslAGvF+9qprirwVVEAhz6vQnvR3C7U8m5eJwnIyTbcRqNImkea49p5k5f6mKrXxbecGWwjgdh3K2RPWhYupJ/f/cW/OtME4ihKX+ppOekWr0SWPp7lOvdqd8U8zWbI6qty5rv+ro0j+QhZAPEQkl9jhfzUYw3iu1rg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=A0OTi1DSZdqRmMNfs+m5/ON1L7LgydKWqJ2ZAnT5xa4=; b=Ut6ntZpjwXLz4N0gU2SgZTcWS+ARt56ldpe/N0zVTVezze5jo18LinzARRQVX/eh4k34iVUsqqriolxsjxi5HRMHq1PoSPJNZ75NsVNYre1GrjLXjn85rqG/LDZxU6gzWFRWB7keYIubmGaJCNwDVNfphfeUVBPVx+IfMyIWOwu4lh7VDS3tmBGsLiEatPcX1pVu+9DQAZBajGCmXDap+SMUjDWwKZoEplJQCI+gDxNZweNz5NbPCffEOvr2lijgbXnOo/tQLiXZFAUIYsy642vVrY5RrIyQawmRFuyIJPLxSJXekldMzrxN+jngbyzUUIbAwumoDo1aixXUssP8og== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uoguelph.ca; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=A0OTi1DSZdqRmMNfs+m5/ON1L7LgydKWqJ2ZAnT5xa4=; b=nO+zS2XdyZ9wdQ3z8ZHp60Q7nNo8igGwAkA8QuesjJh/+NSrCrunaedFgjz4RCuIm/n8j9UTYdronpmSw2bRjwnL+MBc5QVDGFjYH7Rx1vyRlp22ifIv5cDJ3Dx0O3bbHeadXw43Ds+49/A+1icZSSXDZqdCfKLS9T9xtnv9hPWMxmOZYqkmkIMQxEHsL3EcZM2SXml1p0ruS4FqnBFhFg0KPAz9xYWS8ipjok5FBEawphoH3rzfzD3NHXAajAfR8rjCtUbhuGaw5zw7LHH/Tle2I8pPyMV11VFP3Qpwten0D63C379zecCZIBJOT+Oz9Jxs8ffN9YMM+ssminhYKQ== Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:19::29) by YQBPR0101MB4668.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c01:17::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4042.24; Mon, 19 Apr 2021 15:03:35 +0000 Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::1c05:585a:132a:f08e]) by YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::1c05:585a:132a:f08e%4]) with mapi id 15.20.3999.038; Mon, 19 Apr 2021 15:03:35 +0000 From: Rick Macklem To: Olav Gjerde CC: Allan Jude , "freebsd-current@freebsd.org" Subject: Re: NFS issues since upgrading to 13-RELEASE Thread-Topic: NFS issues since upgrading to 13-RELEASE Thread-Index: AQHXMhVsKLCb5bI16E6Ydj5qgkLmjKq16AQAgAAI7ACAAAPdgIAB0dePgAPBloCAAGfJ3g== Date: Mon, 19 Apr 2021 15:03:35 +0000 Message-ID: References: <902a3c81-2ce8-49c0-b163-5ffa4b90afe5@www.fastmail.com> , In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 59d90e9b-4d85-4fd6-1df0-08d903444e15 x-ms-traffictypediagnostic: YQBPR0101MB4668: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:3173; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: g877dNRWC3bWskIve3E+15V2W3XMx3UhiqYwTkIprBseEuXmEbrYxi/Z4Fhh3Ik9PnS2WLRJAkTvVsA/clDVlEtBNVYR1uko3ikkQUBEcioFAprYvfQ7jbf65XWQ7u29to0QDaWDUTyq2b4bNAnBU1sqDAMR1LYG/fC2VUKH3b5GoQv77lVfBrbruvAlbL4GBokL9dtY9rLWAYx2qjQGwYmZ79dng4ZJ54e2nTYd1WjvkbpOc8rcCEdoWBkVAq5k/8Cibf499u2nDIHGPf4Ula06FbljELT913rn/NcYVr3pPivHjJi87xl5lGjd2deU8/xKJHlbFV7JUXLQNa4tdCl5Eouj+TcdvZxa5EAkKaPrBheNBKUqzetpofrLjYf6E6QLek8TCHoobE5CYgp7r/aFjIm1ka6WgrJ8pe9fHOkT8Ii+1gxGH/lhm+K9OwgmlLxZg4fUbKcTiwuP1wBy4NI/vKwf4LTWe8aKW964z2e/G8sP8PmOeG1arJ0fJwob+QGgBmkBfYrCdAd85xY0jX5g0K7OlhjqbWxshIKQKcPPCH9EVyezipHf+IA4U9Ff2EJpDF5DzyaS++YWKumIvivPkpZWvR7v7wkx9WMEYRWcelTzHPvNnPKfynjf6sNWNkDEEPzvzUKUNSqn6I8SOvIgLURmssPYrYcW/CUQ84wOcJGWOacXGEyYeuvkUb1f x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(39860400002)(136003)(366004)(396003)(376002)(346002)(66574015)(91956017)(66556008)(76116006)(66476007)(66616009)(55016002)(786003)(7696005)(86362001)(71200400001)(316002)(66446008)(2906002)(6506007)(9686003)(53546011)(66946007)(5660300002)(8936002)(122000001)(83380400001)(52536014)(186003)(478600001)(6916009)(54906003)(966005)(4326008)(99936003)(64756008)(33656002)(38100700002)(8676002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?iso-8859-1?Q?E/bNAJiRX/eLRNELiFAm/kmBFkWKS0W4YDOc4a+eZ6Fnl8I7Ea2v2YdT3n?= =?iso-8859-1?Q?61kT90MdQHvDXA1dSWweUZyyxS1uoN4L6E/+fv3y92PXVp+Nnb3ltj6r+z?= =?iso-8859-1?Q?gHLnAOgQI8jHc8WdLNd3rpyvT80shRr87AT8oNNG+jwGfRllMnmQI97lkD?= =?iso-8859-1?Q?eKyBdmPPrAJqfT+a3dfGL30FHw1v2USNgO6MJRp02CKf358+mY24cPUDBq?= =?iso-8859-1?Q?EndhKNwuS/zV29b0X/hOgcQvOzFhMufKaOdzeeaLWgl85pqULbrPwuWNSv?= =?iso-8859-1?Q?4g20c7uKRlbFpeOnGBXWU86ZfCR5Ezp7Rc9+MG4XPDN29N7rmxY29rdplZ?= =?iso-8859-1?Q?DasUjdfbTJZFt2avupHs6heBDSR+31gTa0P7g7jih90/cWyCI2FZvTgUby?= =?iso-8859-1?Q?4M748ipqqSpfn6bIK7y1u/YijG5116sHsE2bZuxMg8QxBVDdYa3Jh8m0Dv?= =?iso-8859-1?Q?zMCuTVIz2ndTlezZcXYPcd96QxYq4qoqxAu1MIN/G84Z0IIqpFznF3ISiK?= =?iso-8859-1?Q?aIw8r5iP3eMuIlygdzcvxlS4OjwXev5kIEmq/A1PCkUfhYdUQhTcE7FJih?= =?iso-8859-1?Q?yXrsbkC4VBO1vJWrN/C2nAhaXq8CMF5qaEw+JF6lvpkGMOwsHVaHzHbw6z?= =?iso-8859-1?Q?FJaURIrOj4mqM9gF5YVGFfRIzkwPdbgYkKFNPJEPzs07g4FV0FkOHZfXx+?= =?iso-8859-1?Q?77GH+NZDjLYRRZpP0gh3e9Bz21wzOQCw4OIMJyRCZu9jNdZXF7lqOfy2HE?= =?iso-8859-1?Q?p1cHn1Pr11ebMaa8DEvYbHQu7NCXVI8LqITAgBkIfCcJOzfszcWe27cTHh?= =?iso-8859-1?Q?Jshb7ePzZESHhIfoUhQlf5Cpjkf0EadatpZe4z2CprT339fEMOmd9/P/7u?= =?iso-8859-1?Q?qITEUTqrjVks7hK3mjNeOFXpMa902qigLzM2iYAqGTP+TQ3D+Vcv12Nzh6?= =?iso-8859-1?Q?S/NcA6IAp+0JuWCbz/EnzkzW6I+goo8wcEnZ/k0/IC9jgc9lBgFg7vcS7Z?= =?iso-8859-1?Q?tnBvIj7gNWe0875S54Ur4A7fCYebbtTqKlmyzpMvIIAZA/UR2/lKFbhHz1?= =?iso-8859-1?Q?u10VgHtB61qyRU6L1jnU6ywkImjAR3quLwnz/rJSkS5Ff10kZfdrRTs1De?= =?iso-8859-1?Q?S1KYbGw51LL2DdmwoXdNOKm/eOYOYMD0Lv2TQLS+UzIt/fnIoV+HxTUbDM?= =?iso-8859-1?Q?RVxGodaZWF8SmrA64z45iwobyk02Ie8ek/BeoqnVmnYqaN1HtNWA774Srg?= =?iso-8859-1?Q?Lq+O/66McZ5Yz45RRtcqjUx7u24C0TNM7peugo4z5tT+5fe7cf1RL22CeT?= =?iso-8859-1?Q?Ct/eOSFfJjv+alYgaNXNzWLD9wdcQq6bslpA7XVPQZS1dG77X8O6WyZBDa?= =?iso-8859-1?Q?g7htobFN5bPw4Gu3/oPW2N1+HQGTUpbnN9BHMpIwcEy6DG3iEBNGD0IKHL?= =?iso-8859-1?Q?GEHpV4aKji2QiuB4?= x-ms-exchange-transport-forked: True Content-Type: multipart/mixed; boundary="_002_YQXPR0101MB0968924C8AEAB808888D75B1DD499YQXPR0101MB0968_" MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: 59d90e9b-4d85-4fd6-1df0-08d903444e15 X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Apr 2021 15:03:35.1732 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 6xK1ijM+WBxiXr0+5L8vCznDZwh3r33H6Nl0ln1pkQXwH0NGIidNxUzJ1wpo9Lmt9jlKbWaneBSSk1PHP6fhWA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQBPR0101MB4668 X-Rspamd-Queue-Id: 4FP97F1Z8tz4W8Q X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=uoguelph.ca header.s=selector1 header.b=nO+zS2Xd; arc=pass (microsoft.com:s=arcselector9901:i=1); dmarc=pass (policy=none) header.from=uoguelph.ca; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 40.107.67.60 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-6.10 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RBL_SENDERSCORE_FAIL(0.00)[40.107.67.60:server fail]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; HAS_ATTACHMENT(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[uoguelph.ca:+]; DMARC_POLICY_ALLOW(-0.50)[uoguelph.ca,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCVD_IN_DNSWL_LOW(-0.10)[40.107.67.60:from]; RCVD_TLS_LAST(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[40.107.67.60:from]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:8075, ipnet:40.104.0.0/14, country:US]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector1]; FREEFALL_USER(0.00)[rmacklem]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; SPAMHAUS_ZRD(0.00)[40.107.67.60:from:127.0.2.255]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[40.107.67.60:from]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Apr 2021 15:03:38 -0000 --_002_YQXPR0101MB0968924C8AEAB808888D75B1DD499YQXPR0101MB0968_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Olav Gjerde wrote:=0A= >I have tried D29690 patch and reverting back to r367492 this weekend. Neit= her made any difference for my system.=0A= Just to clarify it, I meant "revert the patch in r367492" and not=0A= "revert to revision r367492". I've attached the patch that=0A= backs out the changes made by the patch in r367492, which=0A= should apply to a fairly recent main/13 kernel.=0A= =0A= This should be done instead of applying D29690, not combined with it.=0A= My testing of D29690 has suggested it is not yet mature, so=0A= I would not recommend choosing that alternative yet.=0A= =0A= If you have tried a kernel with the attached patch applied to it, but not= =0A= D29690 applied to it, then please:=0A= Let us know if you still have Linux clients "hanging" with this kernel.=0A= If still "hanging", try the following to see if they help:=0A= - Use the "minorversion=3D1" mount option on the Linux clients,=0A= to ensure that they are not using NFSv4.2, to see if it is a=0A= NFSv4.2 specific issue.=0A= - Try disabling tso and lro and avoid jumbo frames for drivers=0A= that use jumbo mbufs when handling jumbo frames.=0A= Collect the following info when it happens:=0A= - "netstat -a", to see what the TCP connection is up to.=0A= - "tcpdump -s 0 -w hang.pcap host "=0A= run for several minutes on the server, to see what is going on the=0A= wire. I use wireshark to look at hang.pcap, since it=0A= knows NFS as well as TCP.=0A= You can also do the above with "host " instead=0A= of "host " run on the client.=0A= - "ps axHl" on the server, to see what the nfsd threads=0A= are up to.=0A= If none of the above contains confidential info, please=0A= send it to me, if not the list.=0A= =0A= Good luck with it, rick=0A= ps: Yea, I started this post and then realized I had hit=0A= reply instead of reply all.=0A= =0A= =0A= =0A= There is also a reddit thread about this https://www.reddit.com/r/freebsd/c= omments/mqol4o/nfs_issues_since_upgrading_to_13release/=0A= =0A= On Sat, Apr 17, 2021 at 1:10 AM Rick Macklem > wrote:=0A= Just fyi, I just got a "recursed on non-recursed mutex" panic in=0A= socantrcvmore() with the D29690 patch, so you might not=0A= want to test with that one yet.=0A= =0A= rick=0A= =0A= ________________________________________=0A= From: owner-freebsd-current@freebsd.org > on behalf of Olav Gjerde >=0A= Sent: Thursday, April 15, 2021 3:21 PM=0A= To: Allan Jude=0A= Cc: freebsd-current@freebsd.org=0A= Subject: Re: NFS issues since upgrading to 13-RELEASE=0A= =0A= CAUTION: This email originated from outside of the University of Guelph. Do= not click links or open attachments unless you recognize the sender and kn= ow the content is safe. If in doubt, forward suspicious emails to IThelp@uo= guelph.ca=0A= =0A= =0A= Well something do happen if I restart NFS Service on FreeBSD , it works for= =0A= like 10 seconds then it gets unresponsive again.=0A= =0A= This is my output from `nfsstat -d 1`=0A= =0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= 8.00 1025 8.00 8.02 17170 134.54 2.01 72716 142.54 0.07 51 34=0A= 8.00 2273 17.76 7.99 31273 244.07 2.01 133267 261.83 0.14 20 82= =0A= 8.03 4889 38.33 7.99 25885 202.07 2.06 119340 240.40 0.13 21 81= =0A= [=3D=3D=3D=3D=3D Read =3D=3D=3D=3D=3D] [=3D=3D=3D=3D=3D Write =3D=3D=3D=3D= ] [=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Total =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D]=0A= KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s ms ql %b=0A= 7.98 8811 68.64 8.00 12997 101.54 2.22 78396 170.18 0.15 1 80=0A= 7.99 922 7.20 8.00 3798 29.68 2.10 17965 36.87 0.09 0 11=0A= 8.07 2959 23.31 0.00 0 0.00 2.67 8938 23.31 0.86 32 72=0A= 7.97 7088 55.18 0.00 0 0.00 2.66 21233 55.18 1.05 16 98=0A= 7.98 4666 36.38 0.00 0 0.00 2.66 13986 36.38 0.36 9 29=0A= 8.00 4513 35.24 8.00 7662 59.86 2.20 44188 95.10 0.27 10 49=0A= 7.98 4799 37.40 8.00 11422 89.23 2.16 60076 126.63 0.19 0 51=0A= 8.00 4322 33.76 0.00 0 0.00 2.67 12967 33.76 0.89 0 42=0A= 8.02 4839 37.91 0.00 0 0.00 2.67 14550 37.91 0.54 17 41=0A= 8.01 4516 35.32 0.00 0 0.00 2.67 13569 35.32 0.57 27 38=0A= 7.95 4459 34.62 8.00 1195 9.34 2.49 18109 43.96 0.55 0 45=0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A= =0A= =0A= =0A= On Thu, Apr 15, 2021 at 9:07 PM Olav Gjerde > wrote:=0A= =0A= > I have the same issue, using Ubuntu 20.10 with Linux 5.8 kernel. The Linu= x=0A= > NFS client will get unresponsive and it does not recover in my case, even= =0A= > if I restart NFS on FreeBSD. I upgraded from FreeBSD 12.1-RELEASE though.= =0A= >=0A= > On Thu, Apr 15, 2021 at 8:36 PM Allan Jude > wrote:=0A= >=0A= >> On 4/15/2021 9:22 AM, Chris Roose wrote:=0A= >> > I posted this in -questions and someone suggested I post here as well.= =0A= >> >=0A= >> > I'm having NFS availability issues between my Proxmox client and=0A= >> FreeBSD server (10G link) since upgrading to 13-RELEASE. And unfortunate= ly=0A= >> I upgraded my ZFS pool to v2.0.0 before I noticed the issue, so I'm kind= of=0A= >> stuck.=0A= >> >=0A= >> > Periodically, the NFS server (I've tried both v3 and v4.2 clients) wil= l=0A= >> go unresponsive for several minutes. I never had this problem on 12.2, a= nd=0A= >> as far as I can tell it's not a disk or network I/O issue. I'll get seve= ral=0A= >> "nfs: server not responding, still trying" messages on the client and a = few=0A= >> minutes later it usually recovers. It's not clear to me yet what's causi= ng=0A= >> the block. Restarting nfsd on the server will resolve the issue if it=0A= >> doesn't clear itself.=0A= >> >=0A= >> > Any pointers for troubleshooting this? I've been looking through=0A= >> vmstat, gstat, top, etc. when the problem occurs, but I haven't been abl= e=0A= >> to pinpoint the issue. I can get pcap, but it would be from the hosts,= =0A= >> because I don't have a 10G tap or managed switch.=0A= >> >=0A= >>=0A= >> run `nfsstat -d 1` and try to capture a few lines from before, during,= =0A= >> and after the stall, and that may provide some insight.=0A= >>=0A= >> Specifically, does the queue length grow, suggesting it is waiting on=0A= >> the I/O subsystem, or does it just stop getting traffic all together.=0A= >>=0A= >>=0A= >> --=0A= >> Allan Jude=0A= >> _______________________________________________=0A= >> freebsd-current@freebsd.org mailing = list=0A= >> https://lists.freebsd.org/mailman/listinfo/freebsd-current=0A= >> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.or= g=0A= >> "=0A= >>=0A= >=0A= >=0A= > --=0A= > Kind Regards / Med Vennlig Hilsen=0A= >=0A= > Olav Gr=F8n=E5s Gjerde=0A= >=0A= > BackupBay Gjerde=0A= > Madlaforen 35=0A= > 4042 HAFRSFJORD=0A= > Norway=0A= > Phone: +47 918 000 59=0A= >=0A= =0A= =0A= --=0A= Kind Regards / Med Vennlig Hilsen=0A= =0A= Olav Gr=F8n=E5s Gjerde=0A= =0A= BackupBay Gjerde=0A= Madlaforen 35=0A= 4042 HAFRSFJORD=0A= Norway=0A= Phone: +47 918 000 59=0A= _______________________________________________=0A= freebsd-current@freebsd.org mailing lis= t=0A= https://lists.freebsd.org/mailman/listinfo/freebsd-current=0A= To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"=0A= =0A= =0A= --=0A= Kind Regards / Med Vennlig Hilsen=0A= =0A= Olav Gr=F8n=E5s Gjerde=0A= =0A= BackupBay Gjerde=0A= Madlaforen 35=0A= 4042 HAFRSFJORD=0A= Norway=0A= Phone: +47 918 000 59=0A= --_002_YQXPR0101MB0968924C8AEAB808888D75B1DD499YQXPR0101MB0968_ Content-Type: application/octet-stream; name="r367492-revert.patch" Content-Description: r367492-revert.patch Content-Disposition: attachment; filename="r367492-revert.patch"; size=8952; creation-date="Mon, 19 Apr 2021 15:03:30 GMT"; modification-date="Mon, 19 Apr 2021 15:03:30 GMT" Content-Transfer-Encoding: base64 LS0tIHN5cy9uZXRpbmV0L3RjcF9zdGFja3MvYmJyLmMub3JpZwkyMDIxLTA0LTA5IDA4OjI0OjA1 LjM2NDIxMjAwMCAtMDcwMAorKysgc3lzL25ldGluZXQvdGNwX3N0YWNrcy9iYnIuYwkyMDIxLTA0 LTA5IDA4OjMzOjQ5Ljc5OTkwMjAwMCAtMDcwMApAQCAtNzg3Niw4ICs3ODc2LDcgQEAgYmJyX3By b2Nlc3NfYWNrKHN0cnVjdCBtYnVmICptLCBzdHJ1Y3QgdGNwaGRyICp0aCwgc3RyCiAJYWNrZWRf YW1vdW50ID0gbWluKGFja2VkLCAoaW50KXNiYXZhaWwoJnNvLT5zb19zbmQpKTsKIAl0cC0+c25k X3duZCAtPSBhY2tlZF9hbW91bnQ7CiAJbWZyZWUgPSBzYmN1dF9sb2NrZWQoJnNvLT5zb19zbmQs IGFja2VkX2Ftb3VudCk7Ci0JU09DS0JVRl9VTkxPQ0soJnNvLT5zb19zbmQpOwotCXRwLT50X2Zs YWdzIHw9IFRGX1dBS0VTT1c7CisJc293d2FrZXVwX2xvY2tlZChzbyk7CiAJbV9mcmVlbShtZnJl ZSk7CiAJaWYgKFNFUV9HVCh0aC0+dGhfYWNrLCB0cC0+c25kX3VuYSkpIHsKIAkJYmJyX2NvbGxh cHNlX3J0dCh0cCwgYmJyLCBUQ1BfUkVYTVRWQUwodHApKTsKQEAgLTgzNTMsOCArODM1Miw3IEBA IGJicl9wcm9jZXNzX2RhdGEoc3RydWN0IG1idWYgKm0sIHN0cnVjdCB0Y3BoZHIgKnRoLCBzdAog CQkJCWFwcGVuZGVkID0KICNlbmRpZgogCQkJCQlzYmFwcGVuZHN0cmVhbV9sb2NrZWQoJnNvLT5z b19yY3YsIG0sIDApOwotCQkJU09DS0JVRl9VTkxPQ0soJnNvLT5zb19yY3YpOwotCQkJdHAtPnRf ZmxhZ3MgfD0gVEZfV0FLRVNPUjsKKwkJCXNvcndha2V1cF9sb2NrZWQoc28pOwogI2lmZGVmIE5F VEZMSVhfU0JfTElNSVRTCiAJCQlpZiAoc28tPnNvX3Jjdi5zYl9zaGxpbSAmJiBhcHBlbmRlZCAh PSBtY250KQogCQkJCWNvdW50ZXJfZm9fcmVsZWFzZShzby0+c29fcmN2LnNiX3NobGltLApAQCAt ODQxNCw4ICs4NDEyLDYgQEAgYmJyX3Byb2Nlc3NfZGF0YShzdHJ1Y3QgbWJ1ZiAqbSwgc3RydWN0 IHRjcGhkciAqdGgsIHN0CiAJaWYgKHRoZmxhZ3MgJiBUSF9GSU4pIHsKIAkJaWYgKFRDUFNfSEFW RVJDVkRGSU4odHAtPnRfc3RhdGUpID09IDApIHsKIAkJCXNvY2FudHJjdm1vcmUoc28pOwotCQkJ LyogVGhlIHNvY2tldCB1cGNhbGwgaXMgaGFuZGxlZCBieSBzb2NhbnRyY3Ztb3JlLiAqLwotCQkJ dHAtPnRfZmxhZ3MgJj0gflRGX1dBS0VTT1I7CiAJCQkvKgogCQkJICogSWYgY29ubmVjdGlvbiBp cyBoYWxmLXN5bmNocm9uaXplZCAoaWUgTkVFRFNZTgogCQkJICogZmxhZyBvbikgdGhlbiBkZWxh eSBBQ0ssIHNvIGl0IG1heSBiZSBwaWdneWJhY2tlZApAQCAtODYwNiw4ICs4NjAyLDcgQEAgYmJy X2RvX2Zhc3RuZXdkYXRhKHN0cnVjdCBtYnVmICptLCBzdHJ1Y3QgdGNwaGRyICp0aCwgCiAJCQlz YmFwcGVuZHN0cmVhbV9sb2NrZWQoJnNvLT5zb19yY3YsIG0sIDApOwogCQljdGZfY2FsY19yd2lu KHNvLCB0cCk7CiAJfQotCVNPQ0tCVUZfVU5MT0NLKCZzby0+c29fcmN2KTsKLQl0cC0+dF9mbGFn cyB8PSBURl9XQUtFU09SOworCXNvcndha2V1cF9sb2NrZWQoc28pOwogI2lmZGVmIE5FVEZMSVhf U0JfTElNSVRTCiAJaWYgKHNvLT5zb19yY3Yuc2Jfc2hsaW0gJiYgbWNudCAhPSBhcHBlbmRlZCkK IAkJY291bnRlcl9mb19yZWxlYXNlKHNvLT5zb19yY3Yuc2Jfc2hsaW0sIG1jbnQgLSBhcHBlbmRl ZCk7CkBAIC04Nzk4LDcgKzg3OTMsNyBAQCBiYnJfZmFzdGFjayhzdHJ1Y3QgbWJ1ZiAqbSwgc3Ry dWN0IHRjcGhkciAqdGgsIHN0cnVjdCAKIAkJICAgICZ0Y3Bfc2F2ZXRjcCwgMCk7CiAjZW5kaWYK IAkvKiBXYWtlIHVwIHRoZSBzb2NrZXQgaWYgd2UgaGF2ZSByb29tIHRvIHdyaXRlIG1vcmUgKi8K LQl0cC0+dF9mbGFncyB8PSBURl9XQUtFU09XOworCXNvd3dha2V1cChzbyk7CiAJaWYgKHRwLT5z bmRfdW5hID09IHRwLT5zbmRfbWF4KSB7CiAJCS8qIE5vdGhpbmcgbGVmdCBvdXRzdGFuZGluZyAq LwogCQliYnJfbG9nX3Byb2dyZXNzX2V2ZW50KGJiciwgdHAsIHRpY2tzLCBQUk9HUkVTU19DTEVB UiwgX19MSU5FX18pOwpAQCAtMTE3NTQsMTAgKzExNzQ5LDggQEAgYmJyX2RvX3NlZ21lbnQoc3Ry dWN0IG1idWYgKm0sIHN0cnVjdCB0Y3BoZHIgKnRoLCBzdHJ1CiAJfQogCXJldHZhbCA9IGJicl9k b19zZWdtZW50X25vdW5sb2NrKG0sIHRoLCBzbywgdHAsCiAJCQkJCSBkcm9wX2hkcmxlbiwgdGxl biwgaXB0b3MsIDAsICZ0dik7Ci0JaWYgKHJldHZhbCA9PSAwKSB7Ci0JCXRjcF9oYW5kbGVfd2Fr ZXVwKHRwLCBzbyk7CisJaWYgKHJldHZhbCA9PSAwKQogCQlJTlBfV1VOTE9DSyh0cC0+dF9pbnBj Yik7Ci0JfQogfQogCiAvKgotLS0gc3lzL25ldGluZXQvdGNwX3N0YWNrcy9yYWNrLmMub3JpZwky MDIxLTA0LTA5IDA4OjM2OjIzLjYyMjgyMTAwMCAtMDcwMAorKysgc3lzL25ldGluZXQvdGNwX3N0 YWNrcy9yYWNrLmMJMjAyMS0wNC0wOSAwODo0MToyNC4wOTY2ODcwMDAgLTA3MDAKQEAgLTgzNDQs OCArODM0NCw3IEBAIHJhY2tfcHJvY2Vzc19hY2soc3RydWN0IG1idWYgKm0sIHN0cnVjdCB0Y3Bo ZHIgKnRoLCBzdAogCQkgKi8KIAkJb3VyZmluaXNhY2tlZCA9IDE7CiAJfQotCVNPQ0tCVUZfVU5M T0NLKCZzby0+c29fc25kKTsKLQl0cC0+dF9mbGFncyB8PSBURl9XQUtFU09XOworCXNvd3dha2V1 cF9sb2NrZWQoc28pOwogCW1fZnJlZW0obWZyZWUpOwogCWlmIChyYWNrLT5yX2N0bC5yY19lYXJs eV9yZWNvdmVyeSA9PSAwKSB7CiAJCWlmIChJTl9SRUNPVkVSWSh0cC0+dF9mbGFncykpIHsKQEAg LTg2NjUsOCArODY2NCw3IEBAIHJhY2tfcHJvY2Vzc19kYXRhKHN0cnVjdCBtYnVmICptLCBzdHJ1 Y3QgdGNwaGRyICp0aCwgcwogCQkJCWFwcGVuZGVkID0KICNlbmRpZgogCQkJCQlzYmFwcGVuZHN0 cmVhbV9sb2NrZWQoJnNvLT5zb19yY3YsIG0sIDApOwotCQkJU09DS0JVRl9VTkxPQ0soJnNvLT5z b19yY3YpOwotCQkJdHAtPnRfZmxhZ3MgfD0gVEZfV0FLRVNPUjsKKwkJCXNvcndha2V1cF9sb2Nr ZWQoc28pOwogI2lmZGVmIE5FVEZMSVhfU0JfTElNSVRTCiAJCQlpZiAoc28tPnNvX3Jjdi5zYl9z aGxpbSAmJiBhcHBlbmRlZCAhPSBtY250KQogCQkJCWNvdW50ZXJfZm9fcmVsZWFzZShzby0+c29f cmN2LnNiX3NobGltLApAQCAtODczMSw4ICs4NzI5LDYgQEAgcmFja19wcm9jZXNzX2RhdGEoc3Ry dWN0IG1idWYgKm0sIHN0cnVjdCB0Y3BoZHIgKnRoLCBzCiAJaWYgKHRoZmxhZ3MgJiBUSF9GSU4p IHsKIAkJaWYgKFRDUFNfSEFWRVJDVkRGSU4odHAtPnRfc3RhdGUpID09IDApIHsKIAkJCXNvY2Fu dHJjdm1vcmUoc28pOwotCQkJLyogVGhlIHNvY2tldCB1cGNhbGwgaXMgaGFuZGxlZCBieSBzb2Nh bnRyY3Ztb3JlLiAqLwotCQkJdHAtPnRfZmxhZ3MgJj0gflRGX1dBS0VTT1I7CiAJCQkvKgogCQkJ ICogSWYgY29ubmVjdGlvbiBpcyBoYWxmLXN5bmNocm9uaXplZCAoaWUgTkVFRFNZTgogCQkJICog ZmxhZyBvbikgdGhlbiBkZWxheSBBQ0ssIHNvIGl0IG1heSBiZSBwaWdneWJhY2tlZApAQCAtODky NCw4ICs4OTIwLDcgQEAgcmFja19kb19mYXN0bmV3ZGF0YShzdHJ1Y3QgbWJ1ZiAqbSwgc3RydWN0 IHRjcGhkciAqdGgsCiAJCQlzYmFwcGVuZHN0cmVhbV9sb2NrZWQoJnNvLT5zb19yY3YsIG0sIDAp OwogCQljdGZfY2FsY19yd2luKHNvLCB0cCk7CiAJfQotCVNPQ0tCVUZfVU5MT0NLKCZzby0+c29f cmN2KTsKLQl0cC0+dF9mbGFncyB8PSBURl9XQUtFU09SOworCXNvcndha2V1cF9sb2NrZWQoc28p OwogI2lmZGVmIE5FVEZMSVhfU0JfTElNSVRTCiAJaWYgKHNvLT5zb19yY3Yuc2Jfc2hsaW0gJiYg bWNudCAhPSBhcHBlbmRlZCkKIAkJY291bnRlcl9mb19yZWxlYXNlKHNvLT5zb19yY3Yuc2Jfc2hs aW0sIG1jbnQgLSBhcHBlbmRlZCk7CkBAIC05MTQyLDcgKzkxMzcsNyBAQCByYWNrX2Zhc3RhY2so c3RydWN0IG1idWYgKm0sIHN0cnVjdCB0Y3BoZHIgKnRoLCBzdHJ1Y3QKIAkJcmFja190aW1lcl9j YW5jZWwodHAsIHJhY2ssIHJhY2stPnJfY3RsLnJjX3JjdnRpbWUsIF9fTElORV9fKTsKIAl9CiAJ LyogV2FrZSB1cCB0aGUgc29ja2V0IGlmIHdlIGhhdmUgcm9vbSB0byB3cml0ZSBtb3JlICovCi0J dHAtPnRfZmxhZ3MgfD0gVEZfV0FLRVNPVzsKKwlzb3d3YWtldXAoc28pOwogCWlmIChzYmF2YWls KCZzby0+c29fc25kKSkgewogCQlyYWNrLT5yX3dhbnRlZF9vdXRwdXQgPSAxOwogCX0KQEAgLTEx MjA1LDEwICsxMTIwMCw4IEBAIHJhY2tfZG9fc2VnbWVudChzdHJ1Y3QgbWJ1ZiAqbSwgc3RydWN0 IHRjcGhkciAqdGgsIHN0cgogCQl0Y3BfZ2V0X3VzZWNzKCZ0dik7CiAJfQogCWlmKHJhY2tfZG9f c2VnbWVudF9ub3VubG9jayhtLCB0aCwgc28sIHRwLAotCQkJCSAgICBkcm9wX2hkcmxlbiwgdGxl biwgaXB0b3MsIDAsICZ0dikgPT0gMCkgewotCQl0Y3BfaGFuZGxlX3dha2V1cCh0cCwgc28pOwor CQkJCSAgICBkcm9wX2hkcmxlbiwgdGxlbiwgaXB0b3MsIDAsICZ0dikgPT0gMCkKIAkJSU5QX1dV TkxPQ0sodHAtPnRfaW5wY2IpOwotCX0KIH0KIAogc3RydWN0IHJhY2tfc2VuZG1hcCAqCi0tLSBz eXMvbmV0aW5ldC90Y3Bfc3RhY2tzL3JhY2tfYmJyX2NvbW1vbi5jLm9yaWcJMjAyMS0wNC0wOSAw ODo0NToyNi43MjE1MjEwMDAgLTA3MDAKKysrIHN5cy9uZXRpbmV0L3RjcF9zdGFja3MvcmFja19i YnJfY29tbW9uLmMJMjAyMS0wNC0wOSAwODo0Njo1OC41ODAyMzQwMDAgLTA3MDAKQEAgLTQ1OCw3 ICs0NTgsNiBAQCBjdGZfZG9fcXVldWVkX3NlZ21lbnRzKHN0cnVjdCBzb2NrZXQgKnNvLCBzdHJ1 Y3QgdGNwY2IKIAkJCS8qIFdlIGxvc3QgdGhlIHRjcGNiIChtYXliZSBhIFJTVCBjYW1lIGluKT8g Ki8KIAkJCXJldHVybigxKTsKIAkJfQotCQl0Y3BfaGFuZGxlX3dha2V1cCh0cCwgc28pOwogCX0K IAlyZXR1cm4gKDApOwogfQotLS0gc3lzL25ldGluZXQvdGNwX2lucHV0LmMub3JpZwkyMDIxLTA0 LTA1IDAxOjA3OjAwLjM0MjU1OTAwMCAtMDcwMAorKysgc3lzL25ldGluZXQvdGNwX2lucHV0LmMJ MjAyMS0wNC0wOSAwNzo1ODowMy4yNjI4MTUwMDAgLTA3MDAKQEAgLTE0NzIsMjkgKzE0NzIsNiBA QCB0Y3BfYXV0b3JjdmJ1ZihzdHJ1Y3QgbWJ1ZiAqbSwgc3RydWN0IHRjcGhkciAqdGgsIHN0cnUK IH0KIAogdm9pZAotdGNwX2hhbmRsZV93YWtldXAoc3RydWN0IHRjcGNiICp0cCwgc3RydWN0IHNv Y2tldCAqc28pCi17Ci0JLyoKLQkgKiBTaW5jZSB0cCBtaWdodCBiZSBnb25lIGlmIHRoZSBzZXNz aW9uIGVudGVyZWQKLQkgKiB0aGUgVElNRV9XQUlUIHN0YXRlIGJlZm9yZSBjb21pbmcgaGVyZSwg d2UgbmVlZAotCSAqIHRvIGNoZWNrIGlmIHRoZSBzb2NrZXQgaXMgc3RpbGwgY29ubmVjdGVkLgot CSAqLwotCWlmICgoc28tPnNvX3N0YXRlICYgU1NfSVNDT05ORUNURUQpID09IDApCi0JCXJldHVy bjsKLQlJTlBfTE9DS19BU1NFUlQodHAtPnRfaW5wY2IpOwotCWlmICh0cC0+dF9mbGFncyAmIFRG X1dBS0VTT1IpIHsKLQkJdHAtPnRfZmxhZ3MgJj0gflRGX1dBS0VTT1I7Ci0JCVNPQ0tCVUZfVU5M T0NLX0FTU0VSVCgmc28tPnNvX3Jjdik7Ci0JCXNvcndha2V1cChzbyk7Ci0JfQotCWlmICh0cC0+ dF9mbGFncyAmIFRGX1dBS0VTT1cpIHsKLQkJdHAtPnRfZmxhZ3MgJj0gflRGX1dBS0VTT1c7Ci0J CVNPQ0tCVUZfVU5MT0NLX0FTU0VSVCgmc28tPnNvX3NuZCk7Ci0JCXNvd3dha2V1cChzbyk7Ci0J fQotfQotCi12b2lkCiB0Y3BfZG9fc2VnbWVudChzdHJ1Y3QgbWJ1ZiAqbSwgc3RydWN0IHRjcGhk ciAqdGgsIHN0cnVjdCBzb2NrZXQgKnNvLAogICAgIHN0cnVjdCB0Y3BjYiAqdHAsIGludCBkcm9w X2hkcmxlbiwgaW50IHRsZW4sIHVpbnQ4X3QgaXB0b3MpCiB7CkBAIC0xODYzLDcgKzE4NDAsNyBA QCB0Y3BfZG9fc2VnbWVudChzdHJ1Y3QgbWJ1ZiAqbSwgc3RydWN0IHRjcGhkciAqdGgsIHN0cnUK IAkJCQllbHNlIGlmICghdGNwX3RpbWVyX2FjdGl2ZSh0cCwgVFRfUEVSU0lTVCkpCiAJCQkJCXRj cF90aW1lcl9hY3RpdmF0ZSh0cCwgVFRfUkVYTVQsCiAJCQkJCQkgICAgICB0cC0+dF9yeHRjdXIp OwotCQkJCXRwLT50X2ZsYWdzIHw9IFRGX1dBS0VTT1c7CisJCQkJc293d2FrZXVwKHNvKTsKIAkJ CQlpZiAoc2JhdmFpbCgmc28tPnNvX3NuZCkpCiAJCQkJCSh2b2lkKSB0cC0+dF9mYi0+dGZiX3Rj cF9vdXRwdXQodHApOwogCQkJCWdvdG8gY2hlY2tfZGVsYWNrOwpAQCAtMTkyOCw4ICsxOTA1LDgg QEAgdGNwX2RvX3NlZ21lbnQoc3RydWN0IG1idWYgKm0sIHN0cnVjdCB0Y3BoZHIgKnRoLCBzdHJ1 CiAJCQkJbV9hZGoobSwgZHJvcF9oZHJsZW4pOwkvKiBkZWxheWVkIGhlYWRlciBkcm9wICovCiAJ CQkJc2JhcHBlbmRzdHJlYW1fbG9ja2VkKCZzby0+c29fcmN2LCBtLCAwKTsKIAkJCX0KLQkJCVNP Q0tCVUZfVU5MT0NLKCZzby0+c29fcmN2KTsKLQkJCXRwLT50X2ZsYWdzIHw9IFRGX1dBS0VTT1I7 CisJCQkvKiBOQjogc29yd2FrZXVwX2xvY2tlZCgpIGRvZXMgYW4gaW1wbGljaXQgdW5sb2NrLiAq LworCQkJc29yd2FrZXVwX2xvY2tlZChzbyk7CiAJCQlpZiAoREVMQVlfQUNLKHRwLCB0bGVuKSkg ewogCQkJCXRwLT50X2ZsYWdzIHw9IFRGX0RFTEFDSzsKIAkJCX0gZWxzZSB7CkBAIC0yOTI1LDgg KzI5MDIsOCBAQCBwcm9jZXNzX0FDSzoKIAkJCQl0cC0+c25kX3duZCA9IDA7CiAJCQlvdXJmaW5p c2Fja2VkID0gMDsKIAkJfQotCQlTT0NLQlVGX1VOTE9DSygmc28tPnNvX3NuZCk7Ci0JCXRwLT50 X2ZsYWdzIHw9IFRGX1dBS0VTT1c7CisJCS8qIE5COiBzb3d3YWtldXBfbG9ja2VkKCkgZG9lcyBh biBpbXBsaWNpdCB1bmxvY2suICovCisJCXNvd3dha2V1cF9sb2NrZWQoc28pOwogCQltX2ZyZWVt KG1mcmVlKTsKIAkJLyogRGV0ZWN0IHVuYSB3cmFwYXJvdW5kLiAqLwogCQlpZiAoIUlOX1JFQ09W RVJZKHRwLT50X2ZsYWdzKSAmJgpAQCAtMzE0Nyw4ICszMTI0LDggQEAgZG9kYXRhOgkJCQkJCQkv KiBYWFggKi8KIAkJCQltX2ZyZWVtKG0pOwogCQkJZWxzZQogCQkJCXNiYXBwZW5kc3RyZWFtX2xv Y2tlZCgmc28tPnNvX3JjdiwgbSwgMCk7Ci0JCQlTT0NLQlVGX1VOTE9DSygmc28tPnNvX3Jjdik7 Ci0JCQl0cC0+dF9mbGFncyB8PSBURl9XQUtFU09SOworCQkJLyogTkI6IHNvcndha2V1cF9sb2Nr ZWQoKSBkb2VzIGFuIGltcGxpY2l0IHVubG9jay4gKi8KKwkJCXNvcndha2V1cF9sb2NrZWQoc28p OwogCQl9IGVsc2UgewogCQkJLyoKIAkJCSAqIFhYWDogRHVlIHRvIHRoZSBoZWFkZXIgZHJvcCBh Ym92ZSAidGgiIGlzCkBAIC0zMjE1LDggKzMxOTIsNiBAQCBkb2RhdGE6CQkJCQkJCS8qIFhYWCAq LwogCWlmICh0aGZsYWdzICYgVEhfRklOKSB7CiAJCWlmIChUQ1BTX0hBVkVSQ1ZERklOKHRwLT50 X3N0YXRlKSA9PSAwKSB7CiAJCQlzb2NhbnRyY3Ztb3JlKHNvKTsKLQkJCS8qIFRoZSBzb2NrZXQg dXBjYWxsIGlzIGhhbmRsZWQgYnkgc29jYW50cmN2bW9yZS4gKi8KLQkJCXRwLT50X2ZsYWdzICY9 IH5URl9XQUtFU09SOwogCQkJLyoKIAkJCSAqIElmIGNvbm5lY3Rpb24gaXMgaGFsZi1zeW5jaHJv bml6ZWQKIAkJCSAqIChpZSBORUVEU1lOIGZsYWcgb24pIHRoZW4gZGVsYXkgQUNLLApAQCAtMzI4 MCw3ICszMjU1LDYgQEAgY2hlY2tfZGVsYWNrOgogCQl0cC0+dF9mbGFncyAmPSB+VEZfREVMQUNL OwogCQl0Y3BfdGltZXJfYWN0aXZhdGUodHAsIFRUX0RFTEFDSywgdGNwX2RlbGFja3RpbWUpOwog CX0KLQl0Y3BfaGFuZGxlX3dha2V1cCh0cCwgc28pOwogCUlOUF9XVU5MT0NLKHRwLT50X2lucGNi KTsKIAlyZXR1cm47CiAKQEAgLTMzMTQsNyArMzI4OCw2IEBAIGRyb3BhZnRlcmFjazoKIAlUQ1Bf UFJPQkUzKGRlYnVnX19pbnB1dCwgdHAsIHRoLCBtKTsKIAl0cC0+dF9mbGFncyB8PSBURl9BQ0tO T1c7CiAJKHZvaWQpIHRwLT50X2ZiLT50ZmJfdGNwX291dHB1dCh0cCk7Ci0JdGNwX2hhbmRsZV93 YWtldXAodHAsIHNvKTsKIAlJTlBfV1VOTE9DSyh0cC0+dF9pbnBjYik7CiAJbV9mcmVlbShtKTsK IAlyZXR1cm47CkBAIC0zMzIyLDcgKzMyOTUsNiBAQCBkcm9wYWZ0ZXJhY2s6CiBkcm9wd2l0aHJl c2V0OgogCWlmICh0cCAhPSBOVUxMKSB7CiAJCXRjcF9kcm9wd2l0aHJlc2V0KG0sIHRoLCB0cCwg dGxlbiwgcnN0cmVhc29uKTsKLQkJdGNwX2hhbmRsZV93YWtldXAodHAsIHNvKTsKIAkJSU5QX1dV TkxPQ0sodHAtPnRfaW5wY2IpOwogCX0gZWxzZQogCQl0Y3BfZHJvcHdpdGhyZXNldChtLCB0aCwg TlVMTCwgdGxlbiwgcnN0cmVhc29uKTsKQEAgLTMzMzgsMTAgKzMzMTAsOCBAQCBkcm9wOgogCQkJ ICAmdGNwX3NhdmV0Y3AsIDApOwogI2VuZGlmCiAJVENQX1BST0JFMyhkZWJ1Z19faW5wdXQsIHRw LCB0aCwgbSk7Ci0JaWYgKHRwICE9IE5VTEwpIHsKLQkJdGNwX2hhbmRsZV93YWtldXAodHAsIHNv KTsKKwlpZiAodHAgIT0gTlVMTCkKIAkJSU5QX1dVTkxPQ0sodHAtPnRfaW5wY2IpOwotCX0KIAlt X2ZyZWVtKG0pOwogfQogCi0tLSBzeXMvbmV0aW5ldC90Y3BfcmVhc3MuYy5vcmlnCTIwMjEtMDQt MDkgMDg6MTg6MTAuNTk5MDkyMDAwIC0wNzAwCisrKyBzeXMvbmV0aW5ldC90Y3BfcmVhc3MuYwky MDIxLTA0LTA5IDA4OjE5OjU0LjkxMjM3ODAwMCAtMDcwMApAQCAtOTU5LDggKzk1OSw3IEBAIG5l d19lbnRyeToKIAkJfSBlbHNlIHsKIAkJCXNiYXBwZW5kc3RyZWFtX2xvY2tlZCgmc28tPnNvX3Jj diwgbSwgMCk7CiAJCX0KLQkJU09DS0JVRl9VTkxPQ0soJnNvLT5zb19yY3YpOwotCQl0cC0+dF9m bGFncyB8PSBURl9XQUtFU09SOworCQlzb3J3YWtldXBfbG9ja2VkKHNvKTsKIAkJcmV0dXJuIChm bGFncyk7CiAJfQogCWlmICh0Y3BfbmV3X2xpbWl0cykgewpAQCAtMTEwOCw3ICsxMTA3LDYgQEAg cHJlc2VudDoKICNpZmRlZiBUQ1BfUkVBU1NfTE9HR0lORwogCXRjcF9yZWFzc19sb2dfZHVtcCh0 cCk7CiAjZW5kaWYKLQlTT0NLQlVGX1VOTE9DSygmc28tPnNvX3Jjdik7Ci0JdHAtPnRfZmxhZ3Mg fD0gVEZfV0FLRVNPUjsKKwlzb3J3YWtldXBfbG9ja2VkKHNvKTsKIAlyZXR1cm4gKGZsYWdzKTsK IH0K --_002_YQXPR0101MB0968924C8AEAB808888D75B1DD499YQXPR0101MB0968_--