Date: Mon, 19 Apr 2021 15:03:35 +0000 From: Rick Macklem <rmacklem@uoguelph.ca> To: Olav Gjerde <olav@backupbay.com> Cc: Allan Jude <allanjude@freebsd.org>, "freebsd-current@freebsd.org" <freebsd-current@freebsd.org> Subject: Re: NFS issues since upgrading to 13-RELEASE Message-ID: <YQXPR0101MB0968924C8AEAB808888D75B1DD499@YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM> In-Reply-To: <CAJ7kQyFBoSePYA%2Bz60ZfNpOGn6wspy-xPh4wOypw5U5RL9VUJw@mail.gmail.com> References: <902a3c81-2ce8-49c0-b163-5ffa4b90afe5@www.fastmail.com> <e8f585eb-a2a8-ae9d-7f33-526e412ec462@freebsd.org> <CAJ7kQyGQrxe7wJs%2BMezErdNUoLE1HEvD6ixiu2W5=irDNg2mOA@mail.gmail.com> <CAJ7kQyExd6LLg5mb4V_o9uR1fmEqz3P5JkUFuQ6XYjegba1toA@mail.gmail.com> <YQXPR0101MB096888C9CDFA3BC26833AFB2DD4C9@YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM>, <CAJ7kQyFBoSePYA%2Bz60ZfNpOGn6wspy-xPh4wOypw5U5RL9VUJw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--_002_YQXPR0101MB0968924C8AEAB808888D75B1DD499YQXPR0101MB0968_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Olav Gjerde wrote:=0A=
>I have tried D29690 patch and reverting back to r367492 this weekend. Neit=
her made any difference for my system.=0A=
Just to clarify it, I meant "revert the patch in r367492" and not=0A=
"revert to revision r367492". I've attached the patch that=0A=
backs out the changes made by the patch in r367492, which=0A=
should apply to a fairly recent main/13 kernel.=0A=
=0A=
This should be done instead of applying D29690, not combined with it.=0A=
My testing of D29690 has suggested it is not yet mature, so=0A=
I would not recommend choosing that alternative yet.=0A=
=0A=
If you have tried a kernel with the attached patch applied to it, but not=
=0A=
D29690 applied to it, then please:=0A=
Let us know if you still have Linux clients "hanging" with this kernel.=0A=
If still "hanging", try the following to see if they help:=0A=
- Use the "minorversion=3D1" mount option on the Linux clients,=0A=
to ensure that they are not using NFSv4.2, to see if it is a=0A=
NFSv4.2 specific issue.=0A=
- Try disabling tso and lro and avoid jumbo frames for drivers=0A=
that use jumbo mbufs when handling jumbo frames.=0A=
Collect the following info when it happens:=0A=
- "netstat -a", to see what the TCP connection is up to.=0A=
- "tcpdump -s 0 -w hang.pcap host <nfs-client>"=0A=
run for several minutes on the server, to see what is going on the=0A=
wire. I use wireshark to look at hang.pcap, since it=0A=
knows NFS as well as TCP.=0A=
You can also do the above with "host <nfs-server>" instead=0A=
of "host <nfs-client>" run on the client.=0A=
- "ps axHl" on the server, to see what the nfsd threads=0A=
are up to.=0A=
If none of the above contains confidential info, please=0A=
send it to me, if not the list.=0A=
=0A=
Good luck with it, rick=0A=
ps: Yea, I started this post and then realized I had hit=0A=
reply instead of reply all.=0A=
=0A=
=0A=
=0A=
There is also a reddit thread about this https://www.reddit.com/r/freebsd/c=
omments/mqol4o/nfs_issues_since_upgrading_to_13release/=0A=
=0A=
On Sat, Apr 17, 2021 at 1:10 AM Rick Macklem <rmacklem@uoguelph.ca<mailto:r=
macklem@uoguelph.ca>> wrote:=0A=
Just fyi, I just got a "recursed on non-recursed mutex" panic in=0A=
socantrcvmore() with the D29690 patch, so you might not=0A=
want to test with that one yet.=0A=
=0A=
rick=0A=
=0A=
________________________________________=0A=
From: owner-freebsd-current@freebsd.org<mailto:owner-freebsd-current@freebs=
d.org> <owner-freebsd-current@freebsd.org<mailto:owner-freebsd-current@free=
bsd.org>> on behalf of Olav Gjerde <olav@backupbay.com<mailto:olav@backupba=
y.com>>=0A=
Sent: Thursday, April 15, 2021 3:21 PM=0A=
To: Allan Jude=0A=
Cc: freebsd-current@freebsd.org<mailto:freebsd-current@freebsd.org>=0A=
Subject: Re: NFS issues since upgrading to 13-RELEASE=0A=
=0A=
CAUTION: This email originated from outside of the University of Guelph. Do=
not click links or open attachments unless you recognize the sender and kn=
ow the content is safe. If in doubt, forward suspicious emails to IThelp@uo=
guelph.ca<mailto:IThelp@uoguelph.ca>=0A=
=0A=
=0A=
Well something do happen if I restart NFS Service on FreeBSD , it works for=
=0A=
like 10 seconds then it gets unresponsive again.=0A=
=0A=
This is my output from `nfsstat -d 1`=0A=
=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
8.00 1025 8.00 8.02 17170 134.54 2.01 72716 142.54 0.07 51 34=0A=
8.00 2273 17.76 7.99 31273 244.07 2.01 133267 261.83 0.14 20 82=
=0A=
8.03 4889 38.33 7.99 25885 202.07 2.06 119340 240.40 0.13 21 81=
=0A=
[=3D=3D=3D=3D=3D Read =3D=3D=3D=3D=3D] [=3D=3D=3D=3D=3D Write =3D=3D=3D=3D=
] [=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Total =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D]=0A=
KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s ms ql %b=0A=
7.98 8811 68.64 8.00 12997 101.54 2.22 78396 170.18 0.15 1 80=0A=
7.99 922 7.20 8.00 3798 29.68 2.10 17965 36.87 0.09 0 11=0A=
8.07 2959 23.31 0.00 0 0.00 2.67 8938 23.31 0.86 32 72=0A=
7.97 7088 55.18 0.00 0 0.00 2.66 21233 55.18 1.05 16 98=0A=
7.98 4666 36.38 0.00 0 0.00 2.66 13986 36.38 0.36 9 29=0A=
8.00 4513 35.24 8.00 7662 59.86 2.20 44188 95.10 0.27 10 49=0A=
7.98 4799 37.40 8.00 11422 89.23 2.16 60076 126.63 0.19 0 51=0A=
8.00 4322 33.76 0.00 0 0.00 2.67 12967 33.76 0.89 0 42=0A=
8.02 4839 37.91 0.00 0 0.00 2.67 14550 37.91 0.54 17 41=0A=
8.01 4516 35.32 0.00 0 0.00 2.67 13569 35.32 0.57 27 38=0A=
7.95 4459 34.62 8.00 1195 9.34 2.49 18109 43.96 0.55 0 45=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0=0A=
=0A=
=0A=
=0A=
On Thu, Apr 15, 2021 at 9:07 PM Olav Gjerde <olav@backupbay.com<mailto:olav=
@backupbay.com>> wrote:=0A=
=0A=
> I have the same issue, using Ubuntu 20.10 with Linux 5.8 kernel. The Linu=
x=0A=
> NFS client will get unresponsive and it does not recover in my case, even=
=0A=
> if I restart NFS on FreeBSD. I upgraded from FreeBSD 12.1-RELEASE though.=
=0A=
>=0A=
> On Thu, Apr 15, 2021 at 8:36 PM Allan Jude <allanjude@freebsd.org<mailto:=
allanjude@freebsd.org>> wrote:=0A=
>=0A=
>> On 4/15/2021 9:22 AM, Chris Roose wrote:=0A=
>> > I posted this in -questions and someone suggested I post here as well.=
=0A=
>> >=0A=
>> > I'm having NFS availability issues between my Proxmox client and=0A=
>> FreeBSD server (10G link) since upgrading to 13-RELEASE. And unfortunate=
ly=0A=
>> I upgraded my ZFS pool to v2.0.0 before I noticed the issue, so I'm kind=
of=0A=
>> stuck.=0A=
>> >=0A=
>> > Periodically, the NFS server (I've tried both v3 and v4.2 clients) wil=
l=0A=
>> go unresponsive for several minutes. I never had this problem on 12.2, a=
nd=0A=
>> as far as I can tell it's not a disk or network I/O issue. I'll get seve=
ral=0A=
>> "nfs: server not responding, still trying" messages on the client and a =
few=0A=
>> minutes later it usually recovers. It's not clear to me yet what's causi=
ng=0A=
>> the block. Restarting nfsd on the server will resolve the issue if it=0A=
>> doesn't clear itself.=0A=
>> >=0A=
>> > Any pointers for troubleshooting this? I've been looking through=0A=
>> vmstat, gstat, top, etc. when the problem occurs, but I haven't been abl=
e=0A=
>> to pinpoint the issue. I can get pcap, but it would be from the hosts,=
=0A=
>> because I don't have a 10G tap or managed switch.=0A=
>> >=0A=
>>=0A=
>> run `nfsstat -d 1` and try to capture a few lines from before, during,=
=0A=
>> and after the stall, and that may provide some insight.=0A=
>>=0A=
>> Specifically, does the queue length grow, suggesting it is waiting on=0A=
>> the I/O subsystem, or does it just stop getting traffic all together.=0A=
>>=0A=
>>=0A=
>> --=0A=
>> Allan Jude=0A=
>> _______________________________________________=0A=
>> freebsd-current@freebsd.org<mailto:freebsd-current@freebsd.org> mailing =
list=0A=
>> https://lists.freebsd.org/mailman/listinfo/freebsd-current=0A=
>> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.or=
g<mailto:freebsd-current-unsubscribe@freebsd.org>=0A=
>> "=0A=
>>=0A=
>=0A=
>=0A=
> --=0A=
> Kind Regards / Med Vennlig Hilsen=0A=
>=0A=
> Olav Gr=F8n=E5s Gjerde=0A=
>=0A=
> BackupBay Gjerde=0A=
> Madlaforen 35=0A=
> 4042 HAFRSFJORD=0A=
> Norway=0A=
> Phone: +47 918 000 59=0A=
>=0A=
=0A=
=0A=
--=0A=
Kind Regards / Med Vennlig Hilsen=0A=
=0A=
Olav Gr=F8n=E5s Gjerde=0A=
=0A=
BackupBay Gjerde=0A=
Madlaforen 35=0A=
4042 HAFRSFJORD=0A=
Norway=0A=
Phone: +47 918 000 59=0A=
_______________________________________________=0A=
freebsd-current@freebsd.org<mailto:freebsd-current@freebsd.org> mailing lis=
t=0A=
https://lists.freebsd.org/mailman/listinfo/freebsd-current=0A=
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org<m=
ailto:freebsd-current-unsubscribe@freebsd.org>"=0A=
=0A=
=0A=
--=0A=
Kind Regards / Med Vennlig Hilsen=0A=
=0A=
Olav Gr=F8n=E5s Gjerde=0A=
=0A=
BackupBay Gjerde=0A=
Madlaforen 35=0A=
4042 HAFRSFJORD=0A=
Norway=0A=
Phone: +47 918 000 59=0A=
--_002_YQXPR0101MB0968924C8AEAB808888D75B1DD499YQXPR0101MB0968_
Content-Type: application/octet-stream; name="r367492-revert.patch"
Content-Description: r367492-revert.patch
Content-Disposition: attachment; filename="r367492-revert.patch"; size=8952;
creation-date="Mon, 19 Apr 2021 15:03:30 GMT";
modification-date="Mon, 19 Apr 2021 15:03:30 GMT"
Content-Transfer-Encoding: base64
LS0tIHN5cy9uZXRpbmV0L3RjcF9zdGFja3MvYmJyLmMub3JpZwkyMDIxLTA0LTA5IDA4OjI0OjA1
LjM2NDIxMjAwMCAtMDcwMAorKysgc3lzL25ldGluZXQvdGNwX3N0YWNrcy9iYnIuYwkyMDIxLTA0
LTA5IDA4OjMzOjQ5Ljc5OTkwMjAwMCAtMDcwMApAQCAtNzg3Niw4ICs3ODc2LDcgQEAgYmJyX3By
b2Nlc3NfYWNrKHN0cnVjdCBtYnVmICptLCBzdHJ1Y3QgdGNwaGRyICp0aCwgc3RyCiAJYWNrZWRf
YW1vdW50ID0gbWluKGFja2VkLCAoaW50KXNiYXZhaWwoJnNvLT5zb19zbmQpKTsKIAl0cC0+c25k
X3duZCAtPSBhY2tlZF9hbW91bnQ7CiAJbWZyZWUgPSBzYmN1dF9sb2NrZWQoJnNvLT5zb19zbmQs
IGFja2VkX2Ftb3VudCk7Ci0JU09DS0JVRl9VTkxPQ0soJnNvLT5zb19zbmQpOwotCXRwLT50X2Zs
YWdzIHw9IFRGX1dBS0VTT1c7CisJc293d2FrZXVwX2xvY2tlZChzbyk7CiAJbV9mcmVlbShtZnJl
ZSk7CiAJaWYgKFNFUV9HVCh0aC0+dGhfYWNrLCB0cC0+c25kX3VuYSkpIHsKIAkJYmJyX2NvbGxh
cHNlX3J0dCh0cCwgYmJyLCBUQ1BfUkVYTVRWQUwodHApKTsKQEAgLTgzNTMsOCArODM1Miw3IEBA
IGJicl9wcm9jZXNzX2RhdGEoc3RydWN0IG1idWYgKm0sIHN0cnVjdCB0Y3BoZHIgKnRoLCBzdAog
CQkJCWFwcGVuZGVkID0KICNlbmRpZgogCQkJCQlzYmFwcGVuZHN0cmVhbV9sb2NrZWQoJnNvLT5z
b19yY3YsIG0sIDApOwotCQkJU09DS0JVRl9VTkxPQ0soJnNvLT5zb19yY3YpOwotCQkJdHAtPnRf
ZmxhZ3MgfD0gVEZfV0FLRVNPUjsKKwkJCXNvcndha2V1cF9sb2NrZWQoc28pOwogI2lmZGVmIE5F
VEZMSVhfU0JfTElNSVRTCiAJCQlpZiAoc28tPnNvX3Jjdi5zYl9zaGxpbSAmJiBhcHBlbmRlZCAh
PSBtY250KQogCQkJCWNvdW50ZXJfZm9fcmVsZWFzZShzby0+c29fcmN2LnNiX3NobGltLApAQCAt
ODQxNCw4ICs4NDEyLDYgQEAgYmJyX3Byb2Nlc3NfZGF0YShzdHJ1Y3QgbWJ1ZiAqbSwgc3RydWN0
IHRjcGhkciAqdGgsIHN0CiAJaWYgKHRoZmxhZ3MgJiBUSF9GSU4pIHsKIAkJaWYgKFRDUFNfSEFW
RVJDVkRGSU4odHAtPnRfc3RhdGUpID09IDApIHsKIAkJCXNvY2FudHJjdm1vcmUoc28pOwotCQkJ
LyogVGhlIHNvY2tldCB1cGNhbGwgaXMgaGFuZGxlZCBieSBzb2NhbnRyY3Ztb3JlLiAqLwotCQkJ
dHAtPnRfZmxhZ3MgJj0gflRGX1dBS0VTT1I7CiAJCQkvKgogCQkJICogSWYgY29ubmVjdGlvbiBp
cyBoYWxmLXN5bmNocm9uaXplZCAoaWUgTkVFRFNZTgogCQkJICogZmxhZyBvbikgdGhlbiBkZWxh
eSBBQ0ssIHNvIGl0IG1heSBiZSBwaWdneWJhY2tlZApAQCAtODYwNiw4ICs4NjAyLDcgQEAgYmJy
X2RvX2Zhc3RuZXdkYXRhKHN0cnVjdCBtYnVmICptLCBzdHJ1Y3QgdGNwaGRyICp0aCwgCiAJCQlz
YmFwcGVuZHN0cmVhbV9sb2NrZWQoJnNvLT5zb19yY3YsIG0sIDApOwogCQljdGZfY2FsY19yd2lu
KHNvLCB0cCk7CiAJfQotCVNPQ0tCVUZfVU5MT0NLKCZzby0+c29fcmN2KTsKLQl0cC0+dF9mbGFn
cyB8PSBURl9XQUtFU09SOworCXNvcndha2V1cF9sb2NrZWQoc28pOwogI2lmZGVmIE5FVEZMSVhf
U0JfTElNSVRTCiAJaWYgKHNvLT5zb19yY3Yuc2Jfc2hsaW0gJiYgbWNudCAhPSBhcHBlbmRlZCkK
IAkJY291bnRlcl9mb19yZWxlYXNlKHNvLT5zb19yY3Yuc2Jfc2hsaW0sIG1jbnQgLSBhcHBlbmRl
ZCk7CkBAIC04Nzk4LDcgKzg3OTMsNyBAQCBiYnJfZmFzdGFjayhzdHJ1Y3QgbWJ1ZiAqbSwgc3Ry
dWN0IHRjcGhkciAqdGgsIHN0cnVjdCAKIAkJICAgICZ0Y3Bfc2F2ZXRjcCwgMCk7CiAjZW5kaWYK
IAkvKiBXYWtlIHVwIHRoZSBzb2NrZXQgaWYgd2UgaGF2ZSByb29tIHRvIHdyaXRlIG1vcmUgKi8K
LQl0cC0+dF9mbGFncyB8PSBURl9XQUtFU09XOworCXNvd3dha2V1cChzbyk7CiAJaWYgKHRwLT5z
bmRfdW5hID09IHRwLT5zbmRfbWF4KSB7CiAJCS8qIE5vdGhpbmcgbGVmdCBvdXRzdGFuZGluZyAq
LwogCQliYnJfbG9nX3Byb2dyZXNzX2V2ZW50KGJiciwgdHAsIHRpY2tzLCBQUk9HUkVTU19DTEVB
UiwgX19MSU5FX18pOwpAQCAtMTE3NTQsMTAgKzExNzQ5LDggQEAgYmJyX2RvX3NlZ21lbnQoc3Ry
dWN0IG1idWYgKm0sIHN0cnVjdCB0Y3BoZHIgKnRoLCBzdHJ1CiAJfQogCXJldHZhbCA9IGJicl9k
b19zZWdtZW50X25vdW5sb2NrKG0sIHRoLCBzbywgdHAsCiAJCQkJCSBkcm9wX2hkcmxlbiwgdGxl
biwgaXB0b3MsIDAsICZ0dik7Ci0JaWYgKHJldHZhbCA9PSAwKSB7Ci0JCXRjcF9oYW5kbGVfd2Fr
ZXVwKHRwLCBzbyk7CisJaWYgKHJldHZhbCA9PSAwKQogCQlJTlBfV1VOTE9DSyh0cC0+dF9pbnBj
Yik7Ci0JfQogfQogCiAvKgotLS0gc3lzL25ldGluZXQvdGNwX3N0YWNrcy9yYWNrLmMub3JpZwky
MDIxLTA0LTA5IDA4OjM2OjIzLjYyMjgyMTAwMCAtMDcwMAorKysgc3lzL25ldGluZXQvdGNwX3N0
YWNrcy9yYWNrLmMJMjAyMS0wNC0wOSAwODo0MToyNC4wOTY2ODcwMDAgLTA3MDAKQEAgLTgzNDQs
OCArODM0NCw3IEBAIHJhY2tfcHJvY2Vzc19hY2soc3RydWN0IG1idWYgKm0sIHN0cnVjdCB0Y3Bo
ZHIgKnRoLCBzdAogCQkgKi8KIAkJb3VyZmluaXNhY2tlZCA9IDE7CiAJfQotCVNPQ0tCVUZfVU5M
T0NLKCZzby0+c29fc25kKTsKLQl0cC0+dF9mbGFncyB8PSBURl9XQUtFU09XOworCXNvd3dha2V1
cF9sb2NrZWQoc28pOwogCW1fZnJlZW0obWZyZWUpOwogCWlmIChyYWNrLT5yX2N0bC5yY19lYXJs
eV9yZWNvdmVyeSA9PSAwKSB7CiAJCWlmIChJTl9SRUNPVkVSWSh0cC0+dF9mbGFncykpIHsKQEAg
LTg2NjUsOCArODY2NCw3IEBAIHJhY2tfcHJvY2Vzc19kYXRhKHN0cnVjdCBtYnVmICptLCBzdHJ1
Y3QgdGNwaGRyICp0aCwgcwogCQkJCWFwcGVuZGVkID0KICNlbmRpZgogCQkJCQlzYmFwcGVuZHN0
cmVhbV9sb2NrZWQoJnNvLT5zb19yY3YsIG0sIDApOwotCQkJU09DS0JVRl9VTkxPQ0soJnNvLT5z
b19yY3YpOwotCQkJdHAtPnRfZmxhZ3MgfD0gVEZfV0FLRVNPUjsKKwkJCXNvcndha2V1cF9sb2Nr
ZWQoc28pOwogI2lmZGVmIE5FVEZMSVhfU0JfTElNSVRTCiAJCQlpZiAoc28tPnNvX3Jjdi5zYl9z
aGxpbSAmJiBhcHBlbmRlZCAhPSBtY250KQogCQkJCWNvdW50ZXJfZm9fcmVsZWFzZShzby0+c29f
cmN2LnNiX3NobGltLApAQCAtODczMSw4ICs4NzI5LDYgQEAgcmFja19wcm9jZXNzX2RhdGEoc3Ry
dWN0IG1idWYgKm0sIHN0cnVjdCB0Y3BoZHIgKnRoLCBzCiAJaWYgKHRoZmxhZ3MgJiBUSF9GSU4p
IHsKIAkJaWYgKFRDUFNfSEFWRVJDVkRGSU4odHAtPnRfc3RhdGUpID09IDApIHsKIAkJCXNvY2Fu
dHJjdm1vcmUoc28pOwotCQkJLyogVGhlIHNvY2tldCB1cGNhbGwgaXMgaGFuZGxlZCBieSBzb2Nh
bnRyY3Ztb3JlLiAqLwotCQkJdHAtPnRfZmxhZ3MgJj0gflRGX1dBS0VTT1I7CiAJCQkvKgogCQkJ
ICogSWYgY29ubmVjdGlvbiBpcyBoYWxmLXN5bmNocm9uaXplZCAoaWUgTkVFRFNZTgogCQkJICog
ZmxhZyBvbikgdGhlbiBkZWxheSBBQ0ssIHNvIGl0IG1heSBiZSBwaWdneWJhY2tlZApAQCAtODky
NCw4ICs4OTIwLDcgQEAgcmFja19kb19mYXN0bmV3ZGF0YShzdHJ1Y3QgbWJ1ZiAqbSwgc3RydWN0
IHRjcGhkciAqdGgsCiAJCQlzYmFwcGVuZHN0cmVhbV9sb2NrZWQoJnNvLT5zb19yY3YsIG0sIDAp
OwogCQljdGZfY2FsY19yd2luKHNvLCB0cCk7CiAJfQotCVNPQ0tCVUZfVU5MT0NLKCZzby0+c29f
cmN2KTsKLQl0cC0+dF9mbGFncyB8PSBURl9XQUtFU09SOworCXNvcndha2V1cF9sb2NrZWQoc28p
OwogI2lmZGVmIE5FVEZMSVhfU0JfTElNSVRTCiAJaWYgKHNvLT5zb19yY3Yuc2Jfc2hsaW0gJiYg
bWNudCAhPSBhcHBlbmRlZCkKIAkJY291bnRlcl9mb19yZWxlYXNlKHNvLT5zb19yY3Yuc2Jfc2hs
aW0sIG1jbnQgLSBhcHBlbmRlZCk7CkBAIC05MTQyLDcgKzkxMzcsNyBAQCByYWNrX2Zhc3RhY2so
c3RydWN0IG1idWYgKm0sIHN0cnVjdCB0Y3BoZHIgKnRoLCBzdHJ1Y3QKIAkJcmFja190aW1lcl9j
YW5jZWwodHAsIHJhY2ssIHJhY2stPnJfY3RsLnJjX3JjdnRpbWUsIF9fTElORV9fKTsKIAl9CiAJ
LyogV2FrZSB1cCB0aGUgc29ja2V0IGlmIHdlIGhhdmUgcm9vbSB0byB3cml0ZSBtb3JlICovCi0J
dHAtPnRfZmxhZ3MgfD0gVEZfV0FLRVNPVzsKKwlzb3d3YWtldXAoc28pOwogCWlmIChzYmF2YWls
KCZzby0+c29fc25kKSkgewogCQlyYWNrLT5yX3dhbnRlZF9vdXRwdXQgPSAxOwogCX0KQEAgLTEx
MjA1LDEwICsxMTIwMCw4IEBAIHJhY2tfZG9fc2VnbWVudChzdHJ1Y3QgbWJ1ZiAqbSwgc3RydWN0
IHRjcGhkciAqdGgsIHN0cgogCQl0Y3BfZ2V0X3VzZWNzKCZ0dik7CiAJfQogCWlmKHJhY2tfZG9f
c2VnbWVudF9ub3VubG9jayhtLCB0aCwgc28sIHRwLAotCQkJCSAgICBkcm9wX2hkcmxlbiwgdGxl
biwgaXB0b3MsIDAsICZ0dikgPT0gMCkgewotCQl0Y3BfaGFuZGxlX3dha2V1cCh0cCwgc28pOwor
CQkJCSAgICBkcm9wX2hkcmxlbiwgdGxlbiwgaXB0b3MsIDAsICZ0dikgPT0gMCkKIAkJSU5QX1dV
TkxPQ0sodHAtPnRfaW5wY2IpOwotCX0KIH0KIAogc3RydWN0IHJhY2tfc2VuZG1hcCAqCi0tLSBz
eXMvbmV0aW5ldC90Y3Bfc3RhY2tzL3JhY2tfYmJyX2NvbW1vbi5jLm9yaWcJMjAyMS0wNC0wOSAw
ODo0NToyNi43MjE1MjEwMDAgLTA3MDAKKysrIHN5cy9uZXRpbmV0L3RjcF9zdGFja3MvcmFja19i
YnJfY29tbW9uLmMJMjAyMS0wNC0wOSAwODo0Njo1OC41ODAyMzQwMDAgLTA3MDAKQEAgLTQ1OCw3
ICs0NTgsNiBAQCBjdGZfZG9fcXVldWVkX3NlZ21lbnRzKHN0cnVjdCBzb2NrZXQgKnNvLCBzdHJ1
Y3QgdGNwY2IKIAkJCS8qIFdlIGxvc3QgdGhlIHRjcGNiIChtYXliZSBhIFJTVCBjYW1lIGluKT8g
Ki8KIAkJCXJldHVybigxKTsKIAkJfQotCQl0Y3BfaGFuZGxlX3dha2V1cCh0cCwgc28pOwogCX0K
IAlyZXR1cm4gKDApOwogfQotLS0gc3lzL25ldGluZXQvdGNwX2lucHV0LmMub3JpZwkyMDIxLTA0
LTA1IDAxOjA3OjAwLjM0MjU1OTAwMCAtMDcwMAorKysgc3lzL25ldGluZXQvdGNwX2lucHV0LmMJ
MjAyMS0wNC0wOSAwNzo1ODowMy4yNjI4MTUwMDAgLTA3MDAKQEAgLTE0NzIsMjkgKzE0NzIsNiBA
QCB0Y3BfYXV0b3JjdmJ1ZihzdHJ1Y3QgbWJ1ZiAqbSwgc3RydWN0IHRjcGhkciAqdGgsIHN0cnUK
IH0KIAogdm9pZAotdGNwX2hhbmRsZV93YWtldXAoc3RydWN0IHRjcGNiICp0cCwgc3RydWN0IHNv
Y2tldCAqc28pCi17Ci0JLyoKLQkgKiBTaW5jZSB0cCBtaWdodCBiZSBnb25lIGlmIHRoZSBzZXNz
aW9uIGVudGVyZWQKLQkgKiB0aGUgVElNRV9XQUlUIHN0YXRlIGJlZm9yZSBjb21pbmcgaGVyZSwg
d2UgbmVlZAotCSAqIHRvIGNoZWNrIGlmIHRoZSBzb2NrZXQgaXMgc3RpbGwgY29ubmVjdGVkLgot
CSAqLwotCWlmICgoc28tPnNvX3N0YXRlICYgU1NfSVNDT05ORUNURUQpID09IDApCi0JCXJldHVy
bjsKLQlJTlBfTE9DS19BU1NFUlQodHAtPnRfaW5wY2IpOwotCWlmICh0cC0+dF9mbGFncyAmIFRG
X1dBS0VTT1IpIHsKLQkJdHAtPnRfZmxhZ3MgJj0gflRGX1dBS0VTT1I7Ci0JCVNPQ0tCVUZfVU5M
T0NLX0FTU0VSVCgmc28tPnNvX3Jjdik7Ci0JCXNvcndha2V1cChzbyk7Ci0JfQotCWlmICh0cC0+
dF9mbGFncyAmIFRGX1dBS0VTT1cpIHsKLQkJdHAtPnRfZmxhZ3MgJj0gflRGX1dBS0VTT1c7Ci0J
CVNPQ0tCVUZfVU5MT0NLX0FTU0VSVCgmc28tPnNvX3NuZCk7Ci0JCXNvd3dha2V1cChzbyk7Ci0J
fQotfQotCi12b2lkCiB0Y3BfZG9fc2VnbWVudChzdHJ1Y3QgbWJ1ZiAqbSwgc3RydWN0IHRjcGhk
ciAqdGgsIHN0cnVjdCBzb2NrZXQgKnNvLAogICAgIHN0cnVjdCB0Y3BjYiAqdHAsIGludCBkcm9w
X2hkcmxlbiwgaW50IHRsZW4sIHVpbnQ4X3QgaXB0b3MpCiB7CkBAIC0xODYzLDcgKzE4NDAsNyBA
QCB0Y3BfZG9fc2VnbWVudChzdHJ1Y3QgbWJ1ZiAqbSwgc3RydWN0IHRjcGhkciAqdGgsIHN0cnUK
IAkJCQllbHNlIGlmICghdGNwX3RpbWVyX2FjdGl2ZSh0cCwgVFRfUEVSU0lTVCkpCiAJCQkJCXRj
cF90aW1lcl9hY3RpdmF0ZSh0cCwgVFRfUkVYTVQsCiAJCQkJCQkgICAgICB0cC0+dF9yeHRjdXIp
OwotCQkJCXRwLT50X2ZsYWdzIHw9IFRGX1dBS0VTT1c7CisJCQkJc293d2FrZXVwKHNvKTsKIAkJ
CQlpZiAoc2JhdmFpbCgmc28tPnNvX3NuZCkpCiAJCQkJCSh2b2lkKSB0cC0+dF9mYi0+dGZiX3Rj
cF9vdXRwdXQodHApOwogCQkJCWdvdG8gY2hlY2tfZGVsYWNrOwpAQCAtMTkyOCw4ICsxOTA1LDgg
QEAgdGNwX2RvX3NlZ21lbnQoc3RydWN0IG1idWYgKm0sIHN0cnVjdCB0Y3BoZHIgKnRoLCBzdHJ1
CiAJCQkJbV9hZGoobSwgZHJvcF9oZHJsZW4pOwkvKiBkZWxheWVkIGhlYWRlciBkcm9wICovCiAJ
CQkJc2JhcHBlbmRzdHJlYW1fbG9ja2VkKCZzby0+c29fcmN2LCBtLCAwKTsKIAkJCX0KLQkJCVNP
Q0tCVUZfVU5MT0NLKCZzby0+c29fcmN2KTsKLQkJCXRwLT50X2ZsYWdzIHw9IFRGX1dBS0VTT1I7
CisJCQkvKiBOQjogc29yd2FrZXVwX2xvY2tlZCgpIGRvZXMgYW4gaW1wbGljaXQgdW5sb2NrLiAq
LworCQkJc29yd2FrZXVwX2xvY2tlZChzbyk7CiAJCQlpZiAoREVMQVlfQUNLKHRwLCB0bGVuKSkg
ewogCQkJCXRwLT50X2ZsYWdzIHw9IFRGX0RFTEFDSzsKIAkJCX0gZWxzZSB7CkBAIC0yOTI1LDgg
KzI5MDIsOCBAQCBwcm9jZXNzX0FDSzoKIAkJCQl0cC0+c25kX3duZCA9IDA7CiAJCQlvdXJmaW5p
c2Fja2VkID0gMDsKIAkJfQotCQlTT0NLQlVGX1VOTE9DSygmc28tPnNvX3NuZCk7Ci0JCXRwLT50
X2ZsYWdzIHw9IFRGX1dBS0VTT1c7CisJCS8qIE5COiBzb3d3YWtldXBfbG9ja2VkKCkgZG9lcyBh
biBpbXBsaWNpdCB1bmxvY2suICovCisJCXNvd3dha2V1cF9sb2NrZWQoc28pOwogCQltX2ZyZWVt
KG1mcmVlKTsKIAkJLyogRGV0ZWN0IHVuYSB3cmFwYXJvdW5kLiAqLwogCQlpZiAoIUlOX1JFQ09W
RVJZKHRwLT50X2ZsYWdzKSAmJgpAQCAtMzE0Nyw4ICszMTI0LDggQEAgZG9kYXRhOgkJCQkJCQkv
KiBYWFggKi8KIAkJCQltX2ZyZWVtKG0pOwogCQkJZWxzZQogCQkJCXNiYXBwZW5kc3RyZWFtX2xv
Y2tlZCgmc28tPnNvX3JjdiwgbSwgMCk7Ci0JCQlTT0NLQlVGX1VOTE9DSygmc28tPnNvX3Jjdik7
Ci0JCQl0cC0+dF9mbGFncyB8PSBURl9XQUtFU09SOworCQkJLyogTkI6IHNvcndha2V1cF9sb2Nr
ZWQoKSBkb2VzIGFuIGltcGxpY2l0IHVubG9jay4gKi8KKwkJCXNvcndha2V1cF9sb2NrZWQoc28p
OwogCQl9IGVsc2UgewogCQkJLyoKIAkJCSAqIFhYWDogRHVlIHRvIHRoZSBoZWFkZXIgZHJvcCBh
Ym92ZSAidGgiIGlzCkBAIC0zMjE1LDggKzMxOTIsNiBAQCBkb2RhdGE6CQkJCQkJCS8qIFhYWCAq
LwogCWlmICh0aGZsYWdzICYgVEhfRklOKSB7CiAJCWlmIChUQ1BTX0hBVkVSQ1ZERklOKHRwLT50
X3N0YXRlKSA9PSAwKSB7CiAJCQlzb2NhbnRyY3Ztb3JlKHNvKTsKLQkJCS8qIFRoZSBzb2NrZXQg
dXBjYWxsIGlzIGhhbmRsZWQgYnkgc29jYW50cmN2bW9yZS4gKi8KLQkJCXRwLT50X2ZsYWdzICY9
IH5URl9XQUtFU09SOwogCQkJLyoKIAkJCSAqIElmIGNvbm5lY3Rpb24gaXMgaGFsZi1zeW5jaHJv
bml6ZWQKIAkJCSAqIChpZSBORUVEU1lOIGZsYWcgb24pIHRoZW4gZGVsYXkgQUNLLApAQCAtMzI4
MCw3ICszMjU1LDYgQEAgY2hlY2tfZGVsYWNrOgogCQl0cC0+dF9mbGFncyAmPSB+VEZfREVMQUNL
OwogCQl0Y3BfdGltZXJfYWN0aXZhdGUodHAsIFRUX0RFTEFDSywgdGNwX2RlbGFja3RpbWUpOwog
CX0KLQl0Y3BfaGFuZGxlX3dha2V1cCh0cCwgc28pOwogCUlOUF9XVU5MT0NLKHRwLT50X2lucGNi
KTsKIAlyZXR1cm47CiAKQEAgLTMzMTQsNyArMzI4OCw2IEBAIGRyb3BhZnRlcmFjazoKIAlUQ1Bf
UFJPQkUzKGRlYnVnX19pbnB1dCwgdHAsIHRoLCBtKTsKIAl0cC0+dF9mbGFncyB8PSBURl9BQ0tO
T1c7CiAJKHZvaWQpIHRwLT50X2ZiLT50ZmJfdGNwX291dHB1dCh0cCk7Ci0JdGNwX2hhbmRsZV93
YWtldXAodHAsIHNvKTsKIAlJTlBfV1VOTE9DSyh0cC0+dF9pbnBjYik7CiAJbV9mcmVlbShtKTsK
IAlyZXR1cm47CkBAIC0zMzIyLDcgKzMyOTUsNiBAQCBkcm9wYWZ0ZXJhY2s6CiBkcm9wd2l0aHJl
c2V0OgogCWlmICh0cCAhPSBOVUxMKSB7CiAJCXRjcF9kcm9wd2l0aHJlc2V0KG0sIHRoLCB0cCwg
dGxlbiwgcnN0cmVhc29uKTsKLQkJdGNwX2hhbmRsZV93YWtldXAodHAsIHNvKTsKIAkJSU5QX1dV
TkxPQ0sodHAtPnRfaW5wY2IpOwogCX0gZWxzZQogCQl0Y3BfZHJvcHdpdGhyZXNldChtLCB0aCwg
TlVMTCwgdGxlbiwgcnN0cmVhc29uKTsKQEAgLTMzMzgsMTAgKzMzMTAsOCBAQCBkcm9wOgogCQkJ
ICAmdGNwX3NhdmV0Y3AsIDApOwogI2VuZGlmCiAJVENQX1BST0JFMyhkZWJ1Z19faW5wdXQsIHRw
LCB0aCwgbSk7Ci0JaWYgKHRwICE9IE5VTEwpIHsKLQkJdGNwX2hhbmRsZV93YWtldXAodHAsIHNv
KTsKKwlpZiAodHAgIT0gTlVMTCkKIAkJSU5QX1dVTkxPQ0sodHAtPnRfaW5wY2IpOwotCX0KIAlt
X2ZyZWVtKG0pOwogfQogCi0tLSBzeXMvbmV0aW5ldC90Y3BfcmVhc3MuYy5vcmlnCTIwMjEtMDQt
MDkgMDg6MTg6MTAuNTk5MDkyMDAwIC0wNzAwCisrKyBzeXMvbmV0aW5ldC90Y3BfcmVhc3MuYwky
MDIxLTA0LTA5IDA4OjE5OjU0LjkxMjM3ODAwMCAtMDcwMApAQCAtOTU5LDggKzk1OSw3IEBAIG5l
d19lbnRyeToKIAkJfSBlbHNlIHsKIAkJCXNiYXBwZW5kc3RyZWFtX2xvY2tlZCgmc28tPnNvX3Jj
diwgbSwgMCk7CiAJCX0KLQkJU09DS0JVRl9VTkxPQ0soJnNvLT5zb19yY3YpOwotCQl0cC0+dF9m
bGFncyB8PSBURl9XQUtFU09SOworCQlzb3J3YWtldXBfbG9ja2VkKHNvKTsKIAkJcmV0dXJuIChm
bGFncyk7CiAJfQogCWlmICh0Y3BfbmV3X2xpbWl0cykgewpAQCAtMTEwOCw3ICsxMTA3LDYgQEAg
cHJlc2VudDoKICNpZmRlZiBUQ1BfUkVBU1NfTE9HR0lORwogCXRjcF9yZWFzc19sb2dfZHVtcCh0
cCk7CiAjZW5kaWYKLQlTT0NLQlVGX1VOTE9DSygmc28tPnNvX3Jjdik7Ci0JdHAtPnRfZmxhZ3Mg
fD0gVEZfV0FLRVNPUjsKKwlzb3J3YWtldXBfbG9ja2VkKHNvKTsKIAlyZXR1cm4gKGZsYWdzKTsK
IH0K
--_002_YQXPR0101MB0968924C8AEAB808888D75B1DD499YQXPR0101MB0968_--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?YQXPR0101MB0968924C8AEAB808888D75B1DD499>