Date: Tue, 24 Aug 2021 14:20:17 -1000 From: "parv/freebsd" <parv.0zero9+freebsd@gmail.com> To: freebsd-questions@freebsd.org Subject: Re: NFS v3 - FreeBSD 12 server connection getting timed out on CentOS 8 client Message-ID: <CABObuOpqxqT8N0qLZjoSfrbLVJgw2Oj315_vXjbhoPOTS2Ov%2BA@mail.gmail.com> In-Reply-To: <CABObuOr5rLSoUhm8LsWr6hQZVmpBGrYvJE%2BnruV1kmB8W-9XnQ@mail.gmail.com> References: <CABObuOr5rLSoUhm8LsWr6hQZVmpBGrYvJE%2BnruV1kmB8W-9XnQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, Aug 22, 2021 at 8:05 PM parv/freebsd wrote: (I have rearranged my quoted email) [CentOS 8 NFS v3 client sometimes prints ...] > Aug 21 08:31:04 [CLIENT] kernel: watchdog: BUG: soft lockup - CPU#0 stuck > for 22s! [kworker/u24:0:672075] > > Aug 21 08:31:04 [CLIENT] kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [kworker/u24:10:672632] > > ... > On the [FreeBSD 12.2] server, there is not much CPU load; drive activity > does not match the > connection being timed. On [CentOS 8] client, CPU getting stuck messages > are being printed > to everyone when trying to write to the NFS server > ... > On the server side, I had not seen any output -- from smartctl or nfsd -- > which > could indicate such a problem. > ... > On the server, I have set "vfs.nfsd.debuglevel=2" to see if there would be > any > output when the client would complain after connection getting timed out. > > How do I go about investigating or solve this? > ... I have collected tcpdump output on both server & client ... https://github.com/parv/centos8-nfs3-issue/ ... but am unable to differentiate between error and normal operation in the output, unlike of strace. :-( When I ran rsync, CentOS sent the message "CPU stuck" around 21:20, so links around to that time ... client output (215 KB), https://github.com/parv/centos8-nfs3-issue/blob/main/centos8-client.tcpdump.2118-2120#L1623 server output (414 KB), https://github.com/parv/centos8-nfs3-issue/blob/main/freebsd12-server.tcpdump.2118-2120#L2621 - parv
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CABObuOpqxqT8N0qLZjoSfrbLVJgw2Oj315_vXjbhoPOTS2Ov%2BA>