Date: Mon, 20 Apr 2026 15:19:14 -0400 From: Garrett Wollman <wollman@bimajority.org> To: Mark Johnston <markj@freebsd.org> Cc: Olgun Adak <olgun.adak@trexquant.com>, freebsd-stable@freebsd.org, freebsd-net@freebsd.org Subject: Re: [REGRESSION] nfsd TCP socket lockup on 14.3-RELEASE-p9/p10 - Confirmed on Multiple Systems Message-ID: <27110.31794.617374.30354@hergotha.csail.mit.edu> In-Reply-To: <aeY3j_IKV2eUpD3l@nuc> References: <CAFww=iytpc%2BJJuiPJtTrKKkqZaP%2BZ9OzHnTCzLk=MSEkzGwNzA@mail.gmail.com> <aeY3j_IKV2eUpD3l@nuc>
index | next in thread | previous in thread | raw e-mail
<<On Mon, 20 Apr 2026 10:26:23 -0400, Mark Johnston <markj@freebsd.org> said: > Would you be able to share full "procstat -kka" output from an affected > system? For what it's worth, I have found the following recipe to be helpful: # procstat -hk NFSD_PID | | cut -c 54- | sort | uniq -c | sort -n since many nfsd threads will often be sleeping with the same stack trace, this helps to collapse them and call out the interesting activity. I ran this on our one active 14.3-p10 server, and the result is like this: 1 mi_switch sleepq_timedwait _cv_timedwait_sbt zio_wait dbuf_read dbuf_hold_impl dbuf_hold dmu_buf_hold_array_by_dnode dmu_read_uio_dnode dmu_read_uio_dbuf zfs_read zfs_freebsd_read VOP_READ_APV nfsvno_read nfsrvd_read nfsrvd_dorpc nfssvc_program svc_run_internal 1 mi_switch sleepq_timedwait _cv_timedwait_sbt zio_wait dmu_buf_hold_array_by_dnode dmu_read_uio_dnode dmu_read_uio_dbuf zfs_read zfs_freebsd_read VOP_READ_APV nfsvno_read nfsrvd_read nfsrvd_dorpc nfssvc_program svc_run_internal svc_run nfsrvd_nfsd nfssvc_nfsd 1 soiolock sosend_generic sosend svc_vc_reply svc_sendreply_common svc_sendreply_mbuf nfssvc_program svc_run_internal svc_thread_start fork_exit fork_trampoline 1 uma_zalloc_arg tcp_m_copym tcp_default_output tcp_usr_send sosend_generic_locked sosend_generic sosend svc_vc_reply svc_sendreply_common svc_sendreply_mbuf nfssvc_program svc_run_internal svc_thread_start fork_exit fork_trampoline 2 mi_switch _cv_wait dbuf_read dbuf_hold_impl dbuf_hold dmu_buf_hold_array_by_dnode dmu_read_uio_dnode dmu_read_uio_dbuf zfs_read zfs_freebsd_read VOP_READ_APV nfsvno_read nfsrvd_read nfsrvd_dorpc nfssvc_program svc_run_internal svc_thread_start fork_exit 2 mi_switch sleepq_catch_signals sleepq_wait_sig _cv_wait_sig svc_run_internal svc_thread_start fork_exit fork_trampoline 7 mi_switch sleepq_timedwait _cv_timedwait_sbt zio_wait dmu_buf_hold_array_by_dnode dmu_read_uio_dnode dmu_read_uio_dbuf zfs_read zfs_freebsd_read VOP_READ_APV nfsvno_read nfsrvd_read nfsrvd_dorpc nfssvc_program svc_run_internal svc_thread_start fork_exit fork_trampoline 19 mi_switch _cv_wait dmu_buf_hold_array_by_dnode dmu_read_uio_dnode dmu_read_uio_dbuf zfs_read zfs_freebsd_read VOP_READ_APV nfsvno_read nfsrvd_read nfsrvd_dorpc nfssvc_program svc_run_internal svc_thread_start fork_exit fork_trampoline 46 mi_switch sleepq_catch_signals sleepq_timedwait_sig _cv_timedwait_sig_sbt svc_run_internal svc_thread_start fork_exit fork_trampoline -GAWollmanhome | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?27110.31794.617374.30354>