Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 14 Apr 2021 11:41:29 +0000
From:      bugzilla-noreply@freebsd.org
To:        bugs@FreeBSD.org
Subject:   [Bug 255048] ROCKPRO64 freeze during heavy IO on external USB 3 disk
Message-ID:  <bug-255048-227-qHFZJewHVP@https.bugs.freebsd.org/bugzilla/>
In-Reply-To: <bug-255048-227@https.bugs.freebsd.org/bugzilla/>
References:  <bug-255048-227@https.bugs.freebsd.org/bugzilla/>

next in thread | previous in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D255048

Henri Hennebert <hlh@restart.be> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |hlh@restart.be

--- Comment #1 from Henri Hennebert <hlh@restart.be> ---
FreeBSD keystone.lab.bel 13.0-STABLE FreeBSD 13.0-STABLE #0
stable/13-n245189-236d1f8c1773

The disk is under zfs

During a `git pull` in /usr/ports the system show the da0 disk 100% busy and
afer ~2 minutes the system freeze.

The system respond to ping

I connect on the serial console (CR ~ ^B) to ddb:

db> show msgbuf
...
swap_pager: cannot allocate bio
swap_pager: cannot allocate bio
swap_pager: cannot allocate bio
swap_pager: cannot allocate bio
swap_pager: cannot allocate bio
swap_pager: cannot allocate bio

db> show allchains
chain 1:
 thread 115444 (pid 87479, git) is sleeping on 0xffff0000ef3dfc08 "aw.aew_c=
v"
chain 2:
 thread 115446 (pid 87479, git) is sleeping on 0xffffa00058d653e8 "zio->io_=
cv"
chain 3:
 thread 115447 (pid 87479, git) is sleeping on 0xffffa000b26cf3e8 "zio->io_=
cv"
chain 4:
 thread 115450 (pid 87479, git) is sleeping on 0xffff0000ef3cbc08 "aw.aew_c=
v"
chain 5:
 thread 115452 (pid 87479, git) is sleeping on 0xffffa000091518b8 "zio->io_=
cv"
chain 6:
 thread 115453 (pid 87479, git) is sleeping on 0xffff0000ef3e9c08 "aw.aew_c=
v"
chain 7:
 thread 115455 (pid 87479, git) is sleeping on 0xffffa00058d028b8 "zio->io_=
cv"
chain 8:
 thread 115458 (pid 87479, git) is blocked on sx "zfsvfs->z_hold_mtx[i]" XL=
OCK
 thread 115455 (pid 87479, git) is sleeping on 0xffffa00058d028b8 "zio->io_=
cv"
chain 9:
 thread 115459 (pid 87479, git) is sleeping on 0xffffa000971718b8 "zio->io_=
cv"
chain 10:
 thread 115460 (pid 87479, git) is sleeping on 0xffff00010420dc08 "aw.aew_c=
v"
chain 11:
 thread 115462 (pid 87479, git) is sleeping on 0xffff000104217c08 "aw.aew_c=
v"
chain 12:
 thread 115463 (pid 87479, git) is sleeping on 0xffff00010421cc08 "aw.aew_c=
v"
chain 13:
 thread 101506 (pid 1214, god.plugin) is sleeping on 0xffff0000ecdb70a8
"aw.aew_cv"
chain 14:
 thread 101508 (pid 1214, god.plugin) is sleeping on 0xffffa000e430da80 "vm=
pfw"
chain 15:
 thread 101510 (pid 1214, god.plugin) is sleeping on 0xffff0000ecc5e0a8
"aw.aew_cv"
chain 16:
 thread 101513 (pid 1214, god.plugin) is sleeping on 0xffff0000ecd990a8
"aw.aew_cv"
chain 17:
 thread 101352 (pid 1211, apps.plugin) is sleeping on 0xffffa00058d658b8
"zio->io_cv"
chain 18:
 thread 101448 (pid 1195, httpd) is sleeping on 0xffff0000ecc9f0a8 "aw.aew_=
cv"
chain 19:
 thread 101366 (pid 1179, httpd) is sleeping on 0xffffa0008c253d88 "zio->io=
_cv"
chain 20:
 thread 101346 (pid 1141, netdata) is sleeping on 0xffff0000e9424cb8
"aw.aew_cv"
chain 21:
 thread 101355 (pid 1121, sshd) is sleeping on 0xffff000000b24d80 "pfault"
chain 22:
 thread 101350 (pid 1083, sendmail) is sleeping on 0xffff000000b24d80 "vmwa=
it"
chain 23:
 thread 101349 (pid 1079, inetd) is sleeping on 0xffffa00027577b40 "select"
chain 24:
 thread 101347 (pid 1050, ntpd) is sleeping on 0xffff0000e93fd0a8 "aw.aew_c=
v"
chain 25:
 thread 101314 (pid 970, syslogd) is sleeping on 0xffff000000b24d80 "pfault"
chain 26:
 thread 100152 (pid 31, syncer) is sleeping on 0xffff000000d5f480 "syncer"
chain 27:
 thread 100151 (pid 30, vnlru) is blocked on lockmgr  DEXCL
 thread 100151 (pid 30, vnlru) is blocked on lockmgr  DEXCL
 thread 100151 (pid 30, vnlru) is blocked on lockmgr  DEXCL
 thread 100151 (pid 30, vnlru) is blocked on lockmgr  DEXCL
 thread 100151 (pid 30, vnlru) is blocked on lockmgr  DEXCL
 thread 100151 (pid 30, vnlru) is blocked on lockmgr  DEXCL
 thread 100151 (pid 30, vnlru) is blocked on lockmgr  DEXCL
 thread 100151 (pid 30, vnlru) is blocked on lockmgr  DEXCL
 thread 100151 (pid 30, vnlru) is blocked on lockmgr  DEXCL
 thread 100151 (pid 30, vnlru) is blocked on lockmgr  DEXCL
 thread 100151 (pid 30, vnlru) is blocked on lockmgr  DEXCL
 thread 100151 (pid 30, vnlru) is blocked on lockmgr  DEXCL
 thread 100151 (pid 30, vnlru) is blocked on lockmgr  DEXCL
 thread 100151 (pid 30, vnlru) is blocked on lockmgr  DEXCL
 thread 100151 (pid 30, vnlru) is blocked on lockmgr  DEXCL
 thread 100151 (pid 30, vnlru) is blocked on lockmgr  DEXCL
 thread 100151 (pid 30, vnlru) is blocked on lockmgr  DEXCL
 thread 100151 (pid 30, vnlru) is blocked on lockmgr  DEXCL
 thread 100151 (pid 30, vnlru) is blocked on lockmgr  DEXCL
 thread 100151 (pid 30, vnlru) is blocked on lockmgr  DEXCL
 thread 100151 (pid 30, vnlru) is blocked on lockmgr  DEXCL
...=20
...continue for at least 1400 lines.
...
db> show proc 30
Process 30 (vnlru) at 0xffffa00006449000:
 state: NORMAL
 uid: 0  gids: 0
 parent: pid 0 at 0xffff000000cf1828
 ABI: null
 flag: 0x10000204  flag2: 0
 reaper: 0xffff000000cf1828 reapsubtree: 30
 sigparent: 20
 vmspace: 0xffff000000cf22c8
   (map 0xffff000000cf22c8)
   (map.pmap 0xffff000000cf2388)
   (pmap 0xffff000000cf23e8)
 threads: 1
100151                   D       vlruwt  0xffffa00006449000  [vnlru]
db> tr 100151
Tracing pid 30 tid 100151 td 0xffffa0000660f580
sched_switch() at mi_switch+0xf0
         pc =3D 0xffff0000003dc7c0  lr =3D 0xffff0000003b2544
         sp =3D 0xffff0000da1fc760  fp =3D 0xffff0000da1fc7b0

mi_switch() at sleepq_timedwait+0x28
         pc =3D 0xffff0000003b2544  lr =3D 0xffff00000040ce64
         sp =3D 0xffff0000da1fc7c0  fp =3D 0xffff0000da1fc7f0

sleepq_timedwait() at _sleep+0x194
         pc =3D 0xffff00000040ce64  lr =3D 0xffff0000003b1960
         sp =3D 0xffff0000da1fc800  fp =3D 0xffff0000da1fc830

_sleep() at vnlru_proc+0xc54
         pc =3D 0xffff0000003b1960  lr =3D 0xffff0000004a4c18
         sp =3D 0xffff0000da1fc840  fp =3D 0xffff0000da1fc8f0

vnlru_proc() at fork_exit+0x88
         pc =3D 0xffff0000004a4c18  lr =3D 0xffff0000003547d0
         sp =3D 0xffff0000da1fc900  fp =3D 0xffff0000da1fc950
--More--
fork_exit() at fork_trampoline+0x10
         pc =3D 0xffff0000003547d0  lr =3D 0xffff0000007cfef8
         sp =3D 0xffff0000da1fc960  fp =3D 0x0000000000000000

db> show mount
0xffff0000e663e100 keystone/ROOT/default on / (zfs)
0xffff0000bbb25100 devfs on /dev (devfs)
0xffff0000e663d600 procfs on /proc (procfs)
0xffff0000e663cb00 fdescfs on /dev/fd (fdescfs)
0xffff0000e9a4e100 keystone/var/spool on /var/spool (zfs)
0xffff0000e9a4d600 keystone/var/log on /var/log (zfs)
0xffff0000bbb24600 keystone/tmp on /tmp (zfs)
0xffff0000e9a48100 keystone/usr/local on /usr/local (zfs)
0xffff0000e663c000 keystone/var/db on /var/db (zfs)
0xffff0000e9a47600 keystone/home on /home (zfs)
0xffff0000bbb23b00 keystone/usr/ports on /usr/ports (zfs)
0xffff0000e9a4c000 keystone/usr/src on /usr/src (zfs)
0xffff0000e9a59100 keystone/var/tmp on /var/tmp (zfs)
0xffff0000e9a68100 keystone/usr/obj on /usr/obj (zfs)
0xffff0000e9a4cb00 keystone/var/imap on /var/imap (zfs)
0xffff0000e9a4b100 keystone/var/audit on /var/audit (zfs)
0xffff0000bbb23000 keystone/var/spool/openldap on /var/spool/openldap (zfs)
0xffff0000e9a58600 keystone/var/spool/httpd on /var/spool/httpd (zfs)
0xffff0000e9a46b00 keystone/var/spool/imap on /var/spool/imap (zfs)
0xffff0000ea23a100 keystone/var/spool/postgres on /var/spool/postgres (zfs)
0xffff0000ea2ef100 keystone/poudriere on /usr/local/poudriere (zfs)
0xffff0000ea239600 keystone/poudriere/data on /usr/local/poudriere/data (zf=
s)
0xffff0000e9a57b00 keystone/poudriere/ports on /usr/local/poudriere/ports (=
zfs)
0xffff0000e9a4a600 keystone/poudriere/jails on /usr/local/poudriere/jails (=
zfs)
0xffff0000ea238b00 keystone/poudriere/ports/default on
/usr/local/poudriere/ports/default (zfs)
0xffff0000ea238000 keystone/poudriere/data/packages on
/usr/local/poudriere/data/packages (zfs)
0xffff0000ea2ee600 keystone/poudriere/data/cache on
/usr/local/poudriere/data/cache (zfs)
0xffff0000ea2edb00 keystone/poudriere/data/logs on
/usr/local/poudriere/data/logs (zfs)
0xffff0000e9a57000 keystone/poudriere/data/images on
/usr/local/poudriere/data/images (zfs)
0xffff0000e9a66b00 keystone/poudriere/data/.m on /usr/local/poudriere/data/=
.m
(zfs)
0xffff0000e9a49b00 keystone/poudriere/jails/13aarch64 on
/usr/local/poudriere/jails/13aarch64 (zfs)
0xffff0000e9a66000 keystone/poudriere/data/wrkdirs on
/usr/local/poudriere/data/wrkdirs (zfs)
0xffff0000e9a49000 keystone/usr/ports/distfiles on /usr/ports/distfiles (zf=
s)

More info: show mount <addr>

db> show page
vm_cnt.v_free_count: 3095
vm_cnt.v_inactive_count: 80
vm_cnt.v_active_count: 287
vm_cnt.v_laundry_count: 57
vm_cnt.v_wire_count: 979462
vm_cnt.v_free_reserved: 1347
vm_cnt.v_free_min: 6266
vm_cnt.v_free_target: 21023
vm_cnt.v_inactive_target: 31534
db> show pageq
pq_free 3095
dom 0 page_cnt 983828 free 3095 pq_act 287 pq_inact 80 pq_laund 57 pq_unsw 0

I encounter the same problem on another ROCKPRO64 under 13.0-RELEASE

--=20
You are receiving this mail because:
You are the assignee for the bug.=



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-255048-227-qHFZJewHVP>