Date: Sat, 14 Dec 2013 07:13:12 +0000 From: Ryan Baldwin <ryan.baldwin@nexusalpha.com> To: Rod Taylor <rod.taylor@gmail.com> Cc: freebsd-fs@freebsd.org Subject: Re: ZFS related hang with FreeBSD 9.2 Message-ID: <52AC0508.5060704@nexusalpha.com> In-Reply-To: <CAKddOFATpE0U8Z0AYhsBPwfym3XJWeh95Q%2BS3Ug_YZJmkhUKCQ@mail.gmail.com> References: <52AA97B8.8060408@nexusalpha.com> <CAKddOFATpE0U8Z0AYhsBPwfym3XJWeh95Q%2BS3Ug_YZJmkhUKCQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Yes we are using snapshots. The snapshot-manager processes are continually creating and destroying snapshots at a rate of 10,000's a day. We have had such a system running on FreeBSD 9.1 with just one snapshot-manager process several months on end and we are not aware of any problems with this system. I run another type of system currently on FreeBSD 9.1 as a storage server and it serves terrabytes a day and uses snapshots and clones extensively and I have not had any problems with this system. This is the first real problem I have come across. So far the 9.1 based version of the problem server is still going although it is still early days. On 12/13/13 23:21, Rod Taylor wrote: > Are you using snapshots? > > I've found ZFS Snapshots on 9.0, 9.1, and 9.2 regularly crash the > system. Delete the snapshots and don't create any new ones and > suddenly it's stable for months. > > > > On Fri, Dec 13, 2013 at 12:14 AM, Ryan Baldwin > <ryan.baldwin@nexusalpha.com <mailto:ryan.baldwin@nexusalpha.com>> wrote: > > Hi, > > We have a server based on FreeBSD 9.2 which hangs at times on a > daily basis. The longest uptime we have achieved is 5 days > conversely it has stopped daily several days in a row. > > When this occurs it appears there are two proceses stuck in > 'tx->tx' state. In the top output shown these are snapshot-manager > processes which create and destroy snapshots generally and > sometime rollback filesystems to snapshots. When the lockup occurs > other processes which try to access the file system can seem to > end up stuck in state 'rrl->r'. The reboot command that was issued > to try and reboot the server has ended up stuck in this state as > can be seen. > > The server is not under particularly heavy load. > > It has remained in this state for hours. The 'deadman handler'? > does not appear to restart the system. Once this has occurred > there is no further disk activity. > > We did not experience this problem at all previously using 9.1 > although we had less snapshot-manager processes before. We have > built this server against 9.1 again now but it has only been going > one day so far. > > We can try and reproduce this problem again on 9.2 if by doing so > we can gather any additional information that could help resolve > this problem. Please let me know what other information would be > helpful. > > The hardware is a Dell R420 with Perc H310 raid controller in JBOD > mode with the pool mirrored on two SAS disks. > > Thanks > > top and procstat output follow: > > > last pid: 76225; load averages: 0.00, 0.00, 0.00 up 0+20:04:22 > 09:56:24 > 46 processes: 1 running, 44 sleeping, 1 stopped > CPU: % user, % nice, % system, % interrupt, % idle > Mem: 405M Active, 133M Inact, 1206M Wired, 6084M Free > ARC: 797M Total, 184M MFU, 488M MRU, 28M Anon, 32M Header, 65M Other > Swap: 8192M Total, 8192M Free > > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU > COMMAND > 1170 root 1 20 0 16244K 5460K kqread 11 1:28 0.00% > CustomInit > 1531 root 1 20 0 39648K 6368K tx->tx 7 1:17 0.00% > snapshot-manager > 1135 root 1 20 0 16244K 5100K kqread 1 1:10 0.00% > CustomInit > 2097 root 2 20 0 2581M 472M STOP 4 0:46 0.00% > java > 1183 root 1 20 0 16244K 5504K kqread 0 0:22 0.00% > CustomInit > 1145 root 1 20 0 16244K 5464K kqread 9 0:18 0.00% > CustomInit > 1444 root 1 20 0 39648K 6352K tx->tx 5 0:13 0.00% > snapshot-manager > 1628 root 1 20 0 39648K 6348K rrl->r 10 0:12 0.00% > snapshot-manager > 1168 root 1 20 0 16244K 5388K kqread 7 0:07 0.00% > CustomInit > 1535 root 1 20 0 22388K 14296K select 4 0:05 0.00% > openvpn > 1163 root 1 20 0 16244K 5388K kqread 2 0:04 0.00% > CustomInit > 1511 root 1 20 0 18292K 10192K select 0 0:04 0.00% > openvpn > 1156 root 1 20 0 16244K 5392K kqread 10 0:04 0.00% > CustomInit > 1174 root 1 20 0 16244K 5388K kqread 1 0:04 0.00% > CustomInit > 1161 root 1 20 0 16244K 5392K kqread 4 0:03 0.00% > CustomInit > 913 root 1 20 0 12076K 1820K select 7 0:03 0.00% > syslogd > 2102 root 1 20 0 109M 13616K zfsvfs 3 0:03 0.00% > data-processor > 1445 root 1 20 0 22388K 14296K select 5 0:02 0.00% > openvpn > 1014 root 1 20 0 22256K 3360K select 4 0:02 0.00% > ntpd > 1494 root 1 20 0 18292K 10192K select 2 0:01 0.00% > openvpn > 1180 root 1 20 0 18292K 4780K select 0 0:01 0.00% > openvpn > 1505 root 1 20 0 18292K 4712K select 2 0:01 0.00% > openvpn > 1495 root 1 20 0 18292K 4712K select 5 0:00 0.00% > openvpn > 1479 root 1 20 0 18292K 4708K select 1 0:00 0.00% > openvpn > 1567 root 1 20 0 18292K 4712K select 3 0:00 0.00% > openvpn > 1545 root 1 20 0 18292K 4712K select 1 0:00 0.00% > openvpn > 1486 root 1 20 0 18292K 4708K select 0 0:00 0.00% > openvpn > 1443 root 1 20 0 18292K 4436K select 9 0:00 0.00% > openvpn > 1447 root 1 20 0 18292K 4448K select 4 0:00 0.00% > openvpn > 1496 root 1 20 0 18292K 4708K select 3 0:00 0.00% > openvpn > 1478 root 1 20 0 18292K 4704K select 5 0:00 0.00% > openvpn > 1488 root 1 20 0 18292K 10192K select 5 0:00 0.00% > openvpn > 1032 root 1 20 0 14176K 1860K nanslp 1 0:00 0.00% > cron > 76069 root 1 20 0 9948K 1680K rrl->r 7 0:00 0.00% > reboot > 782 root 1 20 0 10376K 4416K select 5 0:00 0.00% > devd > 76196 root 1 20 0 51536K 5828K select 0 0:00 0.00% > sshd > 76203 root 1 20 0 51536K 5828K select 2 0:00 0.00% > sshd > 76205 root 1 20 0 17564K 3252K ttyin 11 0:00 0.00% csh > 76198 root 1 20 0 17564K 3252K pause 8 0:00 0.00% csh > 75718 root 1 20 0 35556K 3956K rrl->r 11 0:00 0.00% > snapshot-manager-counts > 75715 root 1 20 0 35556K 3956K rrl->r 10 0:00 0.00% > snapshot-manager-counts > 1181 root 1 52 0 12084K 1676K ttyin 1 0:00 0.00% > getty > > > 0 100000 kernel swapper mi_switch+0x186 > sleepq_timedwait+0x42 _sleep+0x1c9 scheduler+0x359 mi_startup+0x77 > btext+0x2c > 0 100032 kernel firmware taskq mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100037 kernel ffs_trim taskq mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100038 kernel acpi_task_0 mi_switch+0x186 > sleepq_wait+0x42 msleep_spin+0x194 taskqueue_thread_loop+0x67 > fork_exit+0x11f fork_trampoline+0xe > 0 100039 kernel acpi_task_1 mi_switch+0x186 > sleepq_wait+0x42 msleep_spin+0x194 taskqueue_thread_loop+0x67 > fork_exit+0x11f fork_trampoline+0xe > 0 100040 kernel acpi_task_2 mi_switch+0x186 > sleepq_wait+0x42 msleep_spin+0x194 taskqueue_thread_loop+0x67 > fork_exit+0x11f fork_trampoline+0xe > 0 100041 kernel kqueue taskq mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100044 kernel thread taskq mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100052 kernel bge0 taskq mi_switch+0x186 > sleepq_wait+0x42 msleep_spin+0x194 taskqueue_thread_loop+0x67 > fork_exit+0x11f fork_trampoline+0xe > 0 100053 kernel bge1 taskq mi_switch+0x186 > sleepq_wait+0x42 msleep_spin+0x194 taskqueue_thread_loop+0x67 > fork_exit+0x11f fork_trampoline+0xe > 0 100062 kernel mca taskq mi_switch+0x186 > sleepq_wait+0x42 msleep_spin+0x194 taskqueue_thread_loop+0x67 > fork_exit+0x11f fork_trampoline+0xe > 0 100063 kernel system_taskq_0 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100064 kernel system_taskq_1 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100065 kernel system_taskq_2 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100066 kernel system_taskq_3 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100067 kernel system_taskq_4 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100068 kernel system_taskq_5 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100069 kernel system_taskq_6 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100070 kernel system_taskq_7 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100071 kernel system_taskq_8 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100072 kernel system_taskq_9 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100073 kernel system_taskq_10 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100074 kernel system_taskq_11 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100247 kernel zio_null_issue mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100248 kernel zio_null_intr mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100249 kernel zio_read_issue_0 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100250 kernel zio_read_issue_1 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100251 kernel zio_read_issue_2 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100252 kernel zio_read_issue_3 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100253 kernel zio_read_issue_4 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100254 kernel zio_read_issue_5 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100255 kernel zio_read_issue_6 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100256 kernel zio_read_issue_7 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100257 kernel zio_read_intr_0 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100258 kernel zio_read_intr_1 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100259 kernel zio_read_intr_2 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100260 kernel zio_read_intr_3 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100261 kernel zio_read_intr_4 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100262 kernel zio_read_intr_5 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100263 kernel zio_read_intr_6 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100264 kernel zio_read_intr_7 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100265 kernel zio_read_intr_8 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100266 kernel zio_read_intr_9 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100267 kernel zio_read_intr_10 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100268 kernel zio_read_intr_11 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100269 kernel zio_write_issue_ mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100270 kernel zio_write_issue_ mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100271 kernel zio_write_issue_ mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100272 kernel zio_write_issue_ mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100273 kernel zio_write_issue_ mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100274 kernel zio_write_issue_ mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100275 kernel zio_write_issue_ mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100276 kernel zio_write_issue_ mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100277 kernel zio_write_issue_ mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100278 kernel zio_write_issue_ mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100279 kernel zio_write_issue_ mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100280 kernel zio_write_issue_ mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100281 kernel zio_write_issue_ mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100282 kernel zio_write_issue_ mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100283 kernel zio_write_issue_ mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100284 kernel zio_write_issue_ mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100285 kernel zio_write_issue_ mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100286 kernel zio_write_intr_0 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100287 kernel zio_write_intr_1 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100288 kernel zio_write_intr_2 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100289 kernel zio_write_intr_3 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100290 kernel zio_write_intr_4 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100291 kernel zio_write_intr_5 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100292 kernel zio_write_intr_6 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100293 kernel zio_write_intr_7 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100294 kernel zio_write_intr_h mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100295 kernel zio_write_intr_h mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100296 kernel zio_write_intr_h mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100297 kernel zio_write_intr_h mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100298 kernel zio_write_intr_h mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100299 kernel zio_free_issue_0 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100300 kernel zio_free_issue_1 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100301 kernel zio_free_issue_2 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100302 kernel zio_free_issue_3 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100303 kernel zio_free_issue_4 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100304 kernel zio_free_issue_5 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100305 kernel zio_free_issue_6 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100306 kernel zio_free_issue_7 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100307 kernel zio_free_issue_8 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100308 kernel zio_free_issue_9 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100309 kernel zio_free_issue_1 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100310 kernel zio_free_issue_1 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100311 kernel zio_free_issue_1 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100312 kernel zio_free_issue_1 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100313 kernel zio_free_issue_1 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100314 kernel zio_free_issue_1 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100315 kernel zio_free_issue_1 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100316 kernel zio_free_issue_1 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100317 kernel zio_free_issue_1 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100318 kernel zio_free_issue_1 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100319 kernel zio_free_issue_2 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100320 kernel zio_free_issue_2 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100321 kernel zio_free_issue_2 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100322 kernel zio_free_issue_2 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100323 kernel zio_free_issue_2 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100324 kernel zio_free_issue_2 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100325 kernel zio_free_issue_2 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100326 kernel zio_free_issue_2 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100327 kernel zio_free_issue_2 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100328 kernel zio_free_issue_2 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100329 kernel zio_free_issue_3 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100330 kernel zio_free_issue_3 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100331 kernel zio_free_issue_3 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100332 kernel zio_free_issue_3 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100333 kernel zio_free_issue_3 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100334 kernel zio_free_issue_3 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100335 kernel zio_free_issue_3 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100336 kernel zio_free_issue_3 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100337 kernel zio_free_issue_3 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100338 kernel zio_free_issue_3 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100339 kernel zio_free_issue_4 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100340 kernel zio_free_issue_4 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100341 kernel zio_free_issue_4 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100342 kernel zio_free_issue_4 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100343 kernel zio_free_issue_4 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100344 kernel zio_free_issue_4 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100345 kernel zio_free_issue_4 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100346 kernel zio_free_issue_4 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100347 kernel zio_free_issue_4 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100348 kernel zio_free_issue_4 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100349 kernel zio_free_issue_5 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100350 kernel zio_free_issue_5 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100351 kernel zio_free_issue_5 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100352 kernel zio_free_issue_5 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100353 kernel zio_free_issue_5 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100354 kernel zio_free_issue_5 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100355 kernel zio_free_issue_5 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100356 kernel zio_free_issue_5 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100357 kernel zio_free_issue_5 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100358 kernel zio_free_issue_5 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100359 kernel zio_free_issue_6 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100360 kernel zio_free_issue_6 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100361 kernel zio_free_issue_6 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100362 kernel zio_free_issue_6 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100363 kernel zio_free_issue_6 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100364 kernel zio_free_issue_6 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100365 kernel zio_free_issue_6 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100366 kernel zio_free_issue_6 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100367 kernel zio_free_issue_6 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100368 kernel zio_free_issue_6 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100369 kernel zio_free_issue_7 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100370 kernel zio_free_issue_7 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100371 kernel zio_free_issue_7 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100372 kernel zio_free_issue_7 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100373 kernel zio_free_issue_7 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100374 kernel zio_free_issue_7 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100375 kernel zio_free_issue_7 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100376 kernel zio_free_issue_7 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100377 kernel zio_free_issue_7 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100378 kernel zio_free_issue_7 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100379 kernel zio_free_issue_8 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100380 kernel zio_free_issue_8 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100381 kernel zio_free_issue_8 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100382 kernel zio_free_issue_8 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100383 kernel zio_free_issue_8 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100384 kernel zio_free_issue_8 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100385 kernel zio_free_issue_8 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100386 kernel zio_free_issue_8 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100387 kernel zio_free_issue_8 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100388 kernel zio_free_issue_8 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100389 kernel zio_free_issue_9 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100390 kernel zio_free_issue_9 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100391 kernel zio_free_issue_9 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100392 kernel zio_free_issue_9 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100393 kernel zio_free_issue_9 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100394 kernel zio_free_issue_9 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100395 kernel zio_free_issue_9 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100396 kernel zio_free_issue_9 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100397 kernel zio_free_issue_9 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100398 kernel zio_free_issue_9 mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100399 kernel zio_free_intr mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100400 kernel zio_claim_issue mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100401 kernel zio_claim_intr mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100402 kernel zio_ioctl_issue mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100403 kernel zio_ioctl_intr mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100405 kernel zfs_vn_rele_task mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100418 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100441 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100477 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100478 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100479 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100480 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100481 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100482 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100484 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100486 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100488 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100489 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100490 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100491 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100492 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100493 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100494 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100495 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100496 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100497 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100498 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100499 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100636 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 0 100638 kernel zil_clean mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 taskqueue_thread_loop+0xbb > fork_exit+0x11f fork_trampoline+0xe > 1 100002 init - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_wait_sig+0x16 _sleep+0x295 > kern_wait6+0x8c3 kern_wait+0x9c sys_wait4+0x35 amd64_syscall+0x540 > Xfast_syscall+0xf7 > 2 100033 crypto - mi_switch+0x186 sleepq_wait+0x42 > _sleep+0x379 crypto_proc+0x197 fork_exit+0x11f fork_trampoline+0xe > 3 100034 <tel:3%20100034> crypto returns - > mi_switch+0x186 sleepq_wait+0x42 _sleep+0x379 > crypto_ret_proc+0x192 fork_exit+0x11f fork_trampoline+0xe > 4 100061 ctl_thrd - mi_switch+0x186 sleepq_wait+0x42 > _sleep+0x379 ctl_work_thread+0x2fd0 fork_exit+0x11f > fork_trampoline+0xe > 5 100075 zfskern arc_reclaim_thre mi_switch+0x186 > sleepq_timedwait+0x42 _cv_timedwait+0x135 arc_reclaim_thread+0x29d > fork_exit+0x11f fork_trampoline+0xe > 5 100076 zfskern l2arc_feed_threa mi_switch+0x186 > sleepq_timedwait+0x42 _cv_timedwait+0x135 l2arc_feed_thread+0x1a2 > fork_exit+0x11f fork_trampoline+0xe > 5 100404 zfskern trim nsgroot mi_switch+0x186 > sleepq_timedwait+0x42 _cv_timedwait+0x135 trim_thread+0x68 > fork_exit+0x11f fork_trampoline+0xe > 5 100406 zfskern txg_thread_enter mi_switch+0x186 > sleepq_wait+0x42 _cv_wait+0x112 txg_thread_wait+0x79 > txg_quiesce_thread+0xbb fork_exit+0x11f fork_trampoline+0xe > 5 100407 zfskern txg_thread_enter mi_switch+0x186 > sleepq_wait+0x42 _cv_wait+0x112 rrw_enter_write+0x35 > dsl_sync_task_sync+0xb8 dsl_pool_sync+0x47d spa_sync+0x3ba > txg_sync_thread+0x139 fork_exit+0x11f fork_trampoline+0xe > 5 100408 zfskern zvol nsgroot/swa mi_switch+0x186 > sleepq_wait+0x42 _sleep+0x379 zvol_geom_worker+0xfc > fork_exit+0x11f fork_trampoline+0xe > 6 100077 sctp_iterator - mi_switch+0x186 sleepq_wait+0x42 > _sleep+0x379 sctp_iterator_thread+0x41 fork_exit+0x11f > fork_trampoline+0xe > 7 100078 xpt_thrd - mi_switch+0x186 sleepq_wait+0x42 > _sleep+0x379 xpt_scanner_thread+0xff fork_exit+0x11f > fork_trampoline+0xe > 8 100079 ipmi0: kcs - mi_switch+0x186 sleepq_wait+0x42 > _cv_wait+0x112 ipmi_dequeue_request+0x47 kcs_loop+0x3d > fork_exit+0x11f fork_trampoline+0xe > 9 100080 enc_daemon0 - mi_switch+0x186 sleepq_wait+0x42 > _sleep+0x379 enc_daemon+0xe4 fork_exit+0x11f fork_trampoline+0xe > 10 100001 audit - mi_switch+0x186 sleepq_wait+0x42 > _cv_wait+0x112 audit_worker+0x359 fork_exit+0x11f fork_trampoline+0xe > 11 100003 idle idle: cpu0 <running> > 11 100004 idle idle: cpu1 <running> > 11 100005 idle idle: cpu2 <running> > 11 100006 idle idle: cpu3 <running> > 11 100007 idle idle: cpu4 <running> > 11 100008 idle idle: cpu5 <running> > 11 100009 idle idle: cpu6 <running> > 11 100010 idle idle: cpu7 <running> > 11 100011 idle idle: cpu8 <running> > 11 100012 idle idle: cpu9 <running> > 11 100013 idle idle: cpu10 mi_switch+0x186 > critical_exit+0xa5 sched_idletd+0x118 fork_exit+0x11f > fork_trampoline+0xe > 11 100014 idle idle: cpu11 <running> > 12 100015 intr swi4: clock mi_switch+0x186 > ithread_loop+0x21e fork_exit+0x11f fork_trampoline+0xe > 12 100016 intr swi4: clock mi_switch+0x186 > ithread_loop+0x21e fork_exit+0x11f fork_trampoline+0xe > 12 100017 intr swi4: clock mi_switch+0x186 > ithread_loop+0x21e fork_exit+0x11f fork_trampoline+0xe > 12 100018 intr swi4: clock mi_switch+0x186 > ithread_loop+0x21e fork_exit+0x11f fork_trampoline+0xe > 12 100019 intr swi4: clock mi_switch+0x186 > ithread_loop+0x21e fork_exit+0x11f fork_trampoline+0xe > 12 100020 intr swi4: clock mi_switch+0x186 > ithread_loop+0x21e fork_exit+0x11f fork_trampoline+0xe > 12 100021 intr swi4: clock mi_switch+0x186 > ithread_loop+0x21e fork_exit+0x11f fork_trampoline+0xe > 12 100022 intr swi4: clock mi_switch+0x186 > ithread_loop+0x21e fork_exit+0x11f fork_trampoline+0xe > 12 100023 intr swi4: clock mi_switch+0x186 > ithread_loop+0x21e fork_exit+0x11f fork_trampoline+0xe > 12 100024 intr swi4: clock mi_switch+0x186 > ithread_loop+0x21e fork_exit+0x11f fork_trampoline+0xe > 12 100025 intr swi4: clock mi_switch+0x186 > ithread_loop+0x21e fork_exit+0x11f fork_trampoline+0xe > 12 100026 intr swi4: clock mi_switch+0x186 > ithread_loop+0x21e fork_exit+0x11f fork_trampoline+0xe > 12 100027 intr swi3: vm > 12 100028 intr swi1: netisr 0 mi_switch+0x186 > ithread_loop+0x21e fork_exit+0x11f fork_trampoline+0xe > 12 100036 intr swi6: task queue mi_switch+0x186 > ithread_loop+0x21e fork_exit+0x11f fork_trampoline+0xe > 12 100042 intr swi2: cambio mi_switch+0x186 > ithread_loop+0x21e fork_exit+0x11f fork_trampoline+0xe > 12 100043 intr swi5: fast taskq > 12 100045 intr swi6: Giant task mi_switch+0x186 > ithread_loop+0x21e fork_exit+0x11f fork_trampoline+0xe > 12 100046 intr irq264: mfi0 mi_switch+0x186 > ithread_loop+0x21e fork_exit+0x11f fork_trampoline+0xe > 12 100047 intr irq23: ehci0 mi_switch+0x186 > ithread_loop+0x21e fork_exit+0x11f fork_trampoline+0xe > 12 100054 intr irq22: ehci1 mi_switch+0x186 > ithread_loop+0x21e fork_exit+0x11f fork_trampoline+0xe > 12 100059 intr irq267: ahci0 > 12 100060 intr swi0: uart uart > 13 100029 <tel:13%20100029> geom g_event > mi_switch+0x186 sleepq_wait+0x42 _sleep+0x379 g_run_events+0x440 > fork_exit+0x11f fork_trampoline+0xe > 13 100030 <tel:13%20100030> geom g_up > mi_switch+0x186 sleepq_wait+0x42 _sleep+0x379 > g_io_schedule_up+0xe6 g_up_procbody+0x5c fork_exit+0x11f > fork_trampoline+0xe > 13 100031 <tel:13%20100031> geom g_down > mi_switch+0x186 sleepq_wait+0x42 _sleep+0x379 > g_io_schedule_down+0x25f g_down_procbody+0x5c fork_exit+0x11f > fork_trampoline+0xe > 14 100035 yarrow - mi_switch+0x186 > sleepq_timedwait+0x42 _sleep+0x1c9 random_kthread+0x1ea > fork_exit+0x11f fork_trampoline+0xe > 15 100048 usb usbus0 mi_switch+0x186 > sleepq_wait+0x42 _cv_wait+0x112 usb_process+0x18b fork_exit+0x11f > fork_trampoline+0xe > 15 100049 usb usbus0 mi_switch+0x186 > sleepq_wait+0x42 _cv_wait+0x112 usb_process+0x18b fork_exit+0x11f > fork_trampoline+0xe > 15 100050 usb usbus0 mi_switch+0x186 > sleepq_wait+0x42 _cv_wait+0x112 usb_process+0x18b fork_exit+0x11f > fork_trampoline+0xe > 15 100051 usb usbus0 mi_switch+0x186 > sleepq_wait+0x42 _cv_wait+0x112 usb_process+0x18b fork_exit+0x11f > fork_trampoline+0xe > 15 100055 usb usbus1 mi_switch+0x186 > sleepq_wait+0x42 _cv_wait+0x112 usb_process+0x18b fork_exit+0x11f > fork_trampoline+0xe > 15 100056 usb usbus1 mi_switch+0x186 > sleepq_wait+0x42 _cv_wait+0x112 usb_process+0x18b fork_exit+0x11f > fork_trampoline+0xe > 15 100057 usb usbus1 mi_switch+0x186 > sleepq_wait+0x42 _cv_wait+0x112 usb_process+0x18b fork_exit+0x11f > fork_trampoline+0xe > 15 100058 usb usbus1 mi_switch+0x186 > sleepq_wait+0x42 _cv_wait+0x112 usb_process+0x18b fork_exit+0x11f > fork_trampoline+0xe > 16 100081 pagedaemon - mi_switch+0x186 > sleepq_timedwait+0x42 _sleep+0x1c9 vm_pageout+0xb34 > fork_exit+0x11f fork_trampoline+0xe > 17 100082 vmdaemon - mi_switch+0x186 sleepq_wait+0x42 > _sleep+0x379 vm_daemon+0x58 fork_exit+0x11f fork_trampoline+0xe > 18 100083 pagezero - mi_switch+0x186 > sleepq_timedwait+0x42 _sleep+0x1c9 vm_pagezero+0x83 > fork_exit+0x11f fork_trampoline+0xe > 19 100084 bufdaemon - mi_switch+0x186 > sleepq_timedwait+0x42 _sleep+0x1c9 buf_daemon+0x1e1 > fork_exit+0x11f fork_trampoline+0xe > 20 100085 syncer - mi_switch+0x186 sleepq_wait+0x42 > _cv_wait+0x112 rrw_enter_read+0x4b zfs_sync+0x5e sync_fsync+0x1a2 > VOP_FSYNC_APV+0x68 sync_vnode+0x16b sched_sync+0x1c5 > fork_exit+0x11f fork_trampoline+0xe > 21 100086 vnlru - mi_switch+0x186 sleepq_wait+0x42 > _sx_slock_hard+0x318 _sx_slock+0x56 zfs_freebsd_reclaim+0x4b > VOP_RECLAIM_APV+0x68 vgonel+0x134 vnlru_free+0x362 > vnlru_proc+0x61e fork_exit+0x11f fork_trampoline+0xe > 22 100087 softdepflush - mi_switch+0x186 > sleepq_timedwait+0x42 _sleep+0x1c9 softdep_flush+0x375 > fork_exit+0x11f fork_trampoline+0xe > 782 100432 devd - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_timedwait_sig+0x19 > _cv_timedwait_sig+0x135 seltdwait+0x8d kern_select+0x6ef > sys_select+0x5d amd64_syscall+0x540 Xfast_syscall+0xf7 > 790 100545 pfpurge - mi_switch+0x186 > sleepq_timedwait+0x42 _sleep+0x1c9 pf_purge_thread+0x31 > fork_exit+0x11f fork_trampoline+0xe > 913 100449 syslogd - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_wait_sig+0x16 _cv_wait_sig+0x12a > seltdwait+0xf6 kern_select+0x6ef sys_select+0x5d > amd64_syscall+0x540 Xfast_syscall+0xf7 > 1014 100581 ntpd - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_wait_sig+0x16 _cv_wait_sig+0x12a > seltdwait+0xf6 kern_select+0x6ef sys_select+0x5d > amd64_syscall+0x540 Xfast_syscall+0xf7 > 1028 100532 sshd - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_wait_sig+0x16 _cv_wait_sig+0x12a > seltdwait+0xf6 kern_select+0x6ef sys_select+0x5d > amd64_syscall+0x540 Xfast_syscall+0xf7 > 1032 100473 cron - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_timedwait_sig+0x19 _sleep+0x2ca > kern_nanosleep+0x118 sys_nanosleep+0x6e amd64_syscall+0x540 > Xfast_syscall+0xf7 > 1135 100558 CustomInit - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_wait_sig+0x16 _sleep+0x295 > kern_kevent+0x369 sys_kevent+0x90 amd64_syscall+0x540 > Xfast_syscall+0xf7 > 1145 100509 CustomInit - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_wait_sig+0x16 _sleep+0x295 > kern_kevent+0x369 sys_kevent+0x90 amd64_syscall+0x540 > Xfast_syscall+0xf7 > 1156 100508 CustomInit - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_wait_sig+0x16 _sleep+0x295 > kern_kevent+0x369 sys_kevent+0x90 amd64_syscall+0x540 > Xfast_syscall+0xf7 > 1161 100507 CustomInit - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_wait_sig+0x16 _sleep+0x295 > kern_kevent+0x369 sys_kevent+0x90 amd64_syscall+0x540 > Xfast_syscall+0xf7 > 1163 100521 CustomInit - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_wait_sig+0x16 _sleep+0x295 > kern_kevent+0x369 sys_kevent+0x90 amd64_syscall+0x540 > Xfast_syscall+0xf7 > 1168 100574 CustomInit - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_wait_sig+0x16 _sleep+0x295 > kern_kevent+0x369 sys_kevent+0x90 amd64_syscall+0x540 > Xfast_syscall+0xf7 > 1170 100561 CustomInit - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_wait_sig+0x16 _sleep+0x295 > kern_kevent+0x369 sys_kevent+0x90 amd64_syscall+0x540 > Xfast_syscall+0xf7 > 1174 100560 CustomInit - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_wait_sig+0x16 _sleep+0x295 > kern_kevent+0x369 sys_kevent+0x90 amd64_syscall+0x540 > Xfast_syscall+0xf7 > 1180 100420 openvpn - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_timedwait_sig+0x19 > _cv_timedwait_sig+0x135 seltdwait+0x8d sys_poll+0x478 > amd64_syscall+0x540 Xfast_syscall+0xf7 > 1181 100463 getty - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_wait_sig+0x16 _cv_wait_sig+0x12a > tty_wait+0x25 ttydisc_read+0x2dd ttydev_read+0xc4 > devfs_read_f+0x90 dofileread+0xa1 kern_readv+0x6c sys_read+0x64 > amd64_syscall+0x540 Xfast_syscall+0xf7 > 1182 100430 getty - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_wait_sig+0x16 _cv_wait_sig+0x12a > tty_wait+0x25 ttydisc_read+0x2dd ttydev_read+0xc4 > devfs_read_f+0x90 dofileread+0xa1 kern_readv+0x6c sys_read+0x64 > amd64_syscall+0x540 Xfast_syscall+0xf7 > 1183 100421 CustomInit - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_wait_sig+0x16 _sleep+0x295 > kern_kevent+0x369 sys_kevent+0x90 amd64_syscall+0x540 > Xfast_syscall+0xf7 > 1443 100599 openvpn - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_timedwait_sig+0x19 > _cv_timedwait_sig+0x135 seltdwait+0x8d sys_poll+0x478 > amd64_syscall+0x540 Xfast_syscall+0xf7 > 1444 100527 snapshot-manager - mi_switch+0x186 > sleepq_wait+0x42 _cv_wait+0x112 txg_wait_synced+0x85 > dsl_sync_task+0x139 dsl_destroy_snapshots_nvl+0x71 > dsl_destroy_snapshot+0x4a zfs_ioc_destroy+0x3e zfsdev_ioctl+0x58d > devfs_ioctl_f+0x7b kern_ioctl+0x106 sys_ioctl+0xfd > amd64_syscall+0x540 Xfast_syscall+0xf7 > 1445 100514 openvpn - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_timedwait_sig+0x19 > _cv_timedwait_sig+0x135 seltdwait+0x8d sys_poll+0x478 > amd64_syscall+0x540 Xfast_syscall+0xf7 > 1447 100557 openvpn - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_timedwait_sig+0x19 > _cv_timedwait_sig+0x135 seltdwait+0x8d sys_poll+0x478 > amd64_syscall+0x540 Xfast_syscall+0xf7 > 1478 100555 openvpn - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_timedwait_sig+0x19 > _cv_timedwait_sig+0x135 seltdwait+0x8d sys_poll+0x478 > amd64_syscall+0x540 Xfast_syscall+0xf7 > 1479 100551 openvpn - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_timedwait_sig+0x19 > _cv_timedwait_sig+0x135 seltdwait+0x8d sys_poll+0x478 > amd64_syscall+0x540 Xfast_syscall+0xf7 > 1486 100576 openvpn - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_timedwait_sig+0x19 > _cv_timedwait_sig+0x135 seltdwait+0x8d sys_poll+0x478 > amd64_syscall+0x540 Xfast_syscall+0xf7 > 1488 100563 openvpn - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_timedwait_sig+0x19 > _cv_timedwait_sig+0x135 seltdwait+0x8d sys_poll+0x478 > amd64_syscall+0x540 Xfast_syscall+0xf7 > 1494 100575 openvpn - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_timedwait_sig+0x19 > _cv_timedwait_sig+0x135 seltdwait+0x8d sys_poll+0x478 > amd64_syscall+0x540 Xfast_syscall+0xf7 > 1495 100442 openvpn - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_timedwait_sig+0x19 > _cv_timedwait_sig+0x135 seltdwait+0x8d sys_poll+0x478 > amd64_syscall+0x540 Xfast_syscall+0xf7 > 1496 100556 openvpn - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_timedwait_sig+0x19 > _cv_timedwait_sig+0x135 seltdwait+0x8d sys_poll+0x478 > amd64_syscall+0x540 Xfast_syscall+0xf7 > 1505 100431 openvpn - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_timedwait_sig+0x19 > _cv_timedwait_sig+0x135 seltdwait+0x8d sys_poll+0x478 > amd64_syscall+0x540 Xfast_syscall+0xf7 > 1511 100475 openvpn - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_timedwait_sig+0x19 > _cv_timedwait_sig+0x135 seltdwait+0x8d sys_poll+0x478 > amd64_syscall+0x540 Xfast_syscall+0xf7 > 1531 100429 snapshot-manager - mi_switch+0x186 > sleepq_wait+0x42 _cv_wait+0x112 txg_wait_synced+0x85 > dsl_sync_task+0x139 zfs_ioc_rollback+0xb8 zfsdev_ioctl+0x58d > devfs_ioctl_f+0x7b kern_ioctl+0x106 sys_ioctl+0xfd > amd64_syscall+0x540 Xfast_syscall+0xf7 > 1535 100434 openvpn - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_timedwait_sig+0x19 > _cv_timedwait_sig+0x135 seltdwait+0x8d sys_poll+0x478 > amd64_syscall+0x540 Xfast_syscall+0xf7 > 1545 100459 openvpn - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_timedwait_sig+0x19 > _cv_timedwait_sig+0x135 seltdwait+0x8d sys_poll+0x478 > amd64_syscall+0x540 Xfast_syscall+0xf7 > 1567 100539 openvpn - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_timedwait_sig+0x19 > _cv_timedwait_sig+0x135 seltdwait+0x8d sys_poll+0x478 > amd64_syscall+0x540 Xfast_syscall+0xf7 > 1628 100454 snapshot-manager - mi_switch+0x186 > sleepq_wait+0x42 _cv_wait+0x112 rrw_enter_read+0x4b > dsl_pool_hold+0x44 dmu_objset_hold+0x2a zfs_ioc_objset_stats+0x23 > zfsdev_ioctl+0x58d devfs_ioctl_f+0x7b kern_ioctl+0x106 > sys_ioctl+0xfd amd64_syscall+0x540 Xfast_syscall+0xf7 > 2097 100710 <tel:2097%20100710> java - > mi_switch+0x186 thread_suspend_switch+0xcd thread_single+0x1b2 > exit1+0x72 sys_sys_exit+0xe amd64_syscall+0x540 Xfast_syscall+0xf7 > 2097 100792 <tel:2097%20100792> java - > mi_switch+0x186 sleepq_wait+0x42 __lockmgr_args+0x5cb > vop_stdlock+0x39 VOP_LOCK1_APV+0x70 _vn_lock+0x47 > zfsctl_freebsd_root_lookup+0xd5 VOP_LOOKUP_APV+0x62 lookup+0x437 > namei+0x4ac kern_statat_vnhook+0xb3 kern_statat+0x15 sys_stat+0x2a > amd64_syscall+0x540 Xfast_syscall+0xf7 > 2102 100635 <tel:2102%20100635> data-processor dm_siri_to_chiro > mi_switch+0x186 sleepq_wait+0x42 _sx_slock_hard+0x318 > _sx_slock+0x56 zfs_freebsd_reclaim+0x4b VOP_RECLAIM_APV+0x68 > vgonel+0x134 vnlru_free+0x362 getnewvnode+0x27d > gfs_file_create+0x4b gfs_dir_create+0x16 > zfsctl_snapdir_lookup+0x474 VOP_LOOKUP_APV+0x62 lookup+0x437 > namei+0x4ac kern_statat_vnhook+0xb3 kern_statat+0x15 sys_stat+0x2a > 75715 100424 snapshot-manager-counts mi_switch+0x186 > sleepq_wait+0x42 _cv_wait+0x112 rrw_enter_read+0x4b > dsl_pool_hold+0x44 dmu_objset_hold+0x2a zfs_ioc_objset_stats+0x23 > zfsdev_ioctl+0x58d devfs_ioctl_f+0x7b kern_ioctl+0x106 > sys_ioctl+0xfd amd64_syscall+0x540 Xfast_syscall+0xf7 > 75718 100861 snapshot-manager-counts mi_switch+0x186 > sleepq_wait+0x42 _cv_wait+0x112 rrw_enter_read+0x4b > dsl_pool_hold+0x44 dmu_objset_hold+0x2a zfs_ioc_objset_stats+0x23 > zfsdev_ioctl+0x58d devfs_ioctl_f+0x7b kern_ioctl+0x106 > sys_ioctl+0xfd amd64_syscall+0x540 Xfast_syscall+0xf7 > 75719 100578 snapshot-manager-counts mi_switch+0x186 > sleepq_wait+0x42 _cv_wait+0x112 rrw_enter_read+0x4b > dsl_pool_hold+0x44 dmu_objset_hold+0x2a zfs_ioc_objset_stats+0x23 > zfsdev_ioctl+0x58d devfs_ioctl_f+0x7b kern_ioctl+0x106 > sys_ioctl+0xfd amd64_syscall+0x540 Xfast_syscall+0xf7 > 76069 101690 reboot - mi_switch+0x186 sleepq_wait+0x42 > _cv_wait+0x112 rrw_enter_read+0x4b zfs_sync+0x5e sys_sync+0x1e8 > amd64_syscall+0x540 Xfast_syscall+0xf7 > 76196 100847 sshd - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_wait_sig+0x16 _cv_wait_sig+0x12a > seltdwait+0xf6 kern_select+0x6ef sys_select+0x5d > amd64_syscall+0x540 Xfast_syscall+0xf7 > 76198 101378 csh - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_wait_sig+0x16 _sleep+0x295 > kern_sigsuspend+0xab sys_sigsuspend+0x34 amd64_syscall+0x540 > Xfast_syscall+0xf7 > 76200 100892 top - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_wait_sig+0x16 _cv_wait_sig+0x12a > tty_wait+0x25 ttydisc_read+0x2dd ttydev_read+0xc4 > devfs_read_f+0x90 dofileread+0xa1 kern_readv+0x6c sys_read+0x64 > amd64_syscall+0x540 Xfast_syscall+0xf7 > 76203 100447 sshd - <running> > 76205 101410 csh - mi_switch+0x186 > sleepq_catch_signals+0x2e1 sleepq_wait_sig+0x16 _sleep+0x295 > kern_sigsuspend+0xab sys_sigsuspend+0x34 amd64_syscall+0x540 > Xfast_syscall+0xf7 > 76209 101030 procstat - <running> > > > _______________________________________________ > freebsd-fs@freebsd.org <mailto:freebsd-fs@freebsd.org> mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to > "freebsd-fs-unsubscribe@freebsd.org > <mailto:freebsd-fs-unsubscribe@freebsd.org>" > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?52AC0508.5060704>