Date: Sun, 24 Nov 2019 07:44:16 +0000 From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 241980] panic: I/O to pool appears to be hung on vdev Message-ID: <bug-241980-227-6TbQzXnunE@https.bugs.freebsd.org/bugzilla/> In-Reply-To: <bug-241980-227@https.bugs.freebsd.org/bugzilla/> References: <bug-241980-227@https.bugs.freebsd.org/bugzilla/>
next in thread | previous in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D241980 --- Comment #23 from Eugene Grosbein <eugen@freebsd.org> --- As I've suspected, ZFS issues large number of BIO_DELETE/ZIO_TYPE_FREE requ= ests due to massive removal of hourly/daily snapshots for this SSD-only pool. Clearing of SSD cells may be slow. The system has daily cron job started at early morning at 6:05 local time. = It creates many new daily snapshots for ZVOLs and removes old snapshots. First debugging warning I've got at 6:24:36 when ZIO_TYPE_FREE spent 1006/8=3D125= .75 seconds in the avl_tree for active zio's. Then there were more such events (total 50) with increasing "delta" upto 12= 30 seconds at 06:28:26 after which messages stopped. They were not for ZIO_TYPE_FREE(3) zio's only, but for ZIO_TYPE_WRITE(2) too. And for all components of the pool plust for distinct ZIL device /dev/raid/r0s3 that is part of GEOM_RAID's mirror volume based on another pair of SSDs da[01]. I'd like to show all messages appeared in the log for very first event at 06:24:36. sata SLOW IO: zio io_type 3 timestamp 171750435607373ns, delta 100673493209= 3ns, last io 172757121535759ns I/O to pool 'sata' appears to be hung on vdev guid 3313589389580178043 at '/dev/da2.eli' active zio 65 SLOW IO: sata is child of sata sata SLOW IO: zio io_type 2 timestamp 171757174663043ns, delta 999995876423= ns, last io 172756998535496nsI/O to pool 'sata' appears to be hung on vdev guid 17157972224957886076 at '/dev/da3.eli' active zio 65 SLOW IO: sata is child of sata sata SLOW IO: zio io_type 2 timestamp 171757172636758ns, delta 999997902708= ns, last io 172757029559490ns I/O to pool 'sata' appears to be hung on vdev guid 11111943829132819397 at '/dev/da4.eli' active zio 65 SLOW IO: sata is child of sata sata SLOW IO: zio io_type 2 timestamp 171757172636758ns, delta 999997902708= ns, last io 172756997538959ns I/O to pool 'sata' appears to be hung on vdev guid 13912528878380060504 at '/dev/da5.eli' active zio 65 SLOW IO: sata is child of sata sata SLOW IO: zio io_type 2 timestamp 171757172636758ns, delta 999997902708= ns, last io 172756973537187ns I/O to pool 'sata' appears to be hung on vdev guid 18428510808262122301 at '/dev/da6.eli' active zio 65 SLOW IO: sata is child of sata SLOW IO: sata is child of sata sata SLOW IO: zio io_type 2 timestamp 171985896582124ns, delta 771273957342= ns, last io 172756042541811ns I/O to pool 'sata' appears to be hung on vdev guid 15059311870935027496 at '/dev/raid/r0s3' active zio 2 SLOW IO: sata is child of sata --=20 You are receiving this mail because: You are the assignee for the bug.=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-241980-227-6TbQzXnunE>