Date: Tue, 22 Dec 2020 14:14:43 -0500 From: mike tancsa <mike@sentex.net> To: Mark Johnston <markj@freebsd.org> Cc: FreeBSD-STABLE Mailing List <freebsd-stable@freebsd.org> Subject: Re: zfs panic RELENG_12 Message-ID: <5e2bef97-f124-036f-4e71-874707925ef0@sentex.net> In-Reply-To: <7f49e64d-c875-f12d-744e-7b174e197cbb@sentex.net> References: <878824fe-dde2-b551-4685-e8bd27371275@sentex.net> <5b3415cb-2176-895e-9d22-4f4f0f359d85@sentex.net> <X%2BILmE3TELhUko3B@raichu> <7f49e64d-c875-f12d-744e-7b174e197cbb@sentex.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On 12/22/2020 10:09 AM, mike tancsa wrote: > On 12/22/2020 10:07 AM, Mark Johnston wrote: >> Could you go to frame 11 and print zone->uz_name and >> bucket->ub_bucket[18]? I'm wondering if the item pointer was mangled >> somehow. > Thank you for looking! > > (kgdb) frame 11 > > #11 0xffffffff80ca47d4 in bucket_drain (zone=3D0xfffff800037da000, > bucket=3D0xfffff801c7fd5200) at /usr/src/sys/vm/uma_core.c:758 > 758=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 zone->uz_release(zone->uz_arg, bucket->ub_bucket, > bucket->ub_cnt); > (kgdb) p zone->uz_name > $1 =3D 0xffffffff8102118a "mbuf_jumbo_9k" > (kgdb) p bucket->ub_bucket[18] > $2 =3D (void *) 0xfffff80de4654000 > (kgdb) p bucket->ub_bucket=C2=A0=C2=A0=C2=A0 > $3 =3D 0xfffff801c7fd5218 > > (kgdb) > Not sure if its coincidence or not, but previously I was running with arc being limited to ~30G of the 64G of RAM on the box.=C2=A0 I removed t= hat limit a few weeks ago after upgrading the box to RELENG_12 to pull in the OpenSSL changes.=C2=A0 The panic seems to happen under disk load. I h= ave 3 zfs pools that are pretty busy receiving snapshots. One day a week, we write a full set to a 4th zfs pool off some geli attached drives via USB for offsite cold storage.=C2=A0 The crashes happened with that extra leve= l of disk work.=C2=A0 gstat shows most of the 12 drives off 2 mrsas controller= s at or close to 100% busy during the 18hrs it takes to dump out the files. Trying a new cold storage run now with the arc limit back to vfs.zfs.arc_max=3D29334498304 =C2=A0=C2=A0=C2=A0 ---Mike
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5e2bef97-f124-036f-4e71-874707925ef0>