Date: Wed, 12 Dec 2007 15:17:29 -0800 From: Peter Losher <Peter_Losher@isc.org> To: FreeBSD Current <freebsd-current@freebsd.org> Subject: ZFS melting under postgres... Message-ID: <47606C09.2070209@isc.org>
next in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigAB71B231D3E7261D9266AEBE Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi, As part of our testing 7.0/ZFS we tried putting it thru it's paces having ZFS act as our storage medium for some test pgsql db's (like for sqlgrey, etc) and in both BETA2 and BETA4 (amd64) we get the same results with a RAIDZ2 container: -=3D- Dec 12 14:24:12 nsa sqlgrey: fatal: setconfig error at /usr/local/sbin/sqlgrey line 186. Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad4 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad6 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad8 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad10 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad12 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad14 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad16 offset=3D3665128448 size=3D21504 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad18 offset=3D3665128448 size=3D21504 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad4 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad6 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad8 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad10 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad12 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad14 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad16 offset=3D3665128448 size=3D21504 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad18 offset=3D3665128448 size=3D21504 Dec 12 16:49:53 nsa root: ZFS: zpool I/O failure, zpool=3Dvault error=3D8= 6 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad4 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad6 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad8 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad10 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad12 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad14 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad16 offset=3D3665128448 size=3D21504 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad18 offset=3D3665128448 size=3D21504 Dec 12 16:49:53 nsa postgres[50527]: [5-1] PANIC: could not write to log file 2, segment 53 at offset 7864320, length 8192: Input/output error= Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad4 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad6 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad8 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad10 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad12 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad14 offset=3D3665128448 size=3D22016 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad16 offset=3D3665128448 size=3D21504 Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=3Dvault path=3D/dev/ad18 offset=3D3665128448 size=3D21504 Dec 12 16:49:53 nsa root: ZFS: zpool I/O failure, zpool=3Dvault error=3D8= 6 Dec 12 16:49:53 nsa postgres[50596]: [1-1] FATAL: the database system is starting up Dec 12 16:49:53 nsa kernel: pid 50527 (postgres), uid 70: exited on signal 6 (core dumped) -=3D- It basically corrupts the container from the inside until it fails completely (usually withing 24-48 hours depending on how busy the db is) I had thought it was a bad SATA replicator/controller, but we had that replaced w/ one from Supermicro. So it's either the disks, or something in ZFS. Anyone used ZFS to backend any db's (mysql or pgsql?) If you need more info, let me know... --=20 Peter_Losher@isc.org | ISC | OpenPGP 0xE8048D08 | "The bits must flow" --------------enigAB71B231D3E7261D9266AEBE Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) iD8DBQFHYGwJPtVx9OgEjQgRAsPuAKCUOz2/Jj9V/+mnK6sdTatoFb9nSwCgsJws PbXTr1C+vOsKWqfdfYZpe9Q= =H4Ak -----END PGP SIGNATURE----- --------------enigAB71B231D3E7261D9266AEBE--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?47606C09.2070209>