From owner-freebsd-current@FreeBSD.ORG Thu Dec 13 02:58:29 2007 Return-Path: Delivered-To: freebsd-current@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 43D3C16A41A for ; Thu, 13 Dec 2007 02:58:29 +0000 (UTC) (envelope-from hugo@barafranca.com) Received: from mail.barafranca.com (mail.barafranca.com [67.19.101.164]) by mx1.freebsd.org (Postfix) with ESMTP id 31FD313C4D1 for ; Thu, 13 Dec 2007 02:58:28 +0000 (UTC) (envelope-from hugo@barafranca.com) Received: from localhost (localhost [127.0.0.1]) by mail.barafranca.com (Postfix) with ESMTP id 41094C4D85; Thu, 13 Dec 2007 03:04:40 +0000 (UTC) Received: from mail.barafranca.com ([67.19.101.164]) by localhost (mail.barafranca.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 61901-04; Thu, 13 Dec 2007 03:04:00 +0000 (UTC) Received: from nexus.bsdlan.org (a213-22-38-76.cpe.netcabo.pt [213.22.38.76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.barafranca.com (Postfix) with ESMTP id EB5C3C4D3C; Thu, 13 Dec 2007 03:03:59 +0000 (UTC) Message-ID: <47609FE3.8040606@barafranca.com> Date: Thu, 13 Dec 2007 02:58:43 +0000 From: Hugo Silva User-Agent: Thunderbird 2.0.0.6 (X11/20070816) MIME-Version: 1.0 To: freebsd-current@FreeBSD.ORG References: <47606C09.2070209@isc.org> <47609F0A.7010805@clearchain.com> In-Reply-To: <47609F0A.7010805@clearchain.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at barafranca.com X-Spam-Status: No, score=0 tagged_above=-1 required=4 tests=[none] X-Spam-Score: 0 X-Spam-Level: Cc: Benjamin Close Subject: Re: ZFS melting under postgres... X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2007 02:58:29 -0000 Benjamin Close wrote: > Peter Losher wrote: >> Hi, >> >> As part of our testing 7.0/ZFS we tried putting it thru it's paces >> having ZFS act as our storage medium for some test pgsql db's (like for >> sqlgrey, etc) and in both BETA2 and BETA4 (amd64) we get the same >> results with a RAIDZ2 container: >> >> -=- >> Dec 12 14:24:12 nsa sqlgrey: fatal: setconfig error at >> /usr/local/sbin/sqlgrey line 186. >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad4 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad6 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad8 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad10 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad12 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad14 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad16 offset=3665128448 size=21504 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad18 offset=3665128448 size=21504 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad4 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad6 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad8 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad10 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad12 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad14 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad16 offset=3665128448 size=21504 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad18 offset=3665128448 size=21504 >> Dec 12 16:49:53 nsa root: ZFS: zpool I/O failure, zpool=vault error=86 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad4 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad6 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad8 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad10 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad12 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad14 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad16 offset=3665128448 size=21504 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad18 offset=3665128448 size=21504 >> Dec 12 16:49:53 nsa postgres[50527]: [5-1] PANIC: could not write to >> log file 2, segment 53 at offset 7864320, length 8192: Input/output >> error >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad4 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad6 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad8 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad10 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad12 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad14 offset=3665128448 size=22016 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad16 offset=3665128448 size=21504 >> Dec 12 16:49:53 nsa root: ZFS: checksum mismatch, zpool=vault >> path=/dev/ad18 offset=3665128448 size=21504 >> Dec 12 16:49:53 nsa root: ZFS: zpool I/O failure, zpool=vault error=86 >> Dec 12 16:49:53 nsa postgres[50596]: [1-1] FATAL: the database system >> is starting up >> Dec 12 16:49:53 nsa kernel: pid 50527 (postgres), uid 70: exited on >> signal 6 (core dumped) >> -=- >> >> It basically corrupts the container from the inside until it fails >> completely (usually withing 24-48 hours depending on how busy the db is) >> >> I had thought it was a bad SATA replicator/controller, but we had that >> replaced w/ one from Supermicro. So it's either the disks, or something >> in ZFS. Anyone used ZFS to backend any db's (mysql or pgsql?) >> >> If you need more info, let me know... >> >> > Try turning of zil, whilst I don't use a db, I have zfs under high > load. I've found without zil turned off I see checksum corruption as > well: > > /boot/loader.conf > > vfs.zfs.zil_disable=1 > > Cheers, > Benjamin Wouldn't it be a bad idea to disable ZIL ? http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide#Disabling_the_ZIL_.28Don.27t.29 Regards, Hugo > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to > "freebsd-current-unsubscribe@freebsd.org"