From owner-freebsd-stable@FreeBSD.ORG Sun Feb 19 06:37:14 2012 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7C2311065672 for ; Sun, 19 Feb 2012 06:37:14 +0000 (UTC) (envelope-from ask@develooper.com) Received: from mbox1.develooper.com (mbox1.develooper.com [207.171.7.178]) by mx1.freebsd.org (Postfix) with ESMTP id 4B2468FC16 for ; Sun, 19 Feb 2012 06:37:14 +0000 (UTC) Received: (qmail 4990 invoked from network); 19 Feb 2012 06:10:33 -0000 Received: from cpe-76-172-28-38.socal.res.rr.com (HELO embla.bn.dev) (ask@mail.dev@76.172.28.38) by smtp.develooper.com with ESMTPA; 19 Feb 2012 06:10:33 -0000 From: =?iso-8859-1?Q?Ask_Bj=F8rn_Hansen?= Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Sat, 18 Feb 2012 22:10:32 -0800 Message-Id: <770EEEFF-B41D-4851-AD74-C3F96FFB1683@develooper.com> To: freebsd-stable@freebsd.org, freebsd-zfs@freebsd.org Mime-Version: 1.0 (Apple Message framework v1257) X-Mailer: Apple Mail (2.1257) Cc: Subject: Can't read a full block, only got 8193 bytes. X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Feb 2012 06:37:14 -0000 Hi everyone, We're recycling an old database server with room for 16 disks as a = backup server (our old database servers had 12-20 15k disks; the new = ones one or two SSDs and they're faster). We have a box running FreeBSD 8.2 with 7 disks in a ZFS raidz2 (and a = spare). It's using an older 3ware card with all the disks (2TB WD green = "ears" ones) setup as a "single" unit on the 3ware controller and though = slow is basically working great. We have a small program to smartly = purge old snapshots that I wrote after a year and tens of thousands of = snapshots: https://github.com/abh/zfs-snapshot-cleaner The new box is running 9.0 with a 3ware 9690SA-4I4E card with the latest = firmware (4.10.00.024). We're using Seagate 3TB barracuda disks (big = and cheap; good for backups). Now for the problem: When running bonnie++ we get a few ZFS checksum = errors and (weirder) we get this error from bonnie: "Can't read a full block, only got 8193 bytes." This seems to only be when testing a single ZFS disk or a UFS partition. = Testing a raidz1 we just get checksum errors noted in zpool status, but = no errors reading (though read speeds are ~10MB/second across four disks = -- writing sequentially was ~230MB/second). Any ideas where to start look? Our best guess is that the 3ware controller can't play nicely with the = disks; we're planning to try some older/smaller disks on Monday and then = trying the same system and disks with Linux to see if the 3ware driver = there works differently. Ask=