From owner-freebsd-fs@FreeBSD.ORG Sat Oct 20 09:10:04 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3B13DB67 for ; Sat, 20 Oct 2012 09:10:04 +0000 (UTC) (envelope-from dwimsey@rtsz.com) Received: from smtp.rtsz.com (rrcs-24-199-159-90.midsouth.biz.rr.com [24.199.159.90]) by mx1.freebsd.org (Postfix) with ESMTP id 066058FC0A for ; Sat, 20 Oct 2012 09:10:03 +0000 (UTC) Received: from [10.27.1.242] (cpe-107-015-155-065.nc.res.rr.com [107.15.155.65]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by smtp.rtsz.com (Postfix) with ESMTP id E99581CEBE for ; Sat, 20 Oct 2012 04:49:01 -0400 (EDT) From: David Wimsey Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: gptzfsboot very slow Message-Id: Date: Sat, 20 Oct 2012 04:50:22 -0400 To: "freebsd-fs@freebsd.org" Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) X-Mailer: Apple Mail (2.1499) X-RTS-MailScanner-Information: Please contact the System Administrator for more information X-RTS-MailScanner: Found to be clean X-RTS-MailScanner-MCPCheck: X-Spam-Flag: not spam (whitelisted), SpamAssassin (not cached, score=0, required 4, autolearn=not spam) X-RTS-MailScanner-From: dwimsey@rtsz.com X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Oct 2012 09:10:04 -0000 My file server is configured with zfs root based on = http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/Mirror When booting, after it gets past the BIOS drive C: is disk0 (one line = for each of the 6 drives as expected) part, drops to a new line and the = rotating twiddle starts its bit. At first it moves a long at a almost = normal looking speed, then it starts only ticking away slowly, maybe = once or twice a minute. It appears that its scanning the entire drive or something odd. Its = hard to tell if its doing it on all the drives as half of them are on a = RocketRAID card (configured as JBOD) which doesn't have a LED indicator = attached to it for showing activity. There are a total of 6 drives in the machine. 2 drives are SSDs which are sliced up to provide the root mirror vdev, a = mirrored vdev for the zip if the main pool on the machine and each = provide a slice L2ARC. Some of the remaining space is in a small pool=20= 3 of the HDDs are part of a raidz vdev for my main pool. The remaining HDD is a hot spare. If I remove the HDDs from the system and just let the SSDs handle the = boot, its faster but still far longer than it should be, so when I'm in = a hurry I unplug the 4 HDDs, boot, wait for it to get to the FreeBSD = boot menu, plug the HDDs back in and send it on its marry way. This can not be expected behavior in my mind. Why is it doing so much = disk thrashing when the pools are all perfectly clean.=