From owner-freebsd-questions@FreeBSD.ORG Sun Feb 24 04:24:07 2013 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 7C0A79D4 for ; Sun, 24 Feb 2013 04:24:07 +0000 (UTC) (envelope-from johnl@iecc.com) Received: from leila.iecc.com (leila6.iecc.com [IPv6:2001:470:1f07:1126:0:4c:6569:6c61]) by mx1.freebsd.org (Postfix) with ESMTP id 10A171448 for ; Sun, 24 Feb 2013 04:24:06 +0000 (UTC) Received: (qmail 95449 invoked from network); 24 Feb 2013 04:24:03 -0000 Received: from leila.iecc.com (64.57.183.34) by mail1.iecc.com with QMQP; 24 Feb 2013 04:24:03 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:subject:mime-version:content-type:content-transfer-encoding:vbr-info; s=512995e3.xn--btvx9d.k1302; i=johnl@user.iecc.com; bh=FTDTY1oEuNnYKvwVUkERZ8E2Op+Bn5PmwgxEsjVd2vs=; b=mkH+KpeUwcB1l2GA54yI1qlc6U+8MXl+3sghMT6g+jsGHR/K7eKfk1dIfrEIFOzD2X8See65mBtRHIJ41nhXKw8phd6DfMMOKucIcdGV+9QqmXv2ZwkmgONJhTo4Ld/4KGALXs0FzHCaOSdI0He8DLIIlCOI2n14Dm2Qukb9ypc= VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org Date: 24 Feb 2013 04:23:40 -0000 Message-ID: <20130224042340.11192.qmail@joyce.lan> From: "John Levine" To: freebsd-questions@freebsd.org Subject: Strange delays in ZFS scrub or resilver Organization: X-Headerized: yes Mime-Version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Feb 2013 04:24:07 -0000 I have a raidz of three 1 TB SATA drives, in USB enclosures. One of the disks went bad, so I replaced it last night and it's been resilvering ever since. I can watch the activity lights on the disks and it cranks away for a minute or so, then stops for a minute, then cranks for a minute, and so forth. If I do a zpool status while it's stopped, the zpool waits until the I/O resumes, and a ^T shows it waiting for zio->io_cv. I'm running FreeBSD 9.1, amd64 version, totally vanilla install on a mini-itx box with 4GB of RAM. The root/swap disk is an SSD separate from the zfs disks. When the disks are active, top shows about 10% system time and 4% interrupt. When it isn't, top shows about 99.8% idle. The server isn't doing much else, and nothing else currently touches the disks. (They're for remote backup of a system somewhere else, and I have the backup job turned off until resilvering completes.) I'm running this on the console, and there are no disk error messages. Any idea what's going on or how to fix it? I could move the disks to an ESATA enclosure if USB is losing interrupts or something. My recollection is that when I've done a scrub, it does the same thing, work, pause, work, pause. R's, John