From owner-freebsd-fs@FreeBSD.ORG Tue May 8 14:58:34 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E25B1106564A for ; Tue, 8 May 2012 14:58:34 +0000 (UTC) (envelope-from tevans.uk@googlemail.com) Received: from mail-qa0-f47.google.com (mail-qa0-f47.google.com [209.85.216.47]) by mx1.freebsd.org (Postfix) with ESMTP id 9A7098FC16 for ; Tue, 8 May 2012 14:58:34 +0000 (UTC) Received: by qabg1 with SMTP id g1so629457qab.13 for ; Tue, 08 May 2012 07:58:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=M/KJKOHzKPTwW9o7tCJov0qBFeKC8BbkbOk8z7sGi7Y=; b=tIVF1vpw84Ehh72mcbO7kjPx5psF5Xf0Aen7A09jzVUyPwtYtckW79T9qCKsod7WTn 6Pq5gYfwtwYR8PsKetnWo9mvKToRU8cqK316+CLHzFWWezIJcRZESWdjeeHsYaypC2ek jUjysewnz/fQYhydbta8EoHmL4+CXdSOUEY6aO/Ya7F0ua+Es1EVPgUrf3yuyJamcW1O eXT8J54EUDubJxytYCgpkxy9n2mo1qwCssfV5yxUX8iaHcWewi/4OEldSlVd0kJe/FbA WJmlHFLDcE+N7vx+ndcAqLZrO/ElhJLyO8w0+vgtN9SrJQ/T4NNyh/DrCbHlmIi4NV8Z XsGQ== MIME-Version: 1.0 Received: by 10.220.150.12 with SMTP id w12mr7919764vcv.39.1336489113763; Tue, 08 May 2012 07:58:33 -0700 (PDT) Received: by 10.52.28.240 with HTTP; Tue, 8 May 2012 07:58:33 -0700 (PDT) In-Reply-To: <73F8D020-04F3-44B2-97D4-F08E3B253C32@grem.de> References: <73F8D020-04F3-44B2-97D4-F08E3B253C32@grem.de> Date: Tue, 8 May 2012 15:58:33 +0100 Message-ID: From: Tom Evans To: Michael Gmelin Content-Type: text/plain; charset=UTF-8 Cc: freebsd-fs@freebsd.org Subject: Re: ZFS resilvering strangles IO X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 May 2012 14:58:35 -0000 On Tue, May 8, 2012 at 3:33 PM, Michael Gmelin wrote: > So the question is, is there anything I can do to improve the situation? > Is this because of memory constraints? Are there any other knobs to > adjust? As far as I know zfs_resilver_delay can't be changed in FreeBSD yet. > > I have more drives around, so I could replace another one in the server, > just to replicate the exact situation. > In general, raidz is pretty fast, but when it's resilvering it is just too busy. The first thing I would do to speed up writes is to add a log device, preferably a SSD. Having a log device will allow the pool to buffer writes to the pool much more effectively than normally during a resilver. Having lots of small writes will kill read speed during the resilver, which is the critical thing. If your workload would benefit, you could split the SSD down the middle, use half for a log device, and half for a cache device to accelerate reads. I've never tried using a regular disk as a log device, I wonder if that would speed up resilvering? Cheers Tom