From owner-freebsd-fs@freebsd.org Tue Apr 26 22:26:55 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0DE61B1CF8A for ; Tue, 26 Apr 2016 22:26:55 +0000 (UTC) (envelope-from eborisch@gmail.com) Received: from mail-oi0-x234.google.com (mail-oi0-x234.google.com [IPv6:2607:f8b0:4003:c06::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CC01D1CB0 for ; Tue, 26 Apr 2016 22:26:54 +0000 (UTC) (envelope-from eborisch@gmail.com) Received: by mail-oi0-x234.google.com with SMTP id r78so30963259oie.0 for ; Tue, 26 Apr 2016 15:26:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc; bh=Sg/RTPirXqSwqy7NyiqRx00YtgpAf+3Or26LUwDpy5Y=; b=vOlJs7W7EocObdq1ohRsdhqcQoJLzF/P8lc1Jx+QpqhJLqv1rZ2FGi06pTc/JVax4y BcLbYFiN50bzBZHpUYglbLxE6iJVS/fdyaur+ixQ327pkjnmxd1anzSjg/f2DFO5abpR u8x+5It8BeECJtLeEz3MXwwaCaoXZEOyI9OepESPL02jDjtpOMkxoSbqivb/gsEqBXK8 Tyjcj1BLVWEzqCJ5/fawJOwcMJdyga9aMAwcLsDGn1GvvZ5TupPZeLJ7+Ko2/gSniDFL PsiQwtKnLp+r8qTpkmRcSGjRNQiLIbIwaqGlKkRySqchd+GHHkFG8EPkpv71lFcBX3D4 YB4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc; bh=Sg/RTPirXqSwqy7NyiqRx00YtgpAf+3Or26LUwDpy5Y=; b=c9G98q1lz7h7J1cwdVc/fRQy9nT2onyZGJkH3stbYuOp+thaOdFEF5ejje11ruWxvD Hu/PMXgYniCnwGdYpaAnYFUZz5Z4ZA/82lYA4kLX5nHtVv2ytjPvNOw0WCtL2Hp5ZODF gZAjfQc0RbYI8MUAEO4DHzZlw5BswspFCGF6E3WpC86fp0vIOHlvb5f8qJ/nEpUI19zl FVoNjWHlNdDC9pLG/LNYs7WCLnciraL4rfpIsnfcf7yRsugPdCLS7T3Ny9wE4bGy6Keb XVMwzvTdkhdLa/+rDJ96ES6Vr/GEYPGuAggiNV/+cLEbfHu6tG4ELX/QjqRvcDCmnEcl hdSA== X-Gm-Message-State: AOPr4FVlLx4d7XNrAvP83ry++NBJjZYypvWD+CE4MDGxGwxzzUhX6KEzVzNT0c/vBCwG1y2AC+MJalMv7mskEA== MIME-Version: 1.0 X-Received: by 10.202.189.10 with SMTP id n10mr1911273oif.101.1461709614078; Tue, 26 Apr 2016 15:26:54 -0700 (PDT) Received: by 10.60.52.140 with HTTP; Tue, 26 Apr 2016 15:26:53 -0700 (PDT) In-Reply-To: <571F62AD.6080005@quip.cz> References: <571F62AD.6080005@quip.cz> Date: Tue, 26 Apr 2016 17:26:53 -0500 Message-ID: Subject: Re: How to speed up slow zpool scrub? From: "Eric A. Borisch" To: Miroslav Lachman <000.fbsd@quip.cz> Cc: "freebsd-fs@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Apr 2016 22:26:55 -0000 On Tue, Apr 26, 2016 at 7:44 AM, Miroslav Lachman <000.fbsd@quip.cz> wrote: > Hi, > > is there any way to make zpool scrub faster? > We have one older machine with CPU Pentium(R) Dual E2160 @1.80GHz, 5GB of > RAM and 4x 4TB HDDs. It is just a storage for backups for about 20 machines. > Scrub is scheduled from periodic each 30 days but it takes about 4 days to > complete and everything during scrub is slow. Backups takes 8 hours instead > of 5 (made by rsync), deleting of old files is even more slower. > > The backups are made every night from the midnight to morning, the machine > is idle for the rest of the day. > > Is there any tuning to make scrub faster in this idle time? > Or is it better to do it other way - slower scrub with even lower priority > taking for about one week but not affecting time of normal operations? (is > it dangerous to have scrub running this long or reboot machine during the > scrub?) It's likely easier to make scrub slower (but rsync faster) when the system is not idle, but hopefully no slower otherwise. I would start with increasing the vfs.zfs.scrub_delay (in ticks; typically 1/1000 of a second) settings to reduce scrub's impact to other IO this without impacting resilvers. (Expect longer total time as it will scrub less during rsyncs.) You can also increase vfs.zfs.scan_idle if scrub_delay isn't enough, but do note that vfs.zfs.scan_idle impacts resilvers, too -- so make sure you don't also increase vfs.zfs.resilver_delay, or you may get very slow resilvers during IO. The logic for this is in dsl_scan.c [1] -- look there for the final word on what the different parameters do; there are a number of informative comments throughout the file if you are not into the code itself. See also vfs.zfs.vdev.*_max_active and vfs.zfs.vdev.*_min_active, if you are really want more knobs to try. It's might also be worth trying '/usr/local/bin/perl /usr/share/dtrace/toolkit/hotkernel' to see what the kernel is up to... - Eric [1] https://svnweb.freebsd.org/base/releng/10.3/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c?view=markup or https://github.com/freebsd/freebsd/blob/releng/10.3/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c