From owner-freebsd-current@freebsd.org Thu Dec 14 13:09:41 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3CAF3EA3810 for ; Thu, 14 Dec 2017 13:09:41 +0000 (UTC) (envelope-from dnebdal@gmail.com) Received: from mail-qk0-x234.google.com (mail-qk0-x234.google.com [IPv6:2607:f8b0:400d:c09::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EA991365E for ; Thu, 14 Dec 2017 13:09:40 +0000 (UTC) (envelope-from dnebdal@gmail.com) Received: by mail-qk0-x234.google.com with SMTP id b123so6121934qkg.7 for ; Thu, 14 Dec 2017 05:09:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=s/qlsSOdeAO5goqkCxOA2aCZ9tGYoaGQzBXVAt3IQ/w=; b=gB0Ve7nhEh+tT85IJFVHjBO4ggE/1mSgCnZhQ4TXDjJtRnbX0aPSkHNOInLGcU/kS/ zxsvajErucJAkCpE+XFVPwjBmxGCseDnT47ic6OszVcSpCK5PUvJ/LP7l2Iw6nSuX30U dt5e3dyZiOFxyHwWVFfZNj3kHWS12pJ+926XJCccvyG1AGQb648abu2r6cqs2H9mkD6w 8EQ9+uHemNuqB15lsD8B0lWmk1dA8cy2VUVYDYIQyHcuG00EZKHW3J9CJGtTISSfnbru GlFNqZBM4zOirygwOuJ3//X2j0cwG9sWerdhuMO1u9ZMMglMOBP6tZ0eOmPBSxKj17mp vAqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=s/qlsSOdeAO5goqkCxOA2aCZ9tGYoaGQzBXVAt3IQ/w=; b=Xv24cLbBRqn1qFho8xFsLq3FbSMBHroYEu9WPzdHxXgL4TIEcY8EYAOA7Z5JEISU2a 6KvK2gXxU52DwByf/KL+TlEU6p4D6LIlpCc3QfnGjXtaII18blmy9I0oRTnhP/8VELXT dY2jsDIB/WiWiavcAz3U8erG8UdsRN+ZKqnIybyOUybIH1u7ZHfRH4vf9SvCOr355ilU f5v1z3Bsbv07nKY1yL4aAEwgzxLTARHgRY6IeZBB3yzHFq21iTf2AGqwsFld159zvZZ6 ReDVzySLlrDSpFhb/8TR1eQMnVgc5lfYTTV/iiR0myb87sqt/lt/Rww1FCr44xnPvSG/ gSSQ== X-Gm-Message-State: AKGB3mK0uGGl3FdZaHzGalPzISOVg3nkG2ZwQxOleZcz/NHgfWryZmgj R2rCMre659YHD7gqEQT36nBeBWyLzhJR1mBP8WDjng== X-Google-Smtp-Source: ACJfBov7TjVld/IAyv9gblQV+fUdeSx+oieHATTAWdMtoCdtHd/F9OAatjSivabek/B3wzwKkFnXdmOGt920sPIKwGI= X-Received: by 10.233.220.199 with SMTP id q190mr16362503qkf.72.1513256979878; Thu, 14 Dec 2017 05:09:39 -0800 (PST) MIME-Version: 1.0 Received: by 10.140.108.9 with HTTP; Thu, 14 Dec 2017 05:09:39 -0800 (PST) In-Reply-To: <20171214124900.64211bd9@thor.intern.walstatt.dynvpn.de> References: <20171214124900.64211bd9@thor.intern.walstatt.dynvpn.de> From: Daniel Nebdal Date: Thu, 14 Dec 2017 14:09:39 +0100 Message-ID: Subject: Re: ZFS RAIDZ1: resilvering at <17.3M/s => abyssal slow ... To: "O. Hartmann" Cc: FreeBSD CURRENT Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Dec 2017 13:09:41 -0000 On Thu, Dec 14, 2017 at 12:48 PM, O. Hartmann wrot= e: > Hello out there, > > I had to replace a HDD on a RAIDZ1 pool comprised from 4 HDDs, each 3 TB = (netto size of > the pool is about 8-9 TB, the used amount is reported to be ~ 6TB). > > I just started the rebuild/resilvering process and watch the pool crwalin= g at ~ 18 MB/s. > At the moment, there is no load on the array, the host is a IvyBridge XEO= N with 4 core/8 > threads and 3,4 GHz and 16 GB of RAM. The HDDs are attached to a on-board= SATA II (300 > MB/s max) Intel chip - this just for the record. > > Recently, I switch on the "sync" attribute on most of the defined pools's= zfs filesystems > - I also use a SSD for ZIL/L2ARC caching, but it seems to be unused recen= tly in FreeBSD > CURRENT's ZFS - this from a observers perspective only. > > When scrubbing, I see recently also reduced performance on the pool, so I= 'm wondering > about the low throughput at the very moment when resilvering is in progre= ss. > > If the "perspective" of "zpool status" is correct, then I have to wait af= ter two hours > for another 100 hours - ~ 4 days? Ups ... I think there is something badl= y misconfigured > or missing. > > The pool is not "tuned" in any very sophisticated way, since I trust the = kernels ability > to scale automatically - if I'm not wrong for amd64 ... > > Are there any aspects to look at? > > Thanks in advance, > > Oliver > > -- > O. Hartmann > > Ich widerspreche der Nutzung oder =C3=9Cbermittlung meiner Daten f=C3=BCr > Werbezwecke oder f=C3=BCr die Markt- oder Meinungsforschung (=C2=A7 28 Ab= s. 4 BDSG). This is kind of to be expected - for whatever reason, resilvers seem to go super slow at first and then speed up significantly. Just don't ask me how long "at first" is - I'd give it several (more) hours. I don't _think_ sync affects resilver speed (since that's a filesystem level setting and the resilver happens at the block level), but that's pure speculation. The Tuning guide at https://wiki.freebsd.org/ZFSTuningGuide also suggests a list of sysctls you can tweak, which could be worth a try: ### wiki If you're getting horrible performance during a scrub or resilver, the following sysctls can be set: vfs.zfs.scrub_delay=3D0 vfs.zfs.top_maxinflight=3D128 vfs.zfs.resilver_min_time_ms=3D5000 vfs.zfs.resilver_delay=3D0 Setting those sysctls to those values increased my (Shawn Webb's) resilver performance from 7MB/s to 230MB/s. ### --=20 Daniel Nebdal