From owner-freebsd-current@freebsd.org Thu Dec 14 15:28:10 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4D6C2E83F74 for ; Thu, 14 Dec 2017 15:28:10 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: from mail-it0-x22f.google.com (mail-it0-x22f.google.com [IPv6:2607:f8b0:4001:c0b::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1060B687AC; Thu, 14 Dec 2017 15:28:10 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: by mail-it0-x22f.google.com with SMTP id d137so11940600itc.2; Thu, 14 Dec 2017 07:28:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=gq3L/IY0ReiDeImXhNdCxyXgVcL11/hv3Wfuv8Sjepk=; b=uIZoJm7VT3M+eOjsDRLOn+ZwZ5y7rs2CzHrYzCTf3iNeXu6YsuATwrgGJnSsShpAfs D7RlyimbwJ3u8pzJIzwrfYZLLBmzcTYf8mmGcPpQ9MMUGqJGM7Rt8PdaoQImf4zXuT6j M+GcQ6vA1K58zZnse+lbRm6WN1k3qpNX0YvUEd5BVoZy5HxG3g4FzR6LPJIkGZqJEW4C OWOO6kZ2SKDyIi8RzmEPsHjVqj7g6kooRR02H+Pu/aO7XPIFNii2DntZmoe5jtlpcSmJ +oddsBNYa7FGCc+2yOiMcxoFNPmZGgtbBhQJLpQhEk2wRm+cKMePdjW2q28ONgAxTcHv 9kjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=gq3L/IY0ReiDeImXhNdCxyXgVcL11/hv3Wfuv8Sjepk=; b=d3DEz07ciCsP1tpOa4lnmm4rlZo/TUyWBGCvtncQ1ZLJmIQ/s0qjY1HPD+JkpWA6/h GB4qQNVrvmov1Voa1fsK8uCDEQ7+FVFjNP7F66Ft6TWBpc9tZkpxGbSgQt6W6K/79DTZ envS/r2WRusvXTucQ/e001WzotCewVdmHKYU5ErEmrgQVSf4TsCQFzNadX/E6/Fhl8kT 4QFwMJJPWPwr3BbBk+Q9/kxlHRHXZTNkKJbvKMae3xPJ9AoNNG8IOx9lrwn3W18QieeZ w3JfydJAN5c12OPT7gosoDjO1ojc7IcE6T2JoyE2g0W9EIeGLs/Gd0gO0y3G4z9KfI7q g4Ww== X-Gm-Message-State: AKGB3mLfm1cVi6Ha/7DH1+yhIMjvVIqFr0vS/mEk/pa5V2fDmm2uP5SO A+51JKBynYWoFQJp9ffN9RzfFB3fn6Q45+Voc4g= X-Google-Smtp-Source: ACJfBouL3I2Xf/gHFfugbyzi0Nv2xskmPPGzbnoQHgN9wlB9H59yel2oVGHq2pkw9JiSVi2CYRrQ++5EEfIzqEQykjU= X-Received: by 10.107.112.26 with SMTP id l26mr7596372ioc.230.1513265289250; Thu, 14 Dec 2017 07:28:09 -0800 (PST) MIME-Version: 1.0 Received: by 10.2.138.114 with HTTP; Thu, 14 Dec 2017 07:28:08 -0800 (PST) In-Reply-To: <20171214155254.4736ebe5@thor.intern.walstatt.dynvpn.de> References: <20171214124900.64211bd9@thor.intern.walstatt.dynvpn.de> <20171214144351.24a81faa@thor.intern.walstatt.dynvpn.de> <20171214155254.4736ebe5@thor.intern.walstatt.dynvpn.de> From: Adam Vande More Date: Thu, 14 Dec 2017 09:28:08 -0600 Message-ID: Subject: Re: ZFS RAIDZ1: resilvering at <17.3M/s => abyssal slow ... To: "O. Hartmann" Cc: Dimitry Andric , Daniel Nebdal , FreeBSD CURRENT Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Dec 2017 15:28:10 -0000 On Thu, Dec 14, 2017 at 8:52 AM, O. Hartmann wrote: > Am Thu, 14 Dec 2017 15:46:17 +0100 > Dimitry Andric schrieb: > > > On 14 Dec 2017, at 14:43, O. Hartmann wrote: > > > > > > Am Thu, 14 Dec 2017 14:09:39 +0100 > > > Daniel Nebdal schrieb: > > >> On Thu, Dec 14, 2017 at 12:48 PM, O. Hartmann > wrote: > > >>> I just started the rebuild/resilvering process and watch the pool > crwaling at ~ 18 > > >>> MB/s. At the moment, there is no load on the array, the host is a > IvyBridge XEON > > >>> with 4 core/8 threads and 3,4 GHz and 16 GB of RAM. The HDDs are > attached to a > > >>> on-board SATA II (300 MB/s max) Intel chip - this just for the > record. > > >>> > > >>> Recently, I switch on the "sync" attribute on most of the defined > pools's zfs > > >>> filesystems > > >>> - I also use a SSD for ZIL/L2ARC caching, but it seems to be unused > recently in > > >>> FreeBSD CURRENT's ZFS - this from a observers perspective only. > > >>> > > >>> When scrubbing, I see recently also reduced performance on the pool, > so I'm > > >>> wondering about the low throughput at the very moment when > resilvering is in > > >>> progress. > > >>> > > >>> If the "perspective" of "zpool status" is correct, then I have to > wait after two > > >>> hours for another 100 hours - ~ 4 days? Ups ... I think there is > something badly > > >>> misconfigured or missing. > > ... > > >> This is kind of to be expected - for whatever reason, resilvers seem > > >> to go super slow at first and then speed up significantly. Just don't > > >> ask me how long "at first" is - I'd give it several (more) hours. > > > > Hopefully this will get better in the future, please read: > > > > http://open-zfs.org/wiki/Scrub/Resilver_Performance > > > > -Dimitry > > > > It has already been started to become better ;-) > > After a while now, the throughput is at 128 MBytes/s and the estimated > time decreased to > ~ 8 h now - that is much more appreciable than 4 days ;-) > If you are viewing the rate with zpool status, I don't think that is a close to realtime rate. I don't know of a way to check that either. -- Adam