From owner-freebsd-fs@freebsd.org Sun Jan 22 19:12:31 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DADEECBD83A for ; Sun, 22 Jan 2017 19:12:31 +0000 (UTC) (envelope-from tobias.oberstein@gmail.com) Received: from mail-wm0-x235.google.com (mail-wm0-x235.google.com [IPv6:2a00:1450:400c:c09::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7AE68E4C for ; Sun, 22 Jan 2017 19:12:31 +0000 (UTC) (envelope-from tobias.oberstein@gmail.com) Received: by mail-wm0-x235.google.com with SMTP id c85so104223673wmi.1 for ; Sun, 22 Jan 2017 11:12:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding; bh=OC46dXpKO0+8zfICmH9sNn+wkEhS0Vrg60Vb/+t7UH8=; b=i5SPlboEy5uHCOR7MPuZjdRoih6PcpSw4nQrzpf2E/i9LG9BGTrNiAs/lM6lSCdWt2 AEhYo1PsP1u9qrmQ5qf6I5mxVTrSlm9DCz6IB0vQLZ3qycFBMDAEbXvwupq2jDir/BTH 1ohnTkCbScsGFh+K2AL95YNXc2zjGFgtsbhXnx6GS1fLkyrLdbj1YKqk++z3YOZEBfJ7 GV6Uz0si+SQX8XAevGzxjVF4dbc9aFJcGaN3zCMEGAmuWa2sWxlvKblM0OLwpcgPXnPI AnjDQ+u98/4AMTuSFwJXpZ/o8UvqEmoJ/msH1iuGZzKlpeHA3fm7LxnnQnXfswdYA2W2 MMvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=OC46dXpKO0+8zfICmH9sNn+wkEhS0Vrg60Vb/+t7UH8=; b=HOT9KBk/CAIfmYaNf+QM5yatL9G6uVcd98RE8f8D60qd71DPLomW6Ocmd2E/EluYHw 17toWue3KvW5QKVlZoxLrnfKu3vSPS0SnSHyyTHcBqFqoEj8zXebIONT9jOwlduRdBxS ji4+51joBo9pozrt2kUcubnPaqBaLOXgGYScj3jqxwkHcwM4Ruy/5D58F91GuSnWaEei OHYf7QGzgxAEL2rufg3Ys1MGldP+rJGsSYNmas4dCrNbTx2oyqEGt3fxokxPCbYb18xW xJbDPF7Zn8PBx62JlKPPUGKsX2C+S6Qb36SiaZVb89a7xFlxHEnVsHMREqM/iODPBS4G B7iQ== X-Gm-Message-State: AIkVDXITN2Aou/SjAgk+rDy0hbNgrIGnKf0YBf6Wnol5d0dTC2K8VDc4UnfnZt3e8lhsAg== X-Received: by 10.28.10.68 with SMTP id 65mr11359683wmk.63.1485112349385; Sun, 22 Jan 2017 11:12:29 -0800 (PST) Received: from [192.168.55.138] (ppp-93-104-174-120.dynamic.mnet-online.de. [93.104.174.120]) by smtp.googlemail.com with ESMTPSA id a186sm17333854wmh.1.2017.01.22.11.12.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 22 Jan 2017 11:12:28 -0800 (PST) Subject: Re: stuck on low scrub performance To: Daryl Richards , freebsd-fs@freebsd.org References: <8b408926-768b-8df9-e38c-b617cce30b16@gmail.com> <91c33947-bddc-014e-676d-2a5c3950bb71@isletech.net> From: Tobias Oberstein Message-ID: <4215534c-c249-beb2-e210-a5d31b0ac645@gmail.com> Date: Sun, 22 Jan 2017 20:12:27 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <91c33947-bddc-014e-676d-2a5c3950bb71@isletech.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jan 2017 19:12:32 -0000 >> Note: the pool consists of 4 disk sets (striped) with 6 disks each >> (RAID-Z2). > Also, FWIW, I also find a scrub starts very slow for the first couple > hours, then speeds up drastically. I also only have ~20TB so perhaps > this scales? I am wondering about this too, eg how large can one make a _single_ zpool? Say I create a single zpool of N x 6 disks, where each 6er set is a RAID-Z2. Currently, I have N=4 (with 6TB SAS disks) .. and seems works very well. Will it work as well with N=16? Assuming I have enough SAS bandwidth of course .. Oracle ZFS Storage ZS5-4 maxes out at 9PB. Anyone knows about the zpool layout of this baby? Cheers, /Tobias