From owner-freebsd-fs@freebsd.org Wed Oct 21 04:46:20 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B5A1FA1ACEB; Wed, 21 Oct 2015 04:46:20 +0000 (UTC) (envelope-from jason@broken.net) Received: from broken.net (broken.net [198.134.7.18]) by mx1.freebsd.org (Postfix) with ESMTP id A123530F; Wed, 21 Oct 2015 04:46:20 +0000 (UTC) (envelope-from jason@broken.net) Received: from [192.168.253.69] (c-73-158-46-10.hsd1.ca.comcast.net [73.158.46.10]) by broken.net (Postfix) with ESMTPSA id 81CA1D3AE; Tue, 20 Oct 2015 21:46:18 -0700 (PDT) Subject: Re: [zfs] RE: granularity of performance penalty from resilvering To: zfs@lists.illumos.org, "zfs-discuss@list.zfsonlinux.org" , developer , freebsd-fs , zfs-discuss , "developer@lists.illumos.org" , "zfs-devel@freebsd.org" References: <5626E4B3.2020904@broken.net> <5626EFF1.9020208@broken.net> <5626F196.5080606@broken.net> From: Jason Matthews Message-ID: <56271897.4030209@broken.net> Date: Tue, 20 Oct 2015 21:46:15 -0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2015 04:46:20 -0000 I am not sure we are communicating. Instead of prioritizing resilvering, de-prioritize resilvering using those tunables. j. On 10/20/2015 7:14 PM, Fred Liu wrote: > Yeah, that is a trade-off. Prioritize resilvering high will shorten the time window. > That means brake your application "temporarily??". Even in current situation, the resilvering > eats almost all the IOs. > >> -----Original Message----- >> From: owner-zfs-devel@freebsd.org [mailto:owner-zfs-devel@freebsd.org] >> On Behalf Of Jason Matthews >> Sent: 星期三, 十月 21, 2015 10:00 >> To: zfs@lists.illumos.org; zfs-discuss@list.zfsonlinux.org; developer; >> freebsd-fs; zfs-discuss; developer@lists.illumos.org; zfs- >> devel@freebsd.org >> Subject: Re: [zfs] RE: granularity of performance penalty from >> resilvering >> >> >> >> i just sent you the tunables. turn the knobs in the other direction. >> >> j. >> >> On 10/20/2015 6:58 PM, Fred Liu wrote: >>> This feature is really needed. Even in some low-end RAID controllers, >>> the rebuilding penalty is tunable more than the high-end enterprise >> storage.... >>>> -----Original Message----- >>>> From: owner-freebsd-fs@freebsd.org >>>> [mailto:owner-freebsd-fs@freebsd.org] >>>> On Behalf Of Jason Matthews >>>> Sent: 星期三, 十月 21, 2015 9:53 >>>> To: zfs@lists.illumos.org; zfs-discuss@list.zfsonlinux.org; >>>> developer; freebsd-fs; zfs-discuss; developer@lists.illumos.org >>>> Subject: Re: [zfs] RE: granularity of performance penalty from >>>> resilvering >>>> >>>> >>>> That is the only way to fly... >>>> >>>> j. >>>> >>>> On 10/20/2015 6:35 PM, Fred Liu wrote: >>>>> Yeah. I want to go the other way. Plus, these settings are only >>>>> applicable in illumos. >>>>> >>>>> Therefore I decide to give up the hybrid( ssd+sata) solution to >>>>> underpin applications which need decent RAS. >>>>> >>>>> I am gonna go all-flash array. >>>>> >>>>> Thanks. >>>>> >>>>> Fred >>>>> >>>>> *From:*Jason Matthews [mailto:jason@broken.net] >>>>> *Sent:* 星期三, 十月21, 2015 9:05 >>>>> *To:* zfs@lists.illumos.org; Fred Liu; >>>>> zfs-discuss@list.zfsonlinux.org; developer; freebsd-fs; zfs-discuss >>>>> *Subject:* Re: [zfs] RE: granularity of performance penalty from >>>>> resilvering >>>>> >>>>> >>>>> >>>>> you could look at these tunables (not the settings themselves)... >>>>> >>>>> these settings actually make resilvers have a higher priority. You >>>>> obviously would want to go the other way. >>>>> >>>>> j. >>>>> >>>>> |* Prioritize resilvering by setting the delay to zero| set >>>>> |zfs:zfs_resilver_delay = 0 | >>>>> >>>>> * Prioritize scrubs by setting the delay to zero set >>>>> zfs:zfs_scrub_delay = 0 >>>>> >>>>> |* resilver for five seconds per TXG| set >>>>> |zfs:zfs_resilver_min_time_ms = 5000| >>>>> >>>>> >>>>> |echo zfs_resilver_delay/w0 | mdb -kw| echo zfs_scrub_delay/w0 |mdb >>>>> |-kw| echo zfs_top_maxinflight/w7f |mdb -kw| echo >>>>> |zfs_resilver_min_time_ms/w1388 |mdb -kw| >>>>> >>>>> On 10/19/2015 11:49 PM, Fred Liu wrote: >>>>> >>>>> Yes, “zpool scrub –s” can stop the resilvering. >>>>> >>>>> *From:*Fred Liu >>>>> *Sent:* 星 期二, 十月20, 2015 12:15 >>>>> *To:* 'zfs-discuss@list.zfsonlinux.org >>>>> '; developer; illumos- >> zfs; >>>>> freebsd-fs; zfs-discuss >>>>> *Subject:* granularity of performance penalty from resilvering >>>>> >>>>> Sorry if is a duplicate thread. >>>>> >>>>> The last suffering has been lasted for two weeks for we >>>>> replaced >>>> a >>>>> 6TB HDD. >>>>> >>>>> There should be some IO throttle measure from ZFS software >> stack. >>>>> At least, we can try to stop resilvering like scrubbing >>>>> >>>>> if the realization is quiet complicated. >>>>> >>>>> Besides that, will nice zil/cache be relief? >>>>> >>>>> Thanks. >>>>> >>>>> Fred >>>>> >>>>> *illumos-zfs* | Archives >>>>> >>>>> > 8480fd5f >>>>> | Modify >>>>> >>>> >>> f5b912c9> >>>>> Your Subscription [Powered by Listbox] >>>>> >>>> _______________________________________________ >>>> freebsd-fs@freebsd.org mailing list >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>> To unsubscribe, send any mail to "freebsd-fs- >> unsubscribe@freebsd.org" >>> ------------------------------------------- >>> illumos-zfs >>> Archives: https://www.listbox.com/member/archive/182191/=now >>> RSS Feed: >>> https://www.listbox.com/member/archive/rss/182191/22567878-8480fd5f >>> Modify Your Subscription: >>> >> https://www.listbox.com/member/?& >>> f5b912c9 Powered by Listbox: http://www.listbox.com >> _______________________________________________ >> zfs-devel@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/zfs-devel >> To unsubscribe, send any mail to "zfs-devel-unsubscribe@freebsd.org" > > ------------------------------------------- > illumos-zfs > Archives: https://www.listbox.com/member/archive/182191/=now > RSS Feed: https://www.listbox.com/member/archive/rss/182191/22567878-8480fd5f > Modify Your Subscription: https://www.listbox.com/member/?member_id=22567878&id_secret=22567878-f5b912c9 > Powered by Listbox: http://www.listbox.com