From owner-freebsd-fs@FreeBSD.ORG Wed Aug 14 15:45:59 2013 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id C6012481 for ; Wed, 14 Aug 2013 15:45:59 +0000 (UTC) (envelope-from karl@denninger.net) Received: from fs.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 64AAD235C for ; Wed, 14 Aug 2013 15:45:58 +0000 (UTC) Received: from [192.168.1.40] (localhost [127.0.0.1]) by fs.denninger.net (8.14.6/8.13.1) with ESMTP id r7EFaHgN083242 for ; Wed, 14 Aug 2013 10:36:18 -0500 (CDT) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (TLS/SSL) [192.168.1.40] by Spamblock-sys (LOCAL/AUTH); Wed Aug 14 10:36:18 2013 Message-ID: <520BA3EC.1030304@denninger.net> Date: Wed, 14 Aug 2013 10:36:12 -0500 From: Karl Denninger User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130215 Thunderbird/17.0.3 MIME-Version: 1.0 Subject: Re: Fwd: Disk scheduling activity... References: <520B8B1E.7060002@digiware.nl> <520BA249.8030603@digiware.nl> In-Reply-To: <520BA249.8030603@digiware.nl> X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Aug 2013 15:45:59 -0000 On 8/14/2013 10:29 AM, Willem Jan Withagen wrote: > On 2013-08-14 16:03, Bob Friesenhahn wrote: >> On Wed, 14 Aug 2013, Willem Jan Withagen wrote: >>> >>> Just a point of information or curiosity, and I don't think/know if it >>> is any problem... >>> >>> I have the raidz array with 8 disks, which I'm using to backup to. >>> It is configured >>> 4 disks on a mvs controller >>> 4 disks on an Areca controller (JBODs with battery) >>> Both controllers are on a PCI-E slot >>> >>> Most of the time the source just fully loads the pipe and sends >>> 1Gbit/s. >>> >>> When that happens I see this alternating pattern of writing either to >>> the 4 mvs disks, or writing to the Areca disks. >>> But almost never are all disk accesses at the same time. >>> And really never, never is there a mix of writing between the >>> controller >>> sets. >> > >> Are all 8 disks in the same raidz vdev? > > Yes is a raidz1 with 8 disks. I know it is not optimal in performance, > but I needed the amount of remaining diskspace. > >> Are you basing write activity on the drive LEDs? > > Yup. > >> >> The Areca controller may be caching the writes in its battery-backed >> cache and deferring the writes to when zfs tells it to flush its cache. >> The other controller may be issuing the writes right away. This would >> explain apparent 'split' writing behavior. > > Sounds like a fair assumption. Could remove the battery and see what > happens then. The mvs device is relatively "simple" and has no > significant memory on board. > >> There is even the possibilty that one of the controllers ignores the >> cache flush request and performs the writes later when it feels like it. > > That would then be the Areca controller, bacause I have the feeling > that it always writes later. > > --WjW > _ I very much doubt the ARECA is ignoring the cache-flush request. I have several of these and can get them into a pathological state with TERRIBLE performance when ZFS starts doing things that demand cache flushes - the ARECA will perform the demanded flush which, if you have a lot of RAM on the board, gets real interesting in terms of performance impact. -- Karl Denninger karl@denninger.net /Cuda Systems LLC/