From owner-freebsd-questions@freebsd.org Fri Jan 22 05:09:00 2016 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1F068A8C1BF for ; Fri, 22 Jan 2016 05:09:00 +0000 (UTC) (envelope-from dpchrist@holgerdanske.com) Received: from holgerdanske.com (holgerdanske.com [IPv6:2001:470:0:19b::b869:801b]) by mx1.freebsd.org (Postfix) with SMTP id 12C5F1643 for ; Fri, 22 Jan 2016 05:09:00 +0000 (UTC) (envelope-from dpchrist@holgerdanske.com) Received: from ::ffff:99.100.19.101 ([99.100.19.101]) by holgerdanske.com for ; Thu, 21 Jan 2016 21:08:57 -0800 Subject: Re: ZFS performance help sought To: freebsd-questions@freebsd.org References: <20160121205139.GG4538@blisses.org> From: David Christensen Message-ID: <56A1B969.4020107@holgerdanske.com> Date: Thu, 21 Jan 2016 21:08:57 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Icedove/38.5.0 MIME-Version: 1.0 In-Reply-To: <20160121205139.GG4538@blisses.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Jan 2016 05:09:00 -0000 On 01/21/2016 12:51 PM, Mason Loring Bliss wrote: > I've bounced back and forth between FreeBSD and Linux, and one of the reasons > why I tend to part with FreeBSD is frustration with ZFS performance. ... > Running FreeBSD, a zfs send/receive from one pool to another makes my system > almost unusably slow and even begins to dig me into swap a little. ... > I've moved back to FreeBSD on this box now, and I'd like to resolve this > issue. I don't know if it's a matter of fixing something broken in FreeBSD's > scheduling or tuning ZFS somehow such that it's friendlier. ... > The box has FreeBSD 10.2, eight gigs of RAM, and I'm dealing with pools 1TB > or smaller. No deduplication. ... On 01/21/2016 04:12 PM, Mason Loring Bliss wrote: > This is with the current test transfer going, with the settings noted > previously: > > extended device statistics > device r/s w/s kr/s kw/s qlen svc_t %b > ada0 182.6 0.0 23369.6 0.0 1 3.2 30 > ada1 212.5 0.0 27200.6 0.0 0 3.1 33 > ada2 0.0 486.9 0.0 61652.4 0 4.5 64 > ada3 0.0 798.1 0.0 101236.2 1 4.7 75 > ada4 0.0 0.0 0.0 0.0 0 0.0 0 > da0 0.0 0.0 0.0 0.0 0 0.0 0 > cd0 0.0 0.0 0.0 0.0 0 0.0 0 > > the sending pool is a mirror on ada0,1, and the receiving pool in > this case is a mirror on ada2,3. > > The disks in question: > > ada0: ATA8-ACS SATA 3.x device > ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > ada1: ATA8-ACS SATA 3.x device > ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > ada2: ATA-7 SATA 2.x device > ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > ada3: ACS-2 ATA SATA 3.x device > ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) 1. Have you tested the disks, cables, and controllers individually using disk manufacturer diagnostics? E.g. for Seagate, SeaTools for DOS (bootable CD based on FreeDOS; MS-DOS not required): http://www.seagate.com/support/downloads/seatools/ 2. Have you tested the disks individually using FreeBSD without ZFS? 3. Have you tested the disks individually using FreeBSD with ZFS? 4. Have you tested ada0,1 as a FreeBSD ZFS mirror? 5. Have you tested ada2,3 as a FreeBSD ZFS mirror? David