From owner-freebsd-fs@FreeBSD.ORG Wed Oct 12 20:28:30 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 157C5106566C for ; Wed, 12 Oct 2011 20:28:30 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-vx0-f182.google.com (mail-vx0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id C51568FC17 for ; Wed, 12 Oct 2011 20:28:29 +0000 (UTC) Received: by vcbf13 with SMTP id f13so1356625vcb.13 for ; Wed, 12 Oct 2011 13:28:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=olWouScW6g10OFxlr70Jaqci+M0c5oUzLbLntUVUn0g=; b=snRtjrA1owB6Ri1vfBsxbdRjM0OrIT44hKY83MlkU8ioIKRJkLSNf9QpU9ANUTcFv4 iNao7xGLJPtk7YqsT3FKYywX6A/G3CR898vx4Cj0QjbKnStess59lsW9JpaKDcfSZqJp qYNDeAu2HvOIgmh3trlh1JCHOxZqeEa8sTUn0= MIME-Version: 1.0 Received: by 10.220.149.19 with SMTP id r19mr52069vcv.80.1318451309000; Wed, 12 Oct 2011 13:28:29 -0700 (PDT) Received: by 10.220.176.200 with HTTP; Wed, 12 Oct 2011 13:28:28 -0700 (PDT) In-Reply-To: <20111012165126.GA26562@icarus.home.lan> References: <20111012165126.GA26562@icarus.home.lan> Date: Wed, 12 Oct 2011 13:28:28 -0700 Message-ID: From: Freddie Cash To: Jeremy Chadwick Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: ZFS/compression/performance X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Oct 2011 20:28:30 -0000 On Wed, Oct 12, 2011 at 9:51 AM, Jeremy Chadwick wrote: > That might be the case on OpenSolaris but the performance hit on > FreeBSD RELENG_8 is very high -- enough that enabling compression (using > the defaults) causes stalls when I/O occurs (easily noticeable across > SSH; characters are delayed/stalled (not buffered)), etc.. > > The last time I tried it on RELENG_8 was right after ZFSv28 was MFC'd. > If things have improved I can try again (I don't remember seeing any > commits that could affect this), or if people really think changing the > compression model to lzjb will help. > I would try it again, with the latest RELENG_8 sources. compression=lzjb performance hit is negligible on our newest backups server (8-core Opteron, 16 GB RAM, 4x 6-drive raidz2, dedupe enabled). I am able to connect via SSH, navigate around the filesystem, tail -f multiple log files, monitor network via iftop/trafshow, and watch top, all via a tmux session (multiple tmux "windows"). This is with 5 rsync processes running, transferring data at 200-300 Mbps. compression=gzip-9 performance hit is noticeable on our older backups servers (4-core Opteron, one has 8 GB RAM the other 12 GB RAM, 3x 8-drive raidz2, no dedupe). This pool is 93% full and heavily fragmented, though. SSH connections are slow, typing is choppy, basically everything is choppy, while running 5 rsync processes. This box used to run 12 rsyncs simultaneously, but will now lockup completely if you try to run 7 or more. I've switched to lzjb on these two boxes to see if that helps any. I'm thinking the fullness and fragmenting of the pool is the root cause of the slowness on these boxes now. -- Freddie Cash fjwcash@gmail.com