From owner-freebsd-stable@FreeBSD.ORG Fri Feb 8 23:09:32 2013 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id E40255CB for ; Fri, 8 Feb 2013 23:09:32 +0000 (UTC) (envelope-from jdc@koitsu.org) Received: from qmta07.emeryville.ca.mail.comcast.net (qmta07.emeryville.ca.mail.comcast.net [IPv6:2001:558:fe2d:43:76:96:30:64]) by mx1.freebsd.org (Postfix) with ESMTP id C98F589B for ; Fri, 8 Feb 2013 23:09:32 +0000 (UTC) Received: from omta12.emeryville.ca.mail.comcast.net ([76.96.30.44]) by qmta07.emeryville.ca.mail.comcast.net with comcast id xqAz1k0010x6nqcA7z9Y7G; Fri, 08 Feb 2013 23:09:32 +0000 Received: from koitsu.strangled.net ([67.180.84.87]) by omta12.emeryville.ca.mail.comcast.net with comcast id xz8W1k00d1t3BNj8Yz8Wmh; Fri, 08 Feb 2013 23:08:31 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 8FD4273A31; Fri, 8 Feb 2013 15:08:30 -0800 (PST) Date: Fri, 8 Feb 2013 15:08:30 -0800 From: Jeremy Chadwick To: d@delphij.net Subject: Re: patch which implements ZFS LZ4 compression Message-ID: <20130208230830.GA45081@icarus.home.lan> References: <511581C9.5040608@delphij.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <511581C9.5040608@delphij.net> User-Agent: Mutt/1.5.21 (2010-09-15) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcast.net; s=q20121106; t=1360364972; bh=PxOf/5ZgtrJtqpapSXU0YkT5ao/2tlXRH/oaeKSMEog=; h=Received:Received:Received:Date:From:To:Subject:Message-ID: MIME-Version:Content-Type; b=NXSkUdJXTWJkGV6+R4QTr7V2QafjouVi4BAAMFqkFUUY5XHGRw3+EPAe3wO0A4+Br cr+IEPTU7zpkIhlLQsUitm9hz2NWXJfVU9yyws3jJ0Xj8FSLdm89m3M7yO4PdYmoUz KAgp1rEvFyTvczEL/EyOzHD1TOGNbxrvaSFoPCV7fmfTefzTvOkxgBkT6FD7Etnk1X xiKlnLXGAvc/+MicxRTz/4h7ENXAbgHfTVj+GnGWoyIFTt8Lh+GJbqEIhE30ehKVky PYpmA7VaDM2tVqdrwsKrTo9oPmxv8ZRdE4GsqIwS7Q9lC8AkWV1xjXtXyYa4Jzfbrl 6UKvtDlmO5u4A== Cc: freebsd-stable@freebsd.org, Dan Langille X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Feb 2013 23:09:32 -0000 On Fri, Feb 08, 2013 at 02:52:57PM -0800, Xin Li wrote: > On 02/08/13 14:29, Dan Langille wrote: > > Here is a patch against FreeBSD 9.1 STABLE which implements ZFS LZ4 > > compression. > > > > https://plus.google.com/106386350930626759085/posts/PLbkNfndPiM > > > > short link: http://bpaste.net/show/76095 > > Please DO NOT use this patch! It will ruin your data silently. > > As I already posted on Ivan's Google+ post, I'm doing final universe > builds to make sure that there is no regression and will merge my > changes to -HEAD later today. Another compression algorithm, this time 50%+ faster than lzjb. Great, fine, wonderful, awesome, kudos, huzzah, blah blah blah. So when is someone going to step up to the plate and fix how compression (as well as dedup) destroys interactivity on FreeBSD? Do I need to remind folks of this issue once again? Here you have it, dated October 2011, including the root cause and how it was fixed in Solaris et al: Description: http://lists.freebsd.org/pipermail/freebsd-fs/2011-October/012718.html Explanation and how Solaris et al fixed it, and how on Solaris the problem was major enough that it even caused NFS timeouts (sound familiar to anyone?): http://lists.freebsd.org/pipermail/freebsd-fs/2011-October/012726.html Further testing showing gzip-1 vs. lzjb and interactivity stalls: http://lists.freebsd.org/pipermail/freebsd-fs/2011-October/012752.html This is still a problem with base/stable/9. And as I have said elsewhere on lists, do not ask me to run CURRENT -- it will be a cold day in hell before I ever do that. I assume this same problem exists in CURRENT unless I have some key developer/committer say "I backported this fix in CURRENT, absolutely 100% sure". I'm also wondering why iXSystems hasn't stepped up to the plate to contribute to making this happen, given their business focus. I do not have the knowledge of the kernel (or of threading) to fix this myself, and for that I do apologise. But every time I see compression or dedup mentioned, I use the opportunity to bring up this subject. STOP ADDING FEATURES AND FIX STUFF LIKE THIS INSTEAD -- while new algorithms are neat/fun toys, they do not truly fix issues like this. How this problem has continually gotten overlooked is beyond me. If you want a PR for it, I'll file one, but all it's going to contain is the contents of this Email. -- | Jeremy Chadwick jdc@koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB |