From owner-freebsd-arch Mon Jul 8 14:48:29 2002 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A75EA37B400 for ; Mon, 8 Jul 2002 14:48:27 -0700 (PDT) Received: from flamingo.mail.pas.earthlink.net (flamingo.mail.pas.earthlink.net [207.217.120.232]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3AD3643E31 for ; Mon, 8 Jul 2002 14:48:27 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0159.cvx21-bradley.dialup.earthlink.net ([209.179.192.159] helo=mindspring.com) by flamingo.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 17RgMh-0001DR-00; Mon, 08 Jul 2002 17:48:15 -0400 Message-ID: <3D2A0872.FE692237@mindspring.com> Date: Mon, 08 Jul 2002 14:47:30 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Mark Valentine Cc: Garrett Wollman , arch@freebsd.org Subject: Re: Package system flaws? References: <200207082126.g68LQsCc074173@dotar.thuvia.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Mark Valentine wrote: > > From: Terry Lambert > > Most compression will work better on individual files rather than random > > file contents because of dictionary locality, anyway. > > If you're saying what I think you are, then this is very contrary to what > we've been led to believe to date. ??? The "Welch" part that Unisys's Terry Welch added to Lempel-Ziv-Welch algorithm used by the UNIX "compress" utility was a calculation of a data domain specific dictionary, and a reset of the dictionary every N K in order to hadle file content locality (e.g. text vs. object files vs. other). In other words, the reason LZW is *so* much better than LZ is that it knows about dictionary locality. If you know this going in, then your locality is going to change any time you toggle between data and metadata, and since there is great benefit to being able to access metadata even if content is compressed, then the answer seems obvious... -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message