From owner-freebsd-ports Fri Jun 1 13:42: 9 2001 Delivered-To: freebsd-ports@freebsd.org Received: from wall.polstra.com (rtrwan160.accessone.com [206.213.115.74]) by hub.freebsd.org (Postfix) with ESMTP id A2D9E37B422 for ; Fri, 1 Jun 2001 13:42:02 -0700 (PDT) (envelope-from jdp@wall.polstra.com) Received: from vashon.polstra.com (vashon.polstra.com [206.213.73.13]) by wall.polstra.com (8.11.3/8.11.1) with ESMTP id f51Kg2019746 for ; Fri, 1 Jun 2001 13:42:02 -0700 (PDT) (envelope-from jdp@wall.polstra.com) Received: (from jdp@localhost) by vashon.polstra.com (8.11.3/8.11.0) id f51Kg1U29811; Fri, 1 Jun 2001 13:42:01 -0700 (PDT) (envelope-from jdp) Date: Fri, 1 Jun 2001 13:42:01 -0700 (PDT) Message-Id: <200106012042.f51Kg1U29811@vashon.polstra.com> To: ports@freebsd.org From: John Polstra Subject: Re: Supporting pkg-plist.gz ? In-Reply-To: References: Organization: Polstra & Co., Seattle, WA Sender: owner-freebsd-ports@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org In article , George Reid wrote: > On Fri, 1 Jun 2001, Ernst de Haan wrote: > > > Decompressing all pkg-plist files on forehand doesn't sound like a good idea, > > since the other approach supports a multi-threaded/processing approach. The > > performance will in my expectation be much higher, since downloading is > > typically much slower than (de)compressing for the size of files we are > > talking about (between 1K and 1M). > > If CVSup were to support locally compressed files, each local file would > have to be decompressed first, the delta applied and then recompressed > (since you can't decompress "a bit" of the local file). This is obviously > more efficient than decompressing all of the plist files, cvsupping as > normal and then recompressing. > > It is however, perhaps too drastic a change, since it definitely wouldn't > work in the case where the entire repository was mirrored (i.e. it would > break the standard CVS tools). I know nothing of the cvsup internals or > Modula-3, so this is probably best left to the authors. That would be me. :-) I don't think it's worth pursuing, at least not for pkg-plist files. I looked at my ports tree, which is just a few days old. It contains 4894 pkg-plist files. Of those, 3479 (71%) can't be compressed at all. The reason is that their sizes are 1024 bytes or less. That is the default fragment size (smallest unit of allocation) in UFS filesystems, which means that any file from 1-1024 bytes in size will use the same amount of space on the disk. You could compress each of those files by a factor of 10 without freeing up a single block on your disk. The sum total of all the pkg-plist files is only 22 MB, which isn't worth walking around the corner to save. Megabytes ain't what they used to be, you know. :-) John -- John Polstra jdp@polstra.com John D. Polstra & Co., Inc. Seattle, Washington USA "Disappointment is a good sign of basic intelligence." -- Chögyam Trungpa To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-ports" in the body of the message