From owner-freebsd-current@FreeBSD.ORG Thu Apr 8 18:21:28 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from green.homeunix.org (freefall.freebsd.org [216.136.204.21]) by hub.freebsd.org (Postfix) with ESMTP id 1CD2516A4CE; Thu, 8 Apr 2004 18:21:28 -0700 (PDT) Received: from localhost (green@localhost [127.0.0.1]) by green.homeunix.org (8.12.11/8.12.11) with ESMTP id i391LRlr096539; Thu, 8 Apr 2004 21:21:27 -0400 (EDT) (envelope-from green@green.homeunix.org) Message-Id: <200404090121.i391LRlr096539@green.homeunix.org> X-Mailer: exmh version 2.6.3 04/04/2003 with nmh-1.0.4 To: Tim Kientzle In-Reply-To: Message from Tim Kientzle of "Wed, 07 Apr 2004 10:49:20 PDT." <40743F20.90900@kientzle.com> From: "Brian F. Feldman" Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 08 Apr 2004 21:21:27 -0400 Sender: green@green.homeunix.org cc: "Brian F. Feldman" cc: Ruslan Ermilov cc: current@FreeBSD.org Subject: Re: cvs commit: src/usr.bin/tar Makefile bsdtar.1 bsdtar.c bsdtar.h bsdtar_platform.h matching.c read.c util.c write.c X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Apr 2004 01:21:28 -0000 Tim Kientzle wrote: > Brian F. Feldman wrote: > > Tim Kientzle wrote: > > > > Oh, I was only implementing it inside the libarchive default routines > > because it was easy. I also think that if this is done to speed up tar, it > > should be done in tar and not libarchive. What are you using to benchmark? > > I'm interested in seeing what happens with a worker thread/with a larger > > decompression buffer/etc. > > I think I was just using: > > bsdtar -cvjf /dev/null /usr > > for compression benchmarking. For decompression benchmarking, > something like > > bsdtar -tf archive.tbz >/dev/null > > should suffice. Okay, I tested it out and hammered it into shape so it could actually perform concurrently (see the updated pach to see how I finally implemented that prototype, I saw only the tiniest speed difference in both tar tf and tar xfv -- it's hard to measure any difference whatsoever. With full concurrency it might even be a little slower, and bsdtar is ALWAYS faster than the (multi-process) tar tfy/xfy! So don't bother doing anything else to compete in that respect if that's all you want to beat. One thing I did notice was the huge amount of calls that got done back and forth from the tar reader and the bzip2 reader and the file reader; it should be possible to get more speed out of bsdtar by actually pulling in the entire size of a block of whatever decompressor is being used. For example a bzip2 can be 900KB blocks and the current buffer size is 10KB, and for anything at all to be decompressed the entire thing had to have been run through bzip2, building up its internal state to 3700KB during the process. If there's enough speed benefit, it's silly to save a little bit of space by using a very small "file read" buffer. For S_ISREG() use no KB instead of 10KB by using mmap(2), maybe... It's funny how you say the performance might be bad compared to gnutar even though in decompression you actually save a fair bit of CPU! -- Brian Fundakowski Feldman \'[ FreeBSD ]''''''''''\ <> green@FreeBSD.org \ The Power to Serve! \ Opinions expressed are my own. \,,,,,,,,,,,,,,,,,,,,,,\