Date: Thu, 08 Apr 2004 21:21:27 -0400 From: "Brian F. Feldman" <green@FreeBSD.org> To: Tim Kientzle <tim@kientzle.com> Cc: current@FreeBSD.org Subject: Re: cvs commit: src/usr.bin/tar Makefile bsdtar.1 bsdtar.c bsdtar.h bsdtar_platform.h matching.c read.c util.c write.c Message-ID: <200404090121.i391LRlr096539@green.homeunix.org> In-Reply-To: Message from Tim Kientzle <tim@kientzle.com> of "Wed, 07 Apr 2004 10:49:20 PDT." <40743F20.90900@kientzle.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Tim Kientzle <tim@kientzle.com> wrote: > Brian F. Feldman wrote: > > Tim Kientzle <tim@kientzle.com> wrote: > > > > Oh, I was only implementing it inside the libarchive default routines > > because it was easy. I also think that if this is done to speed up tar, it > > should be done in tar and not libarchive. What are you using to benchmark? > > I'm interested in seeing what happens with a worker thread/with a larger > > decompression buffer/etc. > > I think I was just using: > > bsdtar -cvjf /dev/null /usr > > for compression benchmarking. For decompression benchmarking, > something like > > bsdtar -tf archive.tbz >/dev/null > > should suffice. Okay, I tested it out and hammered it into shape so it could actually perform concurrently (see the updated pach to see how I finally implemented that prototype, <http://green.homeunix.org/~green/libarchive_bz2thread.patch>; I saw only the tiniest speed difference in both tar tf and tar xfv -- it's hard to measure any difference whatsoever. With full concurrency it might even be a little slower, and bsdtar is ALWAYS faster than the (multi-process) tar tfy/xfy! So don't bother doing anything else to compete in that respect if that's all you want to beat. One thing I did notice was the huge amount of calls that got done back and forth from the tar reader and the bzip2 reader and the file reader; it should be possible to get more speed out of bsdtar by actually pulling in the entire size of a block of whatever decompressor is being used. For example a bzip2 can be 900KB blocks and the current buffer size is 10KB, and for anything at all to be decompressed the entire thing had to have been run through bzip2, building up its internal state to 3700KB during the process. If there's enough speed benefit, it's silly to save a little bit of space by using a very small "file read" buffer. For S_ISREG() use no KB instead of 10KB by using mmap(2), maybe... It's funny how you say the performance might be bad compared to gnutar even though in decompression you actually save a fair bit of CPU! -- Brian Fundakowski Feldman \'[ FreeBSD ]''''''''''\ <> green@FreeBSD.org \ The Power to Serve! \ Opinions expressed are my own. \,,,,,,,,,,,,,,,,,,,,,,\
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200404090121.i391LRlr096539>