Date: Wed, 31 Dec 2008 18:31:17 -0800 From: Tim Kientzle <tim@kientzle.com> To: Tim Kientzle <kientzle@freebsd.org> Cc: Max Laier <max@love2party.net>, freebsd-current@freebsd.org, cperciva@freebsd.org, "Simon L. Nielsen" <simon@freebsd.org> Subject: Re: tar/libarchive gzip problem [was: portsnap corrupted] Message-ID: <495C2AF5.6030703@kientzle.com> In-Reply-To: <495C017F.9000408@freebsd.org> References: <200812301616.11132.max@love2party.net> <200812302213.07155.max@love2party.net> <20081231094159.GA964@zaphod.nitro.dk> <200812311109.57955.max@love2party.net> <20081231105952.GB964@zaphod.nitro.dk> <495C017F.9000408@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
I just committed a fix for this (r186670). Please let me know if you see any other problems. Tim Tim Kientzle wrote: > I think I know what this is. I tried recently to add > support for concatenated gzip files. This involves looking > ahead for another GZip header after the end of the compressed > data. The current version screws this up and ends up trying > to match the file CRC as a Gzip header. About 1 in 256 files > will match the first byte, which triggers the subsequent meltdown. > This also explains why neither of us saw it in testing. > (I knew the code didn't actually work for concatenated gzip > files but didn't realize it would break decode of some > regular non-concatenated files.) > > The attached patch simply disables this additional > header check, which should fix the immediate problem. > Please try it and let me know. > > Apologies, > > Tim > > Simon L. Nielsen wrote: > >> Hey Tim, >> >> I think one of the recent changes to tar or libarchive broke gzip >> handling in some cases. See more below. >> >> [portsnap extract fails with gzip error] >> >> I'm not sure why I didn't run into it in my tests, but I think the >> problem is in tar / libarchive's handling of gzip files. Taking one >> random "broken" file [1] it fails with tar's build in decompression, >> but works using external zcat. >> >> [1] >> http://portsnap1.freebsd.org/f/19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz >> >> http://people.freebsd.org/~simon/tmp/19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz >> >> >> >> Broken system: >> >> [simon@eddie:/tmp] tar --version >> bsdtar 2.5.903a - libarchive 2.5.903a >> [simon@eddie:/tmp] uname -a >> FreeBSD eddie.nitro.dk 8.0-CURRENT FreeBSD 8.0-CURRENT #1: Tue Dec 30 >> 22:28:33 CET 2008 >> simon@eddie.nitro.dk:/FreeBSD/obj/FreeBSD/system-CURRENT/sys/EDDIE i386 >> [simon@eddie:/tmp] tar tvf >> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz > >> /dev/null >> tar: Error opening archive: Invalid GZip header (saw 99 at offset 1) >> [simon@eddie:/tmp] zcat >> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz | >> tar tf - > /dev/null [simon@eddie:/tmp] zcat >> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz | >> sha256 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d >> >> >> OK system: >> >> [simon@benji:/tmp] tar --version >> bsdtar 2.5.5 - libarchive 2.5.5 >> [simon@benji:/tmp] uname -a >> FreeBSD benji.s 7.1-RC2 FreeBSD 7.1-RC2 #0: Tue Dec 23 15:18:30 UTC >> 2008 root@logan.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386 >> [simon@benji:/tmp] tar tvf >> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz > >> /dev/null [simon@benji:/tmp] zcat >> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz | >> sha256 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d >> >> >> Another OK system: >> >> [simon@ref8-amd64:files] tar --version >> bsdtar 2.5.5 - libarchive 2.5.5 >> [simon@ref8-amd64:files] uname -a >> FreeBSD ref8-amd64.freebsd.org 8.0-CURRENT FreeBSD 8.0-CURRENT #2 >> r184542:185402: Fri Nov 28 19:14:40 UTC 2008 >> peter@ref8-amd64.freebsd.org:/scratch/src/sys/amd64/compile/REF8-AMD64 >> amd64 >> [simon@ref8-amd64:files] tar tvf >> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz > >> /dev/null >> [simon@ref8-amd64:files] zcat >> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz | >> sha256 >> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d >> > > ------------------------------------------------------------------------ > > Index: archive_read_support_compression_gzip.c > =================================================================== > --- archive_read_support_compression_gzip.c (revision 185679) > +++ archive_read_support_compression_gzip.c (working copy) > @@ -428,8 +428,7 @@ > "Failed to clean up gzip decompressor"); > return (ARCHIVE_FATAL); > } > - /* Restart header parser with the next block. */ > - state->header_state = state->header_done = 0; > + state->eof = 1; > /* FALL THROUGH */ > case Z_OK: /* Decompressor made some progress. */ > /* If we filled our buffer, update stats and return. */
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?495C2AF5.6030703>