From owner-freebsd-bugs@FreeBSD.ORG Thu Sep 18 18:22:37 2014 Return-Path: Delivered-To: freebsd-bugs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 98B29646 for ; Thu, 18 Sep 2014 18:22:37 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7ED6FF7A for ; Thu, 18 Sep 2014 18:22:37 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s8IIMbBG096503 for ; Thu, 18 Sep 2014 18:22:37 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-bugs@FreeBSD.org Subject: [Bug 187315] unzip(1): base unzip does not recognize *.zip archives from dropbox.com Date: Thu, 18 Sep 2014 18:22:37 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: oliver@beefrankly.org X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Sep 2014 18:22:37 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=187315 oliver@beefrankly.org changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |oliver@beefrankly.org --- Comment #2 from oliver@beefrankly.org --- Some time ago I checked out the bug and tried to contact the author, but did not get response...maybe he did not get it...here is a copy of the mail... --- Hello des, I contact you because you are the main author of the /usr/src/usr.bin/unzip utility if I got it correct. Well I took a glimpse into this PR bin/187315 and could need some advice. unzip(1) uses libarchive(3) for working with the archives. To determine the filetype, there is a function called "archive_entry_filetype()" in libarchive. As this function uses the file acl.mode as input, it fails if an entry has no file mode and returns a filetype of 0x0. As the implementation of unzip expects to get a filetype of either a regular file or a directory, it checks for that. And so that sanity check for S_ISREG and S_ISDIR fails and the program skips the entry. unzip.c /* I don't think this can happen in a zipfile.. */ if (!S_ISDIR(filetype) && !S_ISREG(filetype)) { warningx("skipping non-regular entry '%s'", pathname); ac(archive_read_data_skip(a)); free(pathname); return; } The cause of this may be that dropbox creates the zipfile for you on-the-fly. That means streaming it out of a database directly into a zipfile. In this special circumstance, where there is no file and the file comes from stdin, it is allowed by ZIP file archive standard to keep the external file attribute 0x0. (see [1] 4.4.15 external file attributes). As I understand it, the libarchive code uses this field for filetype check. I think that is what happens here (at least in the dropbox-file the filetype is returned zero for all files and directories). I can reproduce the error like that: $ echo "testtext" | python -c "import sys import zipfile z = zipfile.ZipFile(sys.argv[1],'w') z.writestr(sys.argv[2],sys.stdin.read()) z.close() " test.zip testfile1 $ unzip -l test.zip Archive: test.zip Length Date Time Name -------- ---- ---- ---- 9 03-16-14 00:47 testfile1 $ unzip test.zip Archive: test.zip unzip: skipping non-regular entry 'testfile1' $ /usr/local/bin/unzip test.zip Archive: test.zip extracting: testfile1 $ cat testfile1 testtext $ for a correct file zipinfo shows (example): Unix file attributes (100744 octal): -rwxr--r-- Unix file attributes (040744 octal): drwxr--r-- for dropbox or above example: Unix file attributes (000600 octal): ?rw------- recognize the questionmark where filetype should be (=0x00). The extraction seems to work correctly if we remove that sanity check for S_ISDIR and S_ISREG. But as the program uses the information for program flow that may be a problem. As more and more archives are generated on the fly, maybe that issue will get more serious. Maybe you can give me a hint if it's okay to remove that sanity check or if you want to keep it. [1] https://www.pkware.com/documents/casestudies/APPNOTE.TXT -- You are receiving this mail because: You are the assignee for the bug.