From owner-freebsd-hackers@FreeBSD.ORG Sun Mar 30 10:43:54 2003 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2BF2F37B404; Sun, 30 Mar 2003 10:43:54 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6862943FB1; Sun, 30 Mar 2003 10:43:53 -0800 (PST) (envelope-from wes@softweyr.com) Received: from softweyr.homeunix.net (66-91-236-204.san.rr.com [66.91.236.204]) by smtp-relay.omnis.com (Postfix) with ESMTP id 2A2B3434E8; Sun, 30 Mar 2003 10:43:52 -0800 (PST) From: Wes Peters Organization: Softweyr To: kientzle@acm.org, "Bruce A. Mah" Date: Sun, 30 Mar 2003 10:43:51 -0800 User-Agent: KMail/1.5 References: <3E42C148.4050807@acm.org> <20030329012828.GA32891@intruder.bmah.org> <3E85418F.8010201@acm.org> In-Reply-To: <3E85418F.8010201@acm.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200303301043.51087.wes@softweyr.com> cc: freebsd-hackers@FreeBSD.ORG Subject: BSD tar (was Re: Making pkg_XXX tools smarter about file types...) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Mar 2003 18:43:56 -0000 On Friday 28 March 2003 22:47, Tim Kientzle wrote: > > P.S. It's galled me for a while that pkg_add has > to fork 'tar' to extract the archive. Me too, me too. > I've started > piecing together a library that reads/writes tarfiles. Excellent. A general design goal in userland should be to implement functionality in libraries and then wrap small driver programs around them to export the basic functionality to userland. I guess this partly violates the original UNIX tools philosophy, but all it really does is move it from the original pipes interfaces into the dynamic linker. > With this, it should be possible to make pkg_add > considerably more efficient. In particular, rather > than extracting to a temp directory, then parsing important > information, then moving the files, it should be > possible using this library to read the initial > entries ("+CONTENTS", in particular) directly into > memory, process the information there, then extract the > remainder of the package files directly into their > final locations. I'd much rather see the metadata moved outside the file archives, but that's a separate argument and in now way detracts from your proposed work. ;^) > So far, I have a library API > outlined, and functional read support implemented. > Next step is to hack up a minimal tar implementation > that uses it to make sure everything's working > correctly. > > So far, the library automatically detects compression > formats (using techniques like those in my > pkg_install patch) and has some rough support > for detecting the archive format as well. > (One goal of mine: support for 'pax extended > archives', which I understand can handle ACLs.) I have wondered outloud before if pax might be a suitable starting place for such a hacking expedition. Others who've worked on pax assure me it is not. ;^( Sigh. So much code, so few programmers. > Of course, such a library could also form the > basis for a BSD-licensed tar to replace GNU tar. > I understand a few people have wanted such a thing. Why yes, yes they have. -- Where am I, and what am I doing in this handbasket? Wes Peters wes@softweyr.com