From owner-freebsd-arch@FreeBSD.ORG Thu Jul 12 01:58:47 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 46C31106564A for ; Thu, 12 Jul 2012 01:58:47 +0000 (UTC) (envelope-from tim@kientzle.com) Received: from monday.kientzle.com (99-115-135-74.uvs.sntcca.sbcglobal.net [99.115.135.74]) by mx1.freebsd.org (Postfix) with ESMTP id 191F48FC16 for ; Thu, 12 Jul 2012 01:58:46 +0000 (UTC) Received: (from root@localhost) by monday.kientzle.com (8.14.4/8.14.4) id q6C1wZY3038023; Thu, 12 Jul 2012 01:58:35 GMT (envelope-from tim@kientzle.com) Received: from [192.168.2.143] (CiscoE3000 [192.168.1.65]) by kientzle.com with SMTP id f4dfrxnk5ryjg736w48389c8mi; Thu, 12 Jul 2012 01:58:35 +0000 (UTC) (envelope-from tim@kientzle.com) Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=windows-1252 From: Tim Kientzle In-Reply-To: <861ukio8p7.fsf@ds4.des.no> Date: Wed, 11 Jul 2012 18:58:34 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <8E4D9B54-C328-427D-89A1-0CB8E57CD064@kientzle.com> References: <25149679-6B99-4FF0-AB8C-90D5A7880F00@kientzle.com> <861ukio8p7.fsf@ds4.des.no> To: =?iso-8859-1?Q?Dag-Erling_Sm=F8rgrav?= X-Mailer: Apple Mail (2.1278) Cc: Ryan Stone , freebsd-arch@freebsd.org Subject: Re: Generating a tarball directly from make installworld X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jul 2012 01:58:47 -0000 On Jul 11, 2012, at 4:09 AM, Dag-Erling Sm=F8rgrav wrote: > Tim Kientzle writes: >> /bin/ls user=3Droot group=3Dwheel mode=3D0755 = contents=3D/usr/obj/usr/src/bin/ls/ls >=20 > Unfortunately, passing that line to tar is not quite equivalent to = first > installing ls then tarring it, because install normally strips ls. Yes, it does today. But we could easily produce the stripped executable at build time using something like this in bsd.prog.mk: ${PROG}: ${OBJS} ${CC} =85. -o ${.TARGET} ${OBJS} =85 cp ${.TARGET} ${.TARGET}.stripped strip ${.TARGET}.stripped Then ${.TARGET}.stripped is ready to be copied as-is into the tarball. While we're at it, I think the following points out how to address another of Ryan's issues: --- share/mk/bsd.prog.mk.original 2012-07-11 18:48:11.000000000 = -0700 +++ share/mk/bsd.prog.mk 2012-07-11 18:48:35.000000000 -0700 @@ -155,7 +155,7 @@ ${_INSTALLFLAGS} ${PROG} ${DESTDIR}${BINDIR}/${PROGNAME} .else ${INSTALL} ${STRIP} -o ${BINOWN} -g ${BINGRP} -m ${BINMODE} \ - ${_INSTALLFLAGS} ${PROG} ${DESTDIR}${BINDIR} + ${_INSTALLFLAGS} ${PROG} ${DESTDIR}${BINDIR}/${PROG} .endif .endif .endif # !target(realinstall) > There may be other cases as well where the file that ends up being > installed does not exist in that form - or at all - in the object = tree, > but is transformed or even created at install time. Yes. Ryan listed quite a few of these in his original message. I suspect they'll have to be handled very much on a case-by-case basis. There are several different strategies that might apply: Option #1. Move work from install time to build time. This is easy for single-file transforms such as stripping binaries, as above. It might be feasible for other cases: I think you could handle kldxref this way, though it would require a little bit of creativity to find all the modules. Option #2. Handle it using a "mini-install" For makewhatis, the easiest might be to install just the man files to a temporary dir and run the indexing against that. This would work well for makewhatis in particular since the ownership and permissions aren't critical for generating that file. Option #3. Integrate it into a special archiving tool. bsdtar demonstrates that mtree-to-tar.gz conversion is feasible, but that doesn't mean the tool that does this needs to be tar itself. A purpose-built tool could use a lot of the same code (much of which is already in libarchive) and add additional capabilities to generate various install files on-the-fly. That would be a lot of work for makewhatis or kldxref, of course, but might be a reasonable way to handle stripping binaries? Option #4. Defer to system boot. We already have systems that do initialization on the first boot after install (ssh host keys, for instance). A generic framework for this would be easy to add to our current boot scripts. I think this would be a very good fit for makewhatis in particular. I could see any of the above being useful, depending on the particular situation. Tim