From owner-freebsd-ports@FreeBSD.ORG Sun Apr 15 15:25:27 2007 Return-Path: X-Original-To: ports@freebsd.org Delivered-To: freebsd-ports@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 503DF16A400 for ; Sun, 15 Apr 2007 15:25:27 +0000 (UTC) (envelope-from uspoerlein@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.169]) by mx1.freebsd.org (Postfix) with ESMTP id CB07613C4B0 for ; Sun, 15 Apr 2007 15:25:26 +0000 (UTC) (envelope-from uspoerlein@gmail.com) Received: by ug-out-1314.google.com with SMTP id 71so769742ugh for ; Sun, 15 Apr 2007 08:25:25 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:received:received:date:from:to:subject:message-id:mail-followup-to:references:mime-version:content-type:content-disposition:in-reply-to; b=rIKhST2+Xdgd19kasmKufV//BwSSbTi37MSbwWy/d2ESpHJC268N0At2zgL5h1dFwXTfghLynxQl78WPJp8/8RpPNcJ2eX4rWnMkGrFLAZzWWY7nuzT2GKhRyXPv7uyEgIrAQWi+yFNTLCELxamoFdwzg+4x/+Ra6RcB1H4Hx3I= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:date:from:to:subject:message-id:mail-followup-to:references:mime-version:content-type:content-disposition:in-reply-to; b=GRNpZjYY0Fz43P7maBQBsas4p5mYrd+lJfJ+IOQIK40F0vs0oZm2wY+8e8OhIHgUoahhAJLjJicC2cM2jAl0V6RDnOjKlyYOF3b8YFZK40YRsImZjnSIvnRxwZpNdIPEkfZA/AizCFupqWIgw8KRLQn4bKUSXMb1cF8xADpco28= Received: by 10.66.237.9 with SMTP id k9mr3611789ugh.1176650724602; Sun, 15 Apr 2007 08:25:24 -0700 (PDT) Received: from roadrunner.q.local ( [85.180.142.223]) by mx.google.com with ESMTP id w5sm10294654mue.2007.04.15.08.25.22; Sun, 15 Apr 2007 08:25:24 -0700 (PDT) Received: from roadrunner.q.local (localhost [127.0.0.1]) by roadrunner.q.local (8.13.8/8.13.8) with ESMTP id l3FFIZdo008022 for ; Sun, 15 Apr 2007 17:18:35 +0200 (CEST) (envelope-from uspoerlein@gmail.com) Received: (from q@localhost) by roadrunner.q.local (8.13.8/8.13.8/Submit) id l3FFIZvc008021 for ports@freebsd.org; Sun, 15 Apr 2007 17:18:35 +0200 (CEST) (envelope-from uspoerlein@gmail.com) Date: Sun, 15 Apr 2007 17:18:34 +0200 From: Ulrich Spoerlein To: ports@freebsd.org Message-ID: <20070415151834.GE1525@roadrunner.q.local> Mail-Followup-To: ports@freebsd.org References: <20070415092815.GB1525@roadrunner.q.local> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="h56sxpGKRmy85csR" Content-Disposition: inline In-Reply-To: <20070415092815.GB1525@roadrunner.q.local> Cc: Subject: Re: CFT: Parallel package building script X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Apr 2007 15:25:27 -0000 --h56sxpGKRmy85csR Content-Type: multipart/mixed; boundary="yudcn1FV7Hsu/q59" Content-Disposition: inline --yudcn1FV7Hsu/q59 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Ulrich Spoerlein wrote: > I wrote a script=20 That got stripped by the mailling lists ... second try. You can also fetch it from http://coyote.dnsalias.net/create_pkg_makefile.sh Ulrich Spoerlein --=20 "The trouble with the dictionary is you have to know how the word is spelled before you can look it up to see how it is spelled." -- Will Cuppy --yudcn1FV7Hsu/q59 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="create_pkg_makefile.sh" Content-Transfer-Encoding: quoted-printable #!/usr/bin/env zsh # vi:set sw=3D2 tw=3D78: # # "THE BEER-WARE LICENSE" (Revision 42): # wrote this file. As long as you retain this notice= you # can do whatever you want with this stuff. If we meet some day, and you th= ink # this stuff is worth it, you can buy me a beer in return. Ulrich Spoerlein #=20 # What this is: #=20 # This script will output a Makefile, that is self-contained and suitable f= or # building individual sets of packages with maximum parallelism. # # Who it is for: # # For people needing to build clean packages, probably with a custom # make.conf, for system deployment. # # How it works: #=20 # It will recursively walk the $PORTS tree for the specified package origins # (or all POs if none are given) and create the correct dependency tree plus # targets to build those packages. # # To guarantee a clean package without "contamination", as can happen by # configure scripts picking up random installed libraries, it uses a clean # "installworld" provided by either an unionfs mount or by copying the tree= to # the destination. (There are a some problems with unionfs mounts, if packa= ges # don't build, try the cpio method first.) # # How to use: #=20 # First, you need to create a clean environment: # cd $SRC && make buildworld installworld distribution DESTDIR=3D$ROOT/live # mkdir -p $ROOT/live/usr/ports $ROOT/distfiles $ROOT/obj $ROOT/packages/All # # Since we can't change make(1)s .OBJDIR to some arbitrary value, create an # obj link to the package directory: # ln -sf $ROOT/packages/All obj # # Create a make.conf with your desired flags, set CONF=3D$PWD/make.conf # (= default) # vi make.conf # # And torture your machine=20 # ./create_pkg_makefiles.sh editors/vim x11/kde3 | make -f- -j4 all #=20 # To watch the build progressing, use something like: # while sleep 3; do screen -r; done # # The Makefile for all ports can be created by: # ./create_pkg_makefiles.sh > Makefile # # The special "check" target will only print what packages would be build; # use it in conjunction with "all" or your desired package origins. # make -j4 check editors/vim x11/kde3 # make check all # # To update all your installed packages, a heavy handed approach would be t= o: # cp /etc/make.conf . # ./create_pkg_makefiles.sh `pkg_info -qao` | make -f- -j2 all # pkg_delete -fa # pkg_add -F $ROOT/packages/All/* #=20 # User configuration ROOT=3D/vol PORTS=3D/usr/ports SRC=3D/usr/src CONF=3D$PWD/make.conf #CONF=3D/etc/make.conf # End of user configuration. # Global hash, saving pkg-origin -> pkg-name. Is also used to look up # which pkg-origins have already been checked. typeset -A pkg print_make_header() { cat </dev/null; }; parse_time =2ESILENT: check: EOS } print_pkg() { local po po=3D$1; shift cat </dev/null \${CREATE_BUILD_SPACE} $po echo '$po: Building package' \${CREATE_PACKAGE} $po \${PARSE_TIME} $po echo '$po: Cleaning build space' \${CLEAN_BUILD_SPACE} $po =2Eendif EOS } get_pkg_recursive() { local po list depport deplist level po=3D$1; shift level=3D"${1}=3D"; shift # Don't revisit already visited nodes if [ -n "$pkg[$po]" ]; then return fi if [ -z "$pkg[$po]" ]; then pkg[$po]=3D$(make -C $po __MAKE_CONF=3D$CONF -V PKGNAME).tbz fi printf "%-45s %s\n" "$level > $po" $pkg[$po] >&2 # Get dependency dirs list=3D$(make -C $po __MAKE_CONF=3D$CONF -V _DEPEND_DIRS | sed "s|${PORTS= }/*||g") for depport in `eval echo $list`; do if [ -z "$pkg[$depport]" ]; then get_pkg_recursive $depport $level fi case "$deplist" in *${pkg[$depport]}*) ;; *) deplist=3D"$deplist ${pkg[$depport]}" ;; esac done echo "$level < $po" >&2 print_pkg $po `eval echo $deplist` } # Main entry point print_make_header cd $PORTS if [ "$#" -gt 0 ]; then for po; do get_pkg_recursive $po "" done else for sub in $(make -C $PORTS -V SUBDIR); do for port in $(make -C $PORTS/$sub -V SUBDIR); do get_pkg_recursive $sub/$port "" done done fi printf "\nall: $pkg\n" --yudcn1FV7Hsu/q59-- --h56sxpGKRmy85csR Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) iD8DBQFGIkJK524iJyD+6d0RAiGAAJ9ZvF+3MJ3aCz6ljRwt5IsCHVs+LQCeO2IF Ho8SrGfTQrVUImxNZyUlgRU= =ZO6M -----END PGP SIGNATURE----- --h56sxpGKRmy85csR--