From owner-freebsd-current@FreeBSD.ORG Tue Feb 25 22:46:22 2014 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5E84B3DF for ; Tue, 25 Feb 2014 22:46:22 +0000 (UTC) Received: from mout.gmx.net (mout.gmx.net [212.227.15.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E3FB6141E for ; Tue, 25 Feb 2014 22:46:21 +0000 (UTC) Received: from [192.168.0.100] ([87.139.233.65]) by mail.gmx.com (mrgmx103) with ESMTPSA (Nemesis) id 0Lz3JU-1XMeOK0yGW-014Eyv for ; Tue, 25 Feb 2014 23:46:13 +0100 Message-ID: <530D1D36.1020104@gmx.de> Date: Tue, 25 Feb 2014 23:46:14 +0100 From: olli hauer User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: freebsd-current@freebsd.org Subject: Re: Import of DragonFly Mail Agent References: <20140223211155.GS1699@ithaqua.etoilebsd.net> <530B67EA.1090102@FreeBSD.org> <45746.1393257416@critter.freebsd.dk> <17F7065F-5CE1-47CE-AB45-7356E78E243A@FreeBSD.org> <20140225153156.3648c551@gumby.homeunix.com> In-Reply-To: <20140225153156.3648c551@gumby.homeunix.com> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:kPz/Bn660QUzdmf1cpLk91nl4iNghey9zj0ESPply0/LfDlDu1/ JGHXaj5amSOy3ccd2P089EjA7Actqr3I7ScreYbQfINw44n/faW1xVV2BvSFhsiGdiLMq3O 5j3gbmjRC8JFRSiJfExpLwQEWD320m3PvLWlrJMUP6Pg0pYA3R7TFUuljHwdN57gmfipdlB 5KCIzy49M0mJmd3aC8qQQ== Cc: RW , Lyndon Nerenberg X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Feb 2014 22:46:22 -0000 On 2014-02-25 16:31, RW wrote: > On Mon, 24 Feb 2014 19:24:02 -0500 (EST) > Benjamin Kaduk wrote: > >> On Mon, 24 Feb 2014, Lyndon Nerenberg wrote: >> >>> >>> What would really help is if the ports fetch-recursive-list target >>> could extend to reliably include the distfiles for the runtime >>> dependencies as well. But I'm not even sure that's possible. We >>> tried a few different things, but in the end we had to brute force >>> it by running 'make fetch' in every one of the ports directories in >>> order to get all the distfiles onto an external system, which we >>> then rsynced to a USB drive, marched inside, and rsynced to the >>> fileserver. Not pretty ... but with all the distfiles at hand we >>> knew the inside ports builds wouldn't fail due to missing >>> dependencies. >> >> I'm rather confused by why it isn't working for you. >> http://svnweb.freebsd.org/ports/head/Mk/bsd.port.mk?revision=345884&view=markup#l5187 >> is quite clearly looking in ALL-DEPENDS-LIST, which includes runtime >> dependencies. The only thing I can think of is that non-default >> configurations are in play, so that 'make config && make >> config-recursive' should be (re-)run until it does not prompt, and >> only then fetch-recursive-list be used. > > > One oddity is that fetch-recursive-list generates a script that > downloads all the files into the current directory. It doesn't take > account of the fact that some ports look for their files are in a > sub-directory. Some snippets from a script that is used to manage updates, tinderboxe builds, poudriere builds ... I collected all ports that are required to build my environments from tinderbox (./tc listPorts) and others in a plain txt file. in the format $cat/$port. ... databases/php5-pdo databases/php5-pdo_mysql databases/php5-pdo_pgsql databases/php5-pdo_sqlite databases/php5-pgsql databases/postgresql92-client databases/postgresql92-server databases/postgresql93-client databases/postgresql93-server databases/py-gdbm databases/rrdtool databases/rrdtool12 databases/sqlite3 ... Reading this file in a loop with a command like the following will fetch all required distfiles. while read port; do env -i WRKDIRPREFIX=/tmp/rbtrash PKG_DBDIR=/var/empty \ LOCALBASE=/var/empty make fetch -DBATCH -C /usr/ports/${port} \ -DCLEAN_FETCH_ENV -DDISABLE_CONFLICTS done < $path/to/interesting/port/list A list of all required dependency's can be generated with this command (for a single port or in the sample loop (s/fetch/all-depends-list/) $> make all-depends-list /usr/ports/$cat/${port} Ports tree updates (portsnap or svn up) are written to a log which is used to generate a list of ports where the distfile is maybe missing, the loop reads then only this new list. The directory with all distfiles is distributed via httpd to all build systems (make.conf: MASTER_SITE_OVERRIDE=$central/fetch/server/url ) Hope this gives some ideas ;) -- olli