From owner-svn-doc-head@FreeBSD.ORG Fri Feb 15 14:49:57 2013 Return-Path: Delivered-To: svn-doc-head@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 747CC982; Fri, 15 Feb 2013 14:49:57 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) by mx1.freebsd.org (Postfix) with ESMTP id 58665120; Fri, 15 Feb 2013 14:49:57 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.5/8.14.5) with ESMTP id r1FEnvsT083357; Fri, 15 Feb 2013 14:49:57 GMT (envelope-from linimon@svn.freebsd.org) Received: (from linimon@localhost) by svn.freebsd.org (8.14.5/8.14.5/Submit) id r1FEnvob083356; Fri, 15 Feb 2013 14:49:57 GMT (envelope-from linimon@svn.freebsd.org) Message-Id: <201302151449.r1FEnvob083356@svn.freebsd.org> From: Mark Linimon Date: Fri, 15 Feb 2013 14:49:57 +0000 (UTC) To: doc-committers@freebsd.org, svn-doc-all@freebsd.org, svn-doc-head@freebsd.org Subject: svn commit: r40973 - head/en_US.ISO8859-1/articles/portbuild X-SVN-Group: doc-head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-doc-head@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the doc tree for head List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Feb 2013 14:49:57 -0000 Author: linimon Date: Fri Feb 15 14:49:56 2013 New Revision: 40973 URL: http://svnweb.freebsd.org/changeset/doc/40973 Log: Start scrubbing (or flagging) all obsolete information. While here, continue moving from using /var/portbuild to /a/portbuild for server-based instructions, and be more pedantic about labeling things server-side vs. client-side. Modified: head/en_US.ISO8859-1/articles/portbuild/article.xml Modified: head/en_US.ISO8859-1/articles/portbuild/article.xml ============================================================================== --- head/en_US.ISO8859-1/articles/portbuild/article.xml Fri Feb 15 10:23:34 2013 (r40972) +++ head/en_US.ISO8859-1/articles/portbuild/article.xml Fri Feb 15 14:49:56 2013 (r40973) @@ -77,7 +77,7 @@ - Packages are no longer built for branches 4, 5, or 6, nor + FreeBSD no longer builds packages for branches 4, 5, or 6, nor for the alpha architecture. @@ -113,7 +113,7 @@ - Notes on the codebase + Historical notes on the codebase Until mid-2010, the scripts were completely specific to pointyhat.FreeBSD.org as the head (dispatch) node. During @@ -149,18 +149,6 @@ - This document was originally written before these changes - were made. Where things such as script invocations have changed, - they were denoted as new codebase: as opposed - to old codebase:. - - - Up until November 2012, pointyhat had still - been running the old codebase. That installation has now been - permanently offlined. Therefore, all the instructions having - to do with the old codebase have been removed. - - Also during this process, the codebase was migrated to the @@ -175,11 +163,11 @@ Build Client Management - The &i386; clients co-located with pointyhat - netboot from it (connected nodes); all - other clients (disconnected nodes) - are either self-hosted or netboot from some other - pxe host. + You may set up clients to either netboot from the master + (connected nodes) + or have them either self-hosted or netboot from some other + pxe host + (disconnected nodes). In all cases they set themselves up at boot-time to prepare to build packages. @@ -204,29 +192,31 @@ Jail Build Environment Setup - Package builds are performed in a + Package builds are performed by the clients in a jail populated by the portbuild script using the ${arch}/${branch}/builds/${buildid}/bindist.tar file. - The makeworld command builds a world from the + On the server, use the + makeworld command to build a world from the ${arch}/${branch}/builds/${buildid}/src/ - tree and installs it into + tree and install it into ${arch}/${branch}/builds/${buildid}/bindist.tar. The tree will be updated first unless -novcs is - specified. It should be run as root: + specified. &prompt.root; /a/portbuild/admin/scripts/makeworld ${arch} ${branch} ${buildid} [-novcs] - The bindist.tar tarball is created from the + Similiarly on the server, the + bindist.tar tarball is created from the previously installed world by the mkbindist - script. It should be also be run as root: + script. &prompt.root; /a/portbuild/admin/scripts/mkbindist ${arch} ${branch} ${buildid} - The per-machine tarballs are located in + The per-machine tarballs are located on the server in ${arch}/clients. The bindist.tar file is extracted @@ -237,6 +227,16 @@ For both commands above, if ${buildid} is latest, it may be omitted. + + + Currently the above two scripts must be run as + root; otherwise, the install scripts + lack sufficient permissions. This is undesirable for + security reasons. Work is in progress in -HEAD to allow + users to do installations; once that is committed, the + intention is to use that and run these two commands as + srcbuild. + @@ -1018,12 +1018,7 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 &prompt.user; cd ${arch}/${branch} &prompt.user; find distfiles > distfiles-${release} - - This inventory file typically lives in - i386/${branch} - on the cluster master. - - This is useful to aid in periodically cleaning out + You should use that output to periodically clean out the distfiles from ftp-master. When space gets tight, distfiles from recent releases can be kept while others can be thrown away. @@ -1047,6 +1042,16 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 Uploading Packages + + For FreeBSD.org as of 2013, the instructions + about uploading to ftp-master are obsolete. + In the future, ftp-master will pull + from pointyhat, using a mechanism yet + to be implemented. However, the instructions about + RESTRICTED and NO_CDROM + must still be carefully followed. + + Once a build has completed, packages and/or distfiles can be transferred to ftp-master for propagation to the FTP mirror network. If the build was @@ -1124,7 +1129,7 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 a new release), copy packages to the staging area on ftp-master with something like the following: - &prompt.root; cd /var/portbuild/${arch}/${branch} + &prompt.root; cd /a/portbuild/${arch}/${branch} &prompt.root; tar cfv - packages/ | ssh portmgr@ftp-master tar xfC - w/ports/${arch}/tmp/${subdir} Then log into ftp-master, verify that @@ -1152,7 +1157,7 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 Distfiles should be transferred with the cpdistfiles script: - &prompt.root; /var/portbuild/scripts/cpdistfiles ${arch} ${branch} ${buildid} [-yesreally] | tee log2 + &prompt.root; /a/portbuild/scripts/cpdistfiles ${arch} ${branch} ${buildid} [-yesreally] | tee log2 Doing it by hand is deprecated. @@ -1160,6 +1165,11 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 Experimental Patches Builds + + Most of the information in this section is obsolete + as of 2013. + + Experimental patches builds are run from time to time to new features or bugfixes to the ports infrastructure (i.e. bsd.port.mk), or to test large sweeping @@ -1210,12 +1220,6 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 build. This will ensure an apples-to-apples comparison later. - - Once the build finishes, compare the control build failures to those of the experimental patches build. Use the following commands to facilitate this (this assumes the 8 @@ -1300,9 +1304,9 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 The following command will set up the control branch for - the partial build (old codebase): + the partial build: - &prompt.user; /var/portbuild/scripts/dopackages.8 -noportsvcs -nobuild -novcs -nofinish + &prompt.user; /a/portbuild/scripts/dopackages.wrapper 8 -noportsvcs -nobuild -novcs -nofinish The builds must be performed from the @@ -1317,7 +1321,7 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 <#> is the concurrency of the build to attempt. It is usually the sum of the weights listed in - /var/portbuild/i386/mlist unless you have a + /a/portbuild/i386/mlist unless you have a reason to run a heavier or lighter build. The list of packages to build should be a list of package @@ -1350,17 +1354,15 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 Before following these steps, please coordinate with portmgr. - - Due to some generous donations, portmgr is - no longer looking for the loan of &i386; or amd64 - systems. However, we are still interested in borrowing tier-2 - systems. - - Node requirements - portmgr is still working on characterizing + + This section is only of interest when considering + tier-2 architectures. + + + Here are the requirement for what a node needs to be generally useful. @@ -1440,7 +1442,8 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 /usr2/.) The filename chroot is a - historical remnant. + historical remnant. The chroot + command is no longer used. @@ -1481,8 +1484,9 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 Generate a kernel config file. Include - GENERIC (or, if you are using more than - 3.5G on &i386;, PAE). + GENERIC (or, if on &i386, and + you are using more than + 3.5G, PAE). Required options: @@ -1497,9 +1501,14 @@ options SHMMAXPGS=65536 options SEMMNI=40 options SEMMNS=240 options SEMUME=40 -options SEMMNU=120 +options SEMMNU=120 + + If you are interested in debugging general + problems, you may wish to use the following. + However, for unattended operations, it is best + to leave it out: -options ALT_BREAK_TO_DEBUGGER + options ALT_BREAK_TO_DEBUGGER For PAE, it is not currently possible to load modules. Therefore, if you are running an architecture @@ -1916,7 +1925,7 @@ portbuild ALL=(ALL) NOPASSWD: ALL

Create - /var/portbuild/${arch}/clients/bindist-${hostname}.tar. + /a/portbuild/${arch}/clients/bindist-${hostname}.tar. @@ -1955,16 +1964,16 @@ MASTER_SITE_OVERRIDE= \ Hint: you will need one of these for each machine; however, if you have multiple machines at one site, you should create a site-specific one (e.g., in - /var/portbuild/conf/clients/) + /a/portbuild/conf/clients/) and symlink to it. Create -/var/portbuild/${arch}/portbuild-${hostname} +/a/portbuild/${arch}/portbuild-${hostname} using one of the existing ones as a guide. This file contains overrides to -/var/portbuild/${arch}/portbuild.conf. +/a/portbuild/${arch}/portbuild.conf. Suggested values: @@ -2030,7 +2039,7 @@ ssh_cmd="/usr/local/bin/ssh" Populate the client's copy of /var/portbuild/scripts/ by something like - /var/portbuild/scripts/dosetupnode arch major latest hostname. + /a/portbuild/scripts/dosetupnode arch major latest hostname. Verify that you now have files in that directory. @@ -2075,7 +2084,7 @@ ssh_cmd="/usr/local/bin/ssh"Once you are sure that the client is working, tell pollmachine about it by adding it to - /var/portbuild/${arch}/mlist. + /a/portbuild/${arch}/mlist.