From owner-svn-doc-all@FreeBSD.ORG Tue Apr 29 21:38:02 2014 Return-Path: Delivered-To: svn-doc-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4D1E6FC0; Tue, 29 Apr 2014 21:38:02 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3880F1C7E; Tue, 29 Apr 2014 21:38:02 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s3TLc2rl053114; Tue, 29 Apr 2014 21:38:02 GMT (envelope-from wblock@svn.freebsd.org) Received: (from wblock@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s3TLc23J053113; Tue, 29 Apr 2014 21:38:02 GMT (envelope-from wblock@svn.freebsd.org) Message-Id: <201404292138.s3TLc23J053113@svn.freebsd.org> From: Warren Block Date: Tue, 29 Apr 2014 21:38:02 +0000 (UTC) To: doc-committers@freebsd.org, svn-doc-all@freebsd.org, svn-doc-head@freebsd.org Subject: svn commit: r44708 - head/en_US.ISO8859-1/articles/portbuild X-SVN-Group: doc-head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-doc-all@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "SVN commit messages for the entire doc trees \(except for " user" , " projects" , and " translations" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Apr 2014 21:38:02 -0000 Author: wblock Date: Tue Apr 29 21:38:01 2014 New Revision: 44708 URL: http://svnweb.freebsd.org/changeset/doc/44708 Log: Restore lost tags. 100 of them. Modified: head/en_US.ISO8859-1/articles/portbuild/article.xml Modified: head/en_US.ISO8859-1/articles/portbuild/article.xml ============================================================================== --- head/en_US.ISO8859-1/articles/portbuild/article.xml Tue Apr 29 21:16:47 2014 (r44707) +++ head/en_US.ISO8859-1/articles/portbuild/article.xml Tue Apr 29 21:38:01 2014 (r44708) @@ -236,29 +236,29 @@ Package builds are performed by the clients in a jail populated by the portbuild script using the - ${arch}/${branch}/builds/${buildid}/bindist.tar + ${arch}/${branch}/builds/${buildid}/bindist.tar file. On the server, use the makeworld command to build a world from the - ${arch}/${branch}/builds/${buildid}/src/ + ${arch}/${branch}/builds/${buildid}/src/ tree and install it into - ${arch}/${branch}/builds/${buildid}/bindist.tar. + ${arch}/${branch}/builds/${buildid}/bindist.tar. The tree will be updated first unless -novcs is specified. - &prompt.root; /a/portbuild/admin/scripts/makeworld ${arch} ${branch} ${buildid} [-novcs] + &prompt.root; /a/portbuild/admin/scripts/makeworld ${arch} ${branch} ${buildid} [-novcs] Similiarly on the server, the bindist.tar tarball is created from the previously installed world by the mkbindist script. - &prompt.root; /a/portbuild/admin/scripts/mkbindist ${arch} ${branch} ${buildid} + &prompt.root; /a/portbuild/admin/scripts/mkbindist ${arch} ${branch} ${buildid} The per-machine tarballs are located on the server in - ${arch}/clients. + ${arch}/clients. The bindist.tar file is extracted onto each client at client boot time, and at the start of @@ -287,37 +287,37 @@ make.conf and/or src.conf, named - ${arch}/${branch}/builds/${buildid}/make.conf.server + ${arch}/${branch}/builds/${buildid}/make.conf.server and - ${arch}/${branch}/builds/${buildid}/src.conf.server, + ${arch}/${branch}/builds/${buildid}/src.conf.server, respectively. These will be used in lieu of the default-named files on the server side. Similarly, if you wish to also affect the client-side make.conf, you may use - ${arch}/${branch}/builds/${buildid}/make.conf.client. + ${arch}/${branch}/builds/${buildid}/make.conf.client. Due to the fact that individual clients may each have their own per-host make.conf, the contents of - ${arch}/${branch}/builds/${buildid}/make.conf.client + ${arch}/${branch}/builds/${buildid}/make.conf.client will be appended to that make.conf, not supplant it, as is done for - ${arch}/${branch}/builds/${buildid}/make.conf.server. + ${arch}/${branch}/builds/${buildid}/make.conf.server. There is no similar functionality for - ${arch}/${branch}/builds/${buildid}/src.conf.client + ${arch}/${branch}/builds/${buildid}/src.conf.client (what effect would it have?). Sample - <filename>make.conf.target</filename> + <filename>make.conf.<replaceable>target</replaceable></filename> to test new default <application>ruby</application> version @@ -329,7 +329,7 @@ Sample - <filename>make.conf.target</filename> + <filename>make.conf.<replaceable>target</replaceable></filename> for <application>clang</application> builds (For this case, the contents are also identical for both @@ -378,7 +378,7 @@ PKG_BIN=/usr/local/sbin/pkgSeparate builds for various combinations of architecture and branch are supported. All data private to a build (ports tree, src tree, packages, distfiles, log files, bindist, Makefile, etc) are located under the - ${arch}/${branch}/builds/${buildid}/ + ${arch}/${branch}/builds/${buildid}/ directory. The most recently created build can be alternatively referenced using buildid latest, and the one before using @@ -393,7 +393,7 @@ PKG_BIN=/usr/local/sbin/pkgThe scripts/dopackages.wrapper script is used to perform the builds. - &prompt.root; dopackages.wrapper ${arch} ${branch} ${buildid} [-options] + &prompt.root; dopackages.wrapper ${arch} ${branch} ${buildid} [-options] Most often, you will be using latest for the value of buildid. @@ -570,7 +570,7 @@ PKG_BIN=/usr/local/sbin/pkg-nocleanup, you need to clean up clients by running - &prompt.user; build cleanup ${arch} ${branch} ${buildid} -full + &prompt.user; build cleanup ${arch} ${branch} ${buildid} -full When a new build is created, the directories errors/, logs/, packages/, and so @@ -608,7 +608,7 @@ PKG_BIN=/usr/local/sbin/pkg${arch}/${branch}/journal: + ${arch}/${branch}/journal: don't know how to make dns-all(continuing) @@ -656,34 +656,34 @@ PKG_BIN=/usr/local/sbin/pkg - build create arch - branch - [newid] - Creates + build create arch + branch + [newid] - Creates newid (or a datestamp if not specified). Only needed when bringing up a new branch or a new architecture. - build clone arch - branch oldid - [newid] - Clones + build clone arch + branch oldid + [newid] - Clones oldid to newid (or a datestamp if not specified). - build srcupdate arch - branch - buildid - Replaces the src + build srcupdate arch + branch + buildid - Replaces the src tree with a new ZFS snapshot. Do not forget to use -nosrc flag to dopackages later! - build portsupdate arch - branch - buildid - Replaces the ports + build portsupdate arch + branch + buildid - Replaces the ports tree with a new ZFS snapshot. Do not forget to use -noports flag to dopackages later! @@ -698,7 +698,7 @@ PKG_BIN=/usr/local/sbin/pkg - &prompt.root; path/qmanager/packagebuild amd64 7-exp 20080904212103 aclock-0.2.3_2.tbz + &prompt.root; path/qmanager/packagebuild amd64 7-exp 20080904212103 aclock-0.2.3_2.tbz @@ -714,7 +714,7 @@ PKG_BIN=/usr/local/sbin/pkgAn update of the current ports tree from the ZFS snapshot Status of these steps can be found in - ${arch}/${branch}/build.log + ${arch}/${branch}/build.log as well as on stderr of the tty running the dopackages command. @@ -753,12 +753,12 @@ PKG_BIN=/usr/local/sbin/pkg Builds packages (phase 1)Status of these steps can be found in - ${arch}/${branch}/journal. + ${arch}/${branch}/journal. Individual ports will write their build logs to - ${arch}/${branch}/logs/ + ${arch}/${branch}/logs/ and their error logs to - ${arch}/${branch}/errors/. + ${arch}/${branch}/errors/. @@ -882,7 +882,7 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 After you have done all the above, remove the - ${arch}/lock + ${arch}/lock file before trying to restart the build. If you do not, dopackages will simply exit. @@ -900,14 +900,14 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 - build destroy arch - branch - Destroy the + build destroy arch + branch - Destroy the build id. - build list arch - branch - Shows the current set + build list arch + branch - Shows the current set of build ids. @@ -920,11 +920,11 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 You can use qclient command to monitor the status of build nodes, and to list the currently scheduled jobs: - &prompt.user; python path/qmanager/qclient jobs -&prompt.user; python path/qmanager/qclient status + &prompt.user; python path/qmanager/qclient jobs +&prompt.user; python path/qmanager/qclient status The - scripts/stats ${branch} + scripts/stats ${branch} command shows the number of packages already built. Running cat /a/portbuild/*/loads/* @@ -942,7 +942,7 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 idle node. - Running tail -f ${arch}/${branch}/build.log + Running tail -f ${arch}/${branch}/build.log shows the overall build progress. If a port build is failing, and it is not immediately obvious @@ -952,7 +952,7 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 directory. The next time the cluster tries to build this port, it will tar, compress, and copy the WRKDIR to - ${arch}/${branch}/wrkdirs/. + ${arch}/${branch}/wrkdirs/. If you find that the system is looping trying to build the same package over and over again, you may be able to fix the @@ -1027,7 +1027,7 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 the emailed logs and sort them to a folder, so you can maintain a running list of current failures and detect new ones easily. To do this, add an email address to - ${branch}/portbuild.conf. + ${branch}/portbuild.conf. You can easily bounce the new ones to maintainers. After a port appears broken on every build combination @@ -1066,8 +1066,8 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 build is post-processed, take an inventory of the list of files fetched: - &prompt.user; cd ${arch}/${branch} -&prompt.user; find distfiles > distfiles-${release} + &prompt.user; cd ${arch}/${branch} +&prompt.user; find distfiles > distfiles-${release} You should use that output to periodically clean out the distfiles from ftp-master. When space @@ -1077,12 +1077,12 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 Once the distfiles have been uploaded (see below), the final release package set must be created. Just to be on the safe side, run the - ${arch}/${branch}/cdrom.sh + ${arch}/${branch}/cdrom.sh script by hand to make sure all the CD-ROM restricted packages and distfiles have been pruned. Then, copy the - ${arch}/${branch}/packages + ${arch}/${branch}/packages directory to - ${arch}/${branch}/packages-${release}. + ${arch}/${branch}/packages-${release}. Once the packages are safely moved off, contact the &a.re; and inform them of the release package location. @@ -1180,13 +1180,13 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 a new release), copy packages to the staging area on ftp-master with something like the following: - &prompt.root; cd /a/portbuild/${arch}/${branch} -&prompt.root; tar cfv - packages/ | ssh portmgr@ftp-master tar xfC - w/ports/${arch}/tmp/${subdir} + &prompt.root; cd /a/portbuild/${arch}/${branch} +&prompt.root; tar cfv - packages/ | ssh portmgr@ftp-master tar xfC - w/ports/${arch}/tmp/${subdir} Then log into ftp-master, verify that the package set was transferred successfully, remove the package set that the new package set is to replace (in - ~/w/ports/${arch}), + ~/w/ports/${arch}), and move the new set into place. (w/ is merely a shortcut.) @@ -1203,12 +1203,12 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 Example rsync command for incremental package upload: - &prompt.root; rsync -n -r -v -l -t -p --delete packages/ portmgr@ftp-master:w/ports/${arch}/${subdir}/ | tee log + &prompt.root; rsync -n -r -v -l -t -p --delete packages/ portmgr@ftp-master:w/ports/${arch}/${subdir}/ | tee log Distfiles should be transferred with the cpdistfiles script: - &prompt.root; /a/portbuild/scripts/cpdistfiles ${arch} ${branch} ${buildid} [-yesreally] | tee log2 + &prompt.root; /a/portbuild/scripts/cpdistfiles ${arch} ${branch} ${buildid} [-yesreally] | tee log2 Doing it by hand is deprecated. @@ -1523,7 +1523,7 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 directory to contain the install bits. You will probably want to use a subdirectory of /pxeroot, e.g., - /pxeroot/${arch}-${branch}. + /pxeroot/${arch}-${branch}. Export that as DESTDIR. @@ -1584,8 +1584,8 @@ options NFSSERVER As root, do the usual build steps, e.g.: &prompt.root; make -j4 buildworld -&prompt.root; make buildkernel KERNCONF=${kernconf} -&prompt.root; make installkernel KERNCONF=${kernconf} +&prompt.root; make buildkernel KERNCONF=${kernconf} +&prompt.root; make installkernel KERNCONF=${kernconf} &prompt.root; make installworld The install steps use DESTDIR. @@ -1599,15 +1599,15 @@ options NFSSERVER If you are using pxeboot: create a subdirectory of - ${DESTDIR} + ${DESTDIR} called conf/. Create one subdirectory default/etc/, and (if your site will host multiple nodes), subdirectories - ${ip-address}/etc/ + ${ip-address}/etc/ to contain override files for individual hosts. (You may find it handy to symlink each of those directories to a hostname.) Copy the entire contents of - ${DESTDIR}/etc/ + ${DESTDIR}/etc/ to default/etc/; that is where you will edit your files. The by-ip-address etc/ directories will probably only need @@ -1702,8 +1702,8 @@ sshd_program="/usr/local/sbin/sshd" squid_enable="YES" -squid_chdir="/a/squid/logs" -squid_pidfile="/a/squid/logs/squid.pid" +squid_chdir="/a/squid/logs" +squid_pidfile="/a/squid/logs/squid.pid" Required entries for VMWare-based nodes: vmware_guest_vmmemctl_enable="YES" @@ -1724,8 +1724,8 @@ sshd_program="/usr/local/sbin/sshd" gmond_enable="YES" squid_enable="YES" -squid_chdir="/a/squid/logs" -squid_pidfile="/a/squid/logs/squid.pid" +squid_chdir="/a/squid/logs" +squid_pidfile="/a/squid/logs/squid.pid" &man.ntpd.8; should not be enabled for VMWare instances. @@ -1733,7 +1733,7 @@ squid_pidfile="/a/squid/logs/s Also, it may be possible to leave squid disabled by default so as to not have - /a + /a persistent (which should save instantiation time.) Work is still ongoing. @@ -1748,7 +1748,7 @@ squid_pidfile="/a/squid/logs/s Modify etc/sysctl.conf: 9a10,30 -> kern.corefile=/a/%N.core +> kern.corefile=/a/%N.core > kern.sugid_coredump=1 > #debug.witness_ddb=0 > #debug.witness_watch=0 @@ -1856,7 +1856,7 @@ security/sudo # # Configure a package build system post-boot -scratchdir=/a +scratchdir=/a ln -sf ${scratchdir}/portbuild /var/ @@ -1895,7 +1895,7 @@ touch /tmp/.boot_finished Also, change usr/local - to usr2 in + to usr2 in cache_dir, access_log, cache_log, @@ -1940,7 +1940,7 @@ portbuild ALL=(ALL) NOPASSWD: ALL

Change into the port/package directory you picked above, e.g., - cd /usr2. + cd /usr2. @@ -2010,7 +2010,7 @@ portbuild ALL=(ALL) NOPASSWD: ALL

Create - /a/portbuild/${arch}/clients/bindist-${hostname}.tar. + /a/portbuild/${arch}/clients/bindist-${hostname}.tar. @@ -2055,15 +2055,15 @@ MASTER_SITE_OVERRIDE= \ Create -/a/portbuild/${arch}/portbuild-${hostname} +/a/portbuild/${arch}/portbuild-${hostname} using one of the existing ones as a guide. This file contains overrides to -/a/portbuild/${arch}/portbuild.conf. +/a/portbuild/${arch}/portbuild.conf. Suggested values: disconnected=1 -scratchdir=/usr2/pkgbuild +scratchdir=/usr2/pkgbuild client_user=portbuild sudo_cmd="sudo -H" rsync_gzip=-z @@ -2075,7 +2075,7 @@ infoseek_port=${tunelled-tc on the client: http_proxy="http://localhost:3128/" -squid_dir=/usr2/squid +squid_dir=/usr2/squid If, instead, you will be using squid on the server: @@ -2125,7 +2125,7 @@ ssh_cmd="/usr/local/bin/ssh" Ensure that ssh to the client is working by executing - ssh hostname uname -a. + ssh hostname uname -a. The actual command is not important; what is important is to confirm the setup, and also add an entry into known_hosts, once you have confirmed the @@ -2135,13 +2135,13 @@ ssh_cmd="/usr/local/bin/ssh" Populate the client's copy of /var/portbuild/scripts/ by something like - /a/portbuild/scripts/dosetupnode arch major latest hostname. + /a/portbuild/scripts/dosetupnode arch major latest hostname. Verify that you now have files in that directory. Test the other TCP ports by executing - telnet hostname portnumber. + telnet hostname portnumber. 414 (or its tunnel) should give you a few lines of status information including arch and osversion; 8649 should @@ -2157,17 +2157,17 @@ ssh_cmd="/usr/local/bin/ssh" Tell qmanager about the node. Example: - python path/qmanager/qclient add - name=uniquename - arch=arch - osversion=osversion - numcpus=number + python path/qmanager/qclient add + name=uniquename + arch=arch + osversion=osversion + numcpus=number haszfs=0 online=1 - domain=domain + domain=domain primarypool=package pools="package all" maxjobs=1 - acl="ports-arch,deny_all" + acl="ports-arch,deny_all" @@ -2180,7 +2180,7 @@ ssh_cmd="/usr/local/bin/ssh"Once you are sure that the client is working, tell pollmachine about it by adding it to - /a/portbuild/${arch}/mlist. + /a/portbuild/${arch}/mlist. @@ -2216,7 +2216,7 @@ ssh_cmd="/usr/local/bin/ssh" For what was previously head, change SRC_BRANCH_branch_SUBDIR to - releng/branch.0 + releng/branch.0 (literal zero). @@ -2301,22 +2301,22 @@ zfs destroy -r a/snap/src-o Files named - *-old_branch-failure.html + *-old_branch-failure.html Files named - buildlogs_*-old_branch-*-logs.txt + buildlogs_*-old_branch-*-logs.txt Symlinks named - *-old_branch-previous* + *-old_branch-previous* Symlinks named - *-old_branch-latest* + *-old_branch-latest* @@ -2430,7 +2430,7 @@ zfs destroy -r a/snap/src-o It is possible that /dumpster/pointyhat will not have enough space. In that case, create the archive directory as - /dumpster/pointyhat/arch/archive + /dumpster/pointyhat/arch/archive and symlink to that. @@ -2448,7 +2448,7 @@ zfs destroy -r a/snap/src-o Create customized - portbuild.machinename.conf + portbuild.machinename.conf files as appropriate. @@ -2763,7 +2763,7 @@ sh -x ./tmp/mkportbuild Pick a mountpoint and export it. We have used - /a so far to date. + /a so far to date. &prompt.root; export ZFS_MOUNTPOINT=/a @@ -2859,7 +2859,7 @@ sh -x ./tmp/mkportbuild Configure the server by making the following changes to - /a/portbuild/admin/conf/admin.conf: + /a/portbuild/admin/conf/admin.conf: @@ -2906,7 +2906,7 @@ sh -x ./tmp/mkportbuild Configure how build slaves will talk to your server by making the following changes to - /a/portbuild/conf/client.conf: + /a/portbuild/conf/client.conf: @@ -2932,14 +2932,14 @@ sh -x ./tmp/mkportbuild Most of the default values in - /a/portbuild/conf/common.conf + /a/portbuild/conf/common.conf should be fine. This file holds definitions used by both the server and all its clients. Configure the server by making the following changes to - /a/portbuild/conf/server.conf: + /a/portbuild/conf/server.conf: