Date: Sat, 16 Feb 2013 04:50:40 +0000 (UTC) From: Glen Barber <gjb@FreeBSD.org> To: doc-committers@freebsd.org, svn-doc-projects@freebsd.org Subject: svn commit: r40977 - in projects/ISBN_1-57176-407-0/en_US.ISO8859-1: articles/committers-guide articles/contributing articles/contributors articles/portbuild books/arch-handbook/boot books/arch-han... Message-ID: <201302160450.r1G4oeVW041722@svn.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: gjb Date: Sat Feb 16 04:50:40 2013 New Revision: 40977 URL: http://svnweb.freebsd.org/changeset/doc/40977 Log: MFH: /head/en_US.ISO8859-1:r40693-40976 Approved by: doceng (implicit) Deleted: projects/ISBN_1-57176-407-0/en_US.ISO8859-1/htdocs/FAQ/ Modified: projects/ISBN_1-57176-407-0/en_US.ISO8859-1/articles/committers-guide/article.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/articles/contributing/article.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/articles/contributors/contrib.additional.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/articles/contributors/contrib.committers.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/articles/portbuild/article.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/arch-handbook/boot/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/arch-handbook/driverbasics/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/arch-handbook/isa/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/arch-handbook/jail/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/arch-handbook/kobj/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/arch-handbook/newbus/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/arch-handbook/pccard/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/arch-handbook/pci/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/arch-handbook/scsi/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/arch-handbook/smp/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/arch-handbook/sound/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/arch-handbook/usb/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/arch-handbook/vm/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/faq/book.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/advanced-networking/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/audit/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/basics/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/boot/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/config/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/cutting-edge/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/desktop/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/disks/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/dtrace/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/filesystems/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/firewalls/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/geom/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/introduction/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/jails/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/kernelconfig/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/l10n/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/linuxemu/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/mac/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/mail/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/mirrors/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/multimedia/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/network-servers/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/ports/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/ppp-and-slip/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/preface/preface.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/printing/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/security/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/serialcomms/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/vinum/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/handbook/virtualization/chapter.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/books/porters-handbook/book.xml (contents, props changed) projects/ISBN_1-57176-407-0/en_US.ISO8859-1/htdocs/Makefile projects/ISBN_1-57176-407-0/en_US.ISO8859-1/htdocs/about.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/htdocs/advocacy/myths.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/htdocs/applications.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/htdocs/cgi/man.cgi projects/ISBN_1-57176-407-0/en_US.ISO8859-1/htdocs/doc/Makefile projects/ISBN_1-57176-407-0/en_US.ISO8859-1/htdocs/donations/donors.xml (contents, props changed) projects/ISBN_1-57176-407-0/en_US.ISO8859-1/htdocs/donations/wantlist.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/htdocs/features.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/htdocs/internal/doceng.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/htdocs/internal/internal.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/htdocs/layout/css/layout.css projects/ISBN_1-57176-407-0/en_US.ISO8859-1/htdocs/mailto.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/htdocs/projects/newbies.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/htdocs/projects/projects.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/htdocs/releng/index.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/htdocs/relnotes.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/htdocs/security/security.xml projects/ISBN_1-57176-407-0/en_US.ISO8859-1/htdocs/snapshots/index.xml Directory Properties: projects/ISBN_1-57176-407-0/en_US.ISO8859-1/ (props changed) Modified: projects/ISBN_1-57176-407-0/en_US.ISO8859-1/articles/committers-guide/article.xml ============================================================================== --- projects/ISBN_1-57176-407-0/en_US.ISO8859-1/articles/committers-guide/article.xml Fri Feb 15 22:47:36 2013 (r40976) +++ projects/ISBN_1-57176-407-0/en_US.ISO8859-1/articles/committers-guide/article.xml Sat Feb 16 04:50:40 2013 (r40977) @@ -26,6 +26,7 @@ <year>2010</year> <year>2011</year> <year>2012</year> + <year>2013</year> <holder>The FreeBSD Documentation Project</holder> </copyright> @@ -212,7 +213,7 @@ <row> <entry>doc</entry> <entry>doceng@</entry> - <entry>doc/, www/, src/ documentation</entry> + <entry>doc/, src/ documentation</entry> </row> <row> @@ -1720,6 +1721,13 @@ U stable/9/share/man/man4/netmap.4 <screen>&prompt.user; <userinput>svn merge -r179454:179453 svn+ssh://svn.freebsd.org/base/ROADMAP.txt</userinput></screen> + <note> + <para>It is important to ensure that the mergeinfo + is correct when reverting a file in order to permit + <command>svn mergeinfo --eligible</command> to work as + expected.</para> + </note> + <para>Reverting the deletion of a file is slightly different. Copying the version of the file that predates the deletion is required. For example, to restore a file that was @@ -4151,24 +4159,6 @@ bak/packages packages from last complet there are at least some perks:</para> <variablelist> - - <varlistentry> - <term>Direct access to <hostid>cvsup-master</hostid></term> - - <listitem> - <para>As a committer, you may apply to &a.kuriyama; for direct access - to <hostid role="fqdn">cvsup-master.FreeBSD.org</hostid>, - providing the public key output from <command>cvpasswd - <replaceable>yourusername</replaceable>@FreeBSD.org - freefall.FreeBSD.org</command>. Please note: you must - specify <hostid>freefall.FreeBSD.org</hostid> on the - <command>cvpasswd</command> command line even though the - actual server is <hostid>cvsup-master</hostid>. Access to - <hostid>cvsup-master</hostid> should not be overused as it is - a busy machine.</para> - </listitem> - </varlistentry> - <varlistentry> <term>Free 4-CD and DVD Sets</term> Modified: projects/ISBN_1-57176-407-0/en_US.ISO8859-1/articles/contributing/article.xml ============================================================================== --- projects/ISBN_1-57176-407-0/en_US.ISO8859-1/articles/contributing/article.xml Fri Feb 15 22:47:36 2013 (r40976) +++ projects/ISBN_1-57176-407-0/en_US.ISO8859-1/articles/contributing/article.xml Sat Feb 16 04:50:40 2013 (r40977) @@ -518,9 +518,9 @@ THIS SOFTWARE, EVEN IF ADVISED OF THE PO <para>Donations may be sent in check form to: <address> The FreeBSD Foundation - <street>7321 Brockway Dr.</street> + <street>P.O. Box 20247</street>, <city>Boulder</city>, - <state>CO</state> <postcode>80303</postcode> + <state>CO</state> <postcode>80308</postcode> <country>USA</country> </address></para> Modified: projects/ISBN_1-57176-407-0/en_US.ISO8859-1/articles/contributors/contrib.additional.xml ============================================================================== --- projects/ISBN_1-57176-407-0/en_US.ISO8859-1/articles/contributors/contrib.additional.xml Fri Feb 15 22:47:36 2013 (r40976) +++ projects/ISBN_1-57176-407-0/en_US.ISO8859-1/articles/contributors/contrib.additional.xml Sat Feb 16 04:50:40 2013 (r40977) @@ -3366,7 +3366,7 @@ <listitem> <para>Gary Hayers - <email>gary@hayers.net</email></para> + <email>gary@hayers.org</email></para> </listitem> <listitem> @@ -5133,6 +5133,11 @@ </listitem> <listitem> + <para>Joseph Mingrone + <email>jrm@ftfl.ca</email></para> + </listitem> + + <listitem> <para>Joseph Scott <email>joseph@randomnetworks.com</email></para> </listitem> @@ -5312,11 +5317,6 @@ </listitem> <listitem> - <para>Kubilay Kocak - <email>koobs.freebsd@gmail.com</email></para> - </listitem> - - <listitem> <para>KUNISHIMA Takeo <email>kunishi@c.oka-pu.ac.jp</email></para> </listitem> @@ -6476,6 +6476,11 @@ </listitem> <listitem> + <para>Matt Stofko + <email>matt@mjslabs.com</email></para> + </listitem> + + <listitem> <para>Matt Thomas <email>matt@3am-software.com</email></para> </listitem> @@ -6830,6 +6835,11 @@ </listitem> <listitem> + <para>Michael Sanders + <email>mike@topcat.hypermart.net</email></para> + </listitem> + + <listitem> <para>Michael Sardo <email>jaeger16@yahoo.com</email></para> </listitem> @@ -8375,11 +8385,6 @@ </listitem> <listitem> - <para>Po-Chien Lin - <email>linpc@cs.nctu.edu.tw</email></para> - </listitem> - - <listitem> <para>Pomegranate <email>daver@flag.blackened.net</email></para> </listitem> @@ -8787,6 +8792,11 @@ </listitem> <listitem> + <para>Dr. Rolf Jansen + <email>cyclaero@gmail.com</email></para> + </listitem> + + <listitem> <para>Roman Neuhauser <email>neuhauser@chello.cz</email></para> </listitem> Modified: projects/ISBN_1-57176-407-0/en_US.ISO8859-1/articles/contributors/contrib.committers.xml ============================================================================== --- projects/ISBN_1-57176-407-0/en_US.ISO8859-1/articles/contributors/contrib.committers.xml Fri Feb 15 22:47:36 2013 (r40976) +++ projects/ISBN_1-57176-407-0/en_US.ISO8859-1/articles/contributors/contrib.committers.xml Sat Feb 16 04:50:40 2013 (r40977) @@ -700,6 +700,10 @@ </listitem> <listitem> + <para>&a.koobs;</para> + </listitem> + + <listitem> <para>&a.jkois;</para> </listitem> @@ -784,6 +788,10 @@ </listitem> <listitem> + <para>&a.dru;</para> + </listitem> + + <listitem> <para>&a.jlh;</para> </listitem> @@ -812,6 +820,10 @@ </listitem> <listitem> + <para>&a.achim;</para> + </listitem> + + <listitem> <para>&a.truckman;</para> </listitem> @@ -832,6 +844,10 @@ </listitem> <listitem> + <para>&a.pclin;</para> + </listitem> + + <listitem> <para>&a.yzlin;</para> </listitem> Modified: projects/ISBN_1-57176-407-0/en_US.ISO8859-1/articles/portbuild/article.xml ============================================================================== --- projects/ISBN_1-57176-407-0/en_US.ISO8859-1/articles/portbuild/article.xml Fri Feb 15 22:47:36 2013 (r40976) +++ projects/ISBN_1-57176-407-0/en_US.ISO8859-1/articles/portbuild/article.xml Sat Feb 16 04:50:40 2013 (r40977) @@ -63,7 +63,7 @@ <title>The codebase</title> <para>Most of the package building magic occurs under the - <filename>/var/portbuild</filename> directory. Unless + <filename>/a/portbuild</filename> directory. Unless otherwise specified, all paths will be relative to this location. <replaceable>${arch}</replaceable> will be used to specify one of the package architectures @@ -77,15 +77,16 @@ </para> <note> - <para>Packages are no longer built for branches 4, 5, or 6, nor + <para>FreeBSD no longer builds packages for branches 4, 5, or 6, nor for the alpha architecture.</para> </note> - <para>The scripts that control all of this live in - <filename role="directory">/var/portbuild/scripts/</filename>. + <para>The scripts that control all of this live in either + <filename role="directory">/a/portbuild/scripts/</filename> or. + <filename role="directory">/a/portbuild/admin/scripts/</filename>. These are the checked-out copies from the Subversion repository at - <ulink url="http://svnweb.freebsd.org/base/projects/portbuild/scripts/">; - <filename role="directory">base/projects/portbuild/scripts/</filename> + <ulink url="http://svnweb.freebsd.org/base/projects/portbuild/">; + <filename role="directory">base/projects/portbuild/</filename> </ulink>.</para> <para>Typically, incremental builds are done that use previous @@ -112,7 +113,7 @@ </sect2> <sect2 id="codebase-notes"> - <title>Notes on the codebase</title> + <title>Historical notes on the codebase</title> <para>Until mid-2010, the scripts were completely specific to <hostid>pointyhat.FreeBSD.org</hostid> as the head (dispatch) node. During @@ -148,18 +149,6 @@ </listitem> </itemizedlist> - <para>This document was originally written before these changes - were made. Where things such as script invocations have changed, - they were denoted as <literal>new codebase:</literal> as opposed - to <literal>old codebase:</literal>.</para> - - <note> - <para>Up until November 2012, <hostid>pointyhat</hostid> had still - been running the old codebase. That installation has now been - permanently offlined. Therefore, all the instructions having - to do with the old codebase have been removed.</para> - </note> - <note> <para>Also during this process, the codebase was migrated to the <ulink url="http://svnweb.freebsd.org/base/projects/portbuild/scripts/">; @@ -169,16 +158,61 @@ found in CVS</ulink>.</para> </note> </sect2> + + <sect2 id="pointyhat-privsep"> + <title>Notes on privilege separation</title> + + <para>As of January 2013, a rewrite is in progress to further separate + privileges. The following concepts are introduced:</para> + + <itemizedlist> + <listitem> + <para>Server-side user <username>portbuild</username> assumes all + responsiblity for operations involving builds and communicating + with the clients. This user no longer has access to + <application>sudo</application>.</para> + </listitem> + + <listitem> + <para>Server-side user <username>srcbuild</username> is created + and given responsiblity for operations involving both VCS + operations and anything involving src builds for the clients. + This user does not have access to + <application>sudo</application>.</para> + </listitem> + + <listitem> + <para>The server-side + <literal>ports-</literal><replaceable>arch</replaceable> + users go away.</para> + </listitem> + + <listitem> + <para>None of the above server-side users have + <application>ssh</application> keys. Individual + <literal>portmgr</literal> will accomplish all those + tasks using <application>ksu</application>. (This is + still work-in-progress.)</para> + </listitem> + + <listitem> + <para>The only client-side user is also named + <username>portbuild</username> and still has access to + <application>sudo</application> for the purpose of managing + jails.</para> + </listitem> + </itemizedlist> + </sect2> </sect1> <sect1 id="management"> <title>Build Client Management</title> - <para>The &i386; clients co-located with <hostid>pointyhat</hostid> - netboot from it (<replaceable>connected</replaceable> nodes); all - other clients (<replaceable>disconnected</replaceable> nodes) - are either self-hosted or netboot from some other - <literal>pxe</literal> host. + <para>You may set up clients to either netboot from the master + (<replaceable>connected</replaceable> nodes) + or have them either self-hosted or netboot from some other + <literal>pxe</literal> host + (<replaceable>disconnected</replaceable> nodes). In all cases they set themselves up at boot-time to prepare to build packages.</para> @@ -189,7 +223,7 @@ nullfs-mounted for jail builds.</para> <para>The - <username>ports-<replaceable>${arch}</replaceable></username> + <username>portbuild</username> user can &man.ssh.1; to the client nodes to monitor them. Use <command>sudo</command> and check the <hostid>portbuild.<replaceable>hostname</replaceable>.conf</hostid> @@ -203,29 +237,31 @@ <sect1 id="setup"> <title>Jail Build Environment Setup</title> - <para>Package builds are performed in a + <para>Package builds are performed by the clients in a <literal>jail</literal> populated by the <filename>portbuild</filename> script using the <filename><replaceable>${arch}</replaceable>/<replaceable>${branch}</replaceable>/builds/<replaceable>${buildid}</replaceable>/bindist.tar</filename> file.</para> - <para>The <command>makeworld</command> command builds a world from the + <para>On the server, use the + <command>makeworld</command> command to build a world from the <filename><replaceable>${arch}</replaceable>/<replaceable>${branch}</replaceable>/builds/<replaceable>${buildid}</replaceable>/src/</filename> - tree and installs it into + tree and install it into <filename><replaceable>${arch}</replaceable>/<replaceable>${branch}</replaceable>/builds/<replaceable>${buildid}</replaceable>/bindist.tar</filename>. The tree will be updated first unless <literal>-novcs</literal> is - specified. It should be run as <username>root</username>:</para> + specified.</para> - <screen>&prompt.root; <userinput>/var/portbuild/scripts/makeworld <replaceable>${arch}</replaceable> <replaceable>${branch}</replaceable> <replaceable>${buildid}</replaceable> [-novcs]</userinput></screen> + <screen>&prompt.root; <userinput>/a/portbuild/admin/scripts/makeworld <replaceable>${arch}</replaceable> <replaceable>${branch}</replaceable> <replaceable>${buildid}</replaceable> [-novcs]</userinput></screen> - <para>The <filename>bindist.tar</filename> tarball is created from the + <para>Similiarly on the server, the + <filename>bindist.tar</filename> tarball is created from the previously installed world by the <command>mkbindist</command> - script. It should be also be run as <username>root</username>:</para> + script.</para> - <screen>&prompt.root; <userinput>/var/portbuild/scripts/mkbindist <replaceable>${arch}</replaceable> <replaceable>${branch}</replaceable> <replaceable>${buildid}</replaceable></userinput></screen> + <screen>&prompt.root; <userinput>/a/portbuild/admin/scripts/mkbindist <replaceable>${arch}</replaceable> <replaceable>${branch}</replaceable> <replaceable>${buildid}</replaceable></userinput></screen> - <para>The per-machine tarballs are located in + <para>The per-machine tarballs are located on the server in <filename><replaceable>${arch}</replaceable>/clients</filename>.</para> <para>The <filename>bindist.tar</filename> file is extracted @@ -236,6 +272,16 @@ <para>For both commands above, if <replaceable>${buildid}</replaceable> is <literal>latest</literal>, it may be omitted.</para> + + <note> + <para>Currently the above two scripts must be run as + <username>root</username>; otherwise, the install scripts + lack sufficient permissions. This is undesirable for + security reasons. Work is in progress in -HEAD to allow + users to do installations; once that is committed, the + intention is to use that and run these two commands as + <username>srcbuild</username>.</para> + </note> </sect1> <sect1 id="customizing"> @@ -548,7 +594,7 @@ PKG_BIN=/usr/local/sbin/pkg</programlist </note> <para>Make sure the <replaceable>${arch}</replaceable> build - is run as the ports-<replaceable>${arch}</replaceable> user + is run as the <username>portbuild</username> user or it will complain loudly.</para> <note> @@ -785,7 +831,7 @@ PKG_BIN=/usr/local/sbin/pkg</programlist <para>To free up resources, you will need to clean up client machines by running <command>build cleanup</command> command. For example:</para> - <screen>&prompt.user; <userinput>/var/portbuild/scripts/build cleanup i386 8-exp 20080714120411 -full</userinput></screen> + <screen>&prompt.user; <userinput>/a/portbuild/scripts/build cleanup i386 8-exp 20080714120411 -full</userinput></screen> <para>If you forget to do this, then the old build <literal>jail</literal>s will not be cleaned up for 24 hours, and no @@ -800,6 +846,11 @@ PKG_BIN=/usr/local/sbin/pkg</programlist and it is less than the number of jobs that <literal>loads</literal> thinks are in use, you are in trouble.</para> + <note> + <para>The following notes about mounting only apply to + <literal>connected</literal> nodes.</para> + </note> + <para>You may have problem with the <command>umount</command> commands hanging. If so, you are going to have to use the <command>allgohans</command> script to run an &man.ssh.1; @@ -829,6 +880,11 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 <para>You may also see messages about <literal>procfs</literal>.</para> </note> + <note> + <para>The above is the end of the notes that apply only to + <literal>connected</literal> nodes.</para> + </note> + <para>After you have done all the above, remove the <filename><replaceable>${arch}</replaceable>/lock</filename> file before trying to restart the build. If you do not, @@ -875,7 +931,7 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 <userinput>scripts/stats <replaceable>${branch}</replaceable></userinput> command shows the number of packages already built.</para> - <para>Running <userinput>cat /var/portbuild/*/loads/*</userinput> + <para>Running <userinput>cat /a/portbuild/*/loads/*</userinput> shows the client loads and number of concurrent builds in progress. The files that have been recently updated are the clients that are online; the others are the offline clients.</para> @@ -912,7 +968,7 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 it if not.</para> <para>Keep an eye on &man.df.1; output. If the - <filename>/var/portbuild</filename> file system becomes full + <filename>/a/portbuild</filename> file system becomes full then <trademark>Bad Things</trademark> happen.</para> <para>The status of all current builds is generated periodically @@ -1017,12 +1073,7 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 <screen>&prompt.user; <userinput>cd <replaceable>${arch}</replaceable>/<replaceable>${branch}</replaceable></userinput> &prompt.user; <userinput>find distfiles > distfiles-<replaceable>${release}</replaceable></userinput></screen> - <!-- XXX MCL apparently obsolete --> - <para>This inventory file typically lives in - <filename>i386/<replaceable>${branch}</replaceable></filename> - on the cluster master.</para> - - <para>This is useful to aid in periodically cleaning out + <para>You should use that output to periodically clean out the distfiles from <hostid>ftp-master</hostid>. When space gets tight, distfiles from recent releases can be kept while others can be thrown away.</para> @@ -1046,6 +1097,16 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 <sect1 id="uploading"> <title>Uploading Packages</title> + <note> + <para>For FreeBSD.org as of 2013, the instructions + about uploading to <hostid>ftp-master</hostid> are obsolete. + In the future, <hostid>ftp-master</hostid> will pull + from <hostid>pointyhat</hostid>, using a mechanism yet + to be implemented. However, the instructions about + <makevar>RESTRICTED</makevar> and <makevar>NO_CDROM</makevar> + must still be <emphasis>carefully</emphasis> followed.</para> + </note> + <para>Once a build has completed, packages and/or distfiles can be transferred to <hostid>ftp-master</hostid> for propagation to the FTP mirror network. If the build was @@ -1123,7 +1184,7 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 a new release), copy packages to the staging area on <hostid>ftp-master</hostid> with something like the following:</para> - <screen>&prompt.root; <userinput>cd /var/portbuild/<replaceable>${arch}</replaceable>/<replaceable>${branch}</replaceable></userinput> + <screen>&prompt.root; <userinput>cd /a/portbuild/<replaceable>${arch}</replaceable>/<replaceable>${branch}</replaceable></userinput> &prompt.root; <userinput>tar cfv - packages/ | ssh portmgr@ftp-master tar xfC - w/ports/<replaceable>${arch}</replaceable>/tmp/<replaceable>${subdir}</replaceable></userinput></screen> <para>Then log into <hostid>ftp-master</hostid>, verify that @@ -1151,7 +1212,7 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 <para>Distfiles should be transferred with the <command>cpdistfiles</command> script:</para> - <screen>&prompt.root; <userinput>/var/portbuild/scripts/cpdistfiles <replaceable>${arch}</replaceable> <replaceable>${branch}</replaceable> <replaceable>${buildid}</replaceable> [-yesreally] | tee log2</userinput></screen> + <screen>&prompt.root; <userinput>/a/portbuild/scripts/cpdistfiles <replaceable>${arch}</replaceable> <replaceable>${branch}</replaceable> <replaceable>${buildid}</replaceable> [-yesreally] | tee log2</userinput></screen> <para>Doing it by hand is deprecated.</para> </sect1> @@ -1159,6 +1220,11 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 <sect1 id="expbuilds"> <title>Experimental Patches Builds</title> + <note> + <para>Most of the information in this section is obsolete + as of 2013 and needs to be rewritten.</para> + </note> + <para>Experimental patches builds are run from time to time to new features or bugfixes to the ports infrastructure (i.e. <filename>bsd.port.mk</filename>), or to test large sweeping @@ -1209,21 +1275,15 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 build. This will ensure an apples-to-apples comparison later.</para> - <!-- XXX MCL currently there is only one build cluster - <note><para>One build cluster can do the control build while the other - does the experimental patches build. This can be a great - time-saver.</para></note> - --> - <para>Once the build finishes, compare the control build failures to those of the experimental patches build. Use the following commands to facilitate this (this assumes the <literal>8</literal> branch is the control branch, and the <literal>8-exp</literal> branch is the experimental patches branch):</para> - <screen>&prompt.user; <userinput>cd /var/portbuild/i386/8-exp/errors</userinput> + <screen>&prompt.user; <userinput>cd /a/portbuild/i386/8-exp/errors</userinput> &prompt.user; <userinput>find . -name \*.log\* | sort > /tmp/8-exp-errs</userinput> -&prompt.user; <userinput>cd /var/portbuild/i386/8/errors</userinput> +&prompt.user; <userinput>cd /a/portbuild/i386/8/errors</userinput> &prompt.user; <userinput>find . -name \*.log\* | sort > /tmp/8-errs</userinput></screen> <note> @@ -1286,7 +1346,7 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 rebuild of the affected packages under the control branch:</para> - <screen>&prompt.user; <userinput>cd /var/portbuild/i386/8/ports</userinput></screen> + <screen>&prompt.user; <userinput>cd /a/portbuild/i386/8/ports</userinput></screen> <note> <para>The following example is obsolete</para> @@ -1299,9 +1359,9 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 <!-- XXX MCL fix --> <para>The following command will set up the control branch for - the partial build (old codebase):</para> + the partial build:</para> - <screen>&prompt.user; <userinput>/var/portbuild/scripts/dopackages.8 -noportsvcs -nobuild -novcs -nofinish</userinput></screen> + <screen>&prompt.user; <userinput>/a/portbuild/scripts/dopackages.wrapper i386 8 -noportsvcs -nobuild -novcs -nofinish</userinput></screen> <!-- XXX MCL obsolete --> <para>The builds must be performed from the @@ -1309,14 +1369,14 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 initially be empty except for the Makefile symlink. If this symlink does not exist, it must be created:</para> - <screen>&prompt.user; <userinput>cd /var/portbuild/i386/8/packages/All</userinput> + <screen>&prompt.user; <userinput>cd /a/portbuild/i386/8/packages/All</userinput> &prompt.user; <userinput>ln -sf ../../Makefile .</userinput> &prompt.user; <userinput>make -k -j<#> <list of packages to build></userinput></screen> <note> <para><#> is the concurrency of the build to attempt. It is usually the sum of the weights listed in - <filename>/var/portbuild/i386/mlist</filename> unless you have a + <filename>/a/portbuild/i386/mlist</filename> unless you have a reason to run a heavier or lighter build.</para> <para>The list of packages to build should be a list of package @@ -1349,17 +1409,15 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 <para>Before following these steps, please coordinate with <literal>portmgr</literal>.</para> - <note> - <para>Due to some generous donations, <literal>portmgr</literal> is - no longer looking for the loan of &i386; or <literal>amd64</literal> - systems. However, we are still interested in borrowing tier-2 - systems.</para> - </note> - <sect2 id="node-requirements"> <title>Node requirements</title> - <para><literal>portmgr</literal> is still working on characterizing + <note> + <para>This section is only of interest when considering + tier-2 architectures.</para> + </note> + + <para>Here are the requirement for what a node needs to be generally useful.</para> <itemizedlist> @@ -1439,7 +1497,8 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 <filename>/usr2/</filename>.)</para> <note> <para>The filename <filename>chroot</filename> is a - historical remnant.</para> + historical remnant. The <command>chroot</command> + command is no longer used.</para> </note> </step> </procedure> @@ -1480,8 +1539,9 @@ umount: Cleanup of /x/tmp/8-exp/chroot/5 <step> <para>Generate a kernel config file. Include - <filename>GENERIC</filename> (or, if you are using more than - 3.5G on &i386;, <filename>PAE</filename>).</para> + <filename>GENERIC</filename> (or, if on &i386, and + you are using more than + 3.5G, <filename>PAE</filename>).</para> <para>Required options:</para> @@ -1496,9 +1556,14 @@ options SHMMAXPGS=65536 options SEMMNI=40 options SEMMNS=240 options SEMUME=40 -options SEMMNU=120 +options SEMMNU=120</programlisting> + + <para>If you are interested in debugging general + problems, you may wish to use the following. + However, for unattended operations, it is best + to leave it out:</para> -options ALT_BREAK_TO_DEBUGGER</programlisting> + <programlisting>options ALT_BREAK_TO_DEBUGGER</programlisting> <para>For <filename>PAE</filename>, it is not currently possible to load modules. Therefore, if you are running an architecture @@ -1553,11 +1618,11 @@ options NFSSERVER <itemizedlist> <listitem> <para>Create a - <literal>ports-<replaceable>${arch}</replaceable></literal> + <literal>portbuild</literal> user and group. It can have the <literal>'*'</literal> password.</para> <para>Create - <filename>/home/ports-<replaceable>${arch}/.ssh/</replaceable></filename> + <filename>/home/portbuild/.ssh/</filename> and populate <filename>authorized_keys</filename>. </para> </listitem> @@ -1836,8 +1901,7 @@ touch /tmp/.boot_finished</programlistin <filename>usr/local/etc/sudoers/sudoers.d/portbuild</filename>:</para> <programlisting># local changes for package building -%wheel ALL=(ALL) ALL -ports-<replaceable>${arch}</replaceable> ALL=(ALL) NOPASSWD: ALL</programlisting> +portbuild ALL=(ALL) NOPASSWD: ALL</programlisting> </listitem> </itemizedlist> </step> @@ -1858,9 +1922,9 @@ ports-<replaceable>${arch}</replaceable> <para>As root:</para> <screen>&prompt.root; <userinput>mkdir portbuild</userinput> -&prompt.root; <userinput>chown ports-<replaceable>${arch}</replaceable>:ports-<replaceable>${arch}</replaceable> portbuild</userinput> +&prompt.root; <userinput>chown portbuild:portbuild portbuild</userinput> &prompt.root; <userinput>mkdir pkgbuild</userinput> -&prompt.root; <userinput>chown ports-<replaceable>${arch}</replaceable>:ports-<replaceable>${arch}</replaceable> pkgbuild</userinput> +&prompt.root; <userinput>chown portbuild:portbuild pkgbuild</userinput> &prompt.root; <userinput>mkdir squid</userinput> &prompt.root; <userinput>mkdir squid/cache</userinput> &prompt.root; <userinput>mkdir squid/logs</userinput> @@ -1894,7 +1958,7 @@ ports-<replaceable>${arch}</replaceable> <title>Configuration on the server</title> <para>These steps need to be taken by a <literal>portmgr</literal> - acting as <literal>ports-<replaceable>${arch}</replaceable></literal> + acting as <literal>portbuild</literal> on the server.</para> <procedure> @@ -1902,13 +1966,13 @@ ports-<replaceable>${arch}</replaceable> <para>If any of the default TCP ports is not available (see above), you will need to create an <command>ssh</command> tunnel for them and include its invocation command in - <literal>ports-<replaceable>${arch}</replaceable></literal>'s + <literal>portbuild</literal>'s <filename>crontab</filename>.</para> </step> <step> <para>Unless you can use the defaults, add an entry to - <filename>/home/ports-<replaceable>${arch}</replaceable>/.ssh/config</filename> + <filename>/home/portbuild/.ssh/config</filename> to specify the public IP address, TCP port for <command>ssh</command>, username, and any other necessary information.</para> @@ -1916,7 +1980,7 @@ ports-<replaceable>${arch}</replaceable> <step> <para>Create - <filename>/var/portbuild/<replaceable>${arch}</replaceable>/clients/bindist-<replaceable>${hostname}</replaceable>.tar</filename>.</para> + <filename>/a/portbuild/<replaceable>${arch}</replaceable>/clients/bindist-<replaceable>${hostname}</replaceable>.tar</filename>.</para> <itemizedlist> <listitem> @@ -1955,16 +2019,16 @@ MASTER_SITE_OVERRIDE= \ <para>Hint: you will need one of these for each machine; however, if you have multiple machines at one site, you should create a site-specific one (e.g., in - <filename>/var/portbuild/conf/clients/</filename>) + <filename>/a/portbuild/conf/clients/</filename>) and symlink to it.</para> </step> <step> <para>Create -<filename>/var/portbuild/<replaceable>${arch}</replaceable>/portbuild-<replaceable>${hostname}</replaceable></filename> +<filename>/a/portbuild/<replaceable>${arch}</replaceable>/portbuild-<replaceable>${hostname}</replaceable></filename> using one of the existing ones as a guide. This file contains overrides to -<filename>/var/portbuild/<replaceable>${arch}</replaceable>/portbuild.conf</filename>.</para> +<filename>/a/portbuild/<replaceable>${arch}</replaceable>/portbuild.conf</filename>.</para> <para>Suggested values:</para> @@ -1972,7 +2036,7 @@ MASTER_SITE_OVERRIDE= \ http_proxy="http://localhost:3128/" squid_dir=<filename>/<replaceable>usr2</replaceable>/squid</filename> scratchdir=<filename>/<replaceable>usr2</replaceable>/pkgbuild</filename> -client_user=ports-<replaceable>${arch}</replaceable> +client_user=portbuild sudo_cmd="sudo -H" rsync_gzip=-z @@ -2014,7 +2078,7 @@ ssh_cmd="/usr/local/bin/ssh"</programlis <title>Enabling the node</title> <para>These steps need to be taken by a <literal>portmgr</literal> - acting as <literal>ports-<replaceable>arch</replaceable></literal>:</para> + acting as <literal>portbuild</literal>:</para> <procedure> <step> @@ -2030,7 +2094,7 @@ ssh_cmd="/usr/local/bin/ssh"</programlis <step> <para>Populate the client's copy of <filename>/var/portbuild/scripts/</filename> by something like - <userinput>/var/portbuild/scripts/dosetupnode <replaceable>arch</replaceable> <replaceable>major</replaceable> latest <replaceable>hostname</replaceable></userinput>. + <userinput>/a/portbuild/scripts/dosetupnode <replaceable>arch</replaceable> <replaceable>major</replaceable> latest <replaceable>hostname</replaceable></userinput>. Verify that you now have files in that directory.</para> </step> @@ -2068,14 +2132,14 @@ ssh_cmd="/usr/local/bin/ssh"</programlis </procedure> <para>Finally, again as <literal>portmgr</literal> - acting as <literal>ports-<replaceable>arch</replaceable></literal>:</para> + acting as <literal>portbuild</literal>:</para> <procedure> <step> <para>Once you are sure that the client is working, tell <application>pollmachine</application> about it by adding it to - <filename>/var/portbuild/<replaceable>${arch}</replaceable>/mlist</filename>.</para> + <filename>/a/portbuild/<replaceable>${arch}</replaceable>/mlist</filename>.</para> </step> </procedure> </sect2> @@ -2091,10 +2155,15 @@ ssh_cmd="/usr/local/bin/ssh"</programlis be done to specify that the previous branch is no longer equivalent to <literal>HEAD</literal>.</para> + <note> + <para>As + <literal>srcbuild</literal>:</para> + </note> + <itemizedlist> <listitem> <para> - Edit <filename>/var/portbuild/conf/server.conf</filename> + Edit <filename>/a/portbuild/conf/admin/admin.conf</filename> with the following changes:</para> <itemizedlist> @@ -2119,7 +2188,7 @@ ssh_cmd="/usr/local/bin/ssh"</programlis </listitem> <listitem> - <para>Run <command>/var/portbuild/updatesnap</command> manually.</para> + <para>Run <command>/a/portbuild/admin/scripts/updatesnap</command> manually.</para> </listitem> </itemizedlist> </sect2> @@ -2127,24 +2196,21 @@ ssh_cmd="/usr/local/bin/ssh"</programlis <sect2 id="new-branch-post-qmanager"> <title>Steps necessary after <application>qmanager</application> is started</title> - <note> - <para>Again, as - <literal>ports-<replaceable>arch</replaceable></literal>:</para> - </note> - <itemizedlist> <listitem> <para>For each branch that will be supported, do the following:</para> <itemizedlist> <listitem> - <para>Kick-start the build for the branch with:</para> + <para>As <literal>portbuild</literal>, + kick-start the build for the branch with:</para> <screen>build create <replaceable>arch</replaceable> <replaceable>branch</replaceable></screen> </listitem> <listitem> - <para><link linkend="setup">Create + <para>As <literal>srcbuild</literal>, + <link linkend="setup">create <filename>bindist.tar</filename></link>.</para> </listitem> </itemizedlist> @@ -2161,7 +2227,7 @@ ssh_cmd="/usr/local/bin/ssh"</programlis <itemizedlist> <listitem> - <para>Edit <filename>/var/portbuild/conf/server.conf</filename> + <para>Edit <filename>/a/portbuild/admin/conf/admin.conf</filename> with the following changes:</para> <itemizedlist> @@ -2188,7 +2254,7 @@ zfs destroy -r a/snap/src-<replaceable>o <itemizedlist> <listitem> <para>You will probably find that the following files and - symlinks in <filename>/var/portbuild/errorlogs/</filename> + symlinks in <filename>/a/portbuild/errorlogs/</filename> can be removed:</para> <itemizedlist> @@ -2233,7 +2299,7 @@ zfs destroy -r a/snap/src-<replaceable>o <itemizedlist> <listitem> - <para>Edit <filename>/var/portbuild/conf/server.conf</filename> + <para>Edit <filename>/a/portbuild/admin/conf/admin.conf</filename> with the following changes:</para> <itemizedlist> @@ -2248,7 +2314,7 @@ zfs destroy -r a/snap/src-<replaceable>o </listitem> <listitem> - <para>Run <command>/var/portbuild/updatesnap</command> manually.</para> + <para>Run <command>/a/portbuild/admin/scripts/updatesnap</command> manually.</para> </listitem> <listitem> @@ -2272,49 +2338,24 @@ zfs destroy -r a/snap/src-<replaceable>o <title>Steps necessary before <application>qmanager</application> is started</title> <note> - <para>The initial steps need to be done using - <application>sudo</application>.</para> + <para>The next steps are most easily done as user + <literal>portbuild</literal>.</para> </note> - <itemizedlist> - <listitem> - <para>Create a new - <literal>ports-<replaceable>arch</replaceable></literal> - user and group.</para> - </listitem> - - <listitem> - <screen>mkdir /var/portbuild/<replaceable>arch</replaceable></screen> - </listitem> - - <listitem> - <para>Create a new <application>zfs</application> filesystem:</para> - - <screen>&prompt.root; zfs create -o mountpoint=/a/portbuild/<replaceable>arch</replaceable> a/portbuild/<replaceable>arch</replaceable></screen> - </listitem> - - <listitem> - <screen>&prompt.root; chown ports-<replaceable>arch</replaceable>:portbuild /var/portbuild/<replaceable>arch</replaceable>; -&prompt.root; chmod 775 /var/portbuild/<replaceable>arch</replaceable>; -&prompt.root; cd /var/portbuild/<replaceable>arch</replaceable></screen> - </listitem> - - <listitem> - <para>Create the <filename>.ssh</filename> directory.</para> - </listitem> - </itemizedlist> - <note> - <para>The next steps are most easily done as user - <literal>ports-<replaceable>arch</replaceable></literal>.</para> + <para>The following assumes you have already run + <literal>mkportbuild</literal>.</para> </note> <itemizedlist> <listitem> - <para>Create an archive directory for buildlogs and errorlogs - under <filename>archive/</filename>.</para> + <para>As the <literal>portbuild</literal> user, run</para> + + <screen>&prompt.user; /a/portbuild/admin/tools/addarch <replaceable>arch</replaceable></screen> </listitem> + </itemizedlist> + <itemizedlist> <listitem> <para>For each branch that will be supported, do the following:</para> @@ -2358,8 +2399,10 @@ zfs destroy -r a/snap/src-<replaceable>o </listitem> <listitem> - <para>Create a fresh <filename>portbuild.conf</filename> file - from one of the ones for another architecture.</para> + <para>Edit <filename>portbuild.conf</filename> + from one of the ones for another architecture. + <literal>addarch</literal> will have created a default + one for you.</para> </listitem> <listitem> @@ -2369,23 +2412,13 @@ zfs destroy -r a/snap/src-<replaceable>o </listitem> <listitem> - <screen>&prompt.root; cd .ssh && ssh-keygen</screen> - </listitem> - - <listitem> - <para>If desired, - edit the <filename>.ssh/config</filename> file for - convenience in using <application>ssh</application>.</para> - </listitem> - - <listitem> <para>If you need to create any tunnels:</para> <procedure> <step> <para>Make a private configuration directory:</para> - <screen>&prompt.root; mkdir /var/portbuild/conf/<replaceable>arch</replaceable></screen> + <screen>&prompt.root; mkdir /a/portbuild/conf/<replaceable>arch</replaceable></screen> </step> <step> @@ -2397,30 +2430,26 @@ zfs destroy -r a/snap/src-<replaceable>o </itemizedlist> <note> - <para>Once again using <application>sudo</application>:</para> + <para>As <literal>srcbuild</literal>:</para> </note> <itemizedlist> <listitem> <para>Add <replaceable>arch</replaceable> to <makevar>SUPPORTED_ARCHS</makevar> in - <filename>/var/portbuild/conf/server.conf</filename>.</para> + <filename>/a/portbuild/admin/conf/admin.conf</filename>.</para> </listitem> <listitem> <para>Add the <replaceable>arch</replaceable> directory to - <filename>/var/portbuild/scripts/zbackup</filename> and - <filename>/var/portbuild/scripts/zexpire</filename>.</para> + <filename>/a/portbuild/admin/scripts/zbackup</filename>. + (This is a hack and should go away.)</para> </listitem> </itemizedlist> - <note> - <para>One last time using <application>sudo</application>:</para> - </note> - <itemizedlist> <listitem> - <para>Add an appropriate <replaceable>arch</replaceable> entry for - <filename>/var/portbuild/scripts/dologs</filename> to the root + <para>Enable the appropriate <replaceable>arch</replaceable> entry for + <filename>/a/portbuild/scripts/dologs</filename> to the portbuild <filename>crontab</filename>. (This is a hack and should go away.)</para> </listitem> *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201302160450.r1G4oeVW041722>