From owner-p4-projects@FreeBSD.ORG Wed Jun 17 18:23:06 2009 Return-Path: Delivered-To: p4-projects@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 32767) id CD436106567D; Wed, 17 Jun 2009 18:23:05 +0000 (UTC) Delivered-To: perforce@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8739A1065676 for ; Wed, 17 Jun 2009 18:23:05 +0000 (UTC) (envelope-from rene@FreeBSD.org) Received: from repoman.freebsd.org (repoman.freebsd.org [IPv6:2001:4f8:fff6::29]) by mx1.freebsd.org (Postfix) with ESMTP id 736FE8FC24 for ; Wed, 17 Jun 2009 18:23:05 +0000 (UTC) (envelope-from rene@FreeBSD.org) Received: from repoman.freebsd.org (localhost [127.0.0.1]) by repoman.freebsd.org (8.14.3/8.14.3) with ESMTP id n5HIN5xt067678 for ; Wed, 17 Jun 2009 18:23:05 GMT (envelope-from rene@FreeBSD.org) Received: (from perforce@localhost) by repoman.freebsd.org (8.14.3/8.14.3/Submit) id n5HIN5xq067665 for perforce@freebsd.org; Wed, 17 Jun 2009 18:23:05 GMT (envelope-from rene@FreeBSD.org) Date: Wed, 17 Jun 2009 18:23:05 GMT Message-Id: <200906171823.n5HIN5xq067665@repoman.freebsd.org> X-Authentication-Warning: repoman.freebsd.org: perforce set sender to rene@FreeBSD.org using -f From: Rene Ladan To: Perforce Change Reviews Cc: Subject: PERFORCE change 164604 for review X-BeenThere: p4-projects@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: p4 projects tree changes List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jun 2009 18:23:07 -0000 http://perforce.freebsd.org/chv.cgi?CH=164604 Change 164604 by rene@rene_self on 2009/06/17 18:22:19 IFC Affected files ... .. //depot/projects/docproj_nl/en_US.ISO8859-1/articles/portbuild/article.sgml#11 integrate .. //depot/projects/docproj_nl/en_US.ISO8859-1/books/handbook/multimedia/chapter.sgml#4 integrate .. //depot/projects/docproj_nl/share/sgml/man-refs.ent#19 integrate .. //depot/projects/docproj_nl/www/en/gnome/docs/faq2.sgml#5 integrate .. //depot/projects/docproj_nl/www/share/sgml/events.xml#19 integrate Differences ... ==== //depot/projects/docproj_nl/en_US.ISO8859-1/articles/portbuild/article.sgml#11 (text+ko) ==== @@ -11,7 +11,7 @@ The &os; Ports Management Team - $FreeBSD: doc/en_US.ISO8859-1/articles/portbuild/article.sgml,v 1.37 2009/05/14 15:33:07 erwin Exp $ + $FreeBSD: doc/en_US.ISO8859-1/articles/portbuild/article.sgml,v 1.39 2009/06/14 06:02:09 linimon Exp $ 2003 @@ -20,6 +20,7 @@ 2006 2007 2008 + 2009 The &os; Ports Management Team @@ -69,7 +70,8 @@ right after release time, for the -STABLE branches - every month or so, for -CURRENT + periodically to test changes to + -CURRENT for experimental builds @@ -79,16 +81,15 @@ Build Client Management - The &i386; clients currently - netboot from pointyhat; the other clients - are self-hosted. In all cases they set themselves + The &i386; clients co-located with pointyhat + netboot from it (connected nodes); all + other clients (disconnected nodes) + are either self-hosted or netboot from some other + pxe host. + In all cases they set themselves up at boot-time to prepare to build packages. - Although connected nodes are supported, - disconnected cluster node support has - been added. A disconnected node is - one that does not mount the cluster master via NFS. It could be - a remote node, for example. The cluster master rsync's the + The cluster master rsyncs the interesting data (ports and src trees, bindist tarballs, scripts, etc.) to disconnected nodes during the node-setup phase. Then, the disconnected portbuild directory is @@ -140,6 +141,7 @@ /var/portbuild&prompt.root; scripts/mkbindist ${arch} ${branch} ${buildid} + The per-machine tarballs are located in ${arch}/clients. @@ -453,6 +455,9 @@ dopackages.7 sparc64 -finish + + Hint: it us usually best to run the dopackages + command inside of screen(1). @@ -583,6 +588,7 @@ [+] If any of these steps fail, the build will stop cold in its tracks. + [++] Status of these steps can be found in ${arch}/${branch}/make.[0|1], where make.0 is the log file used by @@ -676,7 +682,8 @@ dopackages will simply exit. - If you have to do a cvs update before + + If you have to do a ports tree update before restarting, you may have to rebuild either duds, INDEX, or both. If you are doing the latter manually, you will also have to rebuild @@ -839,6 +846,7 @@ &prompt.user; cd ${arch}/${branch} &prompt.user; find distfiles > distfiles-${release} + This inventory file typically lives in i386/${branch} on the cluster master. @@ -901,7 +909,7 @@ - packages-6.3-release + packages-6.4-release @@ -909,7 +917,7 @@ - packages-7.0-release + packages-7.2-release @@ -990,9 +998,9 @@ Experimental patches builds are run from time to time to new features or bugfixes to the ports infrastructure (i.e. - bsd.port.mk), or to test large sweeping + bsd.port.mk), or to test large sweeping upgrades. The current experimental patches branch is - 7-exp on the &i386; + 7-exp on the amd64 architecture. In general, an experimental patches build is run the same @@ -1026,33 +1034,35 @@ In order to have a good control case with which to compare failures, you should first do a package build of the branch on which the experimental patches branch is based for the &i386; - architecture (currently this is 6). Then, when + architecture (currently this is 7). Then, when preparing for the experimental patches build, checkout a ports tree and a src tree with the same date as was used for the control build. This will ensure an apples-to-apples comparison later. + Once the build finishes, compare the control build failures to those of the experimental patches build. Use the following - commands to facilitate this (this assumes the 6 - branch is the control branch, and the 6-exp + commands to facilitate this (this assumes the 7 + branch is the control branch, and the 7-exp branch is the experimental patches branch): - &prompt.user; cd /var/portbuild/i386/6-exp/errors -&prompt.user; find . -name \*.log\* | sort > /tmp/6-exp-errs -&prompt.user; cd /var/portbuild/i386/6/errors -&prompt.user; find . -name \*.log\* | sort > /tmp/6-errs + &prompt.user; cd /var/portbuild/i386/7-exp/errors +&prompt.user; find . -name \*.log\* | sort > /tmp/7-exp-errs +&prompt.user; cd /var/portbuild/i386/7/errors +&prompt.user; find . -name \*.log\* | sort > /tmp/7-errs If it has been a long time since one of the builds finished, the logs may have been automatically compressed with bzip2. In that case, you must use sort | sed 's,\.bz2,,g' instead. - &prompt.user; comm -3 /tmp/6-errs /tmp/6-exp-errs | less + &prompt.user; comm -3 /tmp/7-errs /tmp/7-exp-errs | less This last command will produce a two-column report. The first column is ports that failed on the control build but not in @@ -1107,7 +1117,7 @@ To differentiate between [1] and [2] above, you can do a rebuild of the affected packages under the control branch: - &prompt.user; cd /var/portbuild/i386/6/ports + &prompt.user; cd /var/portbuild/i386/7/ports Be sure to cvs update this tree to the same date as the experimental patches tree. @@ -1115,14 +1125,14 @@ The following command will set up the control branch for the partial build: - &prompt.user; /var/portbuild/scripts/dopackages.6 -noportscvs -nobuild -nocvs -nofinish + &prompt.user; /var/portbuild/scripts/dopackages.7 -noportscvs -nobuild -nocvs -nofinish The builds must be performed from the packages/All directory. This directory should initially be empty except for the Makefile symlink. If this symlink does not exist, it must be created: - &prompt.user; cd /var/portbuild/i386/6/packages/All + &prompt.user; cd /var/portbuild/i386/7/packages/All &prompt.user; ln -sf ../../Makefile . &prompt.user; make -k -j<#> <list of packages to build> @@ -1154,6 +1164,601 @@ /usr/ports/CHANGES. + + How to configure a new package building node + + Before following these steps, please coordinate with + portmgr. + + + + Node requirements + + portmgr is still working on characterizing + what a node needs to be generally useful. + + + + CPU capacity: TBA. However, we have several dual-CPU + P-III &i386; 1.0GHz machines available, so anything with + less horsepower than that is not as likely to be useful. + (However, many of our &sparc64;s are single-CPU, 500MHz + machines, so our requirements are lower.) + + + We are able to adjust the number of jobs + dispatched to each machine, and we generally tune + the number to use 100% of CPU. + + + + + RAM: TBA. Again, we have been tuning to one job + per 512M of RAM. (Anything less than 512M is very + unlikely to be useful.) + + + + disk: at least 20G is needed for filesystem; 32G is + needed for swap. Best performance will be if multiple + disks are used, and configured as geom + stripes. Performance numbers are also TBA. + + + Package building will test disk drives to destruction. + Be aware of what you are signing up for! + + + + + network bandwidth: TBA. However, an 8-job machine + has been shown to saturate a cable modem line. + + + + + + + Preparation + + + + Pick a unique hostname. It does not have to be + a publicly resolvable hostname (it can be a name on + your internal network). + + + + By default, package building requires the following TCP + ports to be accessible: 22 (ssh), 414 + (infoseek), and 8649 + (ganglia). If these are not accessible, + pick others and ensure that an ssh tunnel + is set up (see below). + + (Note: if you have more than one machine at your site, + you will need an individual TCP port for each service on + each machine, and thus ssh tunnels + will be necessary. As such, you will probably need to + configure port forwarding on your firewall.) + + + + Decide if you will be booting natively or via + pxeboot. You will find that it is + easier to keep up with changes to -current + with the latter, especially if you have multiple machines + at your site. + + + + Pick a directory to hold ports configuration and + chroot subdirectories. It may be + best to put it this on its own partition. (Example: + /usr2/.) + + + + + + + Configuring <literal>src</literal> + + + + Create a directory to contain the latest + -current source tree and check it + out. (Since your machine will likely be asked to build + packages for -current, the kernel it + runs should be reasonably up-to-date with the + bindist that will be exported + by our scripts.) + + + + If you are using pxeboot: create a + directory to contain the install bits. You will probably + want to use a subdirectory of /pxeroot, + e.g., + /pxeroot/${arch}-${branch}. + Export that as DESTDIR. + + + + If you are cross-building, export + TARGET_ARCH=${arch}. + + + The procedure for cross-building ports is not yet + defined. + + + + + Generate a kernel config file. Include + GENERIC (or, if you are using more than + 3.5G on &i386;, PAE). + Suggested options: +options GEOM_CONCAT +options GEOM_STRIPE +options NULLFS +options TMPFS +options SHMMAXPGS=65536 +options SEMMNI=40 +options SEMMNS=240 +options SEMUME=40 +options SEMMNU=120 + +options ALT_BREAK_TO_DEBUGGER +options PRINTF_BUFR_SIZE=128 + + + For PAE, it is not currently possible + to load modules. Therefore, you will need to add: +options COMPAT_LINUX +options LINPROCFS + + + + + As root, do the usual build steps, e.g.: +make -j4 buildworld +make buildkernel KERNCONF=${kernconf} +make installkernel KERNCONF=${kernconf} +make installworld + The install steps use DESTDIR. + + + + + Customize files in etc/. + Whether you do this on the client itself, or another + machine, will depend on whether you are using + pxeboot. + + If you are using pxeboot: create + a subdirectory of + ${DESTDIR} + called conf/. Create one subdirectory + default/etc/, and (if your site will host + multiple nodes), subdirectories + ${ip-address}/etc/ + to contain override files for individual hosts. (You may find + it handy to symlink each of those directories to a hostname.) + Copy the entire contents of + ${DESTDIR}/etc/ + to default/etc/; that is where you will + edit your files. The by-ip-address + etc/ directories will probably only need + customized rc.conf files. + + In either case, apply the following steps: + + + + Create a + ports-${arch} + user and group. Add it to the wheel + group. It can have the '*' password. + + Create + /home/ports-${arch}/.ssh/ + and populate authorized_keys. + + + + Also add the following users: +squid:*:100:100::0:0:User &:/usr/local/squid:/bin/sh +ganglia:*:102:102::0:0:User &:/usr/local/ganglia:/bin/sh + + Add them to etc/group as well. + + + + Create the appropriate files in + etc/.ssh/. + + + + In etc/crontab: add + * * * * * root /var/portbuild/scripts/client-metrics + + + + + Create the appropriate + etc/fstab. (If you have multiple, + different, machines, you will need to put those in + the override directories.) + + + + In etc/inetd.conf: add + infoseek stream tcp nowait nobody /var/portbuild/scripts/reportload reportload ${arch} + + + + + Create the appropriate + etc/rc.conf. (If you have multiple, + different, machines, you will need to put those in + the override directories.) + + Recommended entries: +hostname="${hostname} +inetd_enable="YES" +linux_enable="YES" +nfs_client_enable="YES" +sendmail_enable="NONE" +sshd_enable="YES" +sshd_program="/usr/local/sbin/sshd" + +squid_enable="YES" +gmond_enable="YES" + + + + + + Create etc/resolv.conf, if + necessary. + + + + Modify etc/sysctl.conf: +9a10,30 +> kern.corefile=/tmp/%N.core +> kern.sugid_coredump=1 +> #debug.witness_ddb=0 +> #debug.witness_watch=0 +> +> # squid needs a lot of fds (leak?) +> kern.maxfiles=40000 +> kern.maxfilesperproc=30000 +> +> # Since the NFS root is static we don't need to check frequently for file changes +> # This saves >75% of NFS traffic +> vfs.nfs.access_cache_timeout=300 +> debug.debugger_on_panic=1 +> +> # For jailing +> security.jail.sysvipc_allowed=1 +> security.jail.allow_raw_sockets=1 +> security.jail.chflags_allowed=1 +> security.jail.enforce_statfs=1 +> +> vfs.lookup_shared=1 + + + + + If desired, modify etc/syslog.conf + to change the logging destinations to + @pointyhat.freebsd.org. + + + + + + + + + + + Configuring <literal>ports</literal> + + + + Install the following ports: +net/rsync +security/openssh-portable +security/sudo +sysutils/ganglia-monitor-core (with GMETAD off) +www/squid + + + There is a WIP to create a meta-port, but it is not yet + complete. + + + + + Customize files in usr/local/etc/. + Whether you do this on the client itself, or another + machine, will depend on whether you are using + pxeboot. + + + The trick of using conf + override subdirectories is less effective here, because + you would need to copy over all subdirectories of + usr/. This is an implementation + detail of how the pxeboot works. + + + Apply the following steps: + + + + Modify + usr/local/etc/gmond.conf: +21,22c21,22 +< name = "unspecified" +< owner = "unspecified" +--- +> name = "${arch} package build cluster" +> owner = "portmgr@FreeBSD.org" +24c24 +< url = "unspecified" +--- +> url = "http://pointyhat.freebsd.org" + + + + If there are machines from more than one cluster in the + same multicast domain (basically = LAN) then change the + multicast groups to different values (.71, .72, etc). + + + + Create + usr/local/etc/rc.d/portbuild.sh: +#!/bin/sh +# +# Configure the system post-boot + +ln -sf /usr2/portbuild /var/portbuild + +# Identify builds ready for use +cd /var/portbuild/${arch} +for i in */builds/*; do + if [ -f ${i}/.ready ]; then + mkdir /tmp/.setup-${i##*/} + fi +done + +# Flag that we are ready to accept jobs +touch /tmp/.boot_finished + + + + If there are machines from more than one cluster in the + same multicast domain (basically = LAN) then change the + multicast groups to different values (.71, .72, etc). + + + + Modify + usr/local/etc/squid/squid.conf: +288,290c288,290 +< #auth_param basic children 5 +< #auth_param basic realm Squid proxy-caching web server +< #auth_param basic credentialsttl 2 hours +--- +> auth_param basic children 5 +> auth_param basic realm Squid proxy-caching web server +> auth_param basic credentialsttl 2 hours +611a612 +> acl localnet src 127.0.0.0/255.0.0.0 +655a657 +> http_access allow localnet +2007a2011 +> maximum_object_size 400 MB +2828a2838 +> negative_ttl 0 minutes + + + Also, change usr/local + to usr2 in the definitions of + cache_dir, + access_log, + cache_log, + cache_store_log, + pid_filename, + netdb_filename, + coredump_dir. + + + + + + Configure ssh: copy + /etc/ssh to + /usr/local/etc/ssh and add + NoneEnabled=yes to + sshd_config. + + + + Modify + usr/local/etc/sudoers: +38a39,42 +> +> # local changes for package building +> %wheel ALL=(ALL) ALL +> ports-${arch} ALL=(ALL) NOPASSWD: ALL + + + + + + + + + + Configuration on the client itself + + + + Change into the port/package directory you picked + above, e.g., cd /usr2. + + + + As root: +mkdir portbuild +chown ports-${arch}:ports-${arch} portbuild +mkdir pkgbuild +chown ports-${arch}:ports-${arch} pkgbuild +mkdir squid +mkdir squid/cache +mkdir squid/logs +chown -R squid:squid squid + + + + + If clients preserve /var/portbuild + between boots then they must either preserve their + /tmp, or revalidate their available + builds at boot time (see the script on the amd64 + machines). They must also clean up stale chroots from previous + builds before creating /tmp/.boot_finished. + + + + + Boot the client. + + + + As root, initialize the squid + directories: + squid -z + + + + Test each of the TCP ports by executing + telnet hostname portnumber. + 414 (or its tunnel) should give you a few lines of status + information; 8649 should give you an + XML response. + + + + + + + Enabling the node + + These steps need to be taken by a portmgr + acting as root on pointyhat. + + + + + If any of the default TCP ports is not available (see + above), you will need to create an ssh + tunnel for it and include it in the appropriate + crontab. + + + + Add an entry to + /home/ports-${arch}/.ssh/config + to specify the public IP address, TCP port for + ssh, username, and any other necessary + information. + + + + Test each of the TCP ports by executing + telnet hostname portnumber, + where portnumber is either the native + portnumber or its tunnelled equivalent. + + + + Add the public IP address to + /etc/hosts.allow. (Remember, multiple + machines can be on the same IP address.) + + + + Create +/var/portbuild/${arch}/clients/bindist-${hostname}.tar. + + + + + Copy one of the existing ones as a template and unpack it + in a temporary directory. + + + Customize etc/resolv.conf and + etc/make.conf for the local site. + + + tar it up and move it to the right + location. + + + + Hint: you will need one of these for each machine; + however, if you have multiple machines at one site, you may + be able to create a site-specific one and symlink to it. + + + + Create +/var/portbuild/${arch}/portbuild-${hostname} + using one of the existing ones as a guide. This + file contains overrides to +/var/portbuild/${arch}/portbuild.conf. + + Suggested values: +disconnected=1 +http_proxy="http://localhost:3128/" +squid_dir=/usr2/squid +scratchdir=/usr2/pkgbuild +client_user=ports-${arch} +sudo_cmd="sudo -H" +rsync_gzip=-z + +infoseek_host=localhost +infoseek_port=${tunelled-tcp-port} + + + Possible other values: +use_md_swap=1 +md_size=9g +use_zfs=1 +scp_cmd="/usr/local/bin/scp" +ssh_cmd="/usr/local/bin/ssh" + + + + + + Add an appropriate entry to + /usr/local/etc/gmetad.conf. + + + + + + + Procedures for dealing with disk failures @@ -1216,7 +1821,7 @@ remapping). - When a disk has failed, please inform &a.kris; so he can try to get it - replaced. + When a disk has failed, please inform the cluster administrators + so we can try to get it replaced. ==== //depot/projects/docproj_nl/en_US.ISO8859-1/books/handbook/multimedia/chapter.sgml#4 (text+ko) ==== @@ -1,7 +1,7 @@ @@ -480,10 +480,12 @@ command-line MP3 player. mpg123 can be run by specifying - the sound device and the MP3 file on the command line, as - shown below: + the sound device and the MP3 file on the command line. Assuming your + audio device is /dev/dsp1.0 and you want + to play the MP3 file Foobar-GreatestHits.mp3 + you would enter the following: - &prompt.root; mpg123 -a /dev/dsp1.0 Foobar-GreatestHits.mp3 + &prompt.root; mpg123 -a /dev/dsp1.0 Foobar-GreatestHits.mp3 High Performance MPEG 1.0/2.0/2.5 Audio Player for Layer 1, 2 and 3. Version 0.59r (1999/Jun/15). Written and copyrights by Michael Hipp. Uses code from various people. See 'README' for more! @@ -496,10 +498,6 @@ Playing MPEG stream from Foobar-GreatestHits.mp3 ... MPEG 1.0 layer III, 128 kbit/s, 44100 Hz joint-stereo - - /dev/dsp1.0 should be replaced with the - dsp device entry on your system. - ==== //depot/projects/docproj_nl/share/sgml/man-refs.ent#19 (text+ko) ==== @@ -20,7 +20,7 @@ lexicographical order by the entity (i.e., the dots used in place of special characters should not be expanded when comparing). - $FreeBSD: doc/share/sgml/man-refs.ent,v 1.487 2009/06/02 01:34:15 gabor Exp $ + $FreeBSD: doc/share/sgml/man-refs.ent,v 1.488 2009/06/16 10:28:11 hrs Exp $ --> @@ -3315,6 +3315,7 @@ + ==== //depot/projects/docproj_nl/www/en/gnome/docs/faq2.sgml#5 (text+ko) ==== @@ -1,6 +1,6 @@ - + @@ -137,7 +137,7 @@

This will enable services such as GDM, HAL, D-BUS, and Avahi on system startup. If you do not want to run all - of these services, you should forgo the + of these services, you should forget the gnome_enable property, and manually enable the services you want.

==== //depot/projects/docproj_nl/www/share/sgml/events.xml#19 (text+ko) ==== @@ -10,10 +10,107 @@ - $FreeBSD: www/share/sgml/events.xml,v 1.73 2009/06/08 17:56:30 delphij Exp $ + $FreeBSD: www/share/sgml/events.xml,v 1.74 2009/06/16 22:23:24 jkois Exp $ + + OpenRheinRuhr 2009 + http://www.openrheinruhr.de/ + + 2009 + 11 + 7 + + + 2009 + 11 + 8 + + + Germany + Bottrop + Saalbau + + This new event in the middle of Europe will + see a huge audience and an impressive list of exhibitors. + FreeBSD will have a booth as well as the other major + BSDs. + + + + discuss & discover 2009 + http://en.discuss-discover.com/ + + 2009 + 10 + 20 + + + 2009 + 10 + 22 + + + Germany + Munich + Messe München + + Discuss & discover is the successor of + Systems. FreeBSD will have a booth as well as the + other major BSDs and will try to repeat the great + success of past years. There will be an Open Source + conference with workshops, talks and a theme park for + various projects. + + + + IT Security Expo 2009 + http://www.it-sa.de/en/ + + 2009 + 9 + 23 + + + 2009 + 9 + 24 + + + Germany + Nuremberg + + Starting in 2009, Nuremberg will be staging + the leading IT security event in a new venue: Although + SYSTEMS has closed its doors, the IT SecurityExpo with + its track record of ten successful years is to be + continued and its scope expanded. There will be a + FreeBSD booth and talks likely. + + + + OpenExpo 2009 + http://www.openexpo.ch/ + + 2009 + 9 + 23 + + + 2009 + 9 >>> TRUNCATED FOR MAIL (1000 lines) <<<