From owner-freebsd-doc Fri Apr 12 5:30:41 2002 Delivered-To: freebsd-doc@freebsd.org Received: from mailout.informatik.tu-muenchen.de (mailout.informatik.tu-muenchen.de [131.159.0.5]) by hub.freebsd.org (Postfix) with ESMTP id 0EBC637B404; Fri, 12 Apr 2002 05:30:23 -0700 (PDT) Received: from mailrelay2.informatik.tu-muenchen.de (mailrelay2.informatik.tu-muenchen.de [131.159.254.8]) by mailout.informatik.tu-muenchen.de (Postfix) with ESMTP id 6021C6176; Fri, 12 Apr 2002 14:30:22 +0200 (MEST) Received: from atrbg11.informatik.tu-muenchen.de (atrbg11.informatik.tu-muenchen.de [131.159.24.91]) by mailrelay2.informatik.tu-muenchen.de (Postfix) with ESMTP id 2804947384; Fri, 12 Apr 2002 14:30:22 +0200 (MEST) Received: by atrbg11.informatik.tu-muenchen.de (Postfix, from userid 20455) id C11BE139A1; Fri, 12 Apr 2002 14:30:21 +0200 (CEST) Date: Fri, 12 Apr 2002 14:30:21 +0200 From: Daniel Lang To: kuriyama@freebsd.org, logo@freebsd.org, freebsd-doc@freebsd.org Subject: Update to hubs article Message-ID: <20020412123021.GI24607@atrbg11.informatik.tu-muenchen.de> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="EVF5PPMfhYS0aIcm" Content-Disposition: inline User-Agent: Mutt/1.3.27i X-Geek: GCS/CC d-- s: a- C++$ UBS++++$ P+++$ L- E-(---) W+++(--) N++ o K w--- O? M? V? PS+(++) PE--(+) Y+ PGP+ t++ 5+++ X R+(-) tv+ b+ DI++ D++ G++ e+++ h---(-) r++>+++ y+ Sender: owner-freebsd-doc@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org --EVF5PPMfhYS0aIcm Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, here's another update to the hubs article in: doc/en_US.ISO8859-1/articles/hubs/article.sgml I've sent part's of this a while ago, but it did not get committed. Anyway, I've updated it according to the recent changes regarding email-adress entities, etc. The patch fixes a little bit of speech and adds another hint section about, when to mirror and reasonable schedules. Best regards, Daniel -- IRCnet: Mr-Spock - Eddie would go! - Daniel Lang * dl@leo.org * +49 89 289 25735 * http://www.leo.org/~dl/ --EVF5PPMfhYS0aIcm Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="article.sgml.diff" --- article.sgml.orig Fri Apr 12 14:15:08 2002 +++ article.sgml Fri Apr 12 14:23:21 2002 @@ -50,7 +50,7 @@ Depending on the set of releases, architectures, and degree of completeness you want to mirror, a huge amount of disk space may be consumed. Also keep in mind, - that official mirrors are probably required to be + that official mirrors are probably required to be complete. The CVS repository and the web pages should always be mirrored completely. Also note, that the numbers stated here, are reflecting the current @@ -101,7 +101,22 @@ indeed consume some CPU. For AnonCVS it can even be required to set up a MFS of at least 300 MB, so you need to take this into account - for your memory requirements. + for your memory requirements. The following + are just examples to give you a very rough hint. + + + For a moderately visited site, that offers + rsync, you might + consider a current CPU with around 800Mhz - 1 GHz, + and at least 512MB RAM. This is probably the + minimum you want for an official + site. + + + For a frequently used site, a Dual Athlon MP 1800, + with 2GB RAM may be more appropriate. + (Ok, this would be a very smooth configuration, + if you put like DDR-RAM into it ... :-)) You also want to consider a fast disk subsystem. @@ -113,9 +128,9 @@ Services to offer This section describes the services that are - required for you to offer, and those that are + required for you to offer and those, that are optional. It also gives hints, which software - can be used to provide this service. + can be used to provide this particular service. FTP (required for FTP fileset) @@ -131,11 +146,11 @@ the path /pub/FreeBSD. - There is lot of software available which + There is a lot of software available which can be set up to allow anonymous FTP (in alphabetical order). - /usr/libexec/ftpd: FreeBSD's own ftpd, + /usr/libexec/ftpd: FreeBSD's own ftpd will probably do a very good job. Be sure to read &man.ftpd.8;. @@ -156,32 +171,33 @@ ftp/twoftpd: same as above ftp/vsftpd: the very secure ftpd - ftp/wu-ftpd: The widely used ftpd from Washington + ftp/wu-ftpd: The widely used ftpd from Washington University. It became sort of infamous, because of the huge amount of security issues that became obvious in wu-ftpd. - It also around for a long time, which can be positive + It is also around for a long time. This can be seen as positive (experience) or negative (lots of bloated code). - FreeBSD's ftpd, proftpd, wu-ftpd and maybe ncftpd are among the + FreeBSD's ftpd, proftpd, + wu-ftpd and maybe ncftpd are among the most used ones. The others are probably not that much widespread. RSYNC (optional for FTP fileset) - Rsync is often also offered for convenience, for the + Rsync is often also offered for convenience, for the contents of the FTP area of FreeBSD. The protocol is different from FTP in many ways, and - overall, it can be stated, that it is much more - bandwidth friendly, as only differences between files + overall it can be stated, that it is much more + bandwidth friendly, since only differences between files are transferred, not whole files. - Rsync does require significant amount of memory for + Rsync does require significant amount of memory for each instance. The size depends on the size of the synced module in terms of number of directories and - files. Rsync can use rsh and + files. Rsync can use rsh and ssh (now default) as a transport, - or use it's own protocol for stand-alone access + or use it's own transport for stand-alone access (this is the preferred method for public rsync servers). Authentication, connection limits, and other restrictions may be applied. There is just one software package @@ -197,10 +213,10 @@ If you want to offer the FreeBSD webpages, you need to install a webserver a.k.a httpd. You may offer the FTP fileset via HTTP, as well, if you like. - Some argue HTTP is more efficient for download, but + Some argue HTTP is more efficient for downloads, but I cannot tell. The most commonly - used httpd is Apache, although there are others around, - take a look at /usr/ports/www. + used httpd is Apache, although there are others around. + Take a look at /usr/ports/www. www/apache13 @@ -209,19 +225,20 @@ CVSup (desired for CVS repository) - CVSup is a very efficient way of distributing files. - It works similar as rsync, but was specially designed for + CVSup is a very efficient way of distributing files. + It works similar to rsync, but was specially designed for the use with CVS repositories. If you want to offer the FreeBSD CVS repository, you really want to consider - offering it via CVSup. Still it is possible to offer - the CVS repository via AnonCVS, FTP, Rsync or HTTP, but - people would not benefit from CVSup access. - CVSup was developed by &a.jdp;. + offering it via CVSup. Still it is possible to offer + the CVS repository via AnonCVS, FTP, + Rsync or HTTP, but + people would benefit more from CVSup access. + CVSup was developed by &a.jdp;. It is a bit tricky to install on non-FreeBSD platforms, since it is written in Modula-3 and therefore requires - a Modula-3 environmen. John Polstra has built a + a Modula-3 environment. John Polstra has built a stripped down version of M3, that is sufficient to - run CVSup, and can be installed much easier. + run CVSup, and can be installed much easier. See Ezm3 for details. Related ports are: @@ -254,29 +271,26 @@ CVSup is known to eat some CPU on the server as on the client, since it needs to compare lots of files. - - - Please have look at the CVSup - like &man.cvsup.1; and consider using the - option, as it can reduce the amount of work to be done - a lot. - - AnonCVS (optional for CVS repository) If you have the CVS repository, you may want to offer - anonymous CVS access. There is not that much demand for it, + anonymous CVS access. A short warning first: + There is not that much demand for it, and it requires some experience and you need to know, - what you are doing. Generally there are two ways, how + what you are doing. + + + Generally there are two ways, how to access a CVS repository remotely: via pserver or via ssh (we don't consider rsh). For anonymous access, pserver is very well suited, but some still offer ssh - access as well. There is some custom crafted wrapper - program around, to be used as a login-shell for the + access as well. There is a custom crafted + wrapper + in the CVS repository, to be used as a login-shell for the anonymous ssh account. It does a chroot, and therefore requires the CVS repository to be available under the anonymous user's homedirectory, which may not be possible @@ -291,7 +305,7 @@ cvspserver stream tcp nowait root /usr/bin/cvs cvs -f -l -R -T /anoncvstmp --allow-root=/home/ncvs pserver See the manpage for details of the options. See also the cvs info - page, about how additional ways, to make sure, access is read-only. + page, about additional ways to make sure, access is read-only. It is advisable, that you create an unprivileged account, preferrably called anoncvs. Also you need to create a file passwd @@ -352,7 +366,7 @@ ftp/mirror was very popular, but seemed - to have some drawbacks, as its written in perl, + to have some drawbacks, as it is written in &man.perl.1;, and did have real problems on mirroring large directories like a FreeBSD site. There are rumours, that the current version has fixed this, by allowing @@ -360,7 +374,7 @@ the directory structure. - In general FTP is not really good for mirroring, since it transfers + In general FTP is not really good for mirroring, since it transferes each whole file, if it has changed, and does not create a single data stream, that will benefit from a large TCP congestion window. @@ -375,8 +389,8 @@ rsync is already mentioned in . Since rsync access is not - required, your preferred upstream site may not allow it, - but it already widely used, so chances are small, that + required, your preferred upstream site may not allow it. + Since it is quite common, though, chances are small, that you cannot use it. You can always consider using an upstream server, that offers it, just for the benefits of rsync. @@ -387,7 +401,7 @@ machine, most admins impose limitations on their server. For a mirror, you should ask the site maintainer you are syncing from, about their policy, and maybe - an exception for your host, since you are a mirror. + an exception for your host (since you are a mirror). A command line to mirror FreeBSD could look like that: @@ -433,13 +447,21 @@ way to mirror the archive, in terms of efficiency, but it is only available from few sites. In fact I just know ftp-master.freebsd.org for sure. + + + Please have look at the CVSup documentation + like &man.cvsup.1; and consider using the + option, as it can reduce the amount of work to be done + a lot. + + Mirroring the CVS repository - Again you have various possibilities, but the most + Again you have various possibilies, but the most recommended one, is to use CVSup. @@ -469,6 +491,13 @@ You should also have a look at /usr/share/examples/cvsup + + + Please don't forget to consider the hint, + mentioned in this note + above. + + Using other methods @@ -478,7 +507,7 @@ anyway. Since most sites offer the CVS repository as part of the FTP fileset under the path /pub/FreeBSD/development/FreeBSD-CVS, - the following methods can of course be used. + the following methods could be used. FTP RSYNC @@ -493,7 +522,7 @@ - AFAICT, you can NOT AnonCVS to + AFAIK, you can NOT use AnonCVS to mirror the CVS repository, since CVS does not allow you to access the repository itself, but only checked out versions of the modules. @@ -591,7 +620,7 @@ The building of the documentation, as well as lots - of side issues is documented itself in: + of side issues, is documented itself in: fdp-primer. Please read this piece of documentation, especially if you have problems, building the documentation. @@ -605,6 +634,52 @@ + + How often should I mirror? + + Every mirror should be updated on a regular + basis. You will certainly need some script + framework for it, that will be called by + &man.cron.8;. Since nearly every admin + does this his own way, we cannot give + specific instructions. It could work + like this: + + + + + Put the command to run your mirroring application + in a script. Use of a plain /bin/sh + script is recommended. + + + + + Add some output redirections, so diagnostic + messages are logged to a file. + + + + + Test, if your script works. Check the logs. + + + + + Use &man.crontab.1; to add the script the + appropriate user's &man.crontab.5;. + + + + + Here are some recommended schedules: + + FTP fileset: daily + CVS repository: daily to hourly + WWW pages: daily + + + Where to mirror from @@ -617,15 +692,15 @@ Mirrors are organised by country. All official mirrors have a DNS entry of the form - ftpX.CC.freebsd.org. Whith + ftpN.CC.freebsd.org. Whith CC (i.e. country code) beeing the top level domain of the country, where this mirror is located; - and X is a number, - telling that the host would be the Xth + and N is a number, + telling that the host would be the Nth mirror in that country. - (Same applies to cvsupX.CC.freebsd.org, - wwwX.CC.freebsd.org, etc.) + (Same applies to cvsupN.CC.freebsd.org, + wwwN.CC.freebsd.org, etc.) There are mirrors with no CC part. These are usually located in the US, but don't need to. ftp.freebsd.org is currently @@ -718,13 +793,12 @@ for access. If you are already an official mirror, this certainly helps you getting access. In any other case make sure your country really needs another mirror. - If it already has three or more, ask - the &a.hubs; first. + If it already has three or more, ask the &a.hubs; first. - There are just two master sites, for the FTP fileset and - for the CVS repository (the webpages and docs are obtained from CVS, - so there is no need for it). + There are just two master sites, one for the FTP fileset and + one for the CVS repository (the webpages and docs are + obtained from CVS, so there is no need for it). ftp-master.freebsd.org @@ -769,7 +843,7 @@ Set up the required authentication by following these instructions. Make sure you specify the server as - freefall.freebsd.org on the cvpasswd + freefall.freebsd.org on the cvpasswd command line, as described in this document, even when you are contacting cvsup-master.freebsd.org @@ -851,7 +925,7 @@ How to become official then? - An interesting questions, especially, since the state + An interesting question, especially, since the state of beeing official comes with some benefits, like a much higher bill from your ISP, as more people will be using your site. Also it may be a key requirement, to get access @@ -889,13 +963,14 @@ If there is no subdomain delegated, yet, for your country, you probably need to contact hostmaster@freebsd.org, - however, you can try the &a.hubs; first. + however, you can try the &a.hubs; + first. - Then you can ask the &a.doc;, or the &a.hubs; to add your - mirror site to the mirror list in the + Then you can ask the &a.doc; or the the &a.hubs; + to add your mirror site to the mirror list in the FreeBSD Handbook. Make sure you tell them the email address, to list as the maintainer of the site. --EVF5PPMfhYS0aIcm-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-doc" in the body of the message