From owner-freebsd-hubs@FreeBSD.ORG Mon Mar 31 15:10:12 2014 Return-Path: Delivered-To: freebsd-hubs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 278DE771; Mon, 31 Mar 2014 15:10:12 +0000 (UTC) Received: from mail0.glenbarber.us (mail0.glenbarber.us [208.86.227.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.glenbarber.us", Issuer "RapidSSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C1306B1C; Mon, 31 Mar 2014 15:10:08 +0000 (UTC) Received: from glenbarber.us (70.15.88.86.res-cmts.sewb.ptd.net [70.15.88.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: gjb) by mail0.glenbarber.us (Postfix) with ESMTPSA id BE97B58E1; Mon, 31 Mar 2014 15:10:06 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.8.3 mail0.glenbarber.us BE97B58E1 Authentication-Results: mail0.glenbarber.us; dkim=none reason="no signature"; dkim-adsp=none Date: Mon, 31 Mar 2014 11:10:02 -0400 From: Glen Barber To: freebsd-hubs@FreeBSD.org Subject: Stale mirrors and rsync Message-ID: <20140331151002.GB1736@glenbarber.us> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="TRYliJ5NKNqkz5bu" Content-Disposition: inline X-Operating-System: FreeBSD 11.0-CURRENT amd64 X-SCUD-Definition: Sudden Completely Unexpected Dataloss X-SULE-Definition: Sudden Unexpected Learning Event User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-hubs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "FreeBSD Distributions Hubs: mail sup ftp" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Mar 2014 15:10:12 -0000 --TRYliJ5NKNqkz5bu Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, We (admins) have changed the way we monitor the mirror states to keep track of staleness. We still continue to use the TIMESTAMP files, but since not all mirrors have a copy of all FTP data, it became tricky to determine which TIMESTAMP file makes sense to keep track of. Since all mirrors have the releases/ directory, please make sure the releases/TIMESTAMP file is readable. We are using fetch(1) to grab the file and compare the update time against the current time to determine how many days a mirror is out of sync, but some mirrors do not have a readable releases/TIMESTAMP file, so otherwise makes it impossible to keep track of things. Related, at present, there are 15 mirrors that are out of sync by a significant number of days. The list is below, for those that are unaware that their respective mirror is out of sync. Several weeks ago, it was pointed out that one of the mirrors was not updating, and upon investigation it was determined that the mirror had run out of space. When syncing with ftp-master, please be sure to use the '-H' flag with rsync. This ensures that symlinks are copied as symlinks, and avoids fetching multiple copies of the same file. This appears to be happening quite often, and not only is inefficient use of bandwidth for all involved, but also significantly increases the overall sync time. Currently, the total size of the entire pub/FreeBSD/ directory is: % du -hs . 1.1T . Thank you, as always, for your continued support of FreeBSD. Stale mirror list: ftp.kr.FreeBSD.org: 56 days stale=20 ftp.nz.FreeBSD.org: 12 days stale=20 ftp.ru.FreeBSD.org: 82 days stale=20 ftp.si.FreeBSD.org: 48 days stale=20 ftp.za.FreeBSD.org: 11 days stale=20 ftp1.FreeBSD.org: 48 days stale=20 ftp11.FreeBSD.org: 48 days stale=20 ftp2.FreeBSD.org: 48 days stale=20 ftp3.ca.FreeBSD.org: 17 days stale=20 ftp4.br.FreeBSD.org: 298 days stale=20 ftp4.ru.FreeBSD.org: 34 days stale=20 ftp6.de.FreeBSD.org: 1384 days stale=20 ftp6.us.FreeBSD.org: 48 days stale=20 ftp8.us.FreeBSD.org: 48 days stale=20 moss.cse.buffalo.edu 48 days stale=20 Glen On behalf of: admins --TRYliJ5NKNqkz5bu Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQIcBAEBCAAGBQJTOYVKAAoJELls3eqvi17QXTkP/RyuppR9m5PEA1Wkxdtm3nNc I2DY+InLYR2NyKDwfFK0Z0CZWClQcjZbpN7wd0dShcWDcxwFvzUMHamNUTRyjrif /RVfBKXQqHr5SP1VfzKnYlEJKtFQ1lvPjZXvUalYzHFBHphHYvOn/cCYJMlTnyhn wzcSDU9aobesqG8XCfckrEEiPAzzs+DSvsVB7tU/CRycee/opRbsn3STn777Zi0C y7O4K961sXt3D2N4z1RjONJEGsOT2yCHMnDdfxKhTY4jAAJNb0jt9Yqin0p1F8J4 KBlx9o4GHcgfNG38Xt5CqT6bZlDxqxlQAVp/snAiASxmuCc0i084adak2k5Y21Ky FxyZAhTEcmLSP/6Ls7NMaFLbZO/FmPLzgq+qMkZCvIlng118+G85d26ZS3a0itXM Cvn9ocXJV3mFkeTi8srHUJgvYazKWwGPuqBnsCAskAxxKjz+/dip3oDzPtFLOifW NfR/T32xdB101FXRN6zrtylTm1+NQj3mt+7NTK0dnsjgd9nAJqbRKHbuaLCfhW5I rsM97hEeNJgEXGw3fWhZ67+8au+LIyRrC/YhnsYkfgNUS0aNGqMWHB9c8a3CqvYs wRi7tXf77yO2fMw/E2tUPmlhfoD+wFZ4RXyFAUxJgY0NxsG8FyP+GRafG5T3lDro B8avcjsNPxdo+P88mnA7 =KQao -----END PGP SIGNATURE----- --TRYliJ5NKNqkz5bu--