From owner-freebsd-hubs@FreeBSD.ORG Thu Jul 18 19:45:43 2013 Return-Path: Delivered-To: freebsd-hubs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 1B63F908 for ; Thu, 18 Jul 2013 19:45:43 +0000 (UTC) (envelope-from peter@wemm.org) Received: from mail-vb0-x22f.google.com (mail-vb0-x22f.google.com [IPv6:2607:f8b0:400c:c02::22f]) by mx1.freebsd.org (Postfix) with ESMTP id D4076365 for ; Thu, 18 Jul 2013 19:45:42 +0000 (UTC) Received: by mail-vb0-f47.google.com with SMTP id x14so2683049vbb.34 for ; Thu, 18 Jul 2013 12:45:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wemm.org; s=google; h=mime-version:date:message-id:subject:from:to:content-type; bh=mXMMoB6c3iPNv1DDJHVmjyATsgPVA41DNlPsHKMOY5k=; b=UBDA/fSMQWQIdgmGrIPawaEjZup08NO9c9BCPJkHNI4wJ8AjvfQSXwxjLWucp2GEFD YTXgqvegeayJOqIWfcZk8XuRWS/C6nuVrCQcjZLkN+NqwNpu3RdXUebz9K81q21/j+GK L19P7ew9bUm65h7kGRNnJemg0RhK0sVLT/W8g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type :x-gm-message-state; bh=mXMMoB6c3iPNv1DDJHVmjyATsgPVA41DNlPsHKMOY5k=; b=LecWoTM8p01T2oojq8SfBG0kPeGyTPi0k7Ml4dO6jZ1ZRL51jNoXSMPnYf3lCD/EO1 QSMf0yxddzHK+x8iYrHtIX+oTsXttqPZBP1Cz07H/VNfmvx9IDVBzk2Da4W7AZqoNw8a vasi0dL7HY0mKRqtHC3JJGv/t0/ptKJLtN+qtHmFwSIOHlhCruwLaT3V2NA0cdZkwonn H6GPAV/dMXpJbNdxUlA7UUL75+V25dHdPwY4XYH8Hyntvu2POQNu4Lk1gTz1EW0Og9GM UWIRYV5KNVeFqKEH6Ko1tkeoZTPl92Qv7uq8E3nYnY1FmNa/eKBqB9yxAzP8Ckb/6BET JSDQ== MIME-Version: 1.0 X-Received: by 10.59.9.69 with SMTP id dq5mr4480652ved.87.1374176742246; Thu, 18 Jul 2013 12:45:42 -0700 (PDT) Received: by 10.221.37.198 with HTTP; Thu, 18 Jul 2013 12:45:42 -0700 (PDT) Date: Thu, 18 Jul 2013 12:45:42 -0700 Message-ID: Subject: ftp mirror From: Peter Wemm To: freebsd-hubs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQnn+16OTUhRGuq3FRl9usd6O933BPcoSMYnAAcs/uQTkDycppickZbtJ0WwuvhVcsDTO9ZD X-BeenThere: freebsd-hubs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "FreeBSD Distributions Hubs: mail sup ftp" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jul 2013 19:45:43 -0000 ftp-master: du -sh FreeBSD 914G FreeBSD There was a lot of stuff that was painful to rsync, eg: all the checked out branches, via cvs, cvsup etc. But that's mostly gone now. (If there's more that really needs to go, please say something) Most mirrors are syncing it as a unit, eg from the last 10 days logs: 544 rsync on FreeBSD/ 112 rsync on FreeBSD/ports/ 77 rsync on FreeBSD/releases/ 28 rsync on FreeBSD/snapshots/VM-IMAGES 28 rsync on FreeBSD/releases/i386 28 rsync on FreeBSD/releases/amd64 28 rsync on FreeBSD/ports/local-distfiles 28 rsync on FreeBSD/ports/i386/packages-9-stable 28 rsync on FreeBSD/ports/i386 28 rsync on FreeBSD/ports/amd64 27 rsync on FreeBSD/ports/amd64/packages-9-stable 26 rsync on FreeBSD/releases/sparc64/ISO-IMAGES 26 rsync on FreeBSD/releases/powerpc/ISO-IMAGES 26 rsync on FreeBSD/releases/pc98/ISO-IMAGES 26 rsync on FreeBSD/ports/i386/packages-8-stable 25 rsync on FreeBSD/releases/ia64/ISO-IMAGES 25 rsync on FreeBSD/ports/amd64/packages-8-stable 24 rsync on FreeBSD/development/ 22 rsync on FreeBSD/ports/distfiles/ 21 rsync on FreeBSD/CERT/ 16 rsync on FreeBSD/snapshots/ 14 rsync on FreeBSD/doc/ 14 rsync on FreeBSD/branches/ 11 rsync on FreeBSD/updates/ 11 rsync on FreeBSD/tools/ 8 rsync on FreeBSD/ports/local-distfiles/ 5 rsync on FreeBSD/releases/amd64/ 5 rsync on FreeBSD/ports/i386/packages-9-stable/ 4 rsync on FreeBSD/releases/i386/ 4 rsync on FreeBSD/ports/i386/packages-8-stable/ 3 rsync on FreeBSD/releases/powerpc/ 3 rsync on FreeBSD/releases/pc98/ 3 rsync on FreeBSD/ports/amd64/packages-9-stable/ 2 rsync on FreeBSD/releases/sparc64/ 2 rsync on FreeBSD/releases/ISO-IMAGES/8.4/ 2 rsync on FreeBSD/releases/ISO-IMAGES/ 2 rsync on FreeBSD/ports/amd64/packages-8-stable/ 1 rsync on FreeBSD/releases/ia64/ 1 rsync on FreeBSD/ports/i386/ 1 rsync on FreeBSD/ports/amd64/ There are some really weird outliers.. eg: some mirrors are fetching 95%+ of the content by carefully stepping around the old cvs / cvsup checkouts that have been gone for 8+ months. eg: internat.freebsd.org nephtys.lip6.fr ftp.gamma.ru freebsd.twaren.net It looks like most of those should be just syncing the entire tree. Please, make sure: running rsync-3.0+, the later the better make sure to use at least the rsync -a and -H (preserve hard links) flags use FreeBSD/ (or FreeBSD-master/) as a unit We are looking at the TIMESTAMP files in various places, including the *root of the archive*, to try and get a handle on staleness of mirrors. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV UTF-8: So you can \342\200\231 .. for when a ' just won't do ZFS must be the bacon of file systems. "everything's better with ZFS"