From owner-freebsd-hubs@FreeBSD.ORG Sun Sep 9 18:40:35 2012 Return-Path: Delivered-To: hubs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6D589106564A for ; Sun, 9 Sep 2012 18:40:35 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (wonkity.com [67.158.26.137]) by mx1.freebsd.org (Postfix) with ESMTP id 251D28FC16 for ; Sun, 9 Sep 2012 18:40:34 +0000 (UTC) Received: from lightning.wonkity.com (lightning.wonkity.com [10.0.0.8]) by wonkity.com (8.14.5/8.14.5) with ESMTP id q89IeRDi065349 for ; Sun, 9 Sep 2012 12:40:27 -0600 (MDT) (envelope-from wblock@lightning.wonkity.com) Received: from lightning.wonkity.com (localhost [127.0.0.1]) by lightning.wonkity.com (8.14.5/8.14.5) with ESMTP id q89IeRdq070222 for ; Sun, 9 Sep 2012 12:40:27 -0600 (MDT) (envelope-from wblock@lightning.wonkity.com) Received: from localhost (wblock@localhost) by lightning.wonkity.com (8.14.5/8.14.5/Submit) with ESMTP id q89IeRvD070219 for ; Sun, 9 Sep 2012 12:40:27 -0600 (MDT) (envelope-from wblock@lightning.wonkity.com) Date: Sun, 9 Sep 2012 12:40:27 -0600 (MDT) From: Warren Block To: hubs@freebsd.org Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (wonkity.com [10.0.0.1]); Sun, 09 Sep 2012 12:40:28 -0600 (MDT) Cc: Subject: cvsup9 responding with "Invalid greeting from server" X-BeenThere: freebsd-hubs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "FreeBSD Distributions Hubs: mail sup ftp" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Sep 2012 18:40:35 -0000 cvsup9.FreeBSD.org is giving this response to csup: Connected to 208.83.20.166 Invalid greeting from server supfile: *default host=cvsup9.FreeBSD.org *default base=/var/db *default prefix=/usr *default release=cvs tag=. *default delete use-rel-suffix *default compress ports-all From owner-freebsd-hubs@FreeBSD.ORG Sun Sep 9 20:23:24 2012 Return-Path: Delivered-To: hubs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 970D8106566B for ; Sun, 9 Sep 2012 20:23:24 +0000 (UTC) (envelope-from ben@desync.com) Received: from exile.desync.com (exile.desync.com [IPv6:2607:f178::167]) by mx1.freebsd.org (Postfix) with ESMTP id 6B7968FC17 for ; Sun, 9 Sep 2012 20:23:24 +0000 (UTC) Received: from [172.16.17.11] (unknown [173.169.234.189]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by exile.desync.com (Postfix) with ESMTPSA id 330D3126F; Sun, 9 Sep 2012 16:23:18 -0400 (EDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.0 \(1486\)) From: ben wilber In-Reply-To: Date: Sun, 9 Sep 2012 16:23:16 -0400 Content-Transfer-Encoding: 7bit Message-Id: References: To: Warren Block X-Mailer: Apple Mail (2.1486) Cc: hubs@freebsd.org Subject: Re: cvsup9 responding with "Invalid greeting from server" X-BeenThere: freebsd-hubs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "FreeBSD Distributions Hubs: mail sup ftp" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Sep 2012 20:23:24 -0000 On Sep 9, 2012, at 2:40 PM, Warren Block wrote: > cvsup9.FreeBSD.org is giving this response to csup: > > Connected to 208.83.20.166 > Invalid greeting from server Sorry, looks like cvsupd crashed. From owner-freebsd-hubs@FreeBSD.ORG Sun Sep 9 21:00:34 2012 Return-Path: Delivered-To: hubs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B2FDF1065672 for ; Sun, 9 Sep 2012 21:00:34 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (wonkity.com [67.158.26.137]) by mx1.freebsd.org (Postfix) with ESMTP id 576EA8FC1A for ; Sun, 9 Sep 2012 21:00:34 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.5/8.14.5) with ESMTP id q89L0QYc002171; Sun, 9 Sep 2012 15:00:26 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.5/8.14.5/Submit) with ESMTP id q89L0Qhu002168; Sun, 9 Sep 2012 15:00:26 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Sun, 9 Sep 2012 15:00:26 -0600 (MDT) From: Warren Block To: ben wilber In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (wonkity.com [127.0.0.1]); Sun, 09 Sep 2012 15:00:26 -0600 (MDT) Cc: hubs@freebsd.org Subject: Re: cvsup9 responding with "Invalid greeting from server" X-BeenThere: freebsd-hubs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "FreeBSD Distributions Hubs: mail sup ftp" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Sep 2012 21:00:34 -0000 On Sun, 9 Sep 2012, ben wilber wrote: > > On Sep 9, 2012, at 2:40 PM, Warren Block wrote: > >> cvsup9.FreeBSD.org is giving this response to csup: >> >> Connected to 208.83.20.166 >> Invalid greeting from server > > Sorry, looks like cvsupd crashed. Working now, thanks! From owner-freebsd-hubs@FreeBSD.ORG Sun Sep 9 22:12:28 2012 Return-Path: Delivered-To: hubs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3D30710657C6 for ; Sun, 9 Sep 2012 22:12:28 +0000 (UTC) (envelope-from stephen@missouri.edu) Received: from wilberforce.math.missouri.edu (wilberforce.math.missouri.edu [128.206.184.213]) by mx1.freebsd.org (Postfix) with ESMTP id E38A38FC0C for ; Sun, 9 Sep 2012 22:12:27 +0000 (UTC) Received: from [127.0.0.1] (wilberforce.math.missouri.edu [128.206.184.213]) by wilberforce.math.missouri.edu (8.14.5/8.14.5) with ESMTP id q89MCQrw061861 for ; Sun, 9 Sep 2012 17:12:27 -0500 (CDT) (envelope-from stephen@missouri.edu) Message-ID: <504D144A.4060503@missouri.edu> Date: Sun, 09 Sep 2012 17:12:26 -0500 From: Stephen Montgomery-Smith User-Agent: Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20120827 Thunderbird/15.0 MIME-Version: 1.0 To: hubs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: About 8G to be added to mirrors carrying CTM X-BeenThere: freebsd-hubs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "FreeBSD Distributions Hubs: mail sup ftp" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Sep 2012 22:12:28 -0000 I am going to add about 8G to CTM over the next few months. This is to allow for ctm-svn-ports-cur. Thanks, Stephen From owner-freebsd-hubs@FreeBSD.ORG Mon Sep 10 23:13:25 2012 Return-Path: Delivered-To: freebsd-hubs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1BBA81065675 for ; Mon, 10 Sep 2012 23:13:25 +0000 (UTC) (envelope-from stephen@missouri.edu) Received: from wilberforce.math.missouri.edu (wilberforce.math.missouri.edu [128.206.184.213]) by mx1.freebsd.org (Postfix) with ESMTP id BD1CB8FC16 for ; Mon, 10 Sep 2012 23:13:24 +0000 (UTC) Received: from [127.0.0.1] (wilberforce.math.missouri.edu [128.206.184.213]) by wilberforce.math.missouri.edu (8.14.5/8.14.5) with ESMTP id q8ANDHuE063274 for ; Mon, 10 Sep 2012 18:13:17 -0500 (CDT) (envelope-from stephen@missouri.edu) Message-ID: <504E740D.6040306@missouri.edu> Date: Mon, 10 Sep 2012 18:13:17 -0500 From: Stephen Montgomery-Smith User-Agent: Mozilla/5.0 (X11; Linux i686; rv:15.0) Gecko/20120827 Thunderbird/15.0 MIME-Version: 1.0 To: freebsd-hubs@freebsd.org References: <504D144A.4060503@missouri.edu> In-Reply-To: <504D144A.4060503@missouri.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: About 8G to be added to mirrors carrying CTM X-BeenThere: freebsd-hubs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "FreeBSD Distributions Hubs: mail sup ftp" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Sep 2012 23:13:25 -0000 On 09/09/2012 05:12 PM, Stephen Montgomery-Smith wrote: > I am going to add about 8G to CTM over the next few months. This is to > allow for ctm-svn-ports-cur. OK, it looks like I was way off in my estimate. It will be more like 2 or 3G. What I found interesting was that the complete ports svn, which is 14G in size, compresses down with xz down to 0.6G! Something like a 95% compression! From owner-freebsd-hubs@FreeBSD.ORG Tue Sep 11 20:26:28 2012 Return-Path: Delivered-To: freebsd-hubs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 40824106566B for ; Tue, 11 Sep 2012 20:26:28 +0000 (UTC) (envelope-from mirrors@tds.net) Received: from mail-wi0-f172.google.com (mail-wi0-f172.google.com [209.85.212.172]) by mx1.freebsd.org (Postfix) with ESMTP id BCBA28FC12 for ; Tue, 11 Sep 2012 20:26:26 +0000 (UTC) Received: by wibhi8 with SMTP id hi8so3308049wib.13 for ; Tue, 11 Sep 2012 13:26:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:x-gm-message-state; bh=+AFjm7NG59Ri2DrWpIiEkrilmopC9GCzYYyVhz4jt88=; b=nMYVaGbdBGjgpiI0qsNTAIzYnbYx/gR3B/qTrQKiBDwDu486KkWxdwwztleHTSHgiS ne51aIrQpkftvH9WxjzlZUN92VYdsLINah84r7VMq4gta5fo0Yf34l8Wb6UmlkmrGzi4 JZzdPbYFHlJfel4JEGNxwaZT+Fy4B7T1vLnulwv7azPcBhKZS6iPUas6o7w8/akkYB/k 4MeYF5Cs7w/aEAqN6cDaH0jq1ztN8IoAUDJSSkKbwQwZkEnaVBUkXJJaQmTZrmlKf/Bq D7uh5Ac1tZihpBvUBLrNcEpWdRknhaSJZItjY51chDjmgsQagvUPWUuH6vzWnfiwO1WT pWMQ== MIME-Version: 1.0 Received: by 10.180.94.164 with SMTP id dd4mr27528322wib.1.1347395185903; Tue, 11 Sep 2012 13:26:25 -0700 (PDT) Received: by 10.216.213.31 with HTTP; Tue, 11 Sep 2012 13:26:25 -0700 (PDT) In-Reply-To: References: Date: Tue, 11 Sep 2012 15:26:25 -0500 Message-ID: From: "mirrors tds.net" To: freebsd-hubs@freebsd.org X-Gm-Message-State: ALoCoQng9iD7e7Z/AxSSSlrW2kYKJHbzb+ywM4vNULQdFlVBjFiJ+UiUCjTm2ObeY4FBiu4ovk9B Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: Cvsup & cvsupd on Linux X-BeenThere: freebsd-hubs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "FreeBSD Distributions Hubs: mail sup ftp" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Sep 2012 20:26:28 -0000 Hi List, Is there anyone who can help on this? I am receiving pressure from management, and if I cannot get this working we will have to bring down cvsup8.freebsd.org until both ports & src are brought into subversion. I have captured gdbs to send to the cm3 mailing list but yet to hear back (I emailed them today so no response yet) I am currently running RHEL 5.8, 64bit Any sort of help or insight is appreciated, I am unable to run a freebsd box in this environment. Current cvsup8 is running on an older version of Linux as well. Thank you for the help. On Wed, Aug 29, 2012 at 1:34 PM, mirrors tds.net wrote: > Hi list, > > I have had no success getting cvsup/cvsupd to work correctly on linux (I > know I know...but that's what I'm stuck with) I have reached out to the > folks at cm3, but they're busy. > > the cvsup page references using cm3 on Linux, so that is what I am doing. > I can compile the binaries just fine, but when i run them they just hang > after transferring an amount of data (sometimes it pulls down a bunch of > info sometimes it hangs almost immediately) I then have to ctrl-\ for it > to quit. > > When I run the daemon I connect to it fine and start to pull data down. > However it stops after a period of time and then eventually i see an > "inactivity timeout" > > Does anyone have experience and could shed some light on how to get this > working correctly? > > Thank you. > From owner-freebsd-hubs@FreeBSD.ORG Wed Sep 12 11:43:02 2012 Return-Path: Delivered-To: freebsd-hubs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B160D1065672 for ; Wed, 12 Sep 2012 11:43:02 +0000 (UTC) (envelope-from mat@FreeBSD.org) Received: from prod2.absolight.net (mx3.absolight.net [IPv6:2a01:678:2:100::25]) by mx1.freebsd.org (Postfix) with ESMTP id 4E3308FC1B for ; Wed, 12 Sep 2012 11:43:02 +0000 (UTC) Received: from prod2.absolight.net (localhost [127.0.0.1]) by prod2.absolight.net (Postfix) with ESMTP id 332B6BDC3E for ; Wed, 12 Sep 2012 13:43:01 +0200 (CEST) Received: from ogg.in.absolight.net (ogg.in.absolight.net [79.143.241.239]) by prod2.absolight.net (Postfix) with ESMTPA id C58D0BDC1F for ; Wed, 12 Sep 2012 13:43:00 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by ogg.in.absolight.net (Postfix) with ESMTP id EAE5941E76C5 for ; Wed, 12 Sep 2012 13:42:59 +0200 (CEST) Date: Wed, 12 Sep 2012 13:42:59 +0200 From: Mathieu Arnold To: freebsd-hubs@freebsd.org Message-ID: <021473D8D9E6A7A785543024@ogg.in.absolight.net> X-Mailer: Mulberry/4.0.8 (Mac OS X) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline Subject: freebsd-update "mirror" howto X-BeenThere: freebsd-hubs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "FreeBSD Distributions Hubs: mail sup ftp" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Sep 2012 11:43:02 -0000 Hi, It's not really a hubs thing, but anyway, I thought I'd share my few bits... For a long time, I pondered about upgrading my 300+ freebsd servers, and a few months back freebsd-update seemed like the way to go... Now, I did not want all those servers to hit the main freebsd-update servers, so I setup a nice local "mirror" using varnish. Varnish's configuration is there https://gist.github.com/2637602 and it would be even better if varnish could use SRV records to define it's backends by itself, but for now, there are a couple of shell that I have to run whenever I feel like it to update the list. On the servers, I (well, puppet did) updated the /etc/freebsd-update.conf file with : ServerName freebsd-update.absolight.net and in my absolight.net zone, I have : _http._tcp.freebsd-update IN SRV 1 5 80 1.freebsd-update 1.freebsd-update IN A 79.143.241.136 and that's about it. The only thing, maybe is that freebsd-update doesn't honor the port field of the SRV record, so I had to run varnishd on the 80 port, but it's not really a problem here. -- Mathieu Arnold From owner-freebsd-hubs@FreeBSD.ORG Wed Sep 12 12:53:41 2012 Return-Path: Delivered-To: freebsd-hubs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id F301A106566C for ; Wed, 12 Sep 2012 12:53:40 +0000 (UTC) (envelope-from simon@qxnitro.org) Received: from mail-ie0-f182.google.com (mail-ie0-f182.google.com [209.85.223.182]) by mx1.freebsd.org (Postfix) with ESMTP id ADC118FC16 for ; Wed, 12 Sep 2012 12:53:39 +0000 (UTC) Received: by iebc12 with SMTP id c12so3633887ieb.13 for ; Wed, 12 Sep 2012 05:53:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qxnitro.org; s=google; h=mime-version:sender:x-originating-ip:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=uLDoyApaMvk2qL3zJA5IeO25SX9kReTSPxqxU+qee0o=; b=Ua0WbQIkgShUi8duMiDK+tzaIzqu5sZOLiZ/+A0rUmUND/izoU2p7C8qqJjMq+HHoN hw+okJ4rflxrNxRHmnzQdbHGysbJdUwTA93vmJh/nCEvvqzCCJyxK8CT6O6INgynQbM3 uMjaUlR/k/o3sjksevBYd6gAxZhTawvbrWhG0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:sender:x-originating-ip:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :x-gm-message-state; bh=uLDoyApaMvk2qL3zJA5IeO25SX9kReTSPxqxU+qee0o=; b=nkEpVWTwB6hzg7T3G+CkPmQQp6J7530x39/jwTq+sFWkcvJtC/x05xm5hL8eXfQUiG Vs0/5EK6pBdcZ90jpe09hUtjLTHUYWk9/r0LKtVubI6aSN0hUH/DjOiw+EL0xiZz6UGj aymQOBLSO0mfgsjnvXC84+lAHc9/mnRN0bWhsJvbyrz4JugpbxotdgbBq7XoY48LZfbB AebFcGRWLb4H1X+YeGRr3+FWuB74mLSSWBW7NPOBn0qQ3thaeV1kUK/A8VjwhCzt6UQJ vKQL9CxtrV97MXITfUyEEMIfdq8Q5aIU8FNUS40GVYvdHBaK+SB+IFbs7IjKUW8Hf2n4 a11Q== MIME-Version: 1.0 Received: by 10.50.41.129 with SMTP id f1mr20052212igl.57.1347454418772; Wed, 12 Sep 2012 05:53:38 -0700 (PDT) Sender: simon@qxnitro.org Received: by 10.64.102.104 with HTTP; Wed, 12 Sep 2012 05:53:38 -0700 (PDT) X-Originating-IP: [172.28.8.16] In-Reply-To: References: Date: Wed, 12 Sep 2012 13:53:38 +0100 X-Google-Sender-Auth: YTSTHMY8lFAHjM8Vhrp27Y7dAUc Message-ID: From: "Simon L. B. Nielsen" To: "mirrors tds.net" Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQmJ75PYpyXmJ+k5l93pm6yicD9UVSfjnfNbK2K3Lzqpt8i8UvkAOu6HDuFeqzz+3gveb9jw Cc: freebsd-hubs@freebsd.org Subject: Re: Cvsup & cvsupd on Linux X-BeenThere: freebsd-hubs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "FreeBSD Distributions Hubs: mail sup ftp" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Sep 2012 12:53:41 -0000 On Tue, Sep 11, 2012 at 9:26 PM, mirrors tds.net wrote: > Hi List, Is there anyone who can help on this? I am receiving pressure > from management, and if I cannot get this working we will have to bring > down cvsup8.freebsd.org until both ports & src are brought into subversion. > > I have captured gdbs to send to the cm3 mailing list but yet to hear back > (I emailed them today so no response yet) I am currently running RHEL 5.8, > 64bit > > Any sort of help or insight is appreciated, I am unable to run a freebsd > box in this environment. Current cvsup8 is running on an older version of > Linux as well. I have never tried myself, but I seem to recall hearing problems over the years with running cvsupd on OS's other than FreeBSD, so I wouldn't be too hopeful in getting it running. One thing you might be able to try is running 32bit instead of 64bit. Even on FreeBSD cvsupd 64bit is problematic (for some users). -- Simon L. B. Nielsen From owner-freebsd-hubs@FreeBSD.ORG Wed Sep 12 12:57:36 2012 Return-Path: Delivered-To: freebsd-hubs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0F033106566B for ; Wed, 12 Sep 2012 12:57:36 +0000 (UTC) (envelope-from simon@qxnitro.org) Received: from mail-ie0-f182.google.com (mail-ie0-f182.google.com [209.85.223.182]) by mx1.freebsd.org (Postfix) with ESMTP id BF5EC8FC14 for ; Wed, 12 Sep 2012 12:57:35 +0000 (UTC) Received: by iebc12 with SMTP id c12so3642578ieb.13 for ; Wed, 12 Sep 2012 05:57:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qxnitro.org; s=google; h=mime-version:sender:x-originating-ip:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=yuJsStAysN/HJmoxQz+RaoHHZVf6s/Tr93bV1jYq2zw=; b=ZutPq8j/QL3k3CcYQIKVnU2f+34PvVnwpLJNdwGLleSu3X8KT+fQJ921uwg5jSBsQM geNb1nvCUHd4ss6zOzuFvRjgYhdv5aH3Qon9PmIagJiTTJFIDJsm9gUO1v19FxGXgfXL dxm09weLSDtDfDpV48pJQWh3uIaH7uNnoifgE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:sender:x-originating-ip:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :x-gm-message-state; bh=yuJsStAysN/HJmoxQz+RaoHHZVf6s/Tr93bV1jYq2zw=; b=ezuK33ITkFHuOaUOzu2lVonHThcL6Gizs9lyi+gb0tKO8wZgLvkurY9bjcmMuxnHZN Xrycp496xbhBOB+XAtHVfZrt/sgiMesPMylfoDX0CvFzF3vuoXQr6I1B5s8AW3zu525j UpeSm9oThS6ADYPt80xKjIaPJubj31A495EOIIe4bk2t58M9CSsrh4zXXvkGCOlzUjOA ivcgdk1072SVn5tgwPpVoNN1j+8qHJ2MG6JIpIaftqliET+c3gvLKCECd7Vx6G76RV2d L1dp5jkYSSxFHEDaien5mZIr1ZFDbeV3tMUVCZu4T5ZaLGP+CZ2nYortFJAlLoU5lU4j kPRg== MIME-Version: 1.0 Received: by 10.50.77.227 with SMTP id v3mr16178946igw.64.1347454654886; Wed, 12 Sep 2012 05:57:34 -0700 (PDT) Sender: simon@qxnitro.org Received: by 10.64.102.104 with HTTP; Wed, 12 Sep 2012 05:57:34 -0700 (PDT) X-Originating-IP: [172.28.8.16] In-Reply-To: <021473D8D9E6A7A785543024@ogg.in.absolight.net> References: <021473D8D9E6A7A785543024@ogg.in.absolight.net> Date: Wed, 12 Sep 2012 13:57:34 +0100 X-Google-Sender-Auth: eatIJ-XyiifGzMiybtDViBfDtxI Message-ID: From: "Simon L. B. Nielsen" To: Mathieu Arnold Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQlIJ32Y4jBNHXPSKkt24yWaMHpcCNFLm/w9ppuDJZCI7EaRzZF3iE9DeHRT4/EhQdtVVck7 Cc: freebsd-hubs@freebsd.org Subject: Re: freebsd-update "mirror" howto X-BeenThere: freebsd-hubs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "FreeBSD Distributions Hubs: mail sup ftp" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Sep 2012 12:57:36 -0000 On Wed, Sep 12, 2012 at 12:42 PM, Mathieu Arnold wrote: > It's not really a hubs thing, but anyway, I thought I'd share my few bits... > > For a long time, I pondered about upgrading my 300+ freebsd servers, and a > few months back freebsd-update seemed like the way to go... Very interesting for information like this to be shared. Even better if somebody could be tricked into putting it in the handbook :-). > Now, I did not want all those servers to hit the main freebsd-update > servers, so I setup a nice local "mirror" using varnish. Why did you decide to go with Varnish instead of a program designed as a forward cache? > Varnish's configuration is there https://gist.github.com/2637602 and it > would be even better if varnish could use SRV records to define it's > backends by itself, but for now, there are a couple of shell that I have to > run whenever I feel like it to update the list. Evil workaround thing would be to just parse the SRV records in a script and then generate the varnish config wrt. backend based on that... not pretty though :-). -- Simon L. B. Nielsen From owner-freebsd-hubs@FreeBSD.ORG Wed Sep 12 13:23:18 2012 Return-Path: Delivered-To: freebsd-hubs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7B4C6106566B; Wed, 12 Sep 2012 13:23:18 +0000 (UTC) (envelope-from mat@FreeBSD.org) Received: from prod2.absolight.net (mx3.absolight.net [IPv6:2a01:678:2:100::25]) by mx1.freebsd.org (Postfix) with ESMTP id 1639E8FC08; Wed, 12 Sep 2012 13:23:18 +0000 (UTC) Received: from prod2.absolight.net (localhost [127.0.0.1]) by prod2.absolight.net (Postfix) with ESMTP id 5014ABDC66; Wed, 12 Sep 2012 15:23:17 +0200 (CEST) Received: from ogg.in.absolight.net (ogg.in.absolight.net [79.143.241.239]) by prod2.absolight.net (Postfix) with ESMTPA id 496C9BDC65; Wed, 12 Sep 2012 15:23:17 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by ogg.in.absolight.net (Postfix) with ESMTP id A140541EA303; Wed, 12 Sep 2012 15:23:16 +0200 (CEST) Date: Wed, 12 Sep 2012 15:23:15 +0200 From: Mathieu Arnold To: "Simon L. B. Nielsen" Message-ID: <6D5EF049FD84BF0620E50077@ogg.in.absolight.net> In-Reply-To: References: <021473D8D9E6A7A785543024@ogg.in.absolight.net> X-Mailer: Mulberry/4.0.8 (Mac OS X) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline Cc: freebsd-hubs@freebsd.org Subject: Re: freebsd-update "mirror" howto X-BeenThere: freebsd-hubs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "FreeBSD Distributions Hubs: mail sup ftp" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Sep 2012 13:23:18 -0000 +--On 12 septembre 2012 13:57:34 +0100 "Simon L. B. Nielsen" wrote: | On Wed, Sep 12, 2012 at 12:42 PM, Mathieu Arnold wrote: | |> It's not really a hubs thing, but anyway, I thought I'd share my few |> bits... |> |> For a long time, I pondered about upgrading my 300+ freebsd servers, and |> a few months back freebsd-update seemed like the way to go... | | Very interesting for information like this to be shared. Even better | if somebody could be tricked into putting it in the handbook :-). Hum, I see where you're going with that ;-) The thing is, I wanted a bit of feedback to see if I wasn't doing something stupid before committing myself into doing that. I have not written doc for a long time, I'll try to see if I can coerce myself into doing that next week while I'm vacationing in Corsica. |> Now, I did not want all those servers to hit the main freebsd-update |> servers, so I setup a nice local "mirror" using varnish. | | Why did you decide to go with Varnish instead of a program designed as | a forward cache? Because Varnish is *cool*. :-p |> Varnish's configuration is there https://gist.github.com/2637602 and it |> would be even better if varnish could use SRV records to define it's |> backends by itself, but for now, there are a couple of shell that I have |> to run whenever I feel like it to update the list. | | Evil workaround thing would be to just parse the SRV records in a | script and then generate the varnish config wrt. backend based on | that... not pretty though :-). Well, that's almost already what's there, when I feel the need to update, I edit the file with vim, I remove the old lines, copy the shell line in the script, remove the # and type "V!bash" ;-) -- Mathieu Arnold From owner-freebsd-hubs@FreeBSD.ORG Wed Sep 12 14:43:10 2012 Return-Path: Delivered-To: freebsd-hubs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6D28F106566B; Wed, 12 Sep 2012 14:43:10 +0000 (UTC) (envelope-from jonny@jonny.eng.br) Received: from coe.ufrj.br (roma.coe.ufrj.br [IPv6:2001:470:8ce3:1::65]) by mx1.freebsd.org (Postfix) with ESMTP id 18A678FC08; Wed, 12 Sep 2012 14:43:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by coe.ufrj.br (Postfix) with ESMTP id DC7491480E6F; Wed, 12 Sep 2012 11:43:05 -0300 (BRT) X-Virus-Scanned: amavisd-new at coe.ufrj.br Received: from coe.ufrj.br ([146.164.53.65]) by localhost (roma.coe.ufrj.br [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wiNgOFjFP5JE; Wed, 12 Sep 2012 11:43:00 -0300 (BRT) Received: from [IPv6:2804:294:4080:3:e464:e83a:604:d800] (unknown [IPv6:2804:294:4080:3:e464:e83a:604:d800]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by coe.ufrj.br (Postfix) with ESMTPSA id B1CCC1480A41; Wed, 12 Sep 2012 11:42:58 -0300 (BRT) Message-ID: <50509F69.2010603@jonny.eng.br> Date: Wed, 12 Sep 2012 11:42:49 -0300 From: =?ISO-8859-1?Q?Jo=E3o_Carlos_Mendes_Lu=EDs?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20120828 Thunderbird/15.0 MIME-Version: 1.0 To: Mathieu Arnold References: <021473D8D9E6A7A785543024@ogg.in.absolight.net> In-Reply-To: <021473D8D9E6A7A785543024@ogg.in.absolight.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hubs@freebsd.org Subject: Re: freebsd-update "mirror" howto X-BeenThere: freebsd-hubs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "FreeBSD Distributions Hubs: mail sup ftp" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Sep 2012 14:43:10 -0000 On 09/12/2012 08:42 AM, Mathieu Arnold wrote: > It's not really a hubs thing, but anyway, I thought I'd share my few bits... > > For a long time, I pondered about upgrading my 300+ freebsd servers, and a > few months back freebsd-update seemed like the way to go... > > Now, I did not want all those servers to hit the main freebsd-update > servers, so I setup a nice local "mirror" using varnish. > > Varnish's configuration is there https://gist.github.com/2637602 and it > would be even better if varnish could use SRV records to define it's > backends by itself, but for now, there are a couple of shell that I have to > run whenever I feel like it to update the list. Recent versions of varnish can use DNS servers for backends. Even if you cannot use SRV records, you can always setup a separate DNS server with the "backend" addresses. Some setups use a hack with load balancers where you can use the SAME IP address for the varnish and the "backend" server, making caching almost transparent. From owner-freebsd-hubs@FreeBSD.ORG Thu Sep 13 10:38:38 2012 Return-Path: Delivered-To: freebsd-hubs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8DF8D1065673 for ; Thu, 13 Sep 2012 10:38:38 +0000 (UTC) (envelope-from simon@qxnitro.org) Received: from mail-ie0-f182.google.com (mail-ie0-f182.google.com [209.85.223.182]) by mx1.freebsd.org (Postfix) with ESMTP id 45FAC8FC0C for ; Thu, 13 Sep 2012 10:38:37 +0000 (UTC) Received: by iea17 with SMTP id 17so655159iea.13 for ; Thu, 13 Sep 2012 03:38:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qxnitro.org; s=google; h=mime-version:sender:x-originating-ip:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=aCSyZ/q5PxcRL9PJfnLlsrnL/1p52V73rj1cyr62nOE=; b=A3Hmc/Mrk4C8L2cXscNMO1PGKW61ulDO4v0Wwh9vfDA+nsWXncmOK6HcQ6XzCGUgtq 7APpLRCQtn9JyIzUF/LPUrldBVHxZ43JNT9Gehw1jkKq9xQRU5QOwNyJ2oBqbfOg2746 zflgIXuDx0U0MwsPxe6ojcsnxx2apu6G8k5Xo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:sender:x-originating-ip:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :x-gm-message-state; bh=aCSyZ/q5PxcRL9PJfnLlsrnL/1p52V73rj1cyr62nOE=; b=O6nNCItDpyr8jcoxxDccKNYPl/6EpG8X39//kOvmPGePP5ky4wAij/6+yYCnlxFL3H T/+1opFhmle3b++cWgDbBXKbYNNSYBf2Y4gkb8jrFski5oiR/BRia3PJBO0PBR0jOADb McnsDFUO0PdVrA4hcOzCfW+krVzKdCst6+kWJgT64jf7pn4+cDMwImAQhYVasCRiha/I how7NfQENZGuGWp6VB5NJKoKeVM3OkOem3HwIwIZY11NzIcxxuQQwIUdeuDZBtoV9EVy iPNN1pTtfFC9s9SDg4bqv/UZeUZ6MA8k8pju4V4jxV3moK4fXyy2apLZDd0Z2Oq/khyM LFyQ== MIME-Version: 1.0 Received: by 10.50.157.201 with SMTP id wo9mr1895556igb.57.1347532717295; Thu, 13 Sep 2012 03:38:37 -0700 (PDT) Sender: simon@qxnitro.org Received: by 10.64.102.104 with HTTP; Thu, 13 Sep 2012 03:38:37 -0700 (PDT) X-Originating-IP: [2620:0:1040:201:ec21:fca4:8fbc:892d] In-Reply-To: <6D5EF049FD84BF0620E50077@ogg.in.absolight.net> References: <021473D8D9E6A7A785543024@ogg.in.absolight.net> <6D5EF049FD84BF0620E50077@ogg.in.absolight.net> Date: Thu, 13 Sep 2012 11:38:37 +0100 X-Google-Sender-Auth: CayNhspe0kmyaug44C_l2tr22Vs Message-ID: From: "Simon L. B. Nielsen" To: Mathieu Arnold Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQmNKY+rNHT8epbfJ3SpChTTz7is6lWmLcABEYQPhQ4hAkKJzfsKXZJZ0mOX1uuBKyzadDLi Cc: freebsd-hubs@freebsd.org Subject: Re: freebsd-update "mirror" howto X-BeenThere: freebsd-hubs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "FreeBSD Distributions Hubs: mail sup ftp" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Sep 2012 10:38:38 -0000 On Wed, Sep 12, 2012 at 2:23 PM, Mathieu Arnold wrote: > +--On 12 septembre 2012 13:57:34 +0100 "Simon L. B. Nielsen" > wrote: > | On Wed, Sep 12, 2012 at 12:42 PM, Mathieu Arnold wrote: > | > |> It's not really a hubs thing, but anyway, I thought I'd share my few > |> bits... > |> > |> For a long time, I pondered about upgrading my 300+ freebsd servers, and > |> a few months back freebsd-update seemed like the way to go... > | > | Very interesting for information like this to be shared. Even better > | if somebody could be tricked into putting it in the handbook :-). > > Hum, I see where you're going with that ;-) :-) > The thing is, I wanted a bit of feedback to see if I wasn't doing something > stupid before committing myself into doing that. Sure. I don't see any fundamental problem doing it that way, though I recall reading in the past that you shouldn't use varnish as a forward cache - but I never looked into why. One thing which may or may not be a problem is support for HTTP pipelining. At least using squid I found the lack of pipelining support to really slow down freebsd-update (and portsnap). Can be that varnish is enough faster than squid that this becomes less of an issue. > I have not written doc for a long time, I'll try to see if I can coerce > myself into doing that next week while I'm vacationing in Corsica. Great :-). I'm sure the freebsd-doc people would be happy to help with extra markup etc. > |> Now, I did not want all those servers to hit the main freebsd-update > |> servers, so I setup a nice local "mirror" using varnish. > | > | Why did you decide to go with Varnish instead of a program designed as > | a forward cache? > > Because Varnish is *cool*. :-p OK :-). > |> Varnish's configuration is there https://gist.github.com/2637602 and it > |> would be even better if varnish could use SRV records to define it's > |> backends by itself, but for now, there are a couple of shell that I have > |> to run whenever I feel like it to update the list. > | > | Evil workaround thing would be to just parse the SRV records in a > | script and then generate the varnish config wrt. backend based on > | that... not pretty though :-). > > Well, that's almost already what's there, when I feel the need to update, I > edit the file with vim, I remove the old lines, copy the shell line in the > script, remove the # and type "V!bash" ;-) Ah, right. I hadn't yet looked at the varnish config file. -- Simon L. B. Nielsen