From owner-freebsd-stable@FreeBSD.ORG Tue Jan 6 17:31:10 2009 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C76DB106566B for ; Tue, 6 Jan 2009 17:31:10 +0000 (UTC) (envelope-from vince@unsane.co.uk) Received: from unsane.co.uk (unsane-pt.tunnel.tserv5.lon1.ipv6.he.net [IPv6:2001:470:1f08:110::2]) by mx1.freebsd.org (Postfix) with ESMTP id 2DBB58FC13 for ; Tue, 6 Jan 2009 17:31:09 +0000 (UTC) (envelope-from vince@unsane.co.uk) Received: from vhoffman.lon.namesco.net (150.117-84-212.staticip.namesco.net [212.84.117.150]) (authenticated bits=0) by unsane.co.uk (8.14.0/8.14.0) with ESMTP id n06HVIYJ097926 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 6 Jan 2009 17:31:19 GMT (envelope-from vince@unsane.co.uk) Message-ID: <49639546.5070608@unsane.co.uk> Date: Tue, 06 Jan 2009 17:30:46 +0000 From: Vincent Hoffman User-Agent: Thunderbird 2.0.0.19 (Macintosh/20081209) MIME-Version: 1.0 To: Daniel Bond References: <08B216B4-79AB-45AB-AB4D-C8CD62196B87@danielbond.org> In-Reply-To: X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Christopher Arnold , freebsd-stable@freebsd.org Subject: Re: freebsd-update painfully slow - slower than source code build of world and kernel X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jan 2009 17:31:11 -0000 Daniel Bond wrote: > The same could be said about CVSup, one could write a caching cvsup > proxy-server, and then we could just get rid of all the other > cvsup-servers, except two (like freebsd-update soon will have). The > point is, for portsnap and freebsd-update to scale properly, it needs > to be opened up to the public, like CVSup is. People running a single > server at home, or maybee two, most like won't want to set up a PROXY > server, and they would be required to update both servers at the same > day for the Proxy server to actually cache something - which many may > not want. And there are a lot of people running a few servers, here > and there. > > > > Sure, a national squid-proxy could work - although, there is no > individual proxy setting for portsnap/freebsd-update.. It honors > HTTP_PROXY environment variable, which a lot of other tools also use. > Some tools might not work via this proxy, especially for local > addresses - the administrators of these servers probably don't want > all the ports tarballs to go via these, and people could use them for > nasty things. So, then we are back to manually setting/specifying the > proxy-server, each time one wants to run the commands - which people > might forget. (Is this getting complicated enough yet..?) We would > basically be creating a whole lot of new potential problems for the > users, to solve the problem in question.. > > > I am also interested in learning how the portsnap protocol works, > maybe there are potential issues with it, that a second eye might > spot, or room for improvement? From what I gather, Colin is a very > cleaver guy, so it is not very likely, but still, other people could > learn from it. > well portsnap/freebsd-update are shell scripts so not too hard to read. The actual transfer protocol is piplined http and is done by /usr/libexec/phttpget (in base so src code available /usr/src/usr.sbin/portsnap/phttpget/phttpget.c ) also see http://www.daemonology.net/phttpget/ > I would like to see these tools as the default recommended tools to > use in the future, and that is why I am so worried about this. > The point I am trying to make is, or actually the question is: Why is > freebsd-update (and portsnap) so secretive? Why can't the average Joe > run his own portsnap-mirror at home? What are we afraid of? I seem to remember once reading that Colin wanted to make it a more polished system before he release it, but i cant find that email anymore. Vince > > I don't see any problems with this, except maybe loosing some detail > in Colin's nice graphs (which would be the case for proxies too). > > > Cheers, > > > Daniel. > > > On Jan 6, 2009, at 5:42 PM, Christopher Arnold wrote: > >> >> >> On Tue, 6 Jan 2009, Daniel Bond wrote: >> >>> reading your answer, you are obviously confusing what I am saying >>> about freebsd-update with the portsnap program. Also, I also wrote >>> in my first post >> No i'm not confusing them, just trying to follow two subjects at the >> same time. Sorry if that is confusing. >> >>> that HTTP_PROXY / Caching proxy server does not help me much. This >>> is because I download a lot of "initial tarball snapshots".. I would >>> rarely see "Cache hits" in my proxy log. I guess I could set >>> something up to fetch nightly via proxy, to keep the data in house, >>> for when I need it. I don't want to use a PROXY server, I feel this >>> is attacking the problem at the wrong end. >>> >> Ok, lets go again. Either you mirror (maybe by having a squid proxy >> and walk the tree) and thats going to me even worse for you. Or you >> use a squid proxy to keep stuff you need close to you and share among >> different installations. >> >> Or you setup one or more national squid proxies and configure your >> machines manually just like you do with cvsup. >> >> >> >>> I agree, I am interested to hear the views of the wise ones. >>> Personally I'm going back to CVSup until freebsd-update and portsnap >>> mirrors are in a more distributed or usable state. >>> >> At least portsnap started to work for me earlier today. Havn't tried >> update yet. >> >> But yes i agree, update and portsnap infrastructure could be done >> better. >> I have some ideas and will try to write them down in a while. >> >> /Chris >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to >> "freebsd-stable-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"