Date: Tue, 6 Jan 2009 20:16:21 +0100 From: Daniel Bond <db@danielbond.org> To: Vincent Hoffman <vince@unsane.co.uk> Cc: freebsd-stable@freebsd.org Subject: Re: freebsd-update painfully slow - slower than source code build of world and kernel Message-ID: <484FA671-3B56-4A7C-9551-37AD2D1AF7B3@danielbond.org> In-Reply-To: <49639546.5070608@unsane.co.uk> References: <DA7E7739-0631-4B00-8CA8-D8C9E22B2126@danielbond.org> <alpine.BSF.2.00.0901061121530.1996@localhost> <F6385D88-BFAF-47A0-B598-78C971FCBD7C@danielbond.org> <alpine.BSF.2.00.0901061242120.1996@localhost> <08B216B4-79AB-45AB-AB4D-C8CD62196B87@danielbond.org> <alpine.BSF.2.00.0901061731550.1343@localhost> <C68E4D01-D846-4FE4-B71E-9D7ED2EA55E2@danielbond.org> <49639546.5070608@unsane.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi, thanks for useful and relevant information. However, this is just one part of the process. Generating the diffs, prepping the server, and the whole server-side setup/management of it is another - I am sure there are tools for this too. Cheers, Daniel. On Jan 6, 2009, at 6:30 PM, Vincent Hoffman wrote: > Daniel Bond wrote: >> The same could be said about CVSup, one could write a caching cvsup >> proxy-server, and then we could just get rid of all the other >> cvsup-servers, except two (like freebsd-update soon will have). The >> point is, for portsnap and freebsd-update to scale properly, it needs >> to be opened up to the public, like CVSup is. People running a single >> server at home, or maybee two, most like won't want to set up a PROXY >> server, and they would be required to update both servers at the same >> day for the Proxy server to actually cache something - which many may >> not want. And there are a lot of people running a few servers, here >> and there. >> >> >> >> Sure, a national squid-proxy could work - although, there is no >> individual proxy setting for portsnap/freebsd-update.. It honors >> HTTP_PROXY environment variable, which a lot of other tools also use. >> Some tools might not work via this proxy, especially for local >> addresses - the administrators of these servers probably don't want >> all the ports tarballs to go via these, and people could use them for >> nasty things. So, then we are back to manually setting/specifying the >> proxy-server, each time one wants to run the commands - which people >> might forget. (Is this getting complicated enough yet..?) We would >> basically be creating a whole lot of new potential problems for the >> users, to solve the problem in question.. >> >> >> I am also interested in learning how the portsnap protocol works, >> maybe there are potential issues with it, that a second eye might >> spot, or room for improvement? From what I gather, Colin is a very >> cleaver guy, so it is not very likely, but still, other people could >> learn from it. >> > well portsnap/freebsd-update are shell scripts so not too hard to > read. > The actual transfer protocol is piplined http and is done by > /usr/libexec/phttpget (in base so src code available > /usr/src/usr.sbin/portsnap/phttpget/phttpget.c ) > also see http://www.daemonology.net/phttpget/ > > >> I would like to see these tools as the default recommended tools to >> use in the future, and that is why I am so worried about this. >> The point I am trying to make is, or actually the question is: Why is >> freebsd-update (and portsnap) so secretive? Why can't the average Joe >> run his own portsnap-mirror at home? What are we afraid of? > I seem to remember once reading that Colin wanted to make it a more > polished system before he release it, but i cant find that email > anymore. > > Vince >> >> I don't see any problems with this, except maybe loosing some detail >> in Colin's nice graphs (which would be the case for proxies too). >> >> >> Cheers, >> >> >> Daniel. >> >> >> On Jan 6, 2009, at 5:42 PM, Christopher Arnold wrote: >> >>> >>> >>> On Tue, 6 Jan 2009, Daniel Bond wrote: >>> >>>> reading your answer, you are obviously confusing what I am saying >>>> about freebsd-update with the portsnap program. Also, I also wrote >>>> in my first post >>> No i'm not confusing them, just trying to follow two subjects at the >>> same time. Sorry if that is confusing. >>> >>>> that HTTP_PROXY / Caching proxy server does not help me much. This >>>> is because I download a lot of "initial tarball snapshots".. I >>>> would >>>> rarely see "Cache hits" in my proxy log. I guess I could set >>>> something up to fetch nightly via proxy, to keep the data in house, >>>> for when I need it. I don't want to use a PROXY server, I feel this >>>> is attacking the problem at the wrong end. >>>> >>> Ok, lets go again. Either you mirror (maybe by having a squid proxy >>> and walk the tree) and thats going to me even worse for you. Or you >>> use a squid proxy to keep stuff you need close to you and share >>> among >>> different installations. >>> >>> Or you setup one or more national squid proxies and configure your >>> machines manually just like you do with cvsup. >>> >>> >>> >>>> I agree, I am interested to hear the views of the wise ones. >>>> Personally I'm going back to CVSup until freebsd-update and >>>> portsnap >>>> mirrors are in a more distributed or usable state. >>>> >>> At least portsnap started to work for me earlier today. Havn't tried >>> update yet. >>> >>> But yes i agree, update and portsnap infrastructure could be done >>> better. >>> I have some ideas and will try to write them down in a while. >>> >>> /Chris >>> _______________________________________________ >>> freebsd-stable@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >>> To unsubscribe, send any mail to >>> "freebsd-stable-unsubscribe@freebsd.org" >> >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org >> " > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org > "
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?484FA671-3B56-4A7C-9551-37AD2D1AF7B3>