Date: Mon, 8 Oct 2007 01:47:57 -0700 From: Jeremy Chadwick <koitsu@FreeBSD.org> To: obrien@freebsd.org Cc: freebsd-ports@freebsd.org Subject: editors/vim -- long fetch times Message-ID: <20071008084757.GA59389@eos.sc1.parodius.com>
next in thread | raw e-mail | index | archive | help
This is a subject I've wanted to bring up for a few years now. Why I decided to bring it up now, I don't know; I guess I just felt it was time. Simply put: the time it takes for "make fetch" to complete in editors/vim gets worse and worse as more patches are released: icarus# cd /usr/ports/editors/vim-lite && make distclean icarus# time make fetch ... 10.801u 3.297s 6:16.33 3.7% 568+1347k 0+52io 0pf+0w That's over 6 minutes to fetch 116 patches and the base vim tarball, totalling about 7MBytes of data: icarus# du -sk /usr/ports/distfiles/vim 7248 /usr/ports/distfiles/vim True, it's only excessive when you have none of the patches in DISTFILES/vim (that is to say, if you already have 115 patches and there's 116 of them, the wait time isn't very long). The "wasted" time is caused by a few things. This opens somewhat of a can of worms (in regards to how to enhance fetch(1)), but the main two: * FTP protocol is used for each and every patch. FTP server may be doing things like forward and reverse DNS look-ups on client, as well as an RFC931/1413 ident check. * fetch(1) does not support the ability to fetch multiple files during the same FTP session. The idea would be to modify fetch(1) to support doing multiple RETRs during a single FTP session. Solutions as I see them: * Change the first site in MASTER_SITE_VIM to the HTTP version of the main distribution site: http://ftp.vim.org/pub/vim/unix/. This is already in the list, but the HTTP mirrors are *last*, not first. * Provide "snapshots" of the vim patches as a tgz-ball somewhere. Have "make fetch" download that and untar it into DISTFILES/vim/. Snapshots would need to be updated every time a new patch was added. * Modify fetch(1) as described above. * Get Bram to stop releasing hundreds of individual patches between minor releases. :-) Using the HTTP fetch method, the time drastically decreases to 1.5 minutes for me: icarus# cd /usr/ports/editors/vim-lite && make distclean icarus# time make MASTER_SITE_VIM=http://ftp.vim.org/pub/vim/unix/ fetch 9.282u 2.747s 1:28.83 13.5% 628+1444k 1+52io 0pf+0w Thoughts? -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20071008084757.GA59389>