Date: Sat, 12 Apr 2014 13:20:43 +0200 From: Matthew Rezny <matthew@reztek.cz> To: freebsd-hackers@freebsd.org Cc: Anton Afanasyev <aasoft@gmail.com> Subject: Re: MITM attacks against portsnap and freebsd-update Message-ID: <1637622.xYKOVe0B9t@desktop.reztek> In-Reply-To: <CAEAhP2iV_ze2ogrw9KJqLEwEzKP%2BpNh9km9kA-jrLwXk7G7rHQ@mail.gmail.com> References: <CAHAXwYCGkP-o0VvMXj5S8-KNA45aTvy%2BsrjDL_=8-x9Dza5z5Q@mail.gmail.com> <2012148.SzKMgBGQYg@desktop.reztek> <CAEAhP2iV_ze2ogrw9KJqLEwEzKP%2BpNh9km9kA-jrLwXk7G7rHQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Friday 11 April 2014 14:20:18 Anton Afanasyev wrote: > On Fri, Apr 11, 2014 at 11:04 AM, Matthew Rezny <matthew@reztek.cz> wrote: > > The biggest effort would be adding rsync to base, but being that we have > > svn(lite) in base it should not be a big deal to add rsync. > > I may be too naive and/or just not understand things as well as those who > do move code into base, so excuse my ignorance, but why was svnlite moved > into base, and why even consider moving rsync into base? > Sure, it is nice if the base includes everything needed to allow > development of it; it is also a must to be able to update and build your > ports. But why include tools that do this, rather than a bootstrap for > installing those tools? There is a strong feeling that the base OS should be 'complete', which includes the ability to update and rebuild the entire base OS (world). It was not always possible to fetch updates with just base components, but since csup had been in the base for some years, many users had come to expect that ability. When CVS was retired, this expectation generated lengthy discussion on the MLs. Before the inclusion of svnlite in 10, someone put a good deal of effort into a svnup tool that worked more similarly to csup. While I applaud its author for the effort, I found it to be unreliable; it crashed several times before getting a full checkout, then could never update the tree without crashing, and of course it still had the same issue of clobbering local changes just as c(v)sup had done. Shortly after I setup a local svn mirror with the intent to use rsync for local distribution within my LAN, I got wind of svnlite coming in 10 and so I jumped straight to that. Checkout from local mirror is MUCH faster than from the official repos (with so many small files, the latency across the WAN results in more time spent waiting between files than actually transferring file contents), so it was still a worthwhile exercise. > For developing and updating base, why not include a script that fetches a > (sufficiently fresh) snapshot of the ports tree and let the user decide > whether they want to use svn or any other port to update their sources? If > it is deemed too large a download (a valid concern) - download only svn and > its dependencies, possibly even to a ports tree rooted in a location > different from /usr/ports, and build svn from that. > For keeping ports up to date, why not include a script that fetches a > (sufficiently fresh) copy of the ports tree and tell the user that the > preferred method to update is rsync; heck, create a port that uses rsync to > do what Matthew described above, and /offer/ to install it for the the user > from the tree that was just downloaded. > > Something along the lines of the above would completely remove the need to > keep unrelated code in base - and the need to keep it updated - , while > still allowing the end user to keep base and ports up to date. > > > Anton You have made a most excellent suggestion that had not crossed my mind. Due to the aforementioned discussions, I had it stuck in my head that the tools must be in base. However, pkg(ng) has set precedence with the bootstrap in base which installs the pkg package. Since I always install a ports tree and update it before any ports, and I don't use binary packages (yet, local poudriere is still on my TODO list), I have been getting pkg via the install of portmaster (the first port I install) and thus have never used the bootstrapper. The bootstrap idea is great as it allows more flexibility and rapid development (which is exactly the reason pkg is done the way it is). Svnlite will still be around for those that must use only tools in base. Since this is a proposed portsnap replacement, that would also be capable of doing the base src as a bonus, there is really no reason it can't be a tool that lives in ports with a handy bootstrap in base. The bootstrap could simply be a sh script which uses fetch to fetch the tar.xz files via ftp/http(s), verify hashes, decompress into an appropriate staging area (/var/portsync), verify hashes of the tar files, then extract to /usr/src and/or /usr/ports. There could then be a port, say port-mgmt/portsync, which implements the update functionality. Installation of this port could pull in rsync as a dep. Since it is in ports, the full portsync could be written in Python (that would be my choice as my sh-fu is weak) without waiting for Python to maybe if ever make it's way into base. The portsync.sh in base could automatically invoke portsync.py if present. I think I just added one more thing to my lengthy TODO list... DOH! (from later in the thread) > > I know .. Git-lovers are upset.. > > Was this addressed to me, or just a general rant/reference that I did not > understand? I don't think that was directed at anyone in particular, but there were numerous requests to forget SVN and use git, apparently not realizing that SVN had been used for src for years and CVS was merely a (hackish) export from there. Git's decentralized nature does not fit well with the project's needs. It just does not work well for maintaining a single, central authoritative repo that is the one truth. Those who wish to use git for their local hacking can do so as git allows checkout from SVN to initialize a local repo, and they can generate diffs to submit back to our central repo. Also, it has been my experience on projects with many fewer developers, that not only is git a nightmare of merge races, but its inability to track changes on a per file basis within a changeset puts it on par with CVS (if not worse, in that sometimes its diff output outright lies about the progeny of a change).
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1637622.xYKOVe0B9t>