Date: Sat, 14 Jan 2006 15:16:31 +0000 (GMT) From: Robert Watson <rwatson@FreeBSD.org> To: Jo Rhett <jrhett@svcolo.com> Cc: freebsd-stable@freebsd.org, current@freebsd.org Subject: Re: SUMMARY: Fast releases demand binary updates.. Message-ID: <20060114144115.M38874@fledge.watson.org> In-Reply-To: <20060112080856.GI84964@svcolo.com> References: <20060112080856.GI84964@svcolo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, 12 Jan 2006, Jo Rhett wrote: <much unhappiness elided> > For now we'll have to see if Colin can find ways around the problems without > the tools he needs to do it right. It seems fairly clear to me that Colin's tools provide useful building blocks that can be used to create something like what you're looking for: it is an efficient way to generate, manage, and distribute difference sets between snapshots of a system. What Colin's pieces explicitly don't provide is a administrative infrastructure for imposing heavy weight packaging semantics and version information -- specifically, an explicit version database, roll-back, and a mechanism for configuration merges. I'm sure many sites would like to have the heavier weight mechanism, but it's certainly a non-trivial project that will take a lot of time for someone to get right. Someone(tm) will need to spend several months on it to get it really working well, but I think it would be well worth the effort. I do suspect, however, that the starting point for this is to discard the formal project releases, and instead simply provide infrastructure for someone with their own notion of what releases are, and what versions make sense. That way if a site chooses to deploy our stock releases or patches, that's fine, but they can also manage their own custom snapshots. Presumably a first starting point is to take the assumption that the build system will produce a series of install snapshots, presumably using installworld/installkernel to a specific DESTDIR, and that all updates will be generated between them, with version names and relevant difference combinations to be assigned to them by the admin. The second part will be handling what's normally done by mergemaster in some way -- for some environments, simply generating a second tarball that contains the output of distrib-dirs (etc) and then providing that to bootstrap mergemaster rather than a source tree would be sufficient. Other environments might wish to simply splat down replacement files based on having run mergemaster in advance on the build machine. A bit of minimal but well-crafted glue is presumably required here, such as the ability to point mergemaster at the update blob and tell it "Yeah, so, I want my / to be merged from here, not /usr/src/etc". The final part is the meta-data for these snapshots, both carried with them, and on the target system. If using Colin's binary difference pieces, they will often be quite compact, and it's easy to imagine storing the necessary information to roll back or forwards by several revisions in a relatively small amount of space. Tricky things include how to accomplish relative atomicity in updates. If we assume that the snapshot itself, along with possibly a version name/number (from, to), is basically sufficient meta-data, that's quite advantageous. It becomes the responsibility of the admin to not screw up by assigning the same name to multiple versions, etc. But as always, the tricky thing is someone actually doing the work. It's a non-trivial and time-consuming task, and isn't just a weekend's hacking. There will almost certainly be a number of "Oh damn, I didn't think of that!"'s along the way. I don't think anyone is opposed to it happening, but getting someone to do all that work with little recompense (or even significant recompense) requires some amount of convincing! Robert N M Watson
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20060114144115.M38874>