Date: Sat, 14 Sep 2013 00:01:45 +0200 From: Matthias Andree <mandree@FreeBSD.org> To: freebsd-ports@freebsd.org Subject: Re: Berkeley DB 4.1 Message-ID: <52338B49.3020303@FreeBSD.org> In-Reply-To: <52336365.2020702@gmx.de> References: <l0s6lr$6o0$1@ger.gmane.org> <52336365.2020702@gmx.de>
next in thread | previous in thread | raw e-mail | index | archive | help
Am 13.09.2013 21:11, schrieb olli hauer: > On 2013-09-12 12:57, Ivan Voras wrote: >> Hello, >> >> I'd like to start a discussion on changing the default BDB port from 4.1 >> to something more recent. >> >> bdb version 4.1 was last released in 2002: >> README: Sleepycat Software: Berkeley DB 4.1.25: (December 19, 2002) >> >> There are some ports which have an unexpected dependacy on bdb via APR >> (apache22, subversion), which does is not wrong in itself, but is >> somewhat unelegant (as a personal opinion of course). >> >> I've found this previous discussion: >> http://lists.freebsd.org/pipermail/freebsd-ports/2013-February/081444.html >> >> And while the argument seems valid, it also doesn't have an estimate of >> which / how many ports will break with a more recent bdb. >> >> Could an experimental port build be done with setting WITH_BDB_VER to >> either the most recent 4.x version (WITH_BDB_VER=48), to the last >> Sleepycat Licensed version (50) or the recent version in ports (60) to >> see what breaks? >> > > Long time ago I had a short discussion with mandree@ about a bump to 4.8. > 4.8 should be supported by most ports (also long time the default in many > linux distributions) > > At last one argument I haven't seen in the discussion is "data migration", > for example amavisd-new provides maintenance and backup / recover routines > but other ports ?? > Greetings, I'll pick Olli's post as a stepping stone, more or less randomly. My thoughts on this are (more or less random): - sorry my efforts to axe anything older than db5 stalled on my side. I always meant to have that -exp run done with - as a first step - the default selection logic twisted in a way that anything that states "41+" or similar meant "5+", but somehow no time felt right to me because I shied away from meddling with releases. - about application compatibility with the library (at build time), it depends on how sophisticated interfaces you use. Berkeley DB has a rather rich feature set, is customizable, and so on. Many applications just use BDB as a key/value datastore. You can, however, go all the way to a concurrent and transactional database that gets replicated (I never used the latter, and never received a feature request to enable it). For applications using these more advanced features, there are more API changes that require version comparison. In bogofilter's earlier days (bogofilter is also about a decade old), we had to add DB version checks to support a new BDB version rather often, but the last half dozen or so versions did not change function signatures that bogofilter used. - about data migration, this is not quite as hard. Newer library versions are supposed to read database files written by older versions; usually the on-disk database format remained unchanged. What changed often, though, is the "environment", that is the __db.001-like stuff that keeps locking tables for concurrent access, and thereabouts. The log files for databases in transactional mode also changed format on more than one occasion. However, for applications that can be cleanly shut down (which is more a logistic than a software problem), usually the upgrade path is: 1. shut down application, 2. make sure that no pending corruptions are there, in doubt, run quick recovery -- basically the idea is to roll the log file changes out into the database, quite similar to having a journalling file system write its journal data back into the actual structures. This is the crucial point - the log must have been fully processed, and all pending transactions committed to the .db file, or rolled back. 3. remove the environment (this leaves log files intact) 4. upgrade 5. restart application. BDB will then happily recreate the environment, bump the logfile number, and start writing into the new. The only thing is there's no way back unless there are database updates. Again, I don't know about replicated databases. This is surely in the BDB docs somewhere and I did not care to read it ;-) The downside is that upgrades require manual interaction, and the ports systems is not rigged up to support the user in any way for such upgrades (this is not specific to Berkeley DB) - at the time when I first looked, there were ~17 ports that hardcoded a particular BDB version, among them the high-profile openldap* ports. - don't get me started about the abundant autoconf/configure.ac abuse though. Many build systems do a stupid quadratic search trying to match a db header to a db library. Too much, and globbing would speed things up. I meant to write an autoconf macro, but alas, no time for that either. I think once 9.2 is out the door we should mark any db4* version deprecated, and I am inclined to hack all db4* ports to refuse building on 10-CURRENT, and make 5 the default, so that all practical 10-RELEASE systems will be free of db4*. Thoughts?
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?52338B49.3020303>