Date: Fri, 9 May 2008 19:06:33 +0200 From: Joerg Sonnenberger <joerg@britannica.bec.de> To: freebsd-hackers@freebsd.org Subject: Re: Adding .db support to pkg_tools Message-ID: <20080509170633.GB3571@britannica.bec.de> In-Reply-To: <op.uawbpwud2n4ijf@duckjen.nextgentel.no> References: <op.uavxx8ip2n4ijf@duckjen.nextgentel.no> <20080509124308.GA596@britannica.bec.de> <op.uawbpwud2n4ijf@duckjen.nextgentel.no>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, May 09, 2008 at 06:50:10PM +0200, Anders Nore wrote: > Yes that would probably be bad for the database, but I'm sure one can > manage to get around this problem by copying it before changing the db and > delete the copy if it doesn't fail. At the next time executed it will check > for a copy, use that and assume that the last run was unsuccessful. /var/db/pkg contains 10MB for the various packages installed on my laptop and 10MB for the cache of +CONTENTS. You don't want to copy that around all the time. >> Secondly, I would also advisy against just storing all meta data in a >> single key/value pair. For example, +CONTENTS can be extremely large. >> Check texmf for a good example. > > When it comes to storing large values in a key/value pair, I think that's > what bdb was designed for, handling large amounts of data (in the orders of > gigabytes even in key's) fast. No, actually that is exactly what it was *not* designed for. Having billions of keys is fine, but data that exceeds the size of a database page is going to slow down. While it might not be a problem of you are copying the data to a new file anyway, it also means that fragmentation in the database will be more problematic. My main point is that for the interesting operations you want to actually look up with fine grained keys and that's what is not possible if you store the meta data as blob. In fact, storing the meta data as blob is not faster than just using the filesystem. Joerg
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20080509170633.GB3571>