Date: Sun, 21 Jan 2018 13:15:57 -0800 From: "Ronald F. Guilmette" <rfg@tristatelogic.com> To: Paul Mather <freebsd-lists@gromit.dlib.vt.edu> Cc: freebsd-questions@freebsd.org Subject: Re: Splitting up sets of files for archiving (e.g. to tape, optical media) Message-ID: <81742.1516569357@segfault.tristatelogic.com> In-Reply-To: <2456E55A-D14F-41ED-B8DD-9633BD73ACF1@gromit.dlib.vt.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
In message <2456E55A-D14F-41ED-B8DD-9633BD73ACF1@gromit.dlib.vt.edu>, Paul Mather <freebsd-lists@gromit.dlib.vt.edu> wrote: >Have you looked at fpart (https://github.com/martymac/fpart)? It looks >to me like it is applicable to your problem (which sounds to me like a >variant of the bin packing problem). The fpart README even lists >packing music files onto fixed-size DVD media as one of its examples, >which sounds close to the archiving scenario you give above. Plus, it >claims to be developed on FreeBSD. I had never heard of that tool (fpart) till now, but I have just now skimmed the README file for that and yes, this tool appears to be quite close to what I had in mind. I will have to see if I can make contact with the developer of this tool, and discuss with him what I have been doing, whch is similar but different. (Maybe he will be motivated to include some suggestions from me.) The problem, as I personally have approached it, is indeed a bin packing problem, but it is not clear (from the README) that this fpart tool even attempts to do bin packing, i.e. in order to minimize the number of output "partitions". (That's an important feature if you are trying to minimize the number of archive volumes.) Also, whereas this (fpart) tool is clearly oriented towards creating output "partitions" which will then be the basis for subsequent rsync operations, in my case each group of files comprising one single output blob gets hard-linked into a newly created directory representing that specific output blob (where these new directoties have rather unimaginative names like "00", "01", "02"...) Since these directores are on the same partition as the input files to be archived, and since that whole partition is exported to my local network (courtesy of Samba), I can then just pop over to my Windoze system and use ImgBurn to write each output blob in turn to a blank BD-R disk. (I am actually doing this as we speak.) Where things really get tricky is where one tries to really and truly maximize the usage, down to the very last sector, of the output (archive) volumes. To do this perfectly, one would have to have deep knowledge of -exactly- how ImgBurn formats a set of input files and turns them into either a UDF or an ISO 9660 image. I don't have such knowlegde, at present, so I am forced to add small fudge factors in my calculations so that each of my output blobs will in fact fit onto a single blank BD-R, at least as far as ImgBurn is concerned. Even so, and even using a simple-minded bin packing algorithm, I am generally able to fill up output volumes to 99.9% or better. Anyway, thank you Paul, for pointing me at fpart. I quite definitely knew nothing about it till now... and that's why I posted my queation here in the first place, because there's such a wealth and breadth of knowledge here. Regards, rfg
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?81742.1516569357>