Date: Wed, 7 Aug 2002 16:23:13 -0400 From: Antoine Beaupre <anarcat@anarcat.ath.cx> To: libh@freebsd.org Subject: package format and creation Message-ID: <7F90C363-AA43-11D6-9D65-0050E4A0BB3F@anarcat.ath.cx>
next in thread | raw e-mail | index | archive | help
Hi. Yesterday night I was bored and I wrote up a little something about package formats and sources. Semantics ========= A "package format" is defined here defined as the "media" of the package itself. In the pkg_tools days, it's the .tgz with +METADATA. Right now, in libh, this is a zip file with METADATA/*. But what are the actual possibilities? Why restrict libh to a single format? This would merely make libh more vulnerable to its detractors. As for "package source", I mean by that how the package itself is created. In the pkg_tools scheme, it's the "make install" and pkg_create procedure directed by a plist, and it's quite obscure, IMHO. In the current libh scheme, well, I'm not sure it's even that well defined. People can reference release/scripts/pkgtools/makepackage.sh to see what's the current procedure, but this script shouldn't be considered as how things *should* be, because, obviously, libh new package system will need a re-work in the ports system. Anyhow, right now, this package creator makes an install in a temporary directory and uses the plist to add files to the package. So here are the possibilities I thought about: Package formats =============== Regardless of the physical file format and location, the content of the package should be the same: some meta-data in a SYSINSTALL (or whatever) directory, and the "data files" (files to be installed) in the root of the file. Here are the considered package formats: 1- zip file (current package format): allows comment and customizable inline archive properties, random access and moderatly powerful compression 2- FS image (ISO, FFS or else, possibly compressed): random and fast access, compression dependent on the system's facility to mount compressed images, might be requiring temporary extraction space 3- .tar.[gz|bz2]: no random access without temporary extraction, powerful compression 4- .foo (insert you favorite archive scheme): must allow good compression, preferably accessible through library calls, preferably also allow random access It's been originally thought the zip file was the best choice. I still think it's a pretty good choice, but the FS image format is very attracting too. At any rate, it is clear for me now that libh will have to *architecturally* abstract the package format. Right now, it's too much integrated in the system and a bit of changes will have to be done to libh to eventually support other formats. Package sources =============== How is a package created? The current pkg_tools scheme is not good enough since it requires installing *and* registering the package in the system to package it, which is a bit absurd. The current libh scheme is a bit better since it installs the thing in a temporary location. We can do better. The PackageCreator class concept is in itself interesting. It is a class used to create a package (duh) from an already constructed file hierarchy. The class itself isn't very well done, but it has at least one interesting thing in the fact that it has an ignore list. It can skip files matching a certain pattern in the directory. This made me think. We can improve that design even better by using some kind of hash. The hash would map the location of the compiled files in the working directory to where they belong in the package (or in the tree). For example, in package foo-x.y: $ ls /usr/ports/cat/foo/work/foo-x.y README TODO configure foo foo.1 foo.c foo.o Makefile Makefile.in $ The "mapping file" (that would map the working files with their installed location) would look something like this: $ cat mapping foo:bin/foo:BIN foo.1:man/man.1/foo.1:MAN README:doc/foo/README:DOC TODO:doc/foo/TODO:DOC $ The DOC/BIN/MAN here could be something to tell which sub-package the file belongs to. Heck, it could even be possible to compute the "bin/foo" path simply from wether the file belongs to a "DOC" or "BIN" sub package. So yes, that would mean that we wouldn't depend on the package's install method. And if we really want to depend on it, we could make the mapping file support this too. Say we install the package in a staging directory "/usr/ports/staging/foo", and that the files are all properly placed there. And say we hack globbing support into the mapping file, that would end up being: $ cat mapping *:* $ or simply "*". Or better: $ cat mapping bin/*::BIN doc/*::DOC man/*::MAN etc/*::ETC $ This could even be default! Conclusion ========== I think there's a lot of work to be done there. Libh folks need to sit down with ports folks and figure out how this would be done best. One thing is for sure: in order to accomodate libh's higher-level package function, the port systems will need a big face lift or it (or libh!) won't survive. For the package formats, it's clear it's all bikeshedding fodder, and I'd be surprised if these suggestions lead anything more than flamewar, but at least I recorded my intentions somewhere. And as for the whole thing, I'd be surprised if it would spark a big discussion on libh, given the traffic it had on the recent times, so I'll probably post a reference of the mail to -arch and/or -ports. A. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-libh" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7F90C363-AA43-11D6-9D65-0050E4A0BB3F>