Date: Thu, 02 Nov 2000 11:16:57 -0800 From: Tim Kientzle <kientzle@acm.org> To: "Daniel C. Sobral" <dcs@newsguy.com> Cc: Patrick Bihan-Faou <patrick@rageagainst.com>, libh@FreeBSD.ORG Subject: Re: Making the Packages System Better Message-ID: <3A01BDA9.C637C0AA@acm.org> References: <39DCC860.B04F7D50@acm.org> <20001006155542.A29218@cichlids.cichlids.com> <39F3CDD7.15B889E7@acm.org> <20001023190412.B507@cichlids.cichlids.com> <39F47E98.4BB647AA@acm.org> <20001023202244.B10374@cichlids.cichlids.com> <39F48F4A.38D458C2@acm.org> <39FCF244.5A8C8E59@newsguy.com> <39FDC12E.304B0011@acm.org> <39FE2406.150CA3B1@newsguy.com> <00cb01c042f1$1a347190$040aa8c0@local.mindstep.com> <39FE562C.714DBE7C@newsguy.com> <39FFCD73.7364C2BF@acm.org> <39FFE5B7.8FC4216B@newsguy.com> <3A0063F5.4798055B@acm.org> <3A00CC6B.39522CF0@newsguy.com>
next in thread | previous in thread | raw e-mail | index | archive | help
"Daniel C. Sobral" wrote:
> I plain don't like having dozens of directories under a single path.
You could, of course, have /usr/packages mimic the layout of
/usr/ports. That would largely address that issue and more tightly
bind the ports and packages collections. I think a flat heirarchy
is more appropriate here, but I could be convinced otherwise.
Personally, I generally prefer more structured heirarchies, but
the purpose of any directory organization is to simplify locating
and/or managing files. Sometimes a flat heirarchy is the best
way to do that. There are many directories in a standard FreeBSD
system whose listings don't fit on one screen.
> Furthermore, the main advantage you claim is forfeiting the need for the
> database of installed files, but ... these
> directories of symlinks are a database in themselves, only with
> disadvantages.
Here's an exercise:
* You type the following on your system: rm -rf /var/db/pkg/*
* I'll type the following on mine: rm -rf /usr/packages/{bin,lib}
Now, who can recover more quickly? The symlink "database"
is purely derived data, and hence can always be completely reconstructed
from the filesystem. The package database is additional data
on top of the filesystem; it cannot be easily reconstructed and
the filesystem information is considerably less useful without it.
On of my big concerns with the current package database is
that there is no mechanism that enforces consistency between
the database and the filesystem to which it refers. As a result,
inadvertent change to either one can make the whole system
much less effective. I've also seen people develop "object-oriented"
filesystems which relied on an auxiliary database of information about
each file; such systems tend to be brittle (files get moved or
renamed without the database being updated). The fact that the
current packages install by default into /usr/local which is
assumed by so many source-code distributions makes it rather
easy to change the filesystem without corresponding changes to
the package database. Using link dirs, you simply update those
directories; with a database, there's no easy recovery. (Imagine
someone who installs something from the package collection, then
installs the next version directly from source code; subsequent
versions often dramatically change the file lists. The package/file
information in the database now has many errors and there's no easy
way to fix it.)
Put simply: The package database is subject to irreversible bit-rot;
the link directory is not.
> Another advantage you claim is name clash avoidance. Again, having
> symlinks negates this advantage.
Nothing, of course, can completely eliminate name clashes. With
a single shared /usr/local, name clashes must be handled at
install time by deleting one of the offending files. (Or through
some sort of backup mechanism.) Either makes it difficult to address
the issue later on. Also, name clashes can be reduced because not
every file needs to be linked; only publically-visible files.
Private executables, data files, configuration files remain in the
package directories and therefore avoid name clash issues. Thus,
the system I propose provides another way to deal with name
clashes; one that is not available with the current system.
> It does make it easy to "test-upgrade" a package, *IF* you solve the
> symlink name clash problem, which you didn't.
One easy way to address this is to only link the most recent version
of a package. (e.g., if it finds foo-3.4 and foo-3.5 it can only link
foo-3.5) With this convention, tentative upgrades and version rollbacks
are routine. With the current system, a tentative upgrade of a package
requires repeated de-installs and re-installs, losing any customized
settings in the process. Alternatively, it requires a backup system
to be implemented within the package tools. From where I sit, that
looks like an awfully complex answer to a simple problem.
> OTOH, you *CAN* test-install a port on an alternate directory of
> your choice, without any problems. Though this does precludes use
> of pre-compiled binaries for this purpose, you'll notice that many
> people never use pre-compiled packages at all.
That's true, you can circumvent any limitation of the package system
by simply not using it. What I'm doing is trying to make the
package system itself more useful so that more people can use it.
> Also, symlinks make it harder to "eye-ball" things like /usr/local/etc
> when searching for problems, as they do not contain all information a
> file does (particularly access time and access permissions).
Having separate package directories does change things, yes. Instead
of looking at "all configuration files in /usr/local/etc", you look
at "all files associated with package foo". The current /usr/local
approach leads naturally to the first, the separate
directory-per-package
approach leads naturally to the second. Each has it's benefits and
drawbacks. In my experience, it's more common to discover "something is
wrong with emacs" than to have a user report that "some package is
mis-configured".
> Another big problem is that it would require retraining of personel and
> invalidate all existing documentation on our ports systems, and you can
> bet *THAT* will make people very pissed off.
<chuckle> I never thought I'd see the day that an open-source
project requires discussion of "retraining of personel". My, how
times change. ;-)
I'm actually suggesting no changes to the ports system at all.
Very little documentation should require changing. Ports that
are PREFIX-clean will require no changes at all. Only the package
tools require any changes.
> If you give it up, for example, you
> cannot find out if a file is part of a package or not, as it's presence
> in a directory is assumed to mean it is part of that package.
Yes, being in a certain directory is equivalent to being a part
of that package. This makes it very easy to tell if a particular
file is part of a particular package. On the other hand, it also
largely renders such a question moot. With the current system,
you sometimes need to know whether a particular file is part of a
package
so that you can manually take steps to preserve it. A less-experienced
sysadmin is likely to delete that file and then later find out
it was a bad idea. Automatic backup tools such as Jordan has been
proposing don't save you from that.
I concede that some form of package database is probably a good thing,
even with my proposed scheme, since it does allow you to track the
inevitable
exceptional cases. But it's no longer an absolute requirement.
> The _only_ advantage your method bring is that people who like it like
> it. Well, people who don't like it, don't like it. And people who like
> the present method like the present method.
I believe that many people who "like the present method" have
either not yet encountered some of its limitations or never considered
alternatives; that's my whole reason in continuing this discussion.
> ... and proposes a way of doing things I plainly don't like,
> having used it elsewhere.
I've also used Solaris' "/opt" convention, and I don't like it either.
> Most people are HAPPY with the present way of doing things.
Earlier, you claimed there were many people who never use the
package tools. I suspect those people aren't all that happy
with them. ;-) I used a simpler of my proposed system for
several years, then tried using the package tools for a few
years. The package tools are convenient, but /usr/local
tends to become unmanagable after a few years.
> If you want to solve name clashes, test-upgrades, faster installs
> and the like, there are plenty paths to take within the existing
> framework, which would make it much more likely the adoption of the code.
Please explain how to fix the current tools. Your solution above
to tentative upgrades was to not use pre-compiled packages. I've
read Jordan's paper, and his suggestion of a complete versioning
database on top of /usr/local sounds both needlessly complex and
very brittle. (Certainly more complex than building shadow heirarchies
and symlinks.)
My most serious concern with the current approach relates to the
lack of any enforced relation between the package database and
the filesystem. There's no real solution for that, other than
to contradict several decades of standard Unix practice and
not use /usr/local for local software. Such a solution seems
unenforcable to me. (The other solution is to move /usr/local
off of ufs and onto a custom filesystem, but that seems even
less practical, although no doubt someone has considered it.)
At the very least, the default PREFIX should be changed to something
other than /usr/local. That would reduce the brittleness of the
current system. It wouldn't address many other issues, of course.
- Tim
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-libh" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3A01BDA9.C637C0AA>