Date: Fri, 03 Nov 2000 11:19:09 +0900 From: "Daniel C. Sobral" <dcs@newsguy.com> To: kientzle@acm.org Cc: Patrick Bihan-Faou <patrick@rageagainst.com>, libh@FreeBSD.ORG Subject: Re: Making the Packages System Better Message-ID: <3A02209D.5E7137F8@newsguy.com> References: <39DCC860.B04F7D50@acm.org> <20001006155542.A29218@cichlids.cichlids.com> <39F3CDD7.15B889E7@acm.org> <20001023190412.B507@cichlids.cichlids.com> <39F47E98.4BB647AA@acm.org> <20001023202244.B10374@cichlids.cichlids.com> <39F48F4A.38D458C2@acm.org> <39FCF244.5A8C8E59@newsguy.com> <39FDC12E.304B0011@acm.org> <39FE2406.150CA3B1@newsguy.com> <00cb01c042f1$1a347190$040aa8c0@local.mindstep.com> <39FE562C.714DBE7C@newsguy.com> <39FFCD73.7364C2BF@acm.org> <39FFE5B7.8FC4216B@newsguy.com> <3A0063F5.4798055B@acm.org> <3A00CC6B.39522CF0@newsguy.com> <3A01BDA9.C637C0AA@acm.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Tim Kientzle wrote:
>
> Personally, I generally prefer more structured heirarchies, but
> the purpose of any directory organization is to simplify locating
> and/or managing files. Sometimes a flat heirarchy is the best
> way to do that. There are many directories in a standard FreeBSD
> system whose listings don't fit on one screen.
Me too. Why is it that to me this means cd /usr/local/etc to configure
programs, and to you it means cd /usr/packages/X/etc; configure; cd
/usr/packages/Y/etc; configure; etc...?
> Here's an exercise:
> * You type the following on your system: rm -rf /var/db/pkg/*
> * I'll type the following on mine: rm -rf /usr/packages/{bin,lib}
> Now, who can recover more quickly? The symlink "database"
I'll tell you when such an accident happens to me. Until then, this
exercise is futile.
> is purely derived data, and hence can always be completely reconstructed
> from the filesystem. The package database is additional data
> on top of the filesystem; it cannot be easily reconstructed and
> the filesystem information is considerably less useful without it.
BTW, a hacker inserted some trojan binaries among the installed
packages. Who can find out faster those trojans?
> On of my big concerns with the current package database is
> that there is no mechanism that enforces consistency between
> the database and the filesystem to which it refers. As a result,
> inadvertent change to either one can make the whole system
> much less effective. I've also seen people develop "object-oriented"
> filesystems which relied on an auxiliary database of information about
> each file; such systems tend to be brittle (files get moved or
> renamed without the database being updated). The fact that the
> current packages install by default into /usr/local which is
> assumed by so many source-code distributions makes it rather
> easy to change the filesystem without corresponding changes to
> the package database. Using link dirs, you simply update those
> directories; with a database, there's no easy recovery. (Imagine
> someone who installs something from the package collection, then
> installs the next version directly from source code; subsequent
> versions often dramatically change the file lists. The package/file
> information in the database now has many errors and there's no easy
> way to fix it.)
Err, no. The versions keep their file list, so there is no error. Of
course, one version has probably overwritten other version's files, but
that is an implementation problem. The upgrade must correctly remove the
previous version and install the new version, and make any format
corrections or preserve configuration files. That this doesn't work has
_nothing_ to do with the database being used.
> Put simply: The package database is subject to irreversible bit-rot;
> the link directory is not.
I'll grant you that.
> > Another advantage you claim is name clash avoidance. Again, having
> > symlinks negates this advantage.
>
> Nothing, of course, can completely eliminate name clashes. With
My point.
> a single shared /usr/local, name clashes must be handled at
> install time by deleting one of the offending files. (Or through
> some sort of backup mechanism.) Either makes it difficult to address
> the issue later on. Also, name clashes can be reduced because not
I'd rather address the problem at _install_. *IF* a name clashes, and
this has never happened to me, I'd rather be notified then and there,
and not have the install proceed until some has been done about it.
> every file needs to be linked; only publically-visible files.
> Private executables, data files, configuration files remain in the
> package directories and therefore avoid name clash issues. Thus,
Duh! Like, say, we presently do? You *DID* realize that we use
directories named after the port for private files, didn't you?
> > It does make it easy to "test-upgrade" a package, *IF* you solve the
> > symlink name clash problem, which you didn't.
>
> One easy way to address this is to only link the most recent version
> of a package. (e.g., if it finds foo-3.4 and foo-3.5 it can only link
Clashes between the same version is an upgrade target problem, not a
general problem.
> are routine. With the current system, a tentative upgrade of a package
> requires repeated de-installs and re-installs, losing any customized
> settings in the process. Alternatively, it requires a backup system
Unless, of course, you install them in a temporary directory for test?
> to be implemented within the package tools. From where I sit, that
> looks like an awfully complex answer to a simple problem.
The funny thing is that this is what your proposal looks like to me. :-)
> > OTOH, you *CAN* test-install a port on an alternate directory of
> > your choice, without any problems. Though this does precludes use
> > of pre-compiled binaries for this purpose, you'll notice that many
> > people never use pre-compiled packages at all.
>
> That's true, you can circumvent any limitation of the package system
> by simply not using it. What I'm doing is trying to make the
> package system itself more useful so that more people can use it.
cd /usr/ports/mail/elm; PREFIX=/testinstall make all install
How's that not using the package system?
> approach leads naturally to the second. Each has it's benefits and
> drawbacks. In my experience, it's more common to discover "something is
> wrong with emacs" than to have a user report that "some package is
> mis-configured".
In my experience it is more often necessary to make changes to various
packages to reflect an environmental change than change multiple
configuration files of a single package.
> > Another big problem is that it would require retraining of personel and
> > invalidate all existing documentation on our ports systems, and you can
> > bet *THAT* will make people very pissed off.
>
> <chuckle> I never thought I'd see the day that an open-source
> project requires discussion of "retraining of personel". My, how
> times change. ;-)
Huh? You think the people who have 4000 FreeBSD box installed has any
less of a support/maintenance problem than the people who have 4000
Windows NT box installed? They both need to have people trained to give
user support, to solve simple problems, and generally keep things going.
> I'm actually suggesting no changes to the ports system at all.
> Very little documentation should require changing. Ports that
> are PREFIX-clean will require no changes at all. Only the package
> tools require any changes.
You completely miss the point. Go back up there and think again.
> > The _only_ advantage your method bring is that people who like it like
> > it. Well, people who don't like it, don't like it. And people who like
> > the present method like the present method.
>
> I believe that many people who "like the present method" have
> either not yet encountered some of its limitations or never considered
> alternatives; that's my whole reason in continuing this discussion.
Libh is not the place to do it. This is a small backwater list,
subscribed by just a few people, most of whom have a lot experience with
the different ways of dealing with packages used by everything from NT
to Debian.
> > ... and proposes a way of doing things I plainly don't like,
> > having used it elsewhere.
>
> I've also used Solaris' "/opt" convention, and I don't like it either.
Actually, I have little experience with Solaris. :-)
> > Most people are HAPPY with the present way of doing things.
>
> Earlier, you claimed there were many people who never use the
> package tools. I suspect those people aren't all that happy
Because they prefer to use ports. Ports is the way of doing things,
packages is a side-kick for quick and dirty results.
My remark stands.
> > If you want to solve name clashes, test-upgrades, faster installs
> > and the like, there are plenty paths to take within the existing
> > framework, which would make it much more likely the adoption of the code.
>
> Please explain how to fix the current tools. Your solution above
> to tentative upgrades was to not use pre-compiled packages. I've
> read Jordan's paper, and his suggestion of a complete versioning
> database on top of /usr/local sounds both needlessly complex and
> very brittle. (Certainly more complex than building shadow heirarchies
> and symlinks.)
Pre-compiled packages suck. They are, as far as I'm concerned, a
non-issue, because they simply CANNOT be targetted at _your_
environment, compiled to take advantage of _your_ hardware, use the libs
that _you_ have installed, the defaults _you_ like, etc.
"Package installation," for me, is cd /usr/ports/Category/Application;
make all install clean. Optimize for pkg_add is optimizing for the wrong
target.
You solve name clashes at the port, because that's really the only way
of doing it. Otherwise, EVERYONE has to deal with it, individually.
Upgrades you solve by keeping a list of what files need to be taken
special attention of, scripts to handle them if necessary, and just
paying attention to what IS in the database before installing.
Faster installs are dealt with by removing the requirement of using a
temporary directory, and this is done by using a smarter package format.
> My most serious concern with the current approach relates to the
> lack of any enforced relation between the package database and
> the filesystem. There's no real solution for that, other than
> to contradict several decades of standard Unix practice and
> not use /usr/local for local software. Such a solution seems
Err... /usr/local is NOT several decades of standard Unix practice. If
you search the mailing lists, you'll find that /usr/local has been used
for DIFFERENT purposes by different vendor and people. :-)
I'll grant you that the solution you propose is the only one available
if your most serious concern is the enforced relation between the
package database and the filesystem.
Since this is not a concern of mine, it's understandable that we see
things different.
> unenforcable to me. (The other solution is to move /usr/local
> off of ufs and onto a custom filesystem, but that seems even
> less practical, although no doubt someone has considered it.)
I don't see how any custom filesystem could help, as a matter of fact.
> At the very least, the default PREFIX should be changed to something
> other than /usr/local. That would reduce the brittleness of the
> current system. It wouldn't address many other issues, of course.
Huh? Why something other than /usr/local?
If you claim /usr/local is the place to install locally compiled
software, you'll find plenty people disagreeing with you.
And it would most certainly be easier for you to change such perception
than for the hundreds of thousands of FreeBSD users to adapt to a new
PREFIX.
--
Daniel C. Sobral (8-DCS)
dcs@newsguy.com
dcs@freebsd.org
capo@world.wide.bsdconspiracy.net
He has been convicted of criminal possession of a clue with intent to
distribute.
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-libh" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3A02209D.5E7137F8>