From owner-freebsd-libh Thu Nov 2 11:18:16 2000 Delivered-To: freebsd-libh@freebsd.org Received: from oberon.dnai.com (oberon.dnai.com [207.181.194.97]) by hub.freebsd.org (Postfix) with ESMTP id E8D5037B479 for ; Thu, 2 Nov 2000 11:18:10 -0800 (PST) Received: from neptune.dnai.com (neptune.dnai.com [207.181.194.93]) by oberon.dnai.com (8.9.3/8.9.3) with ESMTP id LAA95977; Thu, 2 Nov 2000 11:18:10 -0800 (PST) Received: from acm.org (207-172-166-25.s25.tnt1.sfrn.ca.dialup.rcn.com [207.172.166.25]) by neptune.dnai.com (8.9.3/8.9.3) with ESMTP id LAA86144; Thu, 2 Nov 2000 11:18:05 -0800 (PST) Message-ID: <3A01BDA9.C637C0AA@acm.org> Date: Thu, 02 Nov 2000 11:16:57 -0800 From: Tim Kientzle Reply-To: kientzle@acm.org X-Mailer: Mozilla 4.5 [en] (X11; I; FreeBSD 3.3-RELEASE i386) X-Accept-Language: en MIME-Version: 1.0 To: "Daniel C. Sobral" Cc: Patrick Bihan-Faou , libh@FreeBSD.ORG Subject: Re: Making the Packages System Better References: <39DCC860.B04F7D50@acm.org> <20001006155542.A29218@cichlids.cichlids.com> <39F3CDD7.15B889E7@acm.org> <20001023190412.B507@cichlids.cichlids.com> <39F47E98.4BB647AA@acm.org> <20001023202244.B10374@cichlids.cichlids.com> <39F48F4A.38D458C2@acm.org> <39FCF244.5A8C8E59@newsguy.com> <39FDC12E.304B0011@acm.org> <39FE2406.150CA3B1@newsguy.com> <00cb01c042f1$1a347190$040aa8c0@local.mindstep.com> <39FE562C.714DBE7C@newsguy.com> <39FFCD73.7364C2BF@acm.org> <39FFE5B7.8FC4216B@newsguy.com> <3A0063F5.4798055B@acm.org> <3A00CC6B.39522CF0@newsguy.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-libh@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG "Daniel C. Sobral" wrote: > I plain don't like having dozens of directories under a single path. You could, of course, have /usr/packages mimic the layout of /usr/ports. That would largely address that issue and more tightly bind the ports and packages collections. I think a flat heirarchy is more appropriate here, but I could be convinced otherwise. Personally, I generally prefer more structured heirarchies, but the purpose of any directory organization is to simplify locating and/or managing files. Sometimes a flat heirarchy is the best way to do that. There are many directories in a standard FreeBSD system whose listings don't fit on one screen. > Furthermore, the main advantage you claim is forfeiting the need for the > database of installed files, but ... these > directories of symlinks are a database in themselves, only with > disadvantages. Here's an exercise: * You type the following on your system: rm -rf /var/db/pkg/* * I'll type the following on mine: rm -rf /usr/packages/{bin,lib} Now, who can recover more quickly? The symlink "database" is purely derived data, and hence can always be completely reconstructed from the filesystem. The package database is additional data on top of the filesystem; it cannot be easily reconstructed and the filesystem information is considerably less useful without it. On of my big concerns with the current package database is that there is no mechanism that enforces consistency between the database and the filesystem to which it refers. As a result, inadvertent change to either one can make the whole system much less effective. I've also seen people develop "object-oriented" filesystems which relied on an auxiliary database of information about each file; such systems tend to be brittle (files get moved or renamed without the database being updated). The fact that the current packages install by default into /usr/local which is assumed by so many source-code distributions makes it rather easy to change the filesystem without corresponding changes to the package database. Using link dirs, you simply update those directories; with a database, there's no easy recovery. (Imagine someone who installs something from the package collection, then installs the next version directly from source code; subsequent versions often dramatically change the file lists. The package/file information in the database now has many errors and there's no easy way to fix it.) Put simply: The package database is subject to irreversible bit-rot; the link directory is not. > Another advantage you claim is name clash avoidance. Again, having > symlinks negates this advantage. Nothing, of course, can completely eliminate name clashes. With a single shared /usr/local, name clashes must be handled at install time by deleting one of the offending files. (Or through some sort of backup mechanism.) Either makes it difficult to address the issue later on. Also, name clashes can be reduced because not every file needs to be linked; only publically-visible files. Private executables, data files, configuration files remain in the package directories and therefore avoid name clash issues. Thus, the system I propose provides another way to deal with name clashes; one that is not available with the current system. > It does make it easy to "test-upgrade" a package, *IF* you solve the > symlink name clash problem, which you didn't. One easy way to address this is to only link the most recent version of a package. (e.g., if it finds foo-3.4 and foo-3.5 it can only link foo-3.5) With this convention, tentative upgrades and version rollbacks are routine. With the current system, a tentative upgrade of a package requires repeated de-installs and re-installs, losing any customized settings in the process. Alternatively, it requires a backup system to be implemented within the package tools. From where I sit, that looks like an awfully complex answer to a simple problem. > OTOH, you *CAN* test-install a port on an alternate directory of > your choice, without any problems. Though this does precludes use > of pre-compiled binaries for this purpose, you'll notice that many > people never use pre-compiled packages at all. That's true, you can circumvent any limitation of the package system by simply not using it. What I'm doing is trying to make the package system itself more useful so that more people can use it. > Also, symlinks make it harder to "eye-ball" things like /usr/local/etc > when searching for problems, as they do not contain all information a > file does (particularly access time and access permissions). Having separate package directories does change things, yes. Instead of looking at "all configuration files in /usr/local/etc", you look at "all files associated with package foo". The current /usr/local approach leads naturally to the first, the separate directory-per-package approach leads naturally to the second. Each has it's benefits and drawbacks. In my experience, it's more common to discover "something is wrong with emacs" than to have a user report that "some package is mis-configured". > Another big problem is that it would require retraining of personel and > invalidate all existing documentation on our ports systems, and you can > bet *THAT* will make people very pissed off. I never thought I'd see the day that an open-source project requires discussion of "retraining of personel". My, how times change. ;-) I'm actually suggesting no changes to the ports system at all. Very little documentation should require changing. Ports that are PREFIX-clean will require no changes at all. Only the package tools require any changes. > If you give it up, for example, you > cannot find out if a file is part of a package or not, as it's presence > in a directory is assumed to mean it is part of that package. Yes, being in a certain directory is equivalent to being a part of that package. This makes it very easy to tell if a particular file is part of a particular package. On the other hand, it also largely renders such a question moot. With the current system, you sometimes need to know whether a particular file is part of a package so that you can manually take steps to preserve it. A less-experienced sysadmin is likely to delete that file and then later find out it was a bad idea. Automatic backup tools such as Jordan has been proposing don't save you from that. I concede that some form of package database is probably a good thing, even with my proposed scheme, since it does allow you to track the inevitable exceptional cases. But it's no longer an absolute requirement. > The _only_ advantage your method bring is that people who like it like > it. Well, people who don't like it, don't like it. And people who like > the present method like the present method. I believe that many people who "like the present method" have either not yet encountered some of its limitations or never considered alternatives; that's my whole reason in continuing this discussion. > ... and proposes a way of doing things I plainly don't like, > having used it elsewhere. I've also used Solaris' "/opt" convention, and I don't like it either. > Most people are HAPPY with the present way of doing things. Earlier, you claimed there were many people who never use the package tools. I suspect those people aren't all that happy with them. ;-) I used a simpler of my proposed system for several years, then tried using the package tools for a few years. The package tools are convenient, but /usr/local tends to become unmanagable after a few years. > If you want to solve name clashes, test-upgrades, faster installs > and the like, there are plenty paths to take within the existing > framework, which would make it much more likely the adoption of the code. Please explain how to fix the current tools. Your solution above to tentative upgrades was to not use pre-compiled packages. I've read Jordan's paper, and his suggestion of a complete versioning database on top of /usr/local sounds both needlessly complex and very brittle. (Certainly more complex than building shadow heirarchies and symlinks.) My most serious concern with the current approach relates to the lack of any enforced relation between the package database and the filesystem. There's no real solution for that, other than to contradict several decades of standard Unix practice and not use /usr/local for local software. Such a solution seems unenforcable to me. (The other solution is to move /usr/local off of ufs and onto a custom filesystem, but that seems even less practical, although no doubt someone has considered it.) At the very least, the default PREFIX should be changed to something other than /usr/local. That would reduce the brittleness of the current system. It wouldn't address many other issues, of course. - Tim To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-libh" in the body of the message