Date: Sun, 7 Jul 2002 22:00:43 -0400 From: Garance A Drosihn <drosih@rpi.edu> To: Wes Peters <wes@softweyr.com>, Dan Moschuk <dan@FreeBSD.ORG> Cc: arch@FreeBSD.ORG Subject: Re: Package system flaws? Message-ID: <p05111745b94e9452f3b3@[128.113.24.47]> In-Reply-To: <3D27A296.D58FB4B4@softweyr.com> References: <20020706220511.GA88651@scoobysnax.jaded.net> <3D27A296.D58FB4B4@softweyr.com>
next in thread | previous in thread | raw e-mail | index | archive | help
At 7:08 PM -0700 7/6/02, Wes Peters wrote:
>Dan Moschuk wrote:
> > I'm not sure that this classifies as an architectual discussion
> > (for now) so if you feel its appropriate please feel free to
> > redirect to ports@.
>
>I'd certainly like to see it evolve into an architectural
>discussion, even if all it accomplishes is to layout the work
>that needs to be done and provide some sort of road map of what
>the next step or two might be.
There is probably room for several different architectural
discussions.
One is whether we use tar vs some other archive format to hold
the data and the metadata for each port. I agree that is an
important topic, but I have no strong opinions on it. Whatever
is decided upon by others would probably work fine for me.
I do find the ports collection one of the most useful things in
the freebsd community, but the more comfortable you get with
depending on it, the more you see little cracks where various
problems crop up -- over and over again.
- - - -
Separate from that, one thing I think we need is some mechanism
which makes certain that the generated package-list for a port
is exactly correct. I install about 70-80 ports, and I know
several of them install files which are not included in their
package-list. Try running
find /usr/local /usr/X11R6 -type f -print0 | \
xargs -0 /usr/local/sbin/pkg_which -v | fgrep '?'
Whatever we do for this, it should be something which is
simple enough to use that it does not add work for the ports
maintainers.
Conceptually what I would like to do is have ports *built* for
/usr/local (just as they are now), but then when it comes time
to do the make-install, the make-install would do something
like:
(chroot /usr/local/TEMP ; make install)
and then copy all files from /usr/local/TEMP/usr/local
to the real /usr/local, based on the package-list for
the port.
It obviously wouldn't be quite *that* simple to do, but the
idea is that you could be certain that the package list
included every file that was actually installed. It would also
be easy to see every file which SHOULD have been installed,
because they would all be still sitting in the
/usr/local/TEMP/usr/local directories. I believe that openbsd
has something like this, but I do not know the details.
I have also thought that maybe it could be done with Kirk's
filesystem snapshots, where you'd do a snapshot before building
a port, and another one afterwards, and find out what files it
changed. This would be a rather heavy-handed way to approach
the problem, but it avoids some of the technical issues that
come up with the chroot approach.
- - - -
Portupgrade is a great step forward in rebuilding ports, but it
also has it's limitations. It has an idea of dependences between
ports, but it only knows about the dependences as of the last time
the port was built. If progAA was built a month ago, and at that
time it only depended on progBB, then 'portupgrade -Rr -n progAA'
will only consider progAA and progBB. If the *new* version of
progAA (the one you will build) now depends on progCC, you will
not find out about that until portupgrade does the 'make', and
it turns around to build progCC.
This also shows up if you have NOCLEANDEPENDS=true in your
/etc/make.conf file. Portupgrade will do a:
make clean ; make ; make install; make clean
for every port that it builts -- but it only knows to do that for
the ports that it *already* knows about. If you ask it to install
progAA, that depends on progBB and progCC, then those 'make clean's
are only done for progAA. The work-directories for progBB and
progCC are not cleaned out before they are made, and they are
not cleaned out after they are installed.
- - - -
Portupgrade also looses track of what dependencies are, as various
things get rebuilt. I just pkg_deinstall'ed a number of ports,
and pkg_deinstall missed a good number of package dependencies. By
that I mean it quite happy removed some port, even though there
were other ports still installed which in fact depended on that
port. [that was fine for me in this case, but it was still a
little disappointing...]
That's a problem when things get new versions. It gets even more
confusing when ports are renamed, or entirely removed.
- - - -
The current ports mechanism has the idea "this port depends on
file XYZZY, and if that file does not exist then you should first
build port PLOUGH". It also needs the idea that "this port
depends on XYZZY, but it must the XYZZY built by XFree86-4, and
not the one built by XFree86-3".
- - - -
As we move into more platforms (sparc64, ia-64, ppc), then I
expect that will add even more to the work of the ports
collection.
- - - -
In addition to the above issues, there is also the issue of how
large and unwieldy the ports collection is, and how it's "one
single giant collection" (as far as something like the INDEX is
concerned, for instance). It would be nice to rearrange it a
bit, such that I can easily clip off all foreign-language ports
that I know I don't want. Right now I can kinda do that by
changing my cvsup file, and changing my portupgrade-config
file, and editing /usr/ports/Makefile, and...
>I'm more interested in the binary package side personally, a
>holdover from a previous professional involvement in this area.
>I have a number of ideas for things that could be improved in
>relatively small projects if someone wants to discuss those
>with me.
I have had a number of ideas, but no time to really pursue
them. I think we try to stuff too much information into the
name of a port, and we try to do too much to shoehorn all
ports-processing into standard makefile variables and standard
make-cmd processing.
[in the background, the sound of people screaming is heard...]
And lastly, one of the problems with our ports and package system
is that it has grown so large without addressing some of these
issues. Any attempt to switch to a significantly different
mechanism would have to include a plan to EASE INTO the new
approach, probably over months of time. There is no way we are
going to convert 7,000 ports to any new mechanism, on multiple
HW platforms, on a wide-range of freebsd-versions, in some single
one-week cut-over period.
--
Garance Alistair Drosehn = gad@gilead.netel.rpi.edu
Senior Systems Programmer or gad@freebsd.org
Rensselaer Polytechnic Institute or drosih@rpi.edu
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?p05111745b94e9452f3b3>