Date: Mon, 30 Oct 2000 23:14:45 -0500 From: "Patrick Bihan-Faou" <patrick@rageagainst.com> To: "\"Daniel C. Sobral\"" <dcs@newsguy.com> Cc: <libh@freebsd.org> Subject: Re: BOF at BSDCon: FreeBSD Installer, Packages System Message-ID: <00cb01c042f1$1a347190$040aa8c0@local.mindstep.com> References: <39DCC860.B04F7D50@acm.org> <20001006155542.A29218@cichlids.cichlids.com> <39F3CDD7.15B889E7@acm.org> <20001023190412.B507@cichlids.cichlids.com> <39F47E98.4BB647AA@acm.org> <20001023202244.B10374@cichlids.cichlids.com> <39F48F4A.38D458C2@acm.org> <39FCF244.5A8C8E59@newsguy.com> <39FDC12E.304B0011@acm.org> <39FE2406.150CA3B1@newsguy.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi,
> > Though I haven't tested it, I wouldn't be surprised if
> > the ports tree was more than twice as large in ZIP format as
> > in tar.gz format.
>
> It would surprise me. I think you are grossly overestimating tar.gz
> advantage. Anyway, the ports tree is very different from the source tree
> or the ports sources.
Well somebody sent in results that show exactly that...
> > Pick one. Personally, I think the most important issue is to
> > be able to build full-featured distributions, which means sticking
> > with high-compression formats such as tar.gz. Package installation
> > is not a particularly time-critical operation, so I don't see
> > fast single-file access as a sufficient imperative to justify
> > losing as much as 20% of the current CD-ROM distribution.
>
> Package installation can be painfully slow because of tar.gz problems.
> Compare the speed with which our packages are installed with the speed
> of the most popular Linux package formats. We can be two times slower
> than them.
Are you so sure that the tar.gz format is the culprit in the installation
slowness ? I build some custom packages using the "pkg_*" tools as they are
today. These packages are really large (30M) since they correspond to a full
(customized) FreeBSD install. Now the slow part is definitely not installing
the content of the package, the really really painfuly slow part is the
cleaning up of the temp directory that holds the uncompressed files. I think
that with a format like ZIP, this is what you are trying to optimize right ?
If I am not mistaken, most linux based packaging system also you the tar.gz
format, and if as you claim they are faster than the FreeBSD system, it only
means that they do it better.
One way to be fast would be as one suggested the "one directory per package"
approach. It is messy to set-up correctly if you do it manually, but this is
precisely why tools are nice when they exist...
> > > Besides, all paths in a package MUST be relative to the base
directory.
> > > I might want it installed on /usr/local, /opt, or even /. Thus,
> > > distribution formats need not be any different from package format.
> >
> > Having a package just be a list of instructions to be blindly followed
> > is acceptable as long as you assume everyone always pulls their packages
> > from a "trusted" source. My understanding was that people wanted
> > to move away from the assumption of a single, universally-available
> > trusted source. That doesn't mean the basic format for distributions
>
> Nope, as far as I know, we DO NOT want to move away from our model. I,
> and a lot of people I have heard from, consider the centralized model we
> use to be one of FreeBSD's main advantages over some Linux
> distributions. Then again, maybe the mood has changed from the last time
> I saw this issue discussed, but I'd rather believe to still be in the
> majority. :-)
>
> Besides, that doesn't really address what I said. It's the USER who
> specifies what should be done. The package says "install to /bin", and
> the installation installs to "${PREFIX}/bin". By the default, for ports,
> that prefix is /usr/local. The distributions would have a prefix of /.
> If the user makes it / for a package install, do what he says.
Except that it is not likely to work. At this time, the motto would be more
along the lines "install in /usr/local or you are doomed". This is not
because of a deficiency of the package system, it is simply because most (if
not all) software compiled in will contain the hardcoded "prefix" value
somewhere in the code, and since the default for building the ports is to
have "PREFIX=/usr/local", the resulting package should (must ?) be installed
in /usr/local or it won't work properly.
The only thing that the prefix option of pkg_add is usefull for (IMHO) is to
install on a disk that is mounted at a different place than what is
"production" location will be (i.e. boot from a CD, mount the HD on /mnt
install all the packages in "/mnt/usr/local", reboot, the HD is now mounted
as "/", done).
> > Putting all packages into /usr/local begs problems with file overwrites
> > (which can't realistically be resolved manually; even when you can
> > tell the user what package each file came from, few users will be
> > able to tell whether it's "safe" for a particular file to be replaced)
> > and creates other ugly problems because /usr/local must generally be
> > shared
> > with locally-compiled software.
>
> Not true. We don't have much of a problem with overwrites, except for
> multiple versions of the same software. The later is an upgrade target
> problem, not a problem in itself. Programs, like tcl, which one would
> naturally want multiple versions of are installed without conflict.
>
> As for /usr/local being shared with locally compiled software... well,
> that's only if you want it to! :-) You may either install the locally
> compiled software elsewhere, or you may put a different base path for
> ports/package installation on /etc/make.conf, and still preserve the
> single directory for all ports model.
>
> > > Err... no. This is a worse nightmare. Ok, you might disagree. You have
> > > the option of changing the base path of the port to be installed.
Write
> > > a shell script that makes the install path get it's name from the
port.
> >
> > "a worse nightmare"? I'm curious to know why you think this.
> > I used exactly this approach on my own FreeBSD system for several
> > years, and found it quite managable. I'd be interested in knowing
> > what problems you've had with it, as I've found it a big improvement
> > over the current /usr/local tarpit.
>
> Somehow I dislike PATHs, MANPATHs and LIBPATHs with 40 or 50 entries, I
> dislike a /usr that won't fit in one screen, I dislike having programs
> all over the place, I dislike having to edit /etc every time I want to
> make a new program available, and I specially dislike having to instruct
> users in setting up their accounts to be able to use a new program .
Well this is precisely why we need good tools that can handle all of this
automatically. Hell, these things are not really complicated, they are
highly repitive, that's about it. If the situation is not good with AIX or
Solaris, maybe it is because the tools are not good enough on these
platforms.
To keep /usr "small" we can use a convention: /usr/package/xyz-version or
something similar to the ports layout (combining the structure of
/var/db/pkg and /usr/ports).
Also the use of symlinks (can be really hugly) can solve the PATH and
friends issue.
Maybe it would be time to revisit how these variables are handled (i.e.
created, maintained etc.), maybe a system wide list of prefixes that trigger
the proper instantiation of these variables when needed could be the
solution. Of course then this is not exactly "tradditional" unix any more...
> Not really. All ports are built and installed in the directory
> ${PREFIX}. If you want elm+ME to be installed on /usr/elm+ME, you type
>
> PREFIX=/usr/elm+ME make all install
>
> So, as you see, the install place is *NOT* fixed at all! The default, in
> the absence of PREFIX, is /usr/local, but the argument that having this
> as default simplifies things from a build/install perspective is
> incorrect.
If you install a pre-compiled package that you obtained from the website,
you DO NOT have a choice: you have to install in /usr/local
Thinking about it, this little war is even a bit silly. The current port
system is already smart enough to let you use a custom prefix. Why not
implement 2 "standard" profiles:
- "tradditional": default PREFIX=/usr/local
- "split": default PREFIX=/usr/package/$PACKAGE_NAME-$VERSION
Set the profile you like best in /etc/make.conf, and now everybody is happy.
Of course we also need to develop the smarts that will handle the "split"
layout effectively (i.e. updating the PATH and friends), but this is not
rocket science and a couple of shell or perl scripts can tackle that easily.
One of the advantages of the "split" version is that binary package installs
can then become really fast no matter what the format of the archive is.
Cheers,
Patrick.
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-libh" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?00cb01c042f1$1a347190$040aa8c0>