Date: Sun, 3 Sep 2000 05:22:26 -0500 From: Will Andrews <will@physics.purdue.edu> To: FreeBSD Ports <ports@FreeBSD.org> Subject: Ports Options Paper Message-ID: <20000903052226.E1205@radon.gryphonsoft.com>
next in thread | raw e-mail | index | archive | help
--ReaqsoxgOBHFXBhH
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Hi all,
This is the fruit of my brain's unused cycles, several nights of
sleepless thought, and several weeks of pondering.
Please note that any egotistical statements (i.e. this proposal will
solve everybody's problems etc.) are hereby retracted if serious flaws
are found with this proposal. :-P
Failing that, I think there's plenty of room for improvement, and my
goal is to have an implementation written by October 3, 2000, and tested
by October 15, 2000.
But I do think I've thought this out quite well.. and now that I've
finally offloaded most of my ideas (maybe there's something I missed
that I thought up previously), I'm hoping we can finally get the ports
collection to become the world's best package management system.
I think I'm going to get some sleep now. Finally.
Till next time,
--
Will Andrews <will@physics.purdue.edu> <will@FreeBSD.org>
GCS/E/S @d- s+:+ a--- C++ UB++++$ P+ L- E--- W+ N-- !o ?K w---
O- M+ V- PS+ PE++ Y+ PGP+>+++ t++ 5 X+ R+ tv+ b++ DI+++ D+
G++ e>++++ h! r- y?
--ReaqsoxgOBHFXBhH
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename=ports-options
program PORTS-OPTIONS
begin
Paper on "Ports Options"
Author: Will Andrews <will@FreeBSD.org>
Date: Thu 29 June 2000
Last: Sun 3 September 2000
PROBLEM
The current ports system has many problems that make it
difficult to extend, upgrade, etc. For example, there is no command
that will upgrade all installed ports for you. There is no way for
one port to find out whether another port has support for a certain
feature. There is no way to ensure library/binary compatibility.
There is no way to build more than one package (i.e. with or without
fonts, w/ or w/o GTK support, w/ or w/o GNOME support, w/ or w/o
LDAP support, w/ or w/o debugging code, and so forth). There is no
easy UI for enabling or disabling "options" in the ports that have
them. There is no consistent interface to add "options".
PROPOSAL
N.B.:
This paper now takes into account advantages given to the
ports system by the PORTREVISION/PORTEPOCH system recently proposed
by Kris Kennaway <kris@FreeBSD.org>, which allows for flawless version
upgrade mechanism. Original ideas by Maxim Sobolev <sobomax@FreeBSD.org>
and Neil-Blakey Milner <nbm@FreeBSD.org>.
The Story:
Some time ago, Neil-Blakey Milner <nbm@FreeBSD.org> proposed
a method allowing users to configure their ports without having to
read the Makefile to figure out what options were available. This
idea was a good one, but its implementation was a little too difficult:
it required XML files in addition to each port. Since porters are a
little more used to Makefile syntax, it seems it would be easier to
set up something in this manner, to solve a large variety of problems
with the current ports system. XML parsers are not easy to write and
do not come with the system. (Read: it's not easy to integrate XML
files so they can work with the base system, and thus this sort of
thing would be too proprietary) [ no offense, Neil ;) ]
Let's take an example of astro/xplanet...:
.if (!defined(WITH_TK80) && !defined(WITH_TK81) && !defined(WITH_TK82) && !defined(WITH_TK83))
.if exists(${LOCALBASE}/bin/wish8.3)
WITH_TK83= yes
.elif exists(${LOCALBASE}/bin/wish8.2)
WITH_TK82= yes
.elif exists(${LOCALBASE}/bin/wish8.1)
WITH_TK81= yes
.else
WITH_TK80= yes
.endif
.endif
.if defined(WITH_TK80)
RUN_DEPENDS= wish8.0:${PORTSDIR}/x11-toolkits/tk80
WISH= ${LOCALBASE}/bin/wish8.0
TKVERMSG= "Using Tk 8.0"
.endif
.if defined(WITH_TK81)
RUN_DEPENDS= wish8.1:${PORTSDIR}/x11-toolkits/tk81
WISH= ${LOCALBASE}/bin/wish8.1
TKVERMSG= "Using Tk 8.1"
.endif
.if defined(WITH_TK82)
RUN_DEPENDS= wish8.2:${PORTSDIR}/x11-toolkits/tk82
WISH= ${LOCALBASE}/bin/wish8.2
TKVERMSG= "Using Tk 8.2"
.endif
.if defined(WITH_TK83)
RUN_DEPENDS= wish8.3:${PORTSDIR}/x11-toolkits/tk83
WISH= ${LOCALBASE}/bin/wish8.3
TKVERMSG= "Using Tk 8.3"
.endif
Basically, we would replace these lines with:
option WITH_TK80
DESCR= "Tk 8.0 support"
RUN_DEPENDS= tk-8.0:${PORTSDIR}/x11-toolkits/tk80
WISH= ${LOCALBASE}/bin/wish8.0
end WITH_TK80
option WITH_TK81
DESCR= "Tk 8.1 support"
RUN_DEPENDS= tk-8.1:${PORTSDIR}/x11-toolkits/tk81
WISH= ${LOCALBASE}/bin/wish8.1
TKVERMSG= "Using Tk 8.1"
end WITH_TK81
[...]
in ${FILESDIR}/options. Additionally, we could have a
bsd.options.mk that will store these variables with preset
defaults. This bsd.options.mk could also contain the magic
involved into turning ${FILESDIR}/options into a useful
Makefile-type thing.
The reasons for putting such things in a separate file is to
simplify parsing for GUI-style programs like gportconf (which
nbm proposed using with his XML format) or dialog(1), which
would be the default user-friendly interface. It would also
reduce clutter in port Makefiles somewhat.
The *_DEPENDS can also be moved to ${FILESDIR}/options:
LIB_DEPENDS= kdecore.4:${PORTSDIR}/x11/kdelibs2
RUN_DEPENDS= konqueror:${PORTSDIR}/x11/kdebase2 \
kjumpingcube:${PORTSDIR}/games/kdegames2 \
kdf:${PORTSDIR}/misc/kdeutils2 \
kword:${PORTSDIR}/editors/koffice \
kppp:${PORTSDIR}/net/kdenetwork2
to
# defined in bsd.options.mk - capitalized so parser doesn't try
# to look for an `end' point.
REQUIRE WITH_KDELIBS
require WITH_KDEBASE
RUN_DEPENDS= kdebase-2.0:${PORTSDIR}/x11/kdebase2
end WITH_KDEBASE
require WITH_KDEGAMES
RUN_DEPENDS= kdegames-2.0:${PORTSDIR}/x11/kdegames2
end WITH_KDEGAMES
[...]
Returning to the idea of interfaces and that of storing info
about a port's options: If a user selects a particular option,
the package generator will note it in the ${PKG_DBDIR}/${PKGNAME}/+OPTS
file. I.e., if GTK support is optional but not required and the
user selects it, the package generator will put it in the +OPTS
file as "WITH_GTK". If, say, a port supports Qt, GTK, and GNOMELIBS
interfaces, it could hold "WITH_GTK,WITH_QT,WITH_GNOMELIBS".
Now let us address the fact that some configure scripts like to assume
that if you have something installed and it finds it, it should use it
while linking programs etc. This skews our package dependency system
in a number of subtle ways. Thus, we should have a make.conf variable,
USE_GRATUITOUS_DEPENDS, which we can check to see if it is allowed or
not. Then, we can add something like this to a port that allows buidling
with GTK but doesn't require it:
option WITH_GTK
LIB_DEPENDS= gtk-1.2.8:${PORTSDIR}/x11-toolkits/gtk12
option GRATUITOUS
SOME_VAR_TO_DEFINE_HERE= blah
CONFIGURE_ARGS= --blah3 --blah6
end GRATUITOUS
option NO_GRATUITOUS
GRATUITOUS_PATCHES= patch-aa patch-ac patch-zb3
CONFIGURE_ARGS= --blah --blah2
end NO_GRATUITOUS
end WITH_GTK
Where GRATUITOUS_PATCHES will be a string containing the names of the
patches to be applied if USE_GRATUITOUS_DEPENDS is not defined (i.e. if
it is necessary to apply patches etc. NOTE: I'm not real solid on any
of these ideas, but this issue needs to be addressed in a satisfactory
manner, to prevent package dependency breakage. The above method seems
to be good enough to take care of 99.99% of gratuitous pkg dependencies.
My major concern would be regarding conflicting patch files. Perhaps
patchfiles for GRATUITOUS stuff should be denoted in a different manner.
OK. Now, let us consider the case where someone wants to install a
port that REQUIRES an option in a dependency, either through one of its
own options or requirements. But WAIT! The port is already installed,
and it doesn't have the optional code installed! Okay, then. If we are
in BATCH mode, force deinstall and reinstall with option. If we are in
OPTIONS mode (i.e. user selected the `options' target in bsd.port.mk on
the original make command), warn the user and ask about reinstalling X
version of the port that's in the tree with the required options PLUS
whatever other options the port originally contained selected by default.
Then, check another make.conf var FORCE_DEPENDS (or similar) as a last
chance check to see if we have permission to override current port. Else
we fail the install of the original port with an appropriate error message.
This sort of thing would be implemented something like:
option WITH_DEBUG
REQUIREDEP gtk-1.2.8 WITH_DEBUG
end WITH_DEBUG
Now, to move on to the age-old question: How can we properly support the
``make upgrade'' target? Now that we have PORTREVISION/PORTEPOCH, we can
ensure that new versions of packages will always have a PKGNAME greater
than previous ones. Hence, we can simply deinstall and install the new
ports. If the user runs this target from /usr/ports, it checks the PKG_DB
for currently installed ports, selects the options that are default, and
reinstall them in this manner with the options as available. Warnings
will be given if an option is no longer available, and fail if a make.conf
or other variable is specified. Obviously, if ``upgrade'' is run in a
category or specific port Makefile, the upgrade will be as specific in
scope as possible, and will only traverse outside its domain to install
dependencies, and this can be disabled with another make.conf variable.
Another issue that I've stumbled across over time has been the inability
of a single port to create multiple packages. Hence, I'd like to reintroduce
this issue by exploiting the flexibility of ports-options. Let us observe
how we can modify easily (or rather, easily enough) the GTK 1.2.8 port to
build a debugging version:
option WITH_DEBUG
PKGNAME= ${PKGNAME}-debug
CFLAGS+= -g
end WITH_DEBUG
and so forth. This sort of thing will allow us to replace all sorts of hacks
currently in the ports tree, such as ports that exist only to provide
localized versions of another port (i.e. chinese/mutt and mail/mutt), and
all of the various Netscape ports can be crunched into *ONE* Netscape port.
Gratuitous extra ports, i.e. kde11 vs kde2, can now be crunched into *ONE*
kde port, eliminating vagueness. And this will also allow us to crunch all
the various +ipv6 and -ssl ports into their master ports, and still generate
the extra packages as needed. Packages can also be generated for the various
options, i.e. debugging, gtk support, and so forth, from *ONE* port! The
major advantages to this includes not only the significant reduction of
inodes required by an average ports installation, but also increased
maintainability. This is not to say that this method would become REQUIRED;
that is, people can still do it the old hackish way, but it WOULD require a
good reason - i.e. too much hacking needs to be done that would not be made
available through the ports-options mechanism.
Some people may wonder about the effects on the current ports system. Well,
one thing is for sure. People are going to have to reinstall *ALL* of their
current packages (assuming nobody comes up with a good way to upgrade all of
the PKG_DBDIRs to account for the changes). But the standard ``make install''
semantics will not change. It will do whatever is the normal method for
installing (i.e. the required [LIB,BUILD,RUN]_DEPENDS and so forth) and nothing
more. People can avoid the dialog(1) etc. interfaces by simply mentioning
in an environment variable OPTIONS which options they desire. The options
target can also print out in standard textual format the various options
available for a particular port to stdout, which the users can then use to
specify the options in said environment variable.
SUMMARY & CONCLUSION
The term "ports-options" comes from the name of the file that the options
would be written in (${FILESDIR}/options). It is designed to solve many of
the problems with the current ports tree as it is. The project involved in
writing an actual ports-options implementation would probably consist of
creating a new bsd.options.mk to contain options code and common options
(although these may be placed in separate files instead as per Akinori's
recent proposal). A separate file containing a parser for these files
(written in Perl or some other language in the base system) will be used
to generate a Makefile.options as needed. It will provide the actual
functionality to the makefile system. Reiterating, the reasons for putting
things in a separate file are to keep cruft out of the main port Makefile,
and to provide an easier way for fancier interfaces to the options method,
like a GTK/QT/GNOME/dialog(1)/text interface. Also reiterating, ports-options
is not just to provide an easier interface to the options available in a
port, but also to solve many of the package dependency problems that currently
exist in ports.
The power and flexibiltiy of ports-options has been so shockingly
unbelievable that I'm sure other people will think up additional methods
to exploit its usefulness. I'd particularly like to thank Neil-Blakey
Milner <nbm@FreeBSD.org> for the inspiration! I believe that the project
involving writing the code to support ports-options will be most enlightening
(and time-consuming!) as well as vastly enjoyable, because it will result
in an ultimately more powerful ports collection than ever before.
Greetings,
Will
end
--ReaqsoxgOBHFXBhH--
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-ports" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20000903052226.E1205>