Date: Fri, 13 Feb 1998 15:54:11 -0800 From: Mike Smith <mike@smith.net.au> To: install@FreeBSD.ORG Subject: New package scheme, early draft Message-ID: <199802132354.PAA05277@dingo.cdrom.com>
next in thread | raw e-mail | index | archive | help
Apolgies for omitting this from the previous message.
New Software Packaging Tools for FreeBSD
======================================================================
(c) Michael Smith and contributors 1997, 1998
What's wrong with the current tools?
------------------------------------
- The current packaging tools have no sense of continuity, ie.
- there is no understanding that foo-1.2 is an upgrade to foo-1.1
- installing foo-1.2 over the top of foo-1.1 does not transfer
dependancies from the old version.
- It is not possible to apply updates to a package; packages can only
be installed wholesale.
- The use of tar/gzip requires an expensive unpacking stage.
- There is no mechanism for reverting to a previous version of a
package.
- The information kept about files installed from a package is
not enough to verify the state of a package.
- A file or directory cannot belong to more than one package.
- Various combinations of the above make it impossible to use the
package tools for distributing components of the base system.
What will the new tools do?
---------------------------
- Continuity will be maintained across versions of a package:
- it will only be possible to have one version of a package
installed at a given time.
- upgrading/downgrading a package will transfer dependancies.
- Packages will be identified using an extended naming scheme that
allows updates to be applied to a package.
- Packages will use the Zip format, allowing direct installation from
the package file rather than using a staging area.
- A comprehensive database will be kept regarding the files belonging
to package(s), including all pertinent data about the file
(eg. ownership, permissions, checksum (if appropriate), package(s)
claiming ownership, etc).
- Packages will be able to be up/downgraded at will.
How new packages will be built.
-------------------------------
A package will consist of a single Zip file containing the package
script and optional data files. The package script will be called
PACKAGE, and be contained at the top level of the Zip file. Other
files will be organised in subdirectories to suit the package script.
Package flavours
----------------
A package may contain one of:
- A complete software component, allowing installation of some
version on a virgin system.
- A component upgrade, allowing the upgrading of a given range of
previous versions to a new version.
- A delta, adding extra functionality or fixes to an existing
installed component.
The package script.
-------------------
All operations on the package are controlled by the package script.
The script is a plain text file, divided into sections as detailed
below.
Each section is formatted as :
<Operation> {
<body lines>
}
where <Operation> is one of the standard operations INFO, INSTALL,
REMOVE, VERIFY, or a package-specific operation.
The INFO section provides general information about the package. It is
processed every time the package information is read.
The INSTALL and REMOVE sections contain procedures for performing the
installation/removal of the package.
The VERIFY section may contain extra code for performing verification
of the installation status of the package. Normal verification
consists of performing MD5 signature and permissions checks of
installed files. Other tests may look for data files, running
processes, etc.
The package script is parsed and executed by a secure Tcl interpreter.
Many standard Tcl commands and variables are also available (xref
command list). The package system also exports a number of variables
(xref variable list).
In addition, the following commands are available, Unless otherwise
stated, commands may appear in any order, and are processed in the
order they appear.
INFO section
------------
NAME <package-name>
Sets the package name to be operated on by this section. If
this name does not match that of the Zip file, an error may be
generated. In the case of a package containing a delta,
nominates the package to which the delta will be applied.
This name is used by the REQUIRE VERSION and REQUIRE DELTA
commands (see below) to locate the "current" package.
VERSION <major>.<minor>
Sets the major/minor version number for the package to assume
following a successful installation.
DELTA <delta-name> [<delta-name>...]
Sets the delta name(s) to be added/removed following a
successful installation/removal.
CANONICALNAME <name>
Packages will often contain third-party software with
completely different naming conventions. This command
supplies a canonical name for the name-major.minor+delta
combination that results from a successful installation of the
contents of the package.
INSTALL section
---------------
REQUIRE PACKAGE <package-spec-expression>
Produces an error if <package-spec-expression> evaluates
false. In addition, the package(s) whose presence contributes
to satisfying the expression are listed in
$pkg_info(package:required), and if the installation completes
successfully the requirement will be permanently recorded.
(Feature: recursive fetch/install of required-but-not-present
packages/deltas)
REQUIRE VERSION <version-spec> (XXX should allow expression?)
Produces an error if the current package (as specified in the
PACKAGE command above) does not meet <version-spec>. This
command is normally used in packages containing deltas
(nominating the version(s) to which the delta may be applied)
or upgrades from earlier versions of the package.
REQUIRE DELTA <delta-expression>
Produces an error if the current package (as specified in the
PACKAGE command above) does not meet <delta-spec-expression>.
This command is normally used in packages containing deltas to
state the prerequisite delta set. The delta(s) whose presence
contributes to satisfying the expression are listed in
$pkg_info(delta:required), and if the installation of the
delta completes successfully the requirement will be
permanently recorded.
REQUIRE USER <user-specification>
REQUIRE GROUP <group-specification>
The user/group specified must be present for the installation to
complete.
(Feature: automatic adding of user/group. Preserve real UID/GID
in variables.)
EXCLUDE PACKAGE <package-spec-expression>
Produces an error if <package-spec-expression> evaluates true.
In addition, a record is kept of the exclusion condition, and
no package/delta may be subsequently added which meets the
exclusion condition.
(Feature: automatic removal of excluded packages)
EXCLUDE DELTA <delta-expression>
Produces an error if the current package (as specified in the
PACKAGE command above) meets <delta-expression>. This command
is normally used in packages containing deltas to state the
set of deltas which are known to not coexist. If the delta is
added successfully, a record is kept of the expression and no
package may be subsequently added which meets the condition.
USE PACKAGE <package-spec>
If a package matching <package-spec> is present, it will be
listed in $pkg_info(use), and if the installation completes
successfully this usage will be upgraded to a requirement.
(Feature: automatic fetch/install of uses-but-not-present
packages/deltas, with user prompting configurable)
REPLACE PACKAGE <package-spec> <reason>
If a package matching <package-spec> is present, the user may
be prompted to indicate that the offending package should be
removed first. The <reason> string may be supplied as
justification for this.
EXEC <relative-path>
Executes the program located in the package at
<relative-path>. If the current security model prohibits
execution of script-supplied programs, an error will be
generated.
SOURCE <relative-path> [<security-level>]
The script located in the package at <relative-path> is
executed in another Tcl interpreter with security set to
<security-level>. If this level is incompatible with the
current security model, an error will be generated. If no
level is specified, the current level is maintained.
INSTALL <destination-path> <source-path> [@<list-path>]
INSTALL <destination-path> <source-path> [<filename-list>]
Installs files from the <source-path> within the package
archive into <destination-path> in the filesystem. If
@<list-path> is supplied, the file at that location in the
package is consulted for a list of files to be installed. If
<filename-list> is supplied, the files listed are
installed. If neither is supplied, all files under
<source-path> are installed.
In all cases, every file installed is recorded. If an error
occurs installing a file, all installed files are removed and
an error is returned.
During an upgrade installation, files need not be present in
the package if the version from the previous version(s) are
suitable. In this case, they should still be passed to the
INSTALL command, which will note that the file is not supplied
but already present. This will prevent the file in question
from being removed as obsolete.
The <destination-path> argument should be an absolute path; it
will normally be one of the paths from $pkg_info(path:*), see
below. Other paths may be specified, but the security model
in force may not allow installation outside the nominated areas.
ATTRIBUTE ADD <attribute> <destination-path> [@<list-path>]
ATTRIBUTE ADD <attribute> <destination-path> [<filename-list>]
ATTRIBUTE ADD <attribute> <destination-path> [<source-path>]
Adds an attribute to file(s). Attributes control how files
are installed and recorded. The destination path should match
that supplied when the file is installed. If @<list-path> is
supplied, the file at that location is consulted for a list of
files to which the attribute will be applied. If
<filename-list> is supplied, the attribute is applied to the
listed files. If a single path is supplied, and it refers to
a relative source path within the Zip archive, and the Zip
archive is available, the names of all files under that path
are used.
Currently supported attributes are:
CONTENTS_VARY The file's contents may vary over the
installed lifetime of the package. As
such, keeping a checksum of the file
is not useful.
INHERIT_FILE The file is not included as part of
the package, but may be produced by
the application and should be
considered as belonging to the
package. Implies CONTENTS_VARY.
(XXX more here!)
PRESERVE <file-spec>
Normally when an upgrade from a previous version is being
performed, the files from the previous version are removed.
This command causes all files matching <file-spec> from
the previous version to be preserved.
This directive is most suitable for preservation of
configuration files when upgrading from a compatible previous
version.
(XXX should this prevent the file being overwritten by an
INSTALL?)
PRESERVE ALL
This command is normally used in an upgrade situation where
the wholesale removal of all files associated with the
previous version is not desired. It should normally be
avoided in preference to supplying the files in question to
the INSTALL command.
KEEP <relative-path>
Specifies that the file(s) at/under <relative-path> should be
preserved by the installation proecess for later use in other
sections (which may be invoked when the Zip file is not present).
Files retained by KEEP should be stored in the same place the
PACKAGE file is kept.
REMOVE section
--------------
(XXX functionality to stop a running daemon etc. before removing)
VERIFY section
--------------
(XXX functionality to verify that a daemon etc. is running)
General commands
----------------
These commands may be used in any section.
MATCH PACKAGE <package-expression>
Returns a list of packages currently installed which match
<package-expression>.
PKG_NAME <package-ident>
PKG_MAJOR <package-ident>
PKG_MINOR <package-ident>
PKG_DELTAS <package-ident>
Return various components of a package name
Variable list
-------------
These variables are defined by the package system, and may be
consulted by the package script.
pkg_info
This array contains fields describing the package.
package:name
Contains the name component of the current package, if
known.
package:required
Lists full package identifiers for packages that have
been established as prerequisites for the current
package.
delta:required
Lists the deltas which have been established as
prerequisites for the current package.
security:policy (readonly)
Names the current security policy.
security:exec (readonly)
Boolean indicating whether the current security policy
will allow the EXEC command.
security:source
Boolean indicating whether the current security policy
will allow the SOURCE command.
security:paths
Lists the path prefix components under which files may
be installed; if this list is empty, files may be
installed anywhere.
path:prefix
The standard installation prefix; where files go by
default.
path:x11prefix
The prefix for X11-related files.
What's missing?
---------------
- Text-file editing primitives (eg. replace PREFIX)
Expressions and names.
----------------------
<package-ident> Package identifiers.
A given package in a given state is described thus:
<name>-<major>.<minor><delta-list>
where
<name> is an alpha token identifying the package. Only one
package with this name may be present in the system at
a given time.
<major> is the major version number for the package. This
number is incremented when the package undergoes a
change which is likely to render it incompatible with
previous versions.
<minor> is the minor version number for the package. This
number is incremented when the package changes but the
change is not likely to result in incompatabilities.
<delta-list> is a list of entries, each formatted as +<delta>,
which identify deltas which have been applied to the
package.
<delta> Package Deltas.
In order to allow minor changes to be made to a package without
requring a change in version numbers, as well as to allow
customisations of a package, "deltas" are supported.
A delta applies to an existing package (possibly to a range of
versions), and may replace or alter files within the package.
(Feature: the package tools may elect to refuse to apply a delta which
alters files previously altered by another delta. The aim
here is to reduce the likelihood of two deltas produced in
igorance of one another being able to interfere.)
<package-spec> Package Specification.
A package specification is a wildcard expression intended to match
against package identifiers. All fields must be present, and matching
occurs separately for each field. A package identifier matches if all
fields match the specification.
<name> Glob-style string patterns are supported here. (xref
Tcl 'string match')
<vesion-spec>
<major>,<minor> These fields may contain '*' (matching any version),
or a numeric value preceeded by >, >=, =, <= or <,
matching if the ident being considered has a field
numerically greater, greater or equal, equal, less or
equal or less than the value supplied. The = prefix
may be omitted for simplicity.
<delta-spec> Items in the delta list may be prefixed with +,
indicating that the named delta is required for a
match, or !, indicating that the named delta must not
be present for a match.
Commands using package specifications may treat the results of a match
as a logical value (eg. in an expression) or as a list of matching packages.
<package-spec-expression>
<delta-spec-expression>
Specifications may be grouped to form expressions. In an
expression, specifications are considered to evaluate 'true' if they
match one or more packages.
Operators supported are '&&' (logical and) and '||' (logical or).
Precedence must be established using parentheses, an expression may
not contain more than one operator. (XXX someone write a better
expression parser?)
When evaluating expressions, the full set of matched items is
retained and may be used by the command evaluating the expression.
Expressions are fully, not lazily, evaluated.
<file-spec>
This is a glob-style file match. It returns a list of files matching
the expression. The leading path may be explicitly matched (if the
spec begins with a /), otherwise the command using the spec may only
match the trailing component, depending on context.
<relative-path>
This is a package-internal relative path. It may be evaluated within
the Zip file, or within some storage area containing files extracted
from the Zip file by a KEEP command.
The database.
-------------
Everything known about files installed from packages, package
requirements and exclusions, etc. is stored in the package management
database.
Package records define packages that have been installed. A package
record matches a given name-major.minor tuple. The package record
contains (XXX enumerate fields).
Delta records are attached to package records. Each delta record
contains details of the delta (XXX enumerate fields). Delta records
are sorted by order of application.
File records contain information pertaining to files under control of
the package system. Files may be claimed by more than one package, so
the file:package association is kept separately from the file record.
(XXX enumerate fields, comment on attributes)
The filestore and backing out.
------------------------------
XXX Elaborate on moving of files from old packages to the store, and
resurrection of old versions from the store when new versions
installed. Also mention purging of old versions from the store.
--
\\ Sometimes you're ahead, \\ Mike Smith
\\ sometimes you're behind. \\ mike@smith.net.au
\\ The race is long, and in the \\ msmith@freebsd.org
\\ end it's only with yourself. \\ msmith@cdrom.com
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-install" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199802132354.PAA05277>