Date: Fri, 13 Feb 1998 17:23:10 -0800 (PST) From: Simon Shapiro <shimon@simon-shapiro.org> To: Mike Smith <mike@smith.net.au> Cc: install@FreeBSD.ORG Subject: RE: New package scheme, early draft Message-ID: <XFMail.980213172310.shimon@simon-shapiro.org> In-Reply-To: <199802132354.PAA05277@dingo.cdrom.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Perfect! On most of my systems the staging space is not a problem, but I
can see how that can be important.
An idea for a package is to use ar(1) for the pacage. It is a well
understood, inhenerntly Unix facility. A bit unusual, but why not?
On 13-Feb-98 Mike Smith wrote:
>
> Apolgies for omitting this from the previous message.
>
> New Software Packaging Tools for FreeBSD
> ======================================================================
> (c) Michael Smith and contributors 1997, 1998
>
>
> What's wrong with the current tools?
> ------------------------------------
>
> - The current packaging tools have no sense of continuity, ie.
> - there is no understanding that foo-1.2 is an upgrade to foo-1.1
> - installing foo-1.2 over the top of foo-1.1 does not transfer
> dependancies from the old version.
>
> - It is not possible to apply updates to a package; packages can only
> be installed wholesale.
>
> - The use of tar/gzip requires an expensive unpacking stage.
>
> - There is no mechanism for reverting to a previous version of a
> package.
>
> - The information kept about files installed from a package is
> not enough to verify the state of a package.
>
> - A file or directory cannot belong to more than one package.
>
> - Various combinations of the above make it impossible to use the
> package tools for distributing components of the base system.
>
> What will the new tools do?
> ---------------------------
>
> - Continuity will be maintained across versions of a package:
> - it will only be possible to have one version of a package
> installed at a given time.
> - upgrading/downgrading a package will transfer dependancies.
>
> - Packages will be identified using an extended naming scheme that
> allows updates to be applied to a package.
>
> - Packages will use the Zip format, allowing direct installation from
> the package file rather than using a staging area.
>
> - A comprehensive database will be kept regarding the files belonging
> to package(s), including all pertinent data about the file
> (eg. ownership, permissions, checksum (if appropriate), package(s)
> claiming ownership, etc).
>
> - Packages will be able to be up/downgraded at will.
>
> How new packages will be built.
> -------------------------------
>
> A package will consist of a single Zip file containing the package
> script and optional data files. The package script will be called
> PACKAGE, and be contained at the top level of the Zip file. Other
> files will be organised in subdirectories to suit the package script.
>
> Package flavours
> ----------------
>
> A package may contain one of:
>
> - A complete software component, allowing installation of some
> version on a virgin system.
> - A component upgrade, allowing the upgrading of a given range of
> previous versions to a new version.
> - A delta, adding extra functionality or fixes to an existing
> installed component.
>
>
> The package script.
> -------------------
>
> All operations on the package are controlled by the package script.
> The script is a plain text file, divided into sections as detailed
> below.
>
> Each section is formatted as :
>
> <Operation> {
> <body lines>
> }
>
> where <Operation> is one of the standard operations INFO, INSTALL,
> REMOVE, VERIFY, or a package-specific operation.
>
> The INFO section provides general information about the package. It is
> processed every time the package information is read.
>
> The INSTALL and REMOVE sections contain procedures for performing the
> installation/removal of the package.
>
> The VERIFY section may contain extra code for performing verification
> of the installation status of the package. Normal verification
> consists of performing MD5 signature and permissions checks of
> installed files. Other tests may look for data files, running
> processes, etc.
>
> The package script is parsed and executed by a secure Tcl interpreter.
> Many standard Tcl commands and variables are also available (xref
> command list). The package system also exports a number of variables
> (xref variable list).
>
> In addition, the following commands are available, Unless otherwise
> stated, commands may appear in any order, and are processed in the
> order they appear.
>
> INFO section
> ------------
>
> NAME <package-name>
>
> Sets the package name to be operated on by this section. If
> this name does not match that of the Zip file, an error may be
> generated. In the case of a package containing a delta,
> nominates the package to which the delta will be applied.
> This name is used by the REQUIRE VERSION and REQUIRE DELTA
> commands (see below) to locate the "current" package.
>
>
> VERSION <major>.<minor>
>
> Sets the major/minor version number for the package to assume
> following a successful installation.
>
> DELTA <delta-name> [<delta-name>...]
>
> Sets the delta name(s) to be added/removed following a
> successful installation/removal.
>
>
> CANONICALNAME <name>
>
> Packages will often contain third-party software with
> completely different naming conventions. This command
> supplies a canonical name for the name-major.minor+delta
> combination that results from a successful installation of the
> contents of the package.
>
> INSTALL section
> ---------------
>
> REQUIRE PACKAGE <package-spec-expression>
>
> Produces an error if <package-spec-expression> evaluates
> false. In addition, the package(s) whose presence contributes
> to satisfying the expression are listed in
> $pkg_info(package:required), and if the installation completes
> successfully the requirement will be permanently recorded.
>
> (Feature: recursive fetch/install of required-but-not-present
> packages/deltas)
>
>
> REQUIRE VERSION <version-spec> (XXX should allow expression?)
>
> Produces an error if the current package (as specified in the
> PACKAGE command above) does not meet <version-spec>. This
> command is normally used in packages containing deltas
> (nominating the version(s) to which the delta may be applied)
> or upgrades from earlier versions of the package.
>
>
> REQUIRE DELTA <delta-expression>
>
> Produces an error if the current package (as specified in the
> PACKAGE command above) does not meet <delta-spec-expression>.
> This command is normally used in packages containing deltas to
> state the prerequisite delta set. The delta(s) whose presence
> contributes to satisfying the expression are listed in
> $pkg_info(delta:required), and if the installation of the
> delta completes successfully the requirement will be
> permanently recorded.
>
>
> REQUIRE USER <user-specification>
> REQUIRE GROUP <group-specification>
>
> The user/group specified must be present for the installation to
> complete.
>
> (Feature: automatic adding of user/group. Preserve real UID/GID
> in variables.)
>
>
> EXCLUDE PACKAGE <package-spec-expression>
>
> Produces an error if <package-spec-expression> evaluates true.
> In addition, a record is kept of the exclusion condition, and
> no package/delta may be subsequently added which meets the
> exclusion condition.
>
> (Feature: automatic removal of excluded packages)
>
>
> EXCLUDE DELTA <delta-expression>
>
> Produces an error if the current package (as specified in the
> PACKAGE command above) meets <delta-expression>. This command
> is normally used in packages containing deltas to state the
> set of deltas which are known to not coexist. If the delta is
> added successfully, a record is kept of the expression and no
> package may be subsequently added which meets the condition.
>
>
> USE PACKAGE <package-spec>
>
> If a package matching <package-spec> is present, it will be
> listed in $pkg_info(use), and if the installation completes
> successfully this usage will be upgraded to a requirement.
>
> (Feature: automatic fetch/install of uses-but-not-present
> packages/deltas, with user prompting configurable)
>
>
> REPLACE PACKAGE <package-spec> <reason>
>
> If a package matching <package-spec> is present, the user may
> be prompted to indicate that the offending package should be
> removed first. The <reason> string may be supplied as
> justification for this.
>
>
> EXEC <relative-path>
>
> Executes the program located in the package at
> <relative-path>. If the current security model prohibits
> execution of script-supplied programs, an error will be
> generated.
>
>
> SOURCE <relative-path> [<security-level>]
>
> The script located in the package at <relative-path> is
> executed in another Tcl interpreter with security set to
> <security-level>. If this level is incompatible with the
> current security model, an error will be generated. If no
> level is specified, the current level is maintained.
>
>
> INSTALL <destination-path> <source-path> [@<list-path>]
> INSTALL <destination-path> <source-path> [<filename-list>]
>
> Installs files from the <source-path> within the package
> archive into <destination-path> in the filesystem. If
> @<list-path> is supplied, the file at that location in the
> package is consulted for a list of files to be installed. If
> <filename-list> is supplied, the files listed are
> installed. If neither is supplied, all files under
> <source-path> are installed.
>
> In all cases, every file installed is recorded. If an error
> occurs installing a file, all installed files are removed and
> an error is returned.
>
> During an upgrade installation, files need not be present in
> the package if the version from the previous version(s) are
> suitable. In this case, they should still be passed to the
> INSTALL command, which will note that the file is not supplied
> but already present. This will prevent the file in question
> from being removed as obsolete.
>
> The <destination-path> argument should be an absolute path; it
> will normally be one of the paths from $pkg_info(path:*), see
> below. Other paths may be specified, but the security model
> in force may not allow installation outside the nominated areas.
>
> ATTRIBUTE ADD <attribute> <destination-path> [@<list-path>]
> ATTRIBUTE ADD <attribute> <destination-path> [<filename-list>]
> ATTRIBUTE ADD <attribute> <destination-path> [<source-path>]
>
> Adds an attribute to file(s). Attributes control how files
> are installed and recorded. The destination path should match
> that supplied when the file is installed. If @<list-path> is
> supplied, the file at that location is consulted for a list of
> files to which the attribute will be applied. If
> <filename-list> is supplied, the attribute is applied to the
> listed files. If a single path is supplied, and it refers to
> a relative source path within the Zip archive, and the Zip
> archive is available, the names of all files under that path
> are used.
>
> Currently supported attributes are:
>
> CONTENTS_VARY The file's contents may vary over the
> installed lifetime of the package. As
> such, keeping a checksum of the file
> is not useful.
>
> INHERIT_FILE The file is not included as part of
> the package, but may be produced by
> the application and should be
> considered as belonging to the
> package. Implies CONTENTS_VARY.
>
> (XXX more here!)
>
>
> PRESERVE <file-spec>
>
> Normally when an upgrade from a previous version is being
> performed, the files from the previous version are removed.
> This command causes all files matching <file-spec> from
> the previous version to be preserved.
>
> This directive is most suitable for preservation of
> configuration files when upgrading from a compatible previous
> version.
>
> (XXX should this prevent the file being overwritten by an
> INSTALL?)
>
> PRESERVE ALL
>
> This command is normally used in an upgrade situation where
> the wholesale removal of all files associated with the
> previous version is not desired. It should normally be
> avoided in preference to supplying the files in question to
> the INSTALL command.
>
> KEEP <relative-path>
>
> Specifies that the file(s) at/under <relative-path> should be
> preserved by the installation proecess for later use in other
> sections (which may be invoked when the Zip file is not present).
>
> Files retained by KEEP should be stored in the same place the
> PACKAGE file is kept.
>
> REMOVE section
> --------------
>
> (XXX functionality to stop a running daemon etc. before removing)
>
> VERIFY section
> --------------
>
> (XXX functionality to verify that a daemon etc. is running)
>
> General commands
> ----------------
>
> These commands may be used in any section.
>
> MATCH PACKAGE <package-expression>
>
> Returns a list of packages currently installed which match
> <package-expression>.
>
>
> PKG_NAME <package-ident>
> PKG_MAJOR <package-ident>
> PKG_MINOR <package-ident>
> PKG_DELTAS <package-ident>
>
> Return various components of a package name
>
>
> Variable list
> -------------
>
> These variables are defined by the package system, and may be
> consulted by the package script.
>
> pkg_info
>
> This array contains fields describing the package.
>
> package:name
>
> Contains the name component of the current package, if
> known.
>
> package:required
>
> Lists full package identifiers for packages that have
> been established as prerequisites for the current
> package.
>
> delta:required
>
> Lists the deltas which have been established as
> prerequisites for the current package.
>
> security:policy (readonly)
>
> Names the current security policy.
>
> security:exec (readonly)
>
> Boolean indicating whether the current security policy
> will allow the EXEC command.
>
> security:source
>
> Boolean indicating whether the current security policy
> will allow the SOURCE command.
>
> security:paths
>
> Lists the path prefix components under which files may
> be installed; if this list is empty, files may be
> installed anywhere.
>
> path:prefix
>
> The standard installation prefix; where files go by
> default.
>
> path:x11prefix
>
> The prefix for X11-related files.
>
> What's missing?
> ---------------
>
> - Text-file editing primitives (eg. replace PREFIX)
>
>
> Expressions and names.
> ----------------------
>
> <package-ident> Package identifiers.
>
> A given package in a given state is described thus:
>
> <name>-<major>.<minor><delta-list>
>
> where
>
> <name> is an alpha token identifying the package. Only one
> package with this name may be present in the system at
> a given time.
> <major> is the major version number for the package. This
> number is incremented when the package undergoes a
> change which is likely to render it incompatible with
> previous versions.
> <minor> is the minor version number for the package. This
> number is incremented when the package changes but the
> change is not likely to result in incompatabilities.
> <delta-list> is a list of entries, each formatted as +<delta>,
> which identify deltas which have been applied to the
> package.
>
> <delta> Package Deltas.
>
> In order to allow minor changes to be made to a package without
> requring a change in version numbers, as well as to allow
> customisations of a package, "deltas" are supported.
>
> A delta applies to an existing package (possibly to a range of
> versions), and may replace or alter files within the package.
>
> (Feature: the package tools may elect to refuse to apply a delta which
> alters files previously altered by another delta. The aim
> here is to reduce the likelihood of two deltas produced in
> igorance of one another being able to interfere.)
>
> <package-spec> Package Specification.
>
> A package specification is a wildcard expression intended to match
> against package identifiers. All fields must be present, and matching
> occurs separately for each field. A package identifier matches if all
> fields match the specification.
>
> <name> Glob-style string patterns are supported here. (xref
> Tcl 'string match')
> <vesion-spec>
> <major>,<minor> These fields may contain '*' (matching any
version),
> or a numeric value preceeded by >, >=, =, <= or <,
> matching if the ident being considered has a field
> numerically greater, greater or equal, equal, less or
> equal or less than the value supplied. The = prefix
> may be omitted for simplicity.
> <delta-spec> Items in the delta list may be prefixed with +,
> indicating that the named delta is required for a
> match, or !, indicating that the named delta must not
> be present for a match.
>
> Commands using package specifications may treat the results of a match
> as a logical value (eg. in an expression) or as a list of matching
> packages.
>
> <package-spec-expression>
> <delta-spec-expression>
>
> Specifications may be grouped to form expressions. In an
> expression, specifications are considered to evaluate 'true' if they
> match one or more packages.
>
> Operators supported are '&&' (logical and) and '||' (logical or).
> Precedence must be established using parentheses, an expression may
> not contain more than one operator. (XXX someone write a better
> expression parser?)
>
> When evaluating expressions, the full set of matched items is
> retained and may be used by the command evaluating the expression.
> Expressions are fully, not lazily, evaluated.
>
> <file-spec>
>
> This is a glob-style file match. It returns a list of files matching
> the expression. The leading path may be explicitly matched (if the
> spec begins with a /), otherwise the command using the spec may only
> match the trailing component, depending on context.
>
> <relative-path>
>
> This is a package-internal relative path. It may be evaluated within
> the Zip file, or within some storage area containing files extracted
> from the Zip file by a KEEP command.
>
>
> The database.
> -------------
>
> Everything known about files installed from packages, package
> requirements and exclusions, etc. is stored in the package management
> database.
>
> Package records define packages that have been installed. A package
> record matches a given name-major.minor tuple. The package record
> contains (XXX enumerate fields).
>
> Delta records are attached to package records. Each delta record
> contains details of the delta (XXX enumerate fields). Delta records
> are sorted by order of application.
>
> File records contain information pertaining to files under control of
> the package system. Files may be claimed by more than one package, so
> the file:package association is kept separately from the file record.
> (XXX enumerate fields, comment on attributes)
>
> The filestore and backing out.
> ------------------------------
>
> XXX Elaborate on moving of files from old packages to the store, and
> resurrection of old versions from the store when new versions
> installed. Also mention purging of old versions from the store.
>
>
>
> --
> \\ Sometimes you're ahead, \\ Mike Smith
> \\ sometimes you're behind. \\ mike@smith.net.au
> \\ The race is long, and in the \\ msmith@freebsd.org
> \\ end it's only with yourself. \\ msmith@cdrom.com
>
>
>
> To Unsubscribe: send mail to majordomo@FreeBSD.org
> with "unsubscribe freebsd-install" in the body of the message
----------
Sincerely Yours,
Simon Shapiro
Shimon@Simon-Shapiro.ORG Voice: 503.708.7858
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-install" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?XFMail.980213172310.shimon>