Date: Fri, 17 Apr 2009 09:54:34 +0200 From: Mel Flynn <mel.flynn+fbsd.questions@mailing.thruhere.net> To: freebsd-questions@freebsd.org Cc: Polytropon <freebsd@edvax.de>, Chris Whitehouse <cwhiteh@onetel.com> Subject: Re: new package system proposal Message-ID: <200904170954.34679.mel.flynn%2Bfbsd.questions@mailing.thruhere.net> In-Reply-To: <49D76B02.4060201@onetel.com> References: <49D76B02.4060201@onetel.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--Boundary-00=_6WD6JfkiLgNOej2
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Sorry to jump in late.
On Saturday 04 April 2009 16:13:22 Chris Whitehouse wrote:
> pkg_add somewhat addresses this but it doesn't work quite as well as
> ports because of possible version mismatches.
>
> The suggestion below is not aimed at servers because they have
> completely different requirements.
>
> My suggestion is to start with a ports tree that is fixed in time. Make
> that ports tree available as part of this package system and compile a
> typical desktop set of ports, particularly choosing ones which are large
> or have many dependencies. When it is all complete release it and start
> again. Surely quite a wide selection of desktops, wm's and apps could be
> compiled in a couple of weeks?
>
> Modify pkg_add so that it can be told to use this 'snapshot' including
> downloading the fixed ports tree that was used.
I've already thought this through and you're going about it the wrong way.
Version/package mismatches are caused by two things:
1) The buildservers do not repack dependants of an updated port
2) Installed slave ports as dependency cannot be identified by a package that
has the master port built in. Ex: if installing kdelibs3-nocups through
pkg_add, anything using it will download kdelibs3, try to install and fail.
The first is easy to solve, by iterating through the built packages and
repacking them after reinstallation so that +CONTENTS is updated correctly
with the new dependency. I've written a tool pkgsync which does that, but
there's still some corner cases I need to fix and I'm not happy with the time
it takes to read deps from 800+ packages, so might consider using aio(4) or
fixing up my algorithm.
The second case is more difficult, as it requires "flexible dependencies".
I've fixed this myself, by hacking port Makefiles to correctly set the
dependency origin, but this only applies to "one build" (meaning: my
buildserver only makes packages with a fixed set of slave ports, if I'd want
to use CUPS, I'd have to create a seperate buildserver or jail).
The most important thing is that you don't need a snapshot ports tree if 1) is
met. What you also need for binary upgrading is an index format that doesn't
need a ports tree and is consistent with what is actually available on the
given build server.
I've solved this like so:
#!/bin/sh
# vim: ts=4 sw=4 noet ai tw=78
# $Coar: pkgtools/pkgindex/pkgindex.sh,v 1.5 2008/04/17 21:43:46 mdev Exp $
AWK=${AWK:="/usr/bin/awk"}
PKGDB=${PKGDB:="/var/db/pkg"}
PACKAGES=${PACKAGES:="/home/packages-6"}
PKGINDEX=${PKGINDEX:="${PACKAGES}/All/INDEX.bz2"}
TMPINDEX=/tmp/INDEX.$$
# clean up a previous halted version that had the same pid
[ -f ${TMPINDEX} ] && rm -f ${TMPINDEX}
[ -f ${TMPINDEX}.bz2 ] && rm -f ${TMPINDEX}.bz2
DONE=0
echo -n "Creating new in index in $PACKAGES/All"
cd ${PACKAGES}/All
for pkg in *.tbz; do
origin=$(pkgorigin ${pkg})
hash=$(sha256 -q ${pkg})
size=$(ls -aLln ${pkg} | ${AWK} '{ print $5}')
echo "$origin:${pkg}:$hash:$size" >> ${TMPINDEX}
DONE=$((${DONE} + 1))
if test $((${DONE} % 10)) -eq 0; then
if test $((${DONE} % 50)) -eq 0; then
echo -n ${DONE}
else
echo -n .
fi
fi
done
echo done.
bzip2 ${TMPINDEX}
mv -v ${TMPINDEX}.bz2 ${PKGINDEX}
This gives you origin, pkgname, sha256 hash and filesize for download
verification. All you need to determine whether a package is eligible for
binary upgrade.
pkgorigin is a utility I wrote that quickly gets an origin out of a package
without the need for a staging directory. Attached as shar if it makes it to
the list.
Btw, I don't think supporting various OPTIONS is a good idea. Port maintainers
should properly use PACKAGE_BUILDING. There's no such thing as "good
defaults", so try to satisfy what you think is the majority and redirect
complaints to the infamous bit bucket. Proper use also includes disabling
RESTRICTED and IS_INTERACTIVE ports.
--
Mel
--Boundary-00=_6WD6JfkiLgNOej2--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200904170954.34679.mel.flynn%2Bfbsd.questions>