From owner-freebsd-pkg@FreeBSD.ORG Fri Nov 7 17:51:53 2014 Return-Path: Delivered-To: freebsd-pkg@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D47B7F73; Fri, 7 Nov 2014 17:51:53 +0000 (UTC) Received: from st11p01mm-asmtp002.mac.com (st11p01mm-asmtp002.mac.com [17.172.204.237]) (using TLSv1 with cipher DES-CBC3-SHA (112/168 bits)) (Client CN "smtp.me.com", Issuer "VeriSign Class 3 Extended Validation SSL SGC CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A8D6A6ED; Fri, 7 Nov 2014 17:51:53 +0000 (UTC) Received: from [192.168.0.42] (unknown [37.82.113.179]) by st11p01mm-asmtp002.mac.com (Oracle Communications Messaging Server 7u4-27.10(7.0.4.27.9) 64bit (built Jun 6 2014)) with ESMTPSA id <0NEO0041KKADJ710@st11p01mm-asmtp002.mac.com>; Fri, 07 Nov 2014 17:51:52 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.12.52,1.0.28,0.0.0000 definitions=2014-11-07_08:2014-11-07,2014-11-07,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1408290000 definitions=main-1411070147 Content-type: text/plain; charset=utf-8 MIME-version: 1.0 (Mac OS X Mail 8.0 \(1990.1\)) Subject: Re: merging pkg databases From: =?utf-8?Q?Sebastian_J=C3=A4schke?= In-reply-to: <545CF0AD.9050509@freebsd.org> Date: Fri, 07 Nov 2014 18:51:49 +0100 Content-transfer-encoding: quoted-printable Message-id: <54C374D4-7AF9-4BAF-854A-69FAA22DB229@mac.com> References: <545CF0AD.9050509@freebsd.org> To: Matthew Seaman X-Mailer: Apple Mail (2.1990.1) Cc: freebsd-pkg@freebsd.org X-BeenThere: freebsd-pkg@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Binary package management and package tools discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Nov 2014 17:51:53 -0000 Am 07.11.2014 um 17:17 schrieb Matthew Seaman : >=20 > On 11/07/14 15:14, Sebastian J=C3=A4schke wrote: >> For some reason we need to merge pkg(ng) databases together, so that >> only the newer packages is in the resulting database, if both merged >> databases have packages of the same type. And if two packages in the >> databases have even the same version, always the same database is >> overwritten. The reason is that we have a ro tree which is handled >> differently than the rw pathes including /usr/local. >>=20 >> It seems to be not possible with the common tools for pkg, so our >> company is willing to pay for a solution on this. >>=20 >> How should be ask whom for this? Is one of the followers of this fine >> list a freelancer, willing and able to do this job? Should we ask the >> foundation itself? We are quite new on a question like this, so >> please advise! >=20 > When you say 'ro tree' and 'rw tree' are these just chunks of the = usual > filesystem layout with custom permissions? New filesystem hierarchies > you've invented (which implies custom package builds with variant > $PREFIX / $LOCALBASE) or else some more complex system using overlays? Within the ro tree (e.g. /usr/roports) there are only Packages build = from the ports to get the pathes work. All of these packages do NOT need anything below /usr/local and are managed with this prefix. PATH etc (e.g. LOCALBASE) in the machine is aware of this and tries to = use these ro files as long as they exist (e.g. libs) when one wants to build = something from source or ports. PREFIX for the user is /usr/local. So it doesn=E2=80=99t matter if he = uses pkg or ports, he can write it. Your idea below is quite nice, but as some pkgs are located under = /usr/roports and some under /usr/local, the resulting pkg database should be aware of = the real location of the files where the latest version of a package is = installed. Recreating the database again on a fresh machine is smart, but it would = fail about this. On the other hand this structure is installed in a big buch of machines, = all with different installations under /usr/local - so it would be quite painfull = to sort it by hand for each single machine :( Is there something else you can see? Or is it even you who could do a = freelancing job about this? :) thanks in any case! Sebastian > If it's the first option, and maybe for the other two options, then a > fairly quick solution is to dump out the lists of packages you want > installed (i.e. the ones not set to autoremove) >=20 > # pkg query -e '%a =3D 0' '%n-%v' > pkg-list1 >=20 > Do that for both of your pkg databases, and then merge the two lists, > removing any duplicates: >=20 > # cat pkg-list1 pkg-list2 | sort -u > pkg-list >=20 > You'll need to check pkg-list manually for any duplicated packages = with > different versions, and choose which one (usually the most recent) you > want installed. Delete any unwanted duplicates from the list. >=20 > Then on a test box -- just a normally setup FreeBSD machine, with no > special permissions settings -- install the edited list of packages: >=20 > # xargs pkg install < pkg-list >=20 > Now compare the set of files installed on the test machine with what = is > on the original machine with the ro and rw filesystems. Delete any > packages on the original system not present on the new one, and = install > any that are missing -- you'll probably want to do some testing at = this > point to make sure you've got the right functionality still working. = The > idea is to end up with an identical package load on both systems. >=20 > Now you should be able to simply copy /var/db/local.sqlite from the = test > box to your original server. >=20 > Does that make sense to you? >=20 > Cheers, >=20 > Matthew >=20 >=20