From owner-freebsd-pkg@FreeBSD.ORG Wed Dec 3 10:17:54 2014 Return-Path: Delivered-To: pkg@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ADDD7790; Wed, 3 Dec 2014 10:17:54 +0000 (UTC) Received: from mailout05.t-online.de (mailout05.t-online.de [194.25.134.82]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mailout00.t-online.de", Issuer "TeleSec ServerPass DE-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 59314DA3; Wed, 3 Dec 2014 10:17:53 +0000 (UTC) Received: from fwd33.aul.t-online.de (fwd33.aul.t-online.de [172.20.27.144]) by mailout05.t-online.de (Postfix) with SMTP id B84253C9EC4; Wed, 3 Dec 2014 11:17:44 +0100 (CET) Received: from [192.168.119.11] (rCCdI+ZZQhUrb71fb8Ai9p7P6Fn7fEDuRCJu6-8FzHPSY8hv-Cb-3oflLrL+nbVQDt@[84.154.125.71]) by fwd33.t-online.de with (TLSv1.2:ECDHE-RSA-AES256-SHA encrypted) esmtp id 1Xw70B-1NBaMa0; Wed, 3 Dec 2014 11:17:43 +0100 Message-ID: <547EE342.1000503@freebsd.org> Date: Wed, 03 Dec 2014 11:17:38 +0100 From: Stefan Esser User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: pkg@freebsd.org Subject: Locking problems in pkg makes portmaster fail Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-ID: rCCdI+ZZQhUrb71fb8Ai9p7P6Fn7fEDuRCJu6-8FzHPSY8hv-Cb-3oflLrL+nbVQDt X-TOI-MSGID: cb6551ac-058b-4a24-a4d6-14769d6de5cc Cc: Ports FreeBSD X-BeenThere: freebsd-pkg@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Binary package management and package tools discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Dec 2014 10:17:54 -0000 Hi all, I've been having problems with portmaster being aborted because of failed attempts of pkg to obtain a lock. The same problem occurs with pkg and pkg-devel (I installed pkg-devel to check whether it was due to an already fixed problem in pkg-1.3.8). Other possibly relevant machine details: -CURRENT/amd64 on ZFS. A typical portmaster/pkg error message is: -------------------------------------------------------------------- ===>>> Creating a backup package for old version cuse4bsd-kmod-0.1.35 Creating package for cuse4bsd-kmod-0.1.35 pkg: Cannot get an advisory lock on a database, it is locked by another process ===> Installing for cuse4bsd-kmod-0.1.35 ===> Checking if cuse4bsd-kmod already installed ===> cuse4bsd-kmod-0.1.35 is already installed You may wish to ``make deinstall'' and install this port again by ``make reinstall'' to upgrade it properly. If you really wish to overwrite the old port of cuse4bsd-kmod without deleting it first, set the variable "FORCE_PKG_REGISTER" in your environment or the "make install" command line. *** Error code 1 Stop. make[1]: stopped in /usr/svn/ports/head/multimedia/cuse4bsd-kmod [...] -------------------------------------------------------------------- In all these cases it was just portmaster that was using pkg, no other process could have held a lock. (BTW: This was with "portmaster -dgw", I have not checked whether other options hide this problem ...) I have further debugged this problem and found, that the locking problem occurs during this command from /usr/ports/mk/bsd.openssl.mk: OPENSSL_SHLIBFILE!= ${PKG_INFO} -ql ${OPENSSL_INSTALLED} | grep "^`pkg query "%p" ${OPENSSL_INSTALLED}`/lib/libcrypto.so.[0-9]*$$" (sorry about possible line-wrap - this should be a single line ...) This command on its own can be executed without problems: # pkg info -ql openssl | grep `pkg query %p openssl`/lib/libcrypto.so.* /usr/local/lib/libcrypto.so.8 But when it is run in parallel to portmaster creating a package, that process can not acquire the necessary lock and portmaster aborts. This problem does only occur, if portmaster runs a background task that parses bsd.openssl.mk (and it appears that every time this file is read by "make", the pkg-grep-pkg command sequence is invoked). (It seems to be due to a "make -V" in some other port's directory, which has a ".include bsd.openssl.mk", which is executed by portmaster in preparation of building the next port, but I have not identified the exact line in portmaster that does that.) I have no other work-around than to manually install the affected ports. Anyway: While it seems that this is a problem in portmaster, I still think that multiple invocations of "pkg info" or "pkg query" at the time a "pkg create" is running should not make the latter fail ... Regards, STefan PS: I'm supprised this problem has not been reported on the mail-lists, since it is unlikely to only affect me. I'll do a search in the PRs and create a new PR if I really cannot find an existing one ...