From owner-freebsd-ports@FreeBSD.ORG Wed Jul 30 15:02:33 2014 Return-Path: Delivered-To: freebsd-ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5AD28FBB for ; Wed, 30 Jul 2014 15:02:33 +0000 (UTC) Received: from fmailer.gwdg.de (fmailer.gwdg.de [134.76.11.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1A2222A0F for ; Wed, 30 Jul 2014 15:02:32 +0000 (UTC) Received: from um-excht-a02.um.gwdg.de ([134.76.11.222] helo=email.gwdg.de) by mailer.gwdg.de with esmtp (Exim 4.80) (envelope-from ) id 1XCUH2-0001DG-Lw; Wed, 30 Jul 2014 15:50:32 +0200 Received: from krabat.raven.hur (91.21.150.246) by email.gwdg.de (134.76.9.211) with Microsoft SMTP Server (TLS) id 14.3.195.1; Wed, 30 Jul 2014 15:50:32 +0200 Message-ID: <53D8F823.4080104@gwdg.de> Date: Wed, 30 Jul 2014 15:50:27 +0200 From: Rainer Hurling User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Kevin Oberman , Kris Moore Subject: Re: pkg: Cannot get a read lock on a database, it is locked by another process References: <53D511D6.7040108@freebsd.org> <53D66DA2.4020009@pcbsd.org> In-Reply-To: Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit X-Spam-Level: - X-Virus-Scanned: (clean) by clamav Cc: FreeBSD Ports ML X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jul 2014 15:02:33 -0000 Am 28.07.2014 23:36, schrieb Kevin Oberman: > On Mon, Jul 28, 2014 at 8:34 AM, Kris Moore wrote: > >> On 07/27/2014 10:51, Stefan Esser wrote: >>> The locking of the pkg database leads to soft failures, but I'm afraid, >>> if these soft failures happen to coincide with certain administrative >>> tasks, they can lead to unexpected results. >>> >>> One example is that portmaster fails to detect PKGNG (and then assumes >>> to be working in a pre-PKGNG environment), if some pkg subcommand locks >>> the database. To repeat: >>> >>> # pkg version -Bs & >>> # pkg info >>> >>> As long as pkg version runs, pkg info fails to get the read lock (at >>> least in the large majority of my tests). You can also prevent any pkg >>> command from running, if you STOP (kill -STOP / ^Z) pkg version. Any >>> other pkg command will fail, until "pkg version" has been "unstopped" >>> and run to completion. This might even be a local DoS, if any command >>> that read-locks the package DB for extended time can be executed by >>> an unprivileged user. >>> >>> Similar error messages are reported by pkg_libcheck, which issues >>> lots of pkg commands in parallel. (I have observed some 5 lock >>> failures per 1000 installed packages on my system). >>> >>> >>> I did not try to test all combinations of simultanous pkg commands >>> and did not verify, whether e.g. "pkg upgrade" might be stopped >>> half way through because of an error accessing the pkg database, >>> but I have seen SQLITE error messages that indicated failed write >>> operations (INSERT/UPDATE). >>> >>> Either the timeouts are too low, or the duration during which the >>> database is locked by a single operation is too large (or there is >>> no fairness and some processes never get access to the database?). >>> >>> >>> I think this should be fixed, since pkg commands that lock the >>> database can be run from CRON or by other means in the background >>> and the operator who issues pkg commands in the foreground (or runs >>> portmaster) receives spurious error messages (which might still be >>> better than the batch jobs doing silly things, after they failed to >>> obtain information from the pkg database ...) >>> >>> Regards, STefan >> >> +1 to this whole thing. >> >> I would prefer that pkg commands simply wait for the DB to become >> unlocked again, instead of just failing outright. >> >> We have many scripts which monitor the system, check for updates, >> display our GUI store-front, and its really annoying to have random bits >> of it fail simply because of bad timing with another pkg process. >> >> -- >> Kris Moore >> PC-BSD Software >> iXsystems > > > This is a real pain, but I only see it on one of my systems. That system is > my only i386 system and also my only system running 9.2. Whether that has > anything to do with it, I can't say, but it makes me very nervous. I just > would like to know why only the one system has the problem. > -- > R. Kevin Oberman, Network Engineer, Retired > E-mail: rkoberman@gmail.com I am running three boxes with recent HEAD. Only one of them was a new installation, from the beginning on with pkgng (no pkg_ before). The other two boxes were upgraded over years now, and so from old pkg_ to new pkgng. Locks, as described above, I only observe on the two upgraded boxes (from older pkg_ to newer pkgng). So, for me, it looks more like a remnant from the conversion towards the newer pkgng. But of course, I may be completely wrong. So excuse me, if this is a misleading contribution. Rainer Hurling