From owner-freebsd-ports@FreeBSD.ORG Wed Jul 30 19:07:38 2014 Return-Path: Delivered-To: freebsd-ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F26275A1 for ; Wed, 30 Jul 2014 19:07:38 +0000 (UTC) Received: from mail-ig0-x236.google.com (mail-ig0-x236.google.com [IPv6:2607:f8b0:4001:c05::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BD9AD281B for ; Wed, 30 Jul 2014 19:07:38 +0000 (UTC) Received: by mail-ig0-f182.google.com with SMTP id c1so3486647igq.15 for ; Wed, 30 Jul 2014 12:07:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=EqmEVHbPFpdqWZZLMwgfldBXfXmQqZM1Yy3eWRUnq2A=; b=f3VOVBjsLutWuLL6tO/d2NLFwpJUfvuAsWS2Fwb7CZCCDxWiBu/zwS3LfpATrO580Q vqtWr5lrWpy+1VeHk6Lafn91S21ah/SIMx2Tiwj8a7z473ETn7Gj91ut/5l3fCmz0nZM sBR5AxlqltWFGwEglJybRLwXNcoBXrtDffgIJLsCU6j2+ogZU6Wu+svYo7pILBUqqqYd s9jbJPPDlRHLgR+zRdP+neANqCLhBj1qetGRagpbOnjpvTjLctTfJYNaw/3rXT6eQdUX QyWjkiS0Www6NjnIEtt5zD2sMMwtJhrxoINKjGmQsRVUab1WsgreDNook6487Lc6N55Q G/og== MIME-Version: 1.0 X-Received: by 10.50.2.71 with SMTP id 7mr57017004igs.32.1406747258224; Wed, 30 Jul 2014 12:07:38 -0700 (PDT) Sender: kob6558@gmail.com Received: by 10.107.163.148 with HTTP; Wed, 30 Jul 2014 12:07:38 -0700 (PDT) In-Reply-To: <53D8F823.4080104@gwdg.de> References: <53D511D6.7040108@freebsd.org> <53D66DA2.4020009@pcbsd.org> <53D8F823.4080104@gwdg.de> Date: Wed, 30 Jul 2014 12:07:38 -0700 X-Google-Sender-Auth: dD0_dNuQCUoy1QF_jBpbNE2PKsU Message-ID: Subject: Re: pkg: Cannot get a read lock on a database, it is locked by another process From: Kevin Oberman To: Rainer Hurling Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: FreeBSD Ports ML , Kris Moore X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jul 2014 19:07:39 -0000 On Wed, Jul 30, 2014 at 6:50 AM, Rainer Hurling wrote: > Am 28.07.2014 23:36, schrieb Kevin Oberman: > > On Mon, Jul 28, 2014 at 8:34 AM, Kris Moore wrote: > > > >> On 07/27/2014 10:51, Stefan Esser wrote: > >>> The locking of the pkg database leads to soft failures, but I'm afraid, > >>> if these soft failures happen to coincide with certain administrative > >>> tasks, they can lead to unexpected results. > >>> > >>> One example is that portmaster fails to detect PKGNG (and then assumes > >>> to be working in a pre-PKGNG environment), if some pkg subcommand locks > >>> the database. To repeat: > >>> > >>> # pkg version -Bs & > >>> # pkg info > >>> > >>> As long as pkg version runs, pkg info fails to get the read lock (at > >>> least in the large majority of my tests). You can also prevent any pkg > >>> command from running, if you STOP (kill -STOP / ^Z) pkg version. Any > >>> other pkg command will fail, until "pkg version" has been "unstopped" > >>> and run to completion. This might even be a local DoS, if any command > >>> that read-locks the package DB for extended time can be executed by > >>> an unprivileged user. > >>> > >>> Similar error messages are reported by pkg_libcheck, which issues > >>> lots of pkg commands in parallel. (I have observed some 5 lock > >>> failures per 1000 installed packages on my system). > >>> > >>> > >>> I did not try to test all combinations of simultanous pkg commands > >>> and did not verify, whether e.g. "pkg upgrade" might be stopped > >>> half way through because of an error accessing the pkg database, > >>> but I have seen SQLITE error messages that indicated failed write > >>> operations (INSERT/UPDATE). > >>> > >>> Either the timeouts are too low, or the duration during which the > >>> database is locked by a single operation is too large (or there is > >>> no fairness and some processes never get access to the database?). > >>> > >>> > >>> I think this should be fixed, since pkg commands that lock the > >>> database can be run from CRON or by other means in the background > >>> and the operator who issues pkg commands in the foreground (or runs > >>> portmaster) receives spurious error messages (which might still be > >>> better than the batch jobs doing silly things, after they failed to > >>> obtain information from the pkg database ...) > >>> > >>> Regards, STefan > >> > >> +1 to this whole thing. > >> > >> I would prefer that pkg commands simply wait for the DB to become > >> unlocked again, instead of just failing outright. > >> > >> We have many scripts which monitor the system, check for updates, > >> display our GUI store-front, and its really annoying to have random bits > >> of it fail simply because of bad timing with another pkg process. > >> > >> -- > >> Kris Moore > >> PC-BSD Software > >> iXsystems > > > > > > This is a real pain, but I only see it on one of my systems. That system > is > > my only i386 system and also my only system running 9.2. Whether that has > > anything to do with it, I can't say, but it makes me very nervous. I just > > would like to know why only the one system has the problem. > > -- > > R. Kevin Oberman, Network Engineer, Retired > > E-mail: rkoberman@gmail.com > > > I am running three boxes with recent HEAD. Only one of them was a new > installation, from the beginning on with pkgng (no pkg_ before). The > other two boxes were upgraded over years now, and so from old pkg_ to > new pkgng. > > Locks, as described above, I only observe on the two upgraded boxes > (from older pkg_ to newer pkgng). So, for me, it looks more like a > remnant from the conversion towards the newer pkgng. > > But of course, I may be completely wrong. So excuse me, if this is a > misleading contribution. > > Rainer Hurling > > No joy. My systems were both upgraded from the old package system. But it was a good idea! -- R. Kevin Oberman, Network Engineer, Retired E-mail: rkoberman@gmail.com