From owner-freebsd-ports@FreeBSD.ORG  Mon Jul 28 21:36:16 2014
Return-Path: <owner-freebsd-ports@FreeBSD.ORG>
Delivered-To: freebsd-ports@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id 66D2A9C3
 for <freebsd-ports@freebsd.org>; Mon, 28 Jul 2014 21:36:16 +0000 (UTC)
Received: from mail-ig0-x235.google.com (mail-ig0-x235.google.com
 [IPv6:2607:f8b0:4001:c05::235])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (Client CN "smtp.gmail.com",
 Issuer "Google Internet Authority G2" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 346382670
 for <freebsd-ports@freebsd.org>; Mon, 28 Jul 2014 21:36:16 +0000 (UTC)
Received: by mail-ig0-f181.google.com with SMTP id h3so4361028igd.8
 for <freebsd-ports@freebsd.org>; Mon, 28 Jul 2014 14:36:15 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=mime-version:sender:in-reply-to:references:date:message-id:subject
 :from:to:cc:content-type;
 bh=z9XPxI6rAGk+ajlrNV+yMX32zF7PSmHXmo66gSXrqK0=;
 b=aWKRqfqsXs6RnOnOjc3Daxy9QJ9uIBDBd37fx9GahynIfzvS/9O+D3WXjJcnHOPn2D
 ARg2wkfXiq4WdvYT/4co1zx0/gdPhyQk3LQJSHaeAoqJ19WKqpIkQER14fJZz3YtmTNU
 Oa1xWs4nAXmrc5piK4JhTAsKTd5TSzoV9F5sTwVfK/gUfU1vD3Y9DaKyCb6DGA82Fxon
 8XtXVquzPDvu4l19N6Hb+LTz5Feoi1mU9PlJzpALCA2tcMwYRtoAKcXCbeoK5kTH8jdW
 Dxzc/LWdx++wIFkQzpv6XNSNZTOq+M0R3F8fAIMOyJlfuCg630JxMqnoFtWzQ/ewcCEu
 uVWg==
MIME-Version: 1.0
X-Received: by 10.50.43.193 with SMTP id y1mr21786890igl.32.1406583375590;
 Mon, 28 Jul 2014 14:36:15 -0700 (PDT)
Sender: kob6558@gmail.com
Received: by 10.107.163.148 with HTTP; Mon, 28 Jul 2014 14:36:15 -0700 (PDT)
In-Reply-To: <53D66DA2.4020009@pcbsd.org>
References: <53D511D6.7040108@freebsd.org>
	<53D66DA2.4020009@pcbsd.org>
Date: Mon, 28 Jul 2014 14:36:15 -0700
X-Google-Sender-Auth: FpvLtydMboraZzGpj232ietkYpg
Message-ID: <CAN6yY1vQVybvD6AwQWy4gxTOo7ZSPCG9kJ4QK3z5AeYh+MoYDg@mail.gmail.com>
Subject: Re: pkg: Cannot get a read lock on a database, it is locked by
 another process
From: Kevin Oberman <rkoberman@gmail.com>
To: Kris Moore <kris@pcbsd.org>
Content-Type: text/plain; charset=UTF-8
X-Content-Filtered-By: Mailman/MimeDel 2.1.18
Cc: FreeBSD Ports ML <freebsd-ports@freebsd.org>
X-BeenThere: freebsd-ports@freebsd.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: Porting software to FreeBSD <freebsd-ports.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-ports>,
 <mailto:freebsd-ports-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-ports/>
List-Post: <mailto:freebsd-ports@freebsd.org>
List-Help: <mailto:freebsd-ports-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-ports>,
 <mailto:freebsd-ports-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 28 Jul 2014 21:36:16 -0000

On Mon, Jul 28, 2014 at 8:34 AM, Kris Moore <kris@pcbsd.org> wrote:

> On 07/27/2014 10:51, Stefan Esser wrote:
> > The locking of the pkg database leads to soft failures, but I'm afraid,
> > if these soft failures happen to coincide with certain administrative
> > tasks, they can lead to unexpected results.
> >
> > One example is that portmaster fails to detect PKGNG (and then assumes
> > to be working in a pre-PKGNG environment), if some pkg subcommand locks
> > the database. To repeat:
> >
> > # pkg version -Bs &
> > # pkg info
> >
> > As long as pkg version runs, pkg info fails to get the read lock (at
> > least in the large majority of my tests). You can also prevent any pkg
> > command from running, if you STOP (kill -STOP / ^Z) pkg version. Any
> > other pkg command will fail, until "pkg version" has been "unstopped"
> > and run to completion. This might even be a local DoS, if any command
> > that read-locks the package DB for extended time can be executed by
> > an unprivileged user.
> >
> > Similar error messages are reported by pkg_libcheck, which issues
> > lots of pkg commands in parallel. (I have observed some 5 lock
> > failures per 1000 installed packages on my system).
> >
> >
> > I did not try to test all combinations of simultanous pkg commands
> > and did not verify, whether e.g. "pkg upgrade" might be stopped
> > half way through because of an error accessing the pkg database,
> > but I have seen SQLITE error messages that indicated failed write
> > operations (INSERT/UPDATE).
> >
> > Either the timeouts are too low, or the duration during which the
> > database is locked by a single operation is too large (or there is
> > no fairness and some processes never get access to the database?).
> >
> >
> > I think this should be fixed, since pkg commands that lock the
> > database can be run from CRON or by other means in the background
> > and the operator who issues pkg commands in the foreground (or runs
> > portmaster) receives spurious error messages (which might still be
> > better than the batch jobs doing silly things, after they failed to
> > obtain information from the pkg database ...)
> >
> > Regards, STefan
> > _______________________________________________
> > freebsd-ports@freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-ports
> > To unsubscribe, send any mail to "freebsd-ports-unsubscribe@freebsd.org"
>
> +1 to this whole thing.
>
> I would prefer that pkg commands simply wait for the DB to become
> unlocked again, instead of just failing outright.
>
> We have many scripts which monitor the system, check for updates,
> display our GUI store-front, and its really annoying to have random bits
> of it fail simply because of bad timing with another pkg process.
>
> --
> Kris Moore
> PC-BSD Software
> iXsystems


This is a real pain, but I only see it on one of my systems. That system is
my only i386 system and also my only system running 9.2. Whether that has
anything to do with it, I can't say, but it makes me very nervous. I just
would like to know why only the one system has the problem.
--
R. Kevin Oberman, Network Engineer, Retired
E-mail: rkoberman@gmail.com