From owner-freebsd-current@FreeBSD.ORG Tue Sep 16 10:42:43 2014 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0FB769C9; Tue, 16 Sep 2014 10:42:43 +0000 (UTC) Received: from mail-wi0-x22f.google.com (mail-wi0-x22f.google.com [IPv6:2a00:1450:400c:c05::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 72529997; Tue, 16 Sep 2014 10:42:42 +0000 (UTC) Received: by mail-wi0-f175.google.com with SMTP id cc10so5930147wib.2 for ; Tue, 16 Sep 2014 03:42:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:mail-followup-to:references :mime-version:content-type:content-disposition:in-reply-to :user-agent; bh=XZg7t4ZfYBKVeLBB3IY7QY+f159w6slsIZQs247M0uI=; b=Au1pAeNKIbM88AF7yUfF+RKDMAXeh3iChrRErD0fdKtzsVS94S8J+ActBCEtUeHjDu OEZtg6pRGLJqDx/d2I4ZM/a6XEwmthCa5m4PyXroiCTralrslWcTqd6362CN6HJwpmlU WnRzQNyiS6fOpWp73B7TUZHnVCID7EXJ1PUt2rNCdXYjjnOSQxenU2qrI0mUH0/XTxfl zFfXnKu4wBO3seEeGhSEHKRAVnIy73ynDf3EFcAheaNWsfiwPrIyR06SkV9vTv0wEWM3 UFx/uKdbUhKUUrOCHanipA637Ux3YRERB1KuRrxq3m4V3qmdA88QStMfCQtEncHp/jH4 QwRQ== X-Received: by 10.180.212.50 with SMTP id nh18mr25771240wic.81.1410864160504; Tue, 16 Sep 2014 03:42:40 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id cz3sm17929742wjb.23.2014.09.16.03.42.38 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Tue, 16 Sep 2014 03:42:39 -0700 (PDT) Date: Tue, 16 Sep 2014 12:42:36 +0200 From: Mateusz Guzik To: Peter Wemm Subject: Re: _ftello() modification requires additional capsicum rights, breaking tcpdump and dhclient Message-ID: <20140916104236.GA21560@dft-labs.eu> Mail-Followup-To: Mateusz Guzik , Peter Wemm , freebsd-current@freebsd.org, Patrick Kelsey , George Neville-Neil References: <540FF706.2050400@freebsd.org> <4252906.DAHIEnKpIF@overcee.wemm.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <4252906.DAHIEnKpIF@overcee.wemm.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Patrick Kelsey , George Neville-Neil , freebsd-current@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Sep 2014 10:42:43 -0000 On Fri, Sep 12, 2014 at 09:29:56PM -0700, Peter Wemm wrote: > On Thursday, September 11, 2014 12:38:02 PM Patrick Kelsey wrote: > > On Wed, Sep 10, 2014 at 3:00 AM, Andrey Chernov wrote: > > > On 09.09.2014 21:53, Patrick Kelsey wrote: > > > > I don't think it is worth the trouble, as given the larger pattern of > > > > libc routines requiring multiple capsicum rights, it seems one will in > > > > general have to have libc implementation knowledge when using it in > > > > concert with capsicum. For example, consider the limitfd() routine in > > > > kdump.c, which provides rights for the TIOCGETA ioctl to be used on > > > > stdout so the eventual call to isatty() via printf() will work as > > > > > > intended. > > > > > > > I think the above kdump example is a good one for the subtle issues that > > > > can arise when using capsicum with libc. That call to isatty() is via a > > > > widely-used internal libc routine __smakebuf(). __smakebuf() also calls > > > > __swhatbuf(), which in turn calls _fstat(), all to make sure that output > > > > to a tty is line buffered by default. It would appear that programs > > > > that restrict rights on stdout without allowing CAP_IOCTL and CAP_FSTAT > > > > could be disabling the normally default line buffering when stdout is a > > > > tty. kdump goes the distance, but dhclient does not (restricting stdout > > > > to CAP_WRITE only). > > > > > > > > In any event, the patch attached to my first message is seeming like the > > > > way to go. > > > > > > Well, then commit it (if capsicum team agrees). > > > > Will do - thanks for the feedback. > > > > -Patrick > > Is there any possibility that this is related to the problem we've recently > hit in the freebsd.org cluster with this month's refresh? > > After running for a while: > Sep 10 02:39:44 ns2 unbound: [65258:0] notice: init module 0: validator > Sep 10 02:39:44 ns2 unbound: [65258:0] notice: init module 1: iterator > Sep 10 11:44:29 ns2 unbound: [65258:3] fatal error: event_dispatch returned > error -1, errno is Capabilities insufficient > > Sep 10 16:21:16 ns2 unbound: [28212:0] warning: did not exit gracefully last > time (65258) > Sep 10 16:21:16 ns2 unbound: [28213:0] notice: init module 0: validator > Sep 10 16:21:16 ns2 unbound: [28213:0] notice: init module 1: iterator > Sep 11 10:23:49 ns2 unbound: [28213:5] fatal error: event_dispatch returned > error -1, errno is Capabilities insufficient > > Sep 11 13:48:46 ns2 unbound: [79419:0] warning: did not exit gracefully last > time (28213) > Sep 11 13:48:46 ns2 unbound: [79420:0] notice: init module 0: validator > Sep 11 13:48:46 ns2 unbound: [79420:0] notice: init module 1: iterator > Sep 11 18:42:56 ns2 unbound: [79420:6] fatal error: event_dispatch returned > error -1, errno is Capabilities insufficient > > I believe this jail was started from the boot process. If I restart the jail > by hand from a ssh session the problem goes away. > > This is unbound from ports and I don't have any more details than this. This > is new this month. > Is this thingy multithreaded? Currently there is a race in the kernel where fd is visible before relevant capabilities are installed. This can result in an error like this one for weird processes. I got a patch for this: http://lists.freebsd.org/pipermail/freebsd-arch/2014-August/015788.html but it got stalled on 'memory barrier' discussion. I'll try to ping people to move it forward. IIRC there was a report of unbound failing this way, apparently fixed with aforementioned patch. -- Mateusz Guzik