From owner-freebsd-fs@FreeBSD.ORG Sun Mar 6 16:30:16 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4E093106566C for ; Sun, 6 Mar 2011 16:30:16 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta04.emeryville.ca.mail.comcast.net (qmta04.emeryville.ca.mail.comcast.net [76.96.30.40]) by mx1.freebsd.org (Postfix) with ESMTP id 2F33D8FC18 for ; Sun, 6 Mar 2011 16:30:15 +0000 (UTC) Received: from omta16.emeryville.ca.mail.comcast.net ([76.96.30.72]) by qmta04.emeryville.ca.mail.comcast.net with comcast id FsCF1g0011ZMdJ4A4sWFBu; Sun, 06 Mar 2011 16:30:15 +0000 Received: from koitsu.dyndns.org ([98.248.33.18]) by omta16.emeryville.ca.mail.comcast.net with comcast id FsWC1g0010PUQVN8csWCx8; Sun, 06 Mar 2011 16:30:14 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id DA2E69B422; Sun, 6 Mar 2011 08:30:11 -0800 (PST) Date: Sun, 6 Mar 2011 08:30:11 -0800 From: Jeremy Chadwick To: Steve Wills Message-ID: <20110306163011.GA95053@icarus.home.lan> References: <20110227202957.GD1992@garage.freebsd.pl> <4D73098F.3000807@FreeBSD.org> <59D664AA-76C6-45C7-94CE-5AA63080368C@FreeBSD.org> <4D738DB0.1090603@FreeBSD.org> <4D739D96.5090705@FreeBSD.org> <20110306153745.GA93530@icarus.home.lan> <4D73B0F1.1040304@FreeBSD.org> <20110306162342.GA94700@icarus.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110306162342.GA94700@icarus.home.lan> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@FreeBSD.org, Robert Watson , freebsd-current@FreeBSD.org, Edward Tomasz Napiera?a Subject: Re: ACL issue (Was Re: HEADS UP: ZFSv28 is in!) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Mar 2011 16:30:16 -0000 On Sun, Mar 06, 2011 at 08:23:42AM -0800, Jeremy Chadwick wrote: > On Sun, Mar 06, 2011 at 11:06:09AM -0500, Steve Wills wrote: > > On 03/06/11 10:37, Jeremy Chadwick wrote: > > > > > > At first glance it looks like acl_set_fd_np(3) isn't working on an > > > md-backed filesystem; specifically, it's returning EOPNOTSUPP. You > > > should be able to reproduce the problem by doing a setfacl on something > > > in /tmp/foobar. > > > > > > Looking through src/bin/cp/utils.c, this is the code: > > > > > > 420 if (acl_set_fd_np(dest_fd, acl, acl_type) < 0) { > > > 421 warn("failed to set acl entries for %s", to.p_path); > > > 422 acl_free(acl); > > > 423 return (1); > > > 424 } > > > > > > EOPNOTSUPP for acl_set_fd_np(3) is defined as: > > > > > > [EOPNOTSUPP] The file system does not support ACL retrieval. > > > > > > This would be referring to the destination filesystem. > > > > > > Looking through the md(4) source for references to EOPNOTSUPP, we do > > > find some references: > > > > > > $ egrep -n -r "EOPNOTSUPP|ENOTSUP" /usr/src/sys/dev/md > > > /usr/src/sys/dev/md/md.c:423: return (EOPNOTSUPP); > > > /usr/src/sys/dev/md/md.c:475: error = EOPNOTSUPP; > > > /usr/src/sys/dev/md/md.c:523: return (EOPNOTSUPP); > > > /usr/src/sys/dev/md/md.c:601: return (EOPNOTSUPP); > > > /usr/src/sys/dev/md/md.c:731: error = EOPNOTSUPP; > > > > > > Line 423 is within mdstart_malloc(), and it returns EOPNOTSUPP on any > > > BIO operation other than READ/WRITE/DELETE. Line 475 is a continuation > > > of that. > > > > > > Line 508 is within mdstart_vnode(), behaving effectively the same as > > > line 423. Line 601 is within mdstart_swap(), behaving effectively the > > > same as line 423. > > > > > > Line 731 is within md_kthread(), and indicates only BIO operation > > > BIO_GETATTR is supported. This would not be an "ACL attribute" thing, > > > but rather getting attributes of the backing device itself. The code > > > hints at that: > > > > > > 722 if (bp->bio_cmd == BIO_GETATTR) { > > > 723 if ((sc->fwsectors && sc->fwheads && > > > 724 (g_handleattr_int(bp, "GEOM::fwsectors", > > > 725 sc->fwsectors) || > > > 726 g_handleattr_int(bp, "GEOM::fwheads", > > > 727 sc->fwheads))) || > > > 728 g_handleattr_int(bp, "GEOM::candelete", 1)) > > > 729 error = -1; > > > 730 else > > > 731 error = EOPNOTSUPP; > > > 732 } else { > > > > Thanks for the investigation! So this seems to be a bug in md? That's > > too bad, I was enjoying using it to make my tinderbox builds faster. > > Sorry, I should have been more clear -- my investigation wasn't to > determine if the issue you're reporting was a bug or not, but more along > the lines of "hmm, where is userland getting EOPNOTSUPP from in the > kernel in this situation?" It could be that some piece hasn't been > implemented somewhere yet (more an "incomplete" than a bug :-) ). > > I tend to trace source the way I did above in hopes that someone (kernel > dev, etc.) will chime in and go "Oh, yes, THAT... let me tell you about > that!" It's also for educational purposes; I figure sharing the innards > along with some simple descriptions might help people feel more > comfortable (vs. thinking everything is a black box; don't let the magic > smoke out!). Sometimes digging through the code helps. > > > > This leaves me with some ideas; just tossing them out here... > > > > > > 1. Maybe/somehow this is caused by swap being used as the backing > > > type/store for md(4)? Try using "mdconfig -t malloc -o reserve" > > > instead, temporarily anyway. > > > > Seems to be the same. > > I'm not too surprised, but at least that rules out swap vs. > non-block-device stuff being somehow responsible. > > I'm not a user of ACLs myself, but Robert Watson might know what's up > with this, or where to go looking. I've CC'd him here. > > > > 2. Are you absolutely 100% sure the kernel you're using was built > > > with "options UFS_ACL" defined in it? Doing a "strings -a > > > /boot/kernel/kernel | grep UFS_ACL" should suffice. > > > > > > > Yep, it does: > > > > % strings -a /boot/kernel/kernel | grep UFS_ACL > > options UFS_ACL > > > > (My kernel config is just "include GENERIC" then a bunch of "nooptions" > > for KDB, DDB, GDB, INVARIANTS, WITNESS, etc.) > > Cool, good to rule out the obvious. Thanks. > > The only other thing I can think of off the top of my head would be to > "ktrace -t+ -i" the cp -p, then provide output of kdump -s -t+ after. > I wouldn't say go about this quite yet (it may not even help determine > what's going on); maybe wait for Robert to take a look first. It would help if I actually added Robert to the CC list, wouldn't it? :-) -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB |