From owner-freebsd-fs@FreeBSD.ORG Sun Mar 6 16:23:47 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5ECFF1065674 for ; Sun, 6 Mar 2011 16:23:47 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta08.emeryville.ca.mail.comcast.net (qmta08.emeryville.ca.mail.comcast.net [76.96.30.80]) by mx1.freebsd.org (Postfix) with ESMTP id 41B678FC17 for ; Sun, 6 Mar 2011 16:23:46 +0000 (UTC) Received: from omta19.emeryville.ca.mail.comcast.net ([76.96.30.76]) by qmta08.emeryville.ca.mail.comcast.net with comcast id FsDw1g0011eYJf8A8sPm93; Sun, 06 Mar 2011 16:23:46 +0000 Received: from koitsu.dyndns.org ([98.248.33.18]) by omta19.emeryville.ca.mail.comcast.net with comcast id FsPj1g00F0PUQVN01sPjEG; Sun, 06 Mar 2011 16:23:44 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 014CA9B422; Sun, 6 Mar 2011 08:23:42 -0800 (PST) Date: Sun, 6 Mar 2011 08:23:42 -0800 From: Jeremy Chadwick To: Steve Wills Message-ID: <20110306162342.GA94700@icarus.home.lan> References: <20110227202957.GD1992@garage.freebsd.pl> <4D73098F.3000807@FreeBSD.org> <59D664AA-76C6-45C7-94CE-5AA63080368C@FreeBSD.org> <4D738DB0.1090603@FreeBSD.org> <4D739D96.5090705@FreeBSD.org> <20110306153745.GA93530@icarus.home.lan> <4D73B0F1.1040304@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4D73B0F1.1040304@FreeBSD.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, Edward Tomasz Napiera?a Subject: Re: ACL issue (Was Re: HEADS UP: ZFSv28 is in!) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Mar 2011 16:23:47 -0000 On Sun, Mar 06, 2011 at 11:06:09AM -0500, Steve Wills wrote: > On 03/06/11 10:37, Jeremy Chadwick wrote: > > > > At first glance it looks like acl_set_fd_np(3) isn't working on an > > md-backed filesystem; specifically, it's returning EOPNOTSUPP. You > > should be able to reproduce the problem by doing a setfacl on something > > in /tmp/foobar. > > > > Looking through src/bin/cp/utils.c, this is the code: > > > > 420 if (acl_set_fd_np(dest_fd, acl, acl_type) < 0) { > > 421 warn("failed to set acl entries for %s", to.p_path); > > 422 acl_free(acl); > > 423 return (1); > > 424 } > > > > EOPNOTSUPP for acl_set_fd_np(3) is defined as: > > > > [EOPNOTSUPP] The file system does not support ACL retrieval. > > > > This would be referring to the destination filesystem. > > > > Looking through the md(4) source for references to EOPNOTSUPP, we do > > find some references: > > > > $ egrep -n -r "EOPNOTSUPP|ENOTSUP" /usr/src/sys/dev/md > > /usr/src/sys/dev/md/md.c:423: return (EOPNOTSUPP); > > /usr/src/sys/dev/md/md.c:475: error = EOPNOTSUPP; > > /usr/src/sys/dev/md/md.c:523: return (EOPNOTSUPP); > > /usr/src/sys/dev/md/md.c:601: return (EOPNOTSUPP); > > /usr/src/sys/dev/md/md.c:731: error = EOPNOTSUPP; > > > > Line 423 is within mdstart_malloc(), and it returns EOPNOTSUPP on any > > BIO operation other than READ/WRITE/DELETE. Line 475 is a continuation > > of that. > > > > Line 508 is within mdstart_vnode(), behaving effectively the same as > > line 423. Line 601 is within mdstart_swap(), behaving effectively the > > same as line 423. > > > > Line 731 is within md_kthread(), and indicates only BIO operation > > BIO_GETATTR is supported. This would not be an "ACL attribute" thing, > > but rather getting attributes of the backing device itself. The code > > hints at that: > > > > 722 if (bp->bio_cmd == BIO_GETATTR) { > > 723 if ((sc->fwsectors && sc->fwheads && > > 724 (g_handleattr_int(bp, "GEOM::fwsectors", > > 725 sc->fwsectors) || > > 726 g_handleattr_int(bp, "GEOM::fwheads", > > 727 sc->fwheads))) || > > 728 g_handleattr_int(bp, "GEOM::candelete", 1)) > > 729 error = -1; > > 730 else > > 731 error = EOPNOTSUPP; > > 732 } else { > > Thanks for the investigation! So this seems to be a bug in md? That's > too bad, I was enjoying using it to make my tinderbox builds faster. Sorry, I should have been more clear -- my investigation wasn't to determine if the issue you're reporting was a bug or not, but more along the lines of "hmm, where is userland getting EOPNOTSUPP from in the kernel in this situation?" It could be that some piece hasn't been implemented somewhere yet (more an "incomplete" than a bug :-) ). I tend to trace source the way I did above in hopes that someone (kernel dev, etc.) will chime in and go "Oh, yes, THAT... let me tell you about that!" It's also for educational purposes; I figure sharing the innards along with some simple descriptions might help people feel more comfortable (vs. thinking everything is a black box; don't let the magic smoke out!). Sometimes digging through the code helps. > > This leaves me with some ideas; just tossing them out here... > > > > 1. Maybe/somehow this is caused by swap being used as the backing > > type/store for md(4)? Try using "mdconfig -t malloc -o reserve" > > instead, temporarily anyway. > > Seems to be the same. I'm not too surprised, but at least that rules out swap vs. non-block-device stuff being somehow responsible. I'm not a user of ACLs myself, but Robert Watson might know what's up with this, or where to go looking. I've CC'd him here. > > 2. Are you absolutely 100% sure the kernel you're using was built > > with "options UFS_ACL" defined in it? Doing a "strings -a > > /boot/kernel/kernel | grep UFS_ACL" should suffice. > > > > Yep, it does: > > % strings -a /boot/kernel/kernel | grep UFS_ACL > options UFS_ACL > > (My kernel config is just "include GENERIC" then a bunch of "nooptions" > for KDB, DDB, GDB, INVARIANTS, WITNESS, etc.) Cool, good to rule out the obvious. Thanks. The only other thing I can think of off the top of my head would be to "ktrace -t+ -i" the cp -p, then provide output of kdump -s -t+ after. I wouldn't say go about this quite yet (it may not even help determine what's going on); maybe wait for Robert to take a look first. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB |