From owner-freebsd-fs@FreeBSD.ORG Thu Feb 15 13:07:27 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5F9AB16A400 for ; Thu, 15 Feb 2007 13:07:27 +0000 (UTC) (envelope-from tol@pdc.kth.se) Received: from mx2.kth.se (mx2.kth.se [130.237.48.98]) by mx1.freebsd.org (Postfix) with ESMTP id E239013C46B for ; Thu, 15 Feb 2007 13:07:24 +0000 (UTC) (envelope-from tol@pdc.kth.se) Received: from localhost (localhost.localdomain [127.0.0.1]) by mx2.kth.se (Postfix) with ESMTP id CFF7614080C; Thu, 15 Feb 2007 14:07:22 +0100 (CET) Received: from mx2.kth.se ([127.0.0.1]) by localhost (mx2.kth.se [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 17148-02-57; Thu, 15 Feb 2007 14:07:20 +0100 (CET) Received: from kashyyyk.ite.kth.se (kashyyyk.ite.kth.se [130.237.31.35]) by mx2.kth.se (Postfix) with ESMTP id 11A70140734; Thu, 15 Feb 2007 14:07:20 +0100 (CET) Received: by kashyyyk.ite.kth.se (Postfix, from userid 18404) id EF4F289CA62; Thu, 15 Feb 2007 14:07:19 +0100 (CET) Sender: tol@kashyyyk.ite.kth.se From: Tomas Olsson To: Kostik Belousov References: <20070213192906.U726@chrishome.localnet> <20070214162938.GA96725@keira.kiwi-computer.com> <20070214173211.L1054@chrishome.localnet> <20070214170808.GC96725@keira.kiwi-computer.com> <20070215044707.GA39168@deviant.kiev.zoral.com.ua> <20070215104537.GC39168@deviant.kiev.zoral.com.ua> <20070215120855.GE39168@deviant.kiev.zoral.com.ua> Date: 15 Feb 2007 14:07:19 +0100 In-Reply-To: <20070215120855.GE39168@deviant.kiev.zoral.com.ua> Message-ID: Lines: 53 User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.3 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: by amavisd-new at kth.se Cc: freebsd-fs@freebsd.org, arla-drinkers@stacken.kth.se Subject: Re: Arla on FreeBSD X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Feb 2007 13:07:27 -0000 Kostik Belousov writes: > On Thu, Feb 15, 2007 at 12:59:04PM +0100, Tomas Olsson wrote: > > Kostik Belousov writes: > > > I made really quick look at the places you mentioned. I have some > > > comment for open_file(). For FreeBSD >= 6.x, the right way to open vnode > > > from the kernel code is to use vn_open() (and then vn_close()) API. > > > > > Great! Sounds reasonable. > > > > We currently open the cache files from nnpfs' VOPs, are there any risks > > (deadlock?) involved if one passes an absolute path to vn_open() in such a > > context? I'd have liked to do use arlad's thread for this, but vput() > There, you already have nnpfs vnode locked. The right lock order for vnodes > is from root down by the tree. As such, you may end up with reversals, that > would result in deadlocks, IMHO. > Ok, vn_open() must be passed curthread so we can't use arlad's thread when in our VOP. And we cannot use an absolute path to the cache. So vn_open() can't be used? > > explicitly uses curthread deep down in namei. Also, users are not normally > > allowed to access the cache files directly so some OSes complain on such a > > lookup with user creds; would that be a problem here? > How is the user access to cache is disabled ? And what is the cache itself ? > Local filesystem (UFS) that stores your blocks in regular files ? > It's just a local dir tree, on UFS or whatever. Currently each node gets it's own subdir, and each "block" (128kB perhaps) a plain file in that dir. The cache is supposed to be readable only by arlad (usually root) and through nnpfs; that's good when handling fancy ACLs etc, so the cache root is chmod:ed to 0700 for root:wheel. > > Of course, we wouldn't have to worry about such things if we just kept the > > vnode handy for each cache block file. Maybe it's a price worth paying. > > Then, you need to take some care of cached vnode lifecircle (e.g., even > keeping the vnode vref'ed would not prevent it from being recycled, so you > may end with dead vnode). > Eep. Tricky. So where does this leave us, is plain lookup() (or VOP_LOOKUP) and VOP_CREATE on the cache a possible way to go? Seems to work on OpenBSD. > Also, as Robert pointed out in his email, you probably need to decide about > MP-safeness of nnpfs. > Well, it's marked "doesn't need giant"; we use our own global lock. I don't trust it 100%, but it seems to work so far. I haven't tried it on any MP FreeBSD boxes though. thanks /t