From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 6 13:18:56 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 80A991065678; Mon, 6 Apr 2009 13:18:56 +0000 (UTC) (envelope-from gabriele.modena@gmail.com) Received: from mail-bw0-f164.google.com (mail-bw0-f164.google.com [209.85.218.164]) by mx1.freebsd.org (Postfix) with ESMTP id A9D0E8FC1C; Mon, 6 Apr 2009 13:18:55 +0000 (UTC) (envelope-from gabriele.modena@gmail.com) Received: by bwz8 with SMTP id 8so1827631bwz.43 for ; Mon, 06 Apr 2009 06:18:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=ynNrQH584n70yTRpk2SxyvlooZFPjH65Wn4iElRxn68=; b=PkPGv6hgeCM3iA369yTSNNR0vK/QCP4YSAF134/dBIqKbaB48Vq9zxgaZXpxVsJCs+ O2QbFPvd3puunroaAWN/Xb2FkQLCwK0qcMawNba7xB9N6iiq9i3GSw/2M0uwkGB9QL2x qkc/awqdmfAwjTPzH3QmB9l4Az28rqCZYxzns= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=tfxmaflI0VtgLhTnFhp6pfunrd0A4r4ojJq8SgrsQ2ip2zuCQT/1i422FfN0G2Q0W9 gn2jKj6FthXW4q4wxYmeBF6PNl3TjuQr7Qg+Zg4YZFKSpoORGRXERnthz4iRiPpPzLkj vp9Y8Xmc8o9KlSLjPTLo7RFkwd1VDm+IEs92g= MIME-Version: 1.0 Received: by 10.204.113.198 with SMTP id b6mr1642135bkq.108.1239023934587; Mon, 06 Apr 2009 06:18:54 -0700 (PDT) In-Reply-To: References: <1fe1d5d60903210422g70efef15hdd685695cdf8df3c@mail.gmail.com> <1fe1d5d60904020904ya6dcb00h54a54d6a00e2bd0@mail.gmail.com> Date: Mon, 6 Apr 2009 15:18:54 +0200 Message-ID: <1fe1d5d60904060618j1f68b45fl7810169d66890fd6@mail.gmail.com> From: Gabriele Modena To: Robert Watson Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: GSoC: Semantic File System X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Apr 2009 13:18:56 -0000 On Thu, Apr 2, 2009 at 7:26 PM, Robert Watson wrote: >> In this moment I am considering also an userspace approach similar to >> Spotlight/Beagles, but I don't know how I could propose this as a FreeBSD >> GSoC project. > > I think that would make a fine GSoC proposal. Keep in mind that one of the > premises of Spotlight is the fsevents kernel feature, and fseventsd, which > allow Spotlight to subscribe to changes in trees and kick off reindexing as > required. Porting the fsevents API to FreeBSD is fairly straight forward, > with one exception: HFS+ offers a much more reliable notion of vnode->path > mapping, but it would be interesting to see how well our current vnode->path > mapping mechanisms would suffice in practice (since a lot of the edge cases > that don't work well with our mapping system are exactly that -- edge > cases). This is in case of using UFS/FFS as the base fs or is a more general VFS issue? And what about ZFS? Since apple has not (partially) started to support it, IMHO it may be an interesting fs to investigate. > Between kernel and userspace parts there's quite a bit to do, but one > possibility would be to borrow parts from Mac OS X/etc that we need. For > example, do a literal port of the fsevents mechanism from XNU, provide our > own implementation that provides a similar API, or provide a new mechanism > that meets fseventd's semantic requirements for monitoring. This would definitely be an interesting approach. One of my original ideas was to have a look at how inotify has been implemented in linux, but probably fsevents would be a better choice (also license wise). > I'm probably blending reality with imagination here, but my vague > recollection is that the model was a slightly different blend of user vs. > application involvement in indexing. For how I see it now the steps for my work could be: 1) port/implement an event notification feature; 2) build a userspace indexer; 3) write a gui/search tool on top of it. > In the BeOS model, or my reinterpretation based on something I read a long > time ago and then presumably had dreams about, the split is a bit different: > the file system maintains indexes of extended attributes, which are written > by applications in order to expose searchable material. This sounds to me like adding another layer/proxy between the applications and the actual data. I found some material about node watch and extended attributes that I'm going to study in these days. > It's also worth observing that one of the authors of BFS was Dominic > Giampaolo, who now works on Apple's HFS+, and implemented fsevents there as > part of their Spotlight project. Thanks a lot for you feedback, you provided me a with lot of very interesting material to look up! Cheers.