From owner-freebsd-hackers@FreeBSD.ORG Wed Mar 11 23:44:03 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 896FA44D for ; Wed, 11 Mar 2015 23:44:03 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 49A26FEC for ; Wed, 11 Mar 2015 23:44:02 +0000 (UTC) Received: from Julian-MBP3.local (50-196-156-133-static.hfc.comcastbusiness.net [50.196.156.133]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id t2BNhm2I017747 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Wed, 11 Mar 2015 16:43:49 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <5500D32D.30703@freebsd.org> Date: Wed, 11 Mar 2015 16:43:41 -0700 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: "O'Connor, Daniel" , Kim Shrier Subject: Re: file system change notifications References: <237A50A5-FAB7-4FC1-B8F1-0E40DCBF6137@dons.net.au> In-Reply-To: <237A50A5-FAB7-4FC1-B8F1-0E40DCBF6137@dons.net.au> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Mar 2015 23:44:03 -0000 On 3/11/15 3:46 PM, O'Connor, Daniel wrote: >> On 12 Mar 2015, at 05:31, Kim Shrier wrote: >> I have a project where I need to notice changes to files in a large directory tree. >> I noticed that there was a project in GSOC 2010 to implement such a feature. >> >> https://wiki.freebsd.org/SOC2010IlyaPutsikau >> >> It appears that it was never completed. Is it desirable to have this project >> completed and added into FreeBSD. Or, is there another way to get file >> system change notifications? > The 'standard' way is kqueue + masses of file descriptors. > > I am looking at using auditpipe(4) since you can register to be notified for all file modifications and you get a path. > > I wrote some test code at https://gist.github.com/DanielO/e36de242e79fed3fe4f7 > > Ideally we could add an inotify() syscall although I think that is still suboptimal since you need to add a watch per directory so it can be relatively expensive to setup. That said working out what to do in the face of links and so on is tricky.. > > -- > Daniel O'Connor > "The nice thing about standards is that there > are so many of them to choose from." > -- Andrew Tanenbaum > GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > I think there are a few things that could lead to a relatively efficient way of doing this.. if you wanted to watch all the files in a subtree, you should be able to put an annotation in the vnode of base of that subtree, that would indicate that fact. Then you could modify lookup/namei so that any vnode returned from that lookup, that went past that notification would also get the annotation. Obviously if the lookup was relative then the anotation initial state would be inherrited from the start point (CWD?). you woudl have to be ready to remove annotations on the way down (..) but that seems doable. The annotations would refer to a specific kevent item and child processes inherritance rules would work as per normal. each vnode coudl end up being watched by many kevents so therw woudl have ot be some many to many mapping.. a cancelled kevent woudl need ot be able to remove all related anotations, and a touched file woudl need to be able to make several notifications. The "hole" would be that vnodes that already exist within the watched zone would not have the anotation, so you'd have to do root-wards searches to add them when the zone was added (or just live with that). If you search downwards towards root you have to add any annotation you find on any ancestor vnode. but all file activities have to start with an opened vnode somewhere.