From owner-freebsd-questions@FreeBSD.ORG Mon Oct 25 04:48:41 2010 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 167C9106566C for ; Mon, 25 Oct 2010 04:48:41 +0000 (UTC) (envelope-from bonomi@mail.r-bonomi.com) Received: from mail.r-bonomi.com (ns2.r-bonomi.com [204.87.227.129]) by mx1.freebsd.org (Postfix) with ESMTP id DDDDD8FC15 for ; Mon, 25 Oct 2010 04:48:40 +0000 (UTC) Received: (from bonomi@localhost) by mail.r-bonomi.com (8.14.3/rdb1) id o9P4kcid004004; Sun, 24 Oct 2010 23:46:38 -0500 (CDT) Date: Sun, 24 Oct 2010 23:46:38 -0500 (CDT) From: Robert Bonomi Message-ID: <201010250446.o9P4kcid004004@mail.r-bonomi.com> To: freebsd-questions@freebsd.org, temporal@gmail.com Cc: Subject: Re: EVFILT_VNODE doesn't scale to large directory trees? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Oct 2010 04:48:41 -0000 > From owner-freebsd-questions@freebsd.org Sun Oct 24 22:17:42 2010 > Date: Sun, 24 Oct 2010 18:05:34 -0700 > From: Kenton Varda > To: freebsd-questions@freebsd.org > Subject: EVFILT_VNODE doesn't scale to large directory trees? > > Hi all, > > I am trying to write some code which monitors a possibly-large directory > tree for changes. Specifically, it's a build system, and I want it to > automatically start rebuilding whenever I modify a source file. > > So far the approach I've taken is to use EVFILT_VNODE to watch every file > and directory in the tree. This seems to work OK so far, but it worries me > that I have to open() every single file. When I ran the same code on > Darwin, it promptly hit the open file descriptor limit, and I'm worried that > FreeBSD will do the same on larger code trees. > > Is there any better way to accomplish this? Hate to say it, but Linux's > inotify() seems more scalable here. From what I can tell from the docs, it > doesn't require opening the watched files and it will even watch all files > in a directory with one call. You're re-inventing the wheel. 1) Set up a 'makefile' for the entire tree. 2) set up a daemon task that a) cd's to the root direcory of the build tree, b) executes a loop, consisting of 1) the 'make all' command, 2) a reasonably short 'sleep' If 'efficiency' is a concern, then establish a procedure for checking-out/ checking-in files from the repository. When a file is checked in, check for (a) it being a new file, *OR* (b) having changes from the prior version. If either condition is true, fire off 'make' to do the necessary re-build. NOTE: 'cvs' has the above feature as a built-in option. simply specify 'make' as a program to be run when you do a 'cvs commit' to store changes back into the repository. Did I say soemthing about re-inventing the wheel??