From owner-freebsd-questions@FreeBSD.ORG Mon Oct 25 06:24:58 2010 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D0B40106564A for ; Mon, 25 Oct 2010 06:24:58 +0000 (UTC) (envelope-from temporal@gmail.com) Received: from mail-ew0-f54.google.com (mail-ew0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 562D68FC17 for ; Mon, 25 Oct 2010 06:24:57 +0000 (UTC) Received: by ewy28 with SMTP id 28so1307481ewy.13 for ; Sun, 24 Oct 2010 23:24:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=GjukVFVmpZtmzKaMHu2vWpLp1rt5zzPCsagl26mhh7g=; b=LWFsF7A7ioHha3q82LKgnOD+hGbSm64RdaAJoidnt3+KumOrcwaXPL8XXVsIhEnJmd siCbLdCEWwnM4gmgf8mYhdJ6ZZCYJweh83NGz9u7SkbDM6rBVTq1fMjhkXXl705BgVBD cIhryiBJBVZA8kkTvuL8uYLkvDJDXbPFkOwRE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=nNzna5UJPNZQN6TM+40qXdrjZQJvR4hfK2DQmWwc42wKemQGAUUP/BsEMjLVg/nxuq yrTLK+yOPCEiJtTjjX2ZKDSkrbfRyVE24yxYo7ogKgfxs0GmjjA8tkGm8fA5cHIoUGMV lER0HRZSFwcQ8gcAx4DqsOYUtIZx9IfcRxIJE= MIME-Version: 1.0 Received: by 10.14.127.199 with SMTP id d47mr4494881eei.45.1287987897162; Sun, 24 Oct 2010 23:24:57 -0700 (PDT) Received: by 10.14.119.141 with HTTP; Sun, 24 Oct 2010 23:24:57 -0700 (PDT) In-Reply-To: <201010250446.o9P4kcid004004@mail.r-bonomi.com> References: <201010250446.o9P4kcid004004@mail.r-bonomi.com> Date: Sun, 24 Oct 2010 23:24:57 -0700 Message-ID: From: Kenton Varda To: Robert Bonomi Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-questions@freebsd.org Subject: Re: EVFILT_VNODE doesn't scale to large directory trees? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Oct 2010 06:24:58 -0000 That doesn't answer my question. I'm not even using make. I could write a few thousand words describing exactly what I'm trying to do and why it does, in fact, make sense, but it's really beside the point. I just want to know if there is any scalable way to monitor a very large directory tree for changes. Is there? On Sun, Oct 24, 2010 at 9:46 PM, Robert Bonomi wrote: > > From owner-freebsd-questions@freebsd.org Sun Oct 24 22:17:42 2010 > > Date: Sun, 24 Oct 2010 18:05:34 -0700 > > From: Kenton Varda > > To: freebsd-questions@freebsd.org > > Subject: EVFILT_VNODE doesn't scale to large directory trees? > > > > Hi all, > > > > I am trying to write some code which monitors a possibly-large directory > > tree for changes. Specifically, it's a build system, and I want it to > > automatically start rebuilding whenever I modify a source file. > > > > So far the approach I've taken is to use EVFILT_VNODE to watch every file > > and directory in the tree. This seems to work OK so far, but it worries > me > > that I have to open() every single file. When I ran the same code on > > Darwin, it promptly hit the open file descriptor limit, and I'm worried > that > > FreeBSD will do the same on larger code trees. > > > > Is there any better way to accomplish this? Hate to say it, but Linux's > > inotify() seems more scalable here. From what I can tell from the docs, > it > > doesn't require opening the watched files and it will even watch all > files > > in a directory with one call. > > > You're re-inventing the wheel. > > 1) Set up a 'makefile' for the entire tree. > > 2) set up a daemon task that > a) cd's to the root direcory of the build tree, > b) executes a loop, consisting of > 1) the 'make all' command, > 2) a reasonably short 'sleep' > > > If 'efficiency' is a concern, then establish a procedure for checking-out/ > checking-in files from the repository. When a file is checked in, check > for (a) it being a new file, *OR* (b) having changes from the prior > version. > If either condition is true, fire off 'make' to do the necessary re-build. > > NOTE: 'cvs' has the above feature as a built-in option. simply specify > 'make' as a program to be run when you do a 'cvs commit' to store changes > back into the repository. > > Did I say soemthing about re-inventing the wheel?? > > > >