From owner-freebsd-hackers@FreeBSD.ORG Tue Jun 3 02:21:28 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0317E1065721; Tue, 3 Jun 2008 02:21:28 +0000 (UTC) (envelope-from drosih@rpi.edu) Received: from smtp8.server.rpi.edu (smtp8.server.rpi.edu [128.113.2.228]) by mx1.freebsd.org (Postfix) with ESMTP id C25AA8FC1D; Tue, 3 Jun 2008 02:21:27 +0000 (UTC) (envelope-from drosih@rpi.edu) Received: from [128.113.124.153] (crustacean-24.dynamic.rpi.edu [128.113.124.153]) by smtp8.server.rpi.edu (8.13.1/8.13.1) with ESMTP id m5317Fpe027528; Mon, 2 Jun 2008 21:07:17 -0400 Mime-Version: 1.0 Message-Id: In-Reply-To: <4844751C.80704@FreeBSD.org> References: <200805281446.m4SEkojn099133@lurza.secnetix.de> <64200F15-4444-44FE-B904-673543441F35@FreeBSD.org> <4844751C.80704@FreeBSD.org> Date: Mon, 2 Jun 2008 21:07:15 -0400 To: Kris Kennaway , ssouhlal@freebsd.org From: Garance A Drosihn Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-RPI-SA-Score: undef - spam scanning disabled X-CanItPRO-Stream: default X-Canit-Stats-ID: Bayes signature not available X-Scanned-By: CanIt (www . roaringpenguin . com) on 128.113.2.228 Cc: freebsd-hackers@freebsd.org Subject: Re: Impact of having a large number of open file descriptors X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jun 2008 02:21:28 -0000 At 12:33 AM +0200 6/3/08, Kris Kennaway wrote: >Ivan Voras wrote: >>Suleiman Souhlal wrote: >> >>>I have an old patch that makes kqueue monitor every file write on >>>the system and return the inode number in the knote's data field: >>>http://people.freebsd.org/~ssouhlal/testing/kqueue-anyvnode-20050503.diff >>>. >>> >>>I'd think it shouldn't be too hard to make it per-mountpoint.. > >FWIW, I would love to use this. I have situations where I have huge >numbers of files and need to cheaply detect changes so I can >resynchronize them to remote machines. I remember a discussion of changes to MacOS10 in Leopard which made it easier to implement features such as Spotlight and TimeMachine. The description starts here, I think: http://arstechnica.com/reviews/os/mac-os-x-10-5.ars/7 the section on file-system events. The idea I thought was interesting was to save the metadata on a directory basis, instead of saving it on the file. So, if file /some/dir/fname was changed, then they'd record that *some* file under /some/dir has changed. So when your userland process comes along later on, it still has to scan all files in that directory to see which file(s) actually changed. But that's a lot less work than scanning all files in the filesystem, and it also means there is much less data that has to be kept track of. I have no idea how easy it would be to implement something similar on FreeBSD, but the strategy seemed like a pretty neat idea. -- Garance Alistair Drosehn = gad@gilead.netel.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu