From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 18 15:48:51 2005 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5459016A41F for ; Fri, 18 Nov 2005 15:48:51 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from mh2.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8C62D43D6E for ; Fri, 18 Nov 2005 15:48:46 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh2.centtech.com (8.13.1/8.13.1) with ESMTP id jAIFmjsE042134; Fri, 18 Nov 2005 09:48:45 -0600 (CST) (envelope-from anderson@centtech.com) Message-ID: <437DF7D3.9020902@centtech.com> Date: Fri, 18 Nov 2005 09:48:35 -0600 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.12) Gecko/20051021 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Cornelis Swanepoel References: <20051118120039.BB3EE16A421@hub.freebsd.org> <20051118094225.E72964@bowser.eecs.harvard.edu> In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.82/1178/Thu Nov 17 23:27:25 2005 on mh2.centtech.com X-Virus-Status: Clean Cc: freebsd-hackers@freebsd.org Subject: Re: Filesystem monitoring question X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Nov 2005 15:48:51 -0000 Cornelis Swanepoel wrote: > Before all else let me just say that my programming experience is confined > to userland but I am very eager to learn about interaction with the FreeBSD > kernel. > > There are many possible answers to this question. Can you explain > >>more about exactly what you need? > > > > > This will be a server setup from scratch to accomodate the requirements. > Currently the share is a full partition but it can be setup on the new > server as a directory. > A number of SMB and NFS clients, around 30, will need to write to this > directory. > Writing to the directory is 24/7 since some clients VPN into the network > from different time zones. > > When they have completed a write operation I need to trigger some code that > will act upon the file just written. > This code generates a unique id for the file, stats the file, compresses the > file(if over set limit), generates a preview of the file and stores some > info about the file and its owner in a MySQL db and a log on a seperate > machine. > It then moves the file to a different location on the same machine (which is > inaccesible to the NFS and SMB clients) and renames it as its unique > identifier. > > At this point it is out of my hands. > > > Depending on what you can and cannot access ... If everything .. goes > >>through VFS, that would work. But if >> > > it's a production server, you might not be able to get permission to > >>fiddle with the kernel. > > > > > This machine will be built from scratch and any kernel fiddling is possible. > > ktrace might do everything you need, but I understand that it has an > >>impact on system performance under load. I don't know if that matters >>to you at all. It also might be awkward if the SMB and NFS servers >>are kernel processes. (I just don't know; I've only used ktrace on >>ordinary user-land processes, and not even much of that.) > > > > >>From the limited research I've done it seems that ktrace might not be the > answer and I've been looking more into kqueue. > I will definitely look into the VFS option. > > As I said at the start, this is all new to me and I greatly appreciate the > feedback so far. > > If my explaination of the requirements is lacking please let me know > specifically and I can elaborate. > > Thanks again and any further suggestions would be much appreciated. Here's a non-optimal way, but just tossing it in anyway: make a clone of nullfs (call it monitorfs say), modify it so that it outputs the write's using a printf (or any other more efficient method). Then you kldload your monitorfs module, and your tool watches the output. Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------