From owner-freebsd-questions@FreeBSD.ORG Sun Jun 27 13:43:23 2004 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5D6C016A4CE for ; Sun, 27 Jun 2004 13:43:23 +0000 (GMT) Received: from cannabis.dataforce.net (cannabis.dataforce.net [195.42.160.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6A41E43D41 for ; Sun, 27 Jun 2004 13:43:22 +0000 (GMT) (envelope-from ky@df.ru) Received: from proxy.rostelekom.ru (proxy.rostelekom.ru [195.42.179.169]) by cannabis.dataforce.net (8.11.6p2-DF5/8.8.8) with ESMTP id i5RDgjb28451; Sun, 27 Jun 2004 17:42:46 +0400 From: "Kentucky Mandeloid Mo." To: Luke Kearney Date: Sun, 27 Jun 2004 17:42:44 +0400 User-Agent: KMail/1.5.4 References: <200406271348.07592.ky@df.ru> <20040627201809.B133.LUKEK@meibin.net> In-Reply-To: <20040627201809.B133.LUKEK@meibin.net> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200406271742.44298.ky@df.ru> cc: freebsd-questions@freebsd.org Subject: Re: File tree replication in FreeBSD X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: ky@df.ru List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Jun 2004 13:43:23 -0000 On Sunday 27 June 2004 15:20, Luke Kearney wrote: > On Sun, 27 Jun 2004 13:48:07 +0400 > > "Kentucky Mandeloid Mo." spake thus: > > I'm looking for file replication solution for FreeBSD. > > > > I have a task to replicate about 11G of files with 500k total files > > count. I say replication not mirroring i.e. I need to copy modified files > > to slaves almost imidiately after the change. > > Tried to find some ready solution but fails. > > There is FAM from SGI but it fails to work properly on FreeBSD (it makes > > silly stat(2) calls). > > kevent(2) also don't have anough API to watch >= 500k files. > > So I see the only way is to have kernel module that watch kernel calls > > for file operations. > > > > So is there some other points to build a file replication? > > Can I suggest you investigate rsync ? It will allow you to replicate > file trees pretty much on demand across a LAN or indeed the internet > should you choose to. Works over SSH so it's nice and secure and if you > need virtually instant replication you could conceivably run this from a > cron job every N minutes. > > /usr/ports/net/rsync Yes. I use rsync already and its a great tool. But actually its too slow with numbers of fles > 100k. Its takes more then 5mins to found just few files to update in list of 500k files. So I decide rsync to be a transport manager for my own replication tool. Right now I'm writing a kernel module that catches all file syscalls that can modify a file. Its almost done exept rmdir & unlink calls - I don't know how to get a cwd from a thread. Function kern___getcwd sometimes fails with "not a directory" error. BTW, whats a good place where can I ask about kernel programming?