From owner-freebsd-questions@FreeBSD.ORG Sun Jun 27 14:27:52 2004 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 130AA16A4CE for ; Sun, 27 Jun 2004 14:27:52 +0000 (GMT) Received: from internet.potentialtech.com (h-66-167-251-6.phlapafg.covad.net [66.167.251.6]) by mx1.FreeBSD.org (Postfix) with ESMTP id C282343D39 for ; Sun, 27 Jun 2004 14:27:51 +0000 (GMT) (envelope-from wmoran@potentialtech.com) Received: from working.potentialtech.com (pa-plum-cmts1e-68-68-113-64.pittpa.adelphia.net [68.68.113.64]) by internet.potentialtech.com (Postfix) with ESMTP id 41FA969A39; Sun, 27 Jun 2004 10:27:32 -0400 (EDT) Date: Sun, 27 Jun 2004 10:27:31 -0400 From: Bill Moran To: ky@df.ru Message-Id: <20040627102731.63243d11.wmoran@potentialtech.com> In-Reply-To: <200406271742.44298.ky@df.ru> References: <200406271348.07592.ky@df.ru> <20040627201809.B133.LUKEK@meibin.net> <200406271742.44298.ky@df.ru> Organization: Potential Technologies X-Mailer: Sylpheed version 0.9.12 (GTK+ 1.2.10; i386-portbld-freebsd4.9) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit cc: freebsd-questions@freebsd.org Subject: Re: File tree replication in FreeBSD X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Jun 2004 14:27:52 -0000 "Kentucky Mandeloid Mo." wrote: > On Sunday 27 June 2004 15:20, Luke Kearney wrote: > > On Sun, 27 Jun 2004 13:48:07 +0400 > > > > "Kentucky Mandeloid Mo." spake thus: > > > I'm looking for file replication solution for FreeBSD. > > > > > > I have a task to replicate about 11G of files with 500k total files > > > count. I say replication not mirroring i.e. I need to copy modified files > > > to slaves almost imidiately after the change. > > > Tried to find some ready solution but fails. > > > There is FAM from SGI but it fails to work properly on FreeBSD (it makes > > > silly stat(2) calls). > > > kevent(2) also don't have anough API to watch >= 500k files. > > > So I see the only way is to have kernel module that watch kernel calls > > > for file operations. > > > > > > So is there some other points to build a file replication? > > > > Can I suggest you investigate rsync ? It will allow you to replicate > > file trees pretty much on demand across a LAN or indeed the internet > > should you choose to. Works over SSH so it's nice and secure and if you > > need virtually instant replication you could conceivably run this from a > > cron job every N minutes. > > > > /usr/ports/net/rsync > > Yes. I use rsync already and its a great tool. > But actually its too slow with numbers of fles > 100k. > Its takes more then 5mins to found just few files to update in list of 500k > files. > > So I decide rsync to be a transport manager for my own replication tool. > Right now I'm writing a kernel module that catches all file syscalls that can > modify a file. Its almost done exept rmdir & unlink calls - I don't know how > to get a cwd from a thread. Function kern___getcwd sometimes fails with "not > a directory" error. > BTW, whats a good place where can I ask about kernel programming? freebsd-hackers@freebsd.org is the best place I know of. -- Bill Moran Potential Technologies http://www.potentialtech.com