Date: Sun, 27 Jun 2004 10:27:31 -0400 From: Bill Moran <wmoran@potentialtech.com> To: ky@df.ru Cc: freebsd-questions@freebsd.org Subject: Re: File tree replication in FreeBSD Message-ID: <20040627102731.63243d11.wmoran@potentialtech.com> In-Reply-To: <200406271742.44298.ky@df.ru> References: <200406271348.07592.ky@df.ru> <20040627201809.B133.LUKEK@meibin.net> <200406271742.44298.ky@df.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
"Kentucky Mandeloid Mo." <ky@df.ru> wrote: > On Sunday 27 June 2004 15:20, Luke Kearney wrote: > > On Sun, 27 Jun 2004 13:48:07 +0400 > > > > "Kentucky Mandeloid Mo." <ky@df.ru> spake thus: > > > I'm looking for file replication solution for FreeBSD. > > > > > > I have a task to replicate about 11G of files with 500k total files > > > count. I say replication not mirroring i.e. I need to copy modified files > > > to slaves almost imidiately after the change. > > > Tried to find some ready solution but fails. > > > There is FAM from SGI but it fails to work properly on FreeBSD (it makes > > > silly stat(2) calls). > > > kevent(2) also don't have anough API to watch >= 500k files. > > > So I see the only way is to have kernel module that watch kernel calls > > > for file operations. > > > > > > So is there some other points to build a file replication? > > > > Can I suggest you investigate rsync ? It will allow you to replicate > > file trees pretty much on demand across a LAN or indeed the internet > > should you choose to. Works over SSH so it's nice and secure and if you > > need virtually instant replication you could conceivably run this from a > > cron job every N minutes. > > > > /usr/ports/net/rsync > > Yes. I use rsync already and its a great tool. > But actually its too slow with numbers of fles > 100k. > Its takes more then 5mins to found just few files to update in list of 500k > files. > > So I decide rsync to be a transport manager for my own replication tool. > Right now I'm writing a kernel module that catches all file syscalls that can > modify a file. Its almost done exept rmdir & unlink calls - I don't know how > to get a cwd from a thread. Function kern___getcwd sometimes fails with "not > a directory" error. > BTW, whats a good place where can I ask about kernel programming? freebsd-hackers@freebsd.org is the best place I know of. -- Bill Moran Potential Technologies http://www.potentialtech.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20040627102731.63243d11.wmoran>