From owner-freebsd-fs@freebsd.org Sun Jan 3 21:36:53 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EBE24A60B6C for ; Sun, 3 Jan 2016 21:36:52 +0000 (UTC) (envelope-from jordanhubbard@icloud.com) Received: from pv33p03im-asmtp002.me.com (pv33p03im-asmtp002.me.com [17.143.180.11]) (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D1E561453; Sun, 3 Jan 2016 21:36:52 +0000 (UTC) (envelope-from jordanhubbard@icloud.com) Received: from [10.11.111.197] (50-250-239-90-static.hfc.comcastbusiness.net [50.250.239.90]) by pv33p03im-asmtp002.me.com (Oracle Communications Messaging Server 7.0.5.36.0 64bit (built Sep 8 2015)) with ESMTPSA id <0O0E00EJDC10C830@pv33p03im-asmtp002.me.com>; Sun, 03 Jan 2016 21:36:37 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-01-03_14:,, signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 clxscore=1011 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1510270003 definitions=main-1601030412 Content-type: text/plain; charset=utf-8 MIME-version: 1.0 (Mac OS X Mail 9.2 \(3112\)) Subject: Re: Monitoring FS changes From: Jordan Hubbard In-reply-to: <1451855317.3739776.481776018.64F6C4D9@webmail.messagingengine.com> Date: Sun, 03 Jan 2016 13:36:36 -0800 Cc: Chris Stankevitz , FreeBSD Filesystems Content-transfer-encoding: quoted-printable Message-id: <046580B0-BF8E-4A83-B1BB-28C632B1B9B5@icloud.com> References: <5684D810.6070700@stankevitz.com> <1451855317.3739776.481776018.64F6C4D9@webmail.messagingengine.com> To: Mark Felder X-Mailer: Apple Mail (2.3112) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jan 2016 21:36:53 -0000 > On Jan 3, 2016, at 1:08 PM, Mark Felder wrote: >=20 > If someone, anyone out there is capable of bringing us something that > does scale it would be greatly appreciated. Lots of nice Linux = software > uses this, but when they do port to FreeBSD we have to do full > filesystem scans. It's such a waste. I=E2=80=99ve been pondering this for awhile since a lot of interesting = enterprise features require a working filesystem change notification = mechanism that scales to thousands or even millions of files (how did we = bump into the 32 bit NFS file handle problem at iXsystems? Somebody = tried to share more than 4 billion files over NFS - Enterprise folks do = some crazy s**t!). The big question is less whether it=E2=80=99s possible and more what = kind of mechanism people will find palatable. The OS X FSEvents = mechanism works reasonably well and is used constantly to trigger things = like spotlight search indexing and such, and I was by no means involved = in its creation at Apple so I can only speak peripherally to the = implementation, but it seems like it took a fairly long time for it to = become =E2=80=9Clight weight=E2=80=9D enough to use without the overhead = being punitive. Any similar mechanism in FreeBSD would also have to go = through some evolutionary performance iterations - do people want it = badly enough to invest in it long-term? I don=E2=80=99t know, but I do = know that a long-term investment would be necessary to really make it = work well and provide all of the appropriate APIs for talking to it. I think we can probably all agree that Linux inotify wouldn=E2=80=99t be = worth the trouble. =46rom the wikipedia page: =E2=80=A2 Inotify does not support recursively watching = directories, meaning that a separate inotify watch must be created for = every subdirectory.[4] =E2=80=A2 Inotify does report some but not all events in sysfs = and procfs. =E2=80=A2 Notification via inotify requires the kernel to be = aware of all relevant filesystem events, which is not always possible = for networked filesystems such as NFS where changes made by one client = are not immediately broadcast to other clients. =E2=80=A2 Rename events are not handled directly; i.e., inotify = issues two separate events that must be examined and matched in a = context of potential race conditions. I think the first issue alone is a deal killer. Having to walk the = filesystem tree posting notifications on every [new] directory just to = watch a filesystem in its entirety would be pretty onerous and = failure-prone to boot. By contrast: = https://en.wikipedia.org/wiki/FSEvents This is also not to say that I would expect anything in FreeBSD to be = API-compatible (though the upstream clients would probably grumble at = yet another notification mechanism API to #ifdef into their code), = simply that there are only so many design patterns to follow. A = filesystem change is a filesystem change. Everything beyond that is = just a glorified pub/sub mechanism. Assuming there=E2=80=99s interest, I could potentially see throwing some = engineering effort into this. - Jordan