From owner-freebsd-hackers@freebsd.org Sat Feb 20 12:42:47 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 65775AAE0AA for ; Sat, 20 Feb 2016 12:42:47 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 4D66C9BB for ; Sat, 20 Feb 2016 12:42:47 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 4C8D7AAE0A9; Sat, 20 Feb 2016 12:42:47 +0000 (UTC) Delivered-To: hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 323C9AAE0A8 for ; Sat, 20 Feb 2016 12:42:47 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CF2879BA; Sat, 20 Feb 2016 12:42:46 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u1KCgW8t051461 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sat, 20 Feb 2016 14:42:32 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u1KCgW8t051461 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u1KCgWio051460; Sat, 20 Feb 2016 14:42:32 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 20 Feb 2016 14:42:32 +0200 From: Konstantin Belousov To: Rick Macklem Cc: hackers@freebsd.org, Pedro Giffuni Subject: Re: kqueue of a nfs mounted file not working Message-ID: <20160220124232.GB91220@kib.kiev.ua> References: <9BC3EFA2-945F-4C86-89F6-778873B58469@cs.huji.ac.il> <20151115152635.GB5854@kib.kiev.ua> <3AEC67FD-2E67-4EF9-9D46-818ABF3D8118@cs.huji.ac.il> <661673285.88370232.1447682409478.JavaMail.zimbra@uoguelph.ca> <20151116151513.GF5854@kib.kiev.ua> <87169538.4319034.1455931622630.JavaMail.zimbra@uoguelph.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87169538.4319034.1455931622630.JavaMail.zimbra@uoguelph.ca> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2016 12:42:47 -0000 On Fri, Feb 19, 2016 at 08:27:02PM -0500, Rick Macklem wrote: > Kostik wrote: > > On Mon, Nov 16, 2015 at 09:00:09AM -0500, Rick Macklem wrote: > > > Daniel Braniss wrote: > > > > > > > > > On 15 Nov 2015, at 17:26, Konstantin Belousov > > > > > wrote: > > > > > > > > > > On Sun, Nov 15, 2015 at 11:22:55AM +0200, Daniel Braniss wrote: > > > > >> HI, > > > > >> I???m writing a program to monitor a file using kqueue(2), if the file > > > > >> is > > > > >> local > > > > >> all is OK, but if the file is via a nfs mounted fs, it only works > > > > >> once. > > > > >> stat shows the file growing, but kevent is not triggered. > > > > > > > > > > Does file grow due to local changes on the nfs client, or some other > > > > > client changes the file, while your client tries to get kevent > > > > > notifications ? > > > > > > > > it gets updated by a host which has the file as local, so yes, it gets > > > > updated > > > > by another client/host. > > > > > > > Hmm, I am not surprised that this doesn't work. The only indication to the > > > client that the file has changed on the server is a change in the file's > > > attributes when they're acquired (via a Getattr RPC or similar) from the > > > server. > > > > > > There is a vfs operation called VFS_SYSCTL(). This isn't implemented on > > > the current NFS client. It was implemented on the old one, but only for > > > NFS locking events and I didn't understand what needed to be done, so I > > > didn't do it. > > You probably mean VOP_KQFILTER, not VFS_SYSCTL(). BTW, the later is > > only used by nfs and I do not quite see why its functionality not subsumed > > by the mount options. > > > > WRT VOP_KQFILTER, the default implementation is adequate. What is missed > > for NFS is the knote activation when the code notes that the cached > > metadata is invalidated by server. > > > > > Kostik, do you know if there is a VFS_SYSCTL() call done when the kevent > > > stuff is probing for a file size change? (Or does it not probe and events > > > get triggered via the write syscall or ???) I took a quick look at the > > > kevent > > > stuff, but admit I got lost and couldn't figure out what triggered events > > > being logged? > > > > > > Also, is the event for "file growing" or "file changed"? > > > If it is the latter, all the NFS client can do is look for a change in > > > the file's modify time and this is often at a resolution of 1sec., which > > > implies that a change within the same second as the previous one may not > > > be noticed. (NFSv4 has a Change attribute that is always guaranteed to > > > change, but that is only NFSv4.) Also, you see metadata changes as well > > > as data changes, at least for the NFSv4 attribute. > > > > Please look at the sys/kern/vfs_subr.c lines 4296-4419. There is a bunch > > of the post-VOP hooks which are executed after the filesystem VOP method > > was executed. You would see the complete list of the notifications which > > are sent, and the way they are sent, by calling VFS_KNOTE{_LOCKED}(vp, > > NOTE_XXX). Similar calls should be spread over the nfs client code > > when the attribute cache entries are deleted, possibly comparing old > > and new values to select proper notification. > > > > But you are of course right that client cannot fully implement the > > notifications without the server notifying it, so whatever efforts > > are done for NFSv3, there are definitely will be cases which cannot > > work. I do not know NFSv4 enough to make similar statement, but I think > > something would prevent the complete implementation, e.g. for rename. > > > > This makes me wonder should we need to add the calls to VFS_KNOTE() into > > the nfs client, at all. It is similar to lockd/nolockd options, in > > that the client might be not interested in the server or other clients > > updates, only in the local notifications. This plus the fact that the > > feature cannot have complete implementation, raises the question. > > > I took a quick look at this (thanks to pfg@ for pointing it out to me) > and it appears that DragonFlyBSD does it local to the client (like nolockd). > (I haven't looked at the actual patch, so I don't know how easy it is to port > into the FreeBSD client, but I suspect it isn't too hard to do.) > > http://gitweb.dragonflybsd.org/dragonfly.git/commit/05c073d67e9920d45c46eed73b9390229a50ec8a > > So,does anyone think doing this is a good idea (kevent local to the client only) > or not? Client-local kevents should already work on NFS as-is, by the way of VFS plumbs the VOPs and activates events on file operations. E.g. tail -f and -F work on NFS as is. What was asked for, from my memory, is the feature where client noticing server changes to the file, also caused kevent triggering. This is why I talked about NFS node invalidation triggering kevents. > > Thanks in advance for your comments, rick > ps: Since Daniel Braniss found a workaround for his case, I had put this on the > back burner, but if others think doing it for "client only" (ie not seeing > changes done by other clients), I'll see if the patch can be ported. > > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > >