From owner-freebsd-current@FreeBSD.ORG Thu Apr 23 21:50:27 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6581EB22; Thu, 23 Apr 2015 21:50:27 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 1E471181F; Thu, 23 Apr 2015 21:50:26 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2D+AwC0ZzlV/95baINbg15cBYMVwwsJgUcMhTROAoFvFAEBAQEBAQGBCoQhAQEEAQEBICsgCxsOCgICDRkCKQEJJgYIBwQBHASICg23WJRkAQEBAQEBBAEBAQEBAQEbgSGKFoQyAQEcNAeCaIFFBY8ZhjKCP4FHg1A8hXlgiUaDTiOBZYIrIjEBBoEEOYEAAQEB X-IronPort-AV: E=Sophos;i="5.11,634,1422939600"; d="scan'208";a="207651558" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 23 Apr 2015 17:50:25 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id CF888B4001; Thu, 23 Apr 2015 17:50:25 -0400 (EDT) Date: Thu, 23 Apr 2015 17:50:25 -0400 (EDT) From: Rick Macklem To: John Baldwin Cc: freebsd-current@freebsd.org Message-ID: <336285737.24821463.1429825825843.JavaMail.root@uoguelph.ca> In-Reply-To: <10872728.5fNYcpCvKN@ralph.baldwin.cx> Subject: Re: readdir/telldir/seekdir problem (i think) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.11] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Apr 2015 21:50:27 -0000 John Baldwin wrote: > On Thursday, April 23, 2015 05:02:08 PM Julian Elischer wrote: > > On 4/23/15 11:20 AM, Julian Elischer wrote: > > > I'm debugging a problem being seen with samba 3.6. > > > > > > basically telldir/seekdir/readdir don't seem to work as > > > advertised.. > > > > ok so it looks like readdir() (and friends) is totally broken in > > the face > > of deletes unless you read the entire directory at once or reset to > > the > > the first file before the deletes, or earlier. > > I'm not sure that Samba isn't assuming non-portable behavior. For > example: > > From > http://pubs.opengroup.org/onlinepubs/009695399/functions/readdir_r.html > > If a file is removed from or added to the directory after the most > recent call > to opendir() or rewinddir(), whether a subsequent call to readdir() > returns an > entry for that file is unspecified. > > While this doesn't speak directly to your case, it does note that you > will > get inconsistencies if you scan a directory concurrent with add and > remove. > > UFS might kind of work actually since deletes do not compact the > backing > directory, but I suspect NFS and ZFS would not work. In addition, > our > current NFS support for seekdir is pretty flaky and can't be fixed > without > changes to return the seek offset for each directory entry (I believe > that > the projects/ino64 patches include this since they are breaking the > ABI of > the relevant structures already). The ABI breakage makes this a very > non-trivial task. However, even if you have that per-item cookie, it > is > likely meaningless in the face of filesystems that use any sort of > more > advanced structure than an array (such as trees, etc.) to store > directory > entries. POSIX specifically mentions this in the rationale for > seekdir: > > http://pubs.opengroup.org/onlinepubs/009695399/functions/seekdir.html > > One of the perceived problems of implementation is that returning to > a given point in a directory is quite difficult to describe > formally, in spite of its intuitive appeal, when systems that use > B-trees, hashing functions, or other similar mechanisms to order > their directories are considered. The definition of seekdir() and > telldir() does not specify whether, when using these interfaces, a > given directory entry will be seen at all, or more than once. > > In fact, given that quote, I would argue that what Samba is doing is > non-portable. This would seem to indicate that a conforming seekdir > could > just change readdir to immediately return EOF until you call > rewinddir. > Loosely related to this, I have been tempted to modify readdir() to read the entire directory on the first readdir() for NFS, to avoid the readdir()/unlink() problem. My concern was doing this for a very large directory. Maybe it could be done for directories up to some size? rick > -- > John Baldwin > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to > "freebsd-current-unsubscribe@freebsd.org" >