From owner-freebsd-current@FreeBSD.ORG Fri Apr 24 05:02:49 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8867717C; Fri, 24 Apr 2015 05:02:49 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5DBB812BA; Fri, 24 Apr 2015 05:02:49 +0000 (UTC) Received: from Julian-MBP3.local (ppp121-45-229-105.lns20.per1.internode.on.net [121.45.229.105]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id t3O52idI024853 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Thu, 23 Apr 2015 22:02:47 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <5539CE6F.5040802@freebsd.org> Date: Fri, 24 Apr 2015 13:02:39 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: John Baldwin , freebsd-current@freebsd.org Subject: Re: readdir/telldir/seekdir problem (i think) References: <55386505.70708@freebsd.org> <5538B510.9040603@freebsd.org> <10872728.5fNYcpCvKN@ralph.baldwin.cx> In-Reply-To: <10872728.5fNYcpCvKN@ralph.baldwin.cx> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Apr 2015 05:02:49 -0000 On 4/23/15 9:54 PM, John Baldwin wrote: > On Thursday, April 23, 2015 05:02:08 PM Julian Elischer wrote: >> On 4/23/15 11:20 AM, Julian Elischer wrote: >>> I'm debugging a problem being seen with samba 3.6. >>> >>> basically telldir/seekdir/readdir don't seem to work as advertised.. >> ok so it looks like readdir() (and friends) is totally broken in the face >> of deletes unless you read the entire directory at once or reset to the >> the first file before the deletes, or earlier. > I'm not sure that Samba isn't assuming non-portable behavior. For example: > > >From http://pubs.opengroup.org/onlinepubs/009695399/functions/readdir_r.html > > If a file is removed from or added to the directory after the most recent call > to opendir() or rewinddir(), whether a subsequent call to readdir() returns an > entry for that file is unspecified. > > While this doesn't speak directly to your case, it does note that you will > get inconsistencies if you scan a directory concurrent with add and remove. > > UFS might kind of work actually since deletes do not compact the backing > directory, but I suspect NFS and ZFS would not work. In addition, our > current NFS support for seekdir is pretty flaky and can't be fixed without > changes to return the seek offset for each directory entry (I believe that > the projects/ino64 patches include this since they are breaking the ABI of > the relevant structures already). The ABI breakage makes this a very > non-trivial task. However, even if you have that per-item cookie, it is > likely meaningless in the face of filesystems that use any sort of more > advanced structure than an array (such as trees, etc.) to store directory > entries. POSIX specifically mentions this in the rationale for seekdir: > > > One of the perceived problems of implementation is that returning to a given point in a directory is quite difficult to describe formally, in spite of its intuitive appeal, when systems that use B-trees, hashing functions, or other similar mechanisms to order their directories are considered. The definition of seekdir() and telldir() does not specify whether, when using these interfaces, a given directory entry will be seen at all, or more than once. > > In fact, given that quote, I would argue that what Samba is doing is > non-portable. This would seem to indicate that a conforming seekdir could > just change readdir to immediately return EOF until you call rewinddir. how does readdir know that the directory has been changed? fstat? >