From owner-freebsd-fs@FreeBSD.ORG  Tue Oct  7 23:01:46 2014
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
Delivered-To: freebsd-fs@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id 5771470D;
 Tue,  7 Oct 2014 23:01:46 +0000 (UTC)
Received: from khavrinen.csail.mit.edu (khavrinen.csail.mit.edu
 [IPv6:2001:470:8b2d:1e1c:21b:21ff:feb8:d7b0])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (Client CN "khavrinen.csail.mit.edu", Issuer "Client CA" (not verified))
 by mx1.freebsd.org (Postfix) with ESMTPS id 18AE0DAF;
 Tue,  7 Oct 2014 23:01:45 +0000 (UTC)
Received: from khavrinen.csail.mit.edu (localhost [127.0.0.1])
 by khavrinen.csail.mit.edu (8.14.7/8.14.7) with ESMTP id s97N1iTN006634
 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256
 verify=FAIL CN=khavrinen.csail.mit.edu issuer=Client+20CA);
 Tue, 7 Oct 2014 19:01:44 -0400 (EDT)
 (envelope-from wollman@khavrinen.csail.mit.edu)
Received: (from wollman@localhost)
 by khavrinen.csail.mit.edu (8.14.7/8.14.7/Submit) id s97N1irN006631;
 Tue, 7 Oct 2014 19:01:44 -0400 (EDT) (envelope-from wollman)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <21556.28888.523838.856998@khavrinen.csail.mit.edu>
Date: Tue, 7 Oct 2014 19:01:44 -0400
From: Garrett Wollman <wollman@csail.mit.edu>
To: Rick Macklem <rmacklem@uoguelph.ca>
Subject: Re: 9.3 NFS client bug?
In-Reply-To: <654363730.60050348.1412720184948.JavaMail.root@uoguelph.ca>
References: <21556.24590.269326.188103@khavrinen.csail.mit.edu>
 <654363730.60050348.1412720184948.JavaMail.root@uoguelph.ca>
X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3
 (khavrinen.csail.mit.edu [127.0.0.1]); Tue, 07 Oct 2014 19:01:44 -0400 (EDT)
Cc: freebsd-fs@freebsd.org
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.18-1
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-fs>,
 <mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs/>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
 <mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 07 Oct 2014 23:01:46 -0000

<<On Tue, 7 Oct 2014 18:16:24 -0400 (EDT), Rick Macklem <rmacklem@uoguelph.ca> said:

> ps: I would like to hear what happens when you test against a UFS volume.

We don't have any UFS volumes, and this 96-drive file server is not
likely ever to have any.

FWIW, I now have a packet capture of the complete test, taken from the
server side, if anyone wants to look at it in detail.

The inner loop of the "sequential delete" benchmark looks like this
(slightly simplified):

    while(1)
    {
      file_ent = readdir(d);
      if(file_ent == NULL)
        break;
      if(file_ent->d_name[0] != '.')
      {
        if(unlink(file_ent->d_name))
        {
          fprintf(stderr, "Can't delete file %s\n", file_ent->d_name);
          return -1;
        }
        count++;
      }
    }
    closedir(d);

This makes me wonder if there isn't some issue with the NFS
VOP_READDIR losing its place in the presence of interleaved deletes.
fts(3) won't see this as it reads the whole directory at once (for
sorting, if requested) before invoking the callback on each entry.

-GAWollman