From owner-freebsd-bugs@FreeBSD.ORG Tue Oct 7 05:40:19 2003 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2911F16A4B3 for ; Tue, 7 Oct 2003 05:40:19 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id C510443FE5 for ; Tue, 7 Oct 2003 05:40:16 -0700 (PDT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h97CeGFY014000 for ; Tue, 7 Oct 2003 05:40:16 -0700 (PDT) (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h97CeGET013999; Tue, 7 Oct 2003 05:40:16 -0700 (PDT) (envelope-from gnats) Resent-Date: Tue, 7 Oct 2003 05:40:16 -0700 (PDT) Resent-Message-Id: <200310071240.h97CeGET013999@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Brian Candler Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BF05B16A4B3 for ; Tue, 7 Oct 2003 05:36:49 -0700 (PDT) Received: from internal.mail.uk.tiscali.com (internal.mail.uk.tiscali.com [212.74.96.51]) by mx1.FreeBSD.org (Postfix) with ESMTP id 13F3B43FF9 for ; Tue, 7 Oct 2003 05:36:34 -0700 (PDT) (envelope-from b.candler@pobox.com) Received: from [10.44.66.27] (helo=bloodhound.uk.tiscali.com) by internal.mail.uk.tiscali.com with esmtp (Exim 4.12) id 1A6r4r-0000ve-00 for FreeBSD-gnats-submit@freebsd.org; Tue, 07 Oct 2003 13:36:33 +0100 Received: from telinco by bloodhound.uk.tiscali.com with local (Exim 4.12) id 1A6r4k-0008qR-00 for FreeBSD-gnats-submit@freebsd.org; Tue, 07 Oct 2003 13:36:26 +0100 Message-Id: Date: Tue, 07 Oct 2003 13:36:26 +0100 From: Brian Candler To: FreeBSD-gnats-submit@FreeBSD.org X-Send-Pr-Version: 3.113 Subject: kern/57696: NFS client readdir terminates prematurely if renaming files X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Brian Candler List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2003 12:40:19 -0000 >Number: 57696 >Category: kern >Synopsis: NFS client readdir terminates prematurely if renaming files >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Tue Oct 07 05:40:14 PDT 2003 >Closed-Date: >Last-Modified: >Originator: B.Candler@pobox.com >Release: FreeBSD 4.8p3 i386 >Organization: >Environment: FreeBSD-4.7-RELEASE, FreeBSD-4.8p3 >Description: If you have an opendir/readdir loop which rename()s files out of that directory while the directory is being read, and the directory is mounted over NFS, the readdir terminates prematurely - i.e. not all files are seen. Problem verified with the following combinations: FreeBSD client - NetApp server: problem [1] FreeBSD client - Solaris 2.8 server: problem [1] FreeBSD client - FreeBSD server: problem [2] FreeBSD client - Linux server: problem [3] Linux client - Linux server: no problem [3] Solaris 2.8 client - Netapp server: no problem [4] That seems to nail it as a FreeBSD NFS client issue. References: [1] http://www.mail-archive.com/sqwebmail%40inter7.com/msg06643.html [2] http://www.mail-archive.com/sqwebmail%40inter7.com/msg06644.html [3] not yet appeared on archive, message from Stefan Kaltenbrunner [4] http://www.mail-archive.com/sqwebmail%40inter7.com/msg06657.html It's a problem in particular for Maildir messages, when moving files from Maildir/new/* to Maildir/cur/* >How-To-Repeat: Run the following program with an NFS-mounted directory as the command-line argument. The failure mode is: bash-2.05a# ./testnfs /na0/testdir Transferred 169 out of 200 files This program also posted at reference [1] above. /* Demonstrate problem with NFS readdir */ #include #include #include /* for mkdir */ #include /* for mkdir */ #include /* opendir/readdir etc */ #define TESTSIZE 200 int main(int argc, char *argv[]) { int i; char fnbuf[1024], fnbuf2[1024]; char *dir = argv[1]; int count; DIR *dp; struct dirent *de; if (argc < 2 || !dir || !dir[0]) dir = "."; sprintf(fnbuf, "%s/new", dir); mkdir(fnbuf, 0777); sprintf(fnbuf, "%s/cur", dir); mkdir(fnbuf, 0777); for (i=0; id_name[0] == '.') continue; sprintf(fnbuf, "%s/new/%s", dir, de->d_name); sprintf(fnbuf2,"%s/cur/%s:2,S", dir, de->d_name); if (rename(fnbuf, fnbuf2) < 0) { perror("rename"); fprintf(stderr, "(from %s to %s)\n", fnbuf, fnbuf2); continue; } count++; } fprintf(stderr, "Transferred %d out of %d files\n", count, TESTSIZE); return count != TESTSIZE; } >Fix: No idea. Workaround implemented in courier-imap is to opendir, readdir 20 items into array, closedir, process the 20 items, rinse and repeat. >Release-Note: >Audit-Trail: >Unformatted: