Date: Tue, 07 Oct 2003 13:36:26 +0100 From: Brian Candler <B.Candler@pobox.com> To: FreeBSD-gnats-submit@FreeBSD.org Subject: kern/57696: NFS client readdir terminates prematurely if renaming files Message-ID: <E1A6r4k-0008qR-00@bloodhound.uk.tiscali.com> Resent-Message-ID: <200310071240.h97CeGET013999@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 57696 >Category: kern >Synopsis: NFS client readdir terminates prematurely if renaming files >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Tue Oct 07 05:40:14 PDT 2003 >Closed-Date: >Last-Modified: >Originator: B.Candler@pobox.com >Release: FreeBSD 4.8p3 i386 >Organization: >Environment: FreeBSD-4.7-RELEASE, FreeBSD-4.8p3 >Description: If you have an opendir/readdir loop which rename()s files out of that directory while the directory is being read, and the directory is mounted over NFS, the readdir terminates prematurely - i.e. not all files are seen. Problem verified with the following combinations: FreeBSD client - NetApp server: problem [1] FreeBSD client - Solaris 2.8 server: problem [1] FreeBSD client - FreeBSD server: problem [2] FreeBSD client - Linux server: problem [3] Linux client - Linux server: no problem [3] Solaris 2.8 client - Netapp server: no problem [4] That seems to nail it as a FreeBSD NFS client issue. References: [1] http://www.mail-archive.com/sqwebmail%40inter7.com/msg06643.html [2] http://www.mail-archive.com/sqwebmail%40inter7.com/msg06644.html [3] not yet appeared on archive, message from Stefan Kaltenbrunner [4] http://www.mail-archive.com/sqwebmail%40inter7.com/msg06657.html It's a problem in particular for Maildir messages, when moving files from Maildir/new/* to Maildir/cur/* >How-To-Repeat: Run the following program with an NFS-mounted directory as the command-line argument. The failure mode is: bash-2.05a# ./testnfs /na0/testdir Transferred 169 out of 200 files This program also posted at reference [1] above. /* Demonstrate problem with NFS readdir */ #include <stdio.h> #include <unistd.h> #include <sys/types.h> /* for mkdir */ #include <sys/stat.h> /* for mkdir */ #include <dirent.h> /* opendir/readdir etc */ #define TESTSIZE 200 int main(int argc, char *argv[]) { int i; char fnbuf[1024], fnbuf2[1024]; char *dir = argv[1]; int count; DIR *dp; struct dirent *de; if (argc < 2 || !dir || !dir[0]) dir = "."; sprintf(fnbuf, "%s/new", dir); mkdir(fnbuf, 0777); sprintf(fnbuf, "%s/cur", dir); mkdir(fnbuf, 0777); for (i=0; i<TESTSIZE; i++) { FILE *f; sprintf(fnbuf, "%s/new/MYTESTFILE%d", dir, i); f = fopen(fnbuf, "w"); if (!f) { perror("fopen"); exit(1); } fprintf(f, "Some dummy content\n"); fclose(f); } sprintf(fnbuf, "%s/new", dir); dp = opendir(fnbuf); if (!dp) { perror("opendir"); exit(1); } count = 0; while ((de = readdir(dp))) { if (de->d_name[0] == '.') continue; sprintf(fnbuf, "%s/new/%s", dir, de->d_name); sprintf(fnbuf2,"%s/cur/%s:2,S", dir, de->d_name); if (rename(fnbuf, fnbuf2) < 0) { perror("rename"); fprintf(stderr, "(from %s to %s)\n", fnbuf, fnbuf2); continue; } count++; } fprintf(stderr, "Transferred %d out of %d files\n", count, TESTSIZE); return count != TESTSIZE; } >Fix: No idea. Workaround implemented in courier-imap is to opendir, readdir 20 items into array, closedir, process the 20 items, rinse and repeat. >Release-Note: >Audit-Trail: >Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E1A6r4k-0008qR-00>