From owner-freebsd-fs@freebsd.org Thu Apr 29 08:24:24 2021 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AE9DD5FD24E for ; Thu, 29 Apr 2021 08:24:24 +0000 (UTC) (envelope-from njm@njm.me.uk) Received: from smtp001-out.apm-internet.net (smtp001-out.apm-internet.net [85.119.248.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4FW7nz33YSz4SXV for ; Thu, 29 Apr 2021 08:24:22 +0000 (UTC) (envelope-from njm@njm.me.uk) Received: (qmail 4274 invoked from network); 29 Apr 2021 08:24:20 -0000 X-APM-Out-ID: 16196846600427 X-APM-Authkey: 18389/1(18389/1) 1 Received: from unknown (HELO meld.njm.me.uk) (86.179.69.31) by smtp001.apm-internet.net with SMTP; 29 Apr 2021 08:24:20 -0000 Received: from triton.njm.me.uk (triton.njm.me.uk [192.168.144.133]) by meld.njm.me.uk (8.16.1/8.16.1) with ESMTPS id 13T8OJkB001518 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 29 Apr 2021 09:24:20 +0100 (BST) (envelope-from njm@njm.me.uk) Received: from localhost (localhost [127.0.0.1]) by triton.njm.me.uk (8.16.1/8.16.1) with ESMTP id 13T8OJtZ073848 for ; Thu, 29 Apr 2021 09:24:19 +0100 (BST) (envelope-from njm@njm.me.uk) Date: Thu, 29 Apr 2021 09:24:19 +0100 From: "N.J. Mann" To: freebsd-fs@freebsd.org Subject: readdir() -> d_type always zero on NFS v3 mounted filesystems Message-ID: <2E84A420CCC10A73504624DE@triton.njm.me.uk> X-Mailer: Mulberry/4.0.8 (Linux/x86) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="==========16324982B0D1D3197C7A==========" X-Rspamd-Queue-Id: 4FW7nz33YSz4SXV X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of njm@njm.me.uk has no SPF policy when checking 85.119.248.222) smtp.mailfrom=njm@njm.me.uk X-Spamd-Result: default: False [-0.77 / 15.00]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[85.119.248.222:from]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[85.119.248.222:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_NONE(0.00)[]; NEURAL_HAM_SHORT(-0.57)[-0.568]; CTYPE_MIXED_BOGUS(1.00)[]; DMARC_NA(0.00)[njm.me.uk]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+]; ASN(0.00)[asn:35259, ipnet:85.119.248.0/21, country:GB]; RCVD_TLS_LAST(0.00)[]; MAILMAN_DEST(0.00)[freebsd-fs]; RCVD_IN_DNSWL_LOW(-0.10)[85.119.248.222:from] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Apr 2021 08:24:24 -0000 --==========16324982B0D1D3197C7A========== Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline Hi, I recently changed over from using svn to gitup to update /usr/ports on my local system and have been experiencing problems since. At first I thought it was an issue with gitup itself, but now I believe it is either a kernel issue or a configuration issue. I originally posted about the problem to the freebsd-ports mailing list: https://lists.freebsd.org/pipermail/freebsd-ports/2021-April/120929.html Since then I have dug deeper and come to the conclusion that it is not a problem with gitup. The issue I am seeing is that gitup is unable to delete files and directories, even complete ports, which have been removed from the repository. gitup basically does the following: prune_tree(base_path) { if ((directory = opendir(base_path)) != NULL) { while ((entry = readdir(directory)) != NULL) { snprintf(full_path, sizeof(full_path), "%s/%s", base_path, entry->d_name); if (entry->d_type == DT_DIR) { prune_tree(full_path); } else { if ((remove(full_path) != 0) && (errno != ENOENT)) err(EXIT_FAILURE, "prune_tree: cannot remove %s", full_path); } } closedir(directory); if (rmdir(base_path) != 0) err(EXIT_FAILURE, "prune_tree: cannot remove %s", base_path); } } When gitup is run on either a UFS or ZFS file system this works. However, when I run it on a NFS v3 mounted filesystem it fails. Liberal addition of printf's shows that for non-NFS mounted filesystems d_type contains the correct value for each file/directory, but for NFS mounted file systems it is zero - other fields such as d_name and d_namlen are correct. While debugging this I added a call to stat() and that always returns the correct values. At this point I started digging in libc and quickly found that readdir() is basically a wrapper around a system call. Digging in the kernel I quickly got out of my depth and hence my posting here. I then wrote a simple test programme which also shows the issue. I have attached the source for the test programme and the output from two runs, the first on an NFS mounted file system and the second on a local UFS file system. Before gitup started failing I had not seen any failures like this and that was why I initially assumed this must be a gitup issue. I now believe this is either a kernel issue or a confuration issue. My configuration is as follows: file server: /exports/ports - ZFS file system for FreeBSD ports repo exported via NFS /remote/ports - FreeBSD ports repo NFS (v3 rw,tcp) mounted from /export/ports on file server /usr/ports - symbolic link to /remote/ports client machines: /remote/ports - FreeBSD ports repo NFS (v3 ro,tcp) mounted from /export/ports on file server /usr/ports - symbolic link to /remote/ports I run gitup in /remote/ports on the file server and then pkg, portmaster, &.c, in /usr/ports on the file server and then the client machines. All systems are running 11-STABLE from about 12 days ago - I usually update every couple of weeks. Any assistance, suggestions, patches, clue bats, will be gratefully accepted. Regards, Nick. -- --==========16324982B0D1D3197C7A========== Content-Type: text/plain; charset=us-ascii; name="my_search.c.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="my_search.c.txt"; size=2044 /* * vim:sw=4 sts=4 expandtab * */ #include #include #include #include #include #include #include #include static void my_recurse(char *path) { DIR *directory = NULL; struct dirent *entry = NULL; struct stat sb; char full_path[strlen(path) + 1 + MAXNAMLEN + 1]; if ((directory = opendir(path)) != NULL) { printf("opendir: %s\n", path); while ((entry = readdir(directory)) != NULL) { printf("readdir: %c %u %s\n", entry->d_type == DT_DIR ? 'D' : '_', entry->d_namlen, entry->d_name); snprintf(full_path, sizeof(full_path), "%s/%s", path, entry->d_name); stat(full_path, &sb); printf("stat: %c\n", S_ISDIR(sb.st_mode) ? 'D' : '_'); if (S_ISDIR(sb.st_mode) != 0) { if ((entry->d_namlen == 1) && (strcmp(entry->d_name, "." ) == 0)) continue; if ((entry->d_namlen == 2) && (strcmp(entry->d_name, "..") == 0)) continue; printf("recurse: %s\n", full_path); my_recurse(full_path); } else printf("file: %s\n", full_path); } printf("closedir: %s\n", path); closedir(directory); } } int main(int argc, char *argv[]) { int ch, i; char *path; while ((ch = getopt(argc, argv, "v")) != -1) switch (ch) { case 'v': /* Do nothing, for now */ break; default: fprintf(stderr, "Illegal option -- %c\n", ch); exit (-1); } argc -= optind; argv += optind; if (argc > 1) { fprintf(stderr, "Too many parameters\n"); exit(-1); } else if (argc < 1) { fprintf(stderr, "Not enough parameters\n"); exit(-1); } path = *argv; my_recurse(path); return 0; } --==========16324982B0D1D3197C7A========== Content-Type: text/plain; charset=us-ascii; name="test1.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="test1.txt"; size=458 opendir: /remote/ports/net/gitup readdir: _ 1 . stat: D readdir: _ 2 .. stat: D readdir: _ 9 pkg-descr stat: _ file: /remote/ports/net/gitup/pkg-descr readdir: _ 8 distinfo stat: _ file: /remote/ports/net/gitup/distinfo readdir: _ 8 Makefile stat: _ file: /remote/ports/net/gitup/Makefile readdir: _ 9 pkg-plist stat: _ file: /remote/ports/net/gitup/pkg-plist closedir: /remote/ports/net/gitup --==========16324982B0D1D3197C7A========== Content-Type: text/plain; charset=us-ascii; name="test2.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="test2.txt"; size=8879 opendir: /local/av/radio/6Music/ readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 13 DJ_Shadow.m4a stat: _ file: /local/av/radio/6Music//DJ_Shadow.m4a readdir: D 4 2020 stat: D recurse: /local/av/radio/6Music//2020 opendir: /local/av/radio/6Music//2020 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 2 05 stat: D recurse: /local/av/radio/6Music//2020/05 opendir: /local/av/radio/6Music//2020/05 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 2 23 stat: D recurse: /local/av/radio/6Music//2020/05/23 opendir: /local/av/radio/6Music//2020/05/23 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/05/23/1100 opendir: /local/av/radio/6Music//2020/05/23/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 44 The_Huey_Show:_Funky_Space_Reincarnation.m4a stat: _ file: /local/av/radio/6Music//2020/05/23/1100/The_Huey_Show:_Funky_Space_Reincarnation.m4a closedir: /local/av/radio/6Music//2020/05/23/1100 closedir: /local/av/radio/6Music//2020/05/23 readdir: D 2 30 stat: D recurse: /local/av/radio/6Music//2020/05/30 opendir: /local/av/radio/6Music//2020/05/30 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/05/30/1100 opendir: /local/av/radio/6Music//2020/05/30/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 29 The_Huey_Show:_Buggin_Out.m4a stat: _ file: /local/av/radio/6Music//2020/05/30/1100/The_Huey_Show:_Buggin_Out.m4a closedir: /local/av/radio/6Music//2020/05/30/1100 closedir: /local/av/radio/6Music//2020/05/30 closedir: /local/av/radio/6Music//2020/05 readdir: D 2 06 stat: D recurse: /local/av/radio/6Music//2020/06 opendir: /local/av/radio/6Music//2020/06 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 2 06 stat: D recurse: /local/av/radio/6Music//2020/06/06 opendir: /local/av/radio/6Music//2020/06/06 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/06/06/1100 opendir: /local/av/radio/6Music//2020/06/06/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 40 The_Huey_Show:_Uptown_meets_Downtown.m4a stat: _ file: /local/av/radio/6Music//2020/06/06/1100/The_Huey_Show:_Uptown_meets_Downtown.m4a closedir: /local/av/radio/6Music//2020/06/06/1100 closedir: /local/av/radio/6Music//2020/06/06 readdir: D 2 13 stat: D recurse: /local/av/radio/6Music//2020/06/13 opendir: /local/av/radio/6Music//2020/06/13 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/06/13/1100 opendir: /local/av/radio/6Music//2020/06/13/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 29 The_Huey_Show:_Heads_Know.m4a stat: _ file: /local/av/radio/6Music//2020/06/13/1100/The_Huey_Show:_Heads_Know.m4a closedir: /local/av/radio/6Music//2020/06/13/1100 closedir: /local/av/radio/6Music//2020/06/13 readdir: D 2 27 stat: D recurse: /local/av/radio/6Music//2020/06/27 opendir: /local/av/radio/6Music//2020/06/27 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/06/27/1100 opendir: /local/av/radio/6Music//2020/06/27/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 41 The_Huey_Show:_Louie_Vega_Block_Party.m4a stat: _ file: /local/av/radio/6Music//2020/06/27/1100/The_Huey_Show:_Louie_Vega_Block_Party.m4a closedir: /local/av/radio/6Music//2020/06/27/1100 closedir: /local/av/radio/6Music//2020/06/27 closedir: /local/av/radio/6Music//2020/06 readdir: D 2 07 stat: D recurse: /local/av/radio/6Music//2020/07 opendir: /local/av/radio/6Music//2020/07 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 2 04 stat: D recurse: /local/av/radio/6Music//2020/07/04 opendir: /local/av/radio/6Music//2020/07/04 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/07/04/1100 opendir: /local/av/radio/6Music//2020/07/04/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 52 The_Huey_Show:_Brazil_Block_Party_and_Louie_Vega.m4a stat: _ file: /local/av/radio/6Music//2020/07/04/1100/The_Huey_Show:_Brazil_Block_Party_and_Louie_Vega.m4a closedir: /local/av/radio/6Music//2020/07/04/1100 closedir: /local/av/radio/6Music//2020/07/04 readdir: D 2 11 stat: D recurse: /local/av/radio/6Music//2020/07/11 opendir: /local/av/radio/6Music//2020/07/11 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/07/11/1100 opendir: /local/av/radio/6Music//2020/07/11/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 35 The_Huey_Show:_Cuba_Block_Party.m4a stat: _ file: /local/av/radio/6Music//2020/07/11/1100/The_Huey_Show:_Cuba_Block_Party.m4a closedir: /local/av/radio/6Music//2020/07/11/1100 closedir: /local/av/radio/6Music//2020/07/11 readdir: D 2 18 stat: D recurse: /local/av/radio/6Music//2020/07/18 opendir: /local/av/radio/6Music//2020/07/18 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/07/18/1100 opendir: /local/av/radio/6Music//2020/07/18/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 42 The_Huey_Show:_Puerto_Rico_Block_Party.m4a stat: _ file: /local/av/radio/6Music//2020/07/18/1100/The_Huey_Show:_Puerto_Rico_Block_Party.m4a closedir: /local/av/radio/6Music//2020/07/18/1100 closedir: /local/av/radio/6Music//2020/07/18 closedir: /local/av/radio/6Music//2020/07 readdir: D 2 08 stat: D recurse: /local/av/radio/6Music//2020/08 opendir: /local/av/radio/6Music//2020/08 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 2 01 stat: D recurse: /local/av/radio/6Music//2020/08/01 opendir: /local/av/radio/6Music//2020/08/01 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/08/01/1100 opendir: /local/av/radio/6Music//2020/08/01/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 35 The_Huey_Show:_Chuck_D_turns_60.m4a stat: _ file: /local/av/radio/6Music//2020/08/01/1100/The_Huey_Show:_Chuck_D_turns_60.m4a closedir: /local/av/radio/6Music//2020/08/01/1100 closedir: /local/av/radio/6Music//2020/08/01 readdir: D 2 08 stat: D recurse: /local/av/radio/6Music//2020/08/08 opendir: /local/av/radio/6Music//2020/08/08 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/08/08/1100 opendir: /local/av/radio/6Music//2020/08/08/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 59 The_Huey_Show:_New_music_from_Felt_Skyzoo_and_Sly5thAve.m4a stat: _ file: /local/av/radio/6Music//2020/08/08/1100/The_Huey_Show:_New_music_from_Felt_Skyzoo_and_Sly5thAve.m4a closedir: /local/av/radio/6Music//2020/08/08/1100 closedir: /local/av/radio/6Music//2020/08/08 readdir: D 2 15 stat: D recurse: /local/av/radio/6Music//2020/08/15 opendir: /local/av/radio/6Music//2020/08/15 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/08/15/1100 opendir: /local/av/radio/6Music//2020/08/15/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 17 The_Huey_Show.m4a stat: _ file: /local/av/radio/6Music//2020/08/15/1100/The_Huey_Show.m4a closedir: /local/av/radio/6Music//2020/08/15/1100 closedir: /local/av/radio/6Music//2020/08/15 readdir: D 2 22 stat: D recurse: /local/av/radio/6Music//2020/08/22 opendir: /local/av/radio/6Music//2020/08/22 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/08/22/1100 opendir: /local/av/radio/6Music//2020/08/22/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 17 The_Huey_Show.m4a stat: _ file: /local/av/radio/6Music//2020/08/22/1100/The_Huey_Show.m4a closedir: /local/av/radio/6Music//2020/08/22/1100 closedir: /local/av/radio/6Music//2020/08/22 closedir: /local/av/radio/6Music//2020/08 closedir: /local/av/radio/6Music//2020 closedir: /local/av/radio/6Music/ --==========16324982B0D1D3197C7A==========--