Date: Thu, 29 Apr 2021 09:24:19 +0100 From: "N.J. Mann" <njm@njm.me.uk> To: freebsd-fs@freebsd.org Subject: readdir() -> d_type always zero on NFS v3 mounted filesystems Message-ID: <2E84A420CCC10A73504624DE@triton.njm.me.uk>
next in thread | raw e-mail | index | archive | help
--==========16324982B0D1D3197C7A========== Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline Hi, I recently changed over from using svn to gitup to update /usr/ports on my local system and have been experiencing problems since. At first I thought it was an issue with gitup itself, but now I believe it is either a kernel issue or a configuration issue. I originally posted about the problem to the freebsd-ports mailing list: https://lists.freebsd.org/pipermail/freebsd-ports/2021-April/120929.html Since then I have dug deeper and come to the conclusion that it is not a problem with gitup. The issue I am seeing is that gitup is unable to delete files and directories, even complete ports, which have been removed from the repository. gitup basically does the following: prune_tree(base_path) { if ((directory = opendir(base_path)) != NULL) { while ((entry = readdir(directory)) != NULL) { snprintf(full_path, sizeof(full_path), "%s/%s", base_path, entry->d_name); if (entry->d_type == DT_DIR) { prune_tree(full_path); } else { if ((remove(full_path) != 0) && (errno != ENOENT)) err(EXIT_FAILURE, "prune_tree: cannot remove %s", full_path); } } closedir(directory); if (rmdir(base_path) != 0) err(EXIT_FAILURE, "prune_tree: cannot remove %s", base_path); } } When gitup is run on either a UFS or ZFS file system this works. However, when I run it on a NFS v3 mounted filesystem it fails. Liberal addition of printf's shows that for non-NFS mounted filesystems d_type contains the correct value for each file/directory, but for NFS mounted file systems it is zero - other fields such as d_name and d_namlen are correct. While debugging this I added a call to stat() and that always returns the correct values. At this point I started digging in libc and quickly found that readdir() is basically a wrapper around a system call. Digging in the kernel I quickly got out of my depth and hence my posting here. I then wrote a simple test programme which also shows the issue. I have attached the source for the test programme and the output from two runs, the first on an NFS mounted file system and the second on a local UFS file system. Before gitup started failing I had not seen any failures like this and that was why I initially assumed this must be a gitup issue. I now believe this is either a kernel issue or a confuration issue. My configuration is as follows: file server: /exports/ports - ZFS file system for FreeBSD ports repo exported via NFS /remote/ports - FreeBSD ports repo NFS (v3 rw,tcp) mounted from /export/ports on file server /usr/ports - symbolic link to /remote/ports client machines: /remote/ports - FreeBSD ports repo NFS (v3 ro,tcp) mounted from /export/ports on file server /usr/ports - symbolic link to /remote/ports I run gitup in /remote/ports on the file server and then pkg, portmaster, &.c, in /usr/ports on the file server and then the client machines. All systems are running 11-STABLE from about 12 days ago - I usually update every couple of weeks. Any assistance, suggestions, patches, clue bats, will be gratefully accepted. Regards, Nick. -- --==========16324982B0D1D3197C7A========== Content-Type: text/plain; charset=us-ascii; name="my_search.c.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="my_search.c.txt"; size=2044 /* * vim:sw=4 sts=4 expandtab * */ #include <dirent.h> #include <err.h> #include <errno.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #include <unistd.h> #include <sys/stat.h> static void my_recurse(char *path) { DIR *directory = NULL; struct dirent *entry = NULL; struct stat sb; char full_path[strlen(path) + 1 + MAXNAMLEN + 1]; if ((directory = opendir(path)) != NULL) { printf("opendir: %s\n", path); while ((entry = readdir(directory)) != NULL) { printf("readdir: %c %u %s\n", entry->d_type == DT_DIR ? 'D' : '_', entry->d_namlen, entry->d_name); snprintf(full_path, sizeof(full_path), "%s/%s", path, entry->d_name); stat(full_path, &sb); printf("stat: %c\n", S_ISDIR(sb.st_mode) ? 'D' : '_'); if (S_ISDIR(sb.st_mode) != 0) { if ((entry->d_namlen == 1) && (strcmp(entry->d_name, "." ) == 0)) continue; if ((entry->d_namlen == 2) && (strcmp(entry->d_name, "..") == 0)) continue; printf("recurse: %s\n", full_path); my_recurse(full_path); } else printf("file: %s\n", full_path); } printf("closedir: %s\n", path); closedir(directory); } } int main(int argc, char *argv[]) { int ch, i; char *path; while ((ch = getopt(argc, argv, "v")) != -1) switch (ch) { case 'v': /* Do nothing, for now */ break; default: fprintf(stderr, "Illegal option -- %c\n", ch); exit (-1); } argc -= optind; argv += optind; if (argc > 1) { fprintf(stderr, "Too many parameters\n"); exit(-1); } else if (argc < 1) { fprintf(stderr, "Not enough parameters\n"); exit(-1); } path = *argv; my_recurse(path); return 0; } --==========16324982B0D1D3197C7A========== Content-Type: text/plain; charset=us-ascii; name="test1.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="test1.txt"; size=458 opendir: /remote/ports/net/gitup readdir: _ 1 . stat: D readdir: _ 2 .. stat: D readdir: _ 9 pkg-descr stat: _ file: /remote/ports/net/gitup/pkg-descr readdir: _ 8 distinfo stat: _ file: /remote/ports/net/gitup/distinfo readdir: _ 8 Makefile stat: _ file: /remote/ports/net/gitup/Makefile readdir: _ 9 pkg-plist stat: _ file: /remote/ports/net/gitup/pkg-plist closedir: /remote/ports/net/gitup --==========16324982B0D1D3197C7A========== Content-Type: text/plain; charset=us-ascii; name="test2.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="test2.txt"; size=8879 opendir: /local/av/radio/6Music/ readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 13 DJ_Shadow.m4a stat: _ file: /local/av/radio/6Music//DJ_Shadow.m4a readdir: D 4 2020 stat: D recurse: /local/av/radio/6Music//2020 opendir: /local/av/radio/6Music//2020 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 2 05 stat: D recurse: /local/av/radio/6Music//2020/05 opendir: /local/av/radio/6Music//2020/05 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 2 23 stat: D recurse: /local/av/radio/6Music//2020/05/23 opendir: /local/av/radio/6Music//2020/05/23 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/05/23/1100 opendir: /local/av/radio/6Music//2020/05/23/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 44 The_Huey_Show:_Funky_Space_Reincarnation.m4a stat: _ file: /local/av/radio/6Music//2020/05/23/1100/The_Huey_Show:_Funky_Space_Reincarnation.m4a closedir: /local/av/radio/6Music//2020/05/23/1100 closedir: /local/av/radio/6Music//2020/05/23 readdir: D 2 30 stat: D recurse: /local/av/radio/6Music//2020/05/30 opendir: /local/av/radio/6Music//2020/05/30 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/05/30/1100 opendir: /local/av/radio/6Music//2020/05/30/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 29 The_Huey_Show:_Buggin_Out.m4a stat: _ file: /local/av/radio/6Music//2020/05/30/1100/The_Huey_Show:_Buggin_Out.m4a closedir: /local/av/radio/6Music//2020/05/30/1100 closedir: /local/av/radio/6Music//2020/05/30 closedir: /local/av/radio/6Music//2020/05 readdir: D 2 06 stat: D recurse: /local/av/radio/6Music//2020/06 opendir: /local/av/radio/6Music//2020/06 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 2 06 stat: D recurse: /local/av/radio/6Music//2020/06/06 opendir: /local/av/radio/6Music//2020/06/06 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/06/06/1100 opendir: /local/av/radio/6Music//2020/06/06/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 40 The_Huey_Show:_Uptown_meets_Downtown.m4a stat: _ file: /local/av/radio/6Music//2020/06/06/1100/The_Huey_Show:_Uptown_meets_Downtown.m4a closedir: /local/av/radio/6Music//2020/06/06/1100 closedir: /local/av/radio/6Music//2020/06/06 readdir: D 2 13 stat: D recurse: /local/av/radio/6Music//2020/06/13 opendir: /local/av/radio/6Music//2020/06/13 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/06/13/1100 opendir: /local/av/radio/6Music//2020/06/13/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 29 The_Huey_Show:_Heads_Know.m4a stat: _ file: /local/av/radio/6Music//2020/06/13/1100/The_Huey_Show:_Heads_Know.m4a closedir: /local/av/radio/6Music//2020/06/13/1100 closedir: /local/av/radio/6Music//2020/06/13 readdir: D 2 27 stat: D recurse: /local/av/radio/6Music//2020/06/27 opendir: /local/av/radio/6Music//2020/06/27 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/06/27/1100 opendir: /local/av/radio/6Music//2020/06/27/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 41 The_Huey_Show:_Louie_Vega_Block_Party.m4a stat: _ file: /local/av/radio/6Music//2020/06/27/1100/The_Huey_Show:_Louie_Vega_Block_Party.m4a closedir: /local/av/radio/6Music//2020/06/27/1100 closedir: /local/av/radio/6Music//2020/06/27 closedir: /local/av/radio/6Music//2020/06 readdir: D 2 07 stat: D recurse: /local/av/radio/6Music//2020/07 opendir: /local/av/radio/6Music//2020/07 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 2 04 stat: D recurse: /local/av/radio/6Music//2020/07/04 opendir: /local/av/radio/6Music//2020/07/04 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/07/04/1100 opendir: /local/av/radio/6Music//2020/07/04/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 52 The_Huey_Show:_Brazil_Block_Party_and_Louie_Vega.m4a stat: _ file: /local/av/radio/6Music//2020/07/04/1100/The_Huey_Show:_Brazil_Block_Party_and_Louie_Vega.m4a closedir: /local/av/radio/6Music//2020/07/04/1100 closedir: /local/av/radio/6Music//2020/07/04 readdir: D 2 11 stat: D recurse: /local/av/radio/6Music//2020/07/11 opendir: /local/av/radio/6Music//2020/07/11 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/07/11/1100 opendir: /local/av/radio/6Music//2020/07/11/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 35 The_Huey_Show:_Cuba_Block_Party.m4a stat: _ file: /local/av/radio/6Music//2020/07/11/1100/The_Huey_Show:_Cuba_Block_Party.m4a closedir: /local/av/radio/6Music//2020/07/11/1100 closedir: /local/av/radio/6Music//2020/07/11 readdir: D 2 18 stat: D recurse: /local/av/radio/6Music//2020/07/18 opendir: /local/av/radio/6Music//2020/07/18 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/07/18/1100 opendir: /local/av/radio/6Music//2020/07/18/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 42 The_Huey_Show:_Puerto_Rico_Block_Party.m4a stat: _ file: /local/av/radio/6Music//2020/07/18/1100/The_Huey_Show:_Puerto_Rico_Block_Party.m4a closedir: /local/av/radio/6Music//2020/07/18/1100 closedir: /local/av/radio/6Music//2020/07/18 closedir: /local/av/radio/6Music//2020/07 readdir: D 2 08 stat: D recurse: /local/av/radio/6Music//2020/08 opendir: /local/av/radio/6Music//2020/08 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 2 01 stat: D recurse: /local/av/radio/6Music//2020/08/01 opendir: /local/av/radio/6Music//2020/08/01 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/08/01/1100 opendir: /local/av/radio/6Music//2020/08/01/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 35 The_Huey_Show:_Chuck_D_turns_60.m4a stat: _ file: /local/av/radio/6Music//2020/08/01/1100/The_Huey_Show:_Chuck_D_turns_60.m4a closedir: /local/av/radio/6Music//2020/08/01/1100 closedir: /local/av/radio/6Music//2020/08/01 readdir: D 2 08 stat: D recurse: /local/av/radio/6Music//2020/08/08 opendir: /local/av/radio/6Music//2020/08/08 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/08/08/1100 opendir: /local/av/radio/6Music//2020/08/08/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 59 The_Huey_Show:_New_music_from_Felt_Skyzoo_and_Sly5thAve.m4a stat: _ file: /local/av/radio/6Music//2020/08/08/1100/The_Huey_Show:_New_music_from_Felt_Skyzoo_and_Sly5thAve.m4a closedir: /local/av/radio/6Music//2020/08/08/1100 closedir: /local/av/radio/6Music//2020/08/08 readdir: D 2 15 stat: D recurse: /local/av/radio/6Music//2020/08/15 opendir: /local/av/radio/6Music//2020/08/15 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/08/15/1100 opendir: /local/av/radio/6Music//2020/08/15/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 17 The_Huey_Show.m4a stat: _ file: /local/av/radio/6Music//2020/08/15/1100/The_Huey_Show.m4a closedir: /local/av/radio/6Music//2020/08/15/1100 closedir: /local/av/radio/6Music//2020/08/15 readdir: D 2 22 stat: D recurse: /local/av/radio/6Music//2020/08/22 opendir: /local/av/radio/6Music//2020/08/22 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: D 4 1100 stat: D recurse: /local/av/radio/6Music//2020/08/22/1100 opendir: /local/av/radio/6Music//2020/08/22/1100 readdir: D 1 . stat: D readdir: D 2 .. stat: D readdir: _ 17 The_Huey_Show.m4a stat: _ file: /local/av/radio/6Music//2020/08/22/1100/The_Huey_Show.m4a closedir: /local/av/radio/6Music//2020/08/22/1100 closedir: /local/av/radio/6Music//2020/08/22 closedir: /local/av/radio/6Music//2020/08 closedir: /local/av/radio/6Music//2020 closedir: /local/av/radio/6Music/ --==========16324982B0D1D3197C7A==========--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2E84A420CCC10A73504624DE>