Date: Tue, 29 Mar 2016 18:49:37 -0400 (EDT) From: Rick Macklem <rmacklem@uoguelph.ca> To: Hiroshi Nishida <nishida@asusa.net> Cc: freebsd-fs@freebsd.org Subject: Re: Problem with FUSE + fts Message-ID: <765991039.37160180.1459291777879.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <56FAD050.2080707@asusa.net> References: <56F42EF4.5000505@asusa.net> <1294209833.31699182.1458950014610.JavaMail.zimbra@uoguelph.ca> <56F6148D.2030706@asusa.net> <56FAD050.2080707@asusa.net>
next in thread | previous in thread | raw e-mail | index | archive | help
Hiroshi Nishida wrote: > Now I figured out what causes this error. > > Originally, fts outputs an ENOENT error when: > > FTSENT *p; > struct stat sb; > > stat(p->fts_accpath, &sb); > p->fts_ino != sb.st_ino > > i.e., the inode of p is different from sb.st_ino. > They are usually same but sometimes a new inode is allocated to p while > scanning the dir in the following way: > > 1. FUSE lowelevel's forget() is called for some reason and removes all > entries from the entry tables, as well as clears all inodes, while scanning > the dir. > 2. Since p is already removed from FUSE's entry table, FUSE adds it again and > allocates a new inode. > > I don't know why forget() is called for the directory which is still open and > clears all inodes, but according to fuse_lowlevel.h > You've never mentioned what version of FreeBSD you are using? FreeBSD10 and later have a fuse client in sys/fs/fuse and I don't recall seeing this code in it. (I will grep for it, although I know there isn't a fuse_lowlevel.h.) If you are using FreeBSD9 or earlier with the fuse client in ports/sysutils, I'd suggest you try upgrading to FreeBSD10 and see if the problem exists there. rick > /** > * Forget about an inode > * > * This function is called when the kernel removes an inode > * from its internal caches. > * > * The inode's lookup count increases by one for every call to > * fuse_reply_entry and fuse_reply_create. The nlookup parameter > * indicates by how much the lookup count should be decreased. > * > * Inodes with a non-zero lookup count may receive request from > * the kernel even after calls to unlink, rmdir or (when > * overwriting an existing file) rename. Filesystems must handle > * such requests properly and it is recommended to defer removal > * of the inode until the lookup count reaches zero. Calls to > * unlink, remdir or rename will be followed closely by forget > * unless the file or directory is open, in which case the > * kernel issues forget only after the release or releasedir > * calls. > * > > removing the inode should be deferred until the dir is closed. > > I haven't checked the ref count of each node yet but there seems to be a bug > in the above process. > > Also, there is a suggestion for the hash table but I will post later. > > Any feedback is appreciated on it. > > Thank you. > > > On 2016/03/25 21:48, Hiroshi Nishida wrote: > > Thank you for your response. > > > > On 3/25/16 4:53 PM, Rick Macklem wrote: > >> I think I see the same thing when doing an "rm -r" on a fuse/GlusterFS > >> volume. > > > > Unfortunately, it happens also with "find XXX -print", though I have > > experienced a similar "rm -r" + "XXX: No such file or directory" problem > > with UFS + SUJ. > > And I also verified with truss that in > > > > _fstat(fd, &sb); > > p->fts_ino != sb.st_ino > > > > stat() system call is called with the same path as p's. > > > > Anyway, the following patch for lib/libc/gen/fts.c prevents the error but > > is far from a good solution. > > https://github.com/scopedog/FUSE-Test/blob/master/fts.c.patch > > It assumes that the filesystem id (f_type in struct statfs) of FUSE is 0xed > > but I am not sure if it's applicable to all FUSE filesystems. > > > > I'll look into FUSE source code next week. > >> To be honest, I just add a "-f" to the command to shut it up and then it > >> deleted > >> the tree. > >> > >> I think, in general, what readdir() returns after an entry is unlink'd is > >> undefined > >> behaviour. As such, the safe way to delete all of a directory is something > >> like: > >> - in a loop until readdir() returns EOF > >> - opendir() > >> - readdir() the first entry > >> - unlink() that entry > >> - closedir() > >> --> So that all you ever do is readdir() the first entry after an > >> opendir(). > > > > By the way, could you delete all the files with "-f"? > > I am testing with a pretty big directory containing 81,000 files/dirs and > > have never used "-f", but have to "rm -r" again for undeleted entries. > > However, the offset problem is very interesting as it seems to be > > applicable to all filesystems. > > > > Thank you. > > > > -- > Hiroshi Nishida > nishida@asusa.net > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?765991039.37160180.1459291777879.JavaMail.zimbra>