Date: Tue, 12 Aug 2008 02:43:43 +0200 From: GW <gw.freebsd@tnode.com> To: freebsd-fs@freebsd.org Cc: Mitar <mitar@tnode.com> Subject: Unionfs move directory problems Message-ID: <48A0DCBF.6050900@tnode.com>
next in thread | raw e-mail | index | archive | help
Hi.
It seems that there are even more bugs in unionfs at least in 7.0-STABLE
as the "unionfs breakout to upper layer" problem submitted by Nejc...
Moving directories around in unionfs mounts isn't handled correctly as
can be seen from the long example below. It behaves the same if there is
no "-o below" parameter.
Original hierarchy:
~# mkdir rw ro ro/orig
~# touch ro/orig/file ro/orig/file2
~# mount -t unionfs -o below ro rw
~/rw# ls -aFloWi
total 6
49874 drwx------ 3 root wheel - 512 Aug 12 01:32 ./
49872 drwx------ 4 root wheel - 512 Aug 12 01:20 ../
49878 drwx------ 2 root wheel - 512 Aug 12 01:20 orig/
~/rw# ls -aFloWi orig/
total 4
49878 drwx------ 2 root wheel - 512 Aug 12 01:20 ./
49874 drwx------ 3 root wheel - 512 Aug 12 01:32 ../
49876 -rw------- 1 root wheel - 0 Aug 12 01:20 file
49877 -rw------- 1 root wheel - 0 Aug 12 01:20 file2
Problem appears (lost directory contents):
~/rw# mv orig moved
~/rw# ls -aFloWi
total 6
49874 drwx------ 3 root wheel - 512 Aug 12 01:33 ./
49872 drwx------ 4 root wheel - 512 Aug 12 01:20 ../
49878 drwx------ 2 root wheel - 512 Aug 12 01:20 moved/
0 w--------- 0 root wheel - 0 Jan 1 1970 orig%
~/rw# ls -aFloWi moved
total 4
49878 drwx------ 2 root wheel - 512 Aug 12 01:20 ./
49874 drwx------ 3 root wheel - 512 Aug 12 01:33 ../
<< WTF!??? PROBLEM 1
And now some magic (content reappears):
~/rw# mv moved orig
~/rw# ls -aFloWi
total 6
49874 drwx------ 3 root wheel - 512 Aug 12 01:33 ./
49872 drwx------ 4 root wheel - 512 Aug 12 01:20 ../
49878 drwx------ 2 root wheel - 512 Aug 12 01:20 orig/
~/rw# ls -aFloWi orig
total 4
49878 drwx------ 2 root wheel - 512 Aug 12 01:20 ./
49874 drwx------ 3 root wheel - 512 Aug 12 01:33 ../
49876 -rw------- 1 root wheel - 0 Aug 12 01:20 file
49877 -rw------- 1 root wheel - 0 Aug 12 01:20 file2
Hm, lets test something else (it creates in upper layer):
~/rw# mkdir new
~/rw# touch new/newfile
~/rw# ls -aFloWi
total 8
49874 drwx------ 4 root wheel - 512 Aug 12 02:04 ./
49872 drwx------ 4 root wheel - 512 Aug 12 02:03 ../
49879 drwx------ 2 root wheel - 512 Aug 12 02:04 new/
49878 drwx------ 2 root wheel - 512 Aug 12 01:20 orig/
~/rw# ls -aFloWi new
total 4
49879 drwx------ 2 root wheel - 512 Aug 12 02:04 ./
49874 drwx------ 4 root wheel - 512 Aug 12 02:04 ../
49880 -rw------- 1 root wheel - 0 Aug 12 02:04 newfile
~/rw# ls -aFloWi orig
total 4
49878 drwx------ 2 root wheel - 512 Aug 12 01:20 ./
49874 drwx------ 4 root wheel - 512 Aug 12 02:04 ../
49876 -rw------- 1 root wheel - 0 Aug 12 01:20 file
49877 -rw------- 1 root wheel - 0 Aug 12 01:20 file2
And lets see if there is another problem:
~/rw# rm -rf orig
~/rw# ls -aFloWi
total 6
49874 drwx------ 3 root wheel - 512 Aug 12 02:06 ./
49872 drwx------ 4 root wheel - 512 Aug 12 02:03 ../
49879 drwx------ 2 root wheel - 512 Aug 12 02:06 new/
0 w--------- 0 root wheel - 0 Jan 1 1970 orig%
~/rw# mv new orig
~/rw# ls -aFloWi
total 6
49874 drwx------ 3 root wheel - 512 Aug 12 02:07 ./
49872 drwx------ 4 root wheel - 512 Aug 12 02:03 ../
49879 drwx------ 2 root wheel - 512 Aug 12 02:06 orig/
Looks fine till now, but (files appear from nowhere):
~/rw# ls -aFloWi orig
total 4
49879 drwx------ 2 root wheel - 512 Aug 12 02:06 ./
49874 drwx------ 3 root wheel - 512 Aug 12 02:07 ../
49876 -rw------- 1 root wheel - 0 Aug 12 01:20 file
49877 -rw------- 1 root wheel - 0 Aug 12 01:20 file2
49880 -rw------- 1 root wheel - 0 Aug 12 02:05 newfile
<< WTF!??? PROBLEM 2
I see two possible solutions to the first problem:
- on directory rename/move the whole file hierarchy should be exactly
duplicated under a new directory tree
- if both layers exist on the same partition, inodes of files can be
reused as it is done with hard links, but the directory structure needs
to be duplicated so that it has correct parent inodes
For the second problem:
- in all cases (option whiteout=always or whenneeded) when new
directories get created unionfs should check whether such a directory
exists in the lower layer and automaticly whiteout all of its entries
Does anyone have any better ideas?
I am using:
~# uname -a
FreeBSD server.domain.com 7.0-STABLE FreeBSD 7.0-STABLE #5: Sun Aug 10
09:54:42 CEST 2008
root@server.domain.com:/usr/src/sys/amd64/compile/SERVER amd64
Is it already fixed in HEAD?
Unrelated:
Will the newest patches for unionfs from HEAD ever appear in 7-STABLE or
do I need to backport them?
Is there or will there be a way to separately access any layer in a
unionfs mount (without using nullfs)? I made a script to detect
duplicates, but in current design (it can be done differently) you need
to unmount the layers before you can use it.
Are whiteouts and the interface to manage them (FTS or sth) specific to
FreeBSD or does it look like other OS (that have something like unionfs,
eg Linux) will adapt this sooner or later?
Cheers,
gw
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?48A0DCBF.6050900>