Date: Sun, 7 Aug 2011 10:22:44 +0100 From: "seanrees@gmail.com" <seanrees@gmail.com> To: Gary Palmer <gpalmer@freebsd.org> Cc: Doug Barton <dougb@freebsd.org>, freebsd-stable@freebsd.org Subject: Re: ZFS directory with a large number of files Message-ID: <CAJGy1F1oDFntqFxGyTLHZ0nRu3BoLc1GRwv05XqDn9EO3=fVvA@mail.gmail.com> In-Reply-To: <010350C0-B3B3-44FC-8D94-A111C579860C@gmail.com> References: <CAJGy1F0d7jeyaFuNdXe%2BucTL2r7R4suCyu8xG7WRHenMFZH-6g@mail.gmail.com> <20110802094226.GA93114@icarus.home.lan> <42039B84-D6CE-4780-AA70-8500B1B32036@gsoft.com.au> <4E37CD13.1070402@digsys.bg> <j1h18t$jh1$1@lorvorc.mips.inka.de> <BA47D829-B2E3-419B-AC50-FD3F6FCC54EF@punkt.de> <6E45CE57-491E-4077-B14C-751C73647EFC@gsoft.com.au> <4E3CBB74.9020208@FreeBSD.org> <20110806062415.GB88904@in-addr.com> <010350C0-B3B3-44FC-8D94-A111C579860C@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, Aug 7, 2011 at 10:20 AM, Sean Rees <seanrees@gmail.com> wrote:
>
> On Aug 6, 2011, at 07:24, Gary Palmer wrote:
>
>> On Fri, Aug 05, 2011 at 08:56:36PM -0700, Doug Barton wrote:
>>> On 08/05/2011 20:38, Daniel O'Connor wrote:
>>>
>>>> Ahh, but OP had moved these files away and performance was still poor.=
. _that_ is the bug.
>>>
>>> I'm no file system expert, but it seems to me the key questions are; ho=
w
>>> long does it take the system to recover from this condition, and if it'=
s
>>> more than N $periods is that a problem? We can't stop users from doing
>>> wacky stuff, but the system should be robust in the face of this.
>>
>> Its been quite a while since I worked on the filesystem stuff in any
>> detail but I believe, at least for UFS, it doesn't GC the directory,
>> just truncate it if enough of the entries at the end are deleted
>> to free up at least one fragment or block. =A0If you create N files and
>> then a directory and move the N files into the directory, the directory
>> entry will still be N+1 records into the directory and the only way to
>> "recover" is to recreate the directory that formerly contained the N
>> files. =A0It is theoretically possible to compat the directory but since
>> the code to do that wasn't written when I last worked with UFS I suspect
>> its non trivial.
>>
>> I don't know what ZFS does in this situation
>
> It sounds like it does something similar.
>
> I re-ran the experiment to see if I could narrow down the problem.
>
> % mkdir foo
> % cd foo && for i in {1..1000}; do touch $i; done
Self-pedant mode enabled:
for i in {1..1000000} :) I truncated the zeros in correcting the
copy/paste from my shell :)
Sean
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJGy1F1oDFntqFxGyTLHZ0nRu3BoLc1GRwv05XqDn9EO3=fVvA>