Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 20 Jan 2010 01:16:41 +0100
From:      "O. Hartmann" <ohartman@mail.zedat.fu-berlin.de>
To:        krad <kraduk@googlemail.com>
Cc:        =?UTF-8?B?TW9yZ2FuIFdlc3N0csO2bQ==?= <freebsd-questions@pp.dyndns.biz>, FreeBSD Stable <freebsd-stable@freebsd.org>, freebsd-questions@freebsd.org
Subject:   Re: immense delayed write to file system (ZFS and UFS2), performance issues
Message-ID:  <4B564B69.6080102@mail.zedat.fu-berlin.de>
In-Reply-To: <d36406631001190109x154ff1c1lb2354d5a07212455@mail.gmail.com>
References:  <4B54C100.9080906@mail.zedat.fu-berlin.de>	<4B54C5EE.5070305@pp.dyndns.biz> <d36406631001190109x154ff1c1lb2354d5a07212455@mail.gmail.com>

Next in thread | Previous in thread | Raw E-Mail | Index | Archive | Help
On 01/19/10 10:09, krad wrote:
> 2010/1/18 Morgan Wesstr=EF=BF=BDm<freebsd-questions@pp.dyndns.biz>
>
>> O. Hartmann wrote:
>>> I realise a strange behaviour of several FreeBSD 8.0-STABLE/amd64 box=
es.
>>> All boxes have the most recent STABLE. One box is a UP system, two
>>> others SMP boxes, one with a Q6600 4-core, another XEON with 2x 4-cor=
es
>>> (Dell Poweredge III).
>>>
>>> Symptome: All boxes have ZFS and UFS2 filesystems. Since two weeks or=

>>> so, sometimes the I/O performance drops massively when doing 'svn
>>> update', 'make world' or even 'make kernel'. It doesn't matter what
>>> memory and how many cpu the box has, it get stuck for several seconds=

>>> and freezing. On the UP box, this is sometimes for 10 - 20 seconds.
>>> A very interesting phenomenon is the massively delayed file writing o=
n
>>> ZFS filesystems I realise. Editing a file in 'vi' running on one XTer=
m
>>> and having in another Xterminal my shell for compiling this file, it
>>> takes sometimes up to 20 seconds to get the file updated after it has=

>>> been written. It's like having an old, slow NFS connection with long
>>> cache delays.
>>> These massively delayed file transactions are not necessarely under
>>> heavy load, sometimes they occur in a relaxed situation. They seem to=

>>> occur much more often on the UP box than on the SMP boxes, but this
>>> strange phenomenon also occur on the Dell Poweredge II, which has 16G=
B
>>> RAM and summa summarum 16 cores. This phenomenon does occur on ZFS- a=
nd
>>> UFS2 filesystems as well. It is hardly reproducable.
>>>
>>> Is there any known issue?
>>>
>>> Ragrds,
>>> Oliver
>>
>>
>> The disks involved don't happen to be Western Digital Green Power disk=
s,
>> do they? The Intelli-Park function in these disks are wrecking havoc
>> with I/O in Linux-land at least, causing massive stalls and iowait
>> through the roof during the 25-30 seconds it takes for the heads to
>> unload after parking. I have two of these disks sitting on my desk now=

>> collecting dust...
>> /Morgan
>> _______________________________________________
>> freebsd-questions@freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
>> To unsubscribe, send any mail to "
>> freebsd-questions-unsubscribe@freebsd.org"
>>
>
>
> ZFS is copy on write, therefore to optimize the write performance it de=
lays
> writes for a long as possible, upto a set maximum time. It will then fl=
ush
> to the disks. How long this time is depends on how much free ram you ha=
ve
> available. Assuming processes are eating up all your ram I would imagin=
e you
> are hitting the max limit. I'm not sure exactly what its set to on bsd =
but I
> know the default on opensolaris is 30s. I think this explains your dela=
yed
> writes.
>
> Not sure what will cause the lock ups though.
> _______________________________________________
> freebsd-stable@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.or=
g"

This could end in a bad situation, where one process writes a files, say =

with some arbitrary stuff and another successing process is intended to=20
read this file. even if the processes are run serial, those 'delays'=20
could break the chain! The delay situation in a development environment=20
is harsh, but in other circumstances it could develop very bad.

I see this strange behaviour now for several weeks, something essential=20
has changed in the code, I guess.
On UP boxes the situation is worse sometimes, on SMp boxes with lots of=20
RAM ( 8 and 16 GB and 4 or 8 CPU cores) it is still bad. I have a server =

that acts as a 'rsync' backup system gathering data from satellite=20
servers from time to time. Since this problem of slowness occured, this=20
4-core 8 gig RAM box crawls for minutes. Even when X11 is disabled=20
working on console is 'bumpy': terminal out slows down, mouse pointer=20
jumps etc.As I wrote, the same on a 8 core/16 gig box, but not that harsh=
=2E




Want to link to this message? Use this URL: <http://docs.FreeBSD.org/cgi/mid.cgi?4B564B69.6080102>