Date: Wed, 27 Jun 2012 09:14:52 GMT From: Levent Serinol <levent.serinol@mynet.com> To: freebsd-gnats-submit@FreeBSD.org Subject: kern/169480: ZFS stalls on heavy I/O Message-ID: <201206270914.q5R9EqsC010501@red.freebsd.org> Resent-Message-ID: <201206270920.q5R9K7DN088705@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 169480 >Category: kern >Synopsis: ZFS stalls on heavy I/O >Confidential: no >Severity: serious >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Wed Jun 27 09:20:07 UTC 2012 >Closed-Date: >Last-Modified: >Originator: Levent Serinol >Release: 9.0 and 10.0 >Organization: >Environment: FreeBSD localhost 9.0-RELEASE FreeBSD 9.0-RELEASE #0: Wed Mar 21 11:15:41 EET 2012 root@webmail122.mynet.com:/usr/obj/usr/src/sys/GENERIC amd64 >Description: Under heavy I/O load we face freeze problems on ZFS volumes on both Freebsd 9 Release and 10 Current versions. Machines are HP servers (64bit) with HP Smart array 6400 raid controllers (with battery units). Every da device is a hardware raid5 where each one includes 9x300GB 10K SCSI hard drivers. Main of I/O pattern happens on local system except some small NFS I/O from some other servers (NFS lookup/getattr/ etc.). These servers are mail servers (qmail) with small I/O patterns (64K Read/Write). Below you can find procstat output on freeze time. write_limit is set to 200MB because of the huge amount of txg_wait_opens observed before. Every process stops on D state I think due to txg queue and other 2 queues are full. Is there any suggestion to fix the problem ? btw inject_compress is the main process injecting emails to user inboxes (databases). Also, those machines were running without problems on Linux/XFS filesystem. For a while ago, we started migration from Linux to Freebsd http://pastebin.com/raw.php?i=ic3YepWQ >How-To-Repeat: >Fix: >Release-Note: >Audit-Trail: >Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201206270914.q5R9EqsC010501>