Date: Tue, 10 Dec 2013 11:41:16 +0100 From: Benjamin Lutz <benjamin.lutz@biolab.ch> To: freebsd-stable@freebsd.org Subject: Swap on ZFS - bad idea? (swap_pager: indefinite wait buffer) Message-ID: <OFB3F5370B.79D00F5F-ONC1257C3D.0038AB15-C1257C3D.003AB574@biotronik.com>
next in thread | raw e-mail | index | archive | help
We have storage server that I've found unresponsive this morning. The console was filled with these messages: swap_pager: indefinite wait buffer: bufobj: 0, blkno: 17344, size: 61440 The lines were all exactly the same, no variation in the numbers. Ctrl-Alt-Delete had no effect. Now, the only thing related to swap that's special about this machine, is that we use swap on ZFS, because we figured, silent disk corruption is just as bad when it happens to swapped memory as it is when it happens to stored data. However, might this be the problem? One thought I was having was: ZFS hits its memory bounds, a process makes the kernel want to allocate a swap buffer which causes more ZFS operations which never complete because ZFS would need to do some housework to free memory, but the kernel is waiting for swap to become available, some sort of deadlock? I'm just speculating here, not knowing the internals of this subsystem. The machine is a quad core system with 12GiB RAM and a RAIDZ3 consisting of 12x2TB disks. It's used as a storage system, with accesses mainly through netatalk (an AFP server, what OS X uses) and a bit of iSCSI. I've set the following sysctls: # http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide#Device_I.2FO_Queue_Size_.28I.2FO_Concurrency.29 vfs.zfs.vdev.max_pending=20 # limit kernel size to 10GB to leave some RAM for processes vm.kmem_size_max="10737418240" # increase number of mbuf clusters, see igb(4). The default is 25600 kern.ipc.nmbclusters="102400" This sort of hang seems to happen every couple of months. I'd really like some pointers on how to increase the stability of this system. Maybe there are some more tunables I need to adjust? Or is swap on ZFS maybe just not a great idea? Cheers, Benjamin -- Benjamin Lutz | Software Engineer | BIOLAB Technology AG Dufourstr. 80 | CH-8008 Zurich | www.biolab.ch | benjamin.lutz@biolab.ch PHONE +41 44 295 97 13 | MOBILE +41 79 558 57 13 | FAX +41 44 295 97 19 This e-mail and the information it contains including attachments are confidential and meant only for use by the intended recipient(s); disclosure or copying is strictly prohibited. If you are not addressed, but in the possession of this e-mail, please notify the sender immediately.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?OFB3F5370B.79D00F5F-ONC1257C3D.0038AB15-C1257C3D.003AB574>