Date: Wed, 30 May 2012 10:58:46 +0300 From: Juris Krumins <resdev80@gmail.com> To: Mikolaj Golub <trociny@freebsd.org> Cc: freebsd-fs@freebsd.org Subject: Re: Hastd activemap cache flush and "request failed: FLUSH.: Operation not supported by device" messages. Message-ID: <1338364726.2417.10.camel@brainbug-nb.office.ctco.lv> In-Reply-To: <86ehq2nn35.fsf@in138.ua3> References: <1338295953.17714.12.camel@brainbug-nb.office.ctco.lv> <86ehq2nn35.fsf@in138.ua3>
next in thread | previous in thread | raw e-mail | index | archive | help
Thanks for your answer Mikolaj. I'v found later last night that BIO_FLUSH request comes from ZFS and You prove it (thank You). More than that, I've also found that it's possible to disable BIO_FLUSH requests as ZFS level by setting following sysctl setting: vfs.zfs.vdev.bio_flush_disable: 1 The only concert I have right now is how this setting can potentially effect ZFS stability and ability to recover in case of power failures and/or other issues ? P.S. Underlying hardware is IBM EXS3000 disk bay with 12 disks combined into 3 RAID5 volumes (4 disks in each) with writeback, disk cache enabled and BBU. On Wed, 2012-05-30 at 10:29 +0300, Mikolaj Golub wrote: > On Tue, 29 May 2012 15:52:33 +0300 Juris Krumins wrote: > > JK> Hi everybody. > JK> I've recently setup two FreeBSD nodes to use them as a storage. > JK> So I'm using hast over "MegaRAID SAS 1078" and ZFS filesystem created on > JK> HAST resource devices from /dev/hast/*. > JK> Everything looks good, but when I execute iozone tests, I'm constantly > JK> getting following messages in /var/log/message: > > JK> primary node: > JK> May 29 15:44:15 rook hastd[2313]: [mfid2] (primary) Local request failed > JK> (Operation not supported by device): FLUSH. > JK> May 29 15:44:15 rook hastd[2311]: [mfid0] (primary) Local request failed > JK> (Operation not supported by device): FLUSH. > JK> May 29 15:44:15 rook hastd[2312]: [mfid1] (primary) Local request failed > JK> (Operation not supported by device): FLUSH. > JK> May 29 15:44:15 rook hastd[2312]: [mfid1] (primary) Remote request > JK> failed (Operation not supported by device): FLUSH. > JK> May 29 15:44:15 rook hastd[2313]: [mfid2] (primary) Remote request > JK> failed (Operation not supported by device): FLUSH. > JK> May 29 15:44:15 rook hastd[2311]: [mfid0] (primary) Remote request > JK> failed (Operation not supported by device): FLUSH. > > JK> secondary node: > JK> May 29 15:43:41 rook2 hastd[2458]: [mfid1] (secondary) Request failed: > JK> FLUSH.: Operation not supported by device. > JK> May 29 15:43:41 rook2 hastd[2457]: [mfid0] (secondary) Request failed: > JK> FLUSH.: Operation not supported by device. > JK> May 29 15:43:41 rook2 hastd[2462]: [mfid2] (secondary) Request failed: > JK> FLUSH.: Operation not supported by device. > > JK> I'v also found following topic > JK> http://lists.freebsd.org/pipermail/freebsd-fs/2012-January/013385.html. > JK> I've updated src, rebuild world and kernel and rebooted. I've also added > JK> metaflush "off" options to /etc/hast.conf configuration file. But still > JK> getting previously mentioned messages. > > Note, there are two sources of BIO_FLUSH operations in HAST: > > 1) BIO_FLUSH operations generated by HAST on metadata (activemap) update, if > enabled (not disabled) in config. > > 2) BIO_FLUSH operations sent by a HAST user (filesystem). > > If the underlying provider doesn't support BIO_FLUSH, in the first case a warning > "The provider doesn't support flushing write cache. Disabling it." will be > generated and there will be no more attempt to flush on activemap update. > > In the second case the error 'Request failed: FLUSH.: Operation not supported > by device' will be observed (as in your case). Before r225832 it logged on > every BIO_FLUSH coming. Since r225832 HAST will try only once and only one > error in logs should be observed after hastd start. > > JK> Can somebody point me to the right direction how to configure hastd to > JK> throw this warning only once. > > JK> I'm using: > JK> FreeBSD primary. 9.0-STABLE FreeBSD 9.0-STABLE #0: Tue May 29 12:53:44 > JK> EEST 2012 root@primary.:/usr/obj/usr/src/sys/GENERIC amd64 > > JK> FreeBSD secondary. 9.0-STABLE FreeBSD 9.0-STABLE #0: Tue May 29 12:53:44 > JK> EEST 2012 root@secondary.:/usr/obj/usr/src/sys/GENERIC amd64 > > r225832 has been merged to 9-STABLE on 4 Jan 2012 as r229509 and you shouldn't > observe 'FLUSH: Operation not supported by device' constantly repeating. If > you do this is very strange and you should recheck your build. Please provide > the output of > > ident /sbin/hastd > > command. > > JK> and hast.conf configuration from both machines: > JK> I've decided to put metaflush option on every single level just to be > JK> sure. No effect. > > No need in disabling metaflush in your case. It will disable automatically > after the first attempt failed. >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1338364726.2417.10.camel>