Date: Wed, 11 Oct 2017 11:25:10 +0100 From: Edward Napierala <trasz@freebsd.org> To: "Eugene M. Zheganin" <emz@norma.perm.ru> Cc: FreeBSD Stable <freebsd-stable@freebsd.org> Subject: Re: zfs, iSCSI and volmode=dev Message-ID: <CAFLM3-rfFJvG%2BnmoSe%2Ba%2BLWzgQ-0Yfhf_bRNhZwxiLhmgpquEQ@mail.gmail.com> In-Reply-To: <d7733530-ce2a-c66e-c854-7f7c26b5e14e@norma.perm.ru> References: <ece540ba-8039-424f-8bad-c4266d70d84b@norma.perm.ru> <CAFLM3-pyh4fbGKbM5Y=26RUFc9ozsxobuwm9ayPkk3hwp0uWig@mail.gmail.com> <d7733530-ce2a-c66e-c854-7f7c26b5e14e@norma.perm.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
2017-10-09 17:38 GMT+01:00 Eugene M. Zheganin <emz@norma.perm.ru>: > Hi, > > On 27.09.2017 16:07, Edward Napierala wrote: > >> 2017-08-30 11:45 GMT+02:00 Eugene M. Zheganin <emz@norma.perm.ru <mailto: >> emz@norma.perm.ru>>: >> >> Hi, >> >> >> I have an iSCSI production system that exports a large number of >> zvols as the iSCSI targets. System is running FreeBSD >> 11.0-RELEASE-p7 and initially all of the zvols were confugured >> with default volmode. I've read that it's recommended to use them >> in dev mode, so the system isn't bothered with all of these geom >> structures, so I've switched all of the zvols to dev mode, then I >> exported/imported the pools back. Surprisingly, the performance >> has fallen down like 10 times (200-300 Mbits/sec against 3-4 >> Gbits/sec previously). After observing for 5 minutes the ESXes >> trying to boot up, and doing this extremely slowly, I switched the >> volmode back to default, then again exported/imported the pools. >> The performance went back to normal. >> >> >> So... why did this happen ? The result seems to be >> counter-intuitive. At least not obvious to me. >> >> >> I don't really have an answer - mav@ would be the best person to ask. >> Based >> on his description, "ZVOLs in GEOM mode don't support DPO/FUA cache >> control >> bits, had to chunk large I/Os into MAXPHYS-sized pieces and go through >> GEOM." >> There also used to be so that TRIM was only supported in the "dev" mode, >> but >> that changed a while ago. >> >> Yeah, but you mean dev is faster by design. So was my first thought too, > but it seems like the opposite. Default volmode is geom, and it's much > faster than dev. I'd expect it to be faster, but it might be it interferes with something. For example, if Windows forces direct media access (bypassing the disk cache), and going through GEOM made the target ignore this bit, that could hurt performance. (Note that it's just an idea; I have no idea what's actually happening there.) My first suspect would be TRIM, but then it should work in both modes. Might be worth checking to be sure, by disabling the "unmap" option in ctl.conf (option "unmap" "off") to see if it makes a difference.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFLM3-rfFJvG%2BnmoSe%2Ba%2BLWzgQ-0Yfhf_bRNhZwxiLhmgpquEQ>