Date: Tue, 22 Jan 2019 16:34:18 +0100 From: Borja Marcos <borjam@sarenet.es> To: Steven Hartland <killing@multiplay.co.uk> Cc: jdelisle <jdelisle@gmail.com>, freebsd-fs <freebsd-fs@freebsd.org> Subject: Re: ZFS on Hardware RAID Message-ID: <F2C97F27-CC2F-4D12-B910-F1CDA9858D0F@sarenet.es> In-Reply-To: <bc417815-9ba3-ef06-38fa-2435532b9969@multiplay.co.uk> References: <1180280695.63420.1547910313494.JavaMail.zimbra@gausus.net> <92646202.63422.1547910433715.JavaMail.zimbra@gausus.net> <CAOeNLurgn-ep1e=Lq9kgxXK%2By5xqq4ULnudKZAbye59Ys7q96Q@mail.gmail.com> <alpine.BSF.2.21.1901200834470.12592@mail0.time-domain.co.uk> <1691666278.63816.1547976245836.JavaMail.zimbra@gausus.net> <alpine.BSF.2.21.1901200930550.12592@mail0.time-domain.co.uk> <CAMdBLfQvF7mXriwAvu-h5BaXBNhb_8VPRRfDmeM0u1NGoyWY_g@mail.gmail.com> <92746659-4B3F-415C-BB6A-6C99837AFAF2@sarenet.es> <335e44ec-7c76-8dbd-f587-46e6a9266efc@multiplay.co.uk> <A593D329-3526-4B5A-BCEF-7C17FA5FFA49@sarenet.es> <bc417815-9ba3-ef06-38fa-2435532b9969@multiplay.co.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
> On 22 Jan 2019, at 16:21, Steven Hartland <killing@multiplay.co.uk> = wrote: >=20 >=20 >> Sorry to insist, but maybe we are talking of different stuff? My = apologies in advance if I am confused and/or my >> information is outdated. >>=20 >> JBOD mode: a mode offered by some LSI Logic IR controllers to create = virtual devices mapped so that each one is effectively. >> For example, with the mfi driver the =E2=80=9Cdisks=E2=80=9D appear = as =E2=80=9Cmfisyspd=E2=80=9D devices while the =E2=80=9Cconventional=E2=80= =9D logical volumes appear as =E2=80=9Cmfid=E2=80=9D. My information = could be outdated though? > Cards that use mfi are quite a different beasts, they are more a pure = RAID controller, than cards which have both IT & IR modes, where the = RAID capabilities are more limited. Yes, that=E2=80=99s right.=20 Going to actual examples, which I hope will be useful: - mps cards (LSI2008) can run either IT or IR firmware. Both present = disks as pure targets.=20 - MFI and others, however, which are pure RAID cards, are the = pathological case I mentioned (the so called JBOD mode is deceiving).=20 For those you can force the driver to really present da (CAM) targets = bypassing the RAID logic. I have done it several times and one of my servers has been doing this since 2007. - To make it even more confusing, you can choose the driver for LSI SAS3 = cards. You can use MFI or the new MRSAS driver. The latter should offer the physical disks as regular CAM targets. > The later present as daX devices where the former presents as mfid = unless you force otherwise using mfi/mfip as you detail below. Yes. Speaking of IT and IR, Stephen Mcconnell from LSI/Avago said that the = performance difference should be really negligible. Of course it=E2=80=99s= safer to run IT firmware avoiding the possibility of making a mistake with the = configuration.=20 >> HBA mode: when the card is in IT mode *or* it does expose the actual = targets to the CAM layer. In the past I did it >> by manually patching the drivers and I=E2=80=99ve kept systems = running smoothly for many years despite using IR cards. >> Currently you can use hw.mfi.allow_cam_disk_passthrough tunable to = achieve the same effect without ugly tinkering. >>=20 >> Unless I am terribly wrong, when I tried the JBOD mode as defined = above (with LSI SAS3 cards and IR firmware) the mfisyspd >> devices were not actual CAM devices. >>=20 >> Actually I find =E2=80=9CJBOD=E2=80=9D an artificial and confusing = term because it can mean so many things. I would have rather preferred >> LSI to avoid it and provide either real, transparent access to the = actual targets insteaf of somewhat =E2=80=9Cvirtualizing=E2=80=9D them. >>=20 >> Again, I may be confused? I haven=E2=80=99t tried to configure a LSI = IR card as =E2=80=9CJBOD=E2=80=9D for a long time. Maybe now they do >> expose the targets instead of defining logical volumes? > I can definitely confirm LSI controllers which present individual daX = devices can support TRIM / UMAP based off what the device reports. Yes, mps and mpr. MFI includes a not widely publicized tunable (allow_cam_passthrough) and = for aac (although these are really old) it=E2=80=99s possible to hack = the driver (aac_cam.c) so that disks are reported to the CAM layer. > As you say its quite confusing, so hope that clarifies? I think so, I was trying to avoid confusion for newcomers, not intending = to be fastidious. The JBOD term is ambiguous. This stuff drove me mad = some time ago, not to mention the puzzled faces of the Dell salesmen when I said =E2=80=9CEGG= S, BACON, RAID, RAID, RAID, I DON=E2=80=99T LIKE RAID!=E2=80=9D ;) They thought I was nuts rejecting a =E2=80=9Chardware=E2=80=9D RAID. And = in some cases they almost refuse to sell servers with plain HBAs.=20 Borja.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?F2C97F27-CC2F-4D12-B910-F1CDA9858D0F>