From owner-freebsd-stable@freebsd.org Fri May 3 10:25:24 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 05317158B78C for ; Fri, 3 May 2019 10:25:24 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id 2601C6ADE3 for ; Fri, 3 May 2019 10:25:18 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Received: from [10.10.0.230] (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQX00GIFC96YO10@hades.sorbs.net> for freebsd-stable@freebsd.org; Fri, 03 May 2019 03:39:09 -0700 (PDT) Subject: Re: ZFS... From: Michelle Sullivan X-Mailer: iPad Mail (16A404) In-reply-to: Date: Fri, 03 May 2019 20:25:08 +1000 Cc: Pete French , freebsd-stable@freebsd.org Content-transfer-encoding: quoted-printable Message-id: <62803130-9C40-4A98-B5A4-A2DFAC0FAD65@sorbs.net> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <58DA896C-5312-47BC-8887-7680941A9AF2@sarenet.es> To: Borja Marcos X-Rspamd-Queue-Id: 2601C6ADE3 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-3.02 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.979,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sorbs.net]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[battlestar.sorbs.net,anaconda.sorbs.net,ninja.sorbs.net,catapilla.sorbs.net,scorpion.sorbs.net,desperado.sorbs.net]; NEURAL_HAM_SHORT(-0.84)[-0.844,0]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.43)[ip: (-1.10), ipnet: 72.12.192.0/19(-0.56), asn: 11114(-0.44), country: US(-0.06)]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2019 10:25:24 -0000 Michelle Sullivan http://www.mhix.org/ Sent from my iPad > On 03 May 2019, at 20:09, Borja Marcos via freebsd-stable wrote: >=20 >=20 >=20 >> On 3 May 2019, at 11:55, Pete French wrote: >>=20 >>=20 >>=20 >>> On 03/05/2019 08:09, Borja Marcos via freebsd-stable wrote: >>>=20 >>> The right way to use disks is to give ZFS access to the plain CAM device= s, not thorugh some so-called JBOD on a RAID >>> controller which, at least for a long time, has been a *logical* =E2=80=9C= RAID0=E2=80=9D volume on a single disk. That additional layer can >>> completely break the semantics of transaction writes and cache flushes. >>> With some older cards it can be tricky to achieve, from patching source d= rivers to enabling a sysctl tunable or even >>> flashing the card to turn it into a plain HBA with no RAID features (or m= inimal ones). >>=20 >> Oddly enough I got bitten by something like this yesteray. I have a machi= ne containing an HP P400 RAID controller, which is nice enough, but I run ZFS= so I have made the drives all into RAID-0 as being as close as I can get to= accessing the raw SAS drives. I got bitten by that on this hardware originally... switching to raid-0 and= separate drives then switching to write-through (not write back and definit= ely not write back with bad bbu) seemed to solve it. Michelle >>=20 >> BSD seems them as da0, da1, da2, da3 - but the RAID controller oly presen= ts one of them to the BIOS, so my booting has to be all from that drive. Thi= s has been da0 for as long as I can remember, but yesteday it decided to sta= rt using what BSD sees as da1. Of course this is very hard to recognise as d= a0 and da1 are pretty much mirrors of each other. Spent a long time trying t= o work out why the fixes I was applying to da0 were not being used at boot t= ime. >=20 > Hmm What happens when you do a =E2=80=9Ccamcontrol devlist=E2=80=9D? >=20 > Camcontrol tags da0 -v? >=20 > How is the controller recognized by FreeBSD? For some of them it=E2=80=99s= possible to instruct the controller to present the physical devices to CAM.= Of course > you need to be careful to avoid any logical volume configuration in that c= ase.=20 >=20 > But I would only tinker with this at system installation time, making such= a change on a running system with valid data can be disastrous. >=20 > For mfi recognized cards there is a tunable: hw.mfi.allow_cam_disk_passthr= ough >=20 > For aac cards it was a matter of commenting a couple of source code lines i= n the driver (at your own risk of course). I=E2=80=99ve been running a=20 > server for years doing that. >=20 >=20 >=20 >=20 >=20 >=20 > Borja. >=20 >=20 > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"