Date: Fri, 11 Mar 2011 17:36:49 -0600 From: Dustin Wenz <dustinwenz@ebureau.com> To: freebsd-stable@freebsd.org Cc: Rumen Telbizov <telbizov@gmail.com> Subject: Re: LSI SAS2008 performance with mps(4) driver Message-ID: <7E4869D6-FE68-4746-BA79-2F709EDE9F67@ebureau.com> In-Reply-To: <AANLkTinuA2XUAd=kjQ26b_3LPGQY%2B3aNCe2mM4h2jDJC@mail.gmail.com> References: <2D5317E3-D96F-4123-88D5-5AF1BEAD5B9F@xtechllc.com> <AANLkTinuA2XUAd=kjQ26b_3LPGQY%2B3aNCe2mM4h2jDJC@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Thanks; It's good to know that it's at least possible to make this work = in some instances. Unfortunately, our SAS2008 controller is integrated with the logic board = (a SuperMicro X8DT6) connected to a SAS-113TQ backplane. It's not so = much of an expander; there are two breakout cables that go from the = SFF8087 connectors to individual SATA connectors for each drive on the = backplane. I've spoken with LSI, and (unsurprisingly) they are unable to = provide firmware for chips which are integrated in 3rd party devices. = SuperMicro also doesn't have any particularly recent updates - the = newest I could find was from last August. If this is indeed an issue with the onboard SAS2008 hardware/firmware, = I'm probably going to have to spring for a separate controller card in = the short term. :/ - .Dustin On Mar 10, 2011, at 10:09 PM, Rumen Telbizov wrote: > Hello Dustin, >=20 > I've been testing this SAS2008 LSI chip (on a LSI 9211-8i) for the = last month or so and I can say > that it makes a pretty good HBA but there are indeed a few caveats you = might need to be aware of. > In support of that - tonight I finished a FreeBSD 8.2-STABLE machine = with 2 x 24 disk chassis (each with > a 3Gbit expander) =3D 48 x 2TB SATA RE4-GP disks in 6 x 8disk raidz2 = and I am able to squeeze out 900MB/s > write and 1200 MB/s read in a sequential (single dd) manner. The limit = here is the backplane speed. >=20 > So back to your problem: >=20 > 1) What kind of backplane are you using: please specify the exact = model. Is it a SAS expander or direct attached? > 3Gbit/s or 6Gbit/s? > 2) What kind of disk controller exactly are you using? More = importantly what kind of firmware does it have? >=20 > Those two are very important. In my case it turned out that if I was = connecting SAS2008 chips > to pretty much every kind of SuperMicro SAS expander backplanes (tried = against 826EL26, 836E1, 846E1) I was > getting around 200-300MB/s read/write speeds (FreeBSD and Linux). = Direct attached backplanes (826A) worked fine. > At the end it turned out that it was some sort of a problem with the = LSI firmware (version 8.00 in my case) and I was given > to try version 9.00 (soon to be released) which completely solved the = problem. Contact LSI support (very high quality) if you want to try this = firmware. >=20 > I can't seem to get any better performance than about 250MB/s writes = through the controller. I'm testing with a zpool of six 250MB magnetic = SATA disks, doing a couple of concurrent sequential writes with dd: >=20 > dd bs=3D128k if=3D/dev/zero of=3D/datadisk/zero1 & > dd bs=3D128k if=3D/dev/zero of=3D/datadisk/zero2 & >=20 >=20 > 3) What kind of zpool raid level do you have those disks organized in? > 4) Running two parallel dd's on the same pool will actually turn the = game into no-so-sequential type and more of a random access. > Please try the following and paste results here: > 4.1) dd if=3D/dev/zero of=3D/datadisk/zero1 bs=3D1M count=3D50000 = (only one dd and use a file size larger than your memory) > 4.2) Destroy the zpool (if you have no useful data on it of course) = and try dd against each and every disk individually. > So something like: > dd if=3D/dev/zero of=3D/dev/da0 bs=3D1M count=3D50000 > dd if=3D/dev/da0 of=3D/dev/null bs=3D1M count=3D50000 > monitor throughput with gstat -f da0 or I can send you a simple C = program that I wrote which resembles dd but=20 > prints stats every second. > =20 > On a related note I also experienced very slow read speeds (200MB/s) = with the above mentioned configuration and after enabling > prefetch (I used to set it to disabled as per Jeremy Chadwick's = advise) everything went back to normal - so keep it in mind.=20 >=20 > Cheers, > Rumen Telbizov >=20 > --=20 > Rumen Telbizov > http://telbizov.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7E4869D6-FE68-4746-BA79-2F709EDE9F67>