Date: Wed, 29 May 2013 20:29:51 -0400 From: Mike Tancsa <mike@sentex.net> To: Alexander Motin <mav@FreeBSD.org> Cc: "freebsd-hardware@freebsd.org" <freebsd-hardware@FreeBSD.org> Subject: Re: Rocket Raid 622 in AHCI mode Message-ID: <51A69D7F.3030605@sentex.net> In-Reply-To: <51A4DC42.30705@FreeBSD.org> References: <51A4B34F.7000505@sentex.net> <51A4DC42.30705@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 5/28/2013 12:33 PM, Alexander Motin wrote:
> On 28.05.2013 16:38, Mike Tancsa wrote:
>
> SiI3124 still was not beaten.
>
>> It sees the PMP and cage as
>>
>> pmp0 at ahcich1 bus 0 scbus1 target 15 lun 0
>> pmp0: <Port Multiplier 38261095 1706> ATA-0 device
>> pmp0: 300.000MB/s transfers (SATA 2.x, NONE, PIO 8192bytes)
>> pmp0: 5 fan-out ports
>> Any ideas on how to fix this controller to make it work better with ZFS
>> and get better speeds ?
>
> Good question. One of several about these Marvell controllers, caused by
> lack of any public documentation. :(
I tried to do a little more testing to see what might be up either with
the cage, the card or the drives. It seems there is something odd about
the first slot in the cage-- I have tried 2 cages, and they both show
the same behaviour. ie. two cages (same make/model) and two of the
rocket raid cards (same make and model).
If I read/write to just the disk in slot zero, it works as expected.
However, if I create some sort of multi-disk array with the disk in the
first slot, its so slow, its almost broken.
e.g.
0{mdttestbox}# gstripe stop data
0{mdttestbox}# gstripe label -v -s 131072 data ada0 ada1
Metadata value stored on ada0.
Metadata value stored on ada1.
Done.
0{mdttestbox}# newfs -U -O2 /dev/stripe/data > /dev/null
0{mdttestbox}# mount /dev/stripe/data /mnt
0{mdttestbox}# dd if=/dev/zero of=/mnt/test bs=4096k count=100
100+0 records in
100+0 records out
419430400 bytes transferred in 56.875768 secs (7374501 bytes/sec)
0{mdttestbox}# umount /mnt;mount /dev/stripe/data /mnt
0{mdttestbox}# dd if=/mnt/test of=/dev/null bs=4096k
^C17+0 records in
17+0 records out
71303168 bytes transferred in 237.424329 secs (300320 bytes/sec)
1{mdttestbox}# gstripe stop data
gstripe: Cannot destroy device data (error=16).
1{mdttestbox}# umount /mnt ; mount /dev/stripe/data /mnt
0{mdttestbox}# umount /mnt
Yet, if I use the other 4 disks, all works well
0{mdttestbox}# gstripe stop data
0{mdttestbox}# gstripe label -v -s 131072 data ada1 ada2 ada3 ada4
Metadata value stored on ada1.
warning: ada2: only 1000204885504 bytes from 2000398933504 bytes used.
Metadata value stored on ada2.
warning: ada3: only 1000204885504 bytes from 1500301909504 bytes used.
Metadata value stored on ada3.
warning: ada4: only 1000204885504 bytes from 2000398933504 bytes used.
Metadata value stored on ada4.
Done.
0{mdttestbox}# newfs -U -O2 /dev/stripe/data > /dev/null
0{mdttestbox}# mount /dev/stripe/data /mnt
0{mdttestbox}# dd if=/dev/zero of=/mnt/test bs=4096k count=100
100+0 records in
100+0 records out
419430400 bytes transferred in 2.131533 secs (196774067 bytes/sec)
0{mdttestbox}# umount /mnt ; mount /dev/stripe/data /mnt
0{mdttestbox}# dd if=/mnt/test of=/dev/null bs=4096k
100+0 records in
100+0 records out
419430400 bytes transferred in 2.574856 secs (162894699 bytes/sec)
0{mdttestbox}#
Although its a bit odd writes are faster than reads ?
ZFS works as expected as well if I exclude the first slot.
1{mdttestbox}# zpool create stripe ada1 ada2 ada3 ada4
0{mdttestbox}# dd if=/dev/zero of=/stripe/test bs=4096k count=100
100+0 records in
100+0 records out
419430400 bytes transferred in 1.283134 secs (326879599 bytes/sec)
0{mdttestbox}# dd if=/dev/zero of=/stripe/test bs=4096k count=1000
1000+0 records in
1000+0 records out
4194304000 bytes transferred in 22.798638 secs (183971691 bytes/sec)
0{mdttestbox}# dd if=/stripe/test of=/dev/null bs=4096k
1000+0 records in
1000+0 records out
4194304000 bytes transferred in 0.472554 secs (8875820076 bytes/sec)
0{mdttestbox}# zpool export stripe
0{mdttestbox}# zpool import stripe
0{mdttestbox}# dd if=/stripe/test of=/dev/null bs=4096k
1000+0 records in
1000+0 records out
4194304000 bytes transferred in 23.068320 secs (181820956 bytes/sec)
0{mdttestbox}#
Whats odd is that the SII card does not have this problem with 5 disks
in the drive cage. Only the RR card does.
0{mdttestbox}# camcontrol devlist
<WDC WD1002FAEX-00Z3A0 05.01D05> at scbus0 target 0 lun 0 (ada0,pass1)
<WDC WD1002FAEX-00Z3A0 05.01D05> at scbus0 target 1 lun 0 (ada1,pass2)
<WDC WD1502FAEX-007BA0 05.01D05> at scbus0 target 2 lun 0 (ada3,pass4)
<WDC WD2002FAEX-007BA0 05.01D05> at scbus0 target 3 lun 0 (ada4,pass5)
<WDC WD2002FAEX-007BA0 05.01D05> at scbus0 target 4 lun 0 (ada2,pass3)
<Port Multiplier 37261095 1706> at scbus0 target 15 lun 0 (pmp0,pass0)
<WDC WD5002AALX-00J37A0 15.01H15> at scbus2 target 0 lun 0 (pass6,ada5)
0{mdttestbox}#
---Mike
--
-------------------
Mike Tancsa, tel +1 519 651 3400
Sentex Communications, mike@sentex.net
Providing Internet services since 1994 www.sentex.net
Cambridge, Ontario Canada http://www.tancsa.com/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?51A69D7F.3030605>