Date: Thu, 21 Nov 2013 22:16:35 GMT From: Petr Cibulka <info@ITSMexpert.EU> To: freebsd-gnats-submit@FreeBSD.org Subject: kern/184154: [cam] QUIRK: SYNC_CACHE not supported on IBM ServeRAID 8k Message-ID: <201311212216.rALMGZOi071391@oldred.freebsd.org> Resent-Message-ID: <201311212220.rALMK0Nu087687@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 184154
>Category: kern
>Synopsis: [cam] QUIRK: SYNC_CACHE not supported on IBM ServeRAID 8k
>Confidential: no
>Severity: non-critical
>Priority: low
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Thu Nov 21 22:20:00 UTC 2013
>Closed-Date:
>Last-Modified:
>Originator: Petr Cibulka
>Release: FreeBSD 9.1 amd64 / FreeNAS 9.1.1
>Organization:
>Environment:
IBM System x3650 with ServeRAID 8k, firmware 5.2-0 (Build 17005)
VMware ESXi 4.1 U3 with IBM Customization, updated driver: aacraid_400.4.1.7.28700
Virtual Machine Version 7, two SCSI Controllers LSI Logic Parallel (one for Virtual Disk and one for RDM (Raw Disk Mapping) disks, RDM disks in Physical Compatibility Mode
>Description:
FreeNAS 9.1.1 based on FreeBSD 9.1 shows in dmesg:
da1 at mpt1 bus 0 scbus2 target 0 lun 0
da1: <ServeRA ZFSdisk0 V1.0> Fixed Direct Access SCSI-2 device
da1: 300.000MB/s transfers
da1: Command Queueing enabled
da1: 204800MB (419430400 512 byte sectors: 255H 63S/T 26108C)
As soon as ZFS filesystem on RDM disks is attempted to mount a flood of error messages for each RDM disk is written to dmesg:
(da1:mpt1:0:0:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00
(da1:mpt1:0:0:0): CAM status: SCSI Status Error
(da1:mpt1:0:0:0): SCSI status: Check Condition
(da1:mpt1:0:0:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command operation code)
(da1:mpt1:0:0:0): Error 22, Unretryable error
Moreover following line is displayed for each RDM disk on console during shutdown:
(da1:mpt1:0:0:0): Synchronize cache failed
The incompatibility effectively prevents using ZFS filesystem on RDM disks.
>How-To-Repeat:
It is probably a bug of ServeRAID firmware, which does not implement mandatory SCSI command SYNCHRONIZE CACHE(10).
The firmware version 5.2-0 (Build 17005) is latest, no corrected version has been released.
The virtualization layer should not affect it, because for RDM disks in Physical Compatibility Mode, SCSI commands are simply tunelled from VM to RAID controller.
May be the same error can be observed on FreeBSD installed directly on hardware, but such configuration has not been tested.
>Fix:
The DA_Q_NO_SYNC_CACHE quirk fixes the problem.
The RDM disks on ServeRAID adapter can be identified with these strings:
"ServeRA", "*", "*"
The second string reports the name of the array / logical disk. It is a user selectable information entered during ServeRAID setup, so that it can not be used for identification.
The fix has been sucessfully tested.
Now this identification is shown in dmesg:
da1 at mpt1 bus 0 scbus2 target 0 lun 0
da1: <ServeRA ZFSdisk0 V1.0> Fixed Direct Access SCSI-2 device
da1: 320.000MB/s transfers (160.000MHz, offset 127, 16bit)
da1: Command Queueing enabled
da1: 204800MB (419430400 512 byte sectors: 255H 63S/T 26108C)
da1: quirks=0x1<NO_SYNC_CACHE>
No SCSI error messages are written in dmesg, ZFS is sucessfully mounted and used, the shutdown is errorfree as well.
It is not clean solution (it only suppress cache flushing), but it is the best effort. I believe the caches are flushed during VMware shutdown.
Patch attached with submission follows:
diff --git sys/cam/scsi/scsi_da.c sys/cam/scsi/scsi_da.c
index d376006..7447c63 100644
--- sys/cam/scsi/scsi_da.c
+++ sys/cam/scsi/scsi_da.c
@@ -230,6 +230,13 @@ static const char microp[] = "MICROP";
static struct da_quirk_entry da_quirk_table[] =
{
/* SPI, FC devices */
+ {
+ /*
+ * The IBM ServeRAID controllers do not support SYNC_CACHE
+ */
+ {T_DIRECT, SIP_MEDIA_FIXED, "ServeRA", "*", "*"},
+ /*quirks*/ DA_Q_NO_SYNC_CACHE
+ },
{
/*
* Fujitsu M2513A MO drives.
>Release-Note:
>Audit-Trail:
>Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201311212216.rALMGZOi071391>