From owner-freebsd-scsi@FreeBSD.ORG Sun Nov 2 21:34:31 2014 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AEAF525E for ; Sun, 2 Nov 2014 21:34:31 +0000 (UTC) Received: from mail-vc0-f170.google.com (mail-vc0-f170.google.com [209.85.220.170]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 533C68E7 for ; Sun, 2 Nov 2014 21:34:30 +0000 (UTC) Received: by mail-vc0-f170.google.com with SMTP id la4so4351923vcb.29 for ; Sun, 02 Nov 2014 13:34:24 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=/Yai3H1eilsigmbZPel37YDYTSMgKK7QCALgDTL/M/U=; b=ZfV/ZmKKnq6CShOXsYHZlbZHBiANtqYJYe5lVX1wZCjgZnRDrLAoVAC9FPg8pGt6jC quDBVaHMMckzLvs7T0g8ruAN3tt2ENvzjuIEQHa9VLI98O/ywxBt1O0D83gP0zFpwPAC bQmmTyzyKA6P6D/SKfjoFIgT4e175ol1qhyQswxXLbiCi5j3PLqAYnuTLfp6PiOYtc/M 2I+GzCkV1Qm5iiSJ56MFesbOVyNhVNTEqPQw91S0cZIAxzO5fHt36DaRBGsFAi5GaljZ FvIQcyev8Ey+9saL+mCz1ynZFw6LJ7mrdeCkKRUecV7wlvkmBzGjqHLG9Sp/0bGoSyhX WEVg== X-Gm-Message-State: ALoCoQmo0zdJamcxHVxN9HYmFRo/x5x6yDSOcc5Ck5bZ7pF8YxwliuUus1YicCcH5giDdcB9mbJ5 X-Received: by 10.220.188.131 with SMTP id da3mr29743116vcb.10.1414964064319; Sun, 02 Nov 2014 13:34:24 -0800 (PST) MIME-Version: 1.0 Received: by 10.31.172.86 with HTTP; Sun, 2 Nov 2014 13:34:04 -0800 (PST) X-Originating-IP: [174.50.25.169] In-Reply-To: References: From: Wes Morgan Date: Sun, 2 Nov 2014 15:34:04 -0600 Message-ID: Subject: Re: MPS driver and latest firmware To: Borja Marcos Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-scsi@freebsd.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Nov 2014 21:34:31 -0000 I was updating the firmware hoping that the targets went back to being assigned in "next available" order rather than sticky. I like each bay to have a specific device name, which is impossible due to the "remembered" target IDs. On Mon, Oct 27, 2014 at 11:20 AM, Borja Marcos wrote: > > On Oct 26, 2014, at 11:00 PM, Wes Morgan wrote: > > > Hi all, > > > > I have an LSI 9201-16I controller that I updated to the latest firmware > > (20.00.00.00). Immediately I began seeing CAM errors like this: > > > > Oct 22 19:43:56 volatile kernel: (da13:mps0:0:13:0): CAM status: SCSI > > Status Error > > Oct 22 19:43:56 volatile kernel: (da13:mps0:0:13:0): SCSI status: Check > > Condition > > Oct 22 19:43:56 volatile kernel: (da13:mps0:0:13:0): SCSI sense: ABORTED > > COMMAND asc:47,3 (Information unit iuCRC error detected) > > Oct 22 19:43:56 volatile kernel: (da13:mps0:0:13:0): Retrying command > (per > > sense data) > > > > It did not seem to cause any data corruption, but needless to say that > kind > > of error is disconcerting. I was able to force a downgrade to the > > 19.00.00.00 firmware in FreeDOS, after which everything returned to > normal. > > I tried in a full functional system and I also suffered errors. > > I had to downgrade as well. In case anyone runs into the same problem, > the card required a flash erase (sas2flsh -o -e 6) before being able to > upload > the P19 firmware, as the NVRAM data was migrated to a new format. > > I wonder if the NVRAM changes include that "sticky" target ID behavior... > > And no, this is not a production system, just testing ;) > > > Cheers, > > > > > > > > Borja. > > > ----------- > > > > This is some log information from my change. > > I upgraded the firmware to P20 from sas2flash running on FreeBSD. Before > doing it I exported the ZFS pool > in order to avoid aborted commands leading to corruption. > > This is the syslog showing the controller having problems after the > update to P20: > > Oct 27 16:26:07 elibm kernel: mps0: Reinitializing controller, > Oct 27 16:26:15 elibm kernel: mps0: Warning: io_cmds_active is out of sync > - resynching to 0 > Oct 27 16:26:16 elibm kernel: mps0: Firmware: 20.00.00.00, Driver: > 19.00.00.00-fbsd > Oct 27 16:26:16 elibm kernel: mps0: IOCCapabilities: > 1285c > Oct 27 16:26:16 elibm kernel: mps0: mps_reinit finished sc > 0xfffffe0000d84000 post 4 free 3 > Oct 27 16:28:30 elibm kernel: (da2:mps0:0:11:0): READ(10). CDB: 28 00 00 > e3 d3 38 00 00 e0 00 length 114688 SMID 706 command timeout cm > 0xfffffe04976a1ea0 ccb 0xfffff80a8d573800 > Oct 27 16:28:30 elibm kernel: (noperiph:mps0:0:4294967295:0): SMID 1 > Aborting command 0xfffffe04976a1ea0 > Oct 27 16:28:30 elibm kernel: (da2:mps0:0:11:0): READ(10). CDB: 28 00 00 > e3 d2 58 00 00 e0 00 length 114688 SMID 738 command timeout cm > 0xfffffe04976a48a0 ccb 0xfffff80a8d5bc800 > Oct 27 16:28:30 elibm kernel: (da2:mps0:0:11:0): READ(10). CDB: 28 00 00 > e3 d2 58 00 00 e0 00 length 114688 SMID 738 terminated ioc 804b scsi 0 > state c xfer 112648 > Oct 27 16:28:30 elibm kernel: (da2:mps0:0:11:0): READ(10). CDB: 28 00 00 > e3 d3 38 00 00 e0 00 > Oct 27 16:28:30 elibm kernel: (da2:mps0:0:11:0): CAM status: Command > timeout > Oct 27 16:28:30 elibm kernel: (da2:mps0:0:11:0): Retrying command > Oct 27 16:28:31 elibm kernel: (da2:mps0:0:11:0): READ(10). CDB: 28 00 00 > e3 d3 38 00 00 e0 00 > Oct 27 16:28:31 elibm kernel: (da2:mps0:0:11:0): CAM status: SCSI Status > Error > Oct 27 16:28:31 elibm kernel: (da2:mps0:0:11:0): SCSI status: Check > Condition > Oct 27 16:28:31 elibm kernel: (da2:mps0:0:11:0): SCSI sense: UNIT > ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) > Oct 27 16:28:31 elibm kernel: (da2:mps0:0:11:0): Retrying command (per > sense data) > Oct 27 16:29:31 elibm kernel: (da0:mps0:0:9:0): READ(10). CDB: 28 00 00 e5 > 25 48 00 00 e0 00 length 114688 SMID 995 command timeout cm > 0xfffffe04976b99f0 ccb 0xfffff80a3b6c5000 > Oct 27 16:29:31 elibm kernel: (noperiph:mps0:0:4294967295:0): SMID 2 > Aborting command 0xfffffe04976b99f0 > Oct 27 16:29:32 elibm kernel: (da0:mps0:0:9:0): READ(10). CDB: 28 00 00 e5 > 25 48 00 00 e0 00 > Oct 27 16:29:32 elibm kernel: (da0:mps0:0:9:0): CAM status: Command timeout > Oct 27 16:29:32 elibm kernel: (da0:mps0:0:9:0): Retrying command > Oct 27 16:29:32 elibm kernel: (da0:mps0:0:9:0): READ(10). CDB: 28 00 00 e5 > 25 48 00 00 e0 00 > Oct 27 16:29:32 elibm kernel: (da0:mps0:0:9:0): CAM status: SCSI Status > Error > Oct 27 16:29:32 elibm kernel: (da0:mps0:0:9:0): SCSI status: Check > Condition > Oct 27 16:29:32 elibm kernel: (da0:mps0:0:9:0): SCSI sense: UNIT ATTENTION > asc:29,0 (Power on, reset, or bus device reset occurred) > Oct 27 16:29:32 elibm kernel: (da0:mps0:0:9:0): Retrying command (per > sense data) > Oct 27 16:30:33 elibm kernel: (da5:mps0:0:14:0): READ(10). CDB: 28 00 01 > 11 26 50 00 00 e0 00 length 114688 SMID 358 command timeout cm > 0xfffffe04976855e0 ccb 0xfffff8017f856800 > Oct 27 16:30:33 elibm kernel: (noperiph:mps0:0:4294967295:0): SMID 3 > Aborting command 0xfffffe04976855e0 > Oct 27 16:30:33 elibm kernel: (da2:mps0:0:11:0): READ(10). CDB: 28 00 01 > 11 26 30 00 00 e0 00 length 114688 SMID 432 command timeout cm > 0xfffffe049768b700 ccb 0xfffff80a8d5e8800 > Oct 27 16:30:33 elibm kernel: (noperiph:mps0:0:4294967295:0): SMID 4 > Aborting command 0xfffffe049768b700 > Oct 27 16:30:33 elibm kernel: (da2:mps0:0:11:0): READ(10). CDB: 28 00 01 > 11 25 50 00 00 e0 00 length 114688 SMID 356 command timeout cm > 0xfffffe0497685340 ccb 0xfffff80a3b9c1000 > Oct 27 16:30:33 elibm kernel: (da5:mps0:0:14:0): READ(10). CDB: 28 00 74 > 70 6a 10 00 00 10 00 length 8192 SMID 346 command timeout cm > 0xfffffe0497684620 ccb 0xfffff80a8d5af000 > Oct 27 16:30:33 elibm kernel: (da5:mps0:0:14:0): READ(10). CDB: 28 00 74 > 70 68 10 00 00 10 00 length 8192 SMID 500 command timeout cm > 0xfffffe0497691040 ccb 0xfffff80a8d57c000 > Oct 27 16:30:33 elibm kernel: (da5:mps0:0:14:0): READ(6). CDB: 08 00 02 10 > 10 00 length 8192 SMID 371 command timeout cm 0xfffffe04976866f0 ccb > 0xfffff80a8d56c800 > Oct 27 16:30:33 elibm kernel: (da5:mps0:0:14:0): READ(10). CDB: 28 00 01 > 11 27 30 00 00 e0 00 length 114688 SMID 321 command timeout cm > 0xfffffe0497682550 ccb 0xfffff80a915c5000 > Oct 27 16:30:33 elibm kernel: (da5:mps0:0:14:0): READ(10). CDB: 28 00 01 > 11 26 50 00 00 e0 00 > Oct 27 16:30:33 elibm kernel: (da5:mps0:0:14:0): READ(10). CDB: 28 00 74 > 70 6a 10 00 00 10 00 length 8192 SMID 346 terminated ioc 804b scsi 0 state > c xfer (da5:mps0:0:14:0): CAM status: Command timeout > Oct 27 16:30:33 elibm kernel: (da5: (da5:mps0:0:14:0): READ(10). CDB: > 28 00 74 70 68 10 00 00 10 00 length 8192 SMID 500 terminated ioc 804b scsi > 0 state c xfer mps0:0:0 > Oct 27 16:30:33 elibm kernel: 14: (da5:mps0:0:14:0): READ(6). CDB: > 08 00 02 10 10 00 length 8192 SMID 371 terminated ioc 804b scsi 0 state c > xfer 0 > Oct 27 16:30:33 elibm kernel: 0): (da5:mps0:0:14:0): READ(10). CDB: > 28 00 01 11 27 30 00 00 e0 00 length 114688 SMID 321 terminated ioc 804b > scsi 0 state c xfeRetrying command > Oct 27 16:30:33 elibm kernel: (da2:mps0:0:11:0): READ(10). CDB: 28 00 01 > 11 25 50 00 00 e0 00 length 114688 SMID 356 terminated ioc 804b scsi 0 > state c xfer 112648 > Oct 27 16:30:33 elibm kernel: (da2:mps0:0:11:0): READ(10). CDB: 28 00 01 > 11 26 30 00 00 e0 00 > Oct 27 16:30:33 elibm kernel: (da2:mps0:0:11:0): CAM status: Command > timeout > Oct 27 16:30:33 elibm kernel: (da2:mps0:0:11:0): Retrying command > Oct 27 16:30:34 elibm kernel: (da2:mps0:0:11:0): READ(10). CDB: 28 00 01 > 11 26 30 00 00 e0 00 > Oct 27 16:30:34 elibm kernel: (da5:mps0:0:14:0): READ(10). CDB: 28 00 74 > 70 6a 10 00 00 10 00 > Oct 27 16:30:34 elibm kernel: (da2:mps0:0:11:0): CAM status: SCSI Status > Error > Oct 27 16:30:34 elibm kernel: (da5:mps0:0:14:0): CAM status: SCSI Status > Error > Oct 27 16:30:34 elibm kernel: (da2:mps0:0:11:0): SCSI status: Check > Condition > Oct 27 16:30:34 elibm kernel: (da5:mps0:0:14:0): SCSI status: Check > Condition > Oct 27 16:30:34 elibm kernel: (da2:mps0:0:11:0): SCSI sense: UNIT > ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) > Oct 27 16:30:34 elibm kernel: (da5:mps0:0:14:0): SCSI sense: UNIT > ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) > Oct 27 16:30:34 elibm kernel: (da2:(da5:mps0:0:mps0:0:11:14:0): 0): > Retrying command (per sense data) > Oct 27 16:36:28 elibm kernel: mps0: port 0x3f00-0x3fff mem > 0x90ebc000-0x90ebffff,0x912c0000-0x912fffff irq 32 at device 0.0 on pci17 > Oct 27 16:36:28 elibm kernel: mps0: Firmware: 20.00.00.00, Driver: > 19.00.00.00-fbsd > Oct 27 16:36:28 elibm kernel: mps0: IOCCapabilities: > 1285c > Oct 27 16:36:28 elibm kernel: ses0 at mps0 bus 0 scbus0 target 15 lun 0 > Oct 27 16:36:28 elibm kernel: ses1 at mps0 bus 0 scbus0 target 26 lun 0 > Oct 27 16:36:28 elibm kernel: da0 at mps0 bus 0 scbus0 target 9 lun 0 > Oct 27 16:36:28 elibm kernel: da1 at mps0 bus 0 scbus0 target 10 lun 0 > Oct 27 16:36:28 elibm kernel: da2 at mps0 bus 0 scbus0 target 11 lun 0 > Oct 27 16:36:28 elibm kernel: da3 at mps0 bus 0 scbus0 target 12 lun 0 > Oct 27 16:36:28 elibm kernel: da5 at mps0 bus 0 scbus0 target 14 lun 0 > Oct 27 16:36:28 elibm kernel: da4 at mps0 bus 0 scbus0 target 13 lun 0 > Oct 27 16:36:28 elibm kernel: da6 at mps0 bus 0 scbus0 target 17 lun 0 > Oct 27 16:36:28 elibm kernel: da7 at mps0 bus 0 scbus0 target 18 lun 0 > Oct 27 16:36:28 elibm kernel: da9 at mps0 bus 0 scbus0 target 21 lun 0 > Oct 27 16:36:28 elibm kernel: da8 at mps0 bus 0 scbus0 target 20 lun 0 > Oct 27 16:36:28 elibm kernel: da12 at mps0 bus 0 scbus0 target 27 lun 0 > Oct 27 16:36:28 elibm kernel: da10 at mps0 bus 0 scbus0 target 22 lun 0 > Oct 27 16:36:28 elibm kernel: da11 at mps0 bus 0 scbus0 target 23 lun 0 > Oct 27 16:37:24 elibm kernel: (da0:mps0:0:9:0): READ(10). CDB: 28 00 00 e6 > 5d 08 00 00 28 00 length 20480 SMID 396 command timeout cm > 0xfffffe0000e547c0 ccb 0xfffff800261ca800 > Oct 27 16:37:24 elibm kernel: (noperiph:mps0:0:4294967295:0): SMID 1 > Aborting command 0xfffffe0000e547c0 > Oct 27 16:37:25 elibm kernel: (da0:mps0:0:9:0): READ(10). CDB: 28 00 00 e6 > 5d 08 00 00 28 00 > Oct 27 16:37:25 elibm kernel: (da0:mps0:0:9:0): CAM status: Command timeout > Oct 27 16:37:25 elibm kernel: (da0:mps0:0:9:0): Retrying command > Oct 27 16:37:25 elibm kernel: (da0:mps0:0:9:0): READ(10). CDB: 28 00 00 e6 > 5d 08 00 00 28 00 > Oct 27 16:37:25 elibm kernel: (da0:mps0:0:9:0): CAM status: SCSI Status > Error > Oct 27 16:37:25 elibm kernel: (da0:mps0:0:9:0): SCSI status: Check > Condition > Oct 27 16:37:25 elibm kernel: (da0:mps0:0:9:0): SCSI sense: UNIT ATTENTION > asc:29,0 (Power on, reset, or bus device reset occurred) > Oct 27 16:37:25 elibm kernel: (da0:mps0:0:9:0): Retrying command (per > sense data) > Oct 27 16:37:25 elibm kernel: (da5:mps0:0:14:0): READ(10). CDB: 28 00 01 > 14 fb 28 00 00 70 00 length 57344 SMID 292 command timeout cm > 0xfffffe0000e4bf40 ccb 0xfffff8000fd0a000 > Oct 27 16:37:25 elibm kernel: (noperiph:mps0:0:4294967295:0): SMID 2 > Aborting command 0xfffffe0000e4bf40 > Oct 27 16:37:25 elibm kernel: (da5:mps0:0:14:0): READ(10). CDB: 28 00 01 > 14 fa 48 00 00 e0 00 length 114688 SMID 276 command timeout cm > 0xfffffe0000e4aa40 ccb 0xfffff8007daa0000 > Oct 27 16:37:26 elibm kernel: (da5:mps0:0:14:0): READ(10). CDB: 28 00 01 > 14 fa 48 00 00 e0 00 length 114688 SMID 276 terminated ioc 804b scsi 0 > state c xfer 113668 > Oct 27 16:37:26 elibm kernel: (da5:mps0:0:14:0): READ(10). CDB: 28 00 01 > 14 fb 28 00 00 70 00 > Oct 27 16:37:26 elibm kernel: (da5:mps0:0:14:0): CAM status: Command > timeout > Oct 27 16:37:26 elibm kernel: (da5:mps0:0:14:0): Retrying command > Oct 27 16:37:27 elibm kernel: (da5:mps0:0:14:0): READ(10). CDB: 28 00 01 > 14 fb 28 00 00 70 00 > Oct 27 16:37:27 elibm kernel: (da5:mps0:0:14:0): CAM status: SCSI Status > Error > Oct 27 16:37:27 elibm kernel: (da5:mps0:0:14:0): SCSI status: Check > Condition > Oct 27 16:37:27 elibm kernel: (da5:mps0:0:14:0): SCSI sense: UNIT > ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) > Oct 27 16:37:27 elibm kernel: (da5:mps0:0:14:0): Retrying command (per > sense data) > Oct 27 16:38:27 elibm kernel: (da6:mps0:0:17:0): READ(10). CDB: 28 00 74 > 70 6a 10 00 00 10 00 length 8192 SMID 464 command timeout cm > 0xfffffe0000e5a100 ccb 0xfffff8002620e000 > Oct 27 16:38:27 elibm kernel: (noperiph:mps0:0:4294967295:0): SMID 3 > Aborting command 0xfffffe0000e5a100 > Oct 27 16:38:27 elibm kernel: (da6:mps0:0:17:0): READ(10). CDB: 28 00 74 > 70 68 10 00 00 10 00 length 8192 SMID 459 command timeout cm > 0xfffffe0000e59a70 ccb 0xfffff8000fcfa800 > Oct 27 16:38:27 elibm kernel: (da6:mps0:0:17:0): READ(6). CDB: 08 00 02 10 > 10 00 length 8192 SMID 452 command timeout cm 0xfffffe0000e59140 ccb > 0xfffff8007dab0800 > Oct 27 16:38:27 elibm kernel: (da6:mps0:0:17:0): READ(10). CDB: 28 00 01 > 15 0d 88 00 00 70 00 length 57344 SMID 443 command timeout cm > 0xfffffe0000e58570 ccb 0xfffff8000cf25800 > Oct 27 16:38:27 elibm kernel: (da6:mps0:0:17:0): READ(10). CDB: 28 00 01 > 15 0d 40 00 00 28 00 length 20480 SMID 489 command timeout cm > 0xfffffe0000e5c1d0 ccb 0xfffff8007daa4800 > Oct 27 16:38:27 elibm kernel: (da6:mps0:0:17:0): READ(10). CDB: 28 00 01 > 15 0d 40 00 00 28 00 length 20480 SMID 489 terminated ioc 804b scsi 0 state > c xfer 19460 > Oct 27 16:38:27 elibm kernel: (da6:mps0:0:17:0): READ(10). CDB: 28 00 74 > 70 68 10 00 00 10 00 length 8192 SMID 459 terminated ioc 804b scsi 0 state > c xfer (da6:mps0:0:17:0): READ(10). CDB: 28 00 74 70 6a 10 00 00 10 00 > Oct 27 16:38:27 elibm kernel: (da6:mps0:0:17:0): CAM status: Command > timeout > Oct 27 16:38:27 elibm kernel: (da6:mps0:0:17:0): READ(6). CDB: 08 00 02 10 > 10 00 length 8192 SMID 452 terminated ioc 804b scsi 0 state c xfer 0 > Oct 27 16:38:27 elibm kernel: (da6: (da6:mps0:0:17:0): READ(10). CDB: > 28 00 01 15 0d 88 00 00 70 00 length 57344 SMID 443 terminated ioc 804b > scsi 0 state c xfermps0:0: 0 > Oct 27 16:38:28 elibm kernel: (da6:mps0:0:17:0): READ(10). CDB: 28 00 74 > 70 6a 10 00 00 10 00 > Oct 27 16:38:28 elibm kernel: (da6:mps0:0:17:0): CAM status: SCSI Status > Error > Oct 27 16:38:28 elibm kernel: (da6:mps0:0:17:0): SCSI status: Check > Condition > Oct 27 16:38:28 elibm kernel: (da6:mps0:0:17:0): SCSI sense: UNIT > ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) > Oct 27 16:38:28 elibm kernel: (da6:mps0:0:17:0): Retrying command (per > sense data) > Oct 27 16:45:21 elibm kernel: mps0: Reinitializing controller, > Oct 27 16:45:22 elibm kernel: mps0: Warning: io_cmds_active is out of sync > - resynching to 0 > Oct 27 16:45:22 elibm kernel: mps0: Firmware: 20.00.00.00, Driver: > 19.00.00.00-fbsd > Oct 27 16:45:22 elibm kernel: mps0: IOCCapabilities: > 1285c > Oct 27 16:45:22 elibm kernel: mps0: mps_reinit finished sc > 0xfffffe0000d84000 post 4 free 3 > > ------------ > > > At 17:13, system rebooted with the freshly downgraded controller, this > time to P19. > > > ------------ > > Oct 27 17:13:16 elibm kernel: mps0: port 0x3f00-0x3fff mem > 0x90ebc000-0x90ebffff,0x912c0000-0x912fffff irq 32 at device 0.0 on pci17 > Oct 27 17:13:16 elibm kernel: mps0: Firmware: 19.00.00.00, Driver: > 19.00.00.00-fbsd > Oct 27 17:13:16 elibm kernel: mps0: IOCCapabilities: > 1285c > Oct 27 17:13:16 elibm kernel: ses0 at mps0 bus 0 scbus0 target 15 lun 0 > Oct 27 17:13:16 elibm kernel: ses1 at mps0 bus 0 scbus0 target 22 lun 0 > Oct 27 17:13:16 elibm kernel: da1 at mps0 bus 0 scbus0 target 9 lun 0 > Oct 27 17:13:16 elibm kernel: da2 at mps0 bus 0 scbus0 target 10 lun 0 > Oct 27 17:13:16 elibm kernel: da0 at mps0 bus 0 scbus0 target 8 lun 0 > Oct 27 17:13:16 elibm kernel: da3 at mps0 bus 0 scbus0 target 11 lun 0 > Oct 27 17:13:16 elibm kernel: da4 at mps0 bus 0 scbus0 target 12 lun 0 > Oct 27 17:13:16 elibm kernel: da5 at mps0 bus 0 scbus0 target 13 lun 0 > Oct 27 17:13:16 elibm kernel: da6 at mps0 bus 0 scbus0 target 14 lun 0 > Oct 27 17:13:16 elibm kernel: da10 at mps0 bus 0 scbus0 target 19 lun 0 > Oct 27 17:13:16 elibm kernel: da7 at mps0 bus 0 scbus0 target 16 lun 0 > Oct 27 17:13:16 elibm kernel: da9 at mps0 bus 0 scbus0 target 18 lun 0 > Oct 27 17:13:16 elibm kernel: da8 at mps0 bus 0 scbus0 target 17 lun 0 > Oct 27 17:13:16 elibm kernel: da11 at mps0 bus 0 scbus0 target 20 lun 0 > Oct 27 17:13:16 elibm kernel: da12 at mps0 bus 0 scbus0 target 21 lun 0 > > > > > > sas2ircu output showing the SAS devices, which are mostly SATA disks. The > setup is > satisfactorily functional with the P19 firmware. > > > > > root@elibm:~ # sas2ircu 0 display > LSI Corporation SAS2 IR Configuration Utility. > Version 18.00.00.00 (2013.11.18) > Copyright (c) 2009-2013 LSI Corporation. All rights reserved. > > Read configuration has been initiated for controller 0 > ------------------------------------------------------------------------ > Controller information > ------------------------------------------------------------------------ > Controller type : SAS2008 > BIOS version : 7.37.00.00 > Firmware version : 19.00.00.00 > Channel description : 1 Serial Attached SCSI > Initiator ID : 0 > Maximum physical devices : 255 > Concurrent commands supported : 3432 > Slot : Unknown > Segment : 0 > Bus : 17 > Device : 0 > Function : 0 > RAID Support : No > ------------------------------------------------------------------------ > IR Volume information > ------------------------------------------------------------------------ > ------------------------------------------------------------------------ > Physical device information > ------------------------------------------------------------------------ > Initiator at ID #0 > > Device is a Hard disk > Enclosure # : 2 > Slot # : 16 > SAS Address : 5000c50-0-05b5-ce25 > State : Ready (RDY) > Size (in MB)/(in sectors) : 140014/286749479 > Manufacturer : SEAGATE > Model Number : ST9146803SS > Firmware Revision : FS03 > Serial No : 3SD02W5L > GUID : N/A > Protocol : SAS > Drive Type : SAS_HDD > > Device is a Hard disk > Enclosure # : 2 > Slot # : 17 > SAS Address : 5005076-0-3e8e-81a2 > State : Ready (RDY) > Size (in MB)/(in sectors) : 953869/1953525167 > Manufacturer : ATA > Model Number : Samsung SSD 840 > Firmware Revision : BB0Q > Serial No : S1D9NEADA08549F > GUID : N/A > Protocol : SATA > Drive Type : SATA_SSD > > Device is a Hard disk > Enclosure # : 2 > Slot # : 18 > SAS Address : 5005076-0-3e8e-81a3 > State : Ready (RDY) > Size (in MB)/(in sectors) : 953869/1953525167 > Manufacturer : ATA > Model Number : Samsung SSD 840 > Firmware Revision : BB0Q > Serial No : S1D9NEADA08548T > GUID : N/A > Protocol : SATA > Drive Type : SATA_SSD > > Device is a Hard disk > Enclosure # : 2 > Slot # : 19 > SAS Address : 5005076-0-3e8e-81a4 > State : Ready (RDY) > Size (in MB)/(in sectors) : 953869/1953525167 > Manufacturer : ATA > Model Number : Samsung SSD 840 > Firmware Revision : BB0Q > Serial No : S1D9NEADA08568E > GUID : N/A > Protocol : SATA > Drive Type : SATA_SSD > > Device is a Hard disk > Enclosure # : 2 > Slot # : 20 > SAS Address : 5005076-0-3e8e-81a5 > State : Ready (RDY) > Size (in MB)/(in sectors) : 953869/1953525167 > Manufacturer : ATA > Model Number : Samsung SSD 840 > Firmware Revision : BB0Q > Serial No : S1D9NEADA08547X > GUID : N/A > Protocol : SATA > Drive Type : SATA_SSD > > Device is a Hard disk > Enclosure # : 2 > Slot # : 21 > SAS Address : 5005076-0-3e8e-81a6 > State : Ready (RDY) > Size (in MB)/(in sectors) : 953869/1953525167 > Manufacturer : ATA > Model Number : Samsung SSD 840 > Firmware Revision : BB0Q > Serial No : S1D9NEADA08518Y > GUID : N/A > Protocol : SATA > Drive Type : SATA_SSD > > Device is a Hard disk > Enclosure # : 2 > Slot # : 22 > SAS Address : 5005076-0-3e8e-81a7 > State : Ready (RDY) > Size (in MB)/(in sectors) : 953869/1953525167 > Manufacturer : ATA > Model Number : Samsung SSD 840 > Firmware Revision : BB0Q > Serial No : S1D9NEADA08556K > GUID : N/A > Protocol : SATA > Drive Type : SATA_SSD > > Device is a Enclosure services device > Enclosure # : 2 > Slot # : 255 > SAS Address : 5005076-0-3e8e-81b9 > State : Standby (SBY) > Manufacturer : IBM-ESXS > Model Number : SAS EXP BP > Firmware Revision : 61A6 > Serial No : 00000006 > GUID : N/A > Protocol : SAS > Device Type : Enclosure services device > > Device is a Hard disk > Enclosure # : 3 > Slot # : 0 > SAS Address : 5005076-0-3e8e-86e9 > State : Ready (RDY) > Size (in MB)/(in sectors) : 953869/1953525167 > Manufacturer : ATA > Model Number : Samsung SSD 840 > Firmware Revision : BB0Q > Serial No : S1D9NEADA08550R > GUID : N/A > Protocol : SATA > Drive Type : SATA_SSD > > Device is a Hard disk > Enclosure # : 3 > Slot # : 1 > SAS Address : 5005076-0-3e8e-86ea > State : Ready (RDY) > Size (in MB)/(in sectors) : 953869/1953525167 > Manufacturer : ATA > Model Number : Samsung SSD 840 > Firmware Revision : BB0Q > Serial No : S1D9NEADA08911Y > GUID : N/A > Protocol : SATA > Drive Type : SATA_SSD > > Device is a Hard disk > Enclosure # : 3 > Slot # : 2 > SAS Address : 5005076-0-3e8e-86eb > State : Ready (RDY) > Size (in MB)/(in sectors) : 953869/1953525167 > Manufacturer : ATA > Model Number : Samsung SSD 840 > Firmware Revision : BB0Q > Serial No : S1D9NEADA08811L > GUID : N/A > Protocol : SATA > Drive Type : SATA_SSD > > Device is a Hard disk > Enclosure # : 3 > Slot # : 13 > SAS Address : 5000c50-0-05b5-e531 > State : Ready (RDY) > Size (in MB)/(in sectors) : 140014/286749479 > Manufacturer : SEAGATE > Model Number : ST9146803SS > Firmware Revision : FS03 > Serial No : 3SD02STR > GUID : N/A > Protocol : SAS > Drive Type : SAS_HDD > > Device is a Hard disk > Enclosure # : 3 > Slot # : 14 > SAS Address : 5000c50-0-05b5-d489 > State : Ready (RDY) > Size (in MB)/(in sectors) : 140014/286749479 > Manufacturer : SEAGATE > Model Number : ST9146803SS > Firmware Revision : FS03 > Serial No : 3SD02TV1 > GUID : N/A > Protocol : SAS > Drive Type : SAS_HDD > > Device is a Hard disk > Enclosure # : 3 > Slot # : 15 > SAS Address : 5000c50-0-05b5-f0ad > State : Ready (RDY) > Size (in MB)/(in sectors) : 140014/286749479 > Manufacturer : SEAGATE > Model Number : ST9146803SS > Firmware Revision : FS03 > Serial No : 3SD03F4C > GUID : N/A > Protocol : SAS > Drive Type : SAS_HDD > > Device is a Enclosure services device > Enclosure # : 3 > Slot # : 255 > SAS Address : 5005076-0-3e8e-86f9 > State : Standby (SBY) > Manufacturer : IBM-ESXS > Model Number : SAS EXP BP > Firmware Revision : 61A6 > Serial No : 00000006 > GUID : N/A > Protocol : SAS > Device Type : Enclosure services device > ------------------------------------------------------------------------ > Enclosure information > ------------------------------------------------------------------------ > Enclosure# : 1 > Logical ID : 500605b0:07ba2100 > Numslots : 8 > StartSlot : 0 > Enclosure# : 2 > Logical ID : 50050760:3e8e81a0 > Numslots : 25 > StartSlot : 0 > Enclosure# : 3 > Logical ID : 50050760:3e8e86e0 > Numslots : 25 > StartSlot : 0 > ------------------------------------------------------------------------ > SAS2IRCU: Command DISPLAY Completed Successfully. > SAS2IRCU: Utility Completed Successfully. > root@elibm:~ # > > From owner-freebsd-scsi@FreeBSD.ORG Mon Nov 3 06:57:37 2014 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7291B43D for ; Mon, 3 Nov 2014 06:57:37 +0000 (UTC) Received: from cu01176a.smtpx.saremail.com (cu01176a.smtpx.saremail.com [195.16.150.151]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9DDCCCA for ; Mon, 3 Nov 2014 06:57:35 +0000 (UTC) Received: from [172.16.2.2] (izaro.sarenet.es [192.148.167.11]) by proxypop03.sare.net (Postfix) with ESMTPSA id 8C8F79DDBBD; Mon, 3 Nov 2014 07:57:25 +0100 (CET) Subject: Re: MPS driver and latest firmware Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Borja Marcos In-Reply-To: Date: Mon, 3 Nov 2014 07:57:22 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Wes Morgan X-Mailer: Apple Mail (2.1283) Cc: freebsd-scsi@freebsd.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Nov 2014 06:57:37 -0000 On Nov 2, 2014, at 10:34 PM, Wes Morgan wrote: > I was updating the firmware hoping that the targets went back to being = assigned in "next available" order rather than sticky. I like each bay = to have a specific device name, which is impossible due to the = "remembered" target IDs. Although complicated, sas2ircu is your friend. It will give you an (as far as I know!) accurate list of disks, = including the backplane slot where they are plugged.=20 For now it's the best solution I have found to identify the disks. The = funny thing is, it's an "IR" configuration utility and my cards are in "IT" mode. But it works :) Cheers, Borja. Output example: root@elibm:~ # sas2ircu 0 display LSI Corporation SAS2 IR Configuration Utility. Version 18.00.00.00 (2013.11.18)=20 Copyright (c) 2009-2013 LSI Corporation. All rights reserved.=20 Read configuration has been initiated for controller 0 ------------------------------------------------------------------------ Controller information ------------------------------------------------------------------------ Controller type : SAS2008 BIOS version : 7.37.00.00 Firmware version : 19.00.00.00 Channel description : 1 Serial Attached SCSI Initiator ID : 0 Maximum physical devices : 255 Concurrent commands supported : 3432 Slot : Unknown Segment : 0 Bus : 17 Device : 0 Function : 0 RAID Support : No ------------------------------------------------------------------------ IR Volume information ------------------------------------------------------------------------ ------------------------------------------------------------------------ Physical device information ------------------------------------------------------------------------ Initiator at ID #0 Device is a Hard disk Enclosure # : 2 Slot # : 16 SAS Address : 5000c50-0-05b5-ce25 State : Ready (RDY) Size (in MB)/(in sectors) : 140014/286749479 Manufacturer : SEAGATE=20 Model Number : ST9146803SS =20 Firmware Revision : FS03 Serial No : 3SD02W5L GUID : N/A Protocol : SAS Drive Type : SAS_HDD Device is a Hard disk Enclosure # : 2 Slot # : 17 SAS Address : 5005076-0-3e8e-81a2 State : Ready (RDY) Size (in MB)/(in sectors) : 953869/1953525167 Manufacturer : ATA =20 Model Number : Samsung SSD 840=20 Firmware Revision : BB0Q Serial No : S1D9NEADA08549F GUID : N/A Protocol : SATA Drive Type : SATA_SSD Device is a Hard disk Enclosure # : 2 Slot # : 18 SAS Address : 5005076-0-3e8e-81a3 State : Ready (RDY) Size (in MB)/(in sectors) : 953869/1953525167 Manufacturer : ATA =20 Model Number : Samsung SSD 840=20 Firmware Revision : BB0Q Serial No : S1D9NEADA08548T GUID : N/A Protocol : SATA Drive Type : SATA_SSD Device is a Hard disk Enclosure # : 2 Slot # : 19 SAS Address : 5005076-0-3e8e-81a4 State : Ready (RDY) Size (in MB)/(in sectors) : 953869/1953525167 Manufacturer : ATA =20 Model Number : Samsung SSD 840=20 Firmware Revision : BB0Q Serial No : S1D9NEADA08568E GUID : N/A Protocol : SATA Drive Type : SATA_SSD Device is a Hard disk Enclosure # : 2 Slot # : 20 SAS Address : 5005076-0-3e8e-81a5 State : Ready (RDY) Size (in MB)/(in sectors) : 953869/1953525167 Manufacturer : ATA =20 Model Number : Samsung SSD 840=20 Firmware Revision : BB0Q Serial No : S1D9NEADA08547X GUID : N/A Protocol : SATA Drive Type : SATA_SSD Device is a Hard disk Enclosure # : 2 Slot # : 21 SAS Address : 5005076-0-3e8e-81a6 State : Ready (RDY) Size (in MB)/(in sectors) : 953869/1953525167 Manufacturer : ATA =20 Model Number : Samsung SSD 840=20 Firmware Revision : BB0Q Serial No : S1D9NEADA08518Y GUID : N/A Protocol : SATA Drive Type : SATA_SSD Device is a Hard disk Enclosure # : 2 Slot # : 22 SAS Address : 5005076-0-3e8e-81a7 State : Ready (RDY) Size (in MB)/(in sectors) : 953869/1953525167 Manufacturer : ATA =20 Model Number : Samsung SSD 840=20 Firmware Revision : BB0Q Serial No : S1D9NEADA08556K GUID : N/A Protocol : SATA Drive Type : SATA_SSD Device is a Enclosure services device Enclosure # : 2 Slot # : 255 SAS Address : 5005076-0-3e8e-81b9 State : Standby (SBY) Manufacturer : IBM-ESXS Model Number : SAS EXP BP =20 Firmware Revision : 61A6 Serial No : 00000006 GUID : N/A Protocol : SAS Device Type : Enclosure services device Device is a Hard disk Enclosure # : 3 Slot # : 0 SAS Address : 5005076-0-3e8e-86e9 State : Ready (RDY) Size (in MB)/(in sectors) : 953869/1953525167 Manufacturer : ATA =20 Model Number : Samsung SSD 840=20 Firmware Revision : BB0Q Serial No : S1D9NEADA08550R GUID : N/A Protocol : SATA Drive Type : SATA_SSD Device is a Hard disk Enclosure # : 3 Slot # : 1 SAS Address : 5005076-0-3e8e-86ea State : Ready (RDY) Size (in MB)/(in sectors) : 953869/1953525167 Manufacturer : ATA =20 Model Number : Samsung SSD 840=20 Firmware Revision : BB0Q Serial No : S1D9NEADA08911Y GUID : N/A Protocol : SATA Drive Type : SATA_SSD Device is a Hard disk Enclosure # : 3 Slot # : 2 SAS Address : 5005076-0-3e8e-86eb State : Ready (RDY) Size (in MB)/(in sectors) : 953869/1953525167 Manufacturer : ATA =20 Model Number : Samsung SSD 840=20 Firmware Revision : BB0Q Serial No : S1D9NEADA08811L GUID : N/A Protocol : SATA Drive Type : SATA_SSD Device is a Hard disk Enclosure # : 3 Slot # : 13 SAS Address : 5000c50-0-05b5-e531 State : Ready (RDY) Size (in MB)/(in sectors) : 140014/286749479 Manufacturer : SEAGATE=20 Model Number : ST9146803SS =20 Firmware Revision : FS03 Serial No : 3SD02STR GUID : N/A Protocol : SAS Drive Type : SAS_HDD Device is a Hard disk Enclosure # : 3 Slot # : 14 SAS Address : 5000c50-0-05b5-d489 State : Ready (RDY) Size (in MB)/(in sectors) : 140014/286749479 Manufacturer : SEAGATE=20 Model Number : ST9146803SS =20 Firmware Revision : FS03 Serial No : 3SD02TV1 GUID : N/A Protocol : SAS Drive Type : SAS_HDD Device is a Hard disk Enclosure # : 3 Slot # : 15 SAS Address : 5000c50-0-05b5-f0ad State : Ready (RDY) Size (in MB)/(in sectors) : 140014/286749479 Manufacturer : SEAGATE=20 Model Number : ST9146803SS =20 Firmware Revision : FS03 Serial No : 3SD03F4C GUID : N/A Protocol : SAS Drive Type : SAS_HDD Device is a Enclosure services device Enclosure # : 3 Slot # : 255 SAS Address : 5005076-0-3e8e-86f9 State : Standby (SBY) Manufacturer : IBM-ESXS Model Number : SAS EXP BP =20 Firmware Revision : 61A6 Serial No : 00000006 GUID : N/A Protocol : SAS Device Type : Enclosure services device ------------------------------------------------------------------------ Enclosure information ------------------------------------------------------------------------ Enclosure# : 1 Logical ID : 500605b0:07ba2100 Numslots : 8 StartSlot : 0 Enclosure# : 2 Logical ID : 50050760:3e8e81a0 Numslots : 25 StartSlot : 0 Enclosure# : 3 Logical ID : 50050760:3e8e86e0 Numslots : 25 StartSlot : 0 ------------------------------------------------------------------------ SAS2IRCU: Command DISPLAY Completed Successfully. SAS2IRCU: Utility Completed Successfully. root@elibm:~ #=20 From owner-freebsd-scsi@FreeBSD.ORG Tue Nov 4 20:20:20 2014 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0A4CF17E; Tue, 4 Nov 2014 20:20:20 +0000 (UTC) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1on0060.outbound.protection.outlook.com [157.56.110.60]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6F0A091E; Tue, 4 Nov 2014 20:20:17 +0000 (UTC) Received: from DM2PR0801MB0944.namprd08.prod.outlook.com (25.160.131.27) by DM2PR0801MB0944.namprd08.prod.outlook.com (25.160.131.27) with Microsoft SMTP Server (TLS) id 15.1.16.15; Tue, 4 Nov 2014 20:05:32 +0000 Received: from DM2PR0801MB0944.namprd08.prod.outlook.com ([25.160.131.27]) by DM2PR0801MB0944.namprd08.prod.outlook.com ([25.160.131.27]) with mapi id 15.01.0016.006; Tue, 4 Nov 2014 20:05:32 +0000 From: "Pokala, Ravi" To: "freebsd-scsi@freebsd.org" , "ken@freebsd.org" , "sbruno@freebsd.org" , "gibbs@freebsd.org" Subject: Low-level trace-buffers in CAM Thread-Topic: Low-level trace-buffers in CAM Thread-Index: AQHP+GqvtYH3n70bj0eIcdTdvXSFZQ== Date: Tue, 4 Nov 2014 20:05:31 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.5.141003 x-originating-ip: [129.253.54.225] x-microsoft-antispam: BCL:0;PCL:0;RULEID:;SRVR:DM2PR0801MB0944; x-exchange-antispam-report-test: UriScan:; x-forefront-prvs: 03853D523D x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(979002)(6009001)(189002)(164054003)(199003)(36756003)(66066001)(2501002)(97736003)(64706001)(46102003)(31966008)(40100003)(122556002)(83506001)(95666004)(99286002)(105586002)(120916001)(99396003)(107886001)(229853001)(4396001)(107046002)(101416001)(575784001)(106356001)(86362001)(2201001)(21056001)(54356999)(551934003)(87936001)(92566001)(450100001)(77096003)(106116001)(62966003)(50986999)(92726001)(77156002)(20776003)(2656002)(21314002)(969003)(989001)(999001)(1009001)(1019001); DIR:OUT; SFP:1101; SCL:1; SRVR:DM2PR0801MB0944; H:DM2PR0801MB0944.namprd08.prod.outlook.com; FPR:; MLV:ovrnspm; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: panasas.com X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Nov 2014 20:20:20 -0000 Re-posting something that I sent to freebsd-geom@ last night; I didn't see a freebsd-cam@ list, but I was told this morning that CAM issues should go to freebsd-scsi@. Also adding some known-interested parties. Thanks, Ravi ---------------- Hi folks, At Panasas, we found it useful to have a trace buffer for each ATA drive in the system, for gathering information about command sequences and latency. I implemented that functionality for our old 7-ish ATA driver, and it works quite well, with fairly low overhead. For example (sorry for the really long lines...): # Tabular format for use w/ other tools # Request time ATA request information Response time ATA response information sts/err flags 1415043680586349 25 0000 0200 000001b8c200 1415043680587357 25 0000 0200 000001b8c200 50 00 0000007d 1415043680587389 25 0000 0200 000001b8c400 1415043680588396 25 0000 0200 000001b8c400 50 00 0000007d 1415043680588430 25 0000 0200 000001b8c600 1415043680589406 25 0000 0200 000001b8c600 50 00 0000007d 1415043681203051 ca 0000 0008 000000000480 1415043681203279 ca 0000 0008 000000000480 50 00 0000007d 1415043681203284 ca 0000 0008 000000002f00 1415043681204022 ca 0000 0008 000000002f00 50 00 0000007d 1415043681596331 25 0000 0200 000001b8c800 1415043681597389 25 0000 0200 000001b8c800 50 00 0000007d 1415043681597423 25 0000 0200 000001b8ca00 1415043681598434 25 0000 0200 000001b8ca00 50 00 0000007d 1415043681598466 25 0000 0200 000001b8cc00 1415043681599484 25 0000 0200 000001b8cc00 50 00 0000007d 1415043681599517 25 0000 0200 000001b8ce00 1415043681600515 25 0000 0200 000001b8ce00 50 00 0000007d 1415043681934541 25 0000 0200 000000586600 1415043681935579 25 0000 0200 000000586600 50 00 0000007d # Human-readable FLAGS REQUEST TIME RESPONSE TIME ELAPSED TIME REQUEST COMMAND STATUS ERROR RESPONSE COMMAND =20 -------------------------------- -------------------- -------------------- -------------------- --------------------------------------------------------------------------- ----- -------- -------- --------------------------------------------------------------------------- ----- ____C...._V 1415043680586349 1415043680587357 1008 READ DMA EXT (LBA 28885504 + 512 sectors) _R_S____ ________ ---- =20 ____C...._V 1415043680587389 1415043680588396 1007 READ DMA EXT (LBA 28886016 + 512 sectors) _R_S____ ________ ---- =20 ____C...._V 1415043680588430 1415043680589406 976 READ DMA EXT (LBA 28886528 + 512 sectors) _R_S____ ________ ---- =20 ____C...._V 1415043681203051 1415043681203279 228 WRITE DMA (LBA 1152 + 8 sectors) _R_S____ ________ ---- =20 ____C...._V 1415043681203284 1415043681204022 738 WRITE DMA (LBA 12032 + 8 sectors) _R_S____ ________ ---- =20 ____C...._V 1415043681596331 1415043681597389 1058 READ DMA EXT (LBA 28887040 + 512 sectors) _R_S____ ________ ---- =20 ____C...._V 1415043681597423 1415043681598434 1011 READ DMA EXT (LBA 28887552 + 512 sectors) _R_S____ ________ ---- =20 ____C...._V 1415043681598466 1415043681599484 1018 READ DMA EXT (LBA 28888064 + 512 sectors) _R_S____ ________ ---- =20 As we start moving toward a more modern base release, we want to re-implement that functionality for ATA, SCSI, NVMe, etc. And push it upsream. The obvious place to put this sort of tracing would be in GEOM, but that actually isn't what we want, for a few reasons. First of all, we're primarily interested in the exact register / taskfile / CDB values sent to the drive - opcode, feature codes, LBAs, etc. Second, we're interested in hardware-level latency - how long does a request stay in the SIM and PERIPH. Both of those pretty much require working below GEOM. Therefore, I think we want to do it in the CAM stack; specifically, the hooks would need to go into some of the SIMs. (Yes, I know, this requires touching every SIM which we want to use. It's not optimal, but the hooks should be pretty lightweight and easy to add, and nothing will break if a given SIM isn't updated.) I took our implementation, and pseudo-coded what it might look like in CAM. (I could probably make diffs against the old ATA stack available for comparison, but I figured no one would care anymore.) A huge caveat is that I've never really poked in CAM before, so I fully expect I'm doing several things wrong. Please educate me. :-) =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D Create a circular buffer of trace records for each CAM PERIPH, getting information as close to the device as possible. Each record will contain the full CDB for the request as sent to the device, the timestamp (in usec) of the request, the CDB as returned by the device, the timestamp of the response, and the status and error code. Additionally, flags are provided to track the state of the trace record, and to indicate which type of periph the record is associated with. typedef struct { u_int32_t flags; #define CAM_TRACE_F_VALID 0x00000001 /* valid record */ #define CAM_TRACE_F_INPROGRESS 0x00000002 /* request in progress */ #define CAM_TRACE_F_REQ_VALID 0x00000004 /* request data valid */ #define CAM_TRACE_F_RESP_VALID 0x00000008 /* response data valid */ #define CAM_TRACE_F_COMPLETE 0x00000010 /* response gathered */ #define CAM_TRACE_F_TIMEDOUT 0x00000020 /* request timed out */ #define CAM_TRACE_F_ABANDONED 0x00000040 /* cancelled, etc. */ #define CAM_TRACE_F_RETRIED 0x00000080 /* retry in later record */ #define CAM_TRACE_F_IS_RETRY 0x00000100 /* retry of earlier req. */ #define CAM_TRACE_F_PERIPH_MASK 0xf0000000 /* up to 16 types of PERIPHs */ #define CAM_TRACE_F_PERIPH_DA 0x0 #define CAM_TRACE_F_PERIPH_ADA 0x1 #define CAM_TRACE_F_PERIPH_NVD 0x2 u_int64_t req_usec; cdb_t req_cdb; u_int64_t resp_usec; cdb_t resp_cdb; u_int8_t resp_status; u_int8_t resp_error; u_int8_t resp_asc; /* Needed for SCSI? */ u_int8_t resp_asq; /* Needed for SCSI? */ u_int8_t padding[???]; /* Pad for alignment? */ /* There's going to be a large-ish array of these in the kernel; make sure they're packed and aligned to minimize space usage and keep things aligned. The original implementation was 48 bytes, which was already aligned. */ } cam_trace_record_t __attribute__((packed, aligned(8))); typedef struct { u_int32_t trace_buf_record_count; cam_trace_record_t *trace_buf; } cam_trace_buffer_t; #define CAM_TRACE_BUFFER_DEFAULT_COUNT 100000 ---------------- ioctl interfaces are used to get the trace buffer size, to retrieve the buffer contents, or to clear the buffer. The buffer can then be analyzed from userspace, or written to a file for post-processing. #define CAM_GET_TRACE_RECORD_COUNT _IOR(???, ???, u_int32_t) #define CAM_GET_TRACE_BUFFER _IOWR(???, ???, cam_trace_buffer_t) #define CAM_CLEAR_TRACE_BUFFER _IOW(???, ???, u_int32_t) ---------------- To get the buffer: cam_trace_buffer_t trace =3D NULL; fd =3D open(periph_name, O_RDONLY); error =3D ioctl(fd, CAM_GET_TRACE_RECORD_COUNT, &trace.trace_buf_record_count); trace.trace_buf =3D malloc(record_count * sizeof (cam_trace_record_t)); error =3D ioctl(fd, CAM_GET_TRACE_BUFFER, &trace); By including the full CDB, the entire command can be decoded by analysis tools. Tools can be written to parse the buffer and translate command data into human-readable descriptions, calculate elapsed times for requests, perform statistical analysis based on opcode, command type, transfer size, etc. Command decoders would need to be written for each PERIPH type (i.e. one for ATA, one for SCSI, one for NVMe, etc.). ---------------- Each PERIPH's softc needs to include some information about the per-device buffer. u_int32_t trace_buf_record_count; u_int64_t trace_buf_request_count; cam_trace_record_t *trace_buf; In each SIM, there should already be some sort of request structure, which associates the softc and the CDB of the request. To that structure, add a pointer to the trace record associated with the request. request structure: cam_trace_record_t *trace_rec; ---------------- When initializing the per-device softc, allocate a trace buffer. The number of records can be hard-coded, or settable via a tunable. /** Initialize per-drive command tracing buffer. **/ /* Get the buffer size from the tunable, if present. Sanity check it. */ sc->trace_buf_request_count =3D 0; sc->trace_buf_record_count =3D CAM_TRACE_BUFFER_DEFAULT_COUNT; TUNABLE_ULONG_FETCH("cam.trace_buf_record_count", &trace_buf_record_count_tunable); if (trace_buf_record_count_tunable) { /* Less than 1K records is unusably few; more than 1M records is unusably many. */ if ((trace_buf_record_count_tunable < (1 << 10)) || (trace_buf_record_count_tunable > (1 << 20))) { device_printf(dev, "cam.trace_buf_record_count=3D%lu" " out of range; should be between %u and %u\n", trace_buf_record_count_tunable, (1 << 10), (1 << 20)); } else { sc->trace_buf_record_count =3D (u_int32_t) trace_buf_record_count_tunabl= e; } } /* Allocate the buffer. */ trace_buf_bytes =3D sc->trace_buf_record_count * sizeof(cam_trace_record_t= ); sc->trace_buf =3D malloc(trace_buf_bytes, M_AD, M_NOWAIT | M_ZERO); if (sc->trace_buf =3D=3D NULL) { device_printf(dev, "unable to allocate %zd for trace_buf\n", trace_buf_bytes); sc->trace_buf_record_count =3D 0; } else { device_printf(dev, "allocated %zd @ %p for trace_buf\n", trace_buf_bytes, sc->trace_buf); } ---------------- /* Initial setup of the trace record for a new request. * * Locate the circular per-device trace buffer in the drive's softc. * Populate the pointer to the next record in the trace buffer; this is * done atomically to prevent concurrent requests from getting the same * index. That could get a bit weird if/when 'trace_buf_request_count' * rolls over, if the maximum counter value is not a multiple of * 'trace_buf_record_count' (i.e. the request counter recycles in the * middle of the buffer). By using a 64-bit request counter, we're pretty * safe - even at one transaction per nanosecond, that will still last * about 585 years. * Initialize the record by zeroing it and then setting the VALID flag. * * This could be done at any point before sending the request to the * hardware. */ void trace_rec_init(cam_request_t req) { ada_softc *sc =3D (request->...->softc) u_int64_t tmp =3D 0; if (sc && sc->trace_buf) { tmp =3D atomic_fetchadd_log(&sc->trace_buf_request_count, 1); req->trace_rec =3D &sc->trace_buf[tmp % sc->trace_buf_record_count]; bzero(req->trace_rec, sizeof(*req->trace_rec)); req->trace_rec->flags =3D (CAM_TRACE_F_PERIPH_XXX << 28); req->trace_rec->flags |=3D CAM_TRACE_F_VALID; } else { req->trace_rec =3D NULL; } } ---------------- /* Stuff the request data into the trace record. * * If the trace record is present and valid, copy the CDB into the * record. Set CAM_TRACE_F_REQ_VALID to indicate that the request data is * valid, and INPROGRESS to show that the request is in-progress. * * This should be done just before sending the request to the hardware. * This should *not* be fused into trace_rec_init(), because * trace_rec_init() is also used in the retry case, where it is not correct * to immediately start the timer. */ void trace_rec_req(cam_request_t req) { struct timeval tv; if (req->trace_rec && (req->trace_rec->flags & CAM_TRACE_F_VALID)) { bcopy(req->cdb, req->trace_rec->req_cdb, sizeof(req->cdb)); microuptime(&tv); req->trace_rec->req_usec =3D ((uint64_t) tv->tv_sec * 1000000) + tv->tv_usec; req->trace_rec->flags |=3D (CAM_TRACE_F_REQ_VALID | CAM_TRACE_F_INPROGRESS); } } ---------------- /* Stuff the response data into the trace record. * * If the trace record is present and valid, copy the CDB **as returned by * the drive** into the record. At this point, we also have the result from * the command (status, error, (asc? ascq?)), so save them too. Set * CAM_TRACE_F_RESP_VALID to indicate that the response data is valid. * Clear INPROGRESS and set COMPLETE instead. * * This should be done just after getting the response information from the * hardware */ void trace_rec_resp(cam_request_t req) { struct timeval tv; if (req->trace_rec && (req->trace_rec->flags & CAM_TRACE_F_VALID)) { bcopy(req->cdb, req->trace_rec->resp_cdb, sizeof(req->cdb)); microuptime(&tv); req->trace_rec->resp_usec =3D ((uint64_t) tv->tv_sec * 1000000) + tv->tv_usec; req->trace_rec->flags |=3D CAM_TRACE_F_RESP_VALID | CAM_TRACE_F_COMPLETE)= ; req->trace_rec->flags &=3D ~CAM_TRACE_F_INPROGRESS; } } ---------------- /* Update the trace record if there was a timeout. * * If the trace record is present and valid, set TIMEDOUT to indicate * that the request associated with the record exceeded its timeout. * Since this attempt is complete, clear INPROGRESS and set COMPLETE. * * This should be done along with other timeout handling. */ void trace_rec_timeout(cam_request_t req) { if (req->trace_rec && (req->trace_rec->flags & CAM_TRACE_F_VALID)) { req->trace_rec->flags &=3D ~CAM_TRACE_F_INPROGRESS; req->trace_rec->flags |=3D (CAM_TRACE_F_TIMEDOUT | CAM_TRACE_F_COMPLETE); } } ---------------- /* Update the trace record if the request is being retried. * =20 * If the trace record is present and valid, set RETRIED to indicate * that the request associated with the record is going to be retried. * Since this attempt is complete, clear INPROGRESS and set COMPLETE. * Then get a new trace record for the new attempt, copy the request * command data to the new record, and set IS_RETRY to show that this * is a retry of an earlier request. * * This should be done at the time the retry is queued. */ void trace_rec_retry(cam_request_t req) { if (req->trace_rec && (req->trace_rec->flags & PAN_ATA_TRACE_F_VALID)) { /* First, mark the original request as being requeued. */ req->trace_rec->flags &=3D ~CAM_TRACE_F_INPROGRESS; req->trace_rec->flags |=3D (CAM_TRACE_F_RETRIED | CAM_TRACE_F_COMPLETE); /* Get a new record for the retry. */ trace_rec_init(req); /* Mark the new record as being a retry. */ req->trace_rec->flags |=3D CAM_TRACE_F_IS_RETRY; } } =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D I'd appreciate feedback, both on the implementation details and on the idea as a whole. Some open questions are: - My assumption is that each SIM has a structure that associates a PERIPH's softc with the CDB that's being processed. It might require dereferencing a chain of pointers, but it should be there in some fashion. - The current implementation adds a few ioctls to the old ad(4) disk-device driver. I would expect to add the new ioctls to da(4) and ada(4); is that the right thing to do? Again, any feedback would be greatly appreciated. I'm at the Dev/Vendor Summit today and tomorrow if anyone wants to discuss this with me in person. Thanks, Ravi From owner-freebsd-scsi@FreeBSD.ORG Thu Nov 6 15:13:29 2014 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1C27D1FE for ; Thu, 6 Nov 2014 15:13:29 +0000 (UTC) Received: from cu01176a.smtpx.saremail.com (cu1176c.smtpx.saremail.com [195.16.148.151]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CF002C25 for ; Thu, 6 Nov 2014 15:13:28 +0000 (UTC) Received: from [172.16.2.2] (izaro.sarenet.es [192.148.167.11]) by proxypop02.sare.net (Postfix) with ESMTPSA id 780D99DCF2E for ; Thu, 6 Nov 2014 16:13:26 +0100 (CET) From: Borja Marcos Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: New quirks for Intel 320 SSDs Date: Thu, 6 Nov 2014 16:13:25 +0100 Message-Id: <789D2893-70C9-4312-ADC6-15AC58C6BC77@sarenet.es> To: FreeBSD-scsi Mime-Version: 1.0 (Apple Message framework v1283) X-Mailer: Apple Mail (2.1283) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Nov 2014 15:13:29 -0000 I stumbled upon two 40 GB Intel 320 SSDs with a new id string. It's very similar to the id string of the Intel 330's, changing just a = letter. The up to date quirks that should be added is: (ata_da.c) { /* * Intel 320 Series SSDs (another ID string) * 4k optimised & trim only works in 4k requests + 4k = aligned */ { T_DIRECT, SIP_MEDIA_FIXED, "*", "INTEL SSDSA2CT*", "*" = }, /*quirks*/ADA_Q_4K }, and of course, (scsi_da.c) { /* * Intel 320 Series SSDs (another ID string) * 4k optimised & trim only works in 4k requests + 4k = aligned */ { T_DIRECT, SIP_MEDIA_FIXED, "*", "INTEL SSDSA2CT*", "*" = }, /*quirks*/DA_Q_4K }, ada1: ATA-8 SATA 2.x device ada1: Serial Number PEPR408501DV040AGN ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada1: Command Queueing enabled ada1: 38166MB (78165360 512 byte sectors: 16H 63S/T 16383C) Cheers, Borja. From owner-freebsd-scsi@FreeBSD.ORG Thu Nov 6 15:15:34 2014 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9B671398 for ; Thu, 6 Nov 2014 15:15:34 +0000 (UTC) Received: from cu01176b.smtpx.saremail.com (cu01176b.smtpx.saremail.com [195.16.151.151]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 58410C49 for ; Thu, 6 Nov 2014 15:15:33 +0000 (UTC) Received: from [172.16.2.2] (izaro.sarenet.es [192.148.167.11]) by proxypop04.sare.net (Postfix) with ESMTPSA id 7BD2E9DEAE0 for ; Thu, 6 Nov 2014 16:15:30 +0100 (CET) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v1283) Subject: Re: New quirks for Intel 320 SSDs From: Borja Marcos In-Reply-To: <789D2893-70C9-4312-ADC6-15AC58C6BC77@sarenet.es> Date: Thu, 6 Nov 2014 16:15:29 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <23A2FC51-A96B-49A8-BEFC-35CA45253554@sarenet.es> References: <789D2893-70C9-4312-ADC6-15AC58C6BC77@sarenet.es> To: FreeBSD-scsi X-Mailer: Apple Mail (2.1283) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Nov 2014 15:15:34 -0000 Sorry for the typo, of course the entry for scsi_da.c is missing the = "ATA" string. On Nov 6, 2014, at 4:13 PM, Borja Marcos wrote: > (scsi_da.c) > { > /* > * Intel 320 Series SSDs (another ID string) > * 4k optimised & trim only works in 4k requests + 4k = aligned > */ > { T_DIRECT, SIP_MEDIA_FIXED, "ATA", "INTEL SSDSA2CT*", = "*" }, > /*quirks*/DA_Q_4K > }, >=20 >=20 >=20 > ada1: ATA-8 SATA 2.x device > ada1: Serial Number PEPR408501DV040AGN > ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > ada1: Command Queueing enabled > ada1: 38166MB (78165360 512 byte sectors: 16H 63S/T 16383C) From owner-freebsd-scsi@FreeBSD.ORG Thu Nov 6 23:38:08 2014 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B24DF2C6 for ; Thu, 6 Nov 2014 23:38:08 +0000 (UTC) Received: from exprod7og105.obsmtp.com (exprod7og105.obsmtp.com [64.18.2.163]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 23704EF7 for ; Thu, 6 Nov 2014 23:38:07 +0000 (UTC) Received: from mail-wi0-f170.google.com ([209.85.212.170]) (using TLSv1) by exprod7ob105.postini.com ([64.18.6.12]) with SMTP ID DSNKVFwGWfhwgdGR5ybiod6qTyOc1VRXnKuK@postini.com; Thu, 06 Nov 2014 15:38:08 PST Received: by mail-wi0-f170.google.com with SMTP id r20so2925084wiv.3 for ; Thu, 06 Nov 2014 15:38:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:mime-version:thread-index:date:message-id :subject:to:content-type; bh=0F9jQMQ047y35xDItcwV83567lsth40VUWH2x7jAKFc=; b=Kd45cValekmGBHTvbUt+hbWFJTo5eWF0RO81UKG/ZelP4V80XLlt+yCHXizVZs+paM 3Svd4gtpBYxAcVG3Frvb4K5oD6t8MyNOBTDne3SyZoKc1HPaHlbJ6hwlglz6C7oadMUc 4YQ39lhg5TmpXCDCS9YRW6YrutiUuMpS/yX8FLkBnZR0VmRe6y/eeKqf5vKK5A6vt6od RecZqoffTCrwQMJ5n5GDHlABy82uSsXbyj7mh2Qc8Gjuod9NM81K60px0bu+LeS5rlIj LHoTEHxpXt7aVtb6yieyXsK4DFB6VazSh6P6AURIIMTI4v6jcWdfTfextQfKeFUM8/nS z+bg== X-Gm-Message-State: ALoCoQnj3wRqkZoGOQJeAWi+Zzz/IMjMF0s3TB5Tg0G8EZqeG88rorHWE0li8vygTOdyteJRtqCcjOx9F4fAuyPPFsLJFr9doji0bXjffMbCfNRKET5UuFOQ+QtBPR3LeJy0IptLd/tUTXvQHQJYNQGgPAR3hxJy4g== X-Received: by 10.180.93.37 with SMTP id cr5mr19339159wib.76.1415317080462; Thu, 06 Nov 2014 15:38:00 -0800 (PST) X-Received: by 10.180.93.37 with SMTP id cr5mr19339151wib.76.1415317080342; Thu, 06 Nov 2014 15:38:00 -0800 (PST) From: Sibananda Sahu MIME-Version: 1.0 X-Mailer: Microsoft Outlook 14.0 Thread-Index: Ac/6GrIKbgSmFCvSRleA0P5ea1lcxQ== Date: Fri, 7 Nov 2014 05:07:59 +0530 Message-ID: Subject: Query regarding Unmapped IO, PIM_UNMAPPED and bus_dmamap_load_ccb() To: freebsd-scsi@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Nov 2014 23:38:08 -0000 Hi All, I have raised this query regarding the following mail: https://lists.freebsd.org/pipermail/freebsd-scsi/2014-July/006407.html Where Alexander Motin talks of Unmapped I/O. As per his direction I have declared the below statement in my driver code: ccb->cpi.hba_misc = PIM_NOBUSRESET | PIM_UNMAPPED; Before enabling PIM_UNMAPPED I was getting the (ccb->ccb_h.flags & CAM_DATA_MASK) as CAM_DATA_VADDR and handling accordingly. But after enabling PIM_UNMAPPED in cpi.hba_misc I am getting the (ccb->ccb_h.flags & CAM_DATA_MASK) as CAM_DATA_BIO and handling the data that comes with bus_dma_load_ccb(). I just did the same thing according to the mps(4) driver as referred by Alexander Motin. So I just wanted to know what exactly happening when I am enabling the PIM_UNMAPPED in the cpi.hba_misc field. And what the CAM_DATA_BIO really means in the CAM_DATA_MASK. It would be great idea if somebody explains what exactly happening or at least give me some references where can I have a look and move forward. Thanks, Sibananda Sahu From owner-freebsd-scsi@FreeBSD.ORG Fri Nov 7 14:23:24 2014 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 92BF6E1D for ; Fri, 7 Nov 2014 14:23:24 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1F308B3A for ; Fri, 7 Nov 2014 14:23:23 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id sA7ENAvv058606 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 7 Nov 2014 16:23:10 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua sA7ENAvv058606 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id sA7EN9PY058605; Fri, 7 Nov 2014 16:23:09 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 7 Nov 2014 16:23:09 +0200 From: Konstantin Belousov To: Sibananda Sahu Subject: Re: Query regarding Unmapped IO, PIM_UNMAPPED and bus_dmamap_load_ccb() Message-ID: <20141107142309.GR53947@kib.kiev.ua> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: freebsd-scsi@freebsd.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Nov 2014 14:23:24 -0000 On Fri, Nov 07, 2014 at 05:07:59AM +0530, Sibananda Sahu wrote: > Hi All, > > > > I have raised this query regarding the following mail: > > https://lists.freebsd.org/pipermail/freebsd-scsi/2014-July/006407.html > > > > Where Alexander Motin talks of Unmapped I/O. > > As per his direction I have declared the below statement in my driver code: > > > > ccb->cpi.hba_misc = PIM_NOBUSRESET | PIM_UNMAPPED; > > > > Before enabling PIM_UNMAPPED I was getting the (ccb->ccb_h.flags & > CAM_DATA_MASK) as CAM_DATA_VADDR and handling accordingly. > > But after enabling PIM_UNMAPPED in cpi.hba_misc I am getting the > (ccb->ccb_h.flags & CAM_DATA_MASK) as CAM_DATA_BIO and handling the data > that comes with bus_dma_load_ccb(). Right. Look at the code. In particular, the definition of the struct bio in sys/sys/bio.h. There, the buffer pointed by bio_data is passed if BIO_UNMAPPED flag is not set, but when the flag is set, bio_ma* fields describe an array of physical memory pages where the data for i/o request are scattered. The pages are not neccessary mapped into the KVA, which explains the terminology. The i/o stack and busdma(9) where modified to transparently handle such requests, and you correctly described the minimal changes, required from the driver side to process the unmapped requests. The passing of a pointer to bio instead of the pointer to buffer allows drivers to not care about mapped/not mapped requests, assuming the suitable KPI like bus_dma_load_ccb() prepares the buffer for dma engine of the controller. > > > > I just did the same thing according to the mps(4) driver as referred by > Alexander Motin. > > > > So I just wanted to know what exactly happening when I am enabling the > PIM_UNMAPPED in the cpi.hba_misc field. > > And what the CAM_DATA_BIO really means in the CAM_DATA_MASK. > > > > > > It would be great idea if somebody explains what exactly happening or at > least give me some references where can I have a look and move forward. > > > > > > Thanks, > > Sibananda Sahu > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org"