From owner-freebsd-stable@FreeBSD.ORG Mon Sep 29 20:25:21 2014 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 52B32E6A for ; Mon, 29 Sep 2014 20:25:21 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id E0589170 for ; Mon, 29 Sep 2014 20:25:20 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 12FD520E7088B; Mon, 29 Sep 2014 20:25:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.2 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id 46A2A20E70885; Mon, 29 Sep 2014 20:25:06 +0000 (UTC) Message-ID: <018BC41041EE4DF589EF5D834AD45BAD@multiplay.co.uk> From: "Steven Hartland" To: "Karl Denninger" , References: <5429BB41.8080609@denninger.net> Subject: Re: MPS Date: Mon, 29 Sep 2014 21:25:02 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Sep 2014 20:25:21 -0000 I seem to remember detection being made parallel for speed, as there can be lots of drives. The Avago guys may be able to shed more light. Regards Steve ----- Original Message ----- From: "Karl Denninger" To: Sent: Monday, September 29, 2014 9:04 PM Subject: MPS Has anyone found a set of configuration settings for the mps driver (LSI SAS series cards) that gets the following situation under control? I have a machine here with one of these cards and a SAS expander. The expander has 16 ports, the card has 8 -- one connector attached to the expander. So I have 20 potential drives associated with this adapter. The issue is that the driver makes utterly no sense when it comes to assigning drive designations. In theory it should enumerate in series; that is, target 0 on the card should be da0, target 1 da1, etc. This assumes there are no gaps in the attached drives; if there are then things may move around, which is why some people wire drives. Ok so far, but it doesn't behave that way! This is the excerpt of what it returns: root@Dbms-10:/boot # dmesg|grep mps mps0: port 0xc000-0xc0ff mem 0xfbb3c000-0xfbb3ffff,0xfbb40000-0xfbb7ffff irq 30 at device 0.0 on pci3 mps0: Firmware: 19.00.00.00, Driver: 19.00.00.00-fbsd mps0: IOCCapabilities: 1285c ses0 at mps0 bus 0 scbus0 target 8 lun 0 da0 at mps0 bus 0 scbus0 target 0 lun 0 da1 at mps0 bus 0 scbus0 target 1 lun 0 da2 at mps0 bus 0 scbus0 target 2 lun 0 da3 at mps0 bus 0 scbus0 target 3 lun 0 *da4 at mps0 bus 0 scbus0 target 6 lun 0** **da5 at mps0 bus 0 scbus0 target 7 lun 0* Those last two are the drives the card identifies as being on targets 0 and 1 when looked at in the BIOS. The other four drives (0-3) are on the SAS expander -- and there's a two drive "gap" in the targets identified as well which has zippo for logic associated with it. Even more-oddly if I swap the plugs -- that is, plug the two boot disks into the OTHER connector (putatively targets 4-7) and put the SAS expander on the 0-3 port connector the BIOS *does* show the target shift but the machine, when it boots, still places those two drives on targets 6 and 7! There is nothing in /boot/loader.conf or /boot/device.hints that relates to these devices at all. I would like to wire down the various ports so Drive Bay #x corresponds to da[x] but how can you when you don't know what order they will come up in predicated on either the physical port they're occupying or, for that matter, what the card's BIOS shows them as? I looked in the driver's man page and saw nothing related to this; I get around it by labeling the drives and then using "glabel list" to see what da[x] number it grabbed if I need to pull a disk while the machine is up, but this is a rather-unfriendly way to handle that. -- Karl Denninger karl@denninger.net /The Market Ticker/