From owner-freebsd-scsi@FreeBSD.ORG Fri May 1 23:02:46 2015 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 967651C4; Fri, 1 May 2015 23:02:46 +0000 (UTC) Received: from mail-oi0-x236.google.com (mail-oi0-x236.google.com [IPv6:2607:f8b0:4003:c06::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5F0541935; Fri, 1 May 2015 23:02:46 +0000 (UTC) Received: by oift201 with SMTP id t201so79250345oif.3; Fri, 01 May 2015 16:02:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=ieUzimk9VP0luthq4ARo3+fP5dV52yKLULQ6NRvAf/U=; b=vhI36fzJG52zyAXp0tDb7/KUgyXlGlyZzT4TzO7xeoD7lNh7AOnZ0+Oxt537zuzKaj SPrA0m9H96g0Tita61zmQNBtiLRyJpukvmGn5wghRPzpYOMshop8qJGZYa2V00tM/8A+ zyDcga2P6dUdmINTmAuXRbbPUT4+Wz4T+71E2DnQfn3xPtilSGt0VQjK1ndp0LH4zRI3 9IgsStbg3E+YuudFVeF0M9tvXOZqwEPCpkqarjelb+uuJQR9D/ir7Plc/zTdq9556Z7P qKX5jjzDvIt3YwYjN0slaStg7JMYPmLeMe5V793NhjwlWmTk67OFVBuVyBI47smqiZeh jjdw== MIME-Version: 1.0 X-Received: by 10.202.173.144 with SMTP id w138mr9247558oie.100.1430521365738; Fri, 01 May 2015 16:02:45 -0700 (PDT) Sender: asomers@gmail.com Received: by 10.202.173.204 with HTTP; Fri, 1 May 2015 16:02:45 -0700 (PDT) In-Reply-To: <20150501225201.GA8570@FreeBSD.org> References: <20150501225201.GA8570@FreeBSD.org> Date: Fri, 1 May 2015 17:02:45 -0600 X-Google-Sender-Auth: GVTEnDoKH0ItND3ebS8oJ_heX4A Message-ID: Subject: Re: 9300-16e does not see 6G shelf From: Alan Somers To: John Cc: FreeBSD-SCSI Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 May 2015 23:02:46 -0000 On Fri, May 1, 2015 at 4:52 PM, John wrote: > Hi Folks, > > I'm running 10-STABLE with some LSI (Avago) 9300-16e HBA > cards with the mpr driver. These cards work great with 12G > shelves (system & shelves from SuperMicro). > > However, if I hook a set of 6G shelves from a system > using LSI 9207-8e HBA cards (a nicely working config), the mpr > driver complains that no active slots are found and the shelf is > not registered - leading to no drives being discovered. > > I am not mixing speeds on the bus. I'm using only 6G shelves > in this configuration. > > The shelf information from a 9207 based system: > > ses0 at mps1 bus 0 scbus7 target 169 lun 0 > ses0: Fixed Enclosure Services SCSI-5 device > ses0: 600.000MB/s transfers > ses0: Command Queueing enabled > ses0: SCSI-3 SES Device > > From the system with the 9300-16e cards (verbose boot): > > mpr0: port 0xf000-0xf0ff mem 0xfbd40000-0xfbd4ffff,0xfbd00000-0xfbd3ffff irq 56 at device 0.0 on pci132 > mpr0: IOCFacts : > MsgVersion: 0x205 > HeaderVersion: 0x2300 > IOCNumber: 0 > IOCExceptions: 0x0 > MaxChainDepth: 128 > NumberOfPorts: 1 > RequestCredit: 10112 > ProductID: 0x2221 > IOCRequestFrameSize: 32 > MaxInitiators: 32 > MaxTargets: 1024 > MaxSasExpanders: 64 > MaxEnclosures: 65 > HighPriorityCredit: 120 > MaxReplyDescriptorPostQueueDepth: 65504 > ReplyFrameSize: 32 > MaxVolumes: 0 > MaxDevHandle: 1128 > MaxPersistentEntries: 128 > mpr0: Firmware: 08.00.00.00, Driver: 05.255.05.00-fbsd > mpr0: IOCCapabilities: 7a85c > mpr0: attempting to allocate 1 MSI-X vectors (96 supported) > msi: routing MSI-X IRQ 302 to local APIC 0 vector 100 > mpr0: using IRQ 302 for MSI-X > > > and then the following: > > > mpr_mapping_enclosure_dev_status_change_event: enclosure with handle = 0x2 reported 0 slots > _mapping_get_dev_info: failed to add the device with handle 0x000a because the enclosure is not in the mapping table > _mapping_get_dev_info: failed to add the device with handle 0x000b because the enclosure is not in the mapping table > _mapping_get_dev_info: failed to add the device with handle 0x000c because the enclosure is not in the mapping table > _mapping_get_dev_info: failed to add the device with handle 0x000d because the enclosure is not in the mapping table > _mapping_get_dev_info: failed to add the device with handle 0x000e because the enclosure is not in the mapping table > > Followed by many more of the above messages for each of the disks. > > I've started looking at the Mpi2EventDataSasEnclDevStatusChange_t > structure - and I'm not convinced it's filled out fully. > > I can drop the 9207 cards in the new system and things > work fine - but I would like to use the newer cards if possible. > > Is there something I've missed? A configuration knob? > > Thoughts? > > Thanks, > John This smells like another Slot Mapping problem. When you connect the 6G shelf, do you see any devices at all? I would expect that you see at least one, though it might be ses(4) instead of da(4). If this is the Slot Mapping problem, then it doesn't really have anything to do with 12G vs 6G. Rather, it's an issue with specific how SAS expanders report their phy info, and how the HBA firmware interprets it. The HBA firmware can use two different modes: persistent mapping or enclosure/slot mapping. By default, LSI's 6G HBAs used persistent mapping, but their 12G HBAs use device/slot mapping by default. This post explains how to change the HBA's mode. https://lists.freebsd.org/pipermail/freebsd-scsi/2014-December/006545.html -Alan